2024-09-15T21:52:30.584 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:52:30.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:52:30.706 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906227' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+4wAQBKq8w36rLpP7nKSDDWcfzzu0W/+s0lBIwEC2vVbXHLmd1xI2oIC8Xkvx6LPc5wnsPzQi5TIEtldPrmkI= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQ6D0GR+BZvTgAo6V9pjPFltCVqAje3gEkgqMYLnqOSb/GEHeLV/vmyXOh+u0ym5JY8QLSgnW77V+7GBoLceEM= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDqPxyfrgNCKahNuF2KDRXlIeQRveA2W95DIDXpjSf1HeRkkK3kgyh0ai68TogR/HxtZ8c7XEJuoTOuP2xFppg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:52:30.706 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T21:52:30.707 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T21:52:30.707 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:52:30.708 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:52:30.731 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T21:52:30.731 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:52:30.731 INFO:teuthology.packaging:ref: None 2024-09-15T21:52:30.731 INFO:teuthology.packaging:tag: None 2024-09-15T21:52:30.731 INFO:teuthology.packaging:branch: squid 2024-09-15T21:52:30.732 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:52:30.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T21:52:30.936 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-326.g8921c8ef 2024-09-15T21:52:30.938 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:52:30.950 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:52:30.950 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:52:30.964 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:52:30.985 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:52:30.997 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:52:31.023 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:47:26.545728', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+4wAQBKq8w36rLpP7nKSDDWcfzzu0W/+s0lBIwEC2vVbXHLmd1xI2oIC8Xkvx6LPc5wnsPzQi5TIEtldPrmkI='} 2024-09-15T21:52:31.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:47:26.544957', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQ6D0GR+BZvTgAo6V9pjPFltCVqAje3gEkgqMYLnqOSb/GEHeLV/vmyXOh+u0ym5JY8QLSgnW77V+7GBoLceEM='} 2024-09-15T21:52:31.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:47:26.543970', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDqPxyfrgNCKahNuF2KDRXlIeQRveA2W95DIDXpjSf1HeRkkK3kgyh0ai68TogR/HxtZ8c7XEJuoTOuP2xFppg='} 2024-09-15T21:52:31.069 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:52:31.076 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T21:52:31.076 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T21:52:31.076 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T21:52:31.076 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:52:31.210 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa526973be0>, signals=[15]) 2024-09-15T21:52:31.210 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:52:31.221 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:52:31.221 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T21:52:31.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:52:31.302 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T21:52:31.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:52:31.378 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-15T21:52:31.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:52:31.465 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:52:31.472 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-09-15T21:52:31.491 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-09-15T21:52:31.491 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:52:31.553 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:52:31.554 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:52:31.554 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:52:31.554 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:52:31.554 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-09-15T21:52:31.581 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-15T21:52:31.599 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-15T21:52:31.599 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-15T21:52:31.657 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-15T21:52:31.657 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:52:31.658 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:52:31.659 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-15T21:52:31.681 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-15T21:52:31.700 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-15T21:52:31.700 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-15T21:52:31.758 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-15T21:52:31.758 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:52:31.759 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:52:31.761 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:52:31.761 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:52:31.761 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:52:31.761 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-15T21:52:31.780 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:52:31.794 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:52:31.804 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:52:31.804 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:52:31.807 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:52:31.810 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:52:31.826 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:52:31.837 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:52:31.838 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:52:31.866 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:52:31.869 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:52:31.911 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:52:31.987 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:52:32.007 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:52:32.007 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-15T21:52:32.007 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-15T21:52:32.008 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:52:32.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:32.025 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-15T21:52:32.083 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:52:32.083 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.14.0-503.el9.x86_64 2024-09-15T21:52:32.083 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2024-09-15T21:52:34.311 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-15T21:52:38.096 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 20 MB 00:02 2024-09-15T21:52:42.681 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.5 MB 00:00 2024-09-15T21:52:44.281 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 19 kB 00:00 2024-09-15T21:52:45.501 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-15T21:52:50.924 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-15T21:52:52.939 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:52:52.939 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:52:53.041 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-15T21:52:53.050 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout:Install 4 Packages 2024-09-15T21:52:53.051 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:52:53.052 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 79 M 2024-09-15T21:52:53.052 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 126 M 2024-09-15T21:52:53.052 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-09-15T21:52:53.242 INFO:teuthology.orchestra.run.smithi077.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 247 kB/s | 23 kB 00:00 2024-09-15T21:52:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-15T21:52:54.093 INFO:teuthology.orchestra.run.smithi077.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-15T21:52:54.418 INFO:teuthology.orchestra.run.smithi077.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-15T21:52:54.419 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:52:54.419 INFO:teuthology.orchestra.run.smithi077.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-15T21:52:55.103 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-15T21:52:55.440 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-15T21:52:55.440 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-15T21:52:56.416 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-15T21:52:56.417 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-15T21:52:58.868 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-15T21:52:59.570 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:52:59.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:53:02.428 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:53:02.574 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:53:07.394 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:53:07.516 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:12.343 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:50.402 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:50.442 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:50.797 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:50.798 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:53:50.798 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:53:50.798 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:53:51.645 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:53:51.646 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:53:51.646 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-09-15T21:53:51.646 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:53:51.646 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:53:51.646 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:53:51.647 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:53:51.647 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:53:51.647 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-15T21:53:51.986 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:53:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:53:52.071 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:53:52.071 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:53:52.071 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:53:52.071 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:53:52.071 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T21:53:52.071 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:53:52.071 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:53:52.129 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:53:52.130 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:53:52.130 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -qi grub2-tools 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Name : grub2-tools 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Epoch : 1 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Version : 2.06 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Release : 82.el9 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: x86_64 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Group : Unspecified 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Size : 8274670 2024-09-15T21:53:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:License : GPLv3+ 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Packager : builder@centos.org 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:Vendor : CentOS 2024-09-15T21:53:52.173 INFO:teuthology.orchestra.run.smithi077.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:Summary : Support tools for GRUB. 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:Description : 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:hardware devices. 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T21:53:52.174 INFO:teuthology.orchestra.run.smithi077.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:53:52.175 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:53:52.175 INFO:teuthology.task.kernel:Updating grub on smithi077 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:53:52.175 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:53:52.804 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2024-09-15T21:53:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:53:54.167 INFO:teuthology.orchestra.run.smithi077.stderr:done 2024-09-15T21:53:54.172 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2024-09-15T21:53:54.187 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.VCV58DEa7b 2024-09-15T21:53:54.187 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VCV58DEa7b 2024-09-15T21:53:54.254 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.VCV58DEa7b 2024-09-15T21:53:54.396 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.VCV58DEa7b is 6KB 2024-09-15T21:53:54.446 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.VCV58DEa7b 2024-09-15T21:53:54.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:53:54.524 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:53:54.525 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:53:54.525 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:53:54.525 INFO:teuthology.orchestra.run.smithi077.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:53:54.527 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:53:54.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2024-09-15T21:54:24.677 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T21:54:24.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:54:24.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:54:59.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-15T21:55:08.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:55:08.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:55:26.653 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-09-15T21:55:38.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:55:38.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:55:39.194 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-15T21:55:39.526 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:55:39.527 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:55:39.527 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-09-15T21:55:39.542 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:55:39.542 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:55:39.542 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:55:39.542 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:55:40.543 DEBUG:teuthology.parallel:result is None 2024-09-15T21:55:40.544 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:55:40.552 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:55:40.552 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:55:40.554 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:55:40.557 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:55:40.576 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:55:40.584 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:55:40.594 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:55:40.594 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:55:40.607 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:55:40.616 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:55:40.727 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:55:40.735 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:55:40.735 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:55:40.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:55:40.752 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:55:40.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:55:40.769 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:55:40.771 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:55:40.795 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:55:40.827 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.864 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.874 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.884 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.893 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.924 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:55:40.926 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:55:40.935 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:55:40.935 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:55:40.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:55:40.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:55:40.968 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:55:40.978 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:55:40.979 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:55:41.001 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:55:41.007 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:55:41.026 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.094 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.133 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.249 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T21:55:41.249 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:55:41.311 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.404 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.489 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.533 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T21:55:41.533 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:55:41.599 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.643 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:55:41.703 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.786 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:55:41.831 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T21:55:41.831 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:55:41.897 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-09-15T21:55:41.900 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-15T21:55:41.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-15T21:55:41.937 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:55:41.942 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:55:41.984 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:55:42.402 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:55:42.410 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:55:42.410 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:55:42.437 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:55:42.446 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T21:55:42.446 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-09-15T21:55:42.508 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-09-15T21:55:42.509 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-15T21:55:42.584 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-15T21:55:42.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-15T21:55:42.653 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-15T21:55:42.655 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:55:42.655 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-09-15T21:55:42.676 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-09-15T21:55:42.676 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-15T21:55:42.701 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-15T21:55:42.701 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-15T21:55:42.721 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-15T21:55:42.721 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:55:42.722 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:55:42.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:55:42.748 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:55:42.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:55:42.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:55:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:55:42.803 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:55:42.803 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:55:42.832 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:55:42.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:55:42.904 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:55:42.914 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T21:55:42.917 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T21:55:43.029 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:55:43.040 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:55:43.041 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi117.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:59:42.307 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-15T21:59:42.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:59:42.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:59:42.390 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-09-15T21:59:42.476 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-09-15T21:59:42.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T21:59:42.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:59:42.555 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-15T21:59:42.634 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T21:59:42.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-15T21:59:42.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:59:42.712 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-15T21:59:42.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-15T21:59:42.799 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:59:42.810 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:59:42.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:59:42.811 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:59:42.813 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:59:42.814 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:59:42.816 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:59:42.816 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:59:42.842 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:59:42.846 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:59:42.857 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:59:42.861 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:59:42.871 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:59:42.882 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-09-15T21:59:42.887 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:59:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-15T21:59:42.894 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-09-15T21:59:42.900 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-15T21:59:42.908 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:59:42.911 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-15T21:59:42.913 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:59:42.922 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:59:42.923 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-15T21:59:42.928 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:59:42.937 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:59:42.952 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:59:42.981 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:59:42.983 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:59:42.983 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-15T21:59:42.983 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.983 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.983 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.984 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.986 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:59:42.989 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:59:42.989 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-15T21:59:42.989 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.990 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.990 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:42.990 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:43.014 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:59:43.017 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:59:43.017 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-15T21:59:43.017 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:43.017 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:43.017 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:43.018 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:59:43.019 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:59:43.027 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:59:43.027 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T21:59:43.027 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:59:43.027 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:59:43.033 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:59:43.033 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T21:59:43.033 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:59:43.034 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:59:43.036 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:59:43.037 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:59:43.038 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:59:43.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:59:43.186 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:59:43.186 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:59:43.198 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:59:43.198 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:59:43.203 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:59:43.203 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:59:43.295 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:59:43.295 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T21:59:43.295 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:59:43.310 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:59:43.311 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T21:59:43.311 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:59:43.314 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:59:43.314 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T21:59:43.314 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:59:43.322 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:59:43.322 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:59:43.341 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:59:43.342 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:59:43.342 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:59:43.343 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:59:43.390 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:59:43.407 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:59:43.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:59:43.465 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:59:43.482 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:59:43.485 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:59:43.529 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-09-15T21:59:43.531 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-09-15T21:59:43.546 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-09-15T21:59:43.548 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-09-15T21:59:43.549 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-15T21:59:43.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-15T21:59:43.772 INFO:teuthology.orchestra.run.smithi077.stdout:45 files removed 2024-09-15T21:59:43.776 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-09-15T21:59:43.780 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-15T21:59:43.798 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:59:43.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:59:43.808 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:59:44.340 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 327 kB/s | 83 kB 00:00 2024-09-15T21:59:44.349 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 309 kB/s | 83 kB 00:00 2024-09-15T21:59:44.366 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 297 kB/s | 83 kB 00:00 2024-09-15T21:59:44.615 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-15T21:59:44.633 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T21:59:44.658 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-15T21:59:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-15T21:59:44.908 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T21:59:44.933 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T21:59:46.283 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.2 MB 00:01 2024-09-15T21:59:46.350 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.2 MB 00:01 2024-09-15T21:59:47.367 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.2 MB 00:02 2024-09-15T21:59:49.543 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-15T21:59:52.070 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-15T21:59:53.642 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 20 MB 00:06 2024-09-15T21:59:54.035 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-15T21:59:55.503 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-15T21:59:56.321 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-15T21:59:56.623 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T21:59:58.588 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-15T21:59:59.117 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-15T21:59:59.651 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T22:00:00.877 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-15T22:00:02.004 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-15T22:00:02.013 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-15T22:00:04.266 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:04.266 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:04.279 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:00:04.280 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:00:04.281 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:00:04.282 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:00:04.284 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:00:04.287 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:00:04.288 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:00:04.349 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T22:00:04.359 INFO:teuthology.orchestra.run.smithi117.stdout:====================================================================================== 2024-09-15T22:00:04.359 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout:====================================================================================== 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:00:04.360 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:00:04.361 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:00:04.362 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:00:04.363 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:00:04.364 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:00:04.365 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:00:04.366 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:00:04.367 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:00:04.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:00:04.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:00:04.370 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:00:04.371 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-09-15T22:00:04.372 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout:====================================================================================== 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout:Install 148 Packages 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 8 Packages 2024-09-15T22:00:04.373 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:04.374 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 226 M 2024-09-15T22:00:04.374 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T22:00:05.148 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-15T22:00:05.163 INFO:teuthology.orchestra.run.smithi117.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-15T22:00:05.347 INFO:teuthology.orchestra.run.smithi117.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-15T22:00:05.422 INFO:teuthology.orchestra.run.smithi117.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.9 MB/s | 145 kB 00:00 2024-09-15T22:00:05.531 INFO:teuthology.orchestra.run.smithi117.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-15T22:00:05.623 INFO:teuthology.orchestra.run.smithi117.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-15T22:00:05.673 INFO:teuthology.orchestra.run.smithi117.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-15T22:00:05.999 INFO:teuthology.orchestra.run.smithi117.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-15T22:00:06.341 INFO:teuthology.orchestra.run.smithi117.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 16 MB/s | 21 MB 00:01 2024-09-15T22:00:06.382 INFO:teuthology.orchestra.run.smithi117.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 607 kB/s | 25 kB 00:00 2024-09-15T22:00:06.657 INFO:teuthology.orchestra.run.smithi117.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 17 MB/s | 17 MB 00:00 2024-09-15T22:00:06.774 INFO:teuthology.orchestra.run.smithi117.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 14 MB/s | 11 MB 00:00 2024-09-15T22:00:06.799 INFO:teuthology.orchestra.run.smithi117.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.6 MB/s | 228 kB 00:00 2024-09-15T22:00:06.824 INFO:teuthology.orchestra.run.smithi117.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-15T22:00:06.858 INFO:teuthology.orchestra.run.smithi117.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 539 kB/s | 32 kB 00:00 2024-09-15T22:00:06.901 INFO:teuthology.orchestra.run.smithi117.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 9.7 MB/s | 751 kB 00:00 2024-09-15T22:00:06.926 INFO:teuthology.orchestra.run.smithi117.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-15T22:00:06.951 INFO:teuthology.orchestra.run.smithi117.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-15T22:00:06.976 INFO:teuthology.orchestra.run.smithi117.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 9.8 MB/s | 506 kB 00:00 2024-09-15T22:00:07.035 INFO:teuthology.orchestra.run.smithi117.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 775 kB/s | 45 kB 00:00 2024-09-15T22:00:07.127 INFO:teuthology.orchestra.run.smithi117.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.5 MB/s | 136 kB 00:00 2024-09-15T22:00:07.256 INFO:teuthology.orchestra.run.smithi117.stdout:(21/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-15T22:00:07.328 INFO:teuthology.orchestra.run.smithi117.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 807 kB/s | 161 kB 00:00 2024-09-15T22:00:07.353 INFO:teuthology.orchestra.run.smithi117.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 3.2 MB/s | 320 kB 00:00 2024-09-15T22:00:07.378 INFO:teuthology.orchestra.run.smithi117.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.8 MB/s | 300 kB 00:00 2024-09-15T22:00:07.403 INFO:teuthology.orchestra.run.smithi117.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-15T22:00:07.429 INFO:teuthology.orchestra.run.smithi117.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-15T22:00:07.433 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:07.434 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:07.446 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:00:07.446 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:00:07.447 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:00:07.448 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:00:07.450 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:00:07.452 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:00:07.453 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:00:07.461 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-15T22:00:07.496 INFO:teuthology.orchestra.run.smithi117.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-15T22:00:07.515 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:00:07.537 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:00:07.538 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:00:07.539 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:00:07.540 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:00:07.541 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:00:07.542 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:00:07.543 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:00:07.544 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:00:07.545 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:00:07.546 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:00:07.547 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:00:07.548 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:00:07.549 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:00:07.550 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-09-15T22:00:07.551 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:07.552 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 226 M 2024-09-15T22:00:07.552 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-15T22:00:07.562 INFO:teuthology.orchestra.run.smithi117.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-15T22:00:07.588 INFO:teuthology.orchestra.run.smithi117.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 347 kB/s | 32 kB 00:00 2024-09-15T22:00:07.621 INFO:teuthology.orchestra.run.smithi117.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.4 MB/s | 146 kB 00:00 2024-09-15T22:00:07.755 INFO:teuthology.orchestra.run.smithi117.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 21 MB/s | 3.6 MB 00:00 2024-09-15T22:00:07.831 INFO:teuthology.orchestra.run.smithi117.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-15T22:00:07.906 INFO:teuthology.orchestra.run.smithi117.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 657 kB/s | 49 kB 00:00 2024-09-15T22:00:07.989 INFO:teuthology.orchestra.run.smithi117.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-15T22:00:08.015 INFO:teuthology.orchestra.run.smithi117.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 152 kB/s | 16 kB 00:00 2024-09-15T22:00:08.048 INFO:teuthology.orchestra.run.smithi117.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-15T22:00:08.082 INFO:teuthology.orchestra.run.smithi117.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-15T22:00:08.107 INFO:teuthology.orchestra.run.smithi117.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-15T22:00:08.249 INFO:teuthology.orchestra.run.smithi117.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-15T22:00:08.266 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-15T22:00:08.341 INFO:teuthology.orchestra.run.smithi117.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-15T22:00:08.376 INFO:teuthology.orchestra.run.smithi117.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-15T22:00:08.418 INFO:teuthology.orchestra.run.smithi117.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 253 kB 00:00 2024-09-15T22:00:08.468 INFO:teuthology.orchestra.run.smithi117.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-09-15T22:00:08.494 INFO:teuthology.orchestra.run.smithi117.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-15T22:00:08.516 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.5 MB/s | 896 kB 00:00 2024-09-15T22:00:08.519 INFO:teuthology.orchestra.run.smithi117.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-15T22:00:08.545 INFO:teuthology.orchestra.run.smithi117.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T22:00:08.575 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.4 MB/s | 145 kB 00:00 2024-09-15T22:00:08.578 INFO:teuthology.orchestra.run.smithi117.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-15T22:00:08.708 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 9.6 MB/s | 5.4 MB 00:00 2024-09-15T22:00:08.825 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 1.5 MB 00:00 2024-09-15T22:00:08.901 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 7.3 MB/s | 2.4 MB 00:00 2024-09-15T22:00:09.028 INFO:teuthology.orchestra.run.smithi117.stdout:(48/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 18 MB/s | 49 MB 00:02 2024-09-15T22:00:09.101 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-15T22:00:09.196 INFO:teuthology.orchestra.run.smithi117.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-15T22:00:09.246 INFO:teuthology.orchestra.run.smithi117.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-15T22:00:09.279 INFO:teuthology.orchestra.run.smithi117.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 938 kB/s | 689 kB 00:00 2024-09-15T22:00:09.338 INFO:teuthology.orchestra.run.smithi117.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-15T22:00:09.363 INFO:teuthology.orchestra.run.smithi117.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-15T22:00:09.405 INFO:teuthology.orchestra.run.smithi117.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-09-15T22:00:09.430 INFO:teuthology.orchestra.run.smithi117.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-15T22:00:09.456 INFO:teuthology.orchestra.run.smithi117.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T22:00:09.489 INFO:teuthology.orchestra.run.smithi117.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-15T22:00:09.514 INFO:teuthology.orchestra.run.smithi117.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-15T22:00:09.618 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 21 MB 00:01 2024-09-15T22:00:09.631 INFO:teuthology.orchestra.run.smithi117.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-15T22:00:09.665 INFO:teuthology.orchestra.run.smithi117.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-15T22:00:09.690 INFO:teuthology.orchestra.run.smithi117.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-15T22:00:09.715 INFO:teuthology.orchestra.run.smithi117.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-15T22:00:09.735 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 17 MB/s | 11 MB 00:00 2024-09-15T22:00:09.760 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 178 kB/s | 25 kB 00:00 2024-09-15T22:00:09.773 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:09.774 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:00:09.774 INFO:teuthology.orchestra.run.smithi117.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-15T22:00:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:00:09.788 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:00:09.789 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:00:09.790 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:00:09.792 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:00:09.795 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:00:09.796 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:00:09.818 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.8 MB/s | 228 kB 00:00 2024-09-15T22:00:09.824 INFO:teuthology.orchestra.run.smithi117.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 771 kB/s | 38 kB 00:00 2024-09-15T22:00:09.861 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-09-15T22:00:09.866 INFO:teuthology.orchestra.run.smithi117.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:00:09.871 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:00:09.872 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:00:09.873 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:00:09.874 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:00:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:00:09.876 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:00:09.877 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:00:09.878 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:00:09.879 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:00:09.880 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:00:09.881 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:00:09.882 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:00:09.883 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:00:09.884 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-09-15T22:00:09.884 INFO:teuthology.orchestra.run.smithi077.stdout:====================================================================================== 2024-09-15T22:00:09.884 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-09-15T22:00:09.884 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 8 Packages 2024-09-15T22:00:09.884 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:00:09.885 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 462 kB/s | 27 kB 00:00 2024-09-15T22:00:09.885 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 226 M 2024-09-15T22:00:09.885 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-09-15T22:00:09.936 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 545 kB/s | 32 kB 00:00 2024-09-15T22:00:10.011 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-15T22:00:10.078 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-15T22:00:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-15T22:00:10.188 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.1 MB/s | 127 kB 00:00 2024-09-15T22:00:10.245 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 8.8 MB/s | 506 kB 00:00 2024-09-15T22:00:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-15T22:00:10.334 INFO:teuthology.orchestra.run.smithi117.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-15T22:00:10.409 INFO:teuthology.orchestra.run.smithi117.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.5 MB/s | 4.9 MB 00:00 2024-09-15T22:00:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-15T22:00:10.502 INFO:teuthology.orchestra.run.smithi117.stdout:(71/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-15T22:00:10.560 INFO:teuthology.orchestra.run.smithi117.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-15T22:00:10.586 INFO:teuthology.orchestra.run.smithi117.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-15T22:00:10.587 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-15T22:00:10.637 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 908 kB/s | 45 kB 00:00 2024-09-15T22:00:10.644 INFO:teuthology.orchestra.run.smithi117.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-15T22:00:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.3 MB/s | 136 kB 00:00 2024-09-15T22:00:10.711 INFO:teuthology.orchestra.run.smithi117.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-15T22:00:10.746 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.2 MB/s | 161 kB 00:00 2024-09-15T22:00:10.795 INFO:teuthology.orchestra.run.smithi117.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-09-15T22:00:10.820 INFO:teuthology.orchestra.run.smithi117.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-09-15T22:00:10.830 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T22:00:10.845 INFO:teuthology.orchestra.run.smithi117.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-15T22:00:10.862 INFO:teuthology.orchestra.run.smithi117.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-09-15T22:00:10.879 INFO:teuthology.orchestra.run.smithi117.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 19 kB 00:00 2024-09-15T22:00:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-15T22:00:10.904 INFO:teuthology.orchestra.run.smithi117.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-15T22:00:10.921 INFO:teuthology.orchestra.run.smithi117.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-09-15T22:00:10.954 INFO:teuthology.orchestra.run.smithi077.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-15T22:00:10.963 INFO:teuthology.orchestra.run.smithi117.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-15T22:00:10.989 INFO:teuthology.orchestra.run.smithi117.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-15T22:00:11.014 INFO:teuthology.orchestra.run.smithi117.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T22:00:11.039 INFO:teuthology.orchestra.run.smithi117.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-15T22:00:11.073 INFO:teuthology.orchestra.run.smithi117.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-15T22:00:11.098 INFO:teuthology.orchestra.run.smithi117.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-15T22:00:11.132 INFO:teuthology.orchestra.run.smithi117.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-15T22:00:11.157 INFO:teuthology.orchestra.run.smithi117.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-15T22:00:11.182 INFO:teuthology.orchestra.run.smithi117.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-15T22:00:11.196 INFO:teuthology.orchestra.run.smithi077.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.6 MB/s | 896 kB 00:00 2024-09-15T22:00:11.232 INFO:teuthology.orchestra.run.smithi117.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-15T22:00:11.283 INFO:teuthology.orchestra.run.smithi117.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-15T22:00:11.288 INFO:teuthology.orchestra.run.smithi077.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-15T22:00:11.313 INFO:teuthology.orchestra.run.smithi077.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.2 MB/s | 145 kB 00:00 2024-09-15T22:00:11.330 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 31 MB/s | 49 MB 00:01 2024-09-15T22:00:11.341 INFO:teuthology.orchestra.run.smithi117.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T22:00:11.347 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 221 kB/s | 99 kB 00:00 2024-09-15T22:00:11.400 INFO:teuthology.orchestra.run.smithi117.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-15T22:00:11.414 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-15T22:00:11.455 INFO:teuthology.orchestra.run.smithi077.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 2.4 MB 00:00 2024-09-15T22:00:11.498 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-15T22:00:11.559 INFO:teuthology.orchestra.run.smithi117.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-09-15T22:00:11.584 INFO:teuthology.orchestra.run.smithi117.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-15T22:00:11.589 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 349 kB/s | 32 kB 00:00 2024-09-15T22:00:11.617 INFO:teuthology.orchestra.run.smithi117.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-15T22:00:11.643 INFO:teuthology.orchestra.run.smithi117.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 982 kB/s | 58 kB 00:00 2024-09-15T22:00:11.665 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 3.1 MB 00:00 2024-09-15T22:00:11.668 INFO:teuthology.orchestra.run.smithi117.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T22:00:11.690 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.4 MB/s | 146 kB 00:00 2024-09-15T22:00:11.693 INFO:teuthology.orchestra.run.smithi117.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T22:00:11.697 INFO:teuthology.orchestra.run.smithi077.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-09-15T22:00:11.718 INFO:teuthology.orchestra.run.smithi117.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-15T22:00:11.769 INFO:teuthology.orchestra.run.smithi117.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-15T22:00:11.836 INFO:teuthology.orchestra.run.smithi117.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-15T22:00:11.864 INFO:teuthology.orchestra.run.smithi077.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-15T22:00:11.869 INFO:teuthology.orchestra.run.smithi117.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-15T22:00:11.895 INFO:teuthology.orchestra.run.smithi117.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-15T22:00:11.937 INFO:teuthology.orchestra.run.smithi117.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-15T22:00:11.987 INFO:teuthology.orchestra.run.smithi117.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-15T22:00:12.012 INFO:teuthology.orchestra.run.smithi117.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T22:00:12.046 INFO:teuthology.orchestra.run.smithi117.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-15T22:00:12.080 INFO:teuthology.orchestra.run.smithi117.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-15T22:00:12.132 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 5.2 MB/s | 17 MB 00:03 2024-09-15T22:00:12.172 INFO:teuthology.orchestra.run.smithi117.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi077.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 15 MB/s | 21 MB 00:01 2024-09-15T22:00:12.199 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.7 MB/s | 3.6 MB 00:00 2024-09-15T22:00:12.205 INFO:teuthology.orchestra.run.smithi117.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-15T22:00:12.214 INFO:teuthology.orchestra.run.smithi077.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 609 kB/s | 25 kB 00:00 2024-09-15T22:00:12.224 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-15T22:00:12.230 INFO:teuthology.orchestra.run.smithi117.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:00:12.250 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 983 kB/s | 49 kB 00:00 2024-09-15T22:00:12.264 INFO:teuthology.orchestra.run.smithi117.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-15T22:00:12.289 INFO:teuthology.orchestra.run.smithi117.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 547 kB/s | 32 kB 00:00 2024-09-15T22:00:12.300 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 219 kB/s | 16 kB 00:00 2024-09-15T22:00:12.325 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.7 MB/s | 285 kB 00:00 2024-09-15T22:00:12.366 INFO:teuthology.orchestra.run.smithi117.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 138 kB/s | 14 kB 00:00 2024-09-15T22:00:12.390 INFO:teuthology.orchestra.run.smithi117.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-09-15T22:00:12.415 INFO:teuthology.orchestra.run.smithi117.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.2 MB/s | 358 kB 00:00 2024-09-15T22:00:12.417 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-15T22:00:12.440 INFO:teuthology.orchestra.run.smithi117.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-15T22:00:12.445 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 156 kB/s | 18 kB 00:00 2024-09-15T22:00:12.466 INFO:teuthology.orchestra.run.smithi117.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-15T22:00:12.499 INFO:teuthology.orchestra.run.smithi117.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-15T22:00:12.525 INFO:teuthology.orchestra.run.smithi117.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-15T22:00:12.526 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 3.3 MB/s | 765 kB 00:00 2024-09-15T22:00:12.543 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-15T22:00:12.550 INFO:teuthology.orchestra.run.smithi117.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-15T22:00:12.575 INFO:teuthology.orchestra.run.smithi117.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-15T22:00:12.600 INFO:teuthology.orchestra.run.smithi117.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-15T22:00:12.610 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-15T22:00:12.626 INFO:teuthology.orchestra.run.smithi117.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 794 kB/s | 20 kB 00:00 2024-09-15T22:00:12.635 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-15T22:00:12.669 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-15T22:00:12.676 INFO:teuthology.orchestra.run.smithi117.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-15T22:00:12.702 INFO:teuthology.orchestra.run.smithi117.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:00:12.710 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-15T22:00:12.727 INFO:teuthology.orchestra.run.smithi117.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T22:00:12.736 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-15T22:00:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 12 MB/s | 11 MB 00:00 2024-09-15T22:00:12.752 INFO:teuthology.orchestra.run.smithi117.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T22:00:12.761 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T22:00:12.794 INFO:teuthology.orchestra.run.smithi117.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-15T22:00:12.795 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-09-15T22:00:12.820 INFO:teuthology.orchestra.run.smithi117.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-15T22:00:12.820 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-15T22:00:12.870 INFO:teuthology.orchestra.run.smithi117.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-15T22:00:12.895 INFO:teuthology.orchestra.run.smithi117.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-15T22:00:12.920 INFO:teuthology.orchestra.run.smithi117.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-09-15T22:00:12.931 INFO:teuthology.orchestra.run.smithi077.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-15T22:00:12.945 INFO:teuthology.orchestra.run.smithi117.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-15T22:00:12.957 INFO:teuthology.orchestra.run.smithi077.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.0 MB/s | 228 kB 00:00 2024-09-15T22:00:12.971 INFO:teuthology.orchestra.run.smithi117.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T22:00:12.982 INFO:teuthology.orchestra.run.smithi077.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 541 kB/s | 27 kB 00:00 2024-09-15T22:00:12.987 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-15T22:00:13.004 INFO:teuthology.orchestra.run.smithi117.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-15T22:00:13.007 INFO:teuthology.orchestra.run.smithi077.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 635 kB/s | 32 kB 00:00 2024-09-15T22:00:13.030 INFO:teuthology.orchestra.run.smithi117.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-15T22:00:13.037 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-15T22:00:13.043 INFO:teuthology.orchestra.run.smithi077.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 751 kB 00:00 2024-09-15T22:00:13.056 INFO:teuthology.orchestra.run.smithi117.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-15T22:00:13.068 INFO:teuthology.orchestra.run.smithi077.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.6 MB/s | 163 kB 00:00 2024-09-15T22:00:13.071 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-15T22:00:13.080 INFO:teuthology.orchestra.run.smithi117.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-15T22:00:13.094 INFO:teuthology.orchestra.run.smithi077.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-15T22:00:13.106 INFO:teuthology.orchestra.run.smithi117.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-15T22:00:13.119 INFO:teuthology.orchestra.run.smithi077.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 9.9 MB/s | 506 kB 00:00 2024-09-15T22:00:13.121 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-15T22:00:13.131 INFO:teuthology.orchestra.run.smithi117.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-15T22:00:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-15T22:00:13.163 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-15T22:00:13.169 INFO:teuthology.orchestra.run.smithi077.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 902 kB/s | 45 kB 00:00 2024-09-15T22:00:13.189 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-09-15T22:00:13.190 INFO:teuthology.orchestra.run.smithi117.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:00:13.214 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-15T22:00:13.231 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-15T22:00:13.248 INFO:teuthology.orchestra.run.smithi117.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-15T22:00:13.253 INFO:teuthology.orchestra.run.smithi077.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T22:00:13.272 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-15T22:00:13.298 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-15T22:00:13.370 INFO:teuthology.orchestra.run.smithi077.stdout:(21/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-15T22:00:13.395 INFO:teuthology.orchestra.run.smithi077.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.1 MB/s | 161 kB 00:00 2024-09-15T22:00:13.437 INFO:teuthology.orchestra.run.smithi077.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 4.7 MB/s | 320 kB 00:00 2024-09-15T22:00:13.448 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-15T22:00:13.473 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-15T22:00:13.495 INFO:teuthology.orchestra.run.smithi077.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 2.9 MB/s | 300 kB 00:00 2024-09-15T22:00:13.499 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-15T22:00:13.521 INFO:teuthology.orchestra.run.smithi077.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.2 MB/s | 99 kB 00:00 2024-09-15T22:00:13.533 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-09-15T22:00:13.541 INFO:teuthology.orchestra.run.smithi117.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-15T22:00:13.554 INFO:teuthology.orchestra.run.smithi077.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-15T22:00:13.607 INFO:teuthology.orchestra.run.smithi117.stdout:(149/156): python3-numpy-1.20.1-5.el9.x86_64.rp 1.9 MB/s | 5.1 MB 00:02 2024-09-15T22:00:13.608 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-15T22:00:13.629 INFO:teuthology.orchestra.run.smithi077.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-15T22:00:13.633 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-15T22:00:13.661 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 737 kB/s | 38 kB 00:00 2024-09-15T22:00:13.853 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.4 MB/s | 3.0 MB 00:00 2024-09-15T22:00:13.854 INFO:teuthology.orchestra.run.smithi117.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 7.4 MB/s | 3.2 MB 00:00 2024-09-15T22:00:13.854 INFO:teuthology.orchestra.run.smithi117.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-15T22:00:13.854 INFO:teuthology.orchestra.run.smithi077.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-15T22:00:13.855 INFO:teuthology.orchestra.run.smithi077.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 346 kB/s | 32 kB 00:00 2024-09-15T22:00:13.855 INFO:teuthology.orchestra.run.smithi077.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.7 MB/s | 146 kB 00:00 2024-09-15T22:00:13.892 INFO:teuthology.orchestra.run.smithi117.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-15T22:00:13.950 INFO:teuthology.orchestra.run.smithi117.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 5.7 MB/s | 2.3 MB 00:00 2024-09-15T22:00:13.992 INFO:teuthology.orchestra.run.smithi117.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.7 MB/s | 1.8 MB 00:00 2024-09-15T22:00:14.084 INFO:teuthology.orchestra.run.smithi117.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-09-15T22:00:14.102 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-09-15T22:00:14.106 INFO:teuthology.orchestra.run.smithi077.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.3 MB/s | 3.6 MB 00:00 2024-09-15T22:00:14.126 INFO:teuthology.orchestra.run.smithi117.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-15T22:00:14.131 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:00:14.131 INFO:teuthology.orchestra.run.smithi117.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-15T22:00:14.189 INFO:teuthology.orchestra.run.smithi077.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-15T22:00:14.215 INFO:teuthology.orchestra.run.smithi077.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 2.2 MB/s | 249 kB 00:00 2024-09-15T22:00:14.232 INFO:teuthology.orchestra.run.smithi077.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 1.2 MB/s | 49 kB 00:00 2024-09-15T22:00:14.257 INFO:teuthology.orchestra.run.smithi077.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 392 kB/s | 16 kB 00:00 2024-09-15T22:00:14.282 INFO:teuthology.orchestra.run.smithi077.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.5 MB/s | 285 kB 00:00 2024-09-15T22:00:14.294 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-15T22:00:14.333 INFO:teuthology.orchestra.run.smithi077.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 10 MB/s | 765 kB 00:00 2024-09-15T22:00:14.349 INFO:teuthology.orchestra.run.smithi077.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 273 kB/s | 18 kB 00:00 2024-09-15T22:00:14.353 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-15T22:00:14.775 INFO:teuthology.orchestra.run.smithi077.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 19 MB/s | 49 MB 00:02 2024-09-15T22:00:14.778 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-15T22:00:15.003 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-15T22:00:15.037 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-15T22:00:15.062 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-15T22:00:15.113 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-15T22:00:15.221 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-15T22:00:15.312 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T22:00:15.333 INFO:teuthology.orchestra.run.smithi077.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 81 kB/s | 80 kB 00:00 2024-09-15T22:00:15.363 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 5.7 MB/s | 5.8 MB 00:01 2024-09-15T22:00:15.388 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-15T22:00:15.413 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-15T22:00:15.439 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-15T22:00:15.463 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T22:00:15.463 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T22:00:15.464 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-15T22:00:15.489 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-15T22:00:15.509 INFO:teuthology.orchestra.run.smithi077.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 261 kB/s | 191 kB 00:00 2024-09-15T22:00:15.600 INFO:teuthology.orchestra.run.smithi077.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 641 kB/s | 801 kB 00:01 2024-09-15T22:00:15.614 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-15T22:00:15.732 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-15T22:00:15.790 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-15T22:00:15.849 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-15T22:00:15.966 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-15T22:00:16.066 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.8 MB/s | 6.0 MB 00:01 2024-09-15T22:00:16.091 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T22:00:16.159 INFO:teuthology.orchestra.run.smithi077.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-09-15T22:00:16.175 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 5.1 MB 00:00 2024-09-15T22:00:16.192 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-09-15T22:00:16.217 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-15T22:00:16.234 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-09-15T22:00:16.260 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-09-15T22:00:16.267 INFO:teuthology.orchestra.run.smithi077.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-15T22:00:16.285 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-15T22:00:16.310 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-15T22:00:16.336 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-15T22:00:16.360 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-15T22:00:16.394 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-15T22:00:16.419 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-15T22:00:16.445 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T22:00:16.471 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T22:00:16.484 INFO:teuthology.orchestra.run.smithi077.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-15T22:00:16.495 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-15T22:00:16.546 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-15T22:00:16.596 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-15T22:00:16.630 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-15T22:00:16.646 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-15T22:00:16.705 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-15T22:00:16.756 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-15T22:00:16.789 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-15T22:00:16.848 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-15T22:00:16.865 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2024-09-15T22:00:16.890 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-15T22:00:16.926 INFO:teuthology.orchestra.run.smithi077.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 176 kB/s | 135 kB 00:00 2024-09-15T22:00:16.932 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-15T22:00:16.952 INFO:teuthology.orchestra.run.smithi077.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 182 kB/s | 125 kB 00:00 2024-09-15T22:00:16.957 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:00:16.982 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T22:00:17.008 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-15T22:00:17.035 INFO:teuthology.orchestra.run.smithi077.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-09-15T22:00:17.041 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-15T22:00:17.083 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 839 kB 00:00 2024-09-15T22:00:17.085 INFO:teuthology.orchestra.run.smithi077.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-09-15T22:00:17.109 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-15T22:00:17.142 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-15T22:00:17.152 INFO:teuthology.orchestra.run.smithi077.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 915 kB/s | 107 kB 00:00 2024-09-15T22:00:17.176 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-15T22:00:17.194 INFO:teuthology.orchestra.run.smithi077.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-15T22:00:17.210 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-15T22:00:17.228 INFO:teuthology.orchestra.run.smithi077.stdout:(52/156): python3-urllib3-1.26.5-6.el9.noarch.r 292 kB/s | 215 kB 00:00 2024-09-15T22:00:17.235 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-15T22:00:17.253 INFO:teuthology.orchestra.run.smithi077.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-15T22:00:17.260 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 716 kB/s | 18 kB 00:00 2024-09-15T22:00:17.277 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T22:00:17.277 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T22:00:17.278 INFO:teuthology.orchestra.run.smithi077.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-15T22:00:17.285 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-15T22:00:17.312 INFO:teuthology.orchestra.run.smithi077.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-15T22:00:17.328 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 461 kB/s | 19 kB 00:00 2024-09-15T22:00:17.329 INFO:teuthology.orchestra.run.smithi077.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T22:00:17.353 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 806 kB/s | 20 kB 00:00 2024-09-15T22:00:17.388 INFO:teuthology.orchestra.run.smithi077.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-09-15T22:00:17.395 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-15T22:00:17.413 INFO:teuthology.orchestra.run.smithi077.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-15T22:00:17.420 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:00:17.445 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T22:00:17.471 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T22:00:17.471 INFO:teuthology.orchestra.run.smithi077.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-15T22:00:17.504 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-15T22:00:17.513 INFO:teuthology.orchestra.run.smithi077.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-15T22:00:17.530 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-15T22:00:17.538 INFO:teuthology.orchestra.run.smithi077.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-15T22:00:17.555 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-15T22:00:17.564 INFO:teuthology.orchestra.run.smithi077.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-15T22:00:17.580 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-15T22:00:17.589 INFO:teuthology.orchestra.run.smithi077.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-15T22:00:17.605 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-15T22:00:17.631 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-15T22:00:17.639 INFO:teuthology.orchestra.run.smithi077.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-15T22:00:17.656 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T22:00:17.656 INFO:teuthology.orchestra.run.smithi077.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 578 kB/s | 38 kB 00:00 2024-09-15T22:00:17.681 INFO:teuthology.orchestra.run.smithi077.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-15T22:00:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-15T22:00:17.707 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-15T22:00:17.732 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-15T22:00:17.757 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-15T22:00:17.783 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-15T22:00:17.799 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-15T22:00:17.807 INFO:teuthology.orchestra.run.smithi077.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-15T22:00:17.825 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-15T22:00:17.859 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-15T22:00:17.909 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-15T22:00:17.957 INFO:teuthology.orchestra.run.smithi077.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 5.9 MB/s | 910 kB 00:00 2024-09-15T22:00:18.049 INFO:teuthology.orchestra.run.smithi077.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-15T22:00:18.091 INFO:teuthology.orchestra.run.smithi077.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-15T22:00:18.193 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-15T22:00:18.208 INFO:teuthology.orchestra.run.smithi077.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-09-15T22:00:18.260 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-09-15T22:00:18.367 INFO:teuthology.orchestra.run.smithi077.stdout:(72/156): openblas-openmp-0.3.26-2.el9.x86_64.r 6.8 MB/s | 4.9 MB 00:00 2024-09-15T22:00:18.468 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 8.4 MB/s | 2.3 MB 00:00 2024-09-15T22:00:18.483 INFO:teuthology.orchestra.run.smithi077.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 318 kB/s | 87 kB 00:00 2024-09-15T22:00:18.502 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 7.5 MB/s | 1.8 MB 00:00 2024-09-15T22:00:18.527 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 698 kB/s | 41 kB 00:00 2024-09-15T22:00:18.542 INFO:teuthology.orchestra.run.smithi077.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-15T22:00:18.552 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-15T22:00:18.626 INFO:teuthology.orchestra.run.smithi077.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-15T22:00:18.669 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-09-15T22:00:18.717 INFO:teuthology.orchestra.run.smithi077.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-15T22:00:18.784 INFO:teuthology.orchestra.run.smithi077.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-09-15T22:00:18.820 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 1.4 MB 00:00 2024-09-15T22:00:18.852 INFO:teuthology.orchestra.run.smithi077.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-09-15T22:00:18.993 INFO:teuthology.orchestra.run.smithi077.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-15T22:00:19.069 INFO:teuthology.orchestra.run.smithi077.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 8.5 MB/s | 6.0 MB 00:00 2024-09-15T22:00:19.144 INFO:teuthology.orchestra.run.smithi077.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.3 MB/s | 5.8 MB 00:01 2024-09-15T22:00:19.169 INFO:teuthology.orchestra.run.smithi077.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 198 kB/s | 35 kB 00:00 2024-09-15T22:00:19.244 INFO:teuthology.orchestra.run.smithi077.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-15T22:00:19.245 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-09-15T22:00:19.250 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:00:19.250 INFO:teuthology.orchestra.run.smithi196.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-15T22:00:19.270 INFO:teuthology.orchestra.run.smithi077.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-15T22:00:19.337 INFO:teuthology.orchestra.run.smithi077.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-09-15T22:00:19.397 INFO:teuthology.orchestra.run.smithi077.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 332 kB/s | 42 kB 00:00 2024-09-15T22:00:19.504 INFO:teuthology.orchestra.run.smithi077.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-15T22:00:19.529 INFO:teuthology.orchestra.run.smithi077.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-09-15T22:00:19.605 INFO:teuthology.orchestra.run.smithi077.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-15T22:00:19.630 INFO:teuthology.orchestra.run.smithi077.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-09-15T22:00:19.697 INFO:teuthology.orchestra.run.smithi077.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 585 kB/s | 54 kB 00:00 2024-09-15T22:00:19.755 INFO:teuthology.orchestra.run.smithi077.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-09-15T22:00:19.824 INFO:teuthology.orchestra.run.smithi077.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 665 kB/s | 44 kB 00:00 2024-09-15T22:00:19.948 INFO:teuthology.orchestra.run.smithi077.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-15T22:00:19.998 INFO:teuthology.orchestra.run.smithi077.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-15T22:00:20.056 INFO:teuthology.orchestra.run.smithi077.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-15T22:00:20.140 INFO:teuthology.orchestra.run.smithi077.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-15T22:00:20.215 INFO:teuthology.orchestra.run.smithi077.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-15T22:00:20.274 INFO:teuthology.orchestra.run.smithi077.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 433 kB/s | 58 kB 00:00 2024-09-15T22:00:20.299 INFO:teuthology.orchestra.run.smithi077.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-15T22:00:20.349 INFO:teuthology.orchestra.run.smithi077.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-09-15T22:00:20.366 INFO:teuthology.orchestra.run.smithi077.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-15T22:00:20.450 INFO:teuthology.orchestra.run.smithi077.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-15T22:00:20.491 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-15T22:00:20.492 INFO:teuthology.orchestra.run.smithi077.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-09-15T22:00:20.534 INFO:teuthology.orchestra.run.smithi077.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-15T22:00:20.551 INFO:teuthology.orchestra.run.smithi077.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-15T22:00:20.593 INFO:teuthology.orchestra.run.smithi077.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-15T22:00:20.610 INFO:teuthology.orchestra.run.smithi077.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-15T22:00:20.635 INFO:teuthology.orchestra.run.smithi077.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T22:00:20.642 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-15T22:00:20.645 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-15T22:00:20.661 INFO:teuthology.orchestra.run.smithi077.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T22:00:20.819 INFO:teuthology.orchestra.run.smithi077.stdout:(111/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-15T22:00:20.844 INFO:teuthology.orchestra.run.smithi077.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 824 kB/s | 151 kB 00:00 2024-09-15T22:00:20.895 INFO:teuthology.orchestra.run.smithi077.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 12 MB/s | 548 kB 00:00 2024-09-15T22:00:20.911 INFO:teuthology.orchestra.run.smithi077.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-15T22:00:20.937 INFO:teuthology.orchestra.run.smithi077.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T22:00:20.962 INFO:teuthology.orchestra.run.smithi077.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-15T22:00:21.038 INFO:teuthology.orchestra.run.smithi077.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-15T22:00:21.063 INFO:teuthology.orchestra.run.smithi077.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-09-15T22:00:21.088 INFO:teuthology.orchestra.run.smithi077.stdout:(119/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-09-15T22:00:21.113 INFO:teuthology.orchestra.run.smithi077.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-15T22:00:21.139 INFO:teuthology.orchestra.run.smithi077.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-15T22:00:21.163 INFO:teuthology.orchestra.run.smithi077.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-15T22:00:21.189 INFO:teuthology.orchestra.run.smithi077.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-15T22:00:21.214 INFO:teuthology.orchestra.run.smithi077.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-15T22:00:21.239 INFO:teuthology.orchestra.run.smithi077.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-15T22:00:21.265 INFO:teuthology.orchestra.run.smithi077.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-15T22:00:21.290 INFO:teuthology.orchestra.run.smithi077.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-15T22:00:21.315 INFO:teuthology.orchestra.run.smithi077.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-15T22:00:21.341 INFO:teuthology.orchestra.run.smithi077.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:00:21.366 INFO:teuthology.orchestra.run.smithi077.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T22:00:21.391 INFO:teuthology.orchestra.run.smithi077.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T22:00:21.425 INFO:teuthology.orchestra.run.smithi077.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-15T22:00:21.450 INFO:teuthology.orchestra.run.smithi077.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-15T22:00:21.475 INFO:teuthology.orchestra.run.smithi077.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-15T22:00:21.501 INFO:teuthology.orchestra.run.smithi077.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-15T22:00:21.526 INFO:teuthology.orchestra.run.smithi077.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-15T22:00:21.551 INFO:teuthology.orchestra.run.smithi077.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-15T22:00:21.602 INFO:teuthology.orchestra.run.smithi077.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-09-15T22:00:21.627 INFO:teuthology.orchestra.run.smithi077.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-15T22:00:21.652 INFO:teuthology.orchestra.run.smithi077.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-09-15T22:00:21.677 INFO:teuthology.orchestra.run.smithi077.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-09-15T22:00:21.703 INFO:teuthology.orchestra.run.smithi077.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 985 kB/s | 74 kB 00:00 2024-09-15T22:00:21.736 INFO:teuthology.orchestra.run.smithi077.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T22:00:21.761 INFO:teuthology.orchestra.run.smithi077.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-15T22:00:21.787 INFO:teuthology.orchestra.run.smithi077.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-15T22:00:21.812 INFO:teuthology.orchestra.run.smithi077.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T22:00:21.837 INFO:teuthology.orchestra.run.smithi077.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-15T22:00:21.888 INFO:teuthology.orchestra.run.smithi077.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T22:00:22.096 INFO:teuthology.orchestra.run.smithi077.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-15T22:00:22.180 INFO:teuthology.orchestra.run.smithi077.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.2 MB/s | 3.2 MB 00:00 2024-09-15T22:00:22.428 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-15T22:00:22.428 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-15T22:00:22.747 INFO:teuthology.orchestra.run.smithi077.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 72 kB/s | 41 kB 00:00 2024-09-15T22:00:23.064 INFO:teuthology.orchestra.run.smithi077.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-15T22:00:23.559 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T22:00:23.597 INFO:teuthology.orchestra.run.smithi077.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 293 kB/s | 249 kB 00:00 2024-09-15T22:00:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:00:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:00:23.948 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:00:23.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:00:24.031 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:00:24.082 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:00:24.322 INFO:teuthology.orchestra.run.smithi077.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 830 kB/s | 1.8 MB 00:02 2024-09-15T22:00:24.406 INFO:teuthology.orchestra.run.smithi077.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.0 MB/s | 1.4 MB 00:01 2024-09-15T22:00:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:00:24.523 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:24.631 INFO:teuthology.orchestra.run.smithi077.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-15T22:00:24.638 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:00:24.639 INFO:teuthology.orchestra.run.smithi077.stdout:Total 15 MB/s | 226 MB 00:14 2024-09-15T22:00:24.645 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:24.675 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:00:24.951 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:00:25.008 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:00:25.036 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:25.140 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:25.162 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:25.225 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:25.246 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:25.272 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:00:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:25.865 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-09-15T22:00:26.017 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-09-15T22:00:26.017 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-09-15T22:00:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:00:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:00:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:00:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:00:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:00:26.479 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:00:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:00:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:00:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:00:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:00:26.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:00:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:00:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:00:26.832 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:00:26.909 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:00:26.943 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:00:26.986 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:00:27.017 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:00:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:00:27.130 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:00:27.204 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:00:27.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:00:27.399 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:00:27.926 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:00:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:00:28.040 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:00:28.091 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:00:28.138 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:00:28.179 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:00:28.331 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:00:28.400 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:00:28.455 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:00:28.497 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-09-15T22:00:28.498 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-09-15T22:00:28.528 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:00:28.560 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:00:28.596 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:00:28.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:00:28.679 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:00:28.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:00:28.780 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:00:28.820 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-15T22:00:28.906 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:00:28.916 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:00:28.945 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:00:28.957 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:00:28.998 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:00:28.999 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:00:29.031 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:00:29.057 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:00:29.101 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:00:29.133 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:00:29.139 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:00:29.172 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:00:29.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:00:29.308 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:00:29.422 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:00:29.532 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:00:29.548 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:00:29.591 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:00:29.627 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:00:29.642 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:00:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:00:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:00:29.747 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:00:29.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:00:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:00:29.859 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:00:29.889 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:00:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:00:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:00:30.037 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:00:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:00:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:00:30.085 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:00:30.114 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:30.194 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:00:30.217 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:30.240 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:30.243 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:00:30.303 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:30.333 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:30.369 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:30.559 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:30.695 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:30.814 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:00:30.912 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:30.956 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:00:30.999 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:31.007 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:00:31.041 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:00:31.148 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:00:31.205 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:00:31.211 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:31.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:00:31.321 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:00:31.361 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:00:31.394 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:00:31.408 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:00:31.438 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:00:31.472 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:00:31.528 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:00:31.573 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:00:31.578 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:00:31.605 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:00:31.608 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:00:31.649 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:00:31.681 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:00:31.685 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:00:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:00:31.777 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:00:31.831 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:00:31.862 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:00:31.906 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:00:31.983 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:00:32.017 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:00:32.061 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:00:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:00:32.160 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:00:32.225 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:00:32.300 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:00:32.322 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:00:32.432 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:00:32.434 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:00:32.485 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:00:33.031 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:00:33.114 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:00:33.142 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:00:33.193 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:00:33.232 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:00:33.273 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:00:33.435 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:00:33.485 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:00:33.531 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:00:33.604 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:00:33.636 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:00:33.645 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:00:33.672 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:00:33.718 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:00:33.728 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:00:33.754 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:00:33.756 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:00:33.789 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:00:33.797 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:00:33.840 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:00:33.977 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:00:34.017 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:00:34.058 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:00:34.100 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:00:34.144 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:34.163 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:00:34.193 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:00:34.233 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:00:34.235 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:34.259 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:00:34.266 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:00:34.284 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:00:34.302 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:00:34.359 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:00:34.487 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:00:34.518 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:00:34.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:00:34.580 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:00:34.619 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:00:34.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:00:34.681 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:00:34.772 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:00:34.870 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:00:34.897 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:00:34.999 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:00:35.026 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-09-15T22:00:35.042 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:00:35.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:00:35.166 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:00:35.168 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:00:35.207 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:00:35.214 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:00:35.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:00:35.268 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:00:35.311 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:00:35.336 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:00:35.341 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:00:35.390 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:00:35.396 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:00:35.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:00:35.828 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:35.941 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:00:35.963 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:36.055 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:00:36.112 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:00:36.115 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:00:36.159 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:00:36.193 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:00:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:00:36.291 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:00:36.347 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:00:36.349 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:00:36.375 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:36.393 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:00:36.462 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:00:36.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:00:36.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:36.521 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:00:36.547 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:00:36.577 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:36.596 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:00:36.713 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:00:36.743 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:00:36.797 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:36.822 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:00:36.971 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:00:37.108 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:00:37.212 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:37.285 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:37.472 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:00:37.507 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:00:37.584 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:00:37.608 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:37.614 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:00:37.645 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:37.700 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:00:37.732 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:00:37.766 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:00:37.830 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:00:37.875 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:00:37.907 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:00:37.950 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:00:37.984 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:00:38.011 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:00:38.080 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:00:38.134 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:00:38.165 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:00:38.190 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:38.208 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:00:38.232 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:38.283 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:00:38.313 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:38.313 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:00:38.313 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:00:38.313 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:38.318 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:00:38.341 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:00:38.362 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:00:38.390 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:00:38.461 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:00:38.522 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:00:38.597 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:00:38.727 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:00:38.782 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:00:38.815 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:00:38.889 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:00:38.917 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:00:38.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:00:39.268 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:00:39.295 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:00:39.313 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:39.379 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:00:39.404 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:39.407 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:00:39.435 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:00:39.450 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:00:39.471 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:00:39.488 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:00:39.530 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:00:39.700 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:00:39.749 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:00:39.795 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:00:39.869 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:00:39.900 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:00:39.944 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:00:39.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:00:39.983 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:39.991 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:00:40.027 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:00:40.037 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:40.067 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:00:40.070 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:00:40.112 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:00:40.249 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:00:40.289 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:00:40.331 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:00:40.432 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:00:40.465 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:00:40.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:00:40.538 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:00:40.563 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:00:40.652 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:00:40.781 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:00:40.811 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:00:40.840 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:00:40.930 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:00:41.105 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:00:41.196 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:00:41.228 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:00:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:00:41.296 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:00:41.339 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:00:41.372 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:00:41.461 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:00:41.510 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:00:41.541 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:00:41.644 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:00:41.701 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:00:42.413 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:00:42.452 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:42.456 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:00:42.489 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:42.523 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:00:42.619 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:00:42.670 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:00:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:00:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:00:42.834 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:00:43.017 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:00:43.037 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:43.057 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:00:43.093 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:43.141 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:00:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:43.166 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:00:43.166 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:00:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:00:43.802 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:00:43.927 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:00:45.148 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:00:45.235 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:00:45.262 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:00:45.295 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:00:45.604 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:00:45.650 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:00:45.788 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:00:45.824 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:00:46.303 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:00:46.328 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:46.399 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:00:46.429 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:00:48.618 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:48.651 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:49.200 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:00:49.237 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:49.339 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:00:49.339 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:00:49.339 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:00:49.339 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:00:49.372 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:00:50.045 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-09-15T22:00:50.046 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:50.236 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:00:50.252 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:00:50.252 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:50.252 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:00:50.252 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:51.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:00:51.757 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:00:51.758 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:51.758 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:00:51.758 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:00:51.758 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:00:51.758 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:51.989 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:00:52.012 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:00:52.012 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:52.012 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:00:52.013 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:00:52.013 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:00:52.013 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:52.457 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:00:52.479 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:00:52.479 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:52.480 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:00:52.480 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:00:52.480 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:00:52.480 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:52.587 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:00:52.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:00:52.699 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:00:52.892 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:00:52.941 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:00:53.010 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:00:53.099 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:00:53.238 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:00:53.283 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:00:53.314 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:00:53.342 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:00:53.788 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:00:53.816 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:00:53.847 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:00:53.910 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:00:53.939 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:00:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:00:55.015 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:00:55.015 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-15T22:00:55.015 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-15T22:00:55.015 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-15T22:00:55.015 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-15T22:00:55.016 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-15T22:00:55.016 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-15T22:00:55.016 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-15T22:00:55.016 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:00:55.189 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:00:55.205 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:00:55.205 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:55.205 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:00:55.205 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:55.975 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:00:56.007 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:00:56.109 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:00:56.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:00:56.318 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:00:56.341 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:00:56.341 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:56.341 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:00:56.342 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:00:56.342 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:00:56.342 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:56.385 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:00:56.397 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:00:56.494 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:00:56.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:00:56.700 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:00:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:00:56.974 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:00:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:00:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:00:56.975 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:00:56.975 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:57.419 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:00:57.442 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:00:57.485 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:00:57.513 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:00:57.536 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:00:57.536 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:57.536 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:00:57.536 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:00:57.537 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:00:57.537 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:57.557 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:00:57.580 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:00:57.602 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:00:57.602 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:57.603 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:00:57.603 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:57.613 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:00:57.656 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:00:57.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:00:57.678 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:00:57.678 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:57.678 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:00:57.678 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:00:57.679 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:00:57.679 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:57.854 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:00:57.910 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:00:57.979 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:00:57.989 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:00:58.012 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:00:58.013 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:00:58.068 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:00:58.207 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:00:58.261 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:00:58.293 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:00:58.329 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:00:58.717 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:00:58.744 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:00:58.776 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:00:58.863 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:00:58.892 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:00:59.936 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:00:59.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:01:00.934 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:01:00.961 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:01:01.063 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:01:01.119 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:01:01.119 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-09-15T22:01:01.119 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-09-15T22:01:01.119 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-09-15T22:01:01.120 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-09-15T22:01:01.120 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-09-15T22:01:01.120 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-09-15T22:01:01.120 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-09-15T22:01:01.120 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:01.224 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:01:01.255 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:01:01.279 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:01:01.279 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:01.279 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:01:01.279 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:01:01.280 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:01:01.280 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:01.285 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:01:01.302 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:01:01.302 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:01.302 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:01:01.302 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:01.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:01:01.342 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:01:01.439 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:01:01.487 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:01:02.421 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:01:02.450 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:02.526 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:01:02.548 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:01:02.548 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:02.548 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:01:02.549 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:02.568 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:01:02.601 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:01:02.609 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:01:02.624 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:02.650 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:01:02.682 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:01:02.710 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:01:02.788 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:01:02.815 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:01:02.815 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:02.815 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:01:02.816 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:01:02.816 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:01:02.816 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:02.891 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:01:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:01:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:01:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:01:02.951 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:02.951 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:01:02.951 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:01:02.951 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:01:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:02.978 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:01:03.008 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:01:03.046 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:01:03.047 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:01:03.071 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:03.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:01:03.530 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:03.628 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:01:03.669 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:03.676 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:01:03.711 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:03.731 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:01:03.731 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:03.915 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:01:03.957 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:01:04.034 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:01:04.131 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:01:04.269 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:01:04.329 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:01:04.364 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:01:04.391 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:01:04.561 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:01:04.657 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:01:04.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:01:04.779 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:01:04.810 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:01:04.811 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:01:04.835 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:01:04.885 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:01:04.885 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:01:04.885 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:01:04.885 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:04.933 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:01:04.937 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:04.937 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:04.979 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:01:05.021 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:05.046 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:01:05.051 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:05.051 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:05.240 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:05.240 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:05.324 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:05.395 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:06.014 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:01:06.056 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:01:07.010 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:01:07.031 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:01:07.135 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:01:07.308 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:01:07.333 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:01:07.357 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:01:07.357 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:07.358 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:01:07.358 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:01:07.358 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:01:07.358 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:07.401 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:01:07.413 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:01:07.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:01:07.509 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:01:07.522 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:01:07.562 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:01:07.565 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:01:07.595 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:01:07.622 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:01:07.803 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:01:07.860 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:01:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:01:07.921 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:01:07.967 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:01:08.487 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:01:08.520 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:01:08.544 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:08.587 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:01:08.610 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:01:08.610 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:08.610 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:01:08.610 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:08.623 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:08.654 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:01:08.664 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:01:08.677 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:08.686 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:01:09.003 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:09.521 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:01:09.612 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:01:09.652 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:01:09.734 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:01:09.764 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:01:09.790 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:01:09.824 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:01:09.825 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:09.917 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:09.917 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:10.001 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:10.026 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:01:10.031 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:10.031 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:10.211 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:10.211 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:10.295 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:10.359 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:10.881 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:10.881 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:01:10.882 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:01:10.883 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:01:10.884 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:01:10.895 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:01:10.896 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:01:10.897 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:01:10.898 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:01:10.899 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:01:10.900 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:01:10.901 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:01:10.902 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:01:10.903 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:01:10.904 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:01:10.905 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:01:10.906 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:01:10.906 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:01:10.906 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:01:10.906 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:01:10.906 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:13.494 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.496 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:01:13.497 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.498 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:01:13.499 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:01:13.500 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:01:13.501 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:01:13.503 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:01:13.504 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:01:13.505 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:01:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:01:13.507 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:01:13.508 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:01:13.509 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T22:01:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:01:13.608 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:01:13.648 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:01:13.681 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:01:13.708 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:01:13.892 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:01:13.938 DEBUG:teuthology.parallel:result is None 2024-09-15T22:01:13.946 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:01:13.977 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:01:14.007 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:01:14.044 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:01:14.674 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:14.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:14.737 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:01:15.554 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:01:15.646 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:01:15.694 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:01:15.783 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:01:15.816 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:01:15.841 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:01:15.867 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:01:15.867 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:15.960 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:01:15.960 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:01:15.987 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:01:15.988 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:01:15.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:01:16.003 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:01:16.004 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:01:16.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:01:16.006 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:01:16.007 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:01:16.008 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:01:16.009 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:01:16.010 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:01:16.011 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:01:16.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:01:16.013 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:01:16.014 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:01:16.044 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:01:16.069 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:01:16.074 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:16.074 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:16.245 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:01:16.246 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:16.330 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:01:16.394 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:18.777 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.778 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.779 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:01:18.780 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:01:18.781 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:01:18.782 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:01:18.783 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.784 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:01:18.785 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:18.786 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:01:18.787 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:01:18.788 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:18.789 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:01:18.790 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:01:18.790 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:01:18.790 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:01:18.790 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:01:18.790 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:01:18.791 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:01:18.791 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:01:18.791 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-15T22:01:19.189 DEBUG:teuthology.parallel:result is None 2024-09-15T22:01:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:01:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:01:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:01:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:01:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:01:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:01:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:01:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:01:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:01:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:01:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:01:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:01:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:01:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:01:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:01:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:01:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:01:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:01:24.554 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:01:24.554 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:24.554 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:01:24.555 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.556 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:01:24.557 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.558 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:01:24.559 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:01:24.560 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:01:24.561 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:01:24.562 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:01:24.563 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:01:24.564 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:01:24.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:01:24.566 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:01:24.567 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:01:24.568 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:01:24.568 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:01:24.568 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-09-15T22:01:24.991 DEBUG:teuthology.parallel:result is None 2024-09-15T22:01:24.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:25.133 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:01:25.156 INFO:teuthology.orchestra.run.smithi077.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.156 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.156 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:01:25.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:25.294 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:01:25.321 INFO:teuthology.orchestra.run.smithi117.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.348 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.349 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:01:25.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:25.494 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:01:25.520 INFO:teuthology.orchestra.run.smithi196.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.521 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:01:25.521 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:01:25.523 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T22:01:25.523 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:01:25.523 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:01:25.550 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:01:25.550 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:01:25.582 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:01:25.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:01:25.614 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T22:01:25.614 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:01:25.614 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:01:25.638 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:01:25.702 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:01:25.702 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:01:25.732 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:01:25.801 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:01:25.801 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:01:25.831 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:01:25.901 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T22:01:25.901 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:01:25.901 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:01:25.926 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:01:25.990 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:01:25.991 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:01:26.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:01:26.088 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:01:26.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:01:26.118 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:01:26.187 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T22:01:26.188 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:01:26.188 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:01:26.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:01:26.277 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:01:26.278 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:01:26.308 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:01:26.376 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:01:26.376 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:01:26.405 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:01:26.474 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-15T22:01:26.575 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T22:01:26.576 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:26.576 INFO:tasks.cephadm:Cluster fsid is 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:01:26.576 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-15T22:01:26.576 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.77', 'mon.b': '172.21.15.117', 'mon.c': '172.21.15.196'} 2024-09-15T22:01:26.577 INFO:tasks.cephadm:First mon is mon.a on smithi077 2024-09-15T22:01:26.577 INFO:tasks.cephadm:First mgr is x 2024-09-15T22:01:26.577 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-15T22:01:26.577 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-09-15T22:01:26.602 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-09-15T22:01:26.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-15T22:01:26.658 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-15T22:01:26.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:26.800 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-15T22:01:26.938 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:01:26.939 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:01:26.939 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:01:26.940 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:27.279 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:01 /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:27.279 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:27.659 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:01 /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:27.659 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:28.044 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:01 /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:28.045 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:28.065 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:28.086 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:01:28.122 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-15T22:01:28.122 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:01:28.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:01:28.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:01:29.634 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:01:29.785 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:01:29.810 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:01:56.430 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-15T22:01:56.431 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-15T22:01:59.676 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:01:59.677 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:01:59.895 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T22:01:59.895 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:01:59.896 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:01:59.896 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-09-15T22:01:59.896 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:01:59.896 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T22:01:59.896 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T22:01:59.922 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-09-15T22:01:59.952 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-09-15T22:01:59.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-15T22:02:00.016 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-09-15T22:02:00.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-09-15T22:02:00.066 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-15T22:02:00.095 INFO:tasks.cephadm:Writing seed config... 2024-09-15T22:02:00.096 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-15T22:02:00.096 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-15T22:02:00.097 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-15T22:02:00.098 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-15T22:02:00.099 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-15T22:02:00.100 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-15T22:02:00.101 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:02:00.101 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-15T22:02:00.117 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0ddb746c-73ae-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-15T22:02:00.117 DEBUG:teuthology.orchestra.run.smithi077:mon.a> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service 2024-09-15T22:02:00.160 DEBUG:teuthology.orchestra.run.smithi077:mgr.x> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x.service 2024-09-15T22:02:00.204 INFO:tasks.cephadm:Bootstrapping... 2024-09-15T22:02:00.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.77 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:02:00.404 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:02:00.405 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '0ddb746c-73ae-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.77', '--skip-admin-label'] 2024-09-15T22:02:00.405 INFO:teuthology.orchestra.run.smithi077.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-15T22:02:00.405 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying podman|docker is present... 2024-09-15T22:02:00.428 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:02:00.428 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying lvm2 is present... 2024-09-15T22:02:00.428 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying time synchronization is in place... 2024-09-15T22:02:00.435 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:02:00.436 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:02:00.442 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:02:00.442 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:02:00.448 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-09-15T22:02:00.455 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-09-15T22:02:00.455 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:02:00.455 INFO:teuthology.orchestra.run.smithi077.stdout:Repeating the final host check... 2024-09-15T22:02:00.484 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:02:00.484 INFO:teuthology.orchestra.run.smithi077.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-15T22:02:00.484 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl is present 2024-09-15T22:02:00.484 INFO:teuthology.orchestra.run.smithi077.stdout:lvcreate is present 2024-09-15T22:02:00.490 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:02:00.491 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:02:00.497 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:02:00.497 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:02:00.503 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:Host looks OK 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:Cluster fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:Acquiring lock 140647234212864 on /run/cephadm/0ddb746c-73ae-11ef-bceb-c7b262605968.lock 2024-09-15T22:02:00.510 INFO:teuthology.orchestra.run.smithi077.stdout:Lock 140647234212864 acquired on /run/cephadm/0ddb746c-73ae-11ef-bceb-c7b262605968.lock 2024-09-15T22:02:00.511 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 3300 ... 2024-09-15T22:02:00.511 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 6789 ... 2024-09-15T22:02:00.511 INFO:teuthology.orchestra.run.smithi077.stdout:Base mon IP(s) is [172.21.15.77:3300, 172.21.15.77:6789], mon addrv is [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-15T22:02:00.514 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.77 metric 100 2024-09-15T22:02:00.514 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.77 metric 100 2024-09-15T22:02:00.516 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-15T22:02:00.517 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-15T22:02:00.517 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-15T22:02:00.519 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:02:00.520 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-09-15T22:02:00.520 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-09-15T22:02:00.520 INFO:teuthology.orchestra.run.smithi077.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-15T22:02:00.521 INFO:teuthology.orchestra.run.smithi077.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-15T22:02:00.521 INFO:teuthology.orchestra.run.smithi077.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:02:00.962 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:02:00.962 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:02:00.962 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Getting image source signatures 2024-09-15T22:02:00.963 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-15T22:02:00.963 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-15T22:02:00.963 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:02:00.963 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-15T22:02:01.892 INFO:teuthology.orchestra.run.smithi077.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:02:01.892 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:02:01.892 INFO:teuthology.orchestra.run.smithi077.stdout:Extracting ceph user uid/gid from container image... 2024-09-15T22:02:02.782 INFO:teuthology.orchestra.run.smithi077.stdout:stat: stdout 167 167 2024-09-15T22:02:02.782 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial keys... 2024-09-15T22:02:03.703 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBbWedmOkJNBxAAu/9weMeprZ7azt2HxdBt/w== 2024-09-15T22:02:04.600 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBcWedm8dcuABAA6SjByrcnT5BqoXmqBsVvKQ== 2024-09-15T22:02:05.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBcWedmyoiWNxAAz53Z3SBXDN3+8Yz8Ippwsw== 2024-09-15T22:02:05.523 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial monmap... 2024-09-15T22:02:06.436 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool for a [v2:172.21.15.77:3300,v1:172.21.15.77:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = quincy 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: set fsid to 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:02:06.437 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mon... 2024-09-15T22:02:07.367 INFO:teuthology.orchestra.run.smithi077.stdout:create mon.a on 2024-09-15T22:02:07.569 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-15T22:02:07.728 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-15T22:02:07.914 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target. 2024-09-15T22:02:07.914 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target. 2024-09-15T22:02:08.163 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:02:08.163 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-15T22:02:08.335 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target.wants/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@.service. 2024-09-15T22:02:08.850 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:02:08.850 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-15T22:02:08.857 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:02:08.858 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:02:08.858 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-15T22:02:08.858 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:02:08.858 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon to start... 2024-09-15T22:02:08.858 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon... 2024-09-15T22:02:09.901 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:09 smithi077 ceph-mon[30611]: from='client.? 172.21.15.77:0/2195598007' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-15T22:02:09.909 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout cluster: 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout id: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout services: 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.56332s) 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout data: 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-15T22:02:09.910 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pgs: 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.293+0000 7f3c0130c640 1 Processor -- start 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3c0130c640 1 -- start start 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3c0130c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3c0130c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc107010 con 0x7f3bfc106640 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3bfaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3bfaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56438/0 (socket says 172.21.15.77:56438) 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3bfaffd640 1 -- 172.21.15.77:0/3487394196 learned_addr learned my addr 172.21.15.77:0/3487394196 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.294+0000 7f3bfaffd640 1 -- 172.21.15.77:0/3487394196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc1077f0 con 0x7f3bfc106640 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.296+0000 7f3bfaffd640 1 --2- 172.21.15.77:0/3487394196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009820 tx=0x7f3bec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd224381b497f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3bfa7fc640 1 -- 172.21.15.77:0/3487394196 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bec037070 con 0x7f3bfc106640 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3bfa7fc640 1 -- 172.21.15.77:0/3487394196 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec032440 con 0x7f3bfc106640 2024-09-15T22:02:09.911 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3bfa7fc640 1 -- 172.21.15.77:0/3487394196 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bec032740 con 0x7f3bfc106640 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 -- 172.21.15.77:0/3487394196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 msgr2=0x7f3bfc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 --2- 172.21.15.77:0/3487394196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009820 tx=0x7f3bec02a270 comp rx=0 tx=0).stop 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 -- 172.21.15.77:0/3487394196 shutdown_connections 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 --2- 172.21.15.77:0/3487394196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc106a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 -- 172.21.15.77:0/3487394196 >> 172.21.15.77:0/3487394196 conn(0x7f3bfc101db0 msgr2=0x7f3bfc104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 -- 172.21.15.77:0/3487394196 shutdown_connections 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.297+0000 7f3c0130c640 1 -- 172.21.15.77:0/3487394196 wait complete. 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3c0130c640 1 Processor -- start 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3c0130c640 1 -- start start 2024-09-15T22:02:09.912 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3c0130c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3c0130c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc107d90 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3bfaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3bfaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56442/0 (socket says 172.21.15.77:56442) 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.298+0000 7f3bfaffd640 1 -- 172.21.15.77:0/2195598007 learned_addr learned my addr 172.21.15.77:0/2195598007 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bfaffd640 1 -- 172.21.15.77:0/2195598007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc19d640 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bfaffd640 1 --2- 172.21.15.77:0/2195598007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0090b0 tx=0x7f3bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bec041070 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec03c7a0 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc19d8d0 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bec037040 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc1a05c0 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3bec044070 con 0x7f3bfc106640 2024-09-15T22:02:09.913 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.299+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3bec03d3f0 con 0x7f3bfc106640 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.300+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc1083e0 con 0x7f3bfc106640 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.302+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3bec046450 con 0x7f3bfc106640 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.362+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3bfc02d940 con 0x7f3bfc106640 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.362+0000 7f3bf8ff9640 1 -- 172.21.15.77:0/2195598007 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f3bec046630 con 0x7f3bfc106640 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 msgr2=0x7f3bfc19d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 --2- 172.21.15.77:0/2195598007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0090b0 tx=0x7f3bec0046c0 comp rx=0 tx=0).stop 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 shutdown_connections 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 --2- 172.21.15.77:0/2195598007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc106640 0x7f3bfc19d100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 >> 172.21.15.77:0/2195598007 conn(0x7f3bfc101db0 msgr2=0x7f3bfc103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 shutdown_connections 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:09.363+0000 7f3c0130c640 1 -- 172.21.15.77:0/2195598007 wait complete. 2024-09-15T22:02:09.914 INFO:teuthology.orchestra.run.smithi077.stdout:mon is available 2024-09-15T22:02:09.915 INFO:teuthology.orchestra.run.smithi077.stdout:Assimilating anything we can from ceph.conf... 2024-09-15T22:02:10.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:10 smithi077 ceph-mon[30611]: from='client.? 172.21.15.77:0/1807803977' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:02:10.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:10 smithi077 ceph-mon[30611]: from='client.? 172.21.15.77:0/1807803977' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-15T22:02:10.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:10.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:02:10.915 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:02:10.916 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.303+0000 7f4b0b19e640 1 Processor -- start 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.303+0000 7f4b0b19e640 1 -- start start 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.303+0000 7f4b0b19e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.303+0000 7f4b0b19e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04107060 con 0x7f4b04106690 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.304+0000 7f4b08f3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.304+0000 7f4b08f3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56444/0 (socket says 172.21.15.77:56444) 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.304+0000 7f4b08f3a640 1 -- 172.21.15.77:0/2193092017 learned_addr learned my addr 172.21.15.77:0/2193092017 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.304+0000 7f4b08f3a640 1 -- 172.21.15.77:0/2193092017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b04107840 con 0x7f4b04106690 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.304+0000 7f4b08f3a640 1 --2- 172.21.15.77:0/2193092017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009820 tx=0x7f4b0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266616d157303619 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4afb7fe640 1 -- 172.21.15.77:0/2193092017 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00037070 con 0x7f4b04106690 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4afb7fe640 1 -- 172.21.15.77:0/2193092017 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00032440 con 0x7f4b04106690 2024-09-15T22:02:10.917 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4afb7fe640 1 -- 172.21.15.77:0/2193092017 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00032740 con 0x7f4b04106690 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 -- 172.21.15.77:0/2193092017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 msgr2=0x7f4b04106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 --2- 172.21.15.77:0/2193092017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009820 tx=0x7f4b0002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 -- 172.21.15.77:0/2193092017 shutdown_connections 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 --2- 172.21.15.77:0/2193092017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b04106a90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 -- 172.21.15.77:0/2193092017 >> 172.21.15.77:0/2193092017 conn(0x7f4b04101ea0 msgr2=0x7f4b041042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 -- 172.21.15.77:0/2193092017 shutdown_connections 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.305+0000 7f4b0b19e640 1 -- 172.21.15.77:0/2193092017 wait complete. 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b0b19e640 1 Processor -- start 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b0b19e640 1 -- start start 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b0b19e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b0b19e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04107de0 con 0x7f4b04106690 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b08f3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b08f3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56450/0 (socket says 172.21.15.77:56450) 2024-09-15T22:02:10.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b08f3a640 1 -- 172.21.15.77:0/1807803977 learned_addr learned my addr 172.21.15.77:0/1807803977 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.306+0000 7f4b08f3a640 1 -- 172.21.15.77:0/1807803977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0419b450 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4b08f3a640 1 --2- 172.21.15.77:0/1807803977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b000090b0 tx=0x7f4b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00041070 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0003c7a0 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0419b6e0 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b00037040 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0419e3d0 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b00044070 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.307+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b0003d3f0 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.308+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b04108430 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.310+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4b00046450 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.366+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4b0419faa0 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.369+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f4b00046630 con 0x7f4b04106690 2024-09-15T22:02:10.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4af9ffb640 1 -- 172.21.15.77:0/1807803977 <== mon.0 v2:172.21.15.77:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b00056070 con 0x7f4b04106690 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 msgr2=0x7f4b0419af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 --2- 172.21.15.77:0/1807803977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b000090b0 tx=0x7f4b000046c0 comp rx=0 tx=0).stop 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 shutdown_connections 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 --2- 172.21.15.77:0/1807803977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b04106690 0x7f4b0419af10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 >> 172.21.15.77:0/1807803977 conn(0x7f4b04101ea0 msgr2=0x7f4b0407c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 shutdown_connections 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:10.370+0000 7f4b0b19e640 1 -- 172.21.15.77:0/1807803977 wait complete. 2024-09-15T22:02:10.920 INFO:teuthology.orchestra.run.smithi077.stdout:Generating new minimal ceph.conf... 2024-09-15T22:02:11.924 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.285+0000 7f9ee3d83640 1 Processor -- start 2024-09-15T22:02:11.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.285+0000 7f9ee3d83640 1 -- start start 2024-09-15T22:02:11.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.285+0000 7f9ee3d83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:11.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.285+0000 7f9ee3d83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc108170 con 0x7f9edc1077a0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.285+0000 7f9ee1b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.286+0000 7f9ee1b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56464/0 (socket says 172.21.15.77:56464) 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.286+0000 7f9ee1b1f640 1 -- 172.21.15.77:0/3901312536 learned_addr learned my addr 172.21.15.77:0/3901312536 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.286+0000 7f9ee1b1f640 1 -- 172.21.15.77:0/3901312536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc108950 con 0x7f9edc1077a0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.286+0000 7f9ee1b1f640 1 --2- 172.21.15.77:0/3901312536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009820 tx=0x7f9ed802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ba8c7967d11d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee0b1d640 1 -- 172.21.15.77:0/3901312536 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ed8037070 con 0x7f9edc1077a0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee0b1d640 1 -- 172.21.15.77:0/3901312536 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ed8032900 con 0x7f9edc1077a0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee0b1d640 1 -- 172.21.15.77:0/3901312536 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ed8030850 con 0x7f9edc1077a0 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3901312536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 msgr2=0x7f9edc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 --2- 172.21.15.77:0/3901312536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009820 tx=0x7f9ed802a270 comp rx=0 tx=0).stop 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3901312536 shutdown_connections 2024-09-15T22:02:11.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 --2- 172.21.15.77:0/3901312536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc107ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3901312536 >> 172.21.15.77:0/3901312536 conn(0x7f9edc102fd0 msgr2=0x7f9edc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3901312536 shutdown_connections 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.287+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3901312536 wait complete. 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee3d83640 1 Processor -- start 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee3d83640 1 -- start start 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee3d83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee3d83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10bd00 con 0x7f9edc1077a0 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee1b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee1b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56472/0 (socket says 172.21.15.77:56472) 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee1b1f640 1 -- 172.21.15.77:0/3180393653 learned_addr learned my addr 172.21.15.77:0/3180393653 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:11.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.288+0000 7f9ee1b1f640 1 -- 172.21.15.77:0/3180393653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc19bc80 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ee1b1f640 1 --2- 172.21.15.77:0/3180393653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009940 tx=0x7f9ed8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ed8042070 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ed80308c0 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc19bf10 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ed8037040 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc19ec00 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ed8045070 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.289+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ed8047b80 con 0x7f9edc1077a0 2024-09-15T22:02:11.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.290+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc10be80 con 0x7f9edc1077a0 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.292+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9ed80327b0 con 0x7f9edc1077a0 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.349+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9edc19edd0 con 0x7f9edc1077a0 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.349+0000 7f9ed2ffd640 1 -- 172.21.15.77:0/3180393653 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f9ed803c820 con 0x7f9edc1077a0 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 msgr2=0x7f9edc19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 --2- 172.21.15.77:0/3180393653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009940 tx=0x7f9ed8032ce0 comp rx=0 tx=0).stop 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 shutdown_connections 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 --2- 172.21.15.77:0/3180393653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc1077a0 0x7f9edc19b740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 >> 172.21.15.77:0/3180393653 conn(0x7f9edc102fd0 msgr2=0x7f9edc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 shutdown_connections 2024-09-15T22:02:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:11.350+0000 7f9ee3d83640 1 -- 172.21.15.77:0/3180393653 wait complete. 2024-09-15T22:02:11.934 INFO:teuthology.orchestra.run.smithi077.stdout:Restarting the monitor... 2024-09-15T22:02:11.947 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:11 smithi077 ceph-mon[30611]: from='client.? 172.21.15.77:0/3180393653' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:12.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:11 smithi077 systemd[1]: Stopping Ceph mon.a for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:02:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a[30607]: 2024-09-15T22:02:12.207+0000 7fc0837c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:02:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a[30607]: 2024-09-15T22:02:12.207+0000 7fc0837c2640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-15T22:02:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 podman[30856]: 2024-09-15 22:02:12.340579787 +0000 UTC m=+0.244518398 container died a649a8346fb0c7f75379017f6fa9c3f863c5928c6dfd25251dc58d028eb8da20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-15T22:02:12.975 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 podman[30856]: 2024-09-15 22:02:12.715317227 +0000 UTC m=+0.619255841 container cleanup a649a8346fb0c7f75379017f6fa9c3f863c5928c6dfd25251dc58d028eb8da20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-15T22:02:12.975 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 bash[30856]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:02:13.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:12 smithi077 podman[30868]: 2024-09-15 22:02:12.974654566 +0000 UTC m=+0.625933706 container remove a649a8346fb0c7f75379017f6fa9c3f863c5928c6dfd25251dc58d028eb8da20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:02:13.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-15T22:02:13.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 systemd[1]: Stopped Ceph mon.a for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:02:13.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 systemd[1]: Starting Ceph mon.a for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:02:13.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 podman[30961]: 2024-09-15 22:02:13.334199481 +0000 UTC m=+0.111534507 container create ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:02:13.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 podman[30961]: 2024-09-15 22:02:13.238057292 +0000 UTC m=+0.015392327 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:02:13.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 podman[30961]: 2024-09-15 22:02:13.536746277 +0000 UTC m=+0.314081306 container init ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-15T22:02:13.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 podman[30961]: 2024-09-15 22:02:13.547221417 +0000 UTC m=+0.324556443 container start ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: pidfile_write: ignore empty --pid-file 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: load: jerasure load: lrc 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Git sha 0 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: DB SUMMARY 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: DB Session ID: NLGUSE3KL90ZMGKFAJTX 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-15T22:02:13.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106394 ; 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.env: 0x55c771f12940 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.info_log: 0x55c772d9b6e0 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.statistics: (nil) 2024-09-15T22:02:13.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.use_fsync: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.db_log_dir: 2024-09-15T22:02:13.829 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.wal_dir: 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.write_buffer_manager: 0x55c772daadc0 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:02:13.830 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.unordered_write: 0 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.row_cache: None 2024-09-15T22:02:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.wal_filter: None 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.wal_compression: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:02:13.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:02:13.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_open_files: -1 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Compression algorithms supported: 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kZSTD supported: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:02:13.835 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.merge_operator: 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_filter: None 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:02:13.836 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c772d9b320) 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_type: 0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_index_type: 0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_shortening: 1 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: checksum: 4 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: no_block_cache: 0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache: 0x55c772d8f1f0 2024-09-15T22:02:13.837 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_options: 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: capacity : 536870912 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_shard_bits : 4 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: strict_capacity_limit : 0 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_compressed: (nil) 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: persistent_cache: (nil) 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size: 4096 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size_deviation: 10 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_restart_interval: 16 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_block_restart_interval: 1 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: metadata_block_size: 4096 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: partition_filters: 0 2024-09-15T22:02:13.838 INFO:journalctl@ceph.mon.a.smithi077.stdout: use_delta_encoding: 1 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: filter_policy: bloomfilter 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: whole_key_filtering: 1 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: verify_compression: 0 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: format_version: 5 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_index_compression: 1 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_align: 0 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: max_auto_readahead_size: 262144 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: prepopulate_block_cache: 0 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression: NoCompression 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.num_levels: 7 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:02:13.841 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:02:13.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:02:13.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:02:13.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.ttl: 2592000 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 65de01b0-5ad9-45ea-890a-6ae6a7d9168a 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437733573254, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437733574295, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99290, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14564, "raw_average_key_size": 50, "raw_value_size": 91365, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726437733, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "65de01b0-5ad9-45ea-890a-6ae6a7d9168a", "db_session_id": "NLGUSE3KL90ZMGKFAJTX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437733574390, "job": 1, "event": "recovery_finished"} 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c772dbce00 2024-09-15T22:02:13.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: DB pointer 0x55c772eb8000 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** DB Stats ** 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-15T22:02:13.847 INFO:journalctl@ceph.mon.a.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: L0 2/0 100.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: Sum 2/0 100.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.848 INFO:journalctl@ceph.mon.a.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative compaction: 0.00 GB write, 30.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval compaction: 0.00 GB write, 30.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache BinnedLRUCache@0x55c772d8f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.849 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: starting mon.a rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???) e1 preinit fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).mds e1 new map 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).mds e1 print_map 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: e1 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: btime 2024-09-15T22:02:08:800551+0000 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: legacy client fscid: -1 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout: No filesystems configured 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:02:13.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: monmap epoch 1 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: last_changed 2024-09-15T22:02:05.827849+0000 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: min_mon_release 19 (squid) 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: election_strategy: 1 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: fsmap 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T22:02:13.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:13 smithi077 ceph-mon[30980]: mgrmap e1: no daemons active 2024-09-15T22:02:14.529 INFO:teuthology.orchestra.run.smithi077.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-15T22:02:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:14 smithi077 bash[30961]: ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da 2024-09-15T22:02:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:14 smithi077 systemd[1]: Started Ceph mon.a for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:02:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:18 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1796882398' entity='client.admin' 2024-09-15T22:02:24.159 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.780+0000 7f7fc4ee8640 1 Processor -- start 2024-09-15T22:02:24.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.780+0000 7f7fc4ee8640 1 -- start start 2024-09-15T22:02:24.163 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.780+0000 7f7fc4ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.780+0000 7f7fc4ee8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc0108170 con 0x7f7fc01077a0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.781+0000 7f7fbe59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.781+0000 7f7fbe59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56476/0 (socket says 172.21.15.77:56476) 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.781+0000 7f7fbe59c640 1 -- 172.21.15.77:0/2173523699 learned_addr learned my addr 172.21.15.77:0/2173523699 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.781+0000 7f7fbe59c640 1 -- 172.21.15.77:0/2173523699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc0108950 con 0x7f7fc01077a0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.782+0000 7f7fbe59c640 1 --2- 172.21.15.77:0/2173523699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009820 tx=0x7f7fb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f2e4bb6f452889 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fbd59a640 1 -- 172.21.15.77:0/2173523699 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb0037070 con 0x7f7fc01077a0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fbd59a640 1 -- 172.21.15.77:0/2173523699 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fb0032900 con 0x7f7fc01077a0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fbd59a640 1 -- 172.21.15.77:0/2173523699 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb0030850 con 0x7f7fc01077a0 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/2173523699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 msgr2=0x7f7fc0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 --2- 172.21.15.77:0/2173523699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009820 tx=0x7f7fb002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:24.164 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/2173523699 shutdown_connections 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 --2- 172.21.15.77:0/2173523699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc0107ba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/2173523699 >> 172.21.15.77:0/2173523699 conn(0x7f7fc0102fd0 msgr2=0x7f7fc01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/2173523699 shutdown_connections 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.783+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/2173523699 wait complete. 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.784+0000 7f7fc4ee8640 1 Processor -- start 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.784+0000 7f7fc4ee8640 1 -- start start 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.784+0000 7f7fc4ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.784+0000 7f7fc4ee8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc010bd00 con 0x7f7fc01077a0 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fbe59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fbe59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56484/0 (socket says 172.21.15.77:56484) 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fbe59c640 1 -- 172.21.15.77:0/1796882398 learned_addr learned my addr 172.21.15.77:0/1796882398 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fbe59c640 1 -- 172.21.15.77:0/1796882398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc019bc80 con 0x7f7fc01077a0 2024-09-15T22:02:24.165 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fbe59c640 1 --2- 172.21.15.77:0/1796882398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009940 tx=0x7f7fb0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb0042070 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fb00308c0 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb0037040 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc019bf10 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc019ec00 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fb0045070 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.785+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fb0047bb0 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.786+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc010be80 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.788+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7fb00327b0 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.846+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7fc0107c20 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.849+0000 7f7faf7fe640 1 -- 172.21.15.77:0/1796882398 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7fb003c760 con 0x7f7fc01077a0 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 msgr2=0x7f7fc019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:24.166 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 --2- 172.21.15.77:0/1796882398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009940 tx=0x7f7fb0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 shutdown_connections 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 --2- 172.21.15.77:0/1796882398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc01077a0 0x7f7fc019b740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 >> 172.21.15.77:0/1796882398 conn(0x7f7fc0102fd0 msgr2=0x7f7fc0103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 shutdown_connections 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:17.850+0000 7f7fc4ee8640 1 -- 172.21.15.77:0/1796882398 wait complete. 2024-09-15T22:02:24.167 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-15T22:02:24.427 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:02:24.427 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mgr... 2024-09-15T22:02:24.428 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-15T22:02:24.428 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-15T22:02:25.254 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:02:25.254 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-15T22:02:25.403 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968.target.wants/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@.service. 2024-09-15T22:02:25.941 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:02:25.941 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-15T22:02:25.948 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:02:25.948 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:02:25.948 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-15T22:02:25.948 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:02:25.957 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:02:25.957 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-15T22:02:25.964 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:02:25.964 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:02:25.964 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-15T22:02:25.964 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-15T22:02:25.964 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr to start... 2024-09-15T22:02:25.965 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr... 2024-09-15T22:02:26.096 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.028+0000 7fb130d5f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:02:26.384 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.179+0000 7fb130d5f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:02:26.385 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.250+0000 7fb130d5f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:02:26.775 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2549222547' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:02:26.775 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.416+0000 7fb130d5f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:02:26.775 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.566+0000 7fb130d5f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:02:27.075 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.889+0000 7fb130d5f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:02:27.075 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:26.950+0000 7fb130d5f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:02:27.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:27.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:02:27.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:02:27.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:02:27.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:02:27.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:02:27.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:02:08:800551+0000", 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:02:27.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:27.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:02:08.801203+0000", 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.382+0000 7f936b588640 1 Processor -- start 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.382+0000 7f936b588640 1 -- start start 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.382+0000 7f936b588640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.382+0000 7f936b588640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f936407be30 con 0x7f93641081b0 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.383+0000 7f9369324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.383+0000 7f9369324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56192/0 (socket says 172.21.15.77:56192) 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.383+0000 7f9369324640 1 -- 172.21.15.77:0/3541329661 learned_addr learned my addr 172.21.15.77:0/3541329661 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.383+0000 7f9369324640 1 -- 172.21.15.77:0/3541329661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936410aae0 con 0x7f93641081b0 2024-09-15T22:02:27.092 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.384+0000 7f9369324640 1 --2- 172.21.15.77:0/3541329661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9360009820 tx=0x7f936002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865575d59002a58d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.384+0000 7f935bfff640 1 -- 172.21.15.77:0/3541329661 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9360037070 con 0x7f93641081b0 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.384+0000 7f935bfff640 1 -- 172.21.15.77:0/3541329661 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9360032900 con 0x7f93641081b0 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.384+0000 7f935bfff640 1 -- 172.21.15.77:0/3541329661 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9360030850 con 0x7f93641081b0 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 -- 172.21.15.77:0/3541329661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 msgr2=0x7f936410a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 --2- 172.21.15.77:0/3541329661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9360009820 tx=0x7f936002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 -- 172.21.15.77:0/3541329661 shutdown_connections 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 --2- 172.21.15.77:0/3541329661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936410a5a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 -- 172.21.15.77:0/3541329661 >> 172.21.15.77:0/3541329661 conn(0x7f9364101db0 msgr2=0x7f9364104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.385+0000 7f936b588640 1 -- 172.21.15.77:0/3541329661 shutdown_connections 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.386+0000 7f936b588640 1 -- 172.21.15.77:0/3541329661 wait complete. 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.386+0000 7f936b588640 1 Processor -- start 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.386+0000 7f936b588640 1 -- start start 2024-09-15T22:02:27.093 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.386+0000 7f936b588640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.386+0000 7f936b588640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f936410bb20 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f9369324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f9369324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56200/0 (socket says 172.21.15.77:56200) 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f9369324640 1 -- 172.21.15.77:0/2549222547 learned_addr learned my addr 172.21.15.77:0/2549222547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f9369324640 1 -- 172.21.15.77:0/2549222547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936419bc90 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f9369324640 1 --2- 172.21.15.77:0/2549222547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9360031000 tx=0x7f93600326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9360042070 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9360032be0 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9360037040 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936419bf20 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936419ec10 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.387+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9360045070 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.388+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9360047a80 con 0x7f93641081b0 2024-09-15T22:02:27.094 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.388+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9364108610 con 0x7f93641081b0 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.391+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9360032d90 con 0x7f93641081b0 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.468+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f936419ede0 con 0x7f93641081b0 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.469+0000 7f935a7fc640 1 -- 172.21.15.77:0/2549222547 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7f936003d430 con 0x7f93641081b0 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 msgr2=0x7f936419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 --2- 172.21.15.77:0/2549222547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9360031000 tx=0x7f93600326e0 comp rx=0 tx=0).stop 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 shutdown_connections 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 --2- 172.21.15.77:0/2549222547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93641081b0 0x7f936419b750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 >> 172.21.15.77:0/2549222547 conn(0x7f9364101db0 msgr2=0x7f9364102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 shutdown_connections 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:26.470+0000 7f936b588640 1 -- 172.21.15.77:0/2549222547 wait complete. 2024-09-15T22:02:27.095 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (1/15)... 2024-09-15T22:02:27.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:27.137+0000 7fb130d5f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:02:28.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:28.012+0000 7fb130d5f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:02:28.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:28.079+0000 7fb130d5f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:02:29.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:28.825+0000 7fb130d5f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:02:29.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:28.945+0000 7fb130d5f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:02:29.386 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.089+0000 7fb130d5f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:02:29.386 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.155+0000 7fb130d5f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:02:29.386 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.312+0000 7fb130d5f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:02:29.386 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.384+0000 7fb130d5f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:02:29.643 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/978339653' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:02:29.644 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.447+0000 7fb130d5f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:02:29.644 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.641+0000 7fb130d5f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:02:29.924 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.710+0000 7fb130d5f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:02:29.924 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.777+0000 7fb130d5f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:02:29.924 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.862+0000 7fb130d5f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:02:29.924 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.922+0000 7fb130d5f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:02:30.212 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:29.981+0000 7fb130d5f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:02:30.836 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:02:30.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:02:30.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:02:30.849 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:02:08:800551+0000", 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:02:30.850 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:02:08.801203+0000", 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.489+0000 7f79a5a81640 1 Processor -- start 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.489+0000 7f79a5a81640 1 -- start start 2024-09-15T22:02:30.851 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.489+0000 7f79a5a81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.489+0000 7f79a5a81640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79a007bd70 con 0x7f79a0108270 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.490+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.490+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59118/0 (socket says 172.21.15.77:59118) 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.490+0000 7f799f7fe640 1 -- 172.21.15.77:0/2263099483 learned_addr learned my addr 172.21.15.77:0/2263099483 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.490+0000 7f799f7fe640 1 -- 172.21.15.77:0/2263099483 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a010aba0 con 0x7f79a0108270 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.491+0000 7f799f7fe640 1 --2- 172.21.15.77:0/2263099483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7990009820 tx=0x7f799002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d9aad3cb5693b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.491+0000 7f799e7fc640 1 -- 172.21.15.77:0/2263099483 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990037070 con 0x7f79a0108270 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.491+0000 7f799e7fc640 1 -- 172.21.15.77:0/2263099483 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7990032900 con 0x7f79a0108270 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.491+0000 7f799e7fc640 1 -- 172.21.15.77:0/2263099483 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990030850 con 0x7f79a0108270 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 -- 172.21.15.77:0/2263099483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 msgr2=0x7f79a010a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 --2- 172.21.15.77:0/2263099483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7990009820 tx=0x7f799002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 -- 172.21.15.77:0/2263099483 shutdown_connections 2024-09-15T22:02:30.852 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 --2- 172.21.15.77:0/2263099483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a010a660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 -- 172.21.15.77:0/2263099483 >> 172.21.15.77:0/2263099483 conn(0x7f79a0101db0 msgr2=0x7f79a0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 -- 172.21.15.77:0/2263099483 shutdown_connections 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.492+0000 7f79a5a81640 1 -- 172.21.15.77:0/2263099483 wait complete. 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f79a5a81640 1 Processor -- start 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f79a5a81640 1 -- start start 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f79a5a81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f79a5a81640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010bb20 con 0x7f79a0108270 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59120/0 (socket says 172.21.15.77:59120) 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f799f7fe640 1 -- 172.21.15.77:0/978339653 learned_addr learned my addr 172.21.15.77:0/978339653 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.493+0000 7f799f7fe640 1 -- 172.21.15.77:0/978339653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a0108030 con 0x7f79a0108270 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.494+0000 7f799f7fe640 1 --2- 172.21.15.77:0/978339653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79900097f0 tx=0x7f7990030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.494+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990042070 con 0x7f79a0108270 2024-09-15T22:02:30.853 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.494+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7990032ca0 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.494+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a0106220 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.494+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a0106760 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.495+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7990037040 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.495+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7990045070 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.495+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7990047b40 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.495+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a01086d0 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.499+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f799003c7f0 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.580+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f79a0106c90 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.580+0000 7f799cff9640 1 -- 172.21.15.77:0/978339653 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7f799003c9d0 con 0x7f79a0108270 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 msgr2=0x7f79a0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 --2- 172.21.15.77:0/978339653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79900097f0 tx=0x7f7990030ec0 comp rx=0 tx=0).stop 2024-09-15T22:02:30.854 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 shutdown_connections 2024-09-15T22:02:30.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 --2- 172.21.15.77:0/978339653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79a0108270 0x7f79a0107af0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:30.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 >> 172.21.15.77:0/978339653 conn(0x7f79a0101db0 msgr2=0x7f79a0102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:30.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 shutdown_connections 2024-09-15T22:02:30.855 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:29.581+0000 7f79a5a81640 1 -- 172.21.15.77:0/978339653 wait complete. 2024-09-15T22:02:30.855 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (2/15)... 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:30.779+0000 7fb130d5f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: Activating manager daemon x 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: mgrmap e2: x(active, starting, since 0.00317364s) 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:31.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: Manager daemon x is now available 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' 2024-09-15T22:02:31.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:30 smithi077 ceph-mon[30980]: from='mgr.14100 172.21.15.77:0/1181084292' entity='mgr.x' 2024-09-15T22:02:33.075 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:32 smithi077 ceph-mon[30980]: mgrmap e3: x(active, since 1.00745s) 2024-09-15T22:02:34.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:33 smithi077 ceph-mon[30980]: mgrmap e4: x(active, since 2s) 2024-09-15T22:02:34.054 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1983922509' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:02:34.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:34.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:02:34.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:02:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:02:08:800551+0000", 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:02:34.071 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:02:34.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:02:08.801203+0000", 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.236+0000 7f3575a7a640 1 Processor -- start 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.236+0000 7f3575a7a640 1 -- start start 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f3575a7a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f3575a7a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3570107010 con 0x7f3570106640 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f356f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f356f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59190/0 (socket says 172.21.15.77:59190) 2024-09-15T22:02:34.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f356f7fe640 1 -- 172.21.15.77:0/1374622577 learned_addr learned my addr 172.21.15.77:0/1374622577 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.237+0000 7f356f7fe640 1 -- 172.21.15.77:0/1374622577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35701077f0 con 0x7f3570106640 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f356f7fe640 1 --2- 172.21.15.77:0/1374622577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3560009820 tx=0x7f356002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be90159ad0d8b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f356e7fc640 1 -- 172.21.15.77:0/1374622577 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560037070 con 0x7f3570106640 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f356e7fc640 1 -- 172.21.15.77:0/1374622577 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3560032900 con 0x7f3570106640 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f356e7fc640 1 -- 172.21.15.77:0/1374622577 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560030850 con 0x7f3570106640 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f3575a7a640 1 -- 172.21.15.77:0/1374622577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 msgr2=0x7f3570106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.238+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1374622577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3560009820 tx=0x7f356002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- 172.21.15.77:0/1374622577 shutdown_connections 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1374622577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f3570106a40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- 172.21.15.77:0/1374622577 >> 172.21.15.77:0/1374622577 conn(0x7f3570101db0 msgr2=0x7f3570104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:34.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- 172.21.15.77:0/1374622577 shutdown_connections 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- 172.21.15.77:0/1374622577 wait complete. 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 Processor -- start 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- start start 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.239+0000 7f3575a7a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f357010aba0 con 0x7f3570106640 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59206/0 (socket says 172.21.15.77:59206) 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356f7fe640 1 -- 172.21.15.77:0/1983922509 learned_addr learned my addr 172.21.15.77:0/1983922509 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356f7fe640 1 -- 172.21.15.77:0/1983922509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357019bd40 con 0x7f3570106640 2024-09-15T22:02:34.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356f7fe640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35600097f0 tx=0x7f3560030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560042070 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3560032ca0 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357019bfd0 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560037040 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.240+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357019ecc0 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.241+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f3560045070 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.241+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357010ad20 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.241+0000 7f356cff9640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 0x7f355c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.241+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f356006c160 con 0x7f3570106640 2024-09-15T22:02:34.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.241+0000 7f356effd640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 0x7f355c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:34.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.242+0000 7f356effd640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 0x7f355c03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35580098c0 tx=0x7f3558006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:34.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.246+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f356006bb80 con 0x7f3570106640 2024-09-15T22:02:34.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.476+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f357002d9c0 con 0x7f3570106640 2024-09-15T22:02:34.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.476+0000 7f356cff9640 1 -- 172.21.15.77:0/1983922509 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f356003ccd0 con 0x7f3570106640 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 msgr2=0x7f355c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 0x7f355c03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35580098c0 tx=0x7f3558006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 msgr2=0x7f357019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35600097f0 tx=0x7f3560030ec0 comp rx=0 tx=0).stop 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 shutdown_connections 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f355c039160 0x7f355c03b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 --2- 172.21.15.77:0/1983922509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3570106640 0x7f357019b800 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:34.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 >> 172.21.15.77:0/1983922509 conn(0x7f3570101db0 msgr2=0x7f3570102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:34.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 shutdown_connections 2024-09-15T22:02:34.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:33.479+0000 7f3575a7a640 1 -- 172.21.15.77:0/1983922509 wait complete. 2024-09-15T22:02:34.079 INFO:teuthology.orchestra.run.smithi077.stdout:mgr is available 2024-09-15T22:02:34.985 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/809011250' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:02:35.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e92e7b640 1 Processor -- start 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e92e7b640 1 -- start start 2024-09-15T22:02:35.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e92e7b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e92e7b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c07d080 con 0x7f5e8c07e690 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e90c17640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e90c17640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59208/0 (socket says 172.21.15.77:59208) 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.440+0000 7f5e90c17640 1 -- 172.21.15.77:0/2330897883 learned_addr learned my addr 172.21.15.77:0/2330897883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.441+0000 7f5e90c17640 1 -- 172.21.15.77:0/2330897883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c07d200 con 0x7f5e8c07e690 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.441+0000 7f5e90c17640 1 --2- 172.21.15.77:0/2330897883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009820 tx=0x7f5e8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1ba02a80f46a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.441+0000 7f5e8b7fe640 1 -- 172.21.15.77:0/2330897883 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80037070 con 0x7f5e8c07e690 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.441+0000 7f5e8b7fe640 1 -- 172.21.15.77:0/2330897883 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e80032900 con 0x7f5e8c07e690 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.441+0000 7f5e8b7fe640 1 -- 172.21.15.77:0/2330897883 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80030850 con 0x7f5e8c07e690 2024-09-15T22:02:35.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 -- 172.21.15.77:0/2330897883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 msgr2=0x7f5e8c07cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/2330897883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009820 tx=0x7f5e8002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 -- 172.21.15.77:0/2330897883 shutdown_connections 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/2330897883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c07cb40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 -- 172.21.15.77:0/2330897883 >> 172.21.15.77:0/2330897883 conn(0x7f5e8c102fa0 msgr2=0x7f5e8c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 -- 172.21.15.77:0/2330897883 shutdown_connections 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.442+0000 7f5e92e7b640 1 -- 172.21.15.77:0/2330897883 wait complete. 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e92e7b640 1 Processor -- start 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e92e7b640 1 -- start start 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e92e7b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e92e7b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10ddc0 con 0x7f5e8c07e690 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e90c17640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:35.226 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e90c17640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59216/0 (socket says 172.21.15.77:59216) 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e90c17640 1 -- 172.21.15.77:0/809011250 learned_addr learned my addr 172.21.15.77:0/809011250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e90c17640 1 -- 172.21.15.77:0/809011250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19de50 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e90c17640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e80032ce0 tx=0x7f5e80032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80042070 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.443+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e800308c0 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.444+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19e0e0 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.444+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e80037040 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.444+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a0dd0 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.444+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5e80045070 con 0x7f5e8c07e690 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.445+0000 7f5e89ffb640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 0x7f5e7c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:35.227 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.445+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e8006c3b0 con 0x7f5e8c07e690 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.445+0000 7f5e8bfff640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 0x7f5e7c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.445+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c07cbc0 con 0x7f5e8c07e690 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.445+0000 7f5e8bfff640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 0x7f5e7c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e780098c0 tx=0x7f5e78006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.450+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e80045360 con 0x7f5e8c07e690 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.617+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5e8c1a24a0 con 0x7f5e8c07e690 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.618+0000 7f5e89ffb640 1 -- 172.21.15.77:0/809011250 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f5e8002ed90 con 0x7f5e8c07e690 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 msgr2=0x7f5e7c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 0x7f5e7c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e780098c0 tx=0x7f5e78006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 msgr2=0x7f5e8c19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e80032ce0 tx=0x7f5e80032d10 comp rx=0 tx=0).stop 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 shutdown_connections 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.620+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f5e7c0391b0 0x7f5e7c03b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:35.228 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.621+0000 7f5e92e7b640 1 --2- 172.21.15.77:0/809011250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e8c07e690 0x7f5e8c19d910 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:35.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.621+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 >> 172.21.15.77:0/809011250 conn(0x7f5e8c102fa0 msgr2=0x7f5e8c103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:35.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.621+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 shutdown_connections 2024-09-15T22:02:35.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:34.621+0000 7f5e92e7b640 1 -- 172.21.15.77:0/809011250 wait complete. 2024-09-15T22:02:35.229 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling cephadm module... 2024-09-15T22:02:36.176 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:35 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3185514203' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-15T22:02:36.176 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setuser ceph since I am not root 2024-09-15T22:02:36.176 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setgroup ceph since I am not root 2024-09-15T22:02:36.176 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:35.987+0000 7fef8ccf3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:02:36.176 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.044+0000 7fef8ccf3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:02:36.467 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.193+0000 7fef8ccf3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:02:36.467 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.260+0000 7fef8ccf3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:02:36.467 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.409+0000 7fef8ccf3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:02:36.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d169b4640 1 Processor -- start 2024-09-15T22:02:36.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d169b4640 1 -- start start 2024-09-15T22:02:36.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d169b4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:36.474 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d169b4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10108160 con 0x7f2d10107790 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59226/0 (socket says 172.21.15.77:59226) 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.578+0000 7f2d0ffff640 1 -- 172.21.15.77:0/300916791 learned_addr learned my addr 172.21.15.77:0/300916791 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.579+0000 7f2d0ffff640 1 -- 172.21.15.77:0/300916791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d10108940 con 0x7f2d10107790 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.579+0000 7f2d0ffff640 1 --2- 172.21.15.77:0/300916791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009820 tx=0x7f2d0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4113eaee80fb3dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d0effd640 1 -- 172.21.15.77:0/300916791 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d00037070 con 0x7f2d10107790 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d0effd640 1 -- 172.21.15.77:0/300916791 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d00032900 con 0x7f2d10107790 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d0effd640 1 -- 172.21.15.77:0/300916791 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d00030850 con 0x7f2d10107790 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 -- 172.21.15.77:0/300916791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 msgr2=0x7f2d10107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 --2- 172.21.15.77:0/300916791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009820 tx=0x7f2d0002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 -- 172.21.15.77:0/300916791 shutdown_connections 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 --2- 172.21.15.77:0/300916791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d10107b90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:36.475 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 -- 172.21.15.77:0/300916791 >> 172.21.15.77:0/300916791 conn(0x7f2d10102fa0 msgr2=0x7f2d101053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 -- 172.21.15.77:0/300916791 shutdown_connections 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.580+0000 7f2d169b4640 1 -- 172.21.15.77:0/300916791 wait complete. 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d169b4640 1 Processor -- start 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d169b4640 1 -- start start 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d169b4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d169b4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1010bcf0 con 0x7f2d10107790 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59232/0 (socket says 172.21.15.77:59232) 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d0ffff640 1 -- 172.21.15.77:0/3185514203 learned_addr learned my addr 172.21.15.77:0/3185514203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.581+0000 7f2d0ffff640 1 -- 172.21.15.77:0/3185514203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1019deb0 con 0x7f2d10107790 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d0ffff640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009940 tx=0x7f2d00032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d00042070 con 0x7f2d10107790 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d000308c0 con 0x7f2d10107790 2024-09-15T22:02:36.476 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1019e140 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d00037040 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.582+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101a0e30 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f2d00045070 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1010be70 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d0d7fa640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 0x7f2cf803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d0006c240 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d0f7fe640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 0x7f2cf803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.583+0000 7f2d0f7fe640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 0x7f2cf803b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d040098c0 tx=0x7f2d04006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.587+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d0003cd90 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.798+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2d101a14a0 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.850+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2d00045360 con 0x7f2d10107790 2024-09-15T22:02:36.477 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.851+0000 7f2d0d7fa640 1 -- 172.21.15.77:0/3185514203 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f2d0003c760 con 0x7f2d10107790 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 msgr2=0x7f2cf803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 0x7f2cf803b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d040098c0 tx=0x7f2d04006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 msgr2=0x7f2d1019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009940 tx=0x7f2d00032ce0 comp rx=0 tx=0).stop 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 shutdown_connections 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f2cf8039160 0x7f2cf803b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 --2- 172.21.15.77:0/3185514203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d10107790 0x7f2d1019d970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.852+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 >> 172.21.15.77:0/3185514203 conn(0x7f2d10102fa0 msgr2=0x7f2d10103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.853+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 shutdown_connections 2024-09-15T22:02:36.478 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:35.853+0000 7f2d169b4640 1 -- 172.21.15.77:0/3185514203 wait complete. 2024-09-15T22:02:36.745 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.555+0000 7fef8ccf3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:02:37.028 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3185514203' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-15T22:02:37.028 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-mon[30980]: mgrmap e5: x(active, since 5s) 2024-09-15T22:02:37.029 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.882+0000 7fef8ccf3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:02:37.029 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:36 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:36.944+0000 7fef8ccf3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:02:37.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:37 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:37.143+0000 7fef8ccf3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.819+0000 7f4f45962640 1 Processor -- start 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.819+0000 7f4f45962640 1 -- start start 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.819+0000 7f4f45962640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:37.640 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.820+0000 7f4f45962640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40107060 con 0x7f4f40106690 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.820+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.820+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59276/0 (socket says 172.21.15.77:59276) 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.820+0000 7f4f3effd640 1 -- 172.21.15.77:0/2969034325 learned_addr learned my addr 172.21.15.77:0/2969034325 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.820+0000 7f4f3effd640 1 -- 172.21.15.77:0/2969034325 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40107840 con 0x7f4f40106690 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.821+0000 7f4f3effd640 1 --2- 172.21.15.77:0/2969034325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009820 tx=0x7f4f3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49110f5cd9cc17c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.821+0000 7f4f3dffb640 1 -- 172.21.15.77:0/2969034325 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f30037070 con 0x7f4f40106690 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.821+0000 7f4f3dffb640 1 -- 172.21.15.77:0/2969034325 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f30032900 con 0x7f4f40106690 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.821+0000 7f4f3dffb640 1 -- 172.21.15.77:0/2969034325 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f30030850 con 0x7f4f40106690 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 -- 172.21.15.77:0/2969034325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 msgr2=0x7f4f40106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 --2- 172.21.15.77:0/2969034325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009820 tx=0x7f4f3002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 -- 172.21.15.77:0/2969034325 shutdown_connections 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 --2- 172.21.15.77:0/2969034325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f40106a90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:37.641 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 -- 172.21.15.77:0/2969034325 >> 172.21.15.77:0/2969034325 conn(0x7f4f40101ea0 msgr2=0x7f4f401042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 -- 172.21.15.77:0/2969034325 shutdown_connections 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.822+0000 7f4f45962640 1 -- 172.21.15.77:0/2969034325 wait complete. 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f45962640 1 Processor -- start 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f45962640 1 -- start start 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f45962640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f45962640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4010abf0 con 0x7f4f40106690 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59282/0 (socket says 172.21.15.77:59282) 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f3effd640 1 -- 172.21.15.77:0/988917811 learned_addr learned my addr 172.21.15.77:0/988917811 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.823+0000 7f4f3effd640 1 -- 172.21.15.77:0/988917811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019ab80 con 0x7f4f40106690 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f3effd640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009940 tx=0x7f4f30032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f30042070 con 0x7f4f40106690 2024-09-15T22:02:37.642 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f300308c0 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4019ae10 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f30037040 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.824+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4019db00 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f4f30045070 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4010ad70 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f44960640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 0x7f4f2803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f3e7fc640 1 -- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 msgr2=0x7f4f2803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f3006c2f0 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.825+0000 7f4f3e7fc640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 0x7f4f2803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:36.830+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f30045360 con 0x7f4f40106690 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.026+0000 7f4f3e7fc640 1 -- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 msgr2=0x7f4f2803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.026+0000 7f4f3e7fc640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 0x7f4f2803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:02:37.643 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.049+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4f4019e170 con 0x7f4f40106690 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.050+0000 7f4f44960640 1 -- 172.21.15.77:0/988917811 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4f3006bd10 con 0x7f4f40106690 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.051+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 msgr2=0x7f4f2803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 0x7f4f2803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 msgr2=0x7f4f4019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009940 tx=0x7f4f30032ce0 comp rx=0 tx=0).stop 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 shutdown_connections 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f4f28039200 0x7f4f2803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 --2- 172.21.15.77:0/988917811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f40106690 0x7f4f4019a640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 >> 172.21.15.77:0/988917811 conn(0x7f4f40101ea0 msgr2=0x7f4f40102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 shutdown_connections 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:37.052+0000 7f4f45962640 1 -- 172.21.15.77:0/988917811 wait complete. 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-09-15T22:02:37.644 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 5... 2024-09-15T22:02:37.951 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:37 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/988917811' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:02:38.235 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:38.052+0000 7fef8ccf3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:02:38.235 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:38.120+0000 7fef8ccf3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:02:38.942 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:38.825+0000 7fef8ccf3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:02:38.942 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:38.939+0000 7fef8ccf3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:02:39.295 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.084+0000 7fef8ccf3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:02:39.295 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.145+0000 7fef8ccf3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:02:39.295 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.292+0000 7fef8ccf3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:02:39.571 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.360+0000 7fef8ccf3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:02:39.571 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.419+0000 7fef8ccf3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:02:39.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.601+0000 7fef8ccf3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:02:39.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.666+0000 7fef8ccf3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:02:39.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.732+0000 7fef8ccf3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:02:39.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.813+0000 7fef8ccf3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:02:40.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.873+0000 7fef8ccf3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:02:40.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:39.932+0000 7fef8ccf3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:40.708+0000 7fef8ccf3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: Active manager daemon x restarted 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: Activating manager daemon x 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: osdmap e2: 0 total, 0 up, 0 in 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: mgrmap e6: x(active, starting, since 0.00390723s) 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: Manager daemon x is now available 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:02:41.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:40 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:02:42.029 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:41 smithi077 ceph-mon[30980]: Found migration_current of "None". Setting to last migration. 2024-09-15T22:02:42.029 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:41 smithi077 ceph-mon[30980]: mgrmap e7: x(active, since 1.00538s) 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4fefa640 1 Processor -- start 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4fefa640 1 -- start start 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4fefa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:42.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4fefa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4807be30 con 0x7f7c481081b0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4dc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4dc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59288/0 (socket says 172.21.15.77:59288) 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4dc96640 1 -- 172.21.15.77:0/4142569060 learned_addr learned my addr 172.21.15.77:0/4142569060 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.027+0000 7f7c4dc96640 1 -- 172.21.15.77:0/4142569060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4810aae0 con 0x7f7c481081b0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.028+0000 7f7c4dc96640 1 --2- 172.21.15.77:0/4142569060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009820 tx=0x7f7c4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b8c53cf6578f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.028+0000 7f7c4cc94640 1 -- 172.21.15.77:0/4142569060 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44037070 con 0x7f7c481081b0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.028+0000 7f7c4cc94640 1 -- 172.21.15.77:0/4142569060 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c44032900 con 0x7f7c481081b0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.028+0000 7f7c4cc94640 1 -- 172.21.15.77:0/4142569060 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44030850 con 0x7f7c481081b0 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 -- 172.21.15.77:0/4142569060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 msgr2=0x7f7c4810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/4142569060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009820 tx=0x7f7c4402a270 comp rx=0 tx=0).stop 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 -- 172.21.15.77:0/4142569060 shutdown_connections 2024-09-15T22:02:42.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/4142569060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c4810a5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 -- 172.21.15.77:0/4142569060 >> 172.21.15.77:0/4142569060 conn(0x7f7c48101db0 msgr2=0x7f7c48104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 -- 172.21.15.77:0/4142569060 shutdown_connections 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.029+0000 7f7c4fefa640 1 -- 172.21.15.77:0/4142569060 wait complete. 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4fefa640 1 Processor -- start 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4fefa640 1 -- start start 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4fefa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4fefa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4810bb20 con 0x7f7c481081b0 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4dc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4dc96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59304/0 (socket says 172.21.15.77:59304) 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4dc96640 1 -- 172.21.15.77:0/2163028598 learned_addr learned my addr 172.21.15.77:0/2163028598 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4dc96640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48107f70 con 0x7f7c481081b0 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.030+0000 7f7c4dc96640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c44031000 tx=0x7f7c440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.031+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44042070 con 0x7f7c481081b0 2024-09-15T22:02:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.031+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c48106160 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.031+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c44032be0 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.031+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44037040 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.031+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481066a0 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f7c44045070 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c3effd640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c3403bdd0 con 0x7f7c34039200 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c4406c110 con 0x7f7c481081b0 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c4d495640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 msgr2=0x7f7c3403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.032+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.233+0000 7f7c4d495640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 msgr2=0x7f7c3403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.233+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.634+0000 7f7c4d495640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 msgr2=0x7f7c3403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:42.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:38.634+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:39.435+0000 7f7c4d495640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 msgr2=0x7f7c3403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/3382222599 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:39.435+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:40.712+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f7c4406ac90 con 0x7f7c481081b0 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:40.712+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 msgr2=0x7f7c3403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:40.712+0000 7f7c3effd640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.712+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f7c4406afb0 con 0x7f7c481081b0 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.712+0000 7f7c3effd640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 0x7f7c3403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.712+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c3403bdd0 con 0x7f7c3403cb30 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.713+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 0x7f7c3403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.713+0000 7f7c4d495640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 0x7f7c3403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7c38003980 tx=0x7f7c380092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.714+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7c3403bdd0 con 0x7f7c3403cb30 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.720+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c48108610 con 0x7f7c3403cb30 2024-09-15T22:02:42.276 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c3effd640 1 -- 172.21.15.77:0/2163028598 <== mgr.14118 v2:172.21.15.77:6800/1757809354 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7c48108610 con 0x7f7c3403cb30 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 msgr2=0x7f7c3403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 0x7f7c3403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7c38003980 tx=0x7f7c380092b0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 msgr2=0x7f7c48107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c44031000 tx=0x7f7c440326e0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 shutdown_connections 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f7c3403cb30 0x7f7c3403ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:6800/3382222599,v1:172.21.15.77:6801/3382222599] conn(0x7f7c34039200 0x7f7c3403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 --2- 172.21.15.77:0/2163028598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c481081b0 0x7f7c48107a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 >> 172.21.15.77:0/2163028598 conn(0x7f7c48101db0 msgr2=0x7f7c48102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 shutdown_connections 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:41.721+0000 7f7c4fefa640 1 -- 172.21.15.77:0/2163028598 wait complete. 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 5 is available 2024-09-15T22:02:42.277 INFO:teuthology.orchestra.run.smithi077.stdout:Setting orchestrator backend to cephadm... 2024-09-15T22:02:43.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:42 smithi077 ceph-mon[30980]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:02:43.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:42 smithi077 ceph-mon[30980]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:02:43.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:42 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:43.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:42 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3e0e2f640 1 Processor -- start 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3e0e2f640 1 -- start start 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3e0e2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3e0e2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc07be80 con 0x7fe3dc108170 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.616+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39082/0 (socket says 172.21.15.77:39082) 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.617+0000 7fe3da59c640 1 -- 172.21.15.77:0/2415426845 learned_addr learned my addr 172.21.15.77:0/2415426845 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.617+0000 7fe3da59c640 1 -- 172.21.15.77:0/2415426845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc10aaa0 con 0x7fe3dc108170 2024-09-15T22:02:43.443 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.617+0000 7fe3da59c640 1 --2- 172.21.15.77:0/2415426845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009820 tx=0x7fe3cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6607c1e776a342b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3d9d9b640 1 -- 172.21.15.77:0/2415426845 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3cc037070 con 0x7fe3dc108170 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3d9d9b640 1 -- 172.21.15.77:0/2415426845 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3cc032900 con 0x7fe3dc108170 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3d9d9b640 1 -- 172.21.15.77:0/2415426845 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3cc030850 con 0x7fe3dc108170 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/2415426845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 msgr2=0x7fe3dc10a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/2415426845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009820 tx=0x7fe3cc02a270 comp rx=0 tx=0).stop 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/2415426845 shutdown_connections 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/2415426845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc10a560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/2415426845 >> 172.21.15.77:0/2415426845 conn(0x7fe3dc101db0 msgr2=0x7fe3dc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:43.444 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/2415426845 shutdown_connections 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.618+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/2415426845 wait complete. 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3e0e2f640 1 Processor -- start 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3e0e2f640 1 -- start start 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3e0e2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3e0e2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc10bb40 con 0x7fe3dc108170 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39098/0 (socket says 172.21.15.77:39098) 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.619+0000 7fe3da59c640 1 -- 172.21.15.77:0/1929952920 learned_addr learned my addr 172.21.15.77:0/1929952920 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:43.445 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3da59c640 1 -- 172.21.15.77:0/1929952920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc19bca0 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3da59c640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc031000 tx=0x7fe3cc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3cc042070 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3cc032be0 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc19bf30 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3cc037040 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.620+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc19ec20 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.621+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fe3cc045070 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.621+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3dc1085d0 con 0x7fe3dc108170 2024-09-15T22:02:43.446 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.621+0000 7fe3d37fe640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 0x7fe3c803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.621+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3cc06c380 con 0x7fe3dc108170 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.621+0000 7fe3d3fff640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 0x7fe3c803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.622+0000 7fe3d3fff640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 0x7fe3c803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40098c0 tx=0x7fe3c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.630+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3cc042210 con 0x7fe3dc108170 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.816+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe3dc103070 con 0x7fe3c80390a0 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.822+0000 7fe3d37fe640 1 -- 172.21.15.77:0/1929952920 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3dc103070 con 0x7fe3c80390a0 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 msgr2=0x7fe3c803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 0x7fe3c803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40098c0 tx=0x7fe3c4006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 msgr2=0x7fe3dc19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc031000 tx=0x7fe3cc0326e0 comp rx=0 tx=0).stop 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 shutdown_connections 2024-09-15T22:02:43.447 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fe3c80390a0 0x7fe3c803b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:43.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 --2- 172.21.15.77:0/1929952920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3dc108170 0x7fe3dc19b760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:43.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.824+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 >> 172.21.15.77:0/1929952920 conn(0x7fe3dc101db0 msgr2=0x7fe3dc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:43.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.825+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 shutdown_connections 2024-09-15T22:02:43.448 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:42.825+0000 7fe3e0e2f640 1 -- 172.21.15.77:0/1929952920 wait complete. 2024-09-15T22:02:44.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-mon[30980]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:44.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:44.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:44.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-mon[30980]: mgrmap e8: x(active, since 2s) 2024-09-15T22:02:44.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:43.692+0000 7feef400d640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:02:43] ENGINE Error in HTTPServer.serve 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._connections.run(self.expiration_interval) 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._run(expiration_interval) 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:02:44.044 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s = self.context.wrap_socket( 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: return self.sslsocket_class._create( 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.do_handshake() 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._sslobj.do_handshake() 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:02:44.045 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:43 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:44.600 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-15T22:02:44.600 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa95105d640 1 Processor -- start 2024-09-15T22:02:44.600 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa95105d640 1 -- start start 2024-09-15T22:02:44.600 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa95105d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa95105d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c07bdf0 con 0x7fa94c1081f0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39114/0 (socket says 172.21.15.77:39114) 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa94ad9d640 1 -- 172.21.15.77:0/3956623786 learned_addr learned my addr 172.21.15.77:0/3956623786 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.864+0000 7fa94ad9d640 1 -- 172.21.15.77:0/3956623786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c10ab20 con 0x7fa94c1081f0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.865+0000 7fa94ad9d640 1 --2- 172.21.15.77:0/3956623786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009820 tx=0x7fa93c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df146bc7ef137f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.865+0000 7fa949d9b640 1 -- 172.21.15.77:0/3956623786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa93c037070 con 0x7fa94c1081f0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.865+0000 7fa949d9b640 1 -- 172.21.15.77:0/3956623786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa93c032900 con 0x7fa94c1081f0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.865+0000 7fa949d9b640 1 -- 172.21.15.77:0/3956623786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa93c030850 con 0x7fa94c1081f0 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 -- 172.21.15.77:0/3956623786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 msgr2=0x7fa94c10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:44.601 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 --2- 172.21.15.77:0/3956623786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009820 tx=0x7fa93c02a270 comp rx=0 tx=0).stop 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 -- 172.21.15.77:0/3956623786 shutdown_connections 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 --2- 172.21.15.77:0/3956623786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c10a5e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 -- 172.21.15.77:0/3956623786 >> 172.21.15.77:0/3956623786 conn(0x7fa94c101db0 msgr2=0x7fa94c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 -- 172.21.15.77:0/3956623786 shutdown_connections 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 -- 172.21.15.77:0/3956623786 wait complete. 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.866+0000 7fa95105d640 1 Processor -- start 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa95105d640 1 -- start start 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa95105d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa95105d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c10bb00 con 0x7fa94c1081f0 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39118/0 (socket says 172.21.15.77:39118) 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa94ad9d640 1 -- 172.21.15.77:0/2916811572 learned_addr learned my addr 172.21.15.77:0/2916811572 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa94ad9d640 1 -- 172.21.15.77:0/2916811572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c19bcb0 con 0x7fa94c1081f0 2024-09-15T22:02:44.602 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa94ad9d640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0097f0 tx=0x7fa93c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa93c042070 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa93c032ca0 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa93c037040 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.867+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c19bf40 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.868+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c19ec30 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.868+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa93c045070 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.869+0000 7fa933fff640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 0x7fa93403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.869+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c108650 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.869+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa93c06c3b0 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.869+0000 7fa94a59c640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 0x7fa93403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.869+0000 7fa94a59c640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 0x7fa93403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9400098c0 tx=0x7fa940006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:43.874+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa93c03add0 con 0x7fa94c1081f0 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.040+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa94c103120 con 0x7fa934039200 2024-09-15T22:02:44.603 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.041+0000 7fa933fff640 1 -- 172.21.15.77:0/2916811572 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa94c103120 con 0x7fa934039200 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 msgr2=0x7fa93403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 0x7fa93403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9400098c0 tx=0x7fa940006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 msgr2=0x7fa94c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0097f0 tx=0x7fa93c030ec0 comp rx=0 tx=0).stop 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 shutdown_connections 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa934039200 0x7fa93403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 --2- 172.21.15.77:0/2916811572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa94c1081f0 0x7fa94c19b770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 >> 172.21.15.77:0/2916811572 conn(0x7fa94c101db0 msgr2=0x7fa94c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 shutdown_connections 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:44.043+0000 7fa95105d640 1 -- 172.21.15.77:0/2916811572 wait complete. 2024-09-15T22:02:44.604 INFO:teuthology.orchestra.run.smithi077.stdout:Generating ssh key... 2024-09-15T22:02:44.995 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:44 smithi077 ceph-mon[30980]: [15/Sep/2024:22:02:43] ENGINE Bus STARTING 2024-09-15T22:02:44.995 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:44 smithi077 ceph-mon[30980]: [15/Sep/2024:22:02:43] ENGINE Serving on http://172.21.15.77:8765 2024-09-15T22:02:44.995 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:44 smithi077 ceph-mon[30980]: [15/Sep/2024:22:02:43] ENGINE Serving on https://172.21.15.77:7150 2024-09-15T22:02:44.995 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:44 smithi077 ceph-mon[30980]: [15/Sep/2024:22:02:43] ENGINE Bus STARTED 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:44 smithi077 ceph-mon[30980]: [15/Sep/2024:22:02:43] ENGINE Error in HTTPServer.serve 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: Traceback (most recent call last): 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._run(expiration_interval) 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: s = self.context.wrap_socket( 2024-09-15T22:02:44.996 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: return self.sslsocket_class._create( 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: self.do_handshake() 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._sslobj.do_handshake() 2024-09-15T22:02:44.997 INFO:journalctl@ceph.mon.a.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:02:45.824 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Generating public/private rsa key pair. 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Your identification has been saved in /tmp/tmph4clypxu/key 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Your public key has been saved in /tmp/tmph4clypxu/key.pub 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: The key fingerprint is: 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: SHA256:inMuyH2Ov8U5Cuxl3GcbQ36Fl5XPOVWmcNIGaIPIvKE ceph-0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: The key's randomart image is: 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: +---[RSA 3072]----+ 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | o . . .+o. o| 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | = . + +ooo| 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | . o . . ...o| 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | E . . ++| 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | S . . +oo| 2024-09-15T22:02:45.825 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | . o + + o .| 2024-09-15T22:02:45.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | . o= * * * . | 2024-09-15T22:02:45.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | o.oO.o + = | 2024-09-15T22:02:45.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: | o=*. . | 2024-09-15T22:02:45.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: +----[SHA256]-----+ 2024-09-15T22:02:46.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-mon[30980]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:46.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:46.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:45 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.068+0000 7f0b9f672640 1 Processor -- start 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.068+0000 7f0b9f672640 1 -- start start 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.068+0000 7f0b9f672640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.068+0000 7f0b9f672640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9807be30 con 0x7f0b981081b0 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.069+0000 7f0b9d40e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.069+0000 7f0b9d40e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39130/0 (socket says 172.21.15.77:39130) 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.069+0000 7f0b9d40e640 1 -- 172.21.15.77:0/1969634529 learned_addr learned my addr 172.21.15.77:0/1969634529 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.069+0000 7f0b9d40e640 1 -- 172.21.15.77:0/1969634529 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9810aae0 con 0x7f0b981081b0 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9d40e640 1 --2- 172.21.15.77:0/1969634529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009820 tx=0x7f0b9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea4d19632e600af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:46.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b8ffff640 1 -- 172.21.15.77:0/1969634529 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94037070 con 0x7f0b981081b0 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b8ffff640 1 -- 172.21.15.77:0/1969634529 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b94032900 con 0x7f0b981081b0 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b8ffff640 1 -- 172.21.15.77:0/1969634529 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94030850 con 0x7f0b981081b0 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 -- 172.21.15.77:0/1969634529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 msgr2=0x7f0b9810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 --2- 172.21.15.77:0/1969634529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009820 tx=0x7f0b9402a270 comp rx=0 tx=0).stop 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 -- 172.21.15.77:0/1969634529 shutdown_connections 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 --2- 172.21.15.77:0/1969634529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9810a5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 -- 172.21.15.77:0/1969634529 >> 172.21.15.77:0/1969634529 conn(0x7f0b98101db0 msgr2=0x7f0b98104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 -- 172.21.15.77:0/1969634529 shutdown_connections 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.070+0000 7f0b9f672640 1 -- 172.21.15.77:0/1969634529 wait complete. 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9f672640 1 Processor -- start 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9f672640 1 -- start start 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9f672640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9f672640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9810bb20 con 0x7f0b981081b0 2024-09-15T22:02:46.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9d40e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9d40e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39138/0 (socket says 172.21.15.77:39138) 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9d40e640 1 -- 172.21.15.77:0/2964164976 learned_addr learned my addr 172.21.15.77:0/2964164976 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.071+0000 7f0b9d40e640 1 -- 172.21.15.77:0/2964164976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9819bc90 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b9d40e640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b940097f0 tx=0x7f0b94030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94042070 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b94032ca0 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9819bf20 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94037040 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9819ec10 con 0x7f0b981081b0 2024-09-15T22:02:46.108 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.072+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0b94045070 con 0x7f0b981081b0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.073+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b98108610 con 0x7f0b981081b0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.073+0000 7f0b8e7fc640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 0x7f0b8403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.073+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b9406c350 con 0x7f0b981081b0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.073+0000 7f0b9cc0d640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 0x7f0b8403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.073+0000 7f0b9cc0d640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 0x7f0b8403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.077+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b9406bca0 con 0x7f0b981081b0 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.242+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0b98103110 con 0x7f0b84039200 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.510+0000 7f0b8e7fc640 1 -- 172.21.15.77:0/2964164976 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b98103110 con 0x7f0b84039200 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 msgr2=0x7f0b8403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 0x7f0b8403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 msgr2=0x7f0b9819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0b940097f0 tx=0x7f0b94030ec0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 shutdown_connections 2024-09-15T22:02:46.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f0b84039200 0x7f0b8403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 --2- 172.21.15.77:0/2964164976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b981081b0 0x7f0b9819b750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 >> 172.21.15.77:0/2964164976 conn(0x7f0b98101db0 msgr2=0x7f0b98102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:46.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 shutdown_connections 2024-09-15T22:02:46.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:45.512+0000 7f0b9f672640 1 -- 172.21.15.77:0/2964164976 wait complete. 2024-09-15T22:02:47.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:46 smithi077 ceph-mon[30980]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:47.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:46 smithi077 ceph-mon[30980]: Generating ssh key... 2024-09-15T22:02:47.249 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:02:47.284 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bbc58c640 1 Processor -- start 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bbc58c640 1 -- start start 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bbc58c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bbc58c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4108160 con 0x7f6bb4107790 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bba328640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bba328640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39140/0 (socket says 172.21.15.77:39140) 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.479+0000 7f6bba328640 1 -- 172.21.15.77:0/2257315126 learned_addr learned my addr 172.21.15.77:0/2257315126 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.480+0000 7f6bba328640 1 -- 172.21.15.77:0/2257315126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb4108940 con 0x7f6bb4107790 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.480+0000 7f6bba328640 1 --2- 172.21.15.77:0/2257315126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009820 tx=0x7f6bb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1dc341d1890acc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.480+0000 7f6bb9326640 1 -- 172.21.15.77:0/2257315126 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb0037070 con 0x7f6bb4107790 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.480+0000 7f6bb9326640 1 -- 172.21.15.77:0/2257315126 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bb0032900 con 0x7f6bb4107790 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.480+0000 7f6bb9326640 1 -- 172.21.15.77:0/2257315126 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb0030850 con 0x7f6bb4107790 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 -- 172.21.15.77:0/2257315126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 msgr2=0x7f6bb4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:47.285 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/2257315126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009820 tx=0x7f6bb002a270 comp rx=0 tx=0).stop 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 -- 172.21.15.77:0/2257315126 shutdown_connections 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/2257315126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb4107b90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 -- 172.21.15.77:0/2257315126 >> 172.21.15.77:0/2257315126 conn(0x7f6bb4102fa0 msgr2=0x7f6bb41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 -- 172.21.15.77:0/2257315126 shutdown_connections 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.481+0000 7f6bbc58c640 1 -- 172.21.15.77:0/2257315126 wait complete. 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bbc58c640 1 Processor -- start 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bbc58c640 1 -- start start 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bbc58c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:47.286 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bbc58c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb410bcf0 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bba328640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bba328640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39148/0 (socket says 172.21.15.77:39148) 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bba328640 1 -- 172.21.15.77:0/660089737 learned_addr learned my addr 172.21.15.77:0/660089737 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bba328640 1 -- 172.21.15.77:0/660089737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb419bc80 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.482+0000 7f6bba328640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009940 tx=0x7f6bb0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb0042070 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bb00308c0 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb419bf10 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb0037040 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb419ec00 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.483+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6bb0045070 con 0x7f6bb4107790 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.484+0000 7f6bab7fe640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 0x7f6ba003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:47.287 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.484+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6bb006c320 con 0x7f6bb4107790 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.484+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb410be70 con 0x7f6bb4107790 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.484+0000 7f6bb9b27640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 0x7f6ba003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.484+0000 7f6bb9b27640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 0x7f6ba003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098c0 tx=0x7f6ba4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.488+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bb003cd90 con 0x7f6bb4107790 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.654+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6bb4104220 con 0x7f6ba00391b0 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.655+0000 7f6bab7fe640 1 -- 172.21.15.77:0/660089737 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6bb4104220 con 0x7f6ba00391b0 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.656+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 msgr2=0x7f6ba003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.656+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 0x7f6ba003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098c0 tx=0x7f6ba4006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.656+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 msgr2=0x7f6bb419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.656+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009940 tx=0x7f6bb0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 shutdown_connections 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6ba00391b0 0x7f6ba003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:47.288 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 --2- 172.21.15.77:0/660089737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb4107790 0x7f6bb419b740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 >> 172.21.15.77:0/660089737 conn(0x7f6bb4102fa0 msgr2=0x7f6bb4103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 shutdown_connections 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:46.657+0000 7f6bbc58c640 1 -- 172.21.15.77:0/660089737 wait complete. 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:Adding key to root@localhost authorized_keys... 2024-09-15T22:02:47.289 INFO:teuthology.orchestra.run.smithi077.stdout:Adding host smithi077... 2024-09-15T22:02:48.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:47 smithi077 ceph-mon[30980]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:48.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:48 smithi077 ceph-mon[30980]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:50.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:49 smithi077 ceph-mon[30980]: Deploying cephadm binary to smithi077 2024-09-15T22:02:51.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Added host 'smithi077' with addr '172.21.15.77' 2024-09-15T22:02:51.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5eccf69640 1 Processor -- start 2024-09-15T22:02:51.590 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5eccf69640 1 -- start start 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5eccf69640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5eccf69640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec807d150 con 0x7f5ec807e6d0 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5ec659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5ec659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39158/0 (socket says 172.21.15.77:39158) 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.637+0000 7f5ec659c640 1 -- 172.21.15.77:0/362906651 learned_addr learned my addr 172.21.15.77:0/362906651 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.638+0000 7f5ec659c640 1 -- 172.21.15.77:0/362906651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec807d2d0 con 0x7f5ec807e6d0 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.638+0000 7f5ec659c640 1 --2- 172.21.15.77:0/362906651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009820 tx=0x7f5eb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa56854ea376f86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:51.591 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.638+0000 7f5ec559a640 1 -- 172.21.15.77:0/362906651 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb8037070 con 0x7f5ec807e6d0 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.638+0000 7f5ec559a640 1 -- 172.21.15.77:0/362906651 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5eb8032900 con 0x7f5ec807e6d0 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.638+0000 7f5ec559a640 1 -- 172.21.15.77:0/362906651 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb8030850 con 0x7f5ec807e6d0 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 -- 172.21.15.77:0/362906651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 msgr2=0x7f5ec807cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 --2- 172.21.15.77:0/362906651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009820 tx=0x7f5eb802a270 comp rx=0 tx=0).stop 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 -- 172.21.15.77:0/362906651 shutdown_connections 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 --2- 172.21.15.77:0/362906651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec807cb80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 -- 172.21.15.77:0/362906651 >> 172.21.15.77:0/362906651 conn(0x7f5ec8102fd0 msgr2=0x7f5ec81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 -- 172.21.15.77:0/362906651 shutdown_connections 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.639+0000 7f5eccf69640 1 -- 172.21.15.77:0/362906651 wait complete. 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5eccf69640 1 Processor -- start 2024-09-15T22:02:51.592 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5eccf69640 1 -- start start 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5eccf69640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5eccf69640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810dd90 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5ec659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5ec659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39170/0 (socket says 172.21.15.77:39170) 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5ec659c640 1 -- 172.21.15.77:0/3530284742 learned_addr learned my addr 172.21.15.77:0/3530284742 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.640+0000 7f5ec659c640 1 -- 172.21.15.77:0/3530284742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec819de60 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5ec659c640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8032ce0 tx=0x7f5eb8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb8042070 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5eb80308c0 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec819e0f0 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb8037040 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.641+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81a0de0 con 0x7f5ec807e6d0 2024-09-15T22:02:51.593 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5eb8045070 con 0x7f5ec807e6d0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5eb77fe640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 0x7f5eb003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5eb806c4b0 con 0x7f5ec807e6d0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5ec5d9b640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 0x7f5eb003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec807cc00 con 0x7f5ec807e6d0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.642+0000 7f5ec5d9b640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 0x7f5eb003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0098c0 tx=0x7f5ebc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.647+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eb8045520 con 0x7f5ec807e6d0 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:47.816+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}) -- 0x7f5ec81041f0 con 0x7f5eb0039200 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.980+0000 7f5eb77fe640 1 -- 172.21.15.77:0/3530284742 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5ec81041f0 con 0x7f5eb0039200 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 msgr2=0x7f5eb003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 0x7f5eb003b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0098c0 tx=0x7f5ebc006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 msgr2=0x7f5ec819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:51.594 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8032ce0 tx=0x7f5eb8032d10 comp rx=0 tx=0).stop 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 shutdown_connections 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f5eb0039200 0x7f5eb003b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 --2- 172.21.15.77:0/3530284742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ec807e6d0 0x7f5ec819d920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 >> 172.21.15.77:0/3530284742 conn(0x7f5ec8102fd0 msgr2=0x7f5ec8103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 shutdown_connections 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:50.983+0000 7f5eccf69640 1 -- 172.21.15.77:0/3530284742 wait complete. 2024-09-15T22:02:51.595 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mon service... 2024-09-15T22:02:52.223 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:51 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:52.223 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:51 smithi077 ceph-mon[30980]: Added host smithi077 2024-09-15T22:02:52.223 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:51 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.020+0000 7f516c210640 1 Processor -- start 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f516c210640 1 -- start start 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f516c210640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f516c210640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5164108170 con 0x7f51641077a0 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f5169fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:52.941 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f5169fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52578/0 (socket says 172.21.15.77:52578) 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f5169fac640 1 -- 172.21.15.77:0/2304577435 learned_addr learned my addr 172.21.15.77:0/2304577435 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.021+0000 7f5169fac640 1 -- 172.21.15.77:0/2304577435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5164108950 con 0x7f51641077a0 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.022+0000 7f5169fac640 1 --2- 172.21.15.77:0/2304577435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5154009820 tx=0x7f515402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3529c92fc407a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.022+0000 7f5168faa640 1 -- 172.21.15.77:0/2304577435 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154037070 con 0x7f51641077a0 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.022+0000 7f5168faa640 1 -- 172.21.15.77:0/2304577435 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5154032900 con 0x7f51641077a0 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f5168faa640 1 -- 172.21.15.77:0/2304577435 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154030850 con 0x7f51641077a0 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 -- 172.21.15.77:0/2304577435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 msgr2=0x7f5164107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 --2- 172.21.15.77:0/2304577435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5154009820 tx=0x7f515402a270 comp rx=0 tx=0).stop 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 -- 172.21.15.77:0/2304577435 shutdown_connections 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 --2- 172.21.15.77:0/2304577435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f5164107ba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 -- 172.21.15.77:0/2304577435 >> 172.21.15.77:0/2304577435 conn(0x7f5164102fd0 msgr2=0x7f51641053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 -- 172.21.15.77:0/2304577435 shutdown_connections 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.023+0000 7f516c210640 1 -- 172.21.15.77:0/2304577435 wait complete. 2024-09-15T22:02:52.942 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f516c210640 1 Processor -- start 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f516c210640 1 -- start start 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f516c210640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f516c210640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f516410bd00 con 0x7f51641077a0 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f5169fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f5169fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52582/0 (socket says 172.21.15.77:52582) 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f5169fac640 1 -- 172.21.15.77:0/3834047728 learned_addr learned my addr 172.21.15.77:0/3834047728 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f5169fac640 1 -- 172.21.15.77:0/3834047728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516407e910 con 0x7f51641077a0 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.024+0000 7f5169fac640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f51540097f0 tx=0x7f5154030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.025+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154037040 con 0x7f51641077a0 2024-09-15T22:02:52.943 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.025+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5154040070 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.025+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516407cb00 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.025+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f515403a390 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.025+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516407cfe0 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f515403a630 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516410be80 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f515affd640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 0x7f515003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f515406c9c0 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f51697ab640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 0x7f515003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.026+0000 7f51697ab640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 0x7f515003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51600098c0 tx=0x7f5160006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.030+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f515406ee20 con 0x7f51641077a0 2024-09-15T22:02:52.944 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.221+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f516407d890 con 0x7f51500391b0 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.225+0000 7f515affd640 1 -- 172.21.15.77:0/3834047728 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f516407d890 con 0x7f51500391b0 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 msgr2=0x7f515003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 0x7f515003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51600098c0 tx=0x7f5160006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 msgr2=0x7f516407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f51540097f0 tx=0x7f5154030d80 comp rx=0 tx=0).stop 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 shutdown_connections 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f51500391b0 0x7f515003b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 --2- 172.21.15.77:0/3834047728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51641077a0 0x7f516407e3d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 >> 172.21.15.77:0/3834047728 conn(0x7f5164102fd0 msgr2=0x7f5164103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 shutdown_connections 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:52.228+0000 7f516c210640 1 -- 172.21.15.77:0/3834047728 wait complete. 2024-09-15T22:02:52.945 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mgr service... 2024-09-15T22:02:53.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:53 smithi077 ceph-mon[30980]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:53.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:53 smithi077 ceph-mon[30980]: Saving service mon spec with placement count:5 2024-09-15T22:02:53.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:53 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:54 smithi077 ceph-mon[30980]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:54 smithi077 ceph-mon[30980]: Saving service mgr spec with placement count:2 2024-09-15T22:02:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:54 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:54.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:54 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:54.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-15T22:02:54.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.536+0000 7f63458a2640 1 Processor -- start 2024-09-15T22:02:54.964 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.536+0000 7f63458a2640 1 -- start start 2024-09-15T22:02:54.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.536+0000 7f63458a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:54.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.536+0000 7f63458a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f634007be80 con 0x7f6340108170 2024-09-15T22:02:54.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.537+0000 7f633effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:54.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.537+0000 7f633effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52596/0 (socket says 172.21.15.77:52596) 2024-09-15T22:02:54.965 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.537+0000 7f633effd640 1 -- 172.21.15.77:0/1822199574 learned_addr learned my addr 172.21.15.77:0/1822199574 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:54.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.537+0000 7f633effd640 1 -- 172.21.15.77:0/1822199574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634010aaa0 con 0x7f6340108170 2024-09-15T22:02:54.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f633effd640 1 --2- 172.21.15.77:0/1822199574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e80 tx=0x7f633002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4257b9ee11b3100 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:54.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f633dffb640 1 -- 172.21.15.77:0/1822199574 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6330036070 con 0x7f6340108170 2024-09-15T22:02:54.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f633dffb640 1 -- 172.21.15.77:0/1822199574 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63300318c0 con 0x7f6340108170 2024-09-15T22:02:54.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f633dffb640 1 -- 172.21.15.77:0/1822199574 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6330007ab0 con 0x7f6340108170 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 -- 172.21.15.77:0/1822199574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 msgr2=0x7f634010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 --2- 172.21.15.77:0/1822199574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6330009e80 tx=0x7f633002a7a0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 -- 172.21.15.77:0/1822199574 shutdown_connections 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 --2- 172.21.15.77:0/1822199574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634010a560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 -- 172.21.15.77:0/1822199574 >> 172.21.15.77:0/1822199574 conn(0x7f6340101db0 msgr2=0x7f63401041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:54.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 -- 172.21.15.77:0/1822199574 shutdown_connections 2024-09-15T22:02:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.538+0000 7f63458a2640 1 -- 172.21.15.77:0/1822199574 wait complete. 2024-09-15T22:02:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f63458a2640 1 Processor -- start 2024-09-15T22:02:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f63458a2640 1 -- start start 2024-09-15T22:02:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f63458a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f63458a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f634010bb40 con 0x7f6340108170 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f633effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f633effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52608/0 (socket says 172.21.15.77:52608) 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.539+0000 7f633effd640 1 -- 172.21.15.77:0/3612691038 learned_addr learned my addr 172.21.15.77:0/3612691038 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f633effd640 1 -- 172.21.15.77:0/3612691038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634019bca0 con 0x7f6340108170 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f633effd640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6330031ca0 tx=0x7f6330031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:54.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6330036050 con 0x7f6340108170 2024-09-15T22:02:54.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6330042070 con 0x7f6340108170 2024-09-15T22:02:54.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634019bf30 con 0x7f6340108170 2024-09-15T22:02:54.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f633003d940 con 0x7f6340108170 2024-09-15T22:02:54.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.540+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634019ec20 con 0x7f6340108170 2024-09-15T22:02:54.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.541+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f633003dbf0 con 0x7f6340108170 2024-09-15T22:02:54.971 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.541+0000 7f63448a0640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 0x7f632803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:54.971 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.541+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63401085d0 con 0x7f6340108170 2024-09-15T22:02:54.971 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.541+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6330007e00 con 0x7f6340108170 2024-09-15T22:02:54.971 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.541+0000 7f633e7fc640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 0x7f632803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:54.971 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.542+0000 7f633e7fc640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 0x7f632803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63340098c0 tx=0x7f6334006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.546+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6330044310 con 0x7f6340108170 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.730+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6340103070 con 0x7f6328039200 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.735+0000 7f63448a0640 1 -- 172.21.15.77:0/3612691038 <== mgr.14118 v2:172.21.15.77:6800/1757809354 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6340103070 con 0x7f6328039200 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 msgr2=0x7f632803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 0x7f632803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63340098c0 tx=0x7f6334006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.972 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 msgr2=0x7f634019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6330031ca0 tx=0x7f6330031cd0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 shutdown_connections 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6328039200 0x7f632803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 --2- 172.21.15.77:0/3612691038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6340108170 0x7f634019b760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.737+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 >> 172.21.15.77:0/3612691038 conn(0x7f6340101db0 msgr2=0x7f6340102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:54.973 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.738+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 shutdown_connections 2024-09-15T22:02:54.974 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:53.738+0000 7f63458a2640 1 -- 172.21.15.77:0/3612691038 wait complete. 2024-09-15T22:02:56.132 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.357+0000 7f6d41b56640 1 Processor -- start 2024-09-15T22:02:56.133 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.357+0000 7f6d41b56640 1 -- start start 2024-09-15T22:02:56.133 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.357+0000 7f6d41b56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:56.133 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.358+0000 7f6d41b56640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c107010 con 0x7f6d3c106640 2024-09-15T22:02:56.133 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.358+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.358+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52610/0 (socket says 172.21.15.77:52610) 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.358+0000 7f6d3b7fe640 1 -- 172.21.15.77:0/1997220075 learned_addr learned my addr 172.21.15.77:0/1997220075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.358+0000 7f6d3b7fe640 1 -- 172.21.15.77:0/1997220075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1077f0 con 0x7f6d3c106640 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d3b7fe640 1 --2- 172.21.15.77:0/1997220075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009820 tx=0x7f6d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c11f2fe8c8a4093 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d3a7fc640 1 -- 172.21.15.77:0/1997220075 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c037070 con 0x7f6d3c106640 2024-09-15T22:02:56.134 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d3a7fc640 1 -- 172.21.15.77:0/1997220075 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c032920 con 0x7f6d3c106640 2024-09-15T22:02:56.135 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d3a7fc640 1 -- 172.21.15.77:0/1997220075 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c030810 con 0x7f6d3c106640 2024-09-15T22:02:56.135 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 -- 172.21.15.77:0/1997220075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 msgr2=0x7f6d3c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:56.135 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1997220075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009820 tx=0x7f6d2c02a270 comp rx=0 tx=0).stop 2024-09-15T22:02:56.135 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 -- 172.21.15.77:0/1997220075 shutdown_connections 2024-09-15T22:02:56.135 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1997220075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c106a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 -- 172.21.15.77:0/1997220075 >> 172.21.15.77:0/1997220075 conn(0x7f6d3c101db0 msgr2=0x7f6d3c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 -- 172.21.15.77:0/1997220075 shutdown_connections 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.359+0000 7f6d41b56640 1 -- 172.21.15.77:0/1997220075 wait complete. 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d41b56640 1 Processor -- start 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d41b56640 1 -- start start 2024-09-15T22:02:56.136 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d41b56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:56.137 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d41b56640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c10aba0 con 0x7f6d3c106640 2024-09-15T22:02:56.137 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:56.137 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d3b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52624/0 (socket says 172.21.15.77:52624) 2024-09-15T22:02:56.137 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.360+0000 7f6d3b7fe640 1 -- 172.21.15.77:0/1521943586 learned_addr learned my addr 172.21.15.77:0/1521943586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:56.137 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d3b7fe640 1 -- 172.21.15.77:0/1521943586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c19bd20 con 0x7f6d3c106640 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d3b7fe640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c030560 tx=0x7f6d2c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c037040 con 0x7f6d3c106640 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c040070 con 0x7f6d3c106640 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c19bfb0 con 0x7f6d3c106640 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c03a6d0 con 0x7f6d3c106640 2024-09-15T22:02:56.138 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.361+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c19eca0 con 0x7f6d3c106640 2024-09-15T22:02:56.139 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.362+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6d2c03a940 con 0x7f6d3c106640 2024-09-15T22:02:56.139 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.362+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c10ad20 con 0x7f6d3c106640 2024-09-15T22:02:56.139 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.362+0000 7f6d38ff9640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 0x7f6d2803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:56.139 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.362+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6d2c06c820 con 0x7f6d3c106640 2024-09-15T22:02:56.139 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.362+0000 7f6d3affd640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 0x7f6d2803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:56.140 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.363+0000 7f6d3affd640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 0x7f6d2803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d240098c0 tx=0x7f6d24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:56.140 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.367+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d2c044c70 con 0x7f6d3c106640 2024-09-15T22:02:56.140 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.542+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6d3c106ac0 con 0x7f6d3c106640 2024-09-15T22:02:56.140 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.547+0000 7f6d38ff9640 1 -- 172.21.15.77:0/1521943586 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6d2c041270 con 0x7f6d3c106640 2024-09-15T22:02:56.140 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 msgr2=0x7f6d2803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:56.141 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 0x7f6d2803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d240098c0 tx=0x7f6d24006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:56.141 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 msgr2=0x7f6d3c19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:56.141 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c030560 tx=0x7f6d2c032670 comp rx=0 tx=0).stop 2024-09-15T22:02:56.141 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 shutdown_connections 2024-09-15T22:02:56.141 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f6d280391b0 0x7f6d2803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:56.142 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.549+0000 7f6d41b56640 1 --2- 172.21.15.77:0/1521943586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d3c106640 0x7f6d3c19b7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:56.142 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.550+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 >> 172.21.15.77:0/1521943586 conn(0x7f6d3c101db0 msgr2=0x7f6d3c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:56.142 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.550+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 shutdown_connections 2024-09-15T22:02:56.142 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:55.550+0000 7f6d41b56640 1 -- 172.21.15.77:0/1521943586 wait complete. 2024-09-15T22:02:56.691 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:56 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:56.691 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1521943586' entity='client.admin' 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa788d65640 1 Processor -- start 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa788d65640 1 -- start start 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa788d65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa788d65640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa784108170 con 0x7fa7841077a0 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa78259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:57.322 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa78259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52636/0 (socket says 172.21.15.77:52636) 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.507+0000 7fa78259c640 1 -- 172.21.15.77:0/1594819991 learned_addr learned my addr 172.21.15.77:0/1594819991 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.508+0000 7fa78259c640 1 -- 172.21.15.77:0/1594819991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784108950 con 0x7fa7841077a0 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.508+0000 7fa78259c640 1 --2- 172.21.15.77:0/1594819991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa774009820 tx=0x7fa77402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a24328ab45530a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.508+0000 7fa78159a640 1 -- 172.21.15.77:0/1594819991 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa774037070 con 0x7fa7841077a0 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.508+0000 7fa78159a640 1 -- 172.21.15.77:0/1594819991 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa774032920 con 0x7fa7841077a0 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.508+0000 7fa78159a640 1 -- 172.21.15.77:0/1594819991 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa774030810 con 0x7fa7841077a0 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 -- 172.21.15.77:0/1594819991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 msgr2=0x7fa784107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 --2- 172.21.15.77:0/1594819991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa774009820 tx=0x7fa77402a270 comp rx=0 tx=0).stop 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 -- 172.21.15.77:0/1594819991 shutdown_connections 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 --2- 172.21.15.77:0/1594819991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa784107ba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 -- 172.21.15.77:0/1594819991 >> 172.21.15.77:0/1594819991 conn(0x7fa784102fd0 msgr2=0x7fa7841053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 -- 172.21.15.77:0/1594819991 shutdown_connections 2024-09-15T22:02:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.509+0000 7fa788d65640 1 -- 172.21.15.77:0/1594819991 wait complete. 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa788d65640 1 Processor -- start 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa788d65640 1 -- start start 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa788d65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa788d65640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa78410bd00 con 0x7fa7841077a0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa78259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa78259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52652/0 (socket says 172.21.15.77:52652) 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa78259c640 1 -- 172.21.15.77:0/1355973474 learned_addr learned my addr 172.21.15.77:0/1355973474 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.510+0000 7fa78259c640 1 -- 172.21.15.77:0/1355973474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78419bcb0 con 0x7fa7841077a0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa78259c640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa774030560 tx=0x7fa774032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa774037040 con 0x7fa7841077a0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78419bf40 con 0x7fa7841077a0 2024-09-15T22:02:57.324 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa774040070 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa77403a740 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.511+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78419ec30 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa77403a9b0 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78410be80 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa7737fe640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 0x7fa76c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa77406c890 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa781d9b640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 0x7fa76c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.512+0000 7fa781d9b640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 0x7fa76c03b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098c0 tx=0x7fa778006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.517+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa774041270 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.689+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa784107c20 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.695+0000 7fa7737fe640 1 -- 172.21.15.77:0/1355973474 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa77403ada0 con 0x7fa7841077a0 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 msgr2=0x7fa76c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:57.325 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 0x7fa76c03b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098c0 tx=0x7fa778006db0 comp rx=0 tx=0).stop 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 msgr2=0x7fa78419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa774030560 tx=0x7fa774032670 comp rx=0 tx=0).stop 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 shutdown_connections 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fa76c0391b0 0x7fa76c03b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 --2- 172.21.15.77:0/1355973474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7841077a0 0x7fa78419b770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 >> 172.21.15.77:0/1355973474 conn(0x7fa784102fd0 msgr2=0x7fa784103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 shutdown_connections 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:56.698+0000 7fa788d65640 1 -- 172.21.15.77:0/1355973474 wait complete. 2024-09-15T22:02:57.326 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling the dashboard module... 2024-09-15T22:02:57.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:57 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1355973474' entity='client.admin' 2024-09-15T22:02:59.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:59.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/619293531' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-15T22:02:59.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-mon[30980]: from='mgr.14118 172.21.15.77:0/1286618391' entity='mgr.x' 2024-09-15T22:02:59.527 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setuser ceph since I am not root 2024-09-15T22:02:59.527 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setgroup ceph since I am not root 2024-09-15T22:02:59.527 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.393+0000 7f2df771f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:02:59.527 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.453+0000 7f2df771f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:02:59.823 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.606+0000 7f2df771f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:02:59.824 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.673+0000 7f2df771f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:02:59.824 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.822+0000 7f2df771f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:03:00.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:02:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:02:59.967+0000 7f2df771f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.400+0000 7fb7575de640 1 Processor -- start 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb7575de640 1 -- start start 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb7575de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb7575de640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb75007c5a0 con 0x7fb750108270 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb75537a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb75537a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52662/0 (socket says 172.21.15.77:52662) 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb75537a640 1 -- 172.21.15.77:0/1230803530 learned_addr learned my addr 172.21.15.77:0/1230803530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.401+0000 7fb75537a640 1 -- 172.21.15.77:0/1230803530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75010aba0 con 0x7fb750108270 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.402+0000 7fb75537a640 1 --2- 172.21.15.77:0/1230803530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009820 tx=0x7fb74c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb8c387ad7af4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.402+0000 7fb747fff640 1 -- 172.21.15.77:0/1230803530 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c037070 con 0x7fb750108270 2024-09-15T22:03:00.463 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.402+0000 7fb747fff640 1 -- 172.21.15.77:0/1230803530 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c032920 con 0x7fb750108270 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.402+0000 7fb747fff640 1 -- 172.21.15.77:0/1230803530 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c030810 con 0x7fb750108270 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 -- 172.21.15.77:0/1230803530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 msgr2=0x7fb75010a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 --2- 172.21.15.77:0/1230803530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009820 tx=0x7fb74c02a270 comp rx=0 tx=0).stop 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 -- 172.21.15.77:0/1230803530 shutdown_connections 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 --2- 172.21.15.77:0/1230803530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75010a660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 -- 172.21.15.77:0/1230803530 >> 172.21.15.77:0/1230803530 conn(0x7fb750101e70 msgr2=0x7fb750104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 -- 172.21.15.77:0/1230803530 shutdown_connections 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.403+0000 7fb7575de640 1 -- 172.21.15.77:0/1230803530 wait complete. 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb7575de640 1 Processor -- start 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb7575de640 1 -- start start 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb7575de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:00.464 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb7575de640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb75010bb60 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb75537a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb75537a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52670/0 (socket says 172.21.15.77:52670) 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb75537a640 1 -- 172.21.15.77:0/619293531 learned_addr learned my addr 172.21.15.77:0/619293531 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb75537a640 1 -- 172.21.15.77:0/619293531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75019bc50 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.404+0000 7fb75537a640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb74c030380 tx=0x7fb74c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.405+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c037040 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.405+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75019bee0 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.405+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c040070 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.405+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb74c03a7b0 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.405+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75019ebd0 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7501086d0 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb74c03aa30 con 0x7fb750108270 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb7467fc640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 0x7fb73c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb74c06c930 con 0x7fb750108270 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb754b79640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 0x7fb73c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.406+0000 7fb754b79640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 0x7fb73c03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7400098c0 tx=0x7fb740006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.410+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb74c044c70 con 0x7fb750108270 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:58.635+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb75019f1c0 con 0x7fb750108270 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.236+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb74c041270 con 0x7fb750108270 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.237+0000 7fb7467fc640 1 -- 172.21.15.77:0/619293531 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fb74c045980 con 0x7fb750108270 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 msgr2=0x7fb73c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 0x7fb73c03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7400098c0 tx=0x7fb740006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 msgr2=0x7fb75019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb74c030380 tx=0x7fb74c032670 comp rx=0 tx=0).stop 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 shutdown_connections 2024-09-15T22:03:00.466 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7fb73c0391b0 0x7fb73c03b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:00.467 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 --2- 172.21.15.77:0/619293531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750108270 0x7fb75019b710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:00.467 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 >> 172.21.15.77:0/619293531 conn(0x7fb750101e70 msgr2=0x7fb750102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:00.467 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 shutdown_connections 2024-09-15T22:03:00.467 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:02:59.239+0000 7fb7575de640 1 -- 172.21.15.77:0/619293531 wait complete. 2024-09-15T22:03:00.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/619293531' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-15T22:03:00.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:00 smithi077 ceph-mon[30980]: mgrmap e9: x(active, since 18s) 2024-09-15T22:03:00.535 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:00.281+0000 7f2df771f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:03:00.536 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:00.343+0000 7f2df771f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:03:00.536 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:00.533+0000 7f2df771f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:03:01.514 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:01 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/516467778' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:03:01.514 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:01.445+0000 7f2df771f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:03:01.514 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:01.512+0000 7f2df771f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:03:01.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:03:01.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-15T22:03:01.957 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.820+0000 7f9170ffb640 1 Processor -- start 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.820+0000 7f9170ffb640 1 -- start start 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.820+0000 7f9170ffb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.820+0000 7f9170ffb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f916c07c5a0 con 0x7f916c108270 2024-09-15T22:03:01.958 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.820+0000 7f916a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.821+0000 7f916a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33196/0 (socket says 172.21.15.77:33196) 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.821+0000 7f916a59c640 1 -- 172.21.15.77:0/273775095 learned_addr learned my addr 172.21.15.77:0/273775095 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.821+0000 7f916a59c640 1 -- 172.21.15.77:0/273775095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c10aba0 con 0x7f916c108270 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.821+0000 7f916a59c640 1 --2- 172.21.15.77:0/273775095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f915c009820 tx=0x7f915c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b944d013c4612d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9169d9b640 1 -- 172.21.15.77:0/273775095 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c037070 con 0x7f916c108270 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9169d9b640 1 -- 172.21.15.77:0/273775095 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c032920 con 0x7f916c108270 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9169d9b640 1 -- 172.21.15.77:0/273775095 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c030810 con 0x7f916c108270 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9170ffb640 1 -- 172.21.15.77:0/273775095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 msgr2=0x7f916c10a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9170ffb640 1 --2- 172.21.15.77:0/273775095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f915c009820 tx=0x7f915c02a270 comp rx=0 tx=0).stop 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9170ffb640 1 -- 172.21.15.77:0/273775095 shutdown_connections 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9170ffb640 1 --2- 172.21.15.77:0/273775095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c10a660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.822+0000 7f9170ffb640 1 -- 172.21.15.77:0/273775095 >> 172.21.15.77:0/273775095 conn(0x7f916c101e70 msgr2=0x7f916c104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:01.959 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 -- 172.21.15.77:0/273775095 shutdown_connections 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 -- 172.21.15.77:0/273775095 wait complete. 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 Processor -- start 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 -- start start 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.823+0000 7f9170ffb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10bb60 con 0x7f916c108270 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f916a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f916a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33210/0 (socket says 172.21.15.77:33210) 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f916a59c640 1 -- 172.21.15.77:0/516467778 learned_addr learned my addr 172.21.15.77:0/516467778 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f916a59c640 1 -- 172.21.15.77:0/516467778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c19bc50 con 0x7f916c108270 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f916a59c640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f915c030380 tx=0x7f915c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c037040 con 0x7f916c108270 2024-09-15T22:03:01.960 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c19bee0 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c040070 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915c03a7b0 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.824+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c19ebd0 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.825+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f915c03aa30 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.825+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916c1086d0 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.825+0000 7f91637fe640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 0x7f915803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.825+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f915c06ca00 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.825+0000 7f9163fff640 1 -- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 msgr2=0x7f915803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.826+0000 7f9163fff640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 0x7f915803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:00.830+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f915c044c70 con 0x7f916c108270 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.027+0000 7f9163fff640 1 -- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 msgr2=0x7f915803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.027+0000 7f9163fff640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 0x7f915803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:03:01.961 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.051+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f916c19eda0 con 0x7f916c108270 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.051+0000 7f91637fe640 1 -- 172.21.15.77:0/516467778 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f916c19eda0 con 0x7f916c108270 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 msgr2=0x7f915803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 0x7f915803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 msgr2=0x7f916c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f915c030380 tx=0x7f915c032670 comp rx=0 tx=0).stop 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 shutdown_connections 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7f9158039200 0x7f915803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 --2- 172.21.15.77:0/516467778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f916c108270 0x7f916c19b710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.053+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 >> 172.21.15.77:0/516467778 conn(0x7f916c101e70 msgr2=0x7f916c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.054+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 shutdown_connections 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:01.054+0000 7f9170ffb640 1 -- 172.21.15.77:0/516467778 wait complete. 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-09-15T22:03:01.962 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 9... 2024-09-15T22:03:02.501 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.235+0000 7f2df771f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:03:02.501 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.357+0000 7f2df771f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:03:02.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.499+0000 7f2df771f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:03:02.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.560+0000 7f2df771f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:03:02.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.708+0000 7f2df771f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:03:02.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.774+0000 7f2df771f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:03:03.055 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.833+0000 7f2df771f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:03:03.056 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:02.990+0000 7f2df771f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:03:03.056 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:03.054+0000 7f2df771f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:03:03.319 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:03.120+0000 7f2df771f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:03:03.319 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:03.200+0000 7f2df771f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:03:03.319 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:03.259+0000 7f2df771f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:03:03.319 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:03.317+0000 7f2df771f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:04.099+0000 7f2df771f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: Active manager daemon x restarted 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: Activating manager daemon x 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: osdmap e3: 0 total, 0 up, 0 in 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: mgrmap e10: x(active, starting, since 0.00348045s) 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:03:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: Manager daemon x is now available 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:03:04.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:03:05.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:05.274+0000 7f2d5d1e8640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:03:05] ENGINE Error in HTTPServer.serve 2024-09-15T22:03:05.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-15T22:03:05.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._connections.run(self.expiration_interval) 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._run(expiration_interval) 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s = self.context.wrap_socket( 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: return self.sslsocket_class._create( 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:03:05.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.do_handshake() 2024-09-15T22:03:05.584 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:03:05.584 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._sslobj.do_handshake() 2024-09-15T22:03:05.584 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:03:05.584 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:03:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:03:06.052 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.340+0000 7ff96ef20640 1 Processor -- start 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.340+0000 7ff96ef20640 1 -- start start 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ef20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ef20640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff96807be30 con 0x7ff9681081b0 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ccbc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ccbc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33226/0 (socket says 172.21.15.77:33226) 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ccbc640 1 -- 172.21.15.77:0/588019600 learned_addr learned my addr 172.21.15.77:0/588019600 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:06.053 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.341+0000 7ff96ccbc640 1 -- 172.21.15.77:0/588019600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96810aae0 con 0x7ff9681081b0 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.342+0000 7ff96ccbc640 1 --2- 172.21.15.77:0/588019600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009820 tx=0x7ff95c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9265728a3b1f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.342+0000 7ff9677fe640 1 -- 172.21.15.77:0/588019600 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff9681081b0 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.342+0000 7ff9677fe640 1 -- 172.21.15.77:0/588019600 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff95c032920 con 0x7ff9681081b0 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.342+0000 7ff9677fe640 1 -- 172.21.15.77:0/588019600 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c030810 con 0x7ff9681081b0 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- 172.21.15.77:0/588019600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 msgr2=0x7ff96810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 --2- 172.21.15.77:0/588019600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009820 tx=0x7ff95c02a270 comp rx=0 tx=0).stop 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- 172.21.15.77:0/588019600 shutdown_connections 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 --2- 172.21.15.77:0/588019600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96810a5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- 172.21.15.77:0/588019600 >> 172.21.15.77:0/588019600 conn(0x7ff968101db0 msgr2=0x7ff968104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- 172.21.15.77:0/588019600 shutdown_connections 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- 172.21.15.77:0/588019600 wait complete. 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 Processor -- start 2024-09-15T22:03:06.054 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- start start 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ef20640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff96810bb60 con 0x7ff9681081b0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ccbc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ccbc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33242/0 (socket says 172.21.15.77:33242) 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.343+0000 7ff96ccbc640 1 -- 172.21.15.77:0/1373542044 learned_addr learned my addr 172.21.15.77:0/1373542044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff96ccbc640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96819bcb0 con 0x7ff9681081b0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff96ccbc640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff95c030420 tx=0x7ff95c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c037040 con 0x7ff9681081b0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff95c040070 con 0x7ff9681081b0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96819bf40 con 0x7ff9681081b0 2024-09-15T22:03:06.055 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c03a6e0 con 0x7ff9681081b0 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.344+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96819ec30 con 0x7ff9681081b0 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7ff95c03a9e0 con 0x7ff9681081b0 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff965ffb640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff95803bdd0 con 0x7ff958039200 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff95c06c930 con 0x7ff9681081b0 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff967fff640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 msgr2=0x7ff95803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.345+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.546+0000 7ff967fff640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 msgr2=0x7ff95803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.546+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.947+0000 7ff967fff640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 msgr2=0x7ff95803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:02.947+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:03:06.056 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:03.748+0000 7ff967fff640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 msgr2=0x7ff95803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1757809354 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:03.748+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:04.103+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7ff95c06b3e0 con 0x7ff9681081b0 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:04.103+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 msgr2=0x7ff95803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:04.103+0000 7ff965ffb640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.105+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7ff95c06eb60 con 0x7ff9681081b0 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.105+0000 7ff965ffb640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 0x7ff95803f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.105+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff95c043a20 con 0x7ff95803ccc0 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.105+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 0x7ff95803f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.106+0000 7ff967fff640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 0x7ff95803f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff954003980 tx=0x7ff954006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:06.057 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.107+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff95c043a20 con 0x7ff95803ccc0 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff968108610 con 0x7ff95803ccc0 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff965ffb640 1 -- 172.21.15.77:0/1373542044 <== mgr.14150 v2:172.21.15.77:6800/1809193257 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff968108610 con 0x7ff95803ccc0 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 msgr2=0x7ff95803f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 0x7ff95803f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff954003980 tx=0x7ff954006e10 comp rx=0 tx=0).stop 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 msgr2=0x7ff96819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff95c030420 tx=0x7ff95c032670 comp rx=0 tx=0).stop 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 shutdown_connections 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff95803ccc0 0x7ff95803f0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:6800/1757809354,v1:172.21.15.77:6801/1757809354] conn(0x7ff958039200 0x7ff95803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 --2- 172.21.15.77:0/1373542044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9681081b0 0x7ff96819b770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 >> 172.21.15.77:0/1373542044 conn(0x7ff968101db0 msgr2=0x7ff968102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 shutdown_connections 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:05.114+0000 7ff96ef20640 1 -- 172.21.15.77:0/1373542044 wait complete. 2024-09-15T22:03:06.058 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 9 is available 2024-09-15T22:03:06.059 INFO:teuthology.orchestra.run.smithi077.stdout:Generating a dashboard self-signed certificate... 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: [15/Sep/2024:22:03:05] ENGINE Bus STARTING 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: mgrmap e11: x(active, since 1.00562s) 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: [15/Sep/2024:22:03:05] ENGINE Serving on http://172.21.15.77:8765 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: [15/Sep/2024:22:03:05] ENGINE Serving on https://172.21.15.77:7150 2024-09-15T22:03:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: [15/Sep/2024:22:03:05] ENGINE Bus STARTED 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: [15/Sep/2024:22:03:05] ENGINE Error in HTTPServer.serve 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: Traceback (most recent call last): 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._run(expiration_interval) 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: s = self.context.wrap_socket( 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:03:06.333 INFO:journalctl@ceph.mon.a.smithi077.stdout: return self.sslsocket_class._create( 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout: self.do_handshake() 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._sslobj.do_handshake() 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:06.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:07.335 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-15T22:03:07.335 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.463+0000 7f634d278640 1 Processor -- start 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.463+0000 7f634d278640 1 -- start start 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f634d278640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f634d278640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f634807d080 con 0x7f634807e690 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f6346ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f6346ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33324/0 (socket says 172.21.15.77:33324) 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f6346ffd640 1 -- 172.21.15.77:0/3843375200 learned_addr learned my addr 172.21.15.77:0/3843375200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.464+0000 7f6346ffd640 1 -- 172.21.15.77:0/3843375200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634807d200 con 0x7f634807e690 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.465+0000 7f6346ffd640 1 --2- 172.21.15.77:0/3843375200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6338009e80 tx=0x7f633802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d804b35ad94ccb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.465+0000 7f6345ffb640 1 -- 172.21.15.77:0/3843375200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6338036070 con 0x7f634807e690 2024-09-15T22:03:07.336 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.465+0000 7f6345ffb640 1 -- 172.21.15.77:0/3843375200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63380318e0 con 0x7f634807e690 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.465+0000 7f6345ffb640 1 -- 172.21.15.77:0/3843375200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6338007a00 con 0x7f634807e690 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- 172.21.15.77:0/3843375200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 msgr2=0x7f634807cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 --2- 172.21.15.77:0/3843375200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6338009e80 tx=0x7f633802a7a0 comp rx=0 tx=0).stop 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- 172.21.15.77:0/3843375200 shutdown_connections 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 --2- 172.21.15.77:0/3843375200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634807cb40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- 172.21.15.77:0/3843375200 >> 172.21.15.77:0/3843375200 conn(0x7f6348102fd0 msgr2=0x7f63481053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- 172.21.15.77:0/3843375200 shutdown_connections 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- 172.21.15.77:0/3843375200 wait complete. 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 Processor -- start 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.466+0000 7f634d278640 1 -- start start 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f634d278640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:07.337 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f634d278640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f634810ddd0 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f6346ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f6346ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33334/0 (socket says 172.21.15.77:33334) 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f6346ffd640 1 -- 172.21.15.77:0/2386155013 learned_addr learned my addr 172.21.15.77:0/2386155013 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f6346ffd640 1 -- 172.21.15.77:0/2386155013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634819de50 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f6346ffd640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6338031cf0 tx=0x7f6338031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6338036050 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634819e0e0 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6338042070 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f633803d9b0 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.467+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a0dd0 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.468+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f633803dc60 con 0x7f634807e690 2024-09-15T22:03:07.338 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.468+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634807cbc0 con 0x7f634807e690 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.468+0000 7f632ffff640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 0x7f633003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.468+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f633806d8a0 con 0x7f634807e690 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.468+0000 7f63467fc640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 0x7f633003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.469+0000 7f63467fc640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 0x7f633003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098c0 tx=0x7f633c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.473+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6338044310 con 0x7f634807e690 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.652+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6348104260 con 0x7f63300390f0 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.735+0000 7f632ffff640 1 -- 172.21.15.77:0/2386155013 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6348104260 con 0x7f63300390f0 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.737+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 msgr2=0x7f633003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.737+0000 7f634d278640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 0x7f633003b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098c0 tx=0x7f633c006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.737+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 msgr2=0x7f634819d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:07.339 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.737+0000 7f634d278640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6338031cf0 tx=0x7f6338031d20 comp rx=0 tx=0).stop 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 shutdown_connections 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f63300390f0 0x7f633003b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 --2- 172.21.15.77:0/2386155013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634807e690 0x7f634819d910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 >> 172.21.15.77:0/2386155013 conn(0x7f6348102fd0 msgr2=0x7f6348103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 shutdown_connections 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:06.738+0000 7f634d278640 1 -- 172.21.15.77:0/2386155013 wait complete. 2024-09-15T22:03:07.340 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial admin user... 2024-09-15T22:03:07.998 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:07 smithi077 ceph-mon[30980]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:07.998 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:07.998 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:07.998 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:07 smithi077 ceph-mon[30980]: mgrmap e12: x(active, since 2s) 2024-09-15T22:03:08.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9QYQtV3skGpq4VXJR1K72.ayMC1gvzuHthRYuGFJvPLP333bUvBCS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726437788, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93df089640 1 Processor -- start 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93df089640 1 -- start start 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93df089640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93df089640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8108170 con 0x7f93d81077a0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93dce25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93dce25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33336/0 (socket says 172.21.15.77:33336) 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.729+0000 7f93dce25640 1 -- 172.21.15.77:0/3577419648 learned_addr learned my addr 172.21.15.77:0/3577419648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.730+0000 7f93dce25640 1 -- 172.21.15.77:0/3577419648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d8108950 con 0x7f93d81077a0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.730+0000 7f93dce25640 1 --2- 172.21.15.77:0/3577419648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009820 tx=0x7f93cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a03ed126774515e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.730+0000 7f93d77fe640 1 -- 172.21.15.77:0/3577419648 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93cc037070 con 0x7f93d81077a0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.730+0000 7f93d77fe640 1 -- 172.21.15.77:0/3577419648 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc032920 con 0x7f93d81077a0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.730+0000 7f93d77fe640 1 -- 172.21.15.77:0/3577419648 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93cc030810 con 0x7f93d81077a0 2024-09-15T22:03:08.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 -- 172.21.15.77:0/3577419648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 msgr2=0x7f93d8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 --2- 172.21.15.77:0/3577419648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009820 tx=0x7f93cc02a270 comp rx=0 tx=0).stop 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 -- 172.21.15.77:0/3577419648 shutdown_connections 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 --2- 172.21.15.77:0/3577419648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d8107ba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 -- 172.21.15.77:0/3577419648 >> 172.21.15.77:0/3577419648 conn(0x7f93d8102fd0 msgr2=0x7f93d81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 -- 172.21.15.77:0/3577419648 shutdown_connections 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.731+0000 7f93df089640 1 -- 172.21.15.77:0/3577419648 wait complete. 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93df089640 1 Processor -- start 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93df089640 1 -- start start 2024-09-15T22:03:08.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93df089640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93df089640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810bd00 con 0x7f93d81077a0 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93dce25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93dce25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33344/0 (socket says 172.21.15.77:33344) 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93dce25640 1 -- 172.21.15.77:0/3193450762 learned_addr learned my addr 172.21.15.77:0/3193450762 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.732+0000 7f93dce25640 1 -- 172.21.15.77:0/3193450762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d807e910 con 0x7f93d81077a0 2024-09-15T22:03:08.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.733+0000 7f93dce25640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93cc030560 tx=0x7f93cc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.735+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93cc037040 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.735+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc040070 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.735+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d807cb00 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.735+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93cc03a740 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.735+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d807d040 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.736+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f93cc03a8e0 con 0x7f93d81077a0 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.736+0000 7f93d5ffb640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 0x7f93c803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:08.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.736+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93cc06ca40 con 0x7f93d81077a0 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.736+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d8107c20 con 0x7f93d81077a0 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.736+0000 7f93d7fff640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 0x7f93c803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.737+0000 7f93d7fff640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 0x7f93c803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93c40098c0 tx=0x7f93c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.741+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93cc044c70 con 0x7f93d81077a0 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:07.929+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f93d807d2c0 con 0x7f93c8039260 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.151+0000 7f93d5ffb640 1 -- 172.21.15.77:0/3193450762 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f93d807d2c0 con 0x7f93c8039260 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 msgr2=0x7f93c803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 0x7f93c803b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93c40098c0 tx=0x7f93c4006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 msgr2=0x7f93d807e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93cc030560 tx=0x7f93cc032670 comp rx=0 tx=0).stop 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 shutdown_connections 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f93c8039260 0x7f93c803b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 --2- 172.21.15.77:0/3193450762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d81077a0 0x7f93d807e3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 >> 172.21.15.77:0/3193450762 conn(0x7f93d8102fd0 msgr2=0x7f93d8103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 shutdown_connections 2024-09-15T22:03:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:08.153+0000 7f93df089640 1 -- 172.21.15.77:0/3193450762 wait complete. 2024-09-15T22:03:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:Fetching dashboard port number... 2024-09-15T22:03:09.425 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:09 smithi077 ceph-mon[30980]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:09.425 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:09 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:10.264 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:10 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/855942555' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-15T22:03:10.271 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 8443 2024-09-15T22:03:10.271 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01ed308640 1 Processor -- start 2024-09-15T22:03:10.271 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01ed308640 1 -- start start 2024-09-15T22:03:10.271 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01ed308640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:10.271 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01ed308640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01e807c660 con 0x7f01e81081e0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59990/0 (socket says 172.21.15.77:59990) 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01e6ffd640 1 -- 172.21.15.77:0/3600288171 learned_addr learned my addr 172.21.15.77:0/3600288171 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.428+0000 7f01e6ffd640 1 -- 172.21.15.77:0/3600288171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e810ab10 con 0x7f01e81081e0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.429+0000 7f01e6ffd640 1 --2- 172.21.15.77:0/3600288171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009820 tx=0x7f01d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdda317a2888754 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.429+0000 7f01e5ffb640 1 -- 172.21.15.77:0/3600288171 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01d8037070 con 0x7f01e81081e0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.429+0000 7f01e5ffb640 1 -- 172.21.15.77:0/3600288171 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d8032920 con 0x7f01e81081e0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.429+0000 7f01e5ffb640 1 -- 172.21.15.77:0/3600288171 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01d8030810 con 0x7f01e81081e0 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 -- 172.21.15.77:0/3600288171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 msgr2=0x7f01e810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 --2- 172.21.15.77:0/3600288171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009820 tx=0x7f01d802a270 comp rx=0 tx=0).stop 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 -- 172.21.15.77:0/3600288171 shutdown_connections 2024-09-15T22:03:10.272 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 --2- 172.21.15.77:0/3600288171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e810a5d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 -- 172.21.15.77:0/3600288171 >> 172.21.15.77:0/3600288171 conn(0x7f01e8101de0 msgr2=0x7f01e8104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 -- 172.21.15.77:0/3600288171 shutdown_connections 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 -- 172.21.15.77:0/3600288171 wait complete. 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.430+0000 7f01ed308640 1 Processor -- start 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01ed308640 1 -- start start 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01ed308640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01ed308640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810bb90 con 0x7f01e81081e0 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59996/0 (socket says 172.21.15.77:59996) 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01e6ffd640 1 -- 172.21.15.77:0/855942555 learned_addr learned my addr 172.21.15.77:0/855942555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01e6ffd640 1 -- 172.21.15.77:0/855942555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e819bc90 con 0x7f01e81081e0 2024-09-15T22:03:10.273 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01e6ffd640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01d8030380 tx=0x7f01d8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01d8037040 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d8040070 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e819bf20 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01d803a7b0 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.431+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e819ec10 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.432+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f01d803a9e0 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.432+0000 7f01cffff640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 0x7f01d003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.432+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e8108640 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.433+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01d806c9d0 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.433+0000 7f01e67fc640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 0x7f01d003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.433+0000 7f01e67fc640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 0x7f01d003b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0098c0 tx=0x7f01dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.437+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01d8044c70 con 0x7f01e81081e0 2024-09-15T22:03:10.274 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.605+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f01e819f200 con 0x7f01e81081e0 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.605+0000 7f01cffff640 1 -- 172.21.15.77:0/855942555 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f01d8041270 con 0x7f01e81081e0 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 msgr2=0x7f01d003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 0x7f01d003b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0098c0 tx=0x7f01dc006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 msgr2=0x7f01e819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01d8030380 tx=0x7f01d8032670 comp rx=0 tx=0).stop 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 shutdown_connections 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01d0039260 0x7f01d003b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 --2- 172.21.15.77:0/855942555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01e81081e0 0x7f01e819b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 >> 172.21.15.77:0/855942555 conn(0x7f01e8101de0 msgr2=0x7f01e8102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 shutdown_connections 2024-09-15T22:03:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:09.607+0000 7f01ed308640 1 -- 172.21.15.77:0/855942555 wait complete. 2024-09-15T22:03:10.278 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:03:10.278 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-09-15T22:03:10.285 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:03:10.285 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-09-15T22:03:10.285 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-09-15T22:03:10.285 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph Dashboard is now available at: 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout: URL: https://smithi077.front.sepia.ceph.com:8443/ 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout: User: admin 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout: Password: 81t75zpgs2 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:10.287 INFO:teuthology.orchestra.run.smithi077.stdout:Saving cluster configuration to /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config directory 2024-09-15T22:03:11.244 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:11 smithi077 ceph-mon[30980]: mgrmap e13: x(active, since 6s) 2024-09-15T22:03:11.244 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:11 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3237359768' entity='client.admin' 2024-09-15T22:03:11.544 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5f207640 1 Processor -- start 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5f207640 1 -- start start 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5f207640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5f207640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b58108170 con 0x7f1b581077a0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5cfa3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5cfa3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60004/0 (socket says 172.21.15.77:60004) 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5cfa3640 1 -- 172.21.15.77:0/642471372 learned_addr learned my addr 172.21.15.77:0/642471372 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.685+0000 7f1b5cfa3640 1 -- 172.21.15.77:0/642471372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b58108950 con 0x7f1b581077a0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.686+0000 7f1b5cfa3640 1 --2- 172.21.15.77:0/642471372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009820 tx=0x7f1b5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549528d147025e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.686+0000 7f1b577fe640 1 -- 172.21.15.77:0/642471372 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50037070 con 0x7f1b581077a0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.686+0000 7f1b577fe640 1 -- 172.21.15.77:0/642471372 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b50032920 con 0x7f1b581077a0 2024-09-15T22:03:11.545 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.686+0000 7f1b577fe640 1 -- 172.21.15.77:0/642471372 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50030810 con 0x7f1b581077a0 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- 172.21.15.77:0/642471372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 msgr2=0x7f1b58107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 --2- 172.21.15.77:0/642471372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009820 tx=0x7f1b5002a270 comp rx=0 tx=0).stop 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- 172.21.15.77:0/642471372 shutdown_connections 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 --2- 172.21.15.77:0/642471372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b58107ba0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- 172.21.15.77:0/642471372 >> 172.21.15.77:0/642471372 conn(0x7f1b58102fd0 msgr2=0x7f1b581053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- 172.21.15.77:0/642471372 shutdown_connections 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- 172.21.15.77:0/642471372 wait complete. 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 Processor -- start 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- start start 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.687+0000 7f1b5f207640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5810bd00 con 0x7f1b581077a0 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5cfa3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5cfa3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60014/0 (socket says 172.21.15.77:60014) 2024-09-15T22:03:11.546 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5cfa3640 1 -- 172.21.15.77:0/3237359768 learned_addr learned my addr 172.21.15.77:0/3237359768 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5cfa3640 1 -- 172.21.15.77:0/3237359768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5819bcb0 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5cfa3640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097f0 tx=0x7f1b500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50037040 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b50030cc0 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5003a640 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5819bf40 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.688+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5819ec30 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.689+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1b5003b3f0 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.689+0000 7f1b55ffb640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 0x7f1b4403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.689+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1b5006c9c0 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.689+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5810be80 con 0x7f1b581077a0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.689+0000 7f1b57fff640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 0x7f1b4403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:11.547 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.690+0000 7f1b57fff640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 0x7f1b4403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800ad30 tx=0x7f1b480093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.694+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b500313a0 con 0x7f1b581077a0 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.944+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1b58107c20 con 0x7f1b581077a0 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.947+0000 7f1b55ffb640 1 -- 172.21.15.77:0/3237359768 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1b50038250 con 0x7f1b581077a0 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 msgr2=0x7f1b4403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 0x7f1b4403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800ad30 tx=0x7f1b480093f0 comp rx=0 tx=0).stop 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 msgr2=0x7f1b5819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097f0 tx=0x7f1b500327b0 comp rx=0 tx=0).stop 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 shutdown_connections 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1b44039300 0x7f1b4403b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:11.548 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 --2- 172.21.15.77:0/3237359768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b581077a0 0x7f1b5819b770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 >> 172.21.15.77:0/3237359768 conn(0x7f1b58102fd0 msgr2=0x7f1b58103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 shutdown_connections 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-09-15T22:03:10.949+0000 7f1b5f207640 1 -- 172.21.15.77:0/3237359768 wait complete. 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:Or, if you are only running a single cluster on this host: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-15T22:03:11.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: ceph telemetry on 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout:For more information see: 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:11.550 INFO:teuthology.orchestra.run.smithi077.stdout:Bootstrap complete. 2024-09-15T22:03:11.581 INFO:tasks.cephadm:Fetching config... 2024-09-15T22:03:11.582 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:11.582 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-15T22:03:11.598 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-15T22:03:11.598 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:11.598 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-15T22:03:11.654 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-15T22:03:11.654 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:11.654 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-15T22:03:11.719 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-15T22:03:11.719 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:11.719 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-15T22:03:11.775 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-15T22:03:11.775 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:03:11.849 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:11.862 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:03:11.903 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:11.915 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:03:11.957 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLsOibiAY3agdDs461Su0tVSFZH0EJicXxzBzyUNnuMTOzUuzF3NQWBJo5cr5YfozGsXqbO2OQByRNO2MsAFctqCTCIiDzdLS0NBtd7c0NqwSz7s+hkQ27kxYUcHUQcnX10UrYy0g0rqhNqmpPkLbaS8zoIKmaiBuTkXoNMnhwFKi3XLN9KjmuZdo5iAmxpzmzWo2jsHm0hcMBy814iMJqNuZyo/tZaE63IwIsXP0hW+9Ad238ors11O36YwhpVFD5roBZE5dQSzd3uAB9660mwb5Vl03zF9l4nTkS6IhN1dzwPFbd5+CXbVXioy9gCFzic2UC4r+23LdvkFQOUPtDAoPxGwvMCyoT/9TVLVxqlEWsACcUSSwJ+Kd1lVnwFK8lzc2hg8JLFziCi1AB2DMTw03jgBFWjMeDL6+Mj6g35fJs33PN/bTd7/9ZRahO4DrOm47LCsGGHAj/LgxMrE+VhCoFRlLadOEL5ZQ98+lfIZyjH7fKMnhdU9SZ2ArWmIk= ceph-0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:11.969 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-15T22:03:12.215 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:12.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 -- 172.21.15.77:0/4286397774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 msgr2=0x7f38f0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4286397774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009880 tx=0x7f38e002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 -- 172.21.15.77:0/4286397774 shutdown_connections 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4286397774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 -- 172.21.15.77:0/4286397774 >> 172.21.15.77:0/4286397774 conn(0x7f38f00fcdb0 msgr2=0x7f38f00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 -- 172.21.15.77:0/4286397774 shutdown_connections 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.712+0000 7f38f59a0640 1 -- 172.21.15.77:0/4286397774 wait complete. 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38f59a0640 1 Processor -- start 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38f59a0640 1 -- start start 2024-09-15T22:03:12.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38f59a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38f59a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38f0107710 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60030/0 (socket says 172.21.15.77:60030) 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38eeffd640 1 -- 172.21.15.77:0/4272758159 learned_addr learned my addr 172.21.15.77:0/4272758159 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.713+0000 7f38eeffd640 1 -- 172.21.15.77:0/4272758159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f01972f0 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38eeffd640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38e0004f60 tx=0x7f38e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e003f070 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e0004cc0 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f0197580 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e0035030 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.714+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f019a230 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.715+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f38e0042070 con 0x7f38f0101600 2024-09-15T22:03:12.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.715+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f0107e90 con 0x7f38f0101600 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.715+0000 7f38f499e640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 0x7f38dc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.715+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f38e006d070 con 0x7f38f0101600 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.715+0000 7f38e7fff640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 0x7f38dc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.716+0000 7f38e7fff640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 0x7f38dc03b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098c0 tx=0x7f38d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.720+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38e0036da0 con 0x7f38f0101600 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.893+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f38f002d8a0 con 0x7f38f0101600 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.896+0000 7f38f499e640 1 -- 172.21.15.77:0/4272758159 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f38e0030d60 con 0x7f38f0101600 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 msgr2=0x7f38dc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 0x7f38dc03b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098c0 tx=0x7f38d8006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 msgr2=0x7f38f0196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f38e0004f60 tx=0x7f38e00048f0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 shutdown_connections 2024-09-15T22:03:12.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f38dc0392a0 0x7f38dc03b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 --2- 172.21.15.77:0/4272758159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38f0101600 0x7f38f0196db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 >> 172.21.15.77:0/4272758159 conn(0x7f38f00fcdb0 msgr2=0x7f38f00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:12.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 shutdown_connections 2024-09-15T22:03:12.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:12.898+0000 7f38f59a0640 1 -- 172.21.15.77:0/4272758159 wait complete. 2024-09-15T22:03:13.561 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-15T22:03:13.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-15T22:03:14.000 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:14.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:13 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4272758159' entity='client.admin' 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- 172.21.15.77:0/2790724986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 msgr2=0x7f5d90104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/2790724986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90104520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009c20 tx=0x7f5d8002aa90 comp rx=0 tx=0).stop 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- 172.21.15.77:0/2790724986 shutdown_connections 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/2790724986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90104520 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- 172.21.15.77:0/2790724986 >> 172.21.15.77:0/2790724986 conn(0x7f5d900fbcd0 msgr2=0x7f5d900fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- 172.21.15.77:0/2790724986 shutdown_connections 2024-09-15T22:03:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- 172.21.15.77:0/2790724986 wait complete. 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 Processor -- start 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- start start 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.448+0000 7f5d97f6e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901076e0 con 0x7f5d90102130 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d95d0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d95d0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60058/0 (socket says 172.21.15.77:60058) 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d95d0a640 1 -- 172.21.15.77:0/3539496234 learned_addr learned my addr 172.21.15.77:0/3539496234 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d95d0a640 1 -- 172.21.15.77:0/3539496234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d901972e0 con 0x7f5d90102130 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d95d0a640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006010 tx=0x7f5d8002e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d80047070 con 0x7f5d90102130 2024-09-15T22:03:14.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d80003e50 con 0x7f5d90102130 2024-09-15T22:03:14.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d90197570 con 0x7f5d90102130 2024-09-15T22:03:14.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d80033050 con 0x7f5d90102130 2024-09-15T22:03:14.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.449+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9019a220 con 0x7f5d90102130 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.450+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5d8004a070 con 0x7f5d90102130 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.450+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d90107e60 con 0x7f5d90102130 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.450+0000 7f5d8effd640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 0x7f5d7803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.450+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d8003e030 con 0x7f5d90102130 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.450+0000 7f5d95509640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 0x7f5d7803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:14.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.451+0000 7f5d95509640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 0x7f5d7803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d840098c0 tx=0x7f5d84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:14.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.455+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d800040f0 con 0x7f5d90102130 2024-09-15T22:03:14.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.627+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5d9002d940 con 0x7f5d78039300 2024-09-15T22:03:14.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.631+0000 7f5d8effd640 1 -- 172.21.15.77:0/3539496234 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d9002d940 con 0x7f5d78039300 2024-09-15T22:03:14.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.632+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 msgr2=0x7f5d7803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:14.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 0x7f5d7803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d840098c0 tx=0x7f5d84006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 msgr2=0x7f5d90196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006010 tx=0x7f5d8002e3c0 comp rx=0 tx=0).stop 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 shutdown_connections 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5d78039300 0x7f5d7803b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 --2- 172.21.15.77:0/3539496234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d90102130 0x7f5d90196da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 >> 172.21.15.77:0/3539496234 conn(0x7f5d900fbcd0 msgr2=0x7f5d900fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 shutdown_connections 2024-09-15T22:03:14.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:14.633+0000 7f5d97f6e640 1 -- 172.21.15.77:0/3539496234 wait complete. 2024-09-15T22:03:15.541 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-09-15T22:03:15.542 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:03:15.542 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:03:15.562 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:03:15.562 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:15.620 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-09-15T22:03:15.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch host add smithi117 2024-09-15T22:03:15.791 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:15 smithi077 ceph-mon[30980]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:15.791 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:15 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:16.196 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 -- 172.21.15.77:0/1283931706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 msgr2=0x7f59cc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 --2- 172.21.15.77:0/1283931706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc1019e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009880 tx=0x7f59c802a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 -- 172.21.15.77:0/1283931706 shutdown_connections 2024-09-15T22:03:16.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 --2- 172.21.15.77:0/1283931706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc1019e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 -- 172.21.15.77:0/1283931706 >> 172.21.15.77:0/1283931706 conn(0x7f59cc0fcd70 msgr2=0x7f59cc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 -- 172.21.15.77:0/1283931706 shutdown_connections 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.728+0000 7f59d3772640 1 -- 172.21.15.77:0/1283931706 wait complete. 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d3772640 1 Processor -- start 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d3772640 1 -- start start 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d3772640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d3772640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1076f0 con 0x7f59cc1015e0 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d150e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d150e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60082/0 (socket says 172.21.15.77:60082) 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.729+0000 7f59d150e640 1 -- 172.21.15.77:0/1835919731 learned_addr learned my addr 172.21.15.77:0/1835919731 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59d150e640 1 -- 172.21.15.77:0/1835919731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc0763f0 con 0x7f59cc1015e0 2024-09-15T22:03:16.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59d150e640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59c8004f60 tx=0x7f59c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c803f070 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc0745e0 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc074b20 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59c8004cc0 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.730+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c8035030 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f59c8042070 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59c27fc640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 0x7f59b803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59c806d010 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59d0d0d640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 0x7f59b803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b00052c0 con 0x7f59cc1015e0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.731+0000 7f59d0d0d640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 0x7f59b803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098c0 tx=0x7f59bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:16.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.736+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59c80433b0 con 0x7f59cc1015e0 2024-09-15T22:03:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:16.917+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) -- 0x7f59b0000bf0 con 0x7f59b8039300 2024-09-15T22:03:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:17.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:18 smithi077 ceph-mon[30980]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:18 smithi077 ceph-mon[30980]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:18 smithi077 ceph-mon[30980]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:03:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:18 smithi077 ceph-mon[30980]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:19 smithi077 ceph-mon[30980]: Deploying cephadm binary to smithi117 2024-09-15T22:03:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.140+0000 7f59c27fc640 1 -- 172.21.15.77:0/1835919731 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f59b0000bf0 con 0x7f59b8039300 2024-09-15T22:03:20.142 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 msgr2=0x7f59b803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 0x7f59b803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098c0 tx=0x7f59bc006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 msgr2=0x7f59cc075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59c8004f60 tx=0x7f59c80048f0 comp rx=0 tx=0).stop 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 shutdown_connections 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59b8039300 0x7f59b803b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 --2- 172.21.15.77:0/1835919731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1015e0 0x7f59cc075eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.142+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 >> 172.21.15.77:0/1835919731 conn(0x7f59cc0fcd70 msgr2=0x7f59cc0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.143+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 shutdown_connections 2024-09-15T22:03:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:20.143+0000 7f59d3772640 1 -- 172.21.15.77:0/1835919731 wait complete. 2024-09-15T22:03:20.747 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:03:20.982 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:21.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:21.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:21 smithi077 ceph-mon[30980]: Added host smithi117 2024-09-15T22:03:21.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:21.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:21.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:21.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 -- 172.21.15.77:0/3550812223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 msgr2=0x7f6164106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:21.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 --2- 172.21.15.77:0/3550812223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6154009880 tx=0x7f615402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:21.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 -- 172.21.15.77:0/3550812223 shutdown_connections 2024-09-15T22:03:21.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 --2- 172.21.15.77:0/3550812223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 -- 172.21.15.77:0/3550812223 >> 172.21.15.77:0/3550812223 conn(0x7f61640fc1b0 msgr2=0x7f61640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 -- 172.21.15.77:0/3550812223 shutdown_connections 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.425+0000 7f616a951640 1 -- 172.21.15.77:0/3550812223 wait complete. 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f616a951640 1 Processor -- start 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f616a951640 1 -- start start 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f616a951640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f616a951640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f616410ab90 con 0x7f6164106560 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f6163fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f6163fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40046/0 (socket says 172.21.15.77:40046) 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.426+0000 7f6163fff640 1 -- 172.21.15.77:0/553887473 learned_addr learned my addr 172.21.15.77:0/553887473 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f6163fff640 1 -- 172.21.15.77:0/553887473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6164197640 con 0x7f6164106560 2024-09-15T22:03:21.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f6163fff640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6154009230 tx=0x7f6154003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f615403f070 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f615403bc80 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641978d0 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6154035030 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.427+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616419b9d0 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.428+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6154042070 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.428+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6164101bd0 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.428+0000 7f61617fa640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 0x7f614c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.428+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f615406cfb0 con 0x7f6164106560 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.428+0000 7f61637fe640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 0x7f614c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:21.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.429+0000 7f61637fe640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 0x7f614c03b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61580098c0 tx=0x7f6158006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:21.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.433+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f615404c350 con 0x7f6164106560 2024-09-15T22:03:21.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.605+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6164198460 con 0x7f614c0392a0 2024-09-15T22:03:21.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.606+0000 7f61617fa640 1 -- 172.21.15.77:0/553887473 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6164198460 con 0x7f614c0392a0 2024-09-15T22:03:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:21.609 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-09-15T22:03:21.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 msgr2=0x7f614c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:21.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 0x7f614c03b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61580098c0 tx=0x7f6158006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:21.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 msgr2=0x7f6164197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6154009230 tx=0x7f6154003d40 comp rx=0 tx=0).stop 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 shutdown_connections 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f614c0392a0 0x7f614c03b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 --2- 172.21.15.77:0/553887473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164106560 0x7f6164197100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.608+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 >> 172.21.15.77:0/553887473 conn(0x7f61640fc1b0 msgr2=0x7f6164069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.609+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 shutdown_connections 2024-09-15T22:03:21.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:21.609+0000 7f616a951640 1 -- 172.21.15.77:0/553887473 wait complete. 2024-09-15T22:03:22.230 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-15T22:03:22.230 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:03:22.230 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:03:22.250 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:03:22.251 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:22.309 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-15T22:03:22.309 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch host add smithi196 2024-09-15T22:03:22.545 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:22.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:22.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:22 smithi077 ceph-mon[30980]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:03:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- 172.21.15.77:0/4034822720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 msgr2=0x7f3eb8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/4034822720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb8106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009810 tx=0x7f3ea802a540 comp rx=0 tx=0).stop 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- 172.21.15.77:0/4034822720 shutdown_connections 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/4034822720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb8106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- 172.21.15.77:0/4034822720 >> 172.21.15.77:0/4034822720 conn(0x7f3eb8077310 msgr2=0x7f3eb80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- 172.21.15.77:0/4034822720 shutdown_connections 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- 172.21.15.77:0/4034822720 wait complete. 2024-09-15T22:03:22.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 Processor -- start 2024-09-15T22:03:22.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.974+0000 7f3ebd52a640 1 -- start start 2024-09-15T22:03:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3ebd52a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3ebd52a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb810ab70 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40054/0 (socket says 172.21.15.77:40054) 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb6ffd640 1 -- 172.21.15.77:0/3412453594 learned_addr learned my addr 172.21.15.77:0/3412453594 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb6ffd640 1 -- 172.21.15.77:0/3412453594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80ff290 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb6ffd640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80045e0 tx=0x7f3ea8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea8030e70 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea80318e0 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb80fd8e0 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea80049c0 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.975+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb80fdda0 con 0x7f3eb8106540 2024-09-15T22:03:22.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.976+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3ea8039040 con 0x7f3eb8106540 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.976+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8101bb0 con 0x7f3eb8106540 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.976+0000 7f3eb4ff9640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 0x7f3ea403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.976+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ea806d740 con 0x7f3eb8106540 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.977+0000 7f3eaffff640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 0x7f3ea403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.977+0000 7f3eaffff640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 0x7f3ea403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00098c0 tx=0x7f3ea0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:22.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:22.981+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ea8040030 con 0x7f3eb8106540 2024-09-15T22:03:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:23.151+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f3eb80fe940 con 0x7f3ea4039300 2024-09-15T22:03:23.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:23 smithi077 ceph-mon[30980]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: Deploying cephadm binary to smithi196 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:25.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:25.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:25.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:25.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:25 smithi077 ceph-mon[30980]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:03:26.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.283+0000 7f3eb4ff9640 1 -- 172.21.15.77:0/3412453594 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3eb80fe940 con 0x7f3ea4039300 2024-09-15T22:03:26.286 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 msgr2=0x7f3ea403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 0x7f3ea403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00098c0 tx=0x7f3ea0006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 msgr2=0x7f3eb80fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80045e0 tx=0x7f3ea8004610 comp rx=0 tx=0).stop 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 shutdown_connections 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3ea4039300 0x7f3ea403b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 --2- 172.21.15.77:0/3412453594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8106540 0x7f3eb80fd3a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 >> 172.21.15.77:0/3412453594 conn(0x7f3eb8077310 msgr2=0x7f3eb80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 shutdown_connections 2024-09-15T22:03:26.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:26.285+0000 7f3ebd52a640 1 -- 172.21.15.77:0/3412453594 wait complete. 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:26.905 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:03:27.140 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: Added host smithi196 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:27.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:27.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 -- 172.21.15.77:0/5538249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 msgr2=0x7fc390107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:27.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 --2- 172.21.15.77:0/5538249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc390107a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc380009880 tx=0x7fc38002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 -- 172.21.15.77:0/5538249 shutdown_connections 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 --2- 172.21.15.77:0/5538249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc390107a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 -- 172.21.15.77:0/5538249 >> 172.21.15.77:0/5538249 conn(0x7fc3900fd260 msgr2=0x7fc3900ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 -- 172.21.15.77:0/5538249 shutdown_connections 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.573+0000 7fc39805e640 1 -- 172.21.15.77:0/5538249 wait complete. 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc39805e640 1 Processor -- start 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc39805e640 1 -- start start 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc39805e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc39805e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010bca0 con 0x7fc390107670 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc395dfa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:27.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc395dfa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40062/0 (socket says 172.21.15.77:40062) 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc395dfa640 1 -- 172.21.15.77:0/1330566745 learned_addr learned my addr 172.21.15.77:0/1330566745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc395dfa640 1 -- 172.21.15.77:0/1330566745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900763e0 con 0x7fc390107670 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.574+0000 7fc395dfa640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc380009230 tx=0x7fc380003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc38003f070 con 0x7fc390107670 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc38003bc80 con 0x7fc390107670 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc390074a30 con 0x7fc390107670 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc380035030 con 0x7fc390107670 2024-09-15T22:03:27.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc390074ef0 con 0x7fc390107670 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.575+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc380042070 con 0x7fc390107670 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.576+0000 7fc386ffd640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 0x7fc37c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.576+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc38006d040 con 0x7fc390107670 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.576+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390102ce0 con 0x7fc390107670 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.576+0000 7fc3955f9640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 0x7fc37c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:27.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.576+0000 7fc3955f9640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 0x7fc37c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0098c0 tx=0x7fc38c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:27.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.581+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3800314c0 con 0x7fc390107670 2024-09-15T22:03:27.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.764+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc390075a20 con 0x7fc37c039300 2024-09-15T22:03:27.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.766+0000 7fc386ffd640 1 -- 172.21.15.77:0/1330566745 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fc390075a20 con 0x7fc37c039300 2024-09-15T22:03:27.768 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:27.768 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-15T22:03:27.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.767+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 msgr2=0x7fc37c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:27.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.767+0000 7fc39805e640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 0x7fc37c03b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0098c0 tx=0x7fc38c006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:27.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.767+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 msgr2=0x7fc3900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:27.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.767+0000 7fc39805e640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc380009230 tx=0x7fc380003d40 comp rx=0 tx=0).stop 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 shutdown_connections 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc37c039300 0x7fc37c03b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 --2- 172.21.15.77:0/1330566745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc390107670 0x7fc3900744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 >> 172.21.15.77:0/1330566745 conn(0x7fc3900fd260 msgr2=0x7fc3900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 shutdown_connections 2024-09-15T22:03:27.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:27.768+0000 7fc39805e640 1 -- 172.21.15.77:0/1330566745 wait complete. 2024-09-15T22:03:28.335 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-15T22:03:28.335 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-15T22:03:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:28.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:28 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:28.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:28 smithi077 ceph-mon[30980]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:03:28.944 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:28 smithi077 ceph-mon[30980]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- 172.21.15.77:0/356691859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 msgr2=0x7f3264106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 --2- 172.21.15.77:0/356691859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3258009880 tx=0x7f325802a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- 172.21.15.77:0/356691859 shutdown_connections 2024-09-15T22:03:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 --2- 172.21.15.77:0/356691859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- 172.21.15.77:0/356691859 >> 172.21.15.77:0/356691859 conn(0x7f32640fc1b0 msgr2=0x7f32640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- 172.21.15.77:0/356691859 shutdown_connections 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- 172.21.15.77:0/356691859 wait complete. 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 Processor -- start 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.022+0000 7f326af06640 1 -- start start 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f326af06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f326af06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f326410ab90 con 0x7f3264106560 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3268ca2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3268ca2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48958/0 (socket says 172.21.15.77:48958) 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3268ca2640 1 -- 172.21.15.77:0/1705260401 learned_addr learned my addr 172.21.15.77:0/1705260401 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3268ca2640 1 -- 172.21.15.77:0/1705260401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32641976d0 con 0x7f3264106560 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3268ca2640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3258009230 tx=0x7f3258003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f325803f070 con 0x7f3264106560 2024-09-15T22:03:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325803bd40 con 0x7f3264106560 2024-09-15T22:03:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3264197960 con 0x7f3264106560 2024-09-15T22:03:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.023+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3258035030 con 0x7f3264106560 2024-09-15T22:03:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.024+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3264197dc0 con 0x7f3264106560 2024-09-15T22:03:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.025+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3264101bd0 con 0x7f3264106560 2024-09-15T22:03:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.025+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3258042070 con 0x7f3264106560 2024-09-15T22:03:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.025+0000 7f3261ffb640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 0x7f325403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.025+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f325806d080 con 0x7f3264106560 2024-09-15T22:03:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.025+0000 7f3263fff640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 0x7f325403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.026+0000 7f3263fff640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 0x7f325403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32500098c0 tx=0x7f3250006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.030+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f325803a8e0 con 0x7f3264106560 2024-09-15T22:03:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.198+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3264198550 con 0x7f3264106560 2024-09-15T22:03:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.631+0000 7f3261ffb640 1 -- 172.21.15.77:0/1705260401 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f325803a3c0 con 0x7f3264106560 2024-09-15T22:03:29.634 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 msgr2=0x7f325403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 0x7f325403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32500098c0 tx=0x7f3250006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 msgr2=0x7f3264197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3258009230 tx=0x7f3258003d80 comp rx=0 tx=0).stop 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 shutdown_connections 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f3254039300 0x7f325403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 --2- 172.21.15.77:0/1705260401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3264106560 0x7f3264197190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 >> 172.21.15.77:0/1705260401 conn(0x7f32640fc1b0 msgr2=0x7f3264069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 shutdown_connections 2024-09-15T22:03:29.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:29.633+0000 7f326af06640 1 -- 172.21.15.77:0/1705260401 wait complete. 2024-09-15T22:03:29.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1705260401' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:03:30.260 INFO:tasks.cephadm:Adding mon.a on smithi077 2024-09-15T22:03:30.260 INFO:tasks.cephadm:Adding mon.b on smithi117 2024-09-15T22:03:30.261 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-09-15T22:03:30.261 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c' 2024-09-15T22:03:30.519 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T22:03:30.565 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T22:03:30.888 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1705260401' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:03:30.888 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:30.888 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:30.889 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:30 smithi077 ceph-mon[30980]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:03:31.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 -- 172.21.15.196:0/1884849602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 msgr2=0x7fd214101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:31.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 --2- 172.21.15.196:0/1884849602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214101a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009880 tx=0x7fd20c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 -- 172.21.15.196:0/1884849602 shutdown_connections 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 --2- 172.21.15.196:0/1884849602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214101a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 -- 172.21.15.196:0/1884849602 >> 172.21.15.196:0/1884849602 conn(0x7fd2140fcdb0 msgr2=0x7fd2140ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 -- 172.21.15.196:0/1884849602 shutdown_connections 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.050+0000 7fd2190a7640 1 -- 172.21.15.196:0/1884849602 wait complete. 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd2190a7640 1 Processor -- start 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd2190a7640 1 -- start start 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd2190a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd2190a7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd214107730 con 0x7fd214101620 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd212d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd212d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.196:40820/0 (socket says 172.21.15.196:40820) 2024-09-15T22:03:31.086 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.051+0000 7fd212d9d640 1 -- 172.21.15.196:0/541889424 learned_addr learned my addr 172.21.15.196:0/541889424 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd212d9d640 1 -- 172.21.15.196:0/541889424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2141972e0 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd212d9d640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004f60 tx=0x7fd20c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd20c03f070 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c004d40 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd214197570 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd20c035030 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.052+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21419a220 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.053+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd20c042070 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.053+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd214107eb0 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.053+0000 7fd1fbfff640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 0x7fd1fc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.053+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd20c06d0a0 con 0x7fd214101620 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.054+0000 7fd21259c640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 0x7fd1fc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:31.087 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.054+0000 7fd21259c640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 0x7fd1fc03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ad30 tx=0x7fd2080093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:31.088 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.059+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd20c039350 con 0x7fd214101620 2024-09-15T22:03:31.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.241+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fd21402d960 con 0x7fd1fc0392a0 2024-09-15T22:03:31.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.246+0000 7fd1fbfff640 1 -- 172.21.15.196:0/541889424 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd21402d960 con 0x7fd1fc0392a0 2024-09-15T22:03:31.249 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 msgr2=0x7fd1fc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 0x7fd1fc03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ad30 tx=0x7fd2080093f0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 msgr2=0x7fd214196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004f60 tx=0x7fd20c0048f0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 shutdown_connections 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1fc0392a0 0x7fd1fc03b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 --2- 172.21.15.196:0/541889424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd214101620 0x7fd214196da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 >> 172.21.15.196:0/541889424 conn(0x7fd2140fcdb0 msgr2=0x7fd2140fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 shutdown_connections 2024-09-15T22:03:31.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:31.248+0000 7fd2190a7640 1 -- 172.21.15.196:0/541889424 wait complete. 2024-09-15T22:03:31.822 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.b.service 2024-09-15T22:03:31.825 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.c.service 2024-09-15T22:03:31.828 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:03:31.828 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c;count:3 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:32.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:32 smithi077 ceph-mon[30980]: Deploying daemon mon.c on smithi196 2024-09-15T22:03:32.419 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:33.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:33 smithi077 ceph-mon[30980]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:33.776 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 -- 172.21.15.196:0/621315922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 msgr2=0x7f80a8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 --2- 172.21.15.196:0/621315922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a8102b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8098009c20 tx=0x7f809802ab20 comp rx=0 tx=0).stop 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 -- 172.21.15.196:0/621315922 shutdown_connections 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 --2- 172.21.15.196:0/621315922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a8102b30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 -- 172.21.15.196:0/621315922 >> 172.21.15.196:0/621315922 conn(0x7f80a80fc1b0 msgr2=0x7f80a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 -- 172.21.15.196:0/621315922 shutdown_connections 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 -- 172.21.15.196:0/621315922 wait complete. 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.770+0000 7f80ade19640 1 Processor -- start 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80ade19640 1 -- start start 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80ade19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80ade19640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80a810bcc0 con 0x7f80a8102750 2024-09-15T22:03:33.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.196:40840/0 (socket says 172.21.15.196:40840) 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.771+0000 7f80a77fe640 1 -- 172.21.15.196:0/1347043988 learned_addr learned my addr 172.21.15.196:0/1347043988 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80a77fe640 1 -- 172.21.15.196:0/1347043988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a8197710 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80a77fe640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8098030040 tx=0x7f809802eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8098047070 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a81979a0 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80980368e0 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8098042420 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.772+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a8197e00 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.773+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f809804a070 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.773+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a806a7b0 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.773+0000 7f80a4ff9640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 0x7f809003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.773+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f809803d030 con 0x7f80a8102750 2024-09-15T22:03:33.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.774+0000 7f80a6ffd640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 0x7f809003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:33.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.774+0000 7f80a6ffd640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 0x7f809003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f809c0098c0 tx=0x7f809c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:33.781 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:33.779+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8098032030 con 0x7f80a8102750 2024-09-15T22:03:34.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.011+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f80a8198590 con 0x7f80a8102750 2024-09-15T22:03:34.014 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.012+0000 7f80a4ff9640 1 -- 172.21.15.196:0/1347043988 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f8098051090 con 0x7f80a8102750 2024-09-15T22:03:34.015 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:03:34.015 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","modified":"2024-09-15T22:02:05.827849Z","created":"2024-09-15T22:02:05.827849Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T22:03:34.015 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-15T22:03:34.016 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 msgr2=0x7f809003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 0x7f809003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f809c0098c0 tx=0x7f809c006db0 comp rx=0 tx=0).stop 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 msgr2=0x7f80a81971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8098030040 tx=0x7f809802eb50 comp rx=0 tx=0).stop 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 shutdown_connections 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80900392a0 0x7f809003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 --2- 172.21.15.196:0/1347043988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80a8102750 0x7f80a81971d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 >> 172.21.15.196:0/1347043988 conn(0x7f80a80fc1b0 msgr2=0x7f80a8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 shutdown_connections 2024-09-15T22:03:34.017 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:34.014+0000 7f80ade19640 1 -- 172.21.15.196:0/1347043988 wait complete. 2024-09-15T22:03:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/1347043988' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:35 smithi077 ceph-mon[30980]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:35.772 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:03:35.772 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:03:35.922 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 podman[35071]: 2024-09-15 22:03:35.898346005 +0000 UTC m=+0.346730296 container start 2b2a1bd3652be1b233e7cc9547b09a6d3609560e6829506046c79248f30d38ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T22:03:36.050 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:03:36.257 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:03:36.257 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:03:36.257 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: pidfile_write: ignore empty --pid-file 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: load: jerasure load: lrc 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Git sha 0 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: DB SUMMARY 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: DB Session ID: OHD5F3JKOHBF246JCO9Q 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:03:36.258 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.env: 0x55e989aad940 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.info_log: 0x55e98aacf6e0 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.statistics: (nil) 2024-09-15T22:03:36.259 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.use_fsync: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:03:36.260 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.db_log_dir: 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.wal_dir: 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:03:36.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.write_buffer_manager: 0x55e98aadedc0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.unordered_write: 0 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:03:36.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.row_cache: None 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.wal_filter: None 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.wal_compression: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:03:36.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:03:36.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:03:36.265 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:03:36.266 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_open_files: -1 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Compression algorithms supported: 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kZSTD supported: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:03:36.267 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.merge_operator: 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_filter: None 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:03:36.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e98aacf320) 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-09-15T22:03:36.269 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x55e98aac31f0 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:03:36.270 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-09-15T22:03:36.271 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression: NoCompression 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.num_levels: 7 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:03:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:03:36.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:03:36.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:03:36.275 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:03:36.276 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:03:36.277 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.ttl: 2592000 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:03:36.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 853851d8-6eb0-4826-821d-1b601438c1ca 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437815926959, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437815927410, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726437815, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "853851d8-6eb0-4826-821d-1b601438c1ca", "db_session_id": "OHD5F3JKOHBF246JCO9Q", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437815927505, "job": 1, "event": "recovery_finished"} 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e98aaf0e00 2024-09-15T22:03:36.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: DB pointer 0x55e98abec000 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-15T22:03:36.280 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-09-15T22:03:36.281 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.282 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x55e98aac31f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:03:36.283 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(???) e0 preinit fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).mds e1 new map 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: btime 2024-09-15T22:02:08:800551+0000 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:03:36.284 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:03:36.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1705260401' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1705260401' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.286 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:36.287 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c;count:3 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: Deploying daemon mon.c on smithi196 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/1347043988' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 ceph-mon[35109]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T22:03:36.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 bash[35071]: 2b2a1bd3652be1b233e7cc9547b09a6d3609560e6829506046c79248f30d38ab 2024-09-15T22:03:36.289 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:35 smithi196 systemd[1]: Started Ceph mon.c for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:03:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: mon.a calling monitor election 2024-09-15T22:03:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: mon.c calling monitor election 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: monmap epoch 2 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: last_changed 2024-09-15T22:03:35.953444+0000 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: min_mon_release 19 (squid) 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: election_strategy: 1 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: fsmap 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: mgrmap e13: x(active, since 36s) 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:41.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: mon.a calling monitor election 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: mon.c calling monitor election 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: monmap epoch 2 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: last_changed 2024-09-15T22:03:35.953444+0000 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: min_mon_release 19 (squid) 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: election_strategy: 1 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: fsmap 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: mgrmap e13: x(active, since 36s) 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:41.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:42.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:42 smithi196 ceph-mon[35109]: Deploying daemon mon.b on smithi117 2024-09-15T22:03:42.278 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:42 smithi077 ceph-mon[30980]: Deploying daemon mon.b on smithi117 2024-09-15T22:03:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 -- 172.21.15.196:0/3050048675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f00061b0 msgr2=0x7fc1f0008620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 --2- 172.21.15.196:0/3050048675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f00061b0 0x7fc1f0008620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc208102620 tx=0x7fc20002c4a0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 -- 172.21.15.196:0/3050048675 shutdown_connections 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 --2- 172.21.15.196:0/3050048675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1f00061b0 0x7fc1f0008620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 -- 172.21.15.196:0/3050048675 >> 172.21.15.196:0/3050048675 conn(0x7fc2080784e0 msgr2=0x7fc2080788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 -- 172.21.15.196:0/3050048675 shutdown_connections 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.454+0000 7fc207fff640 1 -- 172.21.15.196:0/3050048675 wait complete. 2024-09-15T22:03:42.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.455+0000 7fc207fff640 1 Processor -- start 2024-09-15T22:03:42.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.455+0000 7fc207fff640 1 -- start start 2024-09-15T22:03:42.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc207fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:42.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc207fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 0x7fc20819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:42.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc207fff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081095b0 con 0x7fc208197700 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc207fff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc208109430 con 0x7fc1f00061b0 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40246/0 (socket says 172.21.15.196:40246) 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 -- 172.21.15.196:0/3209783532 learned_addr learned my addr 172.21.15.196:0/3209783532 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 -- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 msgr2=0x7fc2081971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 -- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 msgr2=0x7fc2081971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc2067fc640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 0x7fc20819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc206ffd640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc2067fc640 1 -- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 msgr2=0x7fc2081971c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:03:42.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.456+0000 7fc2067fc640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc2067fc640 1 -- 172.21.15.196:0/3209783532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc208197d20 con 0x7fc208197700 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc2067fc640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 0x7fc20819ba90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b9b0 tx=0x7fc1fc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc1fc00c920 con 0x7fc208197700 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1fc014440 con 0x7fc208197700 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20819bfd0 con 0x7fc208197700 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc1fc012500 con 0x7fc208197700 2024-09-15T22:03:42.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.457+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20819c590 con 0x7fc208197700 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.458+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc1fc01c070 con 0x7fc208197700 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.459+0000 7fc1effff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 0x7fc1f403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.459+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc1fc04bc30 con 0x7fc208197700 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.459+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20819c8a0 con 0x7fc208197700 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.459+0000 7fc206ffd640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 0x7fc1f403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:42.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.459+0000 7fc206ffd640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 0x7fc1f403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc200012a10 tx=0x7fc200005870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:42.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.464+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1fc00e0a0 con 0x7fc208197700 2024-09-15T22:03:42.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.697+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc208190d90 con 0x7fc208197700 2024-09-15T22:03:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.698+0000 7fc1effff640 1 -- 172.21.15.196:0/3209783532 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc1fc013560 con 0x7fc208197700 2024-09-15T22:03:42.700 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:03:42.700 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","modified":"2024-09-15T22:03:35.953444Z","created":"2024-09-15T22:02:05.827849Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:03:42.700 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 msgr2=0x7fc1f403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 0x7fc1f403b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc200012a10 tx=0x7fc200005870 comp rx=0 tx=0).stop 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 msgr2=0x7fc20819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 0x7fc20819ba90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b9b0 tx=0x7fc1fc00be70 comp rx=0 tx=0).stop 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 shutdown_connections 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc1f4039090 0x7fc1f403b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.702 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc208197700 0x7fc20819ba90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 --2- 172.21.15.196:0/3209783532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1f00061b0 0x7fc2081971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 >> 172.21.15.196:0/3209783532 conn(0x7fc2080784e0 msgr2=0x7fc2080ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 shutdown_connections 2024-09-15T22:03:42.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:42.700+0000 7fc207fff640 1 -- 172.21.15.196:0/3209783532 wait complete. 2024-09-15T22:03:43.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:43 smithi196 ceph-mon[35109]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:43.279 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:43 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/3209783532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:43.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:43 smithi077 ceph-mon[30980]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:43.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:43 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/3209783532' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:44.306 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:03:44.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:03:44.560 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:03:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 -- 172.21.15.196:0/1326080528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 msgr2=0x7f5a4c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1326080528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c107a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009880 tx=0x7f5a4002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 -- 172.21.15.196:0/1326080528 shutdown_connections 2024-09-15T22:03:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1326080528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c107a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:44.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 -- 172.21.15.196:0/1326080528 >> 172.21.15.196:0/1326080528 conn(0x7f5a4c0fd330 msgr2=0x7f5a4c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:44.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 -- 172.21.15.196:0/1326080528 shutdown_connections 2024-09-15T22:03:44.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.974+0000 7f5a50f15640 1 -- 172.21.15.196:0/1326080528 wait complete. 2024-09-15T22:03:44.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 Processor -- start 2024-09-15T22:03:44.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 -- start start 2024-09-15T22:03:44.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:44.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:44.978 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c10bd30 con 0x7f5a4c107670 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.975+0000 7f5a50f15640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c10bbb0 con 0x7f5a4c1979a0 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a49d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.196:44124/0 (socket says 172.21.15.196:44124) 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 -- 172.21.15.196:0/1847683035 learned_addr learned my addr 172.21.15.196:0/1847683035 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a49d9b640 1 -- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 msgr2=0x7f5a4c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a49d9b640 1 -- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 msgr2=0x7f5a4c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a49d9b640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a49d9b640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 -- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 msgr2=0x7f5a4c19bd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:44.979 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 -- 172.21.15.196:0/1847683035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c197fc0 con 0x7f5a4c107670 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a4a59c640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a40004650 tx=0x7f5a40032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5a40035070 con 0x7f5a4c107670 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c19c270 con 0x7f5a4c107670 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a40003ce0 con 0x7f5a4c107670 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5a4003bbd0 con 0x7f5a4c107670 2024-09-15T22:03:44.980 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.976+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c19c7b0 con 0x7f5a4c107670 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.977+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5a40042030 con 0x7f5a4c107670 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.977+0000 7f5a3b7fe640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 0x7f5a3403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.977+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c102db0 con 0x7f5a4c107670 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.978+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5a40040070 con 0x7f5a4c107670 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.978+0000 7f5a49d9b640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 0x7f5a3403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:44.981 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.978+0000 7f5a49d9b640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 0x7f5a3403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006fd0 tx=0x7f5a3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:44.984 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:44.983+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a4004c350 con 0x7f5a4c107670 2024-09-15T22:03:45.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:45.170+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40038350 con 0x7f5a4c107670 2024-09-15T22:03:45.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:45 smithi117 ceph-mon[35021]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T22:03:45.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:45 smithi117 bash[35002]: b5703de2a2f14ffb3c412f20e5966c0bcc611cc274c135d81b345bbdb5871177 2024-09-15T22:03:45.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:45.215+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5a4c107ad0 con 0x7f5a4c107670 2024-09-15T22:03:45.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:45 smithi196 ceph-mon[35109]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:45 smithi077 ceph-mon[30980]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:45 smithi117 systemd[1]: Started Ceph mon.b for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:03:50.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.177+0000 7f5a3b7fe640 1 -- 172.21.15.196:0/1847683035 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f5a40039350 con 0x7f5a4c107670 2024-09-15T22:03:50.179 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:03:50.180 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","modified":"2024-09-15T22:03:45.170039Z","created":"2024-09-15T22:02:05.827849Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:03:50.180 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-15T22:03:50.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 msgr2=0x7f5a3403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:50.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 0x7f5a3403b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006fd0 tx=0x7f5a3c008040 comp rx=0 tx=0).stop 2024-09-15T22:03:50.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 msgr2=0x7f5a4c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:50.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5a40004650 tx=0x7f5a40032c50 comp rx=0 tx=0).stop 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 shutdown_connections 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5a34039040 0x7f5a3403b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c1979a0 0x7f5a4c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 --2- 172.21.15.196:0/1847683035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c107670 0x7f5a4c197460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 >> 172.21.15.196:0/1847683035 conn(0x7f5a4c0fd330 msgr2=0x7f5a4c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 shutdown_connections 2024-09-15T22:03:50.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:50.181+0000 7f5a50f15640 1 -- 172.21.15.196:0/1847683035 wait complete. 2024-09-15T22:03:50.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: mon.c calling monitor election 2024-09-15T22:03:50.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:50.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: mon.a calling monitor election 2024-09-15T22:03:50.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/1847683035' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: monmap epoch 3 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:50.516 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: min_mon_release 19 (squid) 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: election_strategy: 1 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: fsmap 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: mgrmap e13: x(active, since 46s) 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.517 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: mon.c calling monitor election 2024-09-15T22:03:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: mon.a calling monitor election 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/1847683035' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: monmap epoch 3 2024-09-15T22:03:50.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: min_mon_release 19 (squid) 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: election_strategy: 1 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: fsmap 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:50.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: mgrmap e13: x(active, since 46s) 2024-09-15T22:03:50.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:03:50.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:50.818 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-15T22:03:50.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-15T22:03:51.070 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:51.450 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:51 smithi077 ceph-mon[30980]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:51.450 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:51.450 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 -- 172.21.15.77:0/443588367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010102730 msgr2=0x7f9010102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 --2- 172.21.15.77:0/443588367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010102730 0x7f9010102b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f900c009810 tx=0x7f900c02a540 comp rx=0 tx=0).stop 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 -- 172.21.15.77:0/443588367 shutdown_connections 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 --2- 172.21.15.77:0/443588367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010102730 0x7f9010102b10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 -- 172.21.15.77:0/443588367 >> 172.21.15.77:0/443588367 conn(0x7f9010077310 msgr2=0x7f90100fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 -- 172.21.15.77:0/443588367 shutdown_connections 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.531+0000 7f90184ff640 1 -- 172.21.15.77:0/443588367 wait complete. 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 Processor -- start 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 -- start start 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9010197890 0x7f901019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f901010bc30 con 0x7f901019c160 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f901010bab0 con 0x7f9010102730 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f90184ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f901010bdb0 con 0x7f9010197890 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f9016a9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f9016a9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47762/0 (socket says 172.21.15.77:47762) 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f9016a9c640 1 -- 172.21.15.77:0/410120066 learned_addr learned my addr 172.21.15.77:0/410120066 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.532+0000 7f901629b640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f901629b640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 msgr2=0x7f9010197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9015a9a640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9010197890 0x7f901019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f901629b640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 msgr2=0x7f9010197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f901629b640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:03:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f901629b640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9010197890 msgr2=0x7f901019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9010197890 0x7f901019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 msgr2=0x7f9010197350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 -- 172.21.15.77:0/410120066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101a3ed0 con 0x7f901019c160 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f9016a9c640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f900800ba50 tx=0x7f900800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008019070 con 0x7f901019c160 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f900800cbe0 con 0x7f901019c160 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90101a4100 con 0x7f901019c160 2024-09-15T22:03:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90080076a0 con 0x7f901019c160 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.533+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90101a45c0 con 0x7f901019c160 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.534+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f900801c070 con 0x7f901019c160 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.534+0000 7f90037fe640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 0x7f8ffc03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.534+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f900804bf70 con 0x7f901019c160 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.534+0000 7f901629b640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 0x7f8ffc03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.534+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9010069f90 con 0x7f901019c160 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.535+0000 7f901629b640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 0x7f8ffc03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f900c0058e0 tx=0x7f900c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:51.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.539+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90080112a0 con 0x7f901019c160 2024-09-15T22:03:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:51 smithi196 ceph-mon[35109]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.771+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f90101a4850 con 0x7f901019c160 2024-09-15T22:03:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.772+0000 7f90037fe640 1 -- 172.21.15.77:0/410120066 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f900800ce00 con 0x7f901019c160 2024-09-15T22:03:51.773 INFO:teuthology.orchestra.run.smithi077.stdout:# minimal ceph.conf for 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:51.773 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2024-09-15T22:03:51.774 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:51.774 INFO:teuthology.orchestra.run.smithi077.stdout: mon_host = [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 msgr2=0x7f8ffc03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 0x7f8ffc03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f900c0058e0 tx=0x7f900c005850 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 msgr2=0x7f901019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f900800ba50 tx=0x7f900800bf10 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 shutdown_connections 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ffc039730 0x7f8ffc03bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f901019c160 0x7f901019c610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9010197890 0x7f901019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 --2- 172.21.15.77:0/410120066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9010102730 0x7f9010197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 >> 172.21.15.77:0/410120066 conn(0x7f9010077310 msgr2=0x7f90101035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 shutdown_connections 2024-09-15T22:03:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:51.775+0000 7f90184ff640 1 -- 172.21.15.77:0/410120066 wait complete. 2024-09-15T22:03:52.553 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-15T22:03:52.553 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:52.553 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:03:52.580 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:52.580 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.c calling monitor election 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.a calling monitor election 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/1847683035' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: monmap epoch 3 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: min_mon_release 19 (squid) 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: election_strategy: 1 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:52.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: fsmap 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mgrmap e13: x(active, since 46s) 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: overall HEALTH_OK 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:52.615 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:52.648 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:03:52.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:03:52.676 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:03:52.676 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:52.741 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:03:52.741 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:03:52.770 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:03:52.770 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:03:52.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mon.b calling monitor election 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mon.b calling monitor election 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mon.c calling monitor election 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mon.a calling monitor election 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:03:52.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: monmap epoch 3 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: min_mon_release 19 (squid) 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: election_strategy: 1 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: fsmap 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: mgrmap e13: x(active, since 48s) 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:03:52.780 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:52.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:52.836 INFO:tasks.cephadm:Adding mgr.x on smithi077 2024-09-15T22:03:52.836 INFO:tasks.cephadm:Adding mgr.y on smithi117 2024-09-15T22:03:52.836 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-09-15T22:03:52.837 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi077=x;smithi117=y;smithi196=z' 2024-09-15T22:03:53.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mon.b calling monitor election 2024-09-15T22:03:53.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mon.b calling monitor election 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mon.c calling monitor election 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mon.a calling monitor election 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: monmap epoch 3 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:53.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: min_mon_release 19 (squid) 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: election_strategy: 1 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: fsmap 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: mgrmap e13: x(active, since 48s) 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:53.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:53.112 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.b calling monitor election 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.b calling monitor election 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.c calling monitor election 2024-09-15T22:03:53.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.a calling monitor election 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: monmap epoch 3 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: last_changed 2024-09-15T22:03:45.170039+0000 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: created 2024-09-15T22:02:05.827849+0000 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: min_mon_release 19 (squid) 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: election_strategy: 1 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-09-15T22:03:53.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: fsmap 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: mgrmap e13: x(active, since 48s) 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: overall HEALTH_OK 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.115 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.116 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.116 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:53.116 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:53.116 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:53.116 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 -- 172.21.15.196:0/1666224085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 msgr2=0x7fc980101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 --2- 172.21.15.196:0/1666224085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980101ae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc974009880 tx=0x7fc97402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 -- 172.21.15.196:0/1666224085 shutdown_connections 2024-09-15T22:03:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 --2- 172.21.15.196:0/1666224085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980101ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 -- 172.21.15.196:0/1666224085 >> 172.21.15.196:0/1666224085 conn(0x7fc9800fce90 msgr2=0x7fc9800ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 -- 172.21.15.196:0/1666224085 shutdown_connections 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.544+0000 7fc98784b640 1 -- 172.21.15.196:0/1666224085 wait complete. 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 Processor -- start 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 -- start start 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 0x7fc980074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980074fa0 0x7fc9801a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9801077f0 con 0x7fc9801016e0 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc980107670 con 0x7fc9800745e0 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc98784b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc980107970 con 0x7fc980074fa0 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc9855e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.545+0000 7fc9855e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.196:44532/0 (socket says 172.21.15.196:44532) 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 -- 172.21.15.196:0/2312341067 learned_addr learned my addr 172.21.15.196:0/2312341067 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc985de8640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980074fa0 0x7fc9801a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc984de6640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 0x7fc980074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 -- 172.21.15.196:0/2312341067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980074fa0 msgr2=0x7fc9801a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980074fa0 0x7fc9801a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 -- 172.21.15.196:0/2312341067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 msgr2=0x7fc980074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 0x7fc980074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 -- 172.21.15.196:0/2312341067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9801a15b0 con 0x7fc9801016e0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc984de6640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 0x7fc980074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9855e7640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc974005ab0 tx=0x7fc974032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97403f030 con 0x7fc9801016e0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc974003ce0 con 0x7fc9801016e0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9801a1840 con 0x7fc9801016e0 2024-09-15T22:03:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974035070 con 0x7fc9801016e0 2024-09-15T22:03:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.546+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9801a1df0 con 0x7fc9801016e0 2024-09-15T22:03:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.547+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc974042030 con 0x7fc9801016e0 2024-09-15T22:03:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.547+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9801905b0 con 0x7fc9801016e0 2024-09-15T22:03:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.547+0000 7fc9727fc640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 0x7fc96c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.548+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc97406cea0 con 0x7fc9801016e0 2024-09-15T22:03:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.548+0000 7fc984de6640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 0x7fc96c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.548+0000 7fc984de6640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 0x7fc96c03bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:53.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.552+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc974030c30 con 0x7fc9801016e0 2024-09-15T22:03:53.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.734+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi117=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7fc9800626c0 con 0x7fc96c039730 2024-09-15T22:03:53.740 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.739+0000 7fc9727fc640 1 -- 172.21.15.196:0/2312341067 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc9800626c0 con 0x7fc96c039730 2024-09-15T22:03:53.740 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 msgr2=0x7fc96c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 0x7fc96c03bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c008040 comp rx=0 tx=0).stop 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 msgr2=0x7fc980075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc974005ab0 tx=0x7fc974032d00 comp rx=0 tx=0).stop 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 shutdown_connections 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc96c039730 0x7fc96c03bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc980074fa0 0x7fc9801a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9800745e0 0x7fc980074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.743 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 --2- 172.21.15.196:0/2312341067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9801016e0 0x7fc980075f90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:53.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 >> 172.21.15.196:0/2312341067 conn(0x7fc9800fce90 msgr2=0x7fc9800fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:53.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 shutdown_connections 2024-09-15T22:03:53.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:03:53.741+0000 7fc98784b640 1 -- 172.21.15.196:0/2312341067 wait complete. 2024-09-15T22:03:53.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:53 smithi196 ceph-mon[35109]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:03:53.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:53 smithi196 ceph-mon[35109]: Reconfiguring daemon mon.a on smithi077 2024-09-15T22:03:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:54.059 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:53 smithi117 ceph-mon[35021]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:03:54.059 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:53 smithi117 ceph-mon[35021]: Reconfiguring daemon mon.a on smithi077 2024-09-15T22:03:54.059 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:54.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:53 smithi077 ceph-mon[30980]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:03:54.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:53 smithi077 ceph-mon[30980]: Reconfiguring daemon mon.a on smithi077 2024-09-15T22:03:54.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:03:54.370 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.y.service 2024-09-15T22:03:54.373 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z.service 2024-09-15T22:03:54.376 INFO:tasks.cephadm:Deploying OSDs... 2024-09-15T22:03:54.376 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:03:54.376 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:03:54.395 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:03:54.395 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:54.452 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-15 22:03:15.389145069 +0000 2024-09-15T22:03:54.453 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-15 21:59:04.403169003 +0000 2024-09-15T22:03:54.453 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-15 21:59:04.403169003 +0000 2024-09-15T22:03:54.453 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-15 21:59:04.403169003 +0000 2024-09-15T22:03:54.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:03:54.518 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-15T22:03:54.519 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-15T22:03:54.519 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000280546 s, 1.8 MB/s 2024-09-15T22:03:54.520 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:03:54.577 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-09-15T22:03:54.633 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:03:54.633 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-15 22:03:15.415144526 +0000 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-15 21:59:04.732162087 +0000 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-15 21:59:04.732162087 +0000 2024-09-15T22:03:54.634 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-15 21:59:04.732162087 +0000 2024-09-15T22:03:54.634 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:03:54.698 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-15T22:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-15T22:03:54.699 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000186619 s, 2.7 MB/s 2024-09-15T22:03:54.700 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:03:54.757 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-15 22:03:15.439144025 +0000 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-15 21:59:05.069155002 +0000 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-15 21:59:05.069155002 +0000 2024-09-15T22:03:54.814 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-15 21:59:05.069155002 +0000 2024-09-15T22:03:54.815 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: Reconfiguring daemon mon.b on smithi117 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='client.14205 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi117=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: Saving service mgr spec with placement smithi077=x;smithi117=y;smithi196=z;count:3 2024-09-15T22:03:54.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:54.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:54.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:54.855 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-15T22:03:54.855 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-15T22:03:54.855 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000172131 s, 3.0 MB/s 2024-09-15T22:03:54.856 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:03:54.912 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-09-15T22:03:54.967 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:03:54.967 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-15 22:03:15.464143502 +0000 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-15 21:59:05.404147960 +0000 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-15 21:59:05.404147960 +0000 2024-09-15T22:03:54.968 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-15 21:59:05.404147960 +0000 2024-09-15T22:03:54.968 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:03:55.000 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.000 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.000 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:03:55.000 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: Reconfiguring daemon mon.b on smithi117 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='client.14205 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi117=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: Saving service mgr spec with placement smithi077=x;smithi117=y;smithi196=z;count:3 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:55.001 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.030 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-09-15T22:03:55.030 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-09-15T22:03:55.031 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000198781 s, 2.6 MB/s 2024-09-15T22:03:55.031 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:03:55.089 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T22:03:55.089 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:03:55.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: Reconfiguring daemon mon.b on smithi117 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='client.14205 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi117=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: Saving service mgr spec with placement smithi077=x;smithi117=y;smithi196=z;count:3 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:55.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:55.106 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:03:55.106 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-15T22:03:55.162 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 22:03:44.629871129 +0000 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 21:59:23.107935965 +0000 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 21:59:23.107935965 +0000 2024-09-15T22:03:55.163 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 21:59:23.107935965 +0000 2024-09-15T22:03:55.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:03:55.227 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T22:03:55.227 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T22:03:55.227 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00025213 s, 2.0 MB/s 2024-09-15T22:03:55.228 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:03:55.285 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-15T22:03:55.341 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:03:55.341 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:55.341 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T22:03:55.341 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:55.342 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:55.342 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 22:03:44.630871110 +0000 2024-09-15T22:03:55.342 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 21:59:23.442929998 +0000 2024-09-15T22:03:55.342 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 21:59:23.442929998 +0000 2024-09-15T22:03:55.342 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 21:59:23.442929998 +0000 2024-09-15T22:03:55.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:03:55.404 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T22:03:55.405 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T22:03:55.405 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000235403 s, 2.2 MB/s 2024-09-15T22:03:55.406 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:03:55.462 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 22:03:44.630871110 +0000 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 21:59:23.753924460 +0000 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 21:59:23.753924460 +0000 2024-09-15T22:03:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 21:59:23.753924460 +0000 2024-09-15T22:03:55.518 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:03:55.581 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T22:03:55.582 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T22:03:55.582 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000213875 s, 2.4 MB/s 2024-09-15T22:03:55.583 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:03:55.639 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-15T22:03:55.694 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:03:55.694 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:55.694 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 22:03:44.630871110 +0000 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 21:59:24.061918975 +0000 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 21:59:24.061918975 +0000 2024-09-15T22:03:55.695 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 21:59:24.061918975 +0000 2024-09-15T22:03:55.695 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:03:55.757 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T22:03:55.757 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T22:03:55.757 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000182795 s, 2.8 MB/s 2024-09-15T22:03:55.759 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:03:55.816 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-15T22:03:55.816 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:03:55.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:03:55.832 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-15T22:03:55.887 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:03:55.887 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:55.887 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-15 22:03:35.362823693 +0000 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-15 21:59:20.600739162 +0000 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-15 21:59:20.600739162 +0000 2024-09-15T22:03:55.888 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-15 21:59:20.600739162 +0000 2024-09-15T22:03:55.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:03:55.952 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-15T22:03:55.952 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-15T22:03:55.952 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000339538 s, 1.5 MB/s 2024-09-15T22:03:55.953 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:03:56.010 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-15 22:03:35.362823693 +0000 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-15 21:59:20.937731274 +0000 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-15 21:59:20.937731274 +0000 2024-09-15T22:03:56.066 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-15 21:59:20.937731274 +0000 2024-09-15T22:03:56.067 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:03:56.130 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-15T22:03:56.130 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-15T22:03:56.131 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000175946 s, 2.9 MB/s 2024-09-15T22:03:56.132 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:03:56.191 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-15T22:03:56.246 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-15 22:03:35.363823670 +0000 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-15 21:59:21.249723970 +0000 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-15 21:59:21.249723970 +0000 2024-09-15T22:03:56.247 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-15 21:59:21.249723970 +0000 2024-09-15T22:03:56.247 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:56.307 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:56.308 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:56 smithi196 ceph-mon[35109]: Reconfiguring daemon mon.c on smithi196 2024-09-15T22:03:56.310 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-15T22:03:56.310 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-15T22:03:56.310 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000194224 s, 2.6 MB/s 2024-09-15T22:03:56.311 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:56 smithi077 ceph-mon[30980]: Reconfiguring daemon mon.c on smithi196 2024-09-15T22:03:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:03:56.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:03:56.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:03:56.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:56.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:56 smithi117 ceph-mon[35021]: Reconfiguring daemon mon.c on smithi196 2024-09-15T22:03:56.367 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-15 22:03:35.363823670 +0000 2024-09-15T22:03:56.423 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-15 21:59:21.569716479 +0000 2024-09-15T22:03:56.424 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-15 21:59:21.569716479 +0000 2024-09-15T22:03:56.424 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-15 21:59:21.569716479 +0000 2024-09-15T22:03:56.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:03:56.487 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-15T22:03:56.488 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-15T22:03:56.488 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000241356 s, 2.1 MB/s 2024-09-15T22:03:56.489 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:03:56.546 INFO:tasks.cephadm:Deploying osd.0 on smithi077 with /dev/vg_nvme/lv_4... 2024-09-15T22:03:56.546 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:03:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:57.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:57.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:57.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:57.209 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:57.210 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:57.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:58.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:58 smithi077 ceph-mon[30980]: Deploying daemon mgr.z on smithi196 2024-09-15T22:03:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-mon[35109]: Deploying daemon mgr.z on smithi196 2024-09-15T22:03:58.354 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 podman[36781]: 2024-09-15 22:03:58.297356049 +0000 UTC m=+0.116125722 container create 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-15T22:03:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:58 smithi117 ceph-mon[35021]: Deploying daemon mgr.z on smithi196 2024-09-15T22:03:58.706 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 podman[36781]: 2024-09-15 22:03:58.515415637 +0000 UTC m=+0.334185279 container init 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 podman[36781]: 2024-09-15 22:03:58.518033833 +0000 UTC m=+0.336803465 container start 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-mgr[36801]: -- 172.21.15.196:0/2584771370 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601490890e0 con 0x560149067000 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 bash[36781]: 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 systemd[1]: Started Ceph mgr.z for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:58.642+0000 7fe1ca4fc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:03:58.707 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:58.704+0000 7fe1ca4fc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:03:59.066 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:58.852+0000 7fe1ca4fc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:03:59.066 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:58.917+0000 7fe1ca4fc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:59.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:03:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:59.106 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:03:59.132 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-09-15T22:03:59.353 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:59.067+0000 7fe1ca4fc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:59.207+0000 7fe1ca4fc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:59.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:03:59.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:03:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 -- 172.21.15.77:0/2066796939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4102e00 msgr2=0x7f30d4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 --2- 172.21.15.77:0/2066796939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4102e00 0x7f30d4103280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009900 tx=0x7f30c402a410 comp rx=0 tx=0).stop 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 -- 172.21.15.77:0/2066796939 shutdown_connections 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 --2- 172.21.15.77:0/2066796939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 0x7f30d410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 --2- 172.21.15.77:0/2066796939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4102e00 0x7f30d4103280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 --2- 172.21.15.77:0/2066796939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4101c00 0x7f30d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 -- 172.21.15.77:0/2066796939 >> 172.21.15.77:0/2066796939 conn(0x7f30d40fd390 msgr2=0x7f30d40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 -- 172.21.15.77:0/2066796939 shutdown_connections 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.812+0000 7f30d9e37640 1 -- 172.21.15.77:0/2066796939 wait complete. 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 Processor -- start 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 -- start start 2024-09-15T22:03:59.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4102e00 0x7f30d41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 0x7f30d419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d860 con 0x7f30d4101c00 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d6e0 con 0x7f30d41037c0 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.813+0000 7f30d9e37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d9e0 con 0x7f30d4102e00 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48922/0 (socket says 172.21.15.77:48922) 2024-09-15T22:03:59.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 -- 172.21.15.77:0/3207466104 learned_addr learned my addr 172.21.15.77:0/3207466104 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d2ffd640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4102e00 0x7f30d41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d3fff640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 0x7f30d419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 -- 172.21.15.77:0/3207466104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4102e00 msgr2=0x7f30d41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4102e00 0x7f30d41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 -- 172.21.15.77:0/3207466104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 msgr2=0x7f30d419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 0x7f30d419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 -- 172.21.15.77:0/3207466104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419fad0 con 0x7f30d4101c00 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d37fe640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30c800d770 tx=0x7f30c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8017070 con 0x7f30d4101c00 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419fdc0 con 0x7f30d4101c00 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8009c30 con 0x7f30d4101c00 2024-09-15T22:03:59.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8003da0 con 0x7f30d4101c00 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.814+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a0220 con 0x7f30d4101c00 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.815+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f30c8010460 con 0x7f30d4101c00 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.815+0000 7f30d0ff9640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 0x7f30bc03b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.815+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f30c80035f0 con 0x7f30d4101c00 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.815+0000 7f30d2ffd640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 0x7f30bc03b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.815+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d4108fa0 con 0x7f30d4101c00 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.816+0000 7f30d2ffd640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 0x7f30bc03b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30c40314e0 tx=0x7f30c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.820+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30c8009de0 con 0x7f30d4101c00 2024-09-15T22:03:59.853 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:59.519+0000 7fe1ca4fc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:03:59.853 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:59.579+0000 7fe1ca4fc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:03:59.853 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:03:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:03:59.762+0000 7fe1ca4fc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:03:59.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:03:59.995+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f30d4062630 con 0x7f30bc039500 2024-09-15T22:04:00.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:00 smithi077 ceph-mon[30980]: Deploying daemon mgr.y on smithi117 2024-09-15T22:04:00.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:00 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:00.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:00 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:00.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:00 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:00.351 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:00 smithi117 ceph-mon[35021]: Deploying daemon mgr.y on smithi117 2024-09-15T22:04:00.351 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:00 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:00.351 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:00 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:00.351 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:00 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-mon[35109]: Deploying daemon mgr.y on smithi117 2024-09-15T22:04:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:00.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 systemd[1]: Starting Ceph mgr.y for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:04:00.968 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 podman[36377]: 2024-09-15 22:04:00.698457854 +0000 UTC m=+0.090561368 container create 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T22:04:00.968 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 podman[36377]: 2024-09-15 22:04:00.623577668 +0000 UTC m=+0.015681147 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:04:00.968 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 podman[36377]: 2024-09-15 22:04:00.90899765 +0000 UTC m=+0.301101122 container init 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T22:04:00.968 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 podman[36377]: 2024-09-15 22:04:00.912263354 +0000 UTC m=+0.304366823 container start 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-15T22:04:00.969 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 ceph-mgr[36396]: -- 172.21.15.117:0/4055210146 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e8295b0e0 con 0x555e82938800 2024-09-15T22:04:00.969 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 bash[36377]: 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c 2024-09-15T22:04:01.071 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:00.645+0000 7fe1ca4fc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:04:01.071 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:00.713+0000 7fe1ca4fc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:04:01.255 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:00 smithi117 systemd[1]: Started Ceph mgr.y for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:04:01.255 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.035+0000 7f038404b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:04:01.255 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.094+0000 7f038404b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.253+0000 7f038404b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:01.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.322+0000 7f038404b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:04:01.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.484+0000 7f038404b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:04:01.723 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:01.454+0000 7fe1ca4fc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:04:01.723 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:01.579+0000 7fe1ca4fc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:04:01.928 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.631+0000 7f038404b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:04:02.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:01.725+0000 7fe1ca4fc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:04:02.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:01.790+0000 7fe1ca4fc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:04:02.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:01.945+0000 7fe1ca4fc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:04:02.017 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.014+0000 7fe1ca4fc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:04:02.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:02 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:02.216 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:01.958+0000 7f038404b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:04:02.216 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:02.019+0000 7f038404b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:04:02.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:02.312 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.075+0000 7fe1ca4fc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:04:02.312 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.244+0000 7fe1ca4fc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:04:02.312 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.310+0000 7fe1ca4fc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:04:02.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:02 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:02.581 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.379+0000 7fe1ca4fc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:04:02.582 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.462+0000 7fe1ca4fc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:04:02.582 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.520+0000 7fe1ca4fc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:04:02.582 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:02.580+0000 7fe1ca4fc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:04:02.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:02.214+0000 7f038404b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:04:03.362 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:03.108+0000 7f038404b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:04:03.362 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:03.176+0000 7f038404b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]: dispatch 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]': finished 2024-09-15T22:04:03.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:04:03.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:03.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:04:03.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:04:03.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2075002346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]': finished 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2075002346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:03.603 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:04:03.365+0000 7fe1ca4fc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:04:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]: dispatch 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1552992802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "286650d4-5c20-4659-a7ef-db7c243e0aba"}]': finished 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:04:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:04:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2075002346' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:04.156 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:03.895+0000 7f038404b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:04:04.156 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.011+0000 7f038404b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:04:04.156 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.154+0000 7f038404b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:04:04.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:04.224+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f30c800f3c0 con 0x7f30d4101c00 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.215+0000 7f038404b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.365+0000 7f038404b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.432+0000 7f038404b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: Reconfiguring daemon mgr.x on smithi077 2024-09-15T22:04:04.434 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: Standby manager daemon z started 2024-09-15T22:04:04.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:04:04.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:04.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:04:04.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:04.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: Reconfiguring daemon mgr.x on smithi077 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: Standby manager daemon z started 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:04:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: Reconfiguring daemon mgr.x on smithi077 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: Standby manager daemon z started 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/2741323612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:04.715 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.491+0000 7f038404b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:04:04.715 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.648+0000 7f038404b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:04:04.716 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.714+0000 7f038404b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:04:04.983 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.779+0000 7f038404b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:04:04.983 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.863+0000 7f038404b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:04:04.983 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.923+0000 7f038404b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:04:04.983 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:04.981+0000 7f038404b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:04:05.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:05 smithi117 ceph-mon[35021]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:05.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:05 smithi117 ceph-mon[35021]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-15T22:04:05.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:04:05.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:05 smithi077 ceph-mon[30980]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:05.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:05 smithi077 ceph-mon[30980]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-15T22:04:05.585 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:04:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:05 smithi196 ceph-mon[35109]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:05 smithi196 ceph-mon[35109]: mgrmap e14: x(active, since 60s), standbys: z 2024-09-15T22:04:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:04:06.112 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:04:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:04:05.767+0000 7f038404b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:04:06.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:06.231+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f30c8010750 con 0x7f30d4101c00 2024-09-15T22:04:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:06 smithi077 ceph-mon[30980]: Standby manager daemon y started 2024-09-15T22:04:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:06 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:04:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:06 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:06 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:04:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:06 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:06 smithi196 ceph-mon[35109]: Standby manager daemon y started 2024-09-15T22:04:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:06 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:04:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:06 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:06 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:04:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:06 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:06 smithi117 ceph-mon[35021]: Standby manager daemon y started 2024-09-15T22:04:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:06 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:04:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:06 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:04:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:06 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:04:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:06 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3442025246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:04:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:07 smithi077 ceph-mon[30980]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:07 smithi077 ceph-mon[30980]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-15T22:04:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:04:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:07 smithi196 ceph-mon[35109]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:07 smithi196 ceph-mon[35109]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-15T22:04:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:04:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:07 smithi117 ceph-mon[35021]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:07 smithi117 ceph-mon[35021]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-09-15T22:04:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:04:09.448 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:09 smithi077 ceph-mon[30980]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:09 smithi196 ceph-mon[35109]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:09 smithi117 ceph-mon[35021]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:10 smithi077 ceph-mon[30980]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:10 smithi196 ceph-mon[35109]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:10 smithi117 ceph-mon[35021]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:12.247 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:04:12.247 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:12.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:04:12.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:04:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:13.160 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:12 smithi077 ceph-mon[30980]: Deploying daemon osd.0 on smithi077 2024-09-15T22:04:13.160 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:12 smithi077 ceph-mon[30980]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:13.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:13 smithi196 ceph-mon[35109]: Deploying daemon osd.0 on smithi077 2024-09-15T22:04:13.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:13 smithi196 ceph-mon[35109]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:13.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:13 smithi117 ceph-mon[35021]: Deploying daemon osd.0 on smithi077 2024-09-15T22:04:13.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:13 smithi117 ceph-mon[35021]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:15 smithi077 ceph-mon[30980]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:15 smithi196 ceph-mon[35109]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:15 smithi117 ceph-mon[35021]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:17.428 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:17 smithi077 ceph-mon[30980]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:17 smithi196 ceph-mon[35109]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:17 smithi117 ceph-mon[35021]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:18.447 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:18.447 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:18.448 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:19 smithi077 ceph-mon[30980]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:19 smithi196 ceph-mon[35109]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:19 smithi117 ceph-mon[35021]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:21 smithi077 ceph-mon[30980]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:21 smithi196 ceph-mon[35109]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:21 smithi117 ceph-mon[35021]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:21.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.771+0000 7f30d0ff9640 1 -- 172.21.15.77:0/3207466104 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30d4062630 con 0x7f30bc039500 2024-09-15T22:04:21.774 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 0 on host 'smithi077' 2024-09-15T22:04:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 msgr2=0x7f30bc03b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 0x7f30bc03b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30c40314e0 tx=0x7f30c4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:04:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 msgr2=0x7f30d4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:21.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30c800d770 tx=0x7f30c800dc30 comp rx=0 tx=0).stop 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 shutdown_connections 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f30bc039500 0x7f30bc03b9c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d41037c0 0x7f30d419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30d4102e00 0x7f30d41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 --2- 172.21.15.77:0/3207466104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4101c00 0x7f30d4197ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 >> 172.21.15.77:0/3207466104 conn(0x7f30d40fd390 msgr2=0x7f30d4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 shutdown_connections 2024-09-15T22:04:21.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:04:21.774+0000 7f30d9e37640 1 -- 172.21.15.77:0/3207466104 wait complete. 2024-09-15T22:04:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:22.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:22.367 DEBUG:teuthology.orchestra.run.smithi077:osd.0> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.0.service 2024-09-15T22:04:22.370 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_4... 2024-09-15T22:04:22.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:04:22.608 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:04:23.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:23 smithi077 ceph-mon[30980]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:23.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:23 smithi196 ceph-mon[35109]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:23.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:23 smithi117 ceph-mon[35021]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:23.987 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:04:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:04:23.750+0000 7fdaa6e9d640 -1 osd.0 0 log_to_monitors true 2024-09-15T22:04:24.237 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:24 smithi077 ceph-mon[30980]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:04:24.302 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:24 smithi117 ceph-mon[35021]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:04:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:24 smithi196 ceph-mon[35109]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:04:25.009 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:04:25.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-09-15T22:04:25.293 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:04:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-mon[30980]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:04:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-mon[30980]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:04:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-mon[30980]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:04:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-mon[30980]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:25.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:25 smithi196 ceph-mon[35109]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:04:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:25 smithi196 ceph-mon[35109]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:04:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:25 smithi196 ceph-mon[35109]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:04:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:25 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:25 smithi196 ceph-mon[35109]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:25 smithi117 ceph-mon[35021]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:04:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:25 smithi117 ceph-mon[35021]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:04:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:25 smithi117 ceph-mon[35021]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:04:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:25 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:25 smithi117 ceph-mon[35021]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:25.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 -- 172.21.15.117:0/3483288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de41079c0 msgr2=0x7f2de4109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:25.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3483288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de41079c0 0x7f2de4109db0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009900 tx=0x7f2ddc02a410 comp rx=0 tx=0).stop 2024-09-15T22:04:25.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 -- 172.21.15.117:0/3483288318 shutdown_connections 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3483288318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de41079c0 0x7f2de4109db0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3483288318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de4105090 0x7f2de4107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3483288318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 -- 172.21.15.117:0/3483288318 >> 172.21.15.117:0/3483288318 conn(0x7f2de40fc2a0 msgr2=0x7f2de40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 -- 172.21.15.117:0/3483288318 shutdown_connections 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.746+0000 7f2de9a72640 1 -- 172.21.15.117:0/3483288318 wait complete. 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 Processor -- start 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 -- start start 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:25.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 0x7f2de419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2de410d8d0 con 0x7f2de4102760 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2de410d750 con 0x7f2de4105090 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de9a72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2de410da50 con 0x7f2de41079c0 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53934/0 (socket says 172.21.15.117:53934) 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.747+0000 7f2de2ffd640 1 -- 172.21.15.117:0/3192246318 learned_addr learned my addr 172.21.15.117:0/3192246318 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de37fe640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de3fff640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 0x7f2de419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 -- 172.21.15.117:0/3192246318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 msgr2=0x7f2de419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 0x7f2de419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 -- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 msgr2=0x7f2de4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 -- 172.21.15.117:0/3192246318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de419fb60 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de3fff640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 0x7f2de419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de37fe640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de2ffd640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800d740 tx=0x7f2dd800dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8017070 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd800de50 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de419fe50 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8004e00 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.748+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de41a02b0 con 0x7f2de4105090 2024-09-15T22:04:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.749+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de40fdd90 con 0x7f2de4105090 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.750+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f2dd8004fa0 con 0x7f2de4105090 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.751+0000 7f2de0ff9640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 0x7f2dcc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.751+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f2dd80cd790 con 0x7f2de4105090 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.751+0000 7f2de37fe640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 0x7f2dcc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.751+0000 7f2de37fe640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 0x7f2dcc0a56a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005fd0 tx=0x7f2dd4009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.755+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dd809c040 con 0x7f2de4105090 2024-09-15T22:04:25.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:04:25.516+0000 7fdaa3634640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T22:04:25.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:04:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:04:25.519+0000 7fda9ac40640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:04:25.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:25.929+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2de4062630 con 0x7f2dcc0a31e0 2024-09-15T22:04:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:04:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:04:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.334 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:26.335 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784]' entity='osd.0' 2024-09-15T22:04:26.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:26.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:26.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi077 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] boot 2024-09-15T22:04:27.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:04:27.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:27.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi077 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] boot 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:04:27.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:27.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:04:27.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:04:27.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi077 2024-09-15T22:04:27.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 17088M 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: osd.0 [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] boot 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:04:27.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:04:28.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/4009070957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]': finished 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:04:28.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:28 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/4009070957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]': finished 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:04:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:28 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:28.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/4009070957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]: dispatch 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cddb21e-cf12-4806-9540-0b7c570af2aa"}]': finished 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:04:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:28 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:29.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/604525447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:29.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:29 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/604525447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:29.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:29 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/604525447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:30.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:30 smithi077 ceph-mon[30980]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:30.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:30 smithi196 ceph-mon[35109]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:30.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:30 smithi117 ceph-mon[35021]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:33.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:33 smithi117 ceph-mon[35021]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:33.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:33 smithi077 ceph-mon[30980]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:33 smithi196 ceph-mon[35109]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:35.296 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:35 smithi117 ceph-mon[35021]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:35.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:35 smithi077 ceph-mon[30980]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:35.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:35 smithi196 ceph-mon[35109]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:37.253 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:37 smithi117 ceph-mon[35021]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:37.253 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:04:37.253 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:37 smithi077 ceph-mon[30980]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:04:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:37 smithi196 ceph-mon[35109]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:04:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:38.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:38 smithi117 ceph-mon[35021]: Deploying daemon osd.1 on smithi117 2024-09-15T22:04:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:38 smithi077 ceph-mon[30980]: Deploying daemon osd.1 on smithi117 2024-09-15T22:04:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:38 smithi196 ceph-mon[35109]: Deploying daemon osd.1 on smithi117 2024-09-15T22:04:39.318 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:39 smithi117 ceph-mon[35021]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:39.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:39 smithi077 ceph-mon[30980]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:39.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:39 smithi196 ceph-mon[35109]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:41 smithi077 ceph-mon[30980]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:41.595 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:41 smithi117 ceph-mon[35021]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:41 smithi196 ceph-mon[35109]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:43.500 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:43 smithi117 ceph-mon[35021]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:43.501 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:43.501 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:43.501 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:43 smithi077 ceph-mon[30980]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:43.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:43.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:43 smithi196 ceph-mon[35109]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:43.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.329 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:45.330 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.330 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.330 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:45.330 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:45.330 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:46.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.870+0000 7f2de0ff9640 1 -- 172.21.15.117:0/3192246318 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2de4062630 con 0x7f2dcc0a31e0 2024-09-15T22:04:46.871 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 msgr2=0x7f2dcc0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 0x7f2dcc0a56a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4005fd0 tx=0x7f2dd4009520 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 msgr2=0x7f2de4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800d740 tx=0x7f2dd800dc00 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 shutdown_connections 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2dcc0a31e0 0x7f2dcc0a56a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de41079c0 0x7f2de419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de4105090 0x7f2de4198470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 --2- 172.21.15.117:0/3192246318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2de4102760 0x7f2de4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 >> 172.21.15.117:0/3192246318 conn(0x7f2de40fc2a0 msgr2=0x7f2de4103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:46.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 shutdown_connections 2024-09-15T22:04:46.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:04:46.872+0000 7f2de9a72640 1 -- 172.21.15.117:0/3192246318 wait complete. 2024-09-15T22:04:47.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:47 smithi117 ceph-mon[35021]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:47.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:47.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.477 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.1.service 2024-09-15T22:04:47.479 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-09-15T22:04:47.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:04:47.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:47 smithi196 ceph-mon[35109]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:47.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:47.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:47 smithi077 ceph-mon[30980]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:47.732 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:04:49.209 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:04:49 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:04:49.024+0000 7fc27b370640 -1 osd.1 0 log_to_monitors true 2024-09-15T22:04:49.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:49 smithi196 ceph-mon[35109]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:49.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:49 smithi196 ceph-mon[35109]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:49 smithi196 ceph-mon[35109]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:49 smithi077 ceph-mon[30980]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:49 smithi077 ceph-mon[30980]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:49 smithi077 ceph-mon[30980]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:49 smithi117 ceph-mon[35021]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:49.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:49 smithi117 ceph-mon[35021]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.601 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:49 smithi117 ceph-mon[35021]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:04:49.947 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:04:49.973 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-09-15T22:04:50.257 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:04:50.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:04:50.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:04:50.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:50.307 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.307 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-mon[35021]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.495 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:50 smithi196 ceph-mon[35109]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: from='osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:04:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:50 smithi077 ceph-mon[30980]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:04:50.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.680+0000 7f0aac77d640 1 -- 172.21.15.196:0/3415563373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa4100af0 msgr2=0x7f0aa4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:50.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.680+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3415563373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa4100af0 0x7f0aa4100ef0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009900 tx=0x7f0a9402a410 comp rx=0 tx=0).stop 2024-09-15T22:04:50.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- 172.21.15.196:0/3415563373 shutdown_connections 2024-09-15T22:04:50.697 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3415563373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa41026b0 0x7f0aa4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3415563373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3415563373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa4100af0 0x7f0aa4100ef0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- 172.21.15.196:0/3415563373 >> 172.21.15.196:0/3415563373 conn(0x7f0aa40fc280 msgr2=0x7f0aa40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- 172.21.15.196:0/3415563373 shutdown_connections 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- 172.21.15.196:0/3415563373 wait complete. 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 Processor -- start 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- start start 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 0x7f0aa4196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa410c780 con 0x7f0aa4100af0 2024-09-15T22:04:50.698 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa410c600 con 0x7f0aa4101cf0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.681+0000 7f0aac77d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa410c900 con 0x7f0aa41026b0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43852/0 (socket says 172.21.15.196:43852) 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 -- 172.21.15.196:0/3958959766 learned_addr learned my addr 172.21.15.196:0/3958959766 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aa9d18640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaa519640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 0x7f0aa4196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 -- 172.21.15.196:0/3958959766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 msgr2=0x7f0aa4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 -- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 msgr2=0x7f0aa4196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 0x7f0aa4196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 -- 172.21.15.196:0/3958959766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa419ea30 con 0x7f0aa41026b0 2024-09-15T22:04:50.699 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aa9d18640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaa519640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 0x7f0aa4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aaad1a640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c007c30 tx=0x7f0a9c00e420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c019020 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c007e40 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa406a7f0 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c005590 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.682+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa406ad30 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.684+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa4100ef0 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.684+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f0a9c005730 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.685+0000 7f0a9b7fe640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 0x7f0a900a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.685+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f0a9c0cf830 con 0x7f0aa41026b0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.685+0000 7f0aaa519640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 0x7f0a900a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:50.700 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.686+0000 7f0aaa519640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 0x7f0a900a56a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a94006010 tx=0x7f0a94032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:50.701 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.689+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a9c023020 con 0x7f0aa41026b0 2024-09-15T22:04:50.862 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:04:50.596+0000 7fc2772f3640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T22:04:50.862 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:04:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:04:50.599+0000 7fc26f113640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:04:50.871 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:04:50.870+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0aa406af90 con 0x7f0a900a31e0 2024-09-15T22:04:51.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:51.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:04:51.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='osd.1 ' entity='osd.1' 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='osd.1 ' entity='osd.1' 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:04:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='osd.1 ' entity='osd.1' 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:51.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:52.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi117 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] boot 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:04:52.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi117 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] boot 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:04:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi117 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: osd.1 [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] boot 2024-09-15T22:04:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:04:52.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/4044708024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]': finished 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:04:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:04:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:04:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/4044708024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]': finished 2024-09-15T22:04:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:04:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/4044708024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]: dispatch 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e1d614a-34ca-45c9-b2c2-09d60c5c0821"}]': finished 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:04:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:04:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:54 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/4212674055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:54 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/4212674055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:54 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/4212674055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:55 smithi077 ceph-mon[30980]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:55 smithi196 ceph-mon[35109]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:55 smithi117 ceph-mon[35021]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:57 smithi077 ceph-mon[30980]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:57.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:57 smithi196 ceph-mon[35109]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:57 smithi117 ceph-mon[35021]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:59.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:04:59 smithi196 ceph-mon[35109]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:04:59 smithi077 ceph-mon[30980]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:04:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:04:59 smithi117 ceph-mon[35021]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:01 smithi077 ceph-mon[30980]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:05:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:01.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:01 smithi196 ceph-mon[35109]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:01.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:05:01.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:01 smithi117 ceph-mon[35021]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:05:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:02.506 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:02 smithi196 ceph-mon[35109]: Deploying daemon osd.2 on smithi196 2024-09-15T22:05:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:02 smithi077 ceph-mon[30980]: Deploying daemon osd.2 on smithi196 2024-09-15T22:05:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:02 smithi117 ceph-mon[35021]: Deploying daemon osd.2 on smithi196 2024-09-15T22:05:03.560 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:03 smithi196 ceph-mon[35109]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:03.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:03 smithi077 ceph-mon[30980]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:03 smithi117 ceph-mon[35021]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:05 smithi077 ceph-mon[30980]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:05 smithi196 ceph-mon[35109]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:05 smithi117 ceph-mon[35021]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:07 smithi077 ceph-mon[30980]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:07 smithi196 ceph-mon[35109]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:07 smithi117 ceph-mon[35021]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:09.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:09 smithi196 ceph-mon[35109]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:09 smithi077 ceph-mon[30980]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:09 smithi117 ceph-mon[35021]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:10 smithi077 ceph-mon[30980]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:10 smithi196 ceph-mon[35109]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:10 smithi117 ceph-mon[35021]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:11.235 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-09-15T22:05:11.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.233+0000 7f0a9b7fe640 1 -- 172.21.15.196:0/3958959766 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0aa406af90 con 0x7f0a900a31e0 2024-09-15T22:05:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 msgr2=0x7f0a900a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 0x7f0a900a56a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a94006010 tx=0x7f0a94032dd0 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 msgr2=0x7f0aa419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c007c30 tx=0x7f0a9c00e420 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 shutdown_connections 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0a900a31e0 0x7f0a900a56a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0aa41026b0 0x7f0aa419e3c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0aa4101cf0 0x7f0aa4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 --2- 172.21.15.196:0/3958959766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa4100af0 0x7f0aa4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 >> 172.21.15.196:0/3958959766 conn(0x7f0aa40fc280 msgr2=0x7f0aa4105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 shutdown_connections 2024-09-15T22:05:11.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:05:11.236+0000 7f0aac77d640 1 -- 172.21.15.196:0/3958959766 wait complete. 2024-09-15T22:05:11.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:11.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:11.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:11.800 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.2.service 2024-09-15T22:05:11.802 INFO:tasks.cephadm:Deploying osd.3 on smithi077 with /dev/vg_nvme/lv_3... 2024-09-15T22:05:11.803 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:05:11.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:11.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:11.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:11.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:11.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:12.040 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:12.763 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:12 smithi117 ceph-mon[35021]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:12.786 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:12 smithi196 ceph-mon[35109]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:12 smithi077 ceph-mon[30980]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:13.354 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:05:13.211+0000 7f1e96884640 -1 osd.2 0 log_to_monitors true 2024-09-15T22:05:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:13 smithi077 ceph-mon[30980]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:13 smithi077 ceph-mon[30980]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:13.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:13 smithi196 ceph-mon[35109]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:13.748 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:13 smithi196 ceph-mon[35109]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:13.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:13 smithi117 ceph-mon[35021]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:13.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:13 smithi117 ceph-mon[35021]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:05:14.517 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:05:14.544 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-09-15T22:05:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:05:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:05:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:05:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:14.689 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.690 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.840 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:05:14.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:05:14.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:14.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:05:14.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:14.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:14.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:14.941 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:05:14.915+0000 7f1e92807640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T22:05:14.941 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:05:14.920+0000 7f1e8a627640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 -- 172.21.15.77:0/583627080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 msgr2=0x7fedc4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 --2- 172.21.15.77:0/583627080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4100f10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009900 tx=0x7fedb402a410 comp rx=0 tx=0).stop 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 -- 172.21.15.77:0/583627080 shutdown_connections 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 --2- 172.21.15.77:0/583627080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 0x7fedc4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 --2- 172.21.15.77:0/583627080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 0x7fedc4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 --2- 172.21.15.77:0/583627080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4100f10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 -- 172.21.15.77:0/583627080 >> 172.21.15.77:0/583627080 conn(0x7fedc40fc2a0 msgr2=0x7fedc40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:15.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 -- 172.21.15.77:0/583627080 shutdown_connections 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.352+0000 7fedc9984640 1 -- 172.21.15.77:0/583627080 wait complete. 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 Processor -- start 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 -- start start 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 0x7fedc4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 0x7fedc419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410c810 con 0x7fedc4100b10 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410c690 con 0x7fedc4101d10 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc9984640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410c990 con 0x7fedc41026d0 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38250/0 (socket says 172.21.15.77:38250) 2024-09-15T22:05:15.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 -- 172.21.15.77:0/4890354 learned_addr learned my addr 172.21.15.77:0/4890354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc37fe640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 0x7fedc419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc27fc640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 0x7fedc4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 -- 172.21.15.77:0/4890354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 msgr2=0x7fedc419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 0x7fedc419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 -- 172.21.15.77:0/4890354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 msgr2=0x7fedc4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 0x7fedc4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.353+0000 7fedc2ffd640 1 -- 172.21.15.77:0/4890354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc419fbf0 con 0x7fedc4100b10 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc2ffd640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedb4031610 tx=0x7fedb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb403f030 con 0x7fedc4100b10 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc419fe80 con 0x7fedc4100b10 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedb4003ad0 con 0x7fedc4100b10 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4035070 con 0x7fedc4100b10 2024-09-15T22:05:15.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.354+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc41a0300 con 0x7fedc4100b10 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.355+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc41a0530 con 0x7fedc4100b10 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.356+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fedb4042030 con 0x7fedc4100b10 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.357+0000 7fedc8982640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 0x7fedac0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.357+0000 7fedc27fc640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 0x7fedac0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.357+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fedb40ee940 con 0x7fedc4100b10 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.357+0000 7fedc27fc640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 0x7fedac0a56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedc41994e0 tx=0x7fedb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:15.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.361+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedb40bcdb0 con 0x7fedc4100b10 2024-09-15T22:05:15.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:15.543+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fedc406aef0 con 0x7fedac0a31e0 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi196 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: from='osd.2 ' entity='osd.2' 2024-09-15T22:05:15.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:15 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi196 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: from='osd.2 ' entity='osd.2' 2024-09-15T22:05:15.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:15 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi196 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: from='osd.2 ' entity='osd.2' 2024-09-15T22:05:15.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:15 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:05:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:05:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] boot 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:16.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:16 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] boot 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:16 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:05:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:05:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: osd.2 [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] boot 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:05:16.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:16 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:18.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:05:18.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:18.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]: dispatch 2024-09-15T22:05:18.177 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]': finished 2024-09-15T22:05:18.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:05:18.178 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]: dispatch 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]': finished 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:05:18.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:05:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:05:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]: dispatch 2024-09-15T22:05:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:05:18.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2215853464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42be2980-1ab3-42e7-9256-eb759230d6e4"}]': finished 2024-09-15T22:05:18.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:05:18.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:18.506 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:18.507 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:18.507 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49150]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:18.507 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49150]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:18.831 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:18.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:18.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49156]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:18.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:05:18 smithi077 sudo[49156]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:18.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:18.935+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fedb4003c80 con 0x7fedc4100b10 2024-09-15T22:05:19.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:18 smithi117 ceph-mon[35021]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:19.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:18 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/337050699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:19.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:19.074 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:18 smithi117 sudo[43194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:19.074 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:18 smithi117 sudo[43194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:19.074 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:18 smithi117 sudo[43194]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:19.075 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:18 smithi117 sudo[43194]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:18 smithi077 ceph-mon[30980]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:18 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/337050699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:19.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:18 smithi196 ceph-mon[35109]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:19.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:18 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/337050699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:19.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:19.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:19 smithi117 sudo[43197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:19.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:19 smithi117 sudo[43197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:19.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:19 smithi117 sudo[43197]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:19.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:05:19 smithi117 sudo[43197]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:19.599 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43884]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:19.599 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:19.599 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43884]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:19.599 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43884]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:19.853 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43887]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:19.854 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43887]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:19.854 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43887]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:19.854 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:05:19 smithi196 sudo[43887]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:20.124 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:19 smithi077 sudo[49393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:20.124 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:19 smithi077 sudo[49393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:20.124 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:19 smithi077 sudo[49393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:20.125 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:19 smithi077 sudo[49393]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:20.385 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 ceph-mon[35109]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:05:20.385 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 ceph-mon[35109]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43890]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:20.386 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43890]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 sudo[49579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 sudo[49579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 sudo[49579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 sudo[49579]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 ceph-mon[30980]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 ceph-mon[30980]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 ceph-mon[35021]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:05:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 ceph-mon[35021]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:05:20.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43893]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43893]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43893]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:20.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:20 smithi196 sudo[43893]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:21.122 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 sudo[43200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:05:21.122 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 sudo[43200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:21.122 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 sudo[43200]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:21.122 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:20 smithi117 sudo[43200]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:21 smithi077 ceph-mon[30980]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:21 smithi196 ceph-mon[35109]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 sudo[43203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:05:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 sudo[43203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:05:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 sudo[43203]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:05:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 sudo[43203]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:05:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:05:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:05:21.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:21 smithi117 ceph-mon[35021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:05:23.410 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:23 smithi077 ceph-mon[30980]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:23.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:23 smithi117 ceph-mon[35021]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:23.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:23 smithi196 ceph-mon[35109]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:25 smithi077 ceph-mon[30980]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:25.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:25 smithi196 ceph-mon[35109]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:25 smithi117 ceph-mon[35021]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:27 smithi077 ceph-mon[30980]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:05:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:27.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:27 smithi077 ceph-mon[30980]: Deploying daemon osd.3 on smithi077 2024-09-15T22:05:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:27 smithi196 ceph-mon[35109]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:05:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:27 smithi196 ceph-mon[35109]: Deploying daemon osd.3 on smithi077 2024-09-15T22:05:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:27 smithi117 ceph-mon[35021]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:05:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:27 smithi117 ceph-mon[35021]: Deploying daemon osd.3 on smithi077 2024-09-15T22:05:29.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:29 smithi077 ceph-mon[30980]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:29.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:29 smithi196 ceph-mon[35109]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:29.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:29 smithi117 ceph-mon[35021]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:31.248 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:31 smithi077 ceph-mon[30980]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:31 smithi196 ceph-mon[35109]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:31 smithi117 ceph-mon[35021]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:33.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:33 smithi077 ceph-mon[30980]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:33.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:33.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:33.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:33 smithi196 ceph-mon[35109]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:33.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:33.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:33.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:33 smithi117 ceph-mon[35021]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:35 smithi077 ceph-mon[30980]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:35.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:35 smithi196 ceph-mon[35109]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:35 smithi117 ceph-mon[35021]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:35.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.985+0000 7fedc8982640 1 -- 172.21.15.77:0/4890354 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fedc406aef0 con 0x7fedac0a31e0 2024-09-15T22:05:35.987 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 3 on host 'smithi077' 2024-09-15T22:05:35.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 msgr2=0x7fedac0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:35.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 0x7fedac0a56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedc41994e0 tx=0x7fedb8008040 comp rx=0 tx=0).stop 2024-09-15T22:05:35.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 msgr2=0x7fedc4197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedb4031610 tx=0x7fedb4004270 comp rx=0 tx=0).stop 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 shutdown_connections 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fedac0a31e0 0x7fedac0a56a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedc41026d0 0x7fedc419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedc4101d10 0x7fedc4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 --2- 172.21.15.77:0/4890354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedc4100b10 0x7fedc4197fc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 >> 172.21.15.77:0/4890354 conn(0x7fedc40fc2a0 msgr2=0x7fedc406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 shutdown_connections 2024-09-15T22:05:35.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:05:35.987+0000 7fedc9984640 1 -- 172.21.15.77:0/4890354 wait complete. 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:36 smithi077 ceph-mon[30980]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:36.612 DEBUG:teuthology.orchestra.run.smithi077:osd.3> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.3.service 2024-09-15T22:05:36.615 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-09-15T22:05:36.615 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:05:36.643 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.643 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.643 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:36.643 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:36.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:36.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:36 smithi117 ceph-mon[35021]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:36.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:36 smithi196 ceph-mon[35109]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:36.866 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:05:38.158 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:38 smithi077 ceph-mon[30980]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:05:38.158 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:05:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:05:38.025+0000 7f2159977640 -1 osd.3 0 log_to_monitors true 2024-09-15T22:05:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:38 smithi196 ceph-mon[35109]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:05:38.360 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:38 smithi117 ceph-mon[35021]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:05:39.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:05:39.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:05:39.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:05:39.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:39.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:39.287 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-mon[30980]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:05:39.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:05:39.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:05:39.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:05:39.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:39.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:39.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:39 smithi196 ceph-mon[35109]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:05:39.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:05:39.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:05:39.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:05:39.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:39.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:39.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:39 smithi117 ceph-mon[35021]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:05:39.501 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:05:39.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-09-15T22:05:39.581 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:05:39.479+0000 7f215610e640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T22:05:39.582 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:05:39 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:05:39.484+0000 7f214d71a640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:05:39.819 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 -- 172.21.15.117:0/3376477699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 msgr2=0x7f9bdc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3376477699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc108f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009ca0 tx=0x7f9bd402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 -- 172.21.15.117:0/3376477699 shutdown_connections 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3376477699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc108f20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3376477699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3376477699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 -- 172.21.15.117:0/3376477699 >> 172.21.15.117:0/3376477699 conn(0x7f9bdc0fc280 msgr2=0x7f9bdc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 -- 172.21.15.117:0/3376477699 shutdown_connections 2024-09-15T22:05:40.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.292+0000 7f9be1c31640 1 -- 172.21.15.117:0/3376477699 wait complete. 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 Processor -- start 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 -- start start 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10c7b0 con 0x7f9bdc101cd0 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10c630 con 0x7f9bdc102690 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9be1c31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10c930 con 0x7f9bdc100ad0 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9bdbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41926/0 (socket says 172.21.15.117:41926) 2024-09-15T22:05:40.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9bdbfff640 1 -- 172.21.15.117:0/3323347487 learned_addr learned my addr 172.21.15.117:0/3323347487 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9bdaffd640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.293+0000 7f9bdb7fe640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 -- 172.21.15.117:0/3323347487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc100ad0 msgr2=0x7f9bdc06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 -- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 msgr2=0x7f9bdc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 -- 172.21.15.117:0/3323347487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc19fb90 con 0x7f9bdc102690 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdaffd640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bdbfff640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005e30 tx=0x7f9bd4031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4003b70 con 0x7f9bdc102690 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd4003d10 con 0x7f9bdc102690 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc19fdc0 con 0x7f9bdc102690 2024-09-15T22:05:40.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd402fd30 con 0x7f9bdc102690 2024-09-15T22:05:40.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.294+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1a0320 con 0x7f9bdc102690 2024-09-15T22:05:40.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.295+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bdc107ea0 con 0x7f9bdc102690 2024-09-15T22:05:40.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.296+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9bd4004050 con 0x7f9bdc102690 2024-09-15T22:05:40.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.297+0000 7f9bd8ff9640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 0x7f9bc40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:40.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.297+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7f9bd40efa00 con 0x7f9bdc102690 2024-09-15T22:05:40.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.297+0000 7f9bdb7fe640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 0x7f9bc40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:40.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.298+0000 7f9bdb7fe640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 0x7f9bc40a5780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc005fd0 tx=0x7f9bcc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:40.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.301+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bd40bd920 con 0x7f9bdc102690 2024-09-15T22:05:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:05:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:05:40.480+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9bdc062630 con 0x7f9bc40a32c0 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217]' entity='osd.3' 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:40.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:40.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:40.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:40.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi077 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] boot 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:05:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:05:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi077 2024-09-15T22:05:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-15T22:05:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] boot 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:41.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:05:41.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:05:41.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi077 2024-09-15T22:05:41.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 8544M 2024-09-15T22:05:41.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: osd.3 [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] boot 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:41.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:42.226 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:42 smithi117 ceph-mon[35021]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:42.226 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:42 smithi117 ceph-mon[35021]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:05:42.226 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:42 smithi117 ceph-mon[35021]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:05:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:42 smithi077 ceph-mon[30980]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:42 smithi077 ceph-mon[30980]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:05:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:42 smithi077 ceph-mon[30980]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:05:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:42 smithi196 ceph-mon[35109]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:42.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:42 smithi196 ceph-mon[35109]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:05:42.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:42 smithi196 ceph-mon[35109]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/3449884103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]': finished 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:05:43.355 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:43.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:05:43.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/3449884103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]': finished 2024-09-15T22:05:43.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:05:43.529 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:05:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/3449884103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]: dispatch 2024-09-15T22:05:43.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7db7bf42-1fed-4197-9bad-58e8a56f816a"}]': finished 2024-09-15T22:05:43.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:05:43.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:44 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/2684506198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:44 smithi117 ceph-mon[35021]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:05:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:44 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:44.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:44 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/2684506198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:44.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:44 smithi196 ceph-mon[35109]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:05:44.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:44 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:44 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/2684506198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:44 smithi077 ceph-mon[30980]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:05:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:44 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:45 smithi077 ceph-mon[30980]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:45 smithi077 ceph-mon[30980]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:05:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:45 smithi196 ceph-mon[35109]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:45 smithi196 ceph-mon[35109]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:05:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:45 smithi117 ceph-mon[35021]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:45 smithi117 ceph-mon[35021]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:05:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:05:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:46 smithi077 ceph-mon[30980]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-15T22:05:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:46 smithi196 ceph-mon[35109]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-15T22:05:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:46 smithi117 ceph-mon[35021]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-15T22:05:47.399 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:47 smithi117 ceph-mon[35021]: pgmap v100: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:47 smithi077 ceph-mon[30980]: pgmap v100: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:47.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:47 smithi196 ceph-mon[35109]: pgmap v100: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-09-15T22:05:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:48 smithi117 ceph-mon[35021]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-15T22:05:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:48 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:05:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:48 smithi077 ceph-mon[30980]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-15T22:05:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:48 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:05:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:48 smithi196 ceph-mon[35109]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-15T22:05:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:48 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:05:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:49 smithi077 ceph-mon[30980]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:49 smithi196 ceph-mon[35109]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:49 smithi117 ceph-mon[35021]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:51 smithi077 ceph-mon[30980]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T22:05:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:05:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:51 smithi196 ceph-mon[35109]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T22:05:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:05:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:51.609 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:51 smithi117 ceph-mon[35021]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-15T22:05:51.609 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:05:51.609 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:52 smithi077 ceph-mon[30980]: Deploying daemon osd.4 on smithi117 2024-09-15T22:05:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:52 smithi196 ceph-mon[35109]: Deploying daemon osd.4 on smithi117 2024-09-15T22:05:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:52 smithi117 ceph-mon[35021]: Deploying daemon osd.4 on smithi117 2024-09-15T22:05:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:53 smithi077 ceph-mon[30980]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:05:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:53 smithi196 ceph-mon[35109]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:05:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:53 smithi117 ceph-mon[35021]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:05:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:55 smithi077 ceph-mon[30980]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:55 smithi196 ceph-mon[35109]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:55 smithi117 ceph-mon[35021]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:57 smithi077 ceph-mon[30980]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-15T22:05:57.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:57 smithi196 ceph-mon[35109]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-15T22:05:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:57 smithi117 ceph-mon[35021]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-15T22:05:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:05:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:05:59 smithi077 ceph-mon[30980]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:05:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:05:59 smithi196 ceph-mon[35109]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:05:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:05:59 smithi117 ceph-mon[35021]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:06:01.311 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:01.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:01.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:01.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:01 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:01 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:01 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:01.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.642+0000 7f9bd8ff9640 1 -- 172.21.15.117:0/3323347487 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9bdc062630 con 0x7f9bc40a32c0 2024-09-15T22:06:01.645 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-09-15T22:06:01.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 msgr2=0x7f9bc40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 0x7f9bc40a5780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc005fd0 tx=0x7f9bcc00b410 comp rx=0 tx=0).stop 2024-09-15T22:06:01.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 msgr2=0x7f9bdc19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005e30 tx=0x7f9bd4031ee0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 shutdown_connections 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9bc40a32c0 0x7f9bc40a5780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bdc102690 0x7f9bdc19f280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bdc101cd0 0x7f9bdc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 --2- 172.21.15.117:0/3323347487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bdc100ad0 0x7f9bdc06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 >> 172.21.15.117:0/3323347487 conn(0x7f9bdc0fc280 msgr2=0x7f9bdc105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 shutdown_connections 2024-09-15T22:06:01.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:01.645+0000 7f9be1c31640 1 -- 172.21.15.117:0/3323347487 wait complete. 2024-09-15T22:06:02.324 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:02 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:02.324 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:02 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:02 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.351 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.4.service 2024-09-15T22:06:02.353 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-09-15T22:06:02.353 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:06:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:02 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:02 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:02 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:02 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:02 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:02 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:02.606 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:06:03.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:03 smithi196 ceph-mon[35109]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:03.506 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:03 smithi117 ceph-mon[35021]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:03 smithi077 ceph-mon[30980]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:04.112 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:06:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:06:03.763+0000 7f32a61b4640 -1 osd.4 0 log_to_monitors true 2024-09-15T22:06:04.398 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:04 smithi196 ceph-mon[35109]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:04.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:04 smithi196 ceph-mon[35109]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:04.506 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:04 smithi117 ceph-mon[35021]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:04.506 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:04 smithi117 ceph-mon[35021]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:04 smithi077 ceph-mon[30980]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:04 smithi077 ceph-mon[30980]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:05.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:05 smithi196 ceph-mon[35109]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.221 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:06:05.247 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-09-15T22:06:05.501 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:05 smithi077 ceph-mon[30980]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.612 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:06:05.153+0000 7f32a2137640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T22:06:05.613 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:06:05.162+0000 7f329df5f640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: from='osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:05 smithi117 ceph-mon[35021]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/3730448032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4102690 msgr2=0x7fe1b4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/3730448032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4102690 0x7fe1b4108f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009ca0 tx=0x7fe1ac02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/3730448032 shutdown_connections 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/3730448032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4102690 0x7fe1b4108f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/3730448032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4101cd0 0x7fe1b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/3730448032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/3730448032 >> 172.21.15.196:0/3730448032 conn(0x7fe1b40fc280 msgr2=0x7fe1b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/3730448032 shutdown_connections 2024-09-15T22:06:05.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/3730448032 wait complete. 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 Processor -- start 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.939+0000 7fe1ba3ae640 1 -- start start 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4101cd0 0x7fe1b41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4102690 0x7fe1b419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410c750 con 0x7fe1b4102690 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410c5d0 con 0x7fe1b4101cd0 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1ba3ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410c8d0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:45406/0 (socket says 172.21.15.196:45406) 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 -- 172.21.15.196:0/120614593 learned_addr learned my addr 172.21.15.196:0/120614593 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b37fe640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4101cd0 0x7fe1b41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:05.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b894b640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4102690 0x7fe1b419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 -- 172.21.15.196:0/120614593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4101cd0 msgr2=0x7fe1b41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4101cd0 0x7fe1b41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 -- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4102690 msgr2=0x7fe1b419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4102690 0x7fe1b419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 -- 172.21.15.196:0/120614593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b4069ff0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b3fff640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9c0 tx=0x7fe1a800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.940+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800cd80 con 0x7fe1b4100ad0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.941+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b406a2e0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.941+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a80035f0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.941+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8010690 con 0x7fe1b4100ad0 2024-09-15T22:06:05.950 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.941+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b406a820 con 0x7fe1b4100ad0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.942+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b4107e20 con 0x7fe1b4100ad0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.943+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe1a80037a0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.944+0000 7fe1b17fa640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 0x7fe19c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.944+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7fe1a80ceaf0 con 0x7fe1b4100ad0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.944+0000 7fe1b37fe640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 0x7fe19c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.945+0000 7fe1b37fe640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 0x7fe19c0a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400bbe0 tx=0x7fe1a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:05.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:05.949+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1a809c980 con 0x7fe1b4100ad0 2024-09-15T22:06:06.131 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:06.130+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe1b406ab50 con 0x7fe19c0a3240 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi117 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.514 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:06.515 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:06 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi117 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:06.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:06 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi117 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:06.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:06.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:06.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:06.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:06.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:06 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:07.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:06:07.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:06:07.420 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:07.421 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:07.421 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] boot 2024-09-15T22:06:07.421 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:06:07.421 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] boot 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:06:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:06:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:06:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:06:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: osd.4 [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] boot 2024-09-15T22:06:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:06:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:06:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:08 smithi077 ceph-mon[30980]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:06:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:08 smithi196 ceph-mon[35109]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:06:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:08 smithi117 ceph-mon[35021]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/740523340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]': finished 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:06:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:09.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:09 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/177128805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/740523340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]': finished 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:09 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/177128805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/740523340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]: dispatch 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d897eb1d-0f68-4ff4-832b-42deabfc6556"}]': finished 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:06:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:09.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:09 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/177128805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:11 smithi077 ceph-mon[30980]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:11 smithi196 ceph-mon[35109]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:11.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:11 smithi117 ceph-mon[35021]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:13.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:13 smithi196 ceph-mon[35109]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:13 smithi077 ceph-mon[30980]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:13 smithi117 ceph-mon[35021]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:15 smithi077 ceph-mon[30980]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:15 smithi196 ceph-mon[35109]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:15 smithi117 ceph-mon[35021]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:17 smithi077 ceph-mon[30980]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:06:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:17 smithi196 ceph-mon[35109]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:06:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:17 smithi117 ceph-mon[35021]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:06:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:18.575 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:18 smithi196 ceph-mon[35109]: Deploying daemon osd.5 on smithi196 2024-09-15T22:06:18.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:18 smithi077 ceph-mon[30980]: Deploying daemon osd.5 on smithi196 2024-09-15T22:06:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:18 smithi117 ceph-mon[35021]: Deploying daemon osd.5 on smithi196 2024-09-15T22:06:19.577 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:19 smithi196 ceph-mon[35109]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:19.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:19 smithi077 ceph-mon[30980]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:19 smithi117 ceph-mon[35021]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:21 smithi077 ceph-mon[30980]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:21 smithi196 ceph-mon[35109]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:21 smithi117 ceph-mon[35021]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:23 smithi077 ceph-mon[30980]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:23 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:23 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:23 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:23.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:23 smithi196 ceph-mon[35109]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:23.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:23 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:23.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:23 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:23.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:23 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:23 smithi117 ceph-mon[35021]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:23 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:23 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:23 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:25 smithi077 ceph-mon[30980]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:25 smithi196 ceph-mon[35109]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:25 smithi117 ceph-mon[35021]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:26.957 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:26 smithi196 ceph-mon[35109]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:26.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.993+0000 7fe1b17fa640 1 -- 172.21.15.196:0/120614593 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1b406ab50 con 0x7fe19c0a3240 2024-09-15T22:06:26.996 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-15T22:06:26.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 msgr2=0x7fe19c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 0x7fe19c0a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400bbe0 tx=0x7fe1a400b390 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 msgr2=0x7fe1b4196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9c0 tx=0x7fe1a800ee80 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 shutdown_connections 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe19c0a3240 0x7fe19c0a5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1b4102690 0x7fe1b419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1b4101cd0 0x7fe1b41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 --2- 172.21.15.196:0/120614593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b4100ad0 0x7fe1b4196d90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 >> 172.21.15.196:0/120614593 conn(0x7fe1b40fc280 msgr2=0x7fe1b4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 shutdown_connections 2024-09-15T22:06:26.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:06:26.996+0000 7fe1ba3ae640 1 -- 172.21.15.196:0/120614593 wait complete. 2024-09-15T22:06:27.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:27.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:27.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:26 smithi077 ceph-mon[30980]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:27.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:27.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:27.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:26 smithi117 ceph-mon[35021]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:27.655 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.5.service 2024-09-15T22:06:27.657 INFO:tasks.cephadm:Deploying osd.6 on smithi077 with /dev/vg_nvme/lv_2... 2024-09-15T22:06:27.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:06:27.912 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:27.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:27.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:27.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:28.033 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:28.033 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:28.033 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:28.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:29.018 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:28 smithi196 ceph-mon[35109]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:29.018 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:06:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:06:28.801+0000 7f7f4196c640 -1 osd.5 0 log_to_monitors true 2024-09-15T22:06:29.035 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:28 smithi077 ceph-mon[30980]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:28 smithi117 ceph-mon[35021]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:29.980 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:29 smithi196 ceph-mon[35109]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:29.980 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:29 smithi196 ceph-mon[35109]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:29 smithi077 ceph-mon[30980]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:29 smithi077 ceph-mon[30980]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:29 smithi117 ceph-mon[35021]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:29 smithi117 ceph-mon[35021]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:06:30.472 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:06:30.498 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-09-15T22:06:30.744 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:06:30.744 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:06:30.744 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:30.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:31.104 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:06:30.739+0000 7f7f3d8ef640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T22:06:31.104 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:06:30.745+0000 7f7f39717640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:31.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:31.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:06:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:06:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:31.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- 172.21.15.77:0/2799029402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101e50 msgr2=0x7f59741086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 --2- 172.21.15.77:0/2799029402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101e50 0x7f59741086e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f596c00adb0 tx=0x7f596c02b7a0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- 172.21.15.77:0/2799029402 shutdown_connections 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 --2- 172.21.15.77:0/2799029402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101e50 0x7f59741086e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 --2- 172.21.15.77:0/2799029402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101490 0x7f5974101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 --2- 172.21.15.77:0/2799029402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 0x7f5974100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- 172.21.15.77:0/2799029402 >> 172.21.15.77:0/2799029402 conn(0x7f59740fba40 msgr2=0x7f59740fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- 172.21.15.77:0/2799029402 shutdown_connections 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- 172.21.15.77:0/2799029402 wait complete. 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 Processor -- start 2024-09-15T22:06:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.207+0000 7f5979d2a640 1 -- start start 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 0x7f5974197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101e50 0x7f597419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f597410b670 con 0x7f5974101490 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f597410b4f0 con 0x7f5974100290 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f597410b7f0 con 0x7f5974101e50 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44704/0 (socket says 172.21.15.77:44704) 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 -- 172.21.15.77:0/3178779864 learned_addr learned my addr 172.21.15.77:0/3178779864 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f59737fe640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 0x7f5974197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5973fff640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101e50 0x7f597419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 -- 172.21.15.77:0/3178779864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101e50 msgr2=0x7f597419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101e50 0x7f597419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 -- 172.21.15.77:0/3178779864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 msgr2=0x7f5974197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 0x7f5974197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 -- 172.21.15.77:0/3178779864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597419fab0 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f596affd640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f595c00d950 tx=0x7f595c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c017070 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c009c30 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597419fda0 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.208+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00f6b0 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.209+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741a0250 con 0x7f5974101490 2024-09-15T22:06:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.210+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59741a0480 con 0x7f5974101490 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.211+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f595c0109d0 con 0x7f5974101490 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.211+0000 7f59717fa640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 0x7f59600a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.212+0000 7f59737fe640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 0x7f59600a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.212+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f595c0ce530 con 0x7f5974101490 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.212+0000 7f59737fe640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 0x7f59600a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59741012f0 tx=0x7f5964005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.216+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f595c009de0 con 0x7f5974101490 2024-09-15T22:06:31.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:31.402+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5974108140 con 0x7f59600a3270 2024-09-15T22:06:32.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi196 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] boot 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:06:32.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:31 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi196 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] boot 2024-09-15T22:06:32.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:06:32.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:31 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi196 2024-09-15T22:06:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-15T22:06:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:06:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: osd.5 [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] boot 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:06:32.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:31 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:06:33.008 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:32 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:06:33.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:32 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:06:33.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:32 smithi077 ceph-mon[30980]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:33.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:32 smithi077 ceph-mon[30980]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:33.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:32 smithi077 ceph-mon[30980]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:06:33.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:32 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:06:33.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:32 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:06:33.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:32 smithi196 ceph-mon[35109]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:33.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:32 smithi196 ceph-mon[35109]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:33.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:32 smithi196 ceph-mon[35109]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:06:33.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:32 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:06:33.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:32 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:06:33.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:32 smithi117 ceph-mon[35021]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:33.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:32 smithi117 ceph-mon[35021]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:33.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:32 smithi117 ceph-mon[35021]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:06:35.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:06:35.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]: dispatch 2024-09-15T22:06:35.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]': finished 2024-09-15T22:06:35.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:06:35.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:35.084 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:35.084 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2068589892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]: dispatch 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]': finished 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:35.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2068589892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]: dispatch 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1769663334' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "916910cf-a42b-453d-be6b-c237a8305b43"}]': finished 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:35.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:35.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2068589892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:06:37.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:36 smithi077 ceph-mon[30980]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:06:37.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:37.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:36 smithi077 ceph-mon[30980]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:37.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:36 smithi196 ceph-mon[35109]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:06:37.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:37.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:36 smithi196 ceph-mon[35109]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:36 smithi117 ceph-mon[35021]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:06:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:36 smithi117 ceph-mon[35021]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:38.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:37 smithi077 ceph-mon[30980]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:06:38.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:38.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:37 smithi196 ceph-mon[35109]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:06:38.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:37 smithi117 ceph-mon[35021]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:06:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:38.985 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:38 smithi077 ceph-mon[30980]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-15T22:06:39.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:38 smithi196 ceph-mon[35109]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-15T22:06:39.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:38 smithi117 ceph-mon[35021]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-15T22:06:41.536 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:41 smithi077 ceph-mon[30980]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T22:06:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:41 smithi196 ceph-mon[35109]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T22:06:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:41 smithi117 ceph-mon[35021]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-15T22:06:42.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:06:42.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:06:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:42 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:06:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:42 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:43.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:43 smithi196 ceph-mon[35109]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:06:43.506 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:43 smithi196 ceph-mon[35109]: Deploying daemon osd.6 on smithi077 2024-09-15T22:06:43.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:43 smithi077 ceph-mon[30980]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:06:43.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:43 smithi077 ceph-mon[30980]: Deploying daemon osd.6 on smithi077 2024-09-15T22:06:43.513 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:43 smithi117 ceph-mon[35021]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:06:43.513 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:43 smithi117 ceph-mon[35021]: Deploying daemon osd.6 on smithi077 2024-09-15T22:06:45.460 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:45 smithi077 ceph-mon[30980]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:06:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:45 smithi196 ceph-mon[35109]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:06:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:45 smithi117 ceph-mon[35021]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:06:47.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:47 smithi077 ceph-mon[30980]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:06:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:47 smithi196 ceph-mon[35109]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:06:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:47 smithi117 ceph-mon[35021]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:06:48.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:48.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:48.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:49.465 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:49 smithi077 ceph-mon[30980]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:06:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:49 smithi196 ceph-mon[35109]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:06:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:49 smithi117 ceph-mon[35021]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.849+0000 7f59717fa640 1 -- 172.21.15.77:0/3178779864 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5974108140 con 0x7f59600a3270 2024-09-15T22:06:51.851 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 6 on host 'smithi077' 2024-09-15T22:06:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 msgr2=0x7f59600a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 0x7f59600a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59741012f0 tx=0x7f5964005f90 comp rx=0 tx=0).stop 2024-09-15T22:06:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 msgr2=0x7f59741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f595c00d950 tx=0x7f595c00de10 comp rx=0 tx=0).stop 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 shutdown_connections 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59600a3270 0x7f59600a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5974101e50 0x7f597419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5974101490 0x7f59741983c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 --2- 172.21.15.77:0/3178779864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5974100290 0x7f5974197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 >> 172.21.15.77:0/3178779864 conn(0x7f59740fba40 msgr2=0x7f5974106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 shutdown_connections 2024-09-15T22:06:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:06:51.852+0000 7f5979d2a640 1 -- 172.21.15.77:0/3178779864 wait complete. 2024-09-15T22:06:52.475 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.475 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.475 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:52 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.501 DEBUG:teuthology.orchestra.run.smithi077:osd.6> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.6.service 2024-09-15T22:06:52.503 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/vg_nvme/lv_2... 2024-09-15T22:06:52.504 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:06:52.532 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.532 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.532 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:52 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:52 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:06:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:53 smithi077 ceph-mon[30980]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:53 smithi196 ceph-mon[35109]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:53 smithi117 ceph-mon[35021]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:54.664 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:06:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:06:54.407+0000 7f712adf8640 -1 osd.6 0 log_to_monitors true 2024-09-15T22:06:55.288 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:55 smithi117 ceph-mon[35021]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:55.288 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:55 smithi117 ceph-mon[35021]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:06:55.291 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:06:55.307 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:55 smithi077 ceph-mon[30980]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:55.307 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:55 smithi077 ceph-mon[30980]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:06:55.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-09-15T22:06:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:55 smithi196 ceph-mon[35109]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:55 smithi196 ceph-mon[35109]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:06:55.616 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 -- 172.21.15.117:0/2826162023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec1037c0 msgr2=0x7fc8ec10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 --2- 172.21.15.117:0/2826162023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec10a050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009900 tx=0x7fc8e402a410 comp rx=0 tx=0).stop 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 -- 172.21.15.117:0/2826162023 shutdown_connections 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 --2- 172.21.15.117:0/2826162023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec10a050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 --2- 172.21.15.117:0/2826162023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec102e00 0x7fc8ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 --2- 172.21.15.117:0/2826162023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec101c00 0x7fc8ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 -- 172.21.15.117:0/2826162023 >> 172.21.15.117:0/2826162023 conn(0x7fc8ec0fd390 msgr2=0x7fc8ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 -- 172.21.15.117:0/2826162023 shutdown_connections 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 -- 172.21.15.117:0/2826162023 wait complete. 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.085+0000 7fc8f2147640 1 Processor -- start 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 -- start start 2024-09-15T22:06:56.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec102e00 0x7fc8ec1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10d860 con 0x7fc8ec102e00 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10d6e0 con 0x7fc8ec101c00 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8f2147640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec10d9e0 con 0x7fc8ec1037c0 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37032/0 (socket says 172.21.15.117:37032) 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 -- 172.21.15.117:0/876664437 learned_addr learned my addr 172.21.15.117:0/876664437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8ebfff640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eaffd640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec102e00 0x7fc8ec1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 -- 172.21.15.117:0/876664437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 msgr2=0x7fc8ec19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 -- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec102e00 msgr2=0x7fc8ec1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec102e00 0x7fc8ec1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 -- 172.21.15.117:0/876664437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec19fac0 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8ebfff640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.086+0000 7fc8eb7fe640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00ebc0 tx=0x7fc8dc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.087+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc003b70 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.087+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec19fdb0 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.087+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8dc003d10 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.087+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc019980 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.087+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a0210 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.088+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec108fb0 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.089+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc8dc019b20 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.090+0000 7fc8e8ff9640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 0x7fc8d80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.090+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4669+0+0 (secure 0 0 0) 0x7fc8dc0cf480 con 0x7fc8ec101c00 2024-09-15T22:06:56.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.090+0000 7fc8eaffd640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 0x7fc8d80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.091+0000 7fc8eaffd640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 0x7fc8d80a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4005fd0 tx=0x7fc8d4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:56.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.093+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8dc09d070 con 0x7fc8ec101c00 2024-09-15T22:06:56.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:06:56.276+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc8ec063020 con 0x7fc8d80a3270 2024-09-15T22:06:56.582 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:06:56.225+0000 7f7126d7b640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T22:06:56.582 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:06:56.234+0000 7f7122ba3640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:06:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:06:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:06:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:06:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi077 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:56 smithi077 ceph-mon[30980]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi077 2024-09-15T22:06:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:56 smithi196 ceph-mon[35109]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:06:56.607 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:06:56.607 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:06:56.607 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi077 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:56.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:06:56.609 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:56 smithi117 ceph-mon[35021]: from='osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: osdmap e44: 7 total, 6 up, 7 in 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:57.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:57.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-15T22:06:57.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: osdmap e44: 7 total, 6 up, 7 in 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 5696M 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: osdmap e44: 7 total, 6 up, 7 in 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:06:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:06:57.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:06:57.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:58 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:06:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:58 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:06:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:58 smithi117 ceph-mon[35021]: osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] boot 2024-09-15T22:06:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:58 smithi117 ceph-mon[35021]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:06:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:58 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:06:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:58 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:06:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:58 smithi077 ceph-mon[30980]: osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] boot 2024-09-15T22:06:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:58 smithi077 ceph-mon[30980]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:06:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:58 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:06:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:58 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:06:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:58 smithi196 ceph-mon[35109]: osd.6 [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] boot 2024-09-15T22:06:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:58 smithi196 ceph-mon[35109]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:06:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: osdmap e46: 7 total, 7 up, 7 in 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/4255464494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]': finished 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:06:59.494 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:06:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:06:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: osdmap e46: 7 total, 7 up, 7 in 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/4255464494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]': finished 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:06:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:06:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: osdmap e46: 7 total, 7 up, 7 in 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/4255464494' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]: dispatch 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4aa2ec-81c6-4365-82ec-6b470840358a"}]': finished 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:06:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:06:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/3422645003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/3422645003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/3422645003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:01 smithi077 ceph-mon[30980]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:01 smithi196 ceph-mon[35109]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:01 smithi117 ceph-mon[35021]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:03.506 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:03 smithi117 ceph-mon[35021]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:03 smithi077 ceph-mon[30980]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:03 smithi196 ceph-mon[35109]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:05.353 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:05 smithi117 ceph-mon[35021]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:05 smithi077 ceph-mon[30980]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:05 smithi196 ceph-mon[35109]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:07 smithi077 ceph-mon[30980]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:07 smithi196 ceph-mon[35109]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:07 smithi117 ceph-mon[35021]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:08 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:07:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:08 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:08 smithi117 ceph-mon[35021]: Deploying daemon osd.7 on smithi117 2024-09-15T22:07:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:08 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:07:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:08 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:08 smithi077 ceph-mon[30980]: Deploying daemon osd.7 on smithi117 2024-09-15T22:07:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:08 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:07:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:08 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:08 smithi196 ceph-mon[35109]: Deploying daemon osd.7 on smithi117 2024-09-15T22:07:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:09 smithi077 ceph-mon[30980]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:09.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:09 smithi196 ceph-mon[35109]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:09.610 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:09 smithi117 ceph-mon[35021]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:11 smithi077 ceph-mon[30980]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:11 smithi196 ceph-mon[35109]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:11.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:11 smithi117 ceph-mon[35021]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:13.332 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:13 smithi117 ceph-mon[35021]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:13 smithi077 ceph-mon[30980]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:13.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:13 smithi196 ceph-mon[35109]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:14.360 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:14.360 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:14.361 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:15.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:15 smithi117 ceph-mon[35021]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:15 smithi077 ceph-mon[30980]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:15 smithi196 ceph-mon[35109]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:17.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.168+0000 7fc8e8ff9640 1 -- 172.21.15.117:0/876664437 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc8ec063020 con 0x7fc8d80a3270 2024-09-15T22:07:17.170 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2024-09-15T22:07:17.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 msgr2=0x7fc8d80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 0x7fc8d80a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4005fd0 tx=0x7fc8d4005850 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 msgr2=0x7fc8ec197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00ebc0 tx=0x7fc8dc00c6a0 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 shutdown_connections 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc8d80a3270 0x7fc8d80a5730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8ec1037c0 0x7fc8ec19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8ec102e00 0x7fc8ec1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 --2- 172.21.15.117:0/876664437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8ec101c00 0x7fc8ec197e90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 >> 172.21.15.117:0/876664437 conn(0x7fc8ec0fd390 msgr2=0x7fc8ec106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 shutdown_connections 2024-09-15T22:07:17.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:07:17.170+0000 7fc8f2147640 1 -- 172.21.15.117:0/876664437 wait complete. 2024-09-15T22:07:17.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:17.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:17.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:17.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:17.842 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.7.service 2024-09-15T22:07:17.845 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-09-15T22:07:17.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:07:18.098 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:07:18.444 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:18.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:18.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:18.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:18.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:19.028 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:07:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:07:18.955+0000 7f0eb76c1640 -1 osd.7 0 log_to_monitors true 2024-09-15T22:07:19.303 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:19 smithi117 ceph-mon[35021]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:19.303 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:19 smithi117 ceph-mon[35021]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:19.303 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:19 smithi117 ceph-mon[35021]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:19.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:19 smithi196 ceph-mon[35109]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:19.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:19 smithi196 ceph-mon[35109]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:19.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:19 smithi196 ceph-mon[35109]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:19 smithi077 ceph-mon[30980]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:19 smithi077 ceph-mon[30980]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:19 smithi077 ceph-mon[30980]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:07:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-mon[35021]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:07:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-mon[35021]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-mon[35021]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:07:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-mon[35021]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.576 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:20 smithi196 ceph-mon[35109]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:07:20.577 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:20 smithi196 ceph-mon[35109]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.577 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:20 smithi196 ceph-mon[35109]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:07:20.577 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:20.577 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:20 smithi196 ceph-mon[35109]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:20 smithi077 ceph-mon[30980]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:07:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:20 smithi077 ceph-mon[30980]: from='osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:20 smithi077 ceph-mon[30980]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:07:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:20 smithi077 ceph-mon[30980]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:07:20.713 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:07:20.564+0000 7f0eb3644640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T22:07:20.713 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:07:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:07:20.573+0000 7f0eab464640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:07:20.945 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:07:20.971 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-09-15T22:07:21.222 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: osdmap e49: 8 total, 7 up, 8 in 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='osd.7 ' entity='osd.7' 2024-09-15T22:07:21.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:21.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:21 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:07:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: osdmap e49: 8 total, 7 up, 8 in 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='osd.7 ' entity='osd.7' 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:21.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:21.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:21 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:07:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: osdmap e49: 8 total, 7 up, 8 in 2024-09-15T22:07:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='osd.7 ' entity='osd.7' 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:21.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:21 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:21.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 -- 172.21.15.196:0/1204188125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb320104fc0 msgr2=0x7fb3201073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 --2- 172.21.15.196:0/1204188125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb320104fc0 0x7fb3201073b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb310009900 tx=0x7fb31002a410 comp rx=0 tx=0).stop 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 -- 172.21.15.196:0/1204188125 shutdown_connections 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 --2- 172.21.15.196:0/1204188125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3201078f0 0x7fb320109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 --2- 172.21.15.196:0/1204188125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb320104fc0 0x7fb3201073b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 --2- 172.21.15.196:0/1204188125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320102690 0x7fb320104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 -- 172.21.15.196:0/1204188125 >> 172.21.15.196:0/1204188125 conn(0x7fb3200fc290 msgr2=0x7fb3200fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 -- 172.21.15.196:0/1204188125 shutdown_connections 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.719+0000 7fb32830a640 1 -- 172.21.15.196:0/1204188125 wait complete. 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 Processor -- start 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 -- start start 2024-09-15T22:07:21.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb320102690 0x7fb320101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320104fc0 0x7fb320100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010d890 con 0x7fb320102690 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010d710 con 0x7fb320104fc0 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb32830a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb32010da10 con 0x7fb3201078f0 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51596/0 (socket says 172.21.15.196:51596) 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 -- 172.21.15.196:0/1327155054 learned_addr learned my addr 172.21.15.196:0/1327155054 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3258a5640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320104fc0 0x7fb320100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3260a6640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb320102690 0x7fb320101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 -- 172.21.15.196:0/1327155054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320104fc0 msgr2=0x7fb320100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320104fc0 0x7fb320100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 -- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb320102690 msgr2=0x7fb320101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb320102690 0x7fb320101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.720+0000 7fb3268a7640 1 -- 172.21.15.196:0/1327155054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb320101440 con 0x7fb3201078f0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb3268a7640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cce0 tx=0x7fb31800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800f450 con 0x7fb3201078f0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31800fad0 con 0x7fb3201078f0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3201a20e0 con 0x7fb3201078f0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318018c10 con 0x7fb3201078f0 2024-09-15T22:07:21.725 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.721+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201a25a0 con 0x7fb3201078f0 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.722+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3201053a0 con 0x7fb3201078f0 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.723+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb31800f5f0 con 0x7fb3201078f0 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.723+0000 7fb3177fe640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 0x7fb30c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.724+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5430+0+0 (secure 0 0 0) 0x7fb3180cf4f0 con 0x7fb3201078f0 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.724+0000 7fb3260a6640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 0x7fb30c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.724+0000 7fb3260a6640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 0x7fb30c0a5700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005fd0 tx=0x7fb31c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:21.728 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.727+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb318014030 con 0x7fb3201078f0 2024-09-15T22:07:21.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:21.901+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb320062630 con 0x7fb30c0a3240 2024-09-15T22:07:22.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi117 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] boot 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:22.532 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:22 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi117 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] boot 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:22.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:22 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:07:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:07:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi117 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: osd.7 [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] boot 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:22.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:22 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:23.302 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:23 smithi196 ceph-mon[35109]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:23.302 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:23 smithi196 ceph-mon[35109]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:23.302 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:23 smithi196 ceph-mon[35109]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:07:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:23 smithi077 ceph-mon[30980]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:23 smithi077 ceph-mon[30980]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:23 smithi077 ceph-mon[30980]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:07:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:23 smithi117 ceph-mon[35021]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:23 smithi117 ceph-mon[35021]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:07:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:23 smithi117 ceph-mon[35021]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:07:24.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:24 smithi196 ceph-mon[35109]: osdmap e52: 8 total, 8 up, 8 in 2024-09-15T22:07:24.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:24 smithi196 ceph-mon[35109]: osdmap e53: 8 total, 8 up, 8 in 2024-09-15T22:07:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:24 smithi077 ceph-mon[30980]: osdmap e52: 8 total, 8 up, 8 in 2024-09-15T22:07:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:24 smithi077 ceph-mon[30980]: osdmap e53: 8 total, 8 up, 8 in 2024-09-15T22:07:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:24 smithi117 ceph-mon[35021]: osdmap e52: 8 total, 8 up, 8 in 2024-09-15T22:07:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:24 smithi117 ceph-mon[35021]: osdmap e53: 8 total, 8 up, 8 in 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/991292234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]': finished 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/957996779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/991292234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]': finished 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/957996779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/991292234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]: dispatch 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dc4187f-569d-4da4-ba2e-3c708dd5b99c"}]': finished 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/957996779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:27 smithi077 ceph-mon[30980]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:27 smithi196 ceph-mon[35109]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:27 smithi117 ceph-mon[35021]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:29.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:29 smithi196 ceph-mon[35109]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:29.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:29 smithi077 ceph-mon[30980]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:29.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:29 smithi117 ceph-mon[35021]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:31.500 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:31 smithi196 ceph-mon[35109]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:31.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:31 smithi077 ceph-mon[30980]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:31 smithi117 ceph-mon[35021]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:32.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:07:32.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:07:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:07:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:33.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:33 smithi077 ceph-mon[30980]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:33.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:33 smithi077 ceph-mon[30980]: Deploying daemon osd.8 on smithi196 2024-09-15T22:07:33.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:33 smithi196 ceph-mon[35109]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:33.589 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:33 smithi196 ceph-mon[35109]: Deploying daemon osd.8 on smithi196 2024-09-15T22:07:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:33 smithi117 ceph-mon[35021]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:33 smithi117 ceph-mon[35021]: Deploying daemon osd.8 on smithi196 2024-09-15T22:07:35.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:35 smithi196 ceph-mon[35109]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:35.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:35 smithi077 ceph-mon[30980]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:35 smithi117 ceph-mon[35021]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:37.499 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:37 smithi196 ceph-mon[35109]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:37 smithi077 ceph-mon[30980]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:37 smithi117 ceph-mon[35021]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:38.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:38.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:38.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:39.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:39 smithi196 ceph-mon[35109]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:39.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:39 smithi077 ceph-mon[30980]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:39.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:39 smithi117 ceph-mon[35021]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:41.518 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:41.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:41.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:41.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:41.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:41.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:41.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:41.857 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-09-15T22:07:41.858 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.855+0000 7fb3177fe640 1 -- 172.21.15.196:0/1327155054 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb320062630 con 0x7fb30c0a3240 2024-09-15T22:07:41.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 msgr2=0x7fb30c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.859 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 0x7fb30c0a5700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005fd0 tx=0x7fb31c005850 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 msgr2=0x7fb320100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cce0 tx=0x7fb31800a620 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 shutdown_connections 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb30c0a3240 0x7fb30c0a5700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3201078f0 0x7fb320100b80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb320104fc0 0x7fb320100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 --2- 172.21.15.196:0/1327155054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb320102690 0x7fb320101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 >> 172.21.15.196:0/1327155054 conn(0x7fb3200fc290 msgr2=0x7fb320102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 shutdown_connections 2024-09-15T22:07:41.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:07:41.858+0000 7fb32830a640 1 -- 172.21.15.196:0/1327155054 wait complete. 2024-09-15T22:07:42.465 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:42.465 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.465 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:42 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.494 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.8.service 2024-09-15T22:07:42.496 INFO:tasks.cephadm:Deploying osd.9 on smithi077 with /dev/vg_nvme/lv_1... 2024-09-15T22:07:42.496 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:07:42.524 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:42.524 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.524 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:42 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:42 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:42 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:42 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:42.750 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:43.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:43 smithi196 ceph-mon[35109]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:43.513 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:43 smithi117 ceph-mon[35021]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:43.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:43 smithi077 ceph-mon[30980]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:44.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:44 smithi196 ceph-mon[35109]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:07:44.354 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:07:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:07:44.151+0000 7f0939516640 -1 osd.8 0 log_to_monitors true 2024-09-15T22:07:44.410 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:44 smithi077 ceph-mon[30980]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:07:44.532 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:44 smithi117 ceph-mon[35021]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:07:45.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:45 smithi077 ceph-mon[30980]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:45.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:45 smithi077 ceph-mon[30980]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:07:45.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:45 smithi077 ceph-mon[30980]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:07:45.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:45 smithi077 ceph-mon[30980]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:07:45.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:45 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:45.553 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:07:45.587 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-09-15T22:07:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-mon[35109]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-mon[35109]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:07:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-mon[35109]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:07:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-mon[35109]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:07:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:45 smithi117 ceph-mon[35021]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:45 smithi117 ceph-mon[35021]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:07:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:45 smithi117 ceph-mon[35021]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:07:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:45 smithi117 ceph-mon[35021]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:07:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:45 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:45.883 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:46.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:07:45.707+0000 7f0935499640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T22:07:46.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:07:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:07:45.718+0000 7f092d2b9640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 -- 172.21.15.77:0/1528019091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c81037c0 msgr2=0x7ff2c810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 --2- 172.21.15.77:0/1528019091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c81037c0 0x7ff2c810a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009ca0 tx=0x7ff2c002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 -- 172.21.15.77:0/1528019091 shutdown_connections 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 --2- 172.21.15.77:0/1528019091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c81037c0 0x7ff2c810a050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 --2- 172.21.15.77:0/1528019091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8102e00 0x7ff2c8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 --2- 172.21.15.77:0/1528019091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c8101c00 0x7ff2c8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 -- 172.21.15.77:0/1528019091 >> 172.21.15.77:0/1528019091 conn(0x7ff2c80fd390 msgr2=0x7ff2c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:46.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 -- 172.21.15.77:0/1528019091 shutdown_connections 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.363+0000 7ff2cf389640 1 -- 172.21.15.77:0/1528019091 wait complete. 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 Processor -- start 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 -- start start 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8101c00 0x7ff2c8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c81037c0 0x7ff2c8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810d880 con 0x7ff2c8102e00 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810d700 con 0x7ff2c8101c00 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cf389640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810da00 con 0x7ff2c81037c0 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42408/0 (socket says 172.21.15.77:42408) 2024-09-15T22:07:46.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 -- 172.21.15.77:0/662285753 learned_addr learned my addr 172.21.15.77:0/662285753 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cd926640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c81037c0 0x7ff2c8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cd125640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8101c00 0x7ff2c8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 -- 172.21.15.77:0/662285753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c81037c0 msgr2=0x7ff2c8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c81037c0 0x7ff2c8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 -- 172.21.15.77:0/662285753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8101c00 msgr2=0x7ff2c8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8101c00 0x7ff2c8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 -- 172.21.15.77:0/662285753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80753e0 con 0x7ff2c8102e00 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.364+0000 7ff2cc924640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d950 tx=0x7ff2b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.365+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8017070 con 0x7ff2c8102e00 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.365+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b8009c30 con 0x7ff2c8102e00 2024-09-15T22:07:46.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.365+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800f6b0 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.365+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c81a1ff0 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.365+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a24a0 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.366+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8108fa0 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.367+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff2b80109d0 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.368+0000 7ff2be7fc640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 0x7ff2b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.368+0000 7ff2cd125640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 0x7ff2b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.368+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7ff2b80ceb80 con 0x7ff2c8102e00 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.368+0000 7ff2cd125640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 0x7ff2b40a5780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4005fd0 tx=0x7ff2c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:46.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.371+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b8009de0 con 0x7ff2c8102e00 2024-09-15T22:07:46.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:07:46.552+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff2c8062630 con 0x7ff2b40a32c0 2024-09-15T22:07:46.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:07:46.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:07:46.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi196 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] boot 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:07:46.555 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi196 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] boot 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:07:46.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi196 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-15T22:07:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979]' entity='osd.8' 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: osd.8 [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] boot 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:07:46.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:47 smithi077 ceph-mon[30980]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:07:47.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:47.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:47 smithi196 ceph-mon[35109]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:07:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:07:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:07:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:07:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:07:47.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:07:47.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:47.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:47 smithi117 ceph-mon[35021]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:07:48.408 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:48 smithi077 ceph-mon[30980]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:48.409 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:48 smithi077 ceph-mon[30980]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:07:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:48 smithi196 ceph-mon[35109]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:48 smithi196 ceph-mon[35109]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:07:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:48 smithi117 ceph-mon[35021]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:48 smithi117 ceph-mon[35021]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:07:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:07:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]: dispatch 2024-09-15T22:07:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]': finished 2024-09-15T22:07:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T22:07:49.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:49 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:07:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:07:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]: dispatch 2024-09-15T22:07:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]': finished 2024-09-15T22:07:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T22:07:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:49 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]: dispatch 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3253252244' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5097d6bd-7d2a-45fc-958c-da3902b80859"}]': finished 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: osdmap e61: 10 total, 9 up, 10 in 2024-09-15T22:07:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:49 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:07:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:50 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/95867631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:50 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/95867631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:50 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/95867631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:07:51.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:51 smithi077 ceph-mon[30980]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:51 smithi196 ceph-mon[35109]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:51 smithi117 ceph-mon[35021]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:53 smithi077 ceph-mon[30980]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:53 smithi196 ceph-mon[35109]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:53 smithi117 ceph-mon[35021]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:07:55.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:55 smithi077 ceph-mon[30980]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:07:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:55 smithi196 ceph-mon[35109]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:07:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:55 smithi117 ceph-mon[35021]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:07:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:57 smithi077 ceph-mon[30980]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:07:57.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:57 smithi196 ceph-mon[35109]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:07:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:57 smithi117 ceph-mon[35021]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:07:58.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:07:58.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:58 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:07:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:58 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:07:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:58 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:59 smithi077 ceph-mon[30980]: Deploying daemon osd.9 on smithi077 2024-09-15T22:07:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:07:59 smithi077 ceph-mon[30980]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:07:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:59 smithi196 ceph-mon[35109]: Deploying daemon osd.9 on smithi077 2024-09-15T22:07:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:07:59 smithi196 ceph-mon[35109]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:07:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:59 smithi117 ceph-mon[35021]: Deploying daemon osd.9 on smithi077 2024-09-15T22:07:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:07:59 smithi117 ceph-mon[35021]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:08:01.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:01 smithi077 ceph-mon[30980]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:08:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:01 smithi196 ceph-mon[35109]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:08:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:01 smithi117 ceph-mon[35021]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:08:03.510 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:03 smithi077 ceph-mon[30980]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:03 smithi196 ceph-mon[35109]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:03 smithi117 ceph-mon[35021]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:04.568 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:04.568 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:04.568 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:05 smithi077 ceph-mon[30980]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:05 smithi196 ceph-mon[35109]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:05 smithi117 ceph-mon[35021]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:07.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:07 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:07 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:07 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.057+0000 7ff2be7fc640 1 -- 172.21.15.77:0/662285753 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2c8062630 con 0x7ff2b40a32c0 2024-09-15T22:08:08.059 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 9 on host 'smithi077' 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 msgr2=0x7ff2b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 0x7ff2b40a5780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4005fd0 tx=0x7ff2c4005850 comp rx=0 tx=0).stop 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 msgr2=0x7ff2c80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d950 tx=0x7ff2b800de10 comp rx=0 tx=0).stop 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 shutdown_connections 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff2b40a32c0 0x7ff2b40a5780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2c81037c0 0x7ff2c8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2c8102e00 0x7ff2c80745e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:08.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 --2- 172.21.15.77:0/662285753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c8101c00 0x7ff2c8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:08.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 >> 172.21.15.77:0/662285753 conn(0x7ff2c80fd390 msgr2=0x7ff2c8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:08.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 shutdown_connections 2024-09-15T22:08:08.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:08:08.059+0000 7ff2cf389640 1 -- 172.21.15.77:0/662285753 wait complete. 2024-09-15T22:08:08.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:08 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:08 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:08 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:08 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:08 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:08 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:08 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:08 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:08 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:08.817 DEBUG:teuthology.orchestra.run.smithi077:osd.9> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.9.service 2024-09-15T22:08:08.819 INFO:tasks.cephadm:Deploying osd.10 on smithi117 with /dev/vg_nvme/lv_1... 2024-09-15T22:08:08.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:08:09.078 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:08:09.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:09 smithi117 ceph-mon[35021]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:09.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:09 smithi077 ceph-mon[30980]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:09.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:09 smithi196 ceph-mon[35109]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:10.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:08:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:08:09.736+0000 7fbac9108640 -1 osd.9 0 log_to_monitors true 2024-09-15T22:08:10.454 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:10 smithi077 ceph-mon[30980]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:08:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:10 smithi196 ceph-mon[35109]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:08:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:10 smithi117 ceph-mon[35021]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:08:11.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:11 smithi117 ceph-mon[35021]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:11.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:11 smithi117 ceph-mon[35021]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:08:11.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:11 smithi117 ceph-mon[35021]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:08:11.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:11 smithi117 ceph-mon[35021]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:08:11.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-mon[30980]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-mon[30980]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:08:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-mon[30980]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:08:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-mon[30980]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:08:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:11 smithi196 ceph-mon[35109]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:11 smithi196 ceph-mon[35109]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:08:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:11 smithi196 ceph-mon[35109]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:08:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:11 smithi196 ceph-mon[35109]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-09-15T22:08:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:11.816 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T22:08:11.842 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-09-15T22:08:12.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:08:11.744+0000 7fbac508b640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T22:08:12.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:08:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:08:11.757+0000 7fbabceab640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:08:12.097 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:08:12.475 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.476 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:12.477 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:12 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:12.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:12.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:12 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544]' entity='osd.9' 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:12.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:12.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:12 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 -- 172.21.15.117:0/1928514450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848100b10 msgr2=0x7f9848100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 --2- 172.21.15.117:0/1928514450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848100b10 0x7f9848100f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9844009900 tx=0x7f984402a410 comp rx=0 tx=0).stop 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 -- 172.21.15.117:0/1928514450 shutdown_connections 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 --2- 172.21.15.117:0/1928514450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98481026d0 0x7f9848108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 --2- 172.21.15.117:0/1928514450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9848101d10 0x7f9848102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 --2- 172.21.15.117:0/1928514450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848100b10 0x7f9848100f10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 -- 172.21.15.117:0/1928514450 >> 172.21.15.117:0/1928514450 conn(0x7f98480fc2a0 msgr2=0x7f98480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 -- 172.21.15.117:0/1928514450 shutdown_connections 2024-09-15T22:08:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.590+0000 7f984f42c640 1 -- 172.21.15.117:0/1928514450 wait complete. 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 Processor -- start 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 -- start start 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 0x7f9848197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 0x7f984819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f984810c7a0 con 0x7f98481026d0 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f984810c620 con 0x7f9848101d10 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984f42c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f984810c920 con 0x7f9848100b10 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42326/0 (socket says 172.21.15.117:42326) 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 -- 172.21.15.117:0/3900679344 learned_addr learned my addr 172.21.15.117:0/3900679344 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-15T22:08:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984d1c8640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 0x7f9848197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984d9c9640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 0x7f984819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 -- 172.21.15.117:0/3900679344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 msgr2=0x7f9848197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 0x7f9848197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 -- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 msgr2=0x7f984819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 0x7f984819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 -- 172.21.15.117:0/3900679344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819fb40 con 0x7f9848101d10 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984d1c8640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 0x7f9848197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984d9c9640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 0x7f984819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.591+0000 7f984c9c7640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f983800d950 tx=0x7f983800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.592+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838017070 con 0x7f9848101d10 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.592+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838009c30 con 0x7f9848101d10 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.592+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984819fe30 con 0x7f9848101d10 2024-09-15T22:08:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.592+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800f6b0 con 0x7f9848101d10 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.592+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a02e0 con 0x7f9848101d10 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.593+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98481a0510 con 0x7f9848101d10 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.594+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f98380109d0 con 0x7f9848101d10 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.594+0000 7f983e7fc640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 0x7f98340a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.595+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7f98380cec70 con 0x7f9848101d10 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.595+0000 7f984d1c8640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 0x7f98340a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.595+0000 7f984d1c8640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 0x7f98340a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f9844032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.598+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9838009de0 con 0x7f9848101d10 2024-09-15T22:08:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:12.776+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f984806aed0 con 0x7f98340a3270 2024-09-15T22:08:13.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:08:13.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:08:13.528 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi077 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] boot 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: osdmap e64: 10 total, 10 up, 10 in 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:13.529 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:13 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi077 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] boot 2024-09-15T22:08:13.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: osdmap e64: 10 total, 10 up, 10 in 2024-09-15T22:08:13.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:13.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:13.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:13 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi077 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 4272M 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: osd.9 [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] boot 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: osdmap e64: 10 total, 10 up, 10 in 2024-09-15T22:08:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:13.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:13.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:13.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:13 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:14.463 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:14 smithi117 ceph-mon[35021]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:14.463 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:14 smithi117 ceph-mon[35021]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:08:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:14 smithi077 ceph-mon[30980]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:14 smithi077 ceph-mon[30980]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:08:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:14 smithi196 ceph-mon[35109]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:14 smithi196 ceph-mon[35109]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/2817565400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]': finished 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:08:15.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:15 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/2817565400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]': finished 2024-09-15T22:08:15.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:08:15.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:15 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/2817565400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]: dispatch 2024-09-15T22:08:15.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67e4b9ba-333a-40a1-a03a-9a31da950d8c"}]': finished 2024-09-15T22:08:15.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:08:15.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:15 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:16 smithi077 ceph-mon[30980]: from='client.? 172.21.15.117:0/478593063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:16 smithi196 ceph-mon[35109]: from='client.? 172.21.15.117:0/478593063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:16 smithi117 ceph-mon[35021]: from='client.? 172.21.15.117:0/478593063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:17 smithi077 ceph-mon[30980]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:17 smithi196 ceph-mon[35109]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:17 smithi117 ceph-mon[35021]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:19.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:19 smithi077 ceph-mon[30980]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:19 smithi196 ceph-mon[35109]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:19 smithi117 ceph-mon[35021]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:21 smithi077 ceph-mon[30980]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:21.585 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:21 smithi117 ceph-mon[35021]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:21.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:21 smithi196 ceph-mon[35109]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:23.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:23 smithi117 ceph-mon[35021]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:23.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:23 smithi196 ceph-mon[35109]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:23.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:23 smithi077 ceph-mon[30980]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:08:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:08:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:24.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:08:24.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:25.481 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:25 smithi117 ceph-mon[35021]: Deploying daemon osd.10 on smithi117 2024-09-15T22:08:25.481 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:25 smithi117 ceph-mon[35021]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:25 smithi196 ceph-mon[35109]: Deploying daemon osd.10 on smithi117 2024-09-15T22:08:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:25 smithi196 ceph-mon[35109]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:25.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:25 smithi077 ceph-mon[30980]: Deploying daemon osd.10 on smithi117 2024-09-15T22:08:25.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:25 smithi077 ceph-mon[30980]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:27.551 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:27 smithi117 ceph-mon[35021]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:27 smithi196 ceph-mon[35109]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:27.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:27 smithi077 ceph-mon[30980]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:29.520 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:29 smithi117 ceph-mon[35021]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:29.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:29 smithi196 ceph-mon[35109]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:29.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:29 smithi077 ceph-mon[30980]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:30.479 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:30.479 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:30.479 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:30.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:30.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:30.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:31.582 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:31 smithi117 ceph-mon[35021]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:31 smithi196 ceph-mon[35109]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:31.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:31 smithi077 ceph-mon[30980]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:33.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.369+0000 7f983e7fc640 1 -- 172.21.15.117:0/3900679344 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f984806aed0 con 0x7f98340a3270 2024-09-15T22:08:33.371 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 10 on host 'smithi117' 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 msgr2=0x7f98340a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 0x7f98340a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f9844032dd0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 msgr2=0x7f9848198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f983800d950 tx=0x7f983800de10 comp rx=0 tx=0).stop 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 shutdown_connections 2024-09-15T22:08:33.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f98340a3270 0x7f98340a5730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98481026d0 0x7f984819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9848101d10 0x7f9848198450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 --2- 172.21.15.117:0/3900679344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9848100b10 0x7f9848197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 >> 172.21.15.117:0/3900679344 conn(0x7f98480fc2a0 msgr2=0x7f984806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 shutdown_connections 2024-09-15T22:08:33.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T22:08:33.371+0000 7f984f42c640 1 -- 172.21.15.117:0/3900679344 wait complete. 2024-09-15T22:08:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:33.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:33.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:33.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:33 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:33.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:33 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:33 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.096 DEBUG:teuthology.orchestra.run.smithi117:osd.10> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.10.service 2024-09-15T22:08:34.099 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-09-15T22:08:34.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:08:34.356 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:08:34.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:34.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.477 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:34.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:34.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:35.496 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:35 smithi117 ceph-mon[35021]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:35.721 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:35 smithi196 ceph-mon[35109]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:35.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:35 smithi077 ceph-mon[30980]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:08:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:08:35.782+0000 7fcaa9348640 -1 osd.10 0 log_to_monitors true 2024-09-15T22:08:36.579 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:36 smithi117 ceph-mon[35021]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.579 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:36 smithi117 ceph-mon[35021]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.580 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:36 smithi117 ceph-mon[35021]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:36 smithi196 ceph-mon[35109]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:36 smithi196 ceph-mon[35109]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:36 smithi196 ceph-mon[35109]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:36 smithi077 ceph-mon[30980]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:36 smithi077 ceph-mon[30980]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:08:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:36 smithi077 ceph-mon[30980]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:36.936 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-15T22:08:36.964 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-09-15T22:08:37.213 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi117 2024-09-15T22:08:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:37.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 -- 172.21.15.196:0/1751832819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c8101c00 msgr2=0x7f33c8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/1751832819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c8101c00 0x7f33c8102000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009900 tx=0x7f33b802a410 comp rx=0 tx=0).stop 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 -- 172.21.15.196:0/1751832819 shutdown_connections 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/1751832819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c81037c0 0x7f33c810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/1751832819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8102e00 0x7f33c8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/1751832819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c8101c00 0x7f33c8102000 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.721 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 -- 172.21.15.196:0/1751832819 >> 172.21.15.196:0/1751832819 conn(0x7f33c80fd390 msgr2=0x7f33c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 -- 172.21.15.196:0/1751832819 shutdown_connections 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.693+0000 7f33cdbe0640 1 -- 172.21.15.196:0/1751832819 wait complete. 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 Processor -- start 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 -- start start 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8101c00 0x7f33c8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c8102e00 0x7f33c8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810d880 con 0x7f33c8102e00 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810d700 con 0x7f33c8101c00 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33cdbe0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810da00 con 0x7f33c81037c0 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:36394/0 (socket says 172.21.15.196:36394) 2024-09-15T22:08:37.722 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8101c00 0x7f33c8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.694+0000 7f33c7fff640 1 -- 172.21.15.196:0/3222648169 learned_addr learned my addr 172.21.15.196:0/3222648169 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c6ffd640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c8102e00 0x7f33c8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 -- 172.21.15.196:0/3222648169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8101c00 msgr2=0x7f33c8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8101c00 0x7f33c8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 -- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c8102e00 msgr2=0x7f33c8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c8102e00 0x7f33c8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 -- 172.21.15.196:0/3222648169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c819fb70 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c7fff640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33c000cad0 tx=0x7f33c000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000ae20 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c0005020 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c819fe60 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000f6f0 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.695+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81a02c0 con 0x7f33c81037c0 2024-09-15T22:08:37.723 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.696+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8109070 con 0x7f33c81037c0 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.697+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f33c00051d0 con 0x7f33c81037c0 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.698+0000 7f33c4ff9640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 0x7f33b00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.698+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7f33c00cf590 con 0x7f33c81037c0 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.698+0000 7f33c77fe640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 0x7f33b00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.699+0000 7f33c77fe640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 0x7f33b00a5730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005ab0 tx=0x7f33b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:37.724 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.701+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33c0014030 con 0x7f33c81037c0 2024-09-15T22:08:37.758 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:08:37.758 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.758 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:08:37.758 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi117 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:37.759 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi117 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:37.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:37.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:37.881+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f33c8062630 con 0x7f33b00a3270 2024-09-15T22:08:38.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:08:37.756+0000 7fcaa52cb640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T22:08:38.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:08:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:08:37.766+0000 7fca9d0eb640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='osd.10 ' entity='osd.10' 2024-09-15T22:08:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:38.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:38.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:38.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:38.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:38 smithi196 ceph-mon[35109]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='osd.10 ' entity='osd.10' 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:38.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:38.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:38 smithi077 ceph-mon[30980]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-15T22:08:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:08:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='osd.10 ' entity='osd.10' 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:38.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:38 smithi117 ceph-mon[35021]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] boot 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: osdmap e69: 11 total, 11 up, 11 in 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:39 smithi196 ceph-mon[35109]: osdmap e70: 11 total, 11 up, 11 in 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] boot 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: osdmap e69: 11 total, 11 up, 11 in 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:39 smithi077 ceph-mon[30980]: osdmap e70: 11 total, 11 up, 11 in 2024-09-15T22:08:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:08:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:08:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: osd.10 [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] boot 2024-09-15T22:08:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: osdmap e69: 11 total, 11 up, 11 in 2024-09-15T22:08:39.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:39.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:39 smithi117 ceph-mon[35021]: osdmap e70: 11 total, 11 up, 11 in 2024-09-15T22:08:40.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:08:40.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:40.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/4194597243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:40.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:40.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]': finished 2024-09-15T22:08:40.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:08:40.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:40 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:40.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:08:40.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:40.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/4194597243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:40.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:40.962 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]': finished 2024-09-15T22:08:40.963 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:08:40.963 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:40 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:08:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/4194597243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]: dispatch 2024-09-15T22:08:41.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35d9686a-188b-4bee-a88a-9c31fe9dc7b0"}]': finished 2024-09-15T22:08:41.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:08:41.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:40 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:42.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:41 smithi077 ceph-mon[30980]: from='client.? 172.21.15.196:0/710645561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:42.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:41 smithi077 ceph-mon[30980]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:08:42.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:41 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:42.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:41 smithi196 ceph-mon[35109]: from='client.? 172.21.15.196:0/710645561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:42.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:41 smithi196 ceph-mon[35109]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:08:42.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:41 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:41 smithi117 ceph-mon[35021]: from='client.? 172.21.15.196:0/710645561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:08:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:41 smithi117 ceph-mon[35021]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:08:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:41 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:43.005 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:42 smithi196 ceph-mon[35109]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:43.013 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:42 smithi117 ceph-mon[35021]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:43.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:42 smithi077 ceph-mon[30980]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:45 smithi077 ceph-mon[30980]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-15T22:08:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:45 smithi196 ceph-mon[35109]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-15T22:08:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:45 smithi117 ceph-mon[35021]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-15T22:08:47.387 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:47 smithi196 ceph-mon[35109]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-15T22:08:47.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:47 smithi077 ceph-mon[30980]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-15T22:08:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:47 smithi117 ceph-mon[35021]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-15T22:08:49.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:49 smithi196 ceph-mon[35109]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:08:49.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:49 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:08:49.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:49 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:49 smithi077 ceph-mon[30980]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:08:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:49 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:08:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:49 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:49 smithi117 ceph-mon[35021]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:08:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:49 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:08:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:49 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:50.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:50 smithi196 ceph-mon[35109]: Deploying daemon osd.11 on smithi196 2024-09-15T22:08:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:50 smithi077 ceph-mon[30980]: Deploying daemon osd.11 on smithi196 2024-09-15T22:08:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:50 smithi117 ceph-mon[35021]: Deploying daemon osd.11 on smithi196 2024-09-15T22:08:51.543 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:51 smithi196 ceph-mon[35109]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:08:51.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:51 smithi077 ceph-mon[30980]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:08:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:51 smithi117 ceph-mon[35021]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:08:53.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:53 smithi196 ceph-mon[35109]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:08:53.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:53 smithi077 ceph-mon[30980]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:08:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:53 smithi117 ceph-mon[35021]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:08:55.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:55 smithi196 ceph-mon[35109]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:55.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:55 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:55 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:55.569 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:55 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:55 smithi077 ceph-mon[30980]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:55 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:55 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:55.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:55 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:55 smithi117 ceph-mon[35021]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:08:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:55 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:55 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:55 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:57.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:57 smithi077 ceph-mon[30980]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:57 smithi196 ceph-mon[35109]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:57 smithi117 ceph-mon[35021]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:58.769 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.767+0000 7f33c4ff9640 1 -- 172.21.15.196:0/3222648169 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f33c8062630 con 0x7f33b00a3270 2024-09-15T22:08:58.769 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 msgr2=0x7f33b00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 0x7f33b00a5730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005ab0 tx=0x7f33b8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 msgr2=0x7f33c819f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33c000cad0 tx=0x7f33c000cf90 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 shutdown_connections 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.769+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f33b00a3270 0x7f33b00a5730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33c81037c0 0x7f33c819f500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c8102e00 0x7f33c8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 --2- 172.21.15.196:0/3222648169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c8101c00 0x7f33c8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.772 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 >> 172.21.15.196:0/3222648169 conn(0x7f33c80fd390 msgr2=0x7f33c8106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 shutdown_connections 2024-09-15T22:08:58.773 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:08:58.770+0000 7f33cdbe0640 1 -- 172.21.15.196:0/3222648169 wait complete. 2024-09-15T22:08:59.398 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.399 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:08:59 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.425 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.11.service 2024-09-15T22:08:59.428 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-15T22:08:59.428 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:08:59 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:08:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:08:59 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:08:59.687 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3972118342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c1037d0 msgr2=0x7f6f7c10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3972118342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c10a060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009ca0 tx=0x7f6f7402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3972118342 shutdown_connections 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3972118342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c10a060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3972118342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c102e10 0x7f6f7c103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3972118342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c101c10 0x7f6f7c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3972118342 >> 172.21.15.77:0/3972118342 conn(0x7f6f7c0fd3c0 msgr2=0x7f6f7c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3972118342 shutdown_connections 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3972118342 wait complete. 2024-09-15T22:09:00.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.192+0000 7f6f83b8b640 1 Processor -- start 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 -- start start 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c102e10 0x7f6f7c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c10d890 con 0x7f6f7c101c10 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c10d710 con 0x7f6f7c102e10 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c10da10 con 0x7f6f7c1037d0 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41560/0 (socket says 172.21.15.77:41560) 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 -- 172.21.15.77:0/3395794200 learned_addr learned my addr 172.21.15.77:0/3395794200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f82128640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81126640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c102e10 0x7f6f7c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 -- 172.21.15.77:0/3395794200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c1037d0 msgr2=0x7f6f7c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 -- 172.21.15.77:0/3395794200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c102e10 msgr2=0x7f6f7c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c102e10 0x7f6f7c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 -- 172.21.15.77:0/3395794200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c19faf0 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f81927640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c970 tx=0x7f6f6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c004220 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c19fde0 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c0043c0 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.193+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00f450 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.194+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c1a0230 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.194+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c108fc0 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.195+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6f6c0079b0 con 0x7f6f7c101c10 2024-09-15T22:09:00.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.196+0000 7f6f72ffd640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 0x7f6f680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.196+0000 7f6f81126640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 0x7f6f680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.196+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f6f6c0cf070 con 0x7f6f7c101c10 2024-09-15T22:09:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.197+0000 7f6f81126640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 0x7f6f680a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f78005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:00.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.200+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f6c014030 con 0x7f6f7c101c10 2024-09-15T22:09:00.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.376+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6f7c02da30 con 0x7f6f7c101c10 2024-09-15T22:09:00.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.377+0000 7f6f72ffd640 1 -- 172.21.15.77:0/3395794200 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f6f6c09c370 con 0x7f6f7c101c10 2024-09-15T22:09:00.379 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:00.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 msgr2=0x7f6f680a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 0x7f6f680a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f78005850 comp rx=0 tx=0).stop 2024-09-15T22:09:00.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 msgr2=0x7f6f7c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c970 tx=0x7f6f6c00ce30 comp rx=0 tx=0).stop 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 shutdown_connections 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f6f680a3270 0x7f6f680a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f7c1037d0 0x7f6f7c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f7c102e10 0x7f6f7c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 --2- 172.21.15.77:0/3395794200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f7c101c10 0x7f6f7c197ec0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 >> 172.21.15.77:0/3395794200 conn(0x7f6f7c0fd3c0 msgr2=0x7f6f7c106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 shutdown_connections 2024-09-15T22:09:00.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:00.379+0000 7f6f83b8b640 1 -- 172.21.15.77:0/3395794200 wait complete. 2024-09-15T22:09:01.247 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:09:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:09:00.934+0000 7feaa87e5640 -1 osd.11 0 log_to_monitors true 2024-09-15T22:09:01.266 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726438118,"num_in_osds":12,"osd_in_since":1726438120,"num_remapped_pgs":0} 2024-09-15T22:09:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:01 smithi077 ceph-mon[30980]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:01 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3395794200' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:01.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:01 smithi077 ceph-mon[30980]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:01.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:01 smithi077 ceph-mon[30980]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:01 smithi196 ceph-mon[35109]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:01 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3395794200' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:01 smithi196 ceph-mon[35109]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:01 smithi196 ceph-mon[35109]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:01 smithi117 ceph-mon[35021]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:01 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3395794200' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:01 smithi117 ceph-mon[35021]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:01 smithi117 ceph-mon[35021]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:09:02.267 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:09:02.532 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:02 smithi077 ceph-mon[30980]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:09:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:02 smithi077 ceph-mon[30980]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:09:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:02 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:02 smithi077 ceph-mon[30980]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:02 smithi077 ceph-mon[30980]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-mon[35109]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:09:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-mon[35109]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:09:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-mon[35109]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:02.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-mon[35109]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:02 smithi117 ceph-mon[35021]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:09:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:02 smithi117 ceph-mon[35021]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:09:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:02 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:02 smithi117 ceph-mon[35021]: from='osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:02.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:02 smithi117 ceph-mon[35021]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-15T22:09:03.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 -- 172.21.15.77:0/1714515961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a0102e00 msgr2=0x7ff9a0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 --2- 172.21.15.77:0/1714515961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a0102e00 0x7ff9a0103280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff99002a410 comp rx=0 tx=0).stop 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 -- 172.21.15.77:0/1714515961 shutdown_connections 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 --2- 172.21.15.77:0/1714515961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a01037c0 0x7ff9a010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 --2- 172.21.15.77:0/1714515961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a0102e00 0x7ff9a0103280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 --2- 172.21.15.77:0/1714515961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0101c00 0x7ff9a0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 -- 172.21.15.77:0/1714515961 >> 172.21.15.77:0/1714515961 conn(0x7ff9a00fd390 msgr2=0x7ff9a00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 -- 172.21.15.77:0/1714515961 shutdown_connections 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 -- 172.21.15.77:0/1714515961 wait complete. 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.991+0000 7ff9a8772640 1 Processor -- start 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 -- start start 2024-09-15T22:09:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a0101c00 0x7ff9a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 0x7ff9a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a010d880 con 0x7ff9a01037c0 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a010d700 con 0x7ff9a0102e00 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a010da00 con 0x7ff9a0101c00 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41564/0 (socket says 172.21.15.77:41564) 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 -- 172.21.15.77:0/3979536434 learned_addr learned my addr 172.21.15.77:0/3979536434 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a650e640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a0101c00 0x7ff9a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 -- 172.21.15.77:0/3979536434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a0101c00 msgr2=0x7ff9a0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a0101c00 0x7ff9a0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 -- 172.21.15.77:0/3979536434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 msgr2=0x7ff9a0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a5d0d640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 0x7ff9a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 0x7ff9a0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 -- 172.21.15.77:0/3979536434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a019fb00 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a5d0d640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 0x7ff9a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a6d0f640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cad0 tx=0x7ff99800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800ae20 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff998005020 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.992+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a019fdf0 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.993+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800f6f0 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.993+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01a02a0 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.994+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a0108ff0 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.995+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff9980051d0 con 0x7ff9a01037c0 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.995+0000 7ff9977fe640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 0x7ff98c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.995+0000 7ff9a650e640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 0x7ff98c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.995+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7ff9980cf660 con 0x7ff9a01037c0 2024-09-15T22:09:03.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.996+0000 7ff9a650e640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 0x7ff98c0a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0102c60 tx=0x7ff99c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:02.999+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff998014030 con 0x7ff9a01037c0 2024-09-15T22:09:03.103 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:09:02.646+0000 7feaa4768640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T22:09:03.104 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:09:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:09:02.659+0000 7fea9c588640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:09:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.164+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff9a002d970 con 0x7ff9a01037c0 2024-09-15T22:09:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.165+0000 7ff9977fe640 1 -- 172.21.15.77:0/3979536434 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7ff99809c940 con 0x7ff9a01037c0 2024-09-15T22:09:03.167 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.166+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 msgr2=0x7ff98c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.166+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 0x7ff98c0a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0102c60 tx=0x7ff99c00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.166+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 msgr2=0x7ff9a019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.166+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cad0 tx=0x7ff99800cf90 comp rx=0 tx=0).stop 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 shutdown_connections 2024-09-15T22:09:03.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff98c0a3270 0x7ff98c0a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9a01037c0 0x7ff9a019f490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0102e00 0x7ff9a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 --2- 172.21.15.77:0/3979536434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a0101c00 0x7ff9a0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 >> 172.21.15.77:0/3979536434 conn(0x7ff9a00fd390 msgr2=0x7ff9a01061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 shutdown_connections 2024-09-15T22:09:03.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:03.167+0000 7ff9a8772640 1 -- 172.21.15.77:0/3979536434 wait complete. 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='osd.11 ' entity='osd.11' 2024-09-15T22:09:03.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3979536434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:09:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:09:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='osd.11 ' entity='osd.11' 2024-09-15T22:09:03.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3979536434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-15T22:09:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:09:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='osd.11 ' entity='osd.11' 2024-09-15T22:09:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3979536434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:03.829 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726438118,"num_in_osds":12,"osd_in_since":1726438120,"num_remapped_pgs":0} 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: purged_snaps scrub starts 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: purged_snaps scrub ok 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: Detected new or changed devices on smithi196 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] boot 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:09:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: purged_snaps scrub starts 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: purged_snaps scrub ok 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: Detected new or changed devices on smithi196 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] boot 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:09:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: purged_snaps scrub starts 2024-09-15T22:09:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: purged_snaps scrub ok 2024-09-15T22:09:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: Detected new or changed devices on smithi196 2024-09-15T22:09:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-15T22:09:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: osd.11 [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] boot 2024-09-15T22:09:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:09:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:09:04.832 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:09:05.089 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:05.479 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:05 smithi077 ceph-mon[30980]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:05.479 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:05 smithi077 ceph-mon[30980]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:09:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:05 smithi196 ceph-mon[35109]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:05 smithi196 ceph-mon[35109]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:09:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- 172.21.15.77:0/3866149574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990102e00 msgr2=0x7f1990103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- 172.21.15.77:0/3866149574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990102e00 0x7f1990103280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f197c009900 tx=0x7f197c02a410 comp rx=0 tx=0).stop 2024-09-15T22:09:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- 172.21.15.77:0/3866149574 shutdown_connections 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- 172.21.15.77:0/3866149574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19901037c0 0x7f199010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- 172.21.15.77:0/3866149574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990102e00 0x7f1990103280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- 172.21.15.77:0/3866149574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1990101c00 0x7f1990102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- 172.21.15.77:0/3866149574 >> 172.21.15.77:0/3866149574 conn(0x7f19900fd390 msgr2=0x7f19900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- 172.21.15.77:0/3866149574 shutdown_connections 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- 172.21.15.77:0/3866149574 wait complete. 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 Processor -- start 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- start start 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1990102e00 0x7f19901982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 0x7f199019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f199010d860 con 0x7f1990101c00 2024-09-15T22:09:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f199010d6e0 con 0x7f19901037c0 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.554+0000 7f19970bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f199010d9e0 con 0x7f1990102e00 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41584/0 (socket says 172.21.15.77:41584) 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 -- 172.21.15.77:0/2647512836 learned_addr learned my addr 172.21.15.77:0/2647512836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1995658640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 0x7f199019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 -- 172.21.15.77:0/2647512836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1990102e00 msgr2=0x7f19901982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1990102e00 0x7f19901982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 -- 172.21.15.77:0/2647512836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 msgr2=0x7f199019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 0x7f199019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 -- 172.21.15.77:0/2647512836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199019f9d0 con 0x7f1990101c00 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1995658640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 0x7f199019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f1994e57640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f198400c970 tx=0x7f198400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984007c10 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1984007db0 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199019fcc0 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400f940 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.555+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901a01b0 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.556+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990108f30 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.557+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f198400fae0 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.558+0000 7f198dffb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 0x7f19800a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.558+0000 7f198ffff640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 0x7f19800a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.558+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f19840cf8d0 con 0x7f1990101c00 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.558+0000 7f198ffff640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 0x7f19800a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f197c0096c0 tx=0x7f197c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.562+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f198409cb20 con 0x7f1990101c00 2024-09-15T22:09:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:05 smithi117 ceph-mon[35021]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:09:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:05 smithi117 ceph-mon[35021]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:09:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.727+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f199002d990 con 0x7f1990101c00 2024-09-15T22:09:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.727+0000 7f198dffb640 1 -- 172.21.15.77:0/2647512836 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f198409c420 con 0x7f1990101c00 2024-09-15T22:09:05.730 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 msgr2=0x7f19800a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 0x7f19800a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f197c0096c0 tx=0x7f197c031400 comp rx=0 tx=0).stop 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 msgr2=0x7f1990197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f198400c970 tx=0x7f198400ce30 comp rx=0 tx=0).stop 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 shutdown_connections 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19800a3270 0x7f19800a5730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19901037c0 0x7f199019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1990102e00 0x7f19901982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 --2- 172.21.15.77:0/2647512836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1990101c00 0x7f1990197da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.729+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 >> 172.21.15.77:0/2647512836 conn(0x7f19900fd390 msgr2=0x7f1990106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.730+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 shutdown_connections 2024-09-15T22:09:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:05.730+0000 7f19970bb640 1 -- 172.21.15.77:0/2647512836 wait complete. 2024-09-15T22:09:06.422 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726438143,"num_in_osds":12,"osd_in_since":1726438120,"num_remapped_pgs":0} 2024-09-15T22:09:06.423 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:09:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:06 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2647512836' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:06 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2647512836' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:06 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2647512836' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:09:06.666 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:07.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 -- 172.21.15.77:0/3374965356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 msgr2=0x7f1588106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 --2- 172.21.15.77:0/3374965356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f1588106e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f157c009930 tx=0x7f157c02a420 comp rx=0 tx=0).stop 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 -- 172.21.15.77:0/3374965356 shutdown_connections 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 --2- 172.21.15.77:0/3374965356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588101420 0x7f158810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 --2- 172.21.15.77:0/3374965356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588100a80 0x7f1588100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 --2- 172.21.15.77:0/3374965356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f1588106e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 -- 172.21.15.77:0/3374965356 >> 172.21.15.77:0/3374965356 conn(0x7f15880fc7a0 msgr2=0x7f15880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 -- 172.21.15.77:0/3374965356 shutdown_connections 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.124+0000 7f158cbff640 1 -- 172.21.15.77:0/3374965356 wait complete. 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 Processor -- start 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 -- start start 2024-09-15T22:09:07.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588100a80 0x7f1588102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101420 0x7f1588102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1588110f20 con 0x7f1588106a80 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1588110da0 con 0x7f1588100a80 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158cbff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15881110a0 con 0x7f1588101420 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41596/0 (socket says 172.21.15.77:41596) 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 -- 172.21.15.77:0/274383702 learned_addr learned my addr 172.21.15.77:0/274383702 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1585d9b640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101420 0x7f1588102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f158659c640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588100a80 0x7f1588102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 -- 172.21.15.77:0/274383702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101420 msgr2=0x7f1588102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101420 0x7f1588102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 -- 172.21.15.77:0/274383702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588100a80 msgr2=0x7f1588102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588100a80 0x7f1588102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 -- 172.21.15.77:0/274383702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15881a6a80 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.125+0000 7f1586d9d640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f158000b970 tx=0x7f158000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.126+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580019070 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.126+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158000cb40 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.126+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15881a6d70 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.126+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580007650 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.126+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15881a72b0 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.127+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1588100f60 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.128+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f158001c070 con 0x7f1588106a80 2024-09-15T22:09:07.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.129+0000 7f15777fe640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 0x7f15700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.129+0000 7f158659c640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 0x7f15700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.129+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f15800ceeb0 con 0x7f1588106a80 2024-09-15T22:09:07.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.129+0000 7f158659c640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 0x7f15700a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f157c0096c0 tx=0x7f157c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:07.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.132+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15800fe1b0 con 0x7f1588106a80 2024-09-15T22:09:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.302+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15881a7540 con 0x7f1588106a80 2024-09-15T22:09:07.305 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:07 smithi077 ceph-mon[30980]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.304+0000 7f15777fe640 1 -- 172.21.15.77:0/274383702 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18505 (secure 0 0 0) 0x7f158009c070 con 0x7f1588106a80 2024-09-15T22:09:07.306 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:07.306 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":77,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:09:04.654118+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:09:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 msgr2=0x7f15700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 0x7f15700a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f157c0096c0 tx=0x7f157c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 msgr2=0x7f15881030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f158000b970 tx=0x7f158000be30 comp rx=0 tx=0).stop 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 shutdown_connections 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f15700a32c0 0x7f15700a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1588106a80 0x7f15881030d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1588101420 0x7f1588102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 --2- 172.21.15.77:0/274383702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1588100a80 0x7f1588102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 >> 172.21.15.77:0/274383702 conn(0x7f15880fc7a0 msgr2=0x7f15881048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 shutdown_connections 2024-09-15T22:09:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:07.307+0000 7f158cbff640 1 -- 172.21.15.77:0/274383702 wait complete. 2024-09-15T22:09:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:07 smithi196 ceph-mon[35109]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:07 smithi117 ceph-mon[35021]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:08.142 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:05:16.403426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T22:09:08.143 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-15T22:09:08.388 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:08.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:08 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/274383702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:08 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/274383702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:08 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/274383702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 -- 172.21.15.77:0/4153070041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc1000d0 msgr2=0x7fbcfc100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/4153070041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc100530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009900 tx=0x7fbcf002a410 comp rx=0 tx=0).stop 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 -- 172.21.15.77:0/4153070041 shutdown_connections 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/4153070041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 0x7fbcfc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/4153070041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc100530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/4153070041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 -- 172.21.15.77:0/4153070041 >> 172.21.15.77:0/4153070041 conn(0x7fbcfc077210 msgr2=0x7fbcfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 -- 172.21.15.77:0/4153070041 shutdown_connections 2024-09-15T22:09:08.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 -- 172.21.15.77:0/4153070041 wait complete. 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.870+0000 7fbd01fe3640 1 Processor -- start 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 -- start start 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 0x7fbcfc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc112090 con 0x7fbcfc106ba0 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc111f10 con 0x7fbcfc1000d0 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbd01fe3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc112210 con 0x7fbcfc100a70 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41600/0 (socket says 172.21.15.77:41600) 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 -- 172.21.15.77:0/1041649659 learned_addr learned my addr 172.21.15.77:0/1041649659 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfaffd640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 0x7fbcfc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:08.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfb7fe640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 -- 172.21.15.77:0/1041649659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 msgr2=0x7fbcfc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 0x7fbcfc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 -- 172.21.15.77:0/1041649659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc1000d0 msgr2=0x7fbcfc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 -- 172.21.15.77:0/1041649659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc19d4a0 con 0x7fbcfc106ba0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.871+0000 7fbcfbfff640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400eaa0 tx=0x7fbcf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.872+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf400cdb0 con 0x7fbcfc106ba0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.872+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc19d790 con 0x7fbcfc106ba0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.872+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf4005020 con 0x7fbcfc106ba0 2024-09-15T22:09:08.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.872+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40116d0 con 0x7fbcfc106ba0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.872+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc1a5070 con 0x7fbcfc106ba0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.873+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc100530 con 0x7fbcfc106ba0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.874+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbcf40054d0 con 0x7fbcfc106ba0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.875+0000 7fbcf8ff9640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 0x7fbce40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.875+0000 7fbcfb7fe640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 0x7fbce40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.875+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbcf40cf690 con 0x7fbcfc106ba0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.875+0000 7fbcfb7fe640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 0x7fbce40a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005fd0 tx=0x7fbcec00b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:08.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:08.879+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcf4016030 con 0x7fbcfc106ba0 2024-09-15T22:09:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.044+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcfc10a0b0 con 0x7fbcfc106ba0 2024-09-15T22:09:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.045+0000 7fbcf8ff9640 1 -- 172.21.15.77:0/1041649659 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fbcf409c850 con 0x7fbcfc106ba0 2024-09-15T22:09:09.046 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 msgr2=0x7fbce40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 0x7fbce40a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005fd0 tx=0x7fbcec00b590 comp rx=0 tx=0).stop 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 msgr2=0x7fbcfc19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400eaa0 tx=0x7fbcf400ef60 comp rx=0 tx=0).stop 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 shutdown_connections 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbce40a3240 0x7fbce40a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcfc106ba0 0x7fbcfc19cdc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcfc100a70 0x7fbcfc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 --2- 172.21.15.77:0/1041649659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcfc1000d0 0x7fbcfc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 >> 172.21.15.77:0/1041649659 conn(0x7fbcfc077210 msgr2=0x7fbcfc104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:09.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.047+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 shutdown_connections 2024-09-15T22:09:09.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:09.048+0000 7fbd01fe3640 1 -- 172.21.15.77:0/1041649659 wait complete. 2024-09-15T22:09:09.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:09 smithi077 ceph-mon[30980]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:09.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:09 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1041649659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:09:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:09 smithi196 ceph-mon[35109]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:09 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1041649659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:09:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:09 smithi117 ceph-mon[35021]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:09 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1041649659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:09:09.818 INFO:tasks.cephadm:Adding mds.a on smithi077 2024-09-15T22:09:09.818 INFO:tasks.cephadm:Adding mds.d on smithi077 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.g on smithi077 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.j on smithi077 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.b on smithi117 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.e on smithi117 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.h on smithi117 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.k on smithi117 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-09-15T22:09:09.819 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-09-15T22:09:09.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-09-15T22:09:10.080 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- 172.21.15.196:0/1218333175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4105050 msgr2=0x7fe2c4107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/1218333175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4105050 0x7fe2c4107440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009900 tx=0x7fe2b002a410 comp rx=0 tx=0).stop 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- 172.21.15.196:0/1218333175 shutdown_connections 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/1218333175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4107980 0x7fe2c4109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/1218333175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4105050 0x7fe2c4107440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/1218333175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4102720 0x7fe2c4104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- 172.21.15.196:0/1218333175 >> 172.21.15.196:0/1218333175 conn(0x7fe2c40fc280 msgr2=0x7fe2c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- 172.21.15.196:0/1218333175 shutdown_connections 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- 172.21.15.196:0/1218333175 wait complete. 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 Processor -- start 2024-09-15T22:09:10.544 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.535+0000 7fe2cabf0640 1 -- start start 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 0x7fe2c41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 0x7fe2c419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410d940 con 0x7fe2c4107980 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410d7c0 con 0x7fe2c4105050 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410dac0 con 0x7fe2c4102720 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47516/0 (socket says 172.21.15.196:47516) 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 -- 172.21.15.196:0/4182300747 learned_addr learned my addr 172.21.15.196:0/4182300747 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c3fff640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 0x7fe2c41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c918d640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 0x7fe2c419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 -- 172.21.15.196:0/4182300747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 msgr2=0x7fe2c41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:10.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 0x7fe2c41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 -- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 msgr2=0x7fe2c419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 0x7fe2c419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 -- 172.21.15.196:0/4182300747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c419fcb0 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c918d640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 0x7fe2c419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c3fff640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 0x7fe2c41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c898c640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800b4a0 tx=0x7fe2b800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8014070 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c419ffa0 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b8010af0 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.536+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8017d10 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.537+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c41a0480 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.538+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c40fdd70 con 0x7fe2c4102720 2024-09-15T22:09:10.546 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.539+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe2b8017070 con 0x7fe2c4102720 2024-09-15T22:09:10.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.539+0000 7fe2c1ffb640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 0x7fe2b40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:10.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.539+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe2b80ce8d0 con 0x7fe2c4102720 2024-09-15T22:09:10.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.540+0000 7fe2c3fff640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 0x7fe2b40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:10.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.540+0000 7fe2c3fff640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 0x7fe2b40a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0006010 tx=0x7fe2b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:10.547 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.544+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2b809ba90 con 0x7fe2c4102720 2024-09-15T22:09:10.727 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.725+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7fe2c4062730 con 0x7fe2b40a3270 2024-09-15T22:09:10.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.734+0000 7fe2c1ffb640 1 -- 172.21.15.196:0/4182300747 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe2c4062730 con 0x7fe2b40a3270 2024-09-15T22:09:10.736 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 msgr2=0x7fe2b40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 0x7fe2b40a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0006010 tx=0x7fe2b0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 msgr2=0x7fe2c4198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800b4a0 tx=0x7fe2b800b960 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 shutdown_connections 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe2b40a3270 0x7fe2b40a5730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c4107980 0x7fe2c419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c4105050 0x7fe2c41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 --2- 172.21.15.196:0/4182300747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2c4102720 0x7fe2c4198080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 >> 172.21.15.196:0/4182300747 conn(0x7fe2c40fc280 msgr2=0x7fe2c4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 shutdown_connections 2024-09-15T22:09:10.739 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-15T22:09:10.737+0000 7fe2cabf0640 1 -- 172.21.15.196:0/4182300747 wait complete. 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:11.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:11 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.461 DEBUG:teuthology.orchestra.run.smithi077:mds.a> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.a.service 2024-09-15T22:09:11.464 DEBUG:teuthology.orchestra.run.smithi077:mds.d> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.d.service 2024-09-15T22:09:11.507 DEBUG:teuthology.orchestra.run.smithi077:mds.g> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.g.service 2024-09-15T22:09:11.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:11.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:11.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:11.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:11.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:11.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:11 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.530 DEBUG:teuthology.orchestra.run.smithi077:mds.j> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.j.service 2024-09-15T22:09:11.573 DEBUG:teuthology.orchestra.run.smithi117:mds.b> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.b.service 2024-09-15T22:09:11.576 DEBUG:teuthology.orchestra.run.smithi117:mds.e> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.e.service 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:11.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:11 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:11.598 DEBUG:teuthology.orchestra.run.smithi117:mds.h> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.h.service 2024-09-15T22:09:11.641 DEBUG:teuthology.orchestra.run.smithi117:mds.k> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.k.service 2024-09-15T22:09:11.684 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.c.service 2024-09-15T22:09:11.687 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.f.service 2024-09-15T22:09:11.730 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.i.service 2024-09-15T22:09:11.774 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.l.service 2024-09-15T22:09:11.816 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-15T22:09:11.817 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:12.061 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:12.474 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:12 smithi077 ceph-mon[30980]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:12.474 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:12 smithi077 ceph-mon[30980]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-15T22:09:12.474 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:12 smithi077 ceph-mon[30980]: Deploying daemon mds.l on smithi196 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- 172.21.15.77:0/133739275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 msgr2=0x7fcc5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 --2- 172.21.15.77:0/133739275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5010da40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc4802a480 comp rx=0 tx=0).stop 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- 172.21.15.77:0/133739275 shutdown_connections 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 --2- 172.21.15.77:0/133739275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5010da40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 --2- 172.21.15.77:0/133739275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 0x7fcc501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 --2- 172.21.15.77:0/133739275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- 172.21.15.77:0/133739275 >> 172.21.15.77:0/133739275 conn(0x7fcc500fc7d0 msgr2=0x7fcc500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:12.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- 172.21.15.77:0/133739275 shutdown_connections 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- 172.21.15.77:0/133739275 wait complete. 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 Processor -- start 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.551+0000 7fcc587f3640 1 -- start start 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc501984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 0x7fcc501989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501120a0 con 0x7fcc50102ef0 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50111f20 con 0x7fcc50100e20 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50112220 con 0x7fcc50102570 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59642/0 (socket says 172.21.15.77:59642) 2024-09-15T22:09:12.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 -- 172.21.15.77:0/2477414381 learned_addr learned my addr 172.21.15.77:0/2477414381 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc55d8e640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 0x7fcc501989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc5658f640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc501984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 -- 172.21.15.77:0/2477414381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 msgr2=0x7fcc501989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 0x7fcc501989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 -- 172.21.15.77:0/2477414381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 msgr2=0x7fcc501984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc501984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 -- 172.21.15.77:0/2477414381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5019d450 con 0x7fcc50102ef0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc5658f640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc501984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc56d90640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc48004770 tx=0x7fcc48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4803f070 con 0x7fcc50102ef0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5019d6e0 con 0x7fcc50102ef0 2024-09-15T22:09:12.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc48031c30 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48035030 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.552+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501a5020 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.553+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50101200 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.554+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcc48042070 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.555+0000 7fcc4f7fe640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 0x7fcc380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.556+0000 7fcc5658f640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 0x7fcc380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.556+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fcc480f04f0 con 0x7fcc50102ef0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.556+0000 7fcc5658f640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 0x7fcc380a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005fd0 tx=0x7fcc40009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:12.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.559+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc480bd610 con 0x7fcc50102ef0 2024-09-15T22:09:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:12 smithi196 ceph-mon[35109]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:12 smithi196 ceph-mon[35109]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-15T22:09:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:12 smithi196 ceph-mon[35109]: Deploying daemon mds.l on smithi196 2024-09-15T22:09:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:12 smithi117 ceph-mon[35021]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:12 smithi117 ceph-mon[35021]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-15T22:09:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:12 smithi117 ceph-mon[35021]: Deploying daemon mds.l on smithi196 2024-09-15T22:09:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.756+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc501a51f0 con 0x7fcc50102ef0 2024-09-15T22:09:12.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.756+0000 7fcc4f7fe640 1 -- 172.21.15.77:0/2477414381 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fcc480bcf10 con 0x7fcc50102ef0 2024-09-15T22:09:12.758 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:12.758 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"btime":"2024-09-15T22:02:08:800551+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-15T22:09:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 1 2024-09-15T22:09:12.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 msgr2=0x7fcc380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:12.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 0x7fcc380a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005fd0 tx=0x7fcc40009520 comp rx=0 tx=0).stop 2024-09-15T22:09:12.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 msgr2=0x7fcc5019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:12.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc48004770 tx=0x7fcc48003c10 comp rx=0 tx=0).stop 2024-09-15T22:09:12.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 shutdown_connections 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcc380a3240 0x7fcc380a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc50102ef0 0x7fcc5019cd70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc50102570 0x7fcc501989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 --2- 172.21.15.77:0/2477414381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc50100e20 0x7fcc501984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 >> 172.21.15.77:0/2477414381 conn(0x7fcc500fc7d0 msgr2=0x7fcc50106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 shutdown_connections 2024-09-15T22:09:12.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:12.758+0000 7fcc587f3640 1 -- 172.21.15.77:0/2477414381 wait complete. 2024-09-15T22:09:13.496 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:13 smithi077 ceph-mon[30980]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:13.496 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:13 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2477414381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:13.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:13 smithi196 ceph-mon[35109]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:13.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:13 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2477414381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:13.565 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 systemd[1]: Starting Ceph mds.l for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:13 smithi117 ceph-mon[35021]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:13 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2477414381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:13.845 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 podman[65154]: 2024-09-15 22:09:13.466491612 +0000 UTC m=+0.015683622 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:13.845 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 podman[65154]: 2024-09-15 22:09:13.590964186 +0000 UTC m=+0.140156190 container create db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T22:09:13.845 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 podman[65154]: 2024-09-15 22:09:13.843783703 +0000 UTC m=+0.392975709 container init db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-15T22:09:14.103 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 podman[65154]: 2024-09-15 22:09:13.846808995 +0000 UTC m=+0.396000999 container start db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T22:09:14.104 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 ceph-mds[65173]: -- 172.21.15.196:0/3394520133 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff6d2b5860 con 0x55ff6d293000 2024-09-15T22:09:14.104 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: starting mds.l at 2024-09-15T22:09:14.104 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 bash[65154]: db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 2024-09-15T22:09:14.104 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:09:13 smithi196 systemd[1]: Started Ceph mds.l for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:15.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6832/3132990127,v1:172.21.15.196:6833/3132990127] up:boot 2024-09-15T22:09:15.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: fsmap 1 up:standby 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: Deploying daemon mds.k on smithi117 2024-09-15T22:09:15.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:14 smithi117 ceph-mon[35021]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6832/3132990127,v1:172.21.15.196:6833/3132990127] up:boot 2024-09-15T22:09:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: fsmap 1 up:standby 2024-09-15T22:09:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:09:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: Deploying daemon mds.k on smithi117 2024-09-15T22:09:15.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:14 smithi077 ceph-mon[30980]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:15.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6832/3132990127,v1:172.21.15.196:6833/3132990127] up:boot 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: fsmap 1 up:standby 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: Deploying daemon mds.k on smithi117 2024-09-15T22:09:15.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:14 smithi196 ceph-mon[35109]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:15.525 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:15.779 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:16.112 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:15 smithi117 systemd[1]: Starting Ceph mds.k for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 -- 172.21.15.77:0/602127966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0101420 msgr2=0x7fddb010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/602127966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0101420 0x7fddb010d950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009930 tx=0x7fdda002a420 comp rx=0 tx=0).stop 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 -- 172.21.15.77:0/602127966 shutdown_connections 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/602127966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0101420 0x7fddb010d950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/602127966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 0x7fddb0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/602127966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0106a80 0x7fddb0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 -- 172.21.15.77:0/602127966 >> 172.21.15.77:0/602127966 conn(0x7fddb00fc7a0 msgr2=0x7fddb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 -- 172.21.15.77:0/602127966 shutdown_connections 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.378+0000 7fddb4b6b640 1 -- 172.21.15.77:0/602127966 wait complete. 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 Processor -- start 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 -- start start 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 0x7fddb01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 0x7fddb01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0110ea0 con 0x7fddb0106a80 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0110d20 con 0x7fddb0100a80 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddb4b6b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0111020 con 0x7fddb0101420 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59664/0 (socket says 172.21.15.77:59664) 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 -- 172.21.15.77:0/3189558791 learned_addr learned my addr 172.21.15.77:0/3189558791 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddae59c640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 0x7fddb01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddadd9b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 0x7fddb01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 -- 172.21.15.77:0/3189558791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 msgr2=0x7fddb01997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 0x7fddb01997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 -- 172.21.15.77:0/3189558791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 msgr2=0x7fddb01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 0x7fddb01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddaed9d640 1 -- 172.21.15.77:0/3189558791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb01a4870 con 0x7fddb0106a80 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.379+0000 7fddadd9b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 0x7fddb01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fddaed9d640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005ab0 tx=0x7fdda00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda003f030 con 0x7fddb0106a80 2024-09-15T22:09:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda0003ce0 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb01a4a40 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0035070 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.380+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb01a4e90 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.381+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb0100ee0 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.382+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdda0042030 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.383+0000 7fdda77fe640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 0x7fdd9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.383+0000 7fddae59c640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 0x7fdd9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.383+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fdda00ef910 con 0x7fddb0106a80 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.383+0000 7fddae59c640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 0x7fdd9c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdda800a400 tx=0x7fdda800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.387+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdda00bcad0 con 0x7fddb0106a80 2024-09-15T22:09:16.488 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 podman[63449]: 2024-09-15 22:09:16.108642604 +0000 UTC m=+0.015812732 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:16.489 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 podman[63449]: 2024-09-15 22:09:16.251908782 +0000 UTC m=+0.159078943 container create a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-15T22:09:16.489 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 podman[63449]: 2024-09-15 22:09:16.487552168 +0000 UTC m=+0.394722287 container init a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:09:16.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.588+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddb01a50c0 con 0x7fddb0106a80 2024-09-15T22:09:16.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.589+0000 7fdda77fe640 1 -- 172.21.15.77:0/3189558791 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fdda00bc3d0 con 0x7fddb0106a80 2024-09-15T22:09:16.590 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:16.591 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":2,"btime":"2024-09-15T22:09:13:875249+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T22:09:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 2 2024-09-15T22:09:16.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 msgr2=0x7fdd9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:16.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 0x7fdd9c0a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdda800a400 tx=0x7fdda800d040 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 msgr2=0x7fddb01a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005ab0 tx=0x7fdda00042f0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 shutdown_connections 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fdd9c0a3240 0x7fdd9c0a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb0106a80 0x7fddb01a4330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb0101420 0x7fddb01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 --2- 172.21.15.77:0/3189558791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb0100a80 0x7fddb01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 >> 172.21.15.77:0/3189558791 conn(0x7fddb00fc7a0 msgr2=0x7fddb00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 shutdown_connections 2024-09-15T22:09:16.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:16.591+0000 7fddb4b6b640 1 -- 172.21.15.77:0/3189558791 wait complete. 2024-09-15T22:09:16.862 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 podman[63449]: 2024-09-15 22:09:16.490600083 +0000 UTC m=+0.397770206 container start a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-15T22:09:16.862 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 ceph-mds[63468]: -- 172.21.15.117:0/1399325239 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fafd4db860 con 0x55fafd4b8800 2024-09-15T22:09:16.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: starting mds.k at 2024-09-15T22:09:16.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 bash[63449]: a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 2024-09-15T22:09:16.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:09:16 smithi117 systemd[1]: Started Ceph mds.k for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3189558791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:17 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3189558791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:17 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3189558791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:17 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:18 smithi077 ceph-mon[30980]: Deploying daemon mds.j on smithi077 2024-09-15T22:09:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:18 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6832/2823149773,v1:172.21.15.117:6833/2823149773] up:boot 2024-09-15T22:09:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:18 smithi077 ceph-mon[30980]: fsmap 2 up:standby 2024-09-15T22:09:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:18 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:09:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:18 smithi196 ceph-mon[35109]: Deploying daemon mds.j on smithi077 2024-09-15T22:09:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:18 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6832/2823149773,v1:172.21.15.117:6833/2823149773] up:boot 2024-09-15T22:09:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:18 smithi196 ceph-mon[35109]: fsmap 2 up:standby 2024-09-15T22:09:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:18 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:09:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:18 smithi117 ceph-mon[35021]: Deploying daemon mds.j on smithi077 2024-09-15T22:09:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:18 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6832/2823149773,v1:172.21.15.117:6833/2823149773] up:boot 2024-09-15T22:09:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:18 smithi117 ceph-mon[35021]: fsmap 2 up:standby 2024-09-15T22:09:18.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:18 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:09:19.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:19 smithi077 ceph-mon[30980]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:19.332 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 systemd[1]: Starting Ceph mds.j for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:19.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:19 smithi196 ceph-mon[35109]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:19 smithi117 ceph-mon[35021]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:19.704 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 podman[69854]: 2024-09-15 22:09:19.350520616 +0000 UTC m=+0.015249372 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:19.704 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 podman[69854]: 2024-09-15 22:09:19.458839561 +0000 UTC m=+0.123568307 container create 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:09:19.704 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 podman[69854]: 2024-09-15 22:09:19.703377653 +0000 UTC m=+0.368106401 container init 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:09:19.852 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:19.974 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 podman[69854]: 2024-09-15 22:09:19.707378455 +0000 UTC m=+0.372107203 container start 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:09:19.975 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 ceph-mds[69915]: -- 172.21.15.77:0/2404878645 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56458dd37860 con 0x56458dd14800 2024-09-15T22:09:19.975 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j[69899]: starting mds.j at 2024-09-15T22:09:19.975 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 bash[69854]: 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 2024-09-15T22:09:19.975 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:09:19 smithi077 systemd[1]: Started Ceph mds.j for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- 172.21.15.77:0/2944815113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474101420 msgr2=0x7f347410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 --2- 172.21.15.77:0/2944815113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474101420 0x7f347410d950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3464009930 tx=0x7f346402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- 172.21.15.77:0/2944815113 shutdown_connections 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 --2- 172.21.15.77:0/2944815113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474101420 0x7f347410d950 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 --2- 172.21.15.77:0/2944815113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 0x7f3474100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 --2- 172.21.15.77:0/2944815113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474106a80 0x7f3474106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- 172.21.15.77:0/2944815113 >> 172.21.15.77:0/2944815113 conn(0x7f34740fc7a0 msgr2=0x7f34740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- 172.21.15.77:0/2944815113 shutdown_connections 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- 172.21.15.77:0/2944815113 wait complete. 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 Processor -- start 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.380+0000 7f3479daa640 1 -- start start 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 0x7f3474197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 0x7f34741977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3474110e80 con 0x7f3474106a80 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3474110d00 con 0x7f3474101420 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3474111000 con 0x7f3474100a80 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32964/0 (socket says 172.21.15.77:32964) 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 -- 172.21.15.77:0/1566667177 learned_addr learned my addr 172.21.15.77:0/1566667177 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3472ffd640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 0x7f34741977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f34737fe640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 0x7f3474197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 -- 172.21.15.77:0/1566667177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 msgr2=0x7f3474197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 0x7f3474197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 -- 172.21.15.77:0/1566667177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 msgr2=0x7f34741977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 0x7f34741977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 -- 172.21.15.77:0/1566667177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347419c230 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3472ffd640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 0x7f34741977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3473fff640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3464005ab0 tx=0x7f34640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346403f030 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347419c4c0 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464003ce0 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464035070 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.381+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347406a790 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.382+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34741021c0 con 0x7f3474106a80 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.383+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3464042030 con 0x7f3474106a80 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.384+0000 7f3470ff9640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 0x7f34600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.384+0000 7f34737fe640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 0x7f34600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.384+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f34640ef9c0 con 0x7f3474106a80 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.385+0000 7f34737fe640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 0x7f34600a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f346c007d50 tx=0x7f346c007730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.388+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34640f1e20 con 0x7f3474106a80 2024-09-15T22:09:20.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.599+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347406a9f0 con 0x7f3474106a80 2024-09-15T22:09:20.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.600+0000 7f3470ff9640 1 -- 172.21.15.77:0/1566667177 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f34640bcb80 con 0x7f3474106a80 2024-09-15T22:09:20.601 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:20.601 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":4,"btime":"2024-09-15T22:09:20:234771+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:09:20.602 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 4 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.601+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 msgr2=0x7f34600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.601+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 0x7f34600a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f346c007d50 tx=0x7f346c007730 comp rx=0 tx=0).stop 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.601+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 msgr2=0x7f347419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3464005ab0 tx=0x7f34640042f0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 shutdown_connections 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f34600a3240 0x7f34600a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3474106a80 0x7f347419bb50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474101420 0x7f34741977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 --2- 172.21.15.77:0/1566667177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3474100a80 0x7f3474197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 >> 172.21.15.77:0/1566667177 conn(0x7f34740fc7a0 msgr2=0x7f34740fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 shutdown_connections 2024-09-15T22:09:20.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:20.602+0000 7f3479daa640 1 -- 172.21.15.77:0/1566667177 wait complete. 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: Deploying daemon mds.d on smithi077 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6834/1898309607,v1:172.21.15.77:6835/1898309607] up:boot 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: fsmap 3 up:standby 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:09:21.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:20 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1566667177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: Deploying daemon mds.d on smithi077 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6834/1898309607,v1:172.21.15.77:6835/1898309607] up:boot 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: fsmap 3 up:standby 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:09:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:20 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1566667177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: Deploying daemon mds.d on smithi077 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6834/1898309607,v1:172.21.15.77:6835/1898309607] up:boot 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: fsmap 3 up:standby 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:09:21.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:20 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1566667177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:23.082 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:22 smithi077 systemd[1]: Starting Ceph mds.d for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:23.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:23 smithi077 ceph-mon[30980]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:23.423 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 podman[70267]: 2024-09-15 22:09:23.035395928 +0000 UTC m=+0.016128029 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:23.423 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 podman[70267]: 2024-09-15 22:09:23.169389895 +0000 UTC m=+0.150122036 container create 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-15T22:09:23.423 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 podman[70267]: 2024-09-15 22:09:23.422255326 +0000 UTC m=+0.402987428 container init 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-15T22:09:23.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:23.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:23 smithi196 ceph-mon[35109]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:23 smithi117 ceph-mon[35021]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:23.703 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:23.824 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 podman[70267]: 2024-09-15 22:09:23.425220955 +0000 UTC m=+0.405953051 container start 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-15T22:09:23.824 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 ceph-mds[70286]: -- 172.21.15.77:0/3732297977 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ac2b87860 con 0x563ac2b64800 2024-09-15T22:09:23.824 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d[70282]: starting mds.d at 2024-09-15T22:09:23.824 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 bash[70267]: 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 2024-09-15T22:09:23.824 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:09:23 smithi077 systemd[1]: Started Ceph mds.d for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 -- 172.21.15.77:0/2962215810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4106f10 msgr2=0x7f62a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/2962215810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4106f10 0x7f62a410ea90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f629c009ca0 tx=0x7f629c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 -- 172.21.15.77:0/2962215810 shutdown_connections 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/2962215810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4106f10 0x7f62a410ea90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/2962215810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4102790 0x7f62a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/2962215810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a4101de0 0x7f62a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 -- 172.21.15.77:0/2962215810 >> 172.21.15.77:0/2962215810 conn(0x7f62a40fd8c0 msgr2=0x7f62a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 -- 172.21.15.77:0/2962215810 shutdown_connections 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.198+0000 7f62ab4e9640 1 -- 172.21.15.77:0/2962215810 wait complete. 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 Processor -- start 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 -- start start 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a4101de0 0x7f62a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4106f10 0x7f62a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4111f70 con 0x7f62a4102790 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4111df0 con 0x7f62a4106f10 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62ab4e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41120f0 con 0x7f62a4101de0 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62a8a84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62a8a84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33004/0 (socket says 172.21.15.77:33004) 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.199+0000 7f62a8a84640 1 -- 172.21.15.77:0/1547997682 learned_addr learned my addr 172.21.15.77:0/1547997682 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a9a86640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4106f10 0x7f62a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 -- 172.21.15.77:0/1547997682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a4101de0 msgr2=0x7f62a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a4101de0 0x7f62a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 -- 172.21.15.77:0/1547997682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4106f10 msgr2=0x7f62a4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4106f10 0x7f62a4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 -- 172.21.15.77:0/1547997682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a41a68b0 con 0x7f62a4102790 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62a8a84640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f629400d950 tx=0x7f629400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294017030 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294004020 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a41a6ba0 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62940047a0 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.200+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41a7130 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.201+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a41021c0 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.202+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f62940041d0 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.203+0000 7f629a7fc640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 0x7f62900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.203+0000 7f62a9285640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 0x7f62900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.203+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f62940cf4c0 con 0x7f62a4102790 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.203+0000 7f62a9285640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 0x7f62900a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f62a000a8f0 tx=0x7f62a0005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.206+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f629409c700 con 0x7f62a4102790 2024-09-15T22:09:24.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.408+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a4075a20 con 0x7f62a4102790 2024-09-15T22:09:24.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.408+0000 7f629a7fc640 1 -- 172.21.15.77:0/1547997682 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f629409c000 con 0x7f62a4102790 2024-09-15T22:09:24.410 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:24.410 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":5,"btime":"2024-09-15T22:09:24:225905+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:09:24.410 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 5 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 msgr2=0x7f62900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 0x7f62900a5780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f62a000a8f0 tx=0x7f62a0005bf0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 msgr2=0x7f62a4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f629400d950 tx=0x7f629400de10 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 shutdown_connections 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f62900a32c0 0x7f62900a5780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a4106f10 0x7f62a4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62a4102790 0x7f62a4074a30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 --2- 172.21.15.77:0/1547997682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62a4101de0 0x7f62a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 >> 172.21.15.77:0/1547997682 conn(0x7f62a40fd8c0 msgr2=0x7f62a40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:24.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 shutdown_connections 2024-09-15T22:09:24.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:24.410+0000 7f62ab4e9640 1 -- 172.21.15.77:0/1547997682 wait complete. 2024-09-15T22:09:24.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:24.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: Deploying daemon mds.e on smithi117 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6836/1847869314,v1:172.21.15.77:6837/1847869314] up:boot 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: fsmap 4 up:standby 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:09:24.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1547997682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:24.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: Deploying daemon mds.e on smithi117 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6836/1847869314,v1:172.21.15.77:6837/1847869314] up:boot 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: fsmap 4 up:standby 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:09:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1547997682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: Deploying daemon mds.e on smithi117 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6836/1847869314,v1:172.21.15.77:6837/1847869314] up:boot 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: fsmap 4 up:standby 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:09:24.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1547997682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:25.820 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:25 smithi117 systemd[1]: Starting Ceph mds.e for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:26.108 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:25 smithi117 podman[63764]: 2024-09-15 22:09:25.721894851 +0000 UTC m=+0.015620024 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:26.108 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:25 smithi117 podman[63764]: 2024-09-15 22:09:25.86547107 +0000 UTC m=+0.159196260 container create 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-15T22:09:26.109 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 podman[63764]: 2024-09-15 22:09:26.107758205 +0000 UTC m=+0.401483375 container init 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:09:26.362 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 podman[63764]: 2024-09-15 22:09:26.110801847 +0000 UTC m=+0.404527018 container start 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-15T22:09:26.362 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 ceph-mds[63783]: -- 172.21.15.117:0/657829007 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccb9c57860 con 0x55ccb9c35400 2024-09-15T22:09:26.362 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: starting mds.e at 2024-09-15T22:09:26.363 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 bash[63764]: 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a 2024-09-15T22:09:26.363 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:09:26 smithi117 systemd[1]: Started Ceph mds.e for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:27.264 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:27.522 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6834/3112549785,v1:172.21.15.117:6835/3112549785] up:boot 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: fsmap 5 up:standby 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:27.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:27.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:27 smithi077 ceph-mon[30980]: Deploying daemon mds.f on smithi196 2024-09-15T22:09:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6834/3112549785,v1:172.21.15.117:6835/3112549785] up:boot 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: fsmap 5 up:standby 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:27.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:27 smithi196 ceph-mon[35109]: Deploying daemon mds.f on smithi196 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6834/3112549785,v1:172.21.15.117:6835/3112549785] up:boot 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: fsmap 5 up:standby 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:27.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:27.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:27.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:27.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:27 smithi117 ceph-mon[35021]: Deploying daemon mds.f on smithi196 2024-09-15T22:09:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 -- 172.21.15.77:0/2252357197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 msgr2=0x7f7980106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 --2- 172.21.15.77:0/2252357197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f7980106e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f797c009930 tx=0x7f797c02a420 comp rx=0 tx=0).stop 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 -- 172.21.15.77:0/2252357197 shutdown_connections 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 --2- 172.21.15.77:0/2252357197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980101420 0x7f798010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 --2- 172.21.15.77:0/2252357197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980100a80 0x7f7980100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 --2- 172.21.15.77:0/2252357197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f7980106e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 -- 172.21.15.77:0/2252357197 >> 172.21.15.77:0/2252357197 conn(0x7f79800fc7a0 msgr2=0x7f79800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 -- 172.21.15.77:0/2252357197 shutdown_connections 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.049+0000 7f79878ba640 1 -- 172.21.15.77:0/2252357197 wait complete. 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 Processor -- start 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 -- start start 2024-09-15T22:09:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980100a80 0x7f79801972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980101420 0x7f7980197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7980110e80 con 0x7f7980106a80 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7980110d00 con 0x7f7980101420 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f79878ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7980111000 con 0x7f7980100a80 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f7985e57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f7985e57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33024/0 (socket says 172.21.15.77:33024) 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f7985656640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980100a80 0x7f79801972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.050+0000 7f7985e57640 1 -- 172.21.15.77:0/2059463488 learned_addr learned my addr 172.21.15.77:0/2059463488 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7984e55640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980101420 0x7f7980197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 -- 172.21.15.77:0/2059463488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980100a80 msgr2=0x7f79801972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980100a80 0x7f79801972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 -- 172.21.15.77:0/2059463488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980101420 msgr2=0x7f7980197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980101420 0x7f7980197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 -- 172.21.15.77:0/2059463488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798019c270 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f7985e57640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f797800eca0 tx=0x7f797800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978004b20 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798019c560 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7978004cc0 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978017670 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.051+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798006a790 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.052+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7980100ee0 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.053+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7978005020 con 0x7f7980106a80 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.054+0000 7f79767fc640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 0x7f796c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.054+0000 7f7985656640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 0x7f796c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.054+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f79780d1700 con 0x7f7980106a80 2024-09-15T22:09:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.054+0000 7f7985656640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 0x7f796c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f797c009900 tx=0x7f797c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.057+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f797809e8c0 con 0x7f7980106a80 2024-09-15T22:09:28.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.258+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7980198620 con 0x7f7980106a80 2024-09-15T22:09:28.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.259+0000 7f79767fc640 1 -- 172.21.15.77:0/2059463488 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f7980198620 con 0x7f7980106a80 2024-09-15T22:09:28.261 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:28.261 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":6,"btime":"2024-09-15T22:09:26:188949+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-15T22:09:28.261 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 6 2024-09-15T22:09:28.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 msgr2=0x7f796c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 0x7f796c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f797c009900 tx=0x7f797c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 msgr2=0x7f798019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f797800eca0 tx=0x7f797800b690 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 shutdown_connections 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f796c0a3240 0x7f796c0a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7980106a80 0x7f798019bb90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7980101420 0x7f7980197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 --2- 172.21.15.77:0/2059463488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7980100a80 0x7f79801972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 >> 172.21.15.77:0/2059463488 conn(0x7f79800fc7a0 msgr2=0x7f79800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 shutdown_connections 2024-09-15T22:09:28.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:28.261+0000 7f79878ba640 1 -- 172.21.15.77:0/2059463488 wait complete. 2024-09-15T22:09:28.297 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 systemd[1]: Starting Ceph mds.f for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:28.565 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 podman[65456]: 2024-09-15 22:09:28.198181108 +0000 UTC m=+0.017020445 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:28.565 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 podman[65456]: 2024-09-15 22:09:28.319789489 +0000 UTC m=+0.138628818 container create a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T22:09:28.565 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 podman[65456]: 2024-09-15 22:09:28.564095435 +0000 UTC m=+0.382934765 container init a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-15T22:09:28.853 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 podman[65456]: 2024-09-15 22:09:28.566969008 +0000 UTC m=+0.385808340 container start a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-15T22:09:28.854 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 ceph-mds[65475]: -- 172.21.15.196:0/1879365468 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d415487860 con 0x55d415464800 2024-09-15T22:09:28.854 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-15T22:09:28.854 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 bash[65456]: a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 2024-09-15T22:09:28.854 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:09:28 smithi196 systemd[1]: Started Ceph mds.f for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:29.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:29.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2059463488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:29.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:29.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:29.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:29 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:29.416 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2059463488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:29.417 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:29 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:29.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2059463488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:29 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:30.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:30 smithi077 ceph-mon[30980]: Deploying daemon mds.c on smithi196 2024-09-15T22:09:30.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:30 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6834/889715770,v1:172.21.15.196:6835/889715770] up:boot 2024-09-15T22:09:30.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:30 smithi077 ceph-mon[30980]: fsmap 6 up:standby 2024-09-15T22:09:30.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:30 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:09:30.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:30 smithi196 ceph-mon[35109]: Deploying daemon mds.c on smithi196 2024-09-15T22:09:30.507 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:30 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6834/889715770,v1:172.21.15.196:6835/889715770] up:boot 2024-09-15T22:09:30.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:30 smithi196 ceph-mon[35109]: fsmap 6 up:standby 2024-09-15T22:09:30.508 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:30 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:09:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:30 smithi117 ceph-mon[35021]: Deploying daemon mds.c on smithi196 2024-09-15T22:09:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:30 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6834/889715770,v1:172.21.15.196:6835/889715770] up:boot 2024-09-15T22:09:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:30 smithi117 ceph-mon[35021]: fsmap 6 up:standby 2024-09-15T22:09:30.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:30 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:09:30.853 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:30 smithi196 systemd[1]: Starting Ceph mds.c for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:30.953 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:31.187 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:30 smithi196 podman[65759]: 2024-09-15 22:09:30.79733716 +0000 UTC m=+0.015746240 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:31.187 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:30 smithi196 podman[65759]: 2024-09-15 22:09:30.916485319 +0000 UTC m=+0.134894385 container create c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-15T22:09:31.187 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 podman[65759]: 2024-09-15 22:09:31.185971848 +0000 UTC m=+0.404380916 container init c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:09:31.203 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:31.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:31 smithi077 ceph-mon[30980]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:31 smithi196 ceph-mon[35109]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:31.604 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 podman[65759]: 2024-09-15 22:09:31.188964226 +0000 UTC m=+0.407373293 container start c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:09:31.604 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 ceph-mds[65778]: -- 172.21.15.196:0/1342320725 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56517bd4bc20 con 0x56517bd29000 2024-09-15T22:09:31.604 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: starting mds.c at 2024-09-15T22:09:31.604 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 bash[65759]: c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a 2024-09-15T22:09:31.604 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:09:31 smithi196 systemd[1]: Started Ceph mds.c for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:31 smithi117 ceph-mon[35021]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 -- 172.21.15.77:0/2256268040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c101420 msgr2=0x7fc73c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 --2- 172.21.15.77:0/2256268040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c101420 0x7fc73c10d950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc734009ca0 tx=0x7fc73402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 -- 172.21.15.77:0/2256268040 shutdown_connections 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 --2- 172.21.15.77:0/2256268040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c101420 0x7fc73c10d950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 --2- 172.21.15.77:0/2256268040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c100a80 0x7fc73c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 --2- 172.21.15.77:0/2256268040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c106a80 0x7fc73c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 -- 172.21.15.77:0/2256268040 >> 172.21.15.77:0/2256268040 conn(0x7fc73c0fc7a0 msgr2=0x7fc73c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:31.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 -- 172.21.15.77:0/2256268040 shutdown_connections 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.717+0000 7fc741905640 1 -- 172.21.15.77:0/2256268040 wait complete. 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 Processor -- start 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 -- start start 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c101420 0x7fc73c10e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c106a80 0x7fc73c10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c111fa0 con 0x7fc73c100a80 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c111e20 con 0x7fc73c106a80 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc741905640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c112120 con 0x7fc73c101420 2024-09-15T22:09:31.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc73affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc73affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35844/0 (socket says 172.21.15.77:35844) 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc73affd640 1 -- 172.21.15.77:0/4270220311 learned_addr learned my addr 172.21.15.77:0/4270220311 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc7327fc640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c101420 0x7fc73c10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.718+0000 7fc73b7fe640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c106a80 0x7fc73c10eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 -- 172.21.15.77:0/4270220311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c101420 msgr2=0x7fc73c10e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c101420 0x7fc73c10e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 -- 172.21.15.77:0/4270220311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c106a80 msgr2=0x7fc73c10eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c106a80 0x7fc73c10eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 -- 172.21.15.77:0/4270220311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c1a6900 con 0x7fc73c100a80 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc73affd640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b4a0 tx=0x7fc72c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c014070 con 0x7fc73c100a80 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c010af0 con 0x7fc73c100a80 2024-09-15T22:09:31.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c1a6bf0 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c017d10 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.719+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c1a7130 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.720+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73c100ee0 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.721+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc72c017070 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.722+0000 7fc740903640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 0x7fc71c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.722+0000 7fc7327fc640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 0x7fc71c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.722+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc72c0ce720 con 0x7fc73c100a80 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.722+0000 7fc7327fc640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 0x7fc71c0a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc73c10fb70 tx=0x7fc72400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:31.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.726+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc72c09b290 con 0x7fc73c100a80 2024-09-15T22:09:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.924+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc73c1a7430 con 0x7fc73c100a80 2024-09-15T22:09:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.925+0000 7fc740903640 1 -- 172.21.15.77:0/4270220311 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7fc72c0b9e20 con 0x7fc73c100a80 2024-09-15T22:09:31.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:31.927 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":7,"btime":"2024-09-15T22:09:29:193254+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-15T22:09:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 7 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 msgr2=0x7fc71c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 0x7fc71c0a5780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc73c10fb70 tx=0x7fc72400b410 comp rx=0 tx=0).stop 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 msgr2=0x7fc73c10e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b4a0 tx=0x7fc72c00b960 comp rx=0 tx=0).stop 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 shutdown_connections 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc71c0a32c0 0x7fc71c0a5780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73c106a80 0x7fc73c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc73c101420 0x7fc73c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 --2- 172.21.15.77:0/4270220311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc73c100a80 0x7fc73c10e0e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.927+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 >> 172.21.15.77:0/4270220311 conn(0x7fc73c0fc7a0 msgr2=0x7fc73c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.928+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 shutdown_connections 2024-09-15T22:09:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:31.928+0000 7fc741905640 1 -- 172.21.15.77:0/4270220311 wait complete. 2024-09-15T22:09:32.295 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: Deploying daemon mds.h on smithi117 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4270220311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:boot 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: fsmap 7 up:standby 2024-09-15T22:09:32.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:32 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:09:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: Deploying daemon mds.h on smithi117 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4270220311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:boot 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: fsmap 7 up:standby 2024-09-15T22:09:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:32 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:09:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: Deploying daemon mds.h on smithi117 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4270220311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:32.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:boot 2024-09-15T22:09:32.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: fsmap 7 up:standby 2024-09-15T22:09:32.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:32 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:09:33.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:33 smithi117 ceph-mon[35021]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:33.535 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 systemd[1]: Starting Ceph mds.h for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:33.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:33 smithi077 ceph-mon[30980]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:33 smithi196 ceph-mon[35109]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:33.787 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 podman[64066]: 2024-09-15 22:09:33.436283826 +0000 UTC m=+0.015384923 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:33.788 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 podman[64066]: 2024-09-15 22:09:33.542794511 +0000 UTC m=+0.121895648 container create 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:09:33.788 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 podman[64066]: 2024-09-15 22:09:33.786956739 +0000 UTC m=+0.366057834 container init 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-15T22:09:34.112 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 podman[64066]: 2024-09-15 22:09:33.789833423 +0000 UTC m=+0.368934519 container start 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:09:34.113 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 ceph-mds[64086]: -- 172.21.15.117:0/3839215501 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe310bd860 con 0x55fe3109b000 2024-09-15T22:09:34.113 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: starting mds.h at 2024-09-15T22:09:34.113 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 bash[64066]: 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 2024-09-15T22:09:34.113 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:09:33 smithi117 systemd[1]: Started Ceph mds.h for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:34.627 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:34.884 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:35.078 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.078 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.078 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.078 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:35.078 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: Deploying daemon mds.i on smithi196 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:boot 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: fsmap 8 up:standby 2024-09-15T22:09:35.079 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:34 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: Deploying daemon mds.i on smithi196 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:boot 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: fsmap 8 up:standby 2024-09-15T22:09:35.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:34 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:09:35.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: Deploying daemon mds.i on smithi196 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:boot 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: fsmap 8 up:standby 2024-09-15T22:09:35.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:34 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 -- 172.21.15.77:0/997190135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80102d40 msgr2=0x7f5e80103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/997190135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80102d40 0x7f5e80103120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009900 tx=0x7f5e7c02a480 comp rx=0 tx=0).stop 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 -- 172.21.15.77:0/997190135 shutdown_connections 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/997190135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e8010c170 0x7f5e8010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/997190135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80103660 0x7f5e8010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/997190135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80102d40 0x7f5e80103120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 -- 172.21.15.77:0/997190135 >> 172.21.15.77:0/997190135 conn(0x7f5e800fc7a0 msgr2=0x7f5e800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 -- 172.21.15.77:0/997190135 shutdown_connections 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.358+0000 7f5e87cd7640 1 -- 172.21.15.77:0/997190135 wait complete. 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 Processor -- start 2024-09-15T22:09:35.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 -- start start 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80102d40 0x7f5e801009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e8010c170 0x7f5e80101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80111fb0 con 0x7f5e80103660 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80111e30 con 0x7f5e8010c170 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e87cd7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112130 con 0x7f5e80102d40 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e85272640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e85272640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35866/0 (socket says 172.21.15.77:35866) 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e85272640 1 -- 172.21.15.77:0/4202939954 learned_addr learned my addr 172.21.15.77:0/4202939954 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.359+0000 7f5e85a73640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80102d40 0x7f5e801009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 -- 172.21.15.77:0/4202939954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80102d40 msgr2=0x7f5e801009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80102d40 0x7f5e801009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 -- 172.21.15.77:0/4202939954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e8010c170 msgr2=0x7f5e80101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:35.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e8010c170 0x7f5e80101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 -- 172.21.15.77:0/4202939954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e801a68b0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e85272640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c970 tx=0x7f5e7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70007bb0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e70007d50 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e801a6ba0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70018820 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.360+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a70e0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.361+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e80103120 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.362+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5e700189c0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.363+0000 7f5e76ffd640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.363+0000 7f5e85a73640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.363+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f5e700cf5d0 con 0x7f5e80103660 2024-09-15T22:09:35.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.363+0000 7f5e85a73640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0096c0 tx=0x7f5e7c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:35.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.367+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e70014030 con 0x7f5e80103660 2024-09-15T22:09:35.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.565+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e80106c50 con 0x7f5e80103660 2024-09-15T22:09:35.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.565+0000 7f5e76ffd640 1 -- 172.21.15.77:0/4202939954 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f5e7000de20 con 0x7f5e80103660 2024-09-15T22:09:35.567 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:35.567 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":9,"btime":"2024-09-15T22:09:34:307408+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:09:35.568 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 9 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 msgr2=0x7f5e6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0096c0 tx=0x7f5e7c031490 comp rx=0 tx=0).stop 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 msgr2=0x7f5e80100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c970 tx=0x7f5e7000ce30 comp rx=0 tx=0).stop 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 shutdown_connections 2024-09-15T22:09:35.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e6c0a3240 0x7f5e6c0a5700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e8010c170 0x7f5e80101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e80103660 0x7f5e80100ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.567+0000 7f5e87cd7640 1 --2- 172.21.15.77:0/4202939954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e80102d40 0x7f5e801009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.568+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 >> 172.21.15.77:0/4202939954 conn(0x7f5e800fc7a0 msgr2=0x7f5e800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.568+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 shutdown_connections 2024-09-15T22:09:35.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:35.568+0000 7f5e87cd7640 1 -- 172.21.15.77:0/4202939954 wait complete. 2024-09-15T22:09:35.953 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:35 smithi196 systemd[1]: Starting Ceph mds.i for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:36.211 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:35 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4202939954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:36.325 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4202939954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:36.326 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 podman[66061]: 2024-09-15 22:09:35.935354238 +0000 UTC m=+0.015815340 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:36.326 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 podman[66061]: 2024-09-15 22:09:36.054920804 +0000 UTC m=+0.135381894 container create 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T22:09:36.326 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 podman[66061]: 2024-09-15 22:09:36.324709345 +0000 UTC m=+0.405170434 container init 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:09:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:35 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4202939954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:36.604 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 podman[66061]: 2024-09-15 22:09:36.327859751 +0000 UTC m=+0.408320849 container start 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T22:09:36.604 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mds[66079]: -- 172.21.15.196:0/3286400117 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636961c5860 con 0x5636961a2800 2024-09-15T22:09:36.604 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: starting mds.i at 2024-09-15T22:09:36.604 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 bash[66061]: 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb 2024-09-15T22:09:36.604 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:09:36 smithi196 systemd[1]: Started Ceph mds.i for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:37.164 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:36 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:37.400 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:36 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:37.401 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:37.402 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:36 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:38.214 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:37 smithi077 ceph-mon[30980]: Deploying daemon mds.g on smithi077 2024-09-15T22:09:38.215 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:37 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:boot 2024-09-15T22:09:38.215 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:37 smithi077 ceph-mon[30980]: fsmap 9 up:standby 2024-09-15T22:09:38.215 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:37 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:09:38.242 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:37 smithi196 ceph-mon[35109]: Deploying daemon mds.g on smithi077 2024-09-15T22:09:38.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:37 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:boot 2024-09-15T22:09:38.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:37 smithi196 ceph-mon[35109]: fsmap 9 up:standby 2024-09-15T22:09:38.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:37 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:09:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:37 smithi117 ceph-mon[35021]: Deploying daemon mds.g on smithi077 2024-09-15T22:09:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:37 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:boot 2024-09-15T22:09:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:37 smithi117 ceph-mon[35021]: fsmap 9 up:standby 2024-09-15T22:09:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:37 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:09:38.582 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 systemd[1]: Starting Ceph mds.g for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:38.900 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 podman[71055]: 2024-09-15 22:09:38.522107665 +0000 UTC m=+0.016120190 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:38.900 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 podman[71055]: 2024-09-15 22:09:38.646352873 +0000 UTC m=+0.140365386 container create 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-15T22:09:38.901 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 podman[71055]: 2024-09-15 22:09:38.899391721 +0000 UTC m=+0.393404233 container init 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-15T22:09:39.090 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:39.211 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:38 smithi077 ceph-mon[30980]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:39.211 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 podman[71055]: 2024-09-15 22:09:38.902456575 +0000 UTC m=+0.396469090 container start 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:09:39.211 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 ceph-mds[71097]: -- 172.21.15.77:0/3163146675 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eef4359860 con 0x55eef4337000 2024-09-15T22:09:39.211 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:38 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: starting mds.g at 2024-09-15T22:09:39.212 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:39 smithi077 bash[71055]: 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de 2024-09-15T22:09:39.212 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:09:39 smithi077 systemd[1]: Started Ceph mds.g for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:39.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:38 smithi196 ceph-mon[35109]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:39.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:38 smithi117 ceph-mon[35021]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- 172.21.15.77:0/4138912851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102570 msgr2=0x7ff1601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 --2- 172.21.15.77:0/4138912851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102570 0x7ff1601029b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff154009900 tx=0x7ff15402a480 comp rx=0 tx=0).stop 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- 172.21.15.77:0/4138912851 shutdown_connections 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 --2- 172.21.15.77:0/4138912851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102ef0 0x7ff16010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 --2- 172.21.15.77:0/4138912851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102570 0x7ff1601029b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 --2- 172.21.15.77:0/4138912851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- 172.21.15.77:0/4138912851 >> 172.21.15.77:0/4138912851 conn(0x7ff1600fc7d0 msgr2=0x7ff1600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- 172.21.15.77:0/4138912851 shutdown_connections 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- 172.21.15.77:0/4138912851 wait complete. 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 Processor -- start 2024-09-15T22:09:39.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.588+0000 7ff16652b640 1 -- start start 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102570 0x7ff160198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff160110fa0 con 0x7ff160102ef0 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff160110e20 con 0x7ff160102570 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff16652b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff160111120 con 0x7ff160100e20 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52370/0 (socket says 172.21.15.77:52370) 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 -- 172.21.15.77:0/2110844832 learned_addr learned my addr 172.21.15.77:0/2110844832 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff15ffff640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff15f7fe640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102570 0x7ff160198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 -- 172.21.15.77:0/2110844832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 msgr2=0x7ff160198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:39.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 -- 172.21.15.77:0/2110844832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102570 msgr2=0x7ff160198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102570 0x7ff160198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 -- 172.21.15.77:0/2110844832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16019d5c0 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff15ffff640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff164ac8640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff158009fd0 tx=0x7ff15800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.589+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158017070 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.590+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16019d8b0 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.590+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158003e50 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.590+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580106c0 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.590+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1601a5190 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.591+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff160101200 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.592+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff158003970 con 0x7ff160102ef0 2024-09-15T22:09:39.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.592+0000 7ff15d7fa640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 0x7ff1480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:39.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.592+0000 7ff15ffff640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 0x7ff1480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:39.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.593+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff1580cf2b0 con 0x7ff160102ef0 2024-09-15T22:09:39.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.593+0000 7ff15ffff640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 0x7ff1480a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff150009930 tx=0x7ff150008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:39.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.596+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff15809c470 con 0x7ff160102ef0 2024-09-15T22:09:39.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.805+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff16010a0a0 con 0x7ff160102ef0 2024-09-15T22:09:39.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.806+0000 7ff15d7fa640 1 -- 172.21.15.77:0/2110844832 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7ff15809bd70 con 0x7ff160102ef0 2024-09-15T22:09:39.808 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:39.808 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":11,"btime":"2024-09-15T22:09:38:960714+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:09:39.808 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 11 2024-09-15T22:09:39.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 msgr2=0x7ff1480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 0x7ff1480a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff150009930 tx=0x7ff150008040 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 msgr2=0x7ff16019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff158009fd0 tx=0x7ff15800b260 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 shutdown_connections 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff1480a3240 0x7ff1480a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff160102ef0 0x7ff16019cee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff160102570 0x7ff160198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 --2- 172.21.15.77:0/2110844832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff160100e20 0x7ff160198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 >> 172.21.15.77:0/2110844832 conn(0x7ff1600fc7d0 msgr2=0x7ff16006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 shutdown_connections 2024-09-15T22:09:39.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:39.808+0000 7ff16652b640 1 -- 172.21.15.77:0/2110844832 wait complete. 2024-09-15T22:09:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6838/2629133747,v1:172.21.15.77:6839/2629133747] up:boot 2024-09-15T22:09:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: fsmap 10 up:standby 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: Deploying daemon mds.a on smithi077 2024-09-15T22:09:40.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:39 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2110844832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6838/2629133747,v1:172.21.15.77:6839/2629133747] up:boot 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: fsmap 10 up:standby 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: Deploying daemon mds.a on smithi077 2024-09-15T22:09:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:39 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2110844832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6838/2629133747,v1:172.21.15.77:6839/2629133747] up:boot 2024-09-15T22:09:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: fsmap 10 up:standby 2024-09-15T22:09:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:09:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: Deploying daemon mds.a on smithi077 2024-09-15T22:09:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:39 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2110844832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:41.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:40 smithi077 ceph-mon[30980]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:41.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:40 smithi196 ceph-mon[35109]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:40 smithi117 ceph-mon[35021]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:42.332 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 systemd[1]: Starting Ceph mds.a for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:09:42.599 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:42.649 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 podman[71451]: 2024-09-15 22:09:42.296772775 +0000 UTC m=+0.015765086 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:09:42.649 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 podman[71451]: 2024-09-15 22:09:42.429226876 +0000 UTC m=+0.148219176 container create 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T22:09:42.649 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 podman[71451]: 2024-09-15 22:09:42.648408495 +0000 UTC m=+0.367400794 container init 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-15T22:09:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:42.992 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 podman[71451]: 2024-09-15 22:09:42.651449984 +0000 UTC m=+0.370442283 container start 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T22:09:42.992 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 ceph-mds[71487]: -- 172.21.15.77:0/4150451734 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d10289860 con 0x564d10267000 2024-09-15T22:09:42.992 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: starting mds.a at 2024-09-15T22:09:42.992 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 bash[71451]: 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 2024-09-15T22:09:42.992 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:09:42 smithi077 systemd[1]: Started Ceph mds.a for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:43.295 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:43.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:43 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:43.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 -- 172.21.15.77:0/3922182571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0100a80 msgr2=0x7f83d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 --2- 172.21.15.77:0/3922182571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0100a80 0x7f83d0100ee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009930 tx=0x7f83bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:09:43.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 -- 172.21.15.77:0/3922182571 shutdown_connections 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 --2- 172.21.15.77:0/3922182571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0101420 0x7f83d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 --2- 172.21.15.77:0/3922182571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0100a80 0x7f83d0100ee0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 --2- 172.21.15.77:0/3922182571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 -- 172.21.15.77:0/3922182571 >> 172.21.15.77:0/3922182571 conn(0x7f83d00fc7a0 msgr2=0x7f83d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 -- 172.21.15.77:0/3922182571 shutdown_connections 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.374+0000 7f83d6e35640 1 -- 172.21.15.77:0/3922182571 wait complete. 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 Processor -- start 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 -- start start 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0100a80 0x7f83d0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110ea0 con 0x7f83d0101420 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110d20 con 0x7f83d0100a80 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d6e35640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0111020 con 0x7f83d0106a80 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52400/0 (socket says 172.21.15.77:52400) 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83cffff640 1 -- 172.21.15.77:0/2969596 learned_addr learned my addr 172.21.15.77:0/2969596 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.375+0000 7f83d4bd1640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0100a80 0x7f83d0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83d53d2640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 -- 172.21.15.77:0/2969596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 msgr2=0x7f83d019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 -- 172.21.15.77:0/2969596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0100a80 msgr2=0x7f83d0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0100a80 0x7f83d0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 -- 172.21.15.77:0/2969596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d019d3d0 con 0x7f83d0101420 2024-09-15T22:09:43.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83d53d2640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cffff640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009a50 tx=0x7f83bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc03f030 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83bc003ad0 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d019d660 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc035070 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.376+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a4fa0 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.377+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d0100ee0 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.378+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f83bc042030 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.379+0000 7f83cdffb640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 0x7f83c00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.379+0000 7f83d4bd1640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 0x7f83c00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.379+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f83bc0efbb0 con 0x7f83d0101420 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.380+0000 7f83d4bd1640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 0x7f83c00a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f83c400abe0 tx=0x7f83c400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.384+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83bc0bcd70 con 0x7f83d0101420 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:43.486 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:43 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:43.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.590+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83d010a050 con 0x7f83d0101420 2024-09-15T22:09:43.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.590+0000 7f83cdffb640 1 -- 172.21.15.77:0/2969596 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7f83bc0bc670 con 0x7f83d0101420 2024-09-15T22:09:43.592 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":12,"btime":"2024-09-15T22:09:43:229122+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:09:43.592 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 12 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 msgr2=0x7f83c00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 0x7f83c00a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f83c400abe0 tx=0x7f83c400a2d0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 msgr2=0x7f83d0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009a50 tx=0x7f83bc004270 comp rx=0 tx=0).stop 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 shutdown_connections 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f83c00a32c0 0x7f83c00a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d0106a80 0x7f83d019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83d0101420 0x7f83d0198960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 --2- 172.21.15.77:0/2969596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83d0100a80 0x7f83d0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.592+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 >> 172.21.15.77:0/2969596 conn(0x7f83d00fc7a0 msgr2=0x7f83d006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.593+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 shutdown_connections 2024-09-15T22:09:43.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:43.593+0000 7f83d6e35640 1 -- 172.21.15.77:0/2969596 wait complete. 2024-09-15T22:09:43.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:43.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:09:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:09:43.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:43 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:44.284 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:44 smithi077 ceph-mon[30980]: Deploying daemon mds.b on smithi117 2024-09-15T22:09:44.284 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:44 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:boot 2024-09-15T22:09:44.284 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:44 smithi077 ceph-mon[30980]: fsmap 11 up:standby 2024-09-15T22:09:44.285 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:44 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:09:44.285 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:44 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2969596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:44.510 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:44 smithi117 ceph-mon[35021]: Deploying daemon mds.b on smithi117 2024-09-15T22:09:44.511 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:44 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:boot 2024-09-15T22:09:44.511 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:44 smithi117 ceph-mon[35021]: fsmap 11 up:standby 2024-09-15T22:09:44.511 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:44 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:09:44.511 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:44 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2969596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:44 smithi196 ceph-mon[35109]: Deploying daemon mds.b on smithi117 2024-09-15T22:09:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:44 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:boot 2024-09-15T22:09:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:44 smithi196 ceph-mon[35109]: fsmap 11 up:standby 2024-09-15T22:09:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:44 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:09:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:44 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2969596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:45 smithi077 ceph-mon[30980]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:45 smithi196 ceph-mon[35109]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:45 smithi117 ceph-mon[35021]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:45.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:09:45 smithi117 systemd[1]: Started Ceph mds.b for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:09:46.316 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:46.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:46.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:boot 2024-09-15T22:09:46.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: fsmap 12 up:standby 2024-09-15T22:09:46.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:46 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:boot 2024-09-15T22:09:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: fsmap 12 up:standby 2024-09-15T22:09:46.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:46 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:boot 2024-09-15T22:09:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: fsmap 12 up:standby 2024-09-15T22:09:46.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:46 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:09:46.896 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:47.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 -- 172.21.15.77:0/3889894471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180100a50 msgr2=0x7f91801057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 --2- 172.21.15.77:0/3889894471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180100a50 0x7f91801057e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9178009ca0 tx=0x7f917802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 -- 172.21.15.77:0/3889894471 shutdown_connections 2024-09-15T22:09:47.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 --2- 172.21.15.77:0/3889894471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180100a50 0x7f91801057e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 --2- 172.21.15.77:0/3889894471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100090 0x7f9180100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 --2- 172.21.15.77:0/3889894471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180106b70 0x7f9180106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 -- 172.21.15.77:0/3889894471 >> 172.21.15.77:0/3889894471 conn(0x7f9180077210 msgr2=0x7f91800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 -- 172.21.15.77:0/3889894471 shutdown_connections 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.411+0000 7f9185ada640 1 -- 172.21.15.77:0/3889894471 wait complete. 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 Processor -- start 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 -- start start 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180100090 0x7f91801a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100a50 0x7f91801a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91801185c0 con 0x7f9180106b70 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9180118440 con 0x7f9180100a50 2024-09-15T22:09:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f9185ada640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9180118740 con 0x7f9180100090 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f917ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f917ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52408/0 (socket says 172.21.15.77:52408) 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f917ffff640 1 -- 172.21.15.77:0/2141834624 learned_addr learned my addr 172.21.15.77:0/2141834624 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f917f7fe640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180100090 0x7f91801a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.412+0000 7f917effd640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100a50 0x7f91801a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 -- 172.21.15.77:0/2141834624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180100090 msgr2=0x7f91801a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180100090 0x7f91801a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 -- 172.21.15.77:0/2141834624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100a50 msgr2=0x7f91801a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100a50 0x7f91801a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 -- 172.21.15.77:0/2141834624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918019ef70 con 0x7f9180106b70 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917ffff640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9178030c30 tx=0x7f9178030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178003b70 con 0x7f9180106b70 2024-09-15T22:09:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918019f200 con 0x7f9180106b70 2024-09-15T22:09:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9178030e80 con 0x7f9180106b70 2024-09-15T22:09:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780046f0 con 0x7f9180106b70 2024-09-15T22:09:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.413+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801ab4b0 con 0x7f9180106b70 2024-09-15T22:09:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.414+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91801012b0 con 0x7f9180106b70 2024-09-15T22:09:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.417+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9178031450 con 0x7f9180106b70 2024-09-15T22:09:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.417+0000 7f917cff9640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 0x7f91680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.418+0000 7f917f7fe640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 0x7f91680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.418+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f91780f0760 con 0x7f9180106b70 2024-09-15T22:09:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.418+0000 7f917f7fe640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 0x7f91680a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9170005fd0 tx=0x7f917000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.421+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91780bd920 con 0x7f9180106b70 2024-09-15T22:09:47.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.664+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91801ab750 con 0x7f9180106b70 2024-09-15T22:09:47.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.665+0000 7f917cff9640 1 -- 172.21.15.77:0/2141834624 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10536 (secure 0 0 0) 0x7f91780bd220 con 0x7f9180106b70 2024-09-15T22:09:47.668 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:47.668 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":13,"btime":"2024-09-15T22:09:46:241645+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-15T22:09:47.668 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 13 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 msgr2=0x7f91680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 0x7f91680a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9170005fd0 tx=0x7f917000b390 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 msgr2=0x7f918019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9178030c30 tx=0x7f9178030c60 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 shutdown_connections 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91680a3240 0x7f91680a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180106b70 0x7f918019e7d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9180100a50 0x7f91801a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 --2- 172.21.15.77:0/2141834624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9180100090 0x7f91801a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 >> 172.21.15.77:0/2141834624 conn(0x7f9180077210 msgr2=0x7f91801037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 shutdown_connections 2024-09-15T22:09:47.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:47.667+0000 7f9185ada640 1 -- 172.21.15.77:0/2141834624 wait complete. 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.411 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:48.412 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:48.412 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.412 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2141834624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.412 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:48 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2141834624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.433 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:48 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:48.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:48.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:48.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2141834624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:48 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:48.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-15T22:09:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:49.487 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:49 smithi077 ceph-mon[30980]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:49.487 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:49 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.488 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:49 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:49 smithi196 ceph-mon[35109]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:49 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:49 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:49 smithi117 ceph-mon[35021]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:49 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:49 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 -- 172.21.15.77:0/3957451145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90106f10 msgr2=0x7f5e9010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/3957451145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90106f10 0x7f5e9010ea90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009ca0 tx=0x7f5e8802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 -- 172.21.15.77:0/3957451145 shutdown_connections 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/3957451145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90106f10 0x7f5e9010ea90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/3957451145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90102790 0x7f5e901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/3957451145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 -- 172.21.15.77:0/3957451145 >> 172.21.15.77:0/3957451145 conn(0x7f5e900fd8c0 msgr2=0x7f5e900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 -- 172.21.15.77:0/3957451145 shutdown_connections 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 -- 172.21.15.77:0/3957451145 wait complete. 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.734+0000 7f5e94aaf640 1 Processor -- start 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 -- start start 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90102790 0x7f5e90198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 0x7f5e9019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90111f70 con 0x7f5e90102790 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90111df0 con 0x7f5e90101de0 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e94aaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e901120f0 con 0x7f5e90106f10 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38616/0 (socket says 172.21.15.77:38616) 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 0x7f5e9019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 -- 172.21.15.77:0/2576101850 learned_addr learned my addr 172.21.15.77:0/2576101850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8dd9b640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90102790 0x7f5e90198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 -- 172.21.15.77:0/2576101850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 msgr2=0x7f5e9019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 0x7f5e9019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 -- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90102790 msgr2=0x7f5e90198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90102790 0x7f5e90198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8e59c640 1 -- 172.21.15.77:0/2576101850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9019d3a0 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.735+0000 7f5e8ed9d640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 0x7f5e9019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e8e59c640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80017070 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9019d690 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80009c30 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000f6b0 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.736+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e90074420 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.737+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e901021c0 con 0x7f5e90101de0 2024-09-15T22:09:49.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.738+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5e800109d0 con 0x7f5e90101de0 2024-09-15T22:09:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.739+0000 7f5e877fe640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 0x7f5e7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.739+0000 7f5e8dd9b640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 0x7f5e7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.739+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f5e800ceff0 con 0x7f5e90101de0 2024-09-15T22:09:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.739+0000 7f5e8dd9b640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 0x7f5e7c0a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e900fdce0 tx=0x7f5e78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.742+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e80009de0 con 0x7f5e90101de0 2024-09-15T22:09:49.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.921+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5e9010a0f0 con 0x7f5e90101de0 2024-09-15T22:09:49.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.921+0000 7f5e877fe640 1 -- 172.21.15.77:0/2576101850 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f5e8009bb60 con 0x7f5e90101de0 2024-09-15T22:09:49.925 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:09:49.925 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2024-09-15T22:09:49.925 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:09:49.925 INFO:teuthology.orchestra.run.smithi077.stdout: "poolnum": 1, 2024-09-15T22:09:49.925 INFO:teuthology.orchestra.run.smithi077.stdout: "poolname": ".mgr" 2024-09-15T22:09:49.926 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:09:49.926 INFO:teuthology.orchestra.run.smithi077.stdout:] 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 msgr2=0x7f5e7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 0x7f5e7c0a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e900fdce0 tx=0x7f5e78005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 msgr2=0x7f5e901983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).stop 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 shutdown_connections 2024-09-15T22:09:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5e7c0a3240 0x7f5e7c0a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e90106f10 0x7f5e9019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e90102790 0x7f5e90198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 --2- 172.21.15.77:0/2576101850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e90101de0 0x7f5e901983f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.923+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 >> 172.21.15.77:0/2576101850 conn(0x7f5e900fd8c0 msgr2=0x7f5e900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.924+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 shutdown_connections 2024-09-15T22:09:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:49.924+0000 7f5e94aaf640 1 -- 172.21.15.77:0/2576101850 wait complete. 2024-09-15T22:09:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:50 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:50 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2576101850' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:09:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:50 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:50 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2576101850' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:09:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:50 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:50 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:50 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:50 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2576101850' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:09:50.657 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T22:09:50.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T22:09:50.912 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 -- 172.21.15.77:0/2973252160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102570 msgr2=0x7f75001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:51.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 --2- 172.21.15.77:0/2973252160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102570 0x7f75001029b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009900 tx=0x7f74f002a480 comp rx=0 tx=0).stop 2024-09-15T22:09:51.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 -- 172.21.15.77:0/2973252160 shutdown_connections 2024-09-15T22:09:51.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 --2- 172.21.15.77:0/2973252160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102ef0 0x7f750010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:51.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 --2- 172.21.15.77:0/2973252160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102570 0x7f75001029b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 --2- 172.21.15.77:0/2973252160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 0x7f7500101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 -- 172.21.15.77:0/2973252160 >> 172.21.15.77:0/2973252160 conn(0x7f75000fc7d0 msgr2=0x7f75000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 -- 172.21.15.77:0/2973252160 shutdown_connections 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.490+0000 7f7507323640 1 -- 172.21.15.77:0/2973252160 wait complete. 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 Processor -- start 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 -- start start 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 0x7f750019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102570 0x7f750019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7500110fa0 con 0x7f7500102ef0 2024-09-15T22:09:51.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7500110e20 con 0x7f7500100e20 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f7507323640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7500111120 con 0x7f7500102570 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35906/0 (socket says 172.21.15.77:35906) 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 -- 172.21.15.77:0/1745190911 learned_addr learned my addr 172.21.15.77:0/1745190911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75050bf640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 0x7f750019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75048be640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102570 0x7f750019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 -- 172.21.15.77:0/1745190911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102570 msgr2=0x7f750019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102570 0x7f750019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 -- 172.21.15.77:0/1745190911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 msgr2=0x7f750019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:51.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 0x7f750019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.491+0000 7f75058c0640 1 -- 172.21.15.77:0/1745190911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750019f660 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f75058c0640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b970 tx=0x7f74f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8019070 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f8014440 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750019f950 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80075d0 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.492+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75001a7230 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.493+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7500101200 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.494+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f74f801c070 con 0x7f7500102ef0 2024-09-15T22:09:51.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.495+0000 7f74f67fc640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 0x7f74ec0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:51.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.495+0000 7f75050bf640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 0x7f74ec0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:51.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.495+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f74f80cf010 con 0x7f7500102ef0 2024-09-15T22:09:51.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.495+0000 7f75050bf640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 0x7f74ec0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00abe0 tx=0x7f74fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:51.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.498+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74f809c130 con 0x7f7500102ef0 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:51.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:09:51.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:51.665+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f75001a7460 con 0x7f7500102ef0 2024-09-15T22:09:52.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.169+0000 7f74f67fc640 1 -- 172.21.15.77:0/1745190911 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f74f809ba30 con 0x7f7500102ef0 2024-09-15T22:09:52.172 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' created 2024-09-15T22:09:52.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 msgr2=0x7f74ec0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 0x7f74ec0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00abe0 tx=0x7f74fc00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 msgr2=0x7f750019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b970 tx=0x7f74f800bd70 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 shutdown_connections 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74ec0a32c0 0x7f74ec0a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7500102ef0 0x7f750019ef80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7500102570 0x7f750019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 --2- 172.21.15.77:0/1745190911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7500100e20 0x7f750019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.172+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 >> 172.21.15.77:0/1745190911 conn(0x7f75000fc7d0 msgr2=0x7f7500069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:52.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.173+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 shutdown_connections 2024-09-15T22:09:52.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:52.174+0000 7f7507323640 1 -- 172.21.15.77:0/1745190911 wait complete. 2024-09-15T22:09:52.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:52 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:09:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:52 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:09:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:52 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:09:52.909 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-15T22:09:53.154 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:53.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:53 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:09:53.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:53 smithi077 ceph-mon[30980]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:09:53.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:53 smithi077 ceph-mon[30980]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:53 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:09:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:53 smithi196 ceph-mon[35109]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:09:53.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:53 smithi196 ceph-mon[35109]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:53 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1745190911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:09:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:53 smithi117 ceph-mon[35021]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:09:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:53 smithi117 ceph-mon[35021]: pgmap v272: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- 172.21.15.77:0/343558479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc102790 msgr2=0x7f9acc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/343558479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc102790 0x7f9acc1067e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02a410 comp rx=0 tx=0).stop 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- 172.21.15.77:0/343558479 shutdown_connections 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/343558479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc106f10 0x7f9acc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/343558479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc102790 0x7f9acc1067e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/343558479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc101de0 0x7f9acc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- 172.21.15.77:0/343558479 >> 172.21.15.77:0/343558479 conn(0x7f9acc0fd8c0 msgr2=0x7f9acc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- 172.21.15.77:0/343558479 shutdown_connections 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- 172.21.15.77:0/343558479 wait complete. 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 Processor -- start 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.651+0000 7f9ad41e6640 1 -- start start 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc101de0 0x7f9acc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102790 0x7f9acc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc111f90 con 0x7f9acc106f10 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc111e10 con 0x7f9acc102790 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc112110 con 0x7f9acc101de0 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35932/0 (socket says 172.21.15.77:35932) 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 -- 172.21.15.77:0/895892948 learned_addr learned my addr 172.21.15.77:0/895892948 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad1f82640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc101de0 0x7f9acc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad1781640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102790 0x7f9acc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 -- 172.21.15.77:0/895892948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc101de0 msgr2=0x7f9acc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc101de0 0x7f9acc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 -- 172.21.15.77:0/895892948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102790 msgr2=0x7f9acc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102790 0x7f9acc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 -- 172.21.15.77:0/895892948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc19d4a0 con 0x7f9acc106f10 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad2783640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b970 tx=0x7f9ac400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4019070 con 0x7f9acc106f10 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc19d790 con 0x7f9acc106f10 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac400cb40 con 0x7f9acc106f10 2024-09-15T22:09:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4007610 con 0x7f9acc106f10 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.652+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a5070 con 0x7f9acc106f10 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.653+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc1021c0 con 0x7f9acc106f10 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.655+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9ac401c070 con 0x7f9acc106f10 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.655+0000 7f9ac2ffd640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 0x7f9ab80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.655+0000 7f9ad1f82640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 0x7f9ab80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.655+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f9ac40cec20 con 0x7f9acc106f10 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.656+0000 7f9ad1f82640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 0x7f9ab80a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009b60 tx=0x7f9ac8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:53.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.660+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac409bc70 con 0x7f9acc106f10 2024-09-15T22:09:53.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:53.833+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9acc10a1b0 con 0x7f9acc106f10 2024-09-15T22:09:54.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.183+0000 7f9ac2ffd640 1 -- 172.21.15.77:0/895892948 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f9ac40bb070 con 0x7f9acc106f10 2024-09-15T22:09:54.186 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' created 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 msgr2=0x7f9ab80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 0x7f9ab80a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009b60 tx=0x7f9ac8005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 msgr2=0x7f9acc19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b970 tx=0x7f9ac400be30 comp rx=0 tx=0).stop 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 shutdown_connections 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9ab80a3240 0x7f9ab80a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9acc106f10 0x7f9acc19cdc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9acc102790 0x7f9acc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 --2- 172.21.15.77:0/895892948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9acc101de0 0x7f9acc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 >> 172.21.15.77:0/895892948 conn(0x7f9acc0fd8c0 msgr2=0x7f9acc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 shutdown_connections 2024-09-15T22:09:54.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:54.185+0000 7f9ad41e6640 1 -- 172.21.15.77:0/895892948 wait complete. 2024-09-15T22:09:54.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:54 smithi077 ceph-mon[30980]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:09:54.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:54 smithi077 ceph-mon[30980]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:09:54.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:54 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:09:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:54 smithi196 ceph-mon[35109]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:09:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:54 smithi196 ceph-mon[35109]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:09:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:54 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:09:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:54 smithi117 ceph-mon[35021]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:09:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:54 smithi117 ceph-mon[35021]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:09:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:54 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:09:54.884 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T22:09:55.125 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:55.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:55 smithi077 ceph-mon[30980]: pgmap v274: 33 pgs: 2 creating+activating, 4 creating+peering, 5 active+clean, 22 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:55.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:55 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:09:55.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:55 smithi077 ceph-mon[30980]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:09:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:55 smithi196 ceph-mon[35109]: pgmap v274: 33 pgs: 2 creating+activating, 4 creating+peering, 5 active+clean, 22 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:55 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:09:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:55 smithi196 ceph-mon[35109]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:09:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:55 smithi117 ceph-mon[35021]: pgmap v274: 33 pgs: 2 creating+activating, 4 creating+peering, 5 active+clean, 22 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:55 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/895892948' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:09:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:55 smithi117 ceph-mon[35021]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:09:55.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 -- 172.21.15.77:0/694825695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4100350 msgr2=0x7f91a4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:55.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/694825695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4100350 0x7f91a4100730 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cd0 tx=0x7f91a002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 -- 172.21.15.77:0/694825695 shutdown_connections 2024-09-15T22:09:55.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/694825695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4106f50 0x7f91a410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/694825695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100d00 0x7f91a4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/694825695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4100350 0x7f91a4100730 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 -- 172.21.15.77:0/694825695 >> 172.21.15.77:0/694825695 conn(0x7f91a4077210 msgr2=0x7f91a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 -- 172.21.15.77:0/694825695 shutdown_connections 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.606+0000 7f91ac4fc640 1 -- 172.21.15.77:0/694825695 wait complete. 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 Processor -- start 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 -- start start 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100350 0x7f91a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4100d00 0x7f91a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4112000 con 0x7f91a4106f50 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4111e80 con 0x7f91a4100350 2024-09-15T22:09:55.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91ac4fc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4112180 con 0x7f91a4100d00 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35948/0 (socket says 172.21.15.77:35948) 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 -- 172.21.15.77:0/1730405940 learned_addr learned my addr 172.21.15.77:0/1730405940 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aa298640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100350 0x7f91a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91a9a97640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4100d00 0x7f91a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 -- 172.21.15.77:0/1730405940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4100d00 msgr2=0x7f91a4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4100d00 0x7f91a4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 -- 172.21.15.77:0/1730405940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100350 msgr2=0x7f91a4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100350 0x7f91a4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.607+0000 7f91aaa99640 1 -- 172.21.15.77:0/1730405940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a419d4d0 con 0x7f91a4106f50 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f91aaa99640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f919c009fd0 tx=0x7f919c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c011830 con 0x7f91a4106f50 2024-09-15T22:09:55.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a419d7c0 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919c016070 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00f2e0 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.608+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a50a0 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.609+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4100730 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.610+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f919c0119d0 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.611+0000 7f919b7fe640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 0x7f91900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.611+0000 7f91aa298640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 0x7f91900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.611+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f919c0cf4a0 con 0x7f91a4106f50 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.611+0000 7f91aa298640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 0x7f91900a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006010 tx=0x7f91a0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:55.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.615+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f919c09c370 con 0x7f91a4106f50 2024-09-15T22:09:55.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:55.817+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f91a402d7c0 con 0x7f91a4106f50 2024-09-15T22:09:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.196+0000 7f919b7fe640 1 -- 172.21.15.77:0/1730405940 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f919c09bc70 con 0x7f91a4106f50 2024-09-15T22:09:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T22:09:56.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 msgr2=0x7f91900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 0x7f91900a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006010 tx=0x7f91a0009360 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 msgr2=0x7f91a419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f919c009fd0 tx=0x7f919c00b8e0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 shutdown_connections 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91900a3240 0x7f91900a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91a4106f50 0x7f91a419cdf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91a4100d00 0x7f91a4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 --2- 172.21.15.77:0/1730405940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4100350 0x7f91a4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 >> 172.21.15.77:0/1730405940 conn(0x7f91a4077210 msgr2=0x7f91a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 shutdown_connections 2024-09-15T22:09:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:56.198+0000 7f91ac4fc640 1 -- 172.21.15.77:0/1730405940 wait complete. 2024-09-15T22:09:56.573 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:56 smithi077 ceph-mon[30980]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:09:56.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:09:56.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:56 smithi077 ceph-mon[30980]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:09:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:56 smithi196 ceph-mon[35109]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:09:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:56 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:09:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:56 smithi196 ceph-mon[35109]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:09:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:56 smithi117 ceph-mon[35021]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:09:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:56 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:09:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:56 smithi117 ceph-mon[35021]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:09:57.011 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-15T22:09:57.012 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-15T22:09:57.257 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:57.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:57 smithi077 ceph-mon[30980]: pgmap v277: 97 pgs: 2 creating+activating, 15 creating+peering, 18 active+clean, 62 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:57.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:57 smithi077 ceph-mon[30980]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:09:57.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:57 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:09:57.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:57 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:09:57.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:57 smithi077 ceph-mon[30980]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:09:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:57 smithi196 ceph-mon[35109]: pgmap v277: 97 pgs: 2 creating+activating, 15 creating+peering, 18 active+clean, 62 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:57 smithi196 ceph-mon[35109]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:09:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:57 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:09:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:57 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:09:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:57 smithi196 ceph-mon[35109]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:09:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:57 smithi117 ceph-mon[35021]: pgmap v277: 97 pgs: 2 creating+activating, 15 creating+peering, 18 active+clean, 62 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:57 smithi117 ceph-mon[35021]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:09:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:57 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1730405940' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:09:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:57 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:09:57.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:57 smithi117 ceph-mon[35021]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:09:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 -- 172.21.15.77:0/412755847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c106f20 msgr2=0x7ffa9c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/412755847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c106f20 0x7ffa9c10eaa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009ca0 tx=0x7ffa9402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 -- 172.21.15.77:0/412755847 shutdown_connections 2024-09-15T22:09:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/412755847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c106f20 0x7ffa9c10eaa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/412755847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/412755847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 0x7ffa9c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 -- 172.21.15.77:0/412755847 >> 172.21.15.77:0/412755847 conn(0x7ffa9c0fd8b0 msgr2=0x7ffa9c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 -- 172.21.15.77:0/412755847 shutdown_connections 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.751+0000 7ffaa27ae640 1 -- 172.21.15.77:0/412755847 wait complete. 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 Processor -- start 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 -- start start 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 0x7ffa9c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa9c106f20 0x7ffa9c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c112020 con 0x7ffa9c1027a0 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111ea0 con 0x7ffa9c106f20 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffaa27ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c1121a0 con 0x7ffa9c101df0 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:57.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35972/0 (socket says 172.21.15.77:35972) 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.752+0000 7ffa9b7fe640 1 -- 172.21.15.77:0/2013157804 learned_addr learned my addr 172.21.15.77:0/2013157804 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9bfff640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 0x7ffa9c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 -- 172.21.15.77:0/2013157804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 msgr2=0x7ffa9c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 0x7ffa9c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 -- 172.21.15.77:0/2013157804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa9c106f20 msgr2=0x7ffa9c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa9c106f20 0x7ffa9c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 -- 172.21.15.77:0/2013157804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c1a69c0 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa9b7fe640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b4a0 tx=0x7ffa9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90014070 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa90010af0 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c1a6cb0 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90017d10 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.753+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c1a71a0 con 0x7ffa9c1027a0 2024-09-15T22:09:57.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.754+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa9c102250 con 0x7ffa9c1027a0 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.755+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ffa90017070 con 0x7ffa9c1027a0 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.756+0000 7ffa997fa640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 0x7ffa840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.756+0000 7ffa9bfff640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 0x7ffa840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.756+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ffa900cefd0 con 0x7ffa9c1027a0 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.756+0000 7ffa9bfff640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 0x7ffa840a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009890 tx=0x7ffa8c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:57.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.759+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa9009be40 con 0x7ffa9c1027a0 2024-09-15T22:09:57.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:57.936+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ffa9c075a20 con 0x7ffa9c1027a0 2024-09-15T22:09:58.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.211+0000 7ffa997fa640 1 -- 172.21.15.77:0/2013157804 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7ffa9c075a20 con 0x7ffa9c1027a0 2024-09-15T22:09:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.213+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 msgr2=0x7ffa840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.213+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 0x7ffa840a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009890 tx=0x7ffa8c008040 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.213+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 msgr2=0x7ffa9c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.213+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b4a0 tx=0x7ffa9000b960 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 shutdown_connections 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ffa840a3240 0x7ffa840a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa9c106f20 0x7ffa9c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa9c1027a0 0x7ffa9c074a30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 --2- 172.21.15.77:0/2013157804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa9c101df0 0x7ffa9c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 >> 172.21.15.77:0/2013157804 conn(0x7ffa9c0fd8b0 msgr2=0x7ffa9c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 shutdown_connections 2024-09-15T22:09:58.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:58.214+0000 7ffaa27ae640 1 -- 172.21.15.77:0/2013157804 wait complete. 2024-09-15T22:09:58.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:58 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:active 2024-09-15T22:09:58.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:58 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:58 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:58 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:09:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:58 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:active 2024-09-15T22:09:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:58 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:58 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:58 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:09:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:58 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] up:active 2024-09-15T22:09:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:58 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:58 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:09:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:58 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:09:58.929 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T22:09:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:59.457 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:59 smithi077 ceph-mon[30980]: pgmap v279: 97 pgs: 2 creating+activating, 11 creating+peering, 76 active+clean, 8 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T22:09:59.457 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:59 smithi077 ceph-mon[30980]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:09:59.457 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:59 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:09:59.457 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:09:59.457 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:09:59 smithi077 ceph-mon[30980]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:09:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:59 smithi196 ceph-mon[35109]: pgmap v279: 97 pgs: 2 creating+activating, 11 creating+peering, 76 active+clean, 8 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T22:09:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:59 smithi196 ceph-mon[35109]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:09:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:59 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:09:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:09:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:09:59 smithi196 ceph-mon[35109]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:09:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:59 smithi117 ceph-mon[35021]: pgmap v279: 97 pgs: 2 creating+activating, 11 creating+peering, 76 active+clean, 8 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-15T22:09:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:59 smithi117 ceph-mon[35021]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:09:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:59 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:09:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2013157804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:09:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:09:59 smithi117 ceph-mon[35021]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:09:59.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 -- 172.21.15.77:0/996577715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4106a80 msgr2=0x7f56a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:59.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/996577715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4106a80 0x7f56a4106e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5694009930 tx=0x7f569402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:59.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 -- 172.21.15.77:0/996577715 shutdown_connections 2024-09-15T22:09:59.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/996577715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:59.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/996577715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4100a80 0x7f56a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/996577715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4106a80 0x7f56a4106e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 -- 172.21.15.77:0/996577715 >> 172.21.15.77:0/996577715 conn(0x7f56a40fc7a0 msgr2=0x7f56a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 -- 172.21.15.77:0/996577715 shutdown_connections 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.663+0000 7f56aa4c7640 1 -- 172.21.15.77:0/996577715 wait complete. 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 Processor -- start 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 -- start start 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 0x7f56a419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4110ea0 con 0x7f56a4100a80 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4110d20 con 0x7f56a4106a80 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56aa4c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4111020 con 0x7f56a4101420 2024-09-15T22:09:59.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52920/0 (socket says 172.21.15.77:52920) 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 -- 172.21.15.77:0/1935598028 learned_addr learned my addr 172.21.15.77:0/1935598028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a8a64640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 0x7f56a419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f569bfff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 -- 172.21.15.77:0/1935598028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 msgr2=0x7f56a41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a41978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 -- 172.21.15.77:0/1935598028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 msgr2=0x7f56a419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 0x7f56a419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a3fff640 1 -- 172.21.15.77:0/1935598028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a419c350 con 0x7f56a4100a80 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f569bfff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.664+0000 7f56a8a64640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 0x7f56a419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:59.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56a3fff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5694009230 tx=0x7f5694004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569403f030 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a419c5e0 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5694003ad0 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694035070 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.665+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a3f20 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.666+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a41021c0 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.667+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5694042030 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.667+0000 7f56a1ffb640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 0x7f56900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.667+0000 7f569bfff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 0x7f56900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.668+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f56940efef0 con 0x7f56a4100a80 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.668+0000 7f569bfff640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 0x7f56900a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56a40fee00 tx=0x7f568c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:59.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.672+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56940bcbe0 con 0x7f56a4100a80 2024-09-15T22:09:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:09:59.838+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f56a4108ff0 con 0x7f56a4100a80 2024-09-15T22:10:00.212 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:10:00.212 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:00 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:10:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:10:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:00 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:10:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:10:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:00 smithi117 ceph-mon[35021]: overall HEALTH_OK 2024-09-15T22:10:01.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.216+0000 7f56a1ffb640 1 -- 172.21.15.77:0/1935598028 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f56940bc4e0 con 0x7f56a4100a80 2024-09-15T22:10:01.218 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data_ec' created 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 msgr2=0x7f56900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 0x7f56900a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56a40fee00 tx=0x7f568c005e50 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 msgr2=0x7f56a41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5694009230 tx=0x7f5694004270 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 shutdown_connections 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56900a3240 0x7f56900a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56a4106a80 0x7f56a419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56a4101420 0x7f56a41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 --2- 172.21.15.77:0/1935598028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56a4100a80 0x7f56a41973a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:01.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 >> 172.21.15.77:0/1935598028 conn(0x7f56a40fc7a0 msgr2=0x7f56a40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:01.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 shutdown_connections 2024-09-15T22:10:01.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:01.220+0000 7f56aa4c7640 1 -- 172.21.15.77:0/1935598028 wait complete. 2024-09-15T22:10:01.551 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:01 smithi077 ceph-mon[30980]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:01.551 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:01 smithi077 ceph-mon[30980]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:10:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:01 smithi196 ceph-mon[35109]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:01 smithi196 ceph-mon[35109]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:10:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:01 smithi117 ceph-mon[35021]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:01 smithi117 ceph-mon[35021]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:10:01.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:10:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:10:01.250+0000 7f48be595640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T22:10:01.895 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-15T22:10:02.142 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:02.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:02 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:10:02.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:02 smithi077 ceph-mon[30980]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:10:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:02 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:10:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:02 smithi196 ceph-mon[35109]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:10:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:02 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1935598028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:10:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:02 smithi117 ceph-mon[35021]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:10:02.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 -- 172.21.15.77:0/4023684883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88100e20 msgr2=0x7f2a88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:02.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/4023684883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88100e20 0x7f2a88101200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009900 tx=0x7f2a7802a480 comp rx=0 tx=0).stop 2024-09-15T22:10:02.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 -- 172.21.15.77:0/4023684883 shutdown_connections 2024-09-15T22:10:02.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/4023684883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88102ef0 0x7f2a8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/4023684883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a88102570 0x7f2a881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/4023684883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88100e20 0x7f2a88101200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 -- 172.21.15.77:0/4023684883 >> 172.21.15.77:0/4023684883 conn(0x7f2a880fc7d0 msgr2=0x7f2a880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 -- 172.21.15.77:0/4023684883 shutdown_connections 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.604+0000 7f2a8ceff640 1 -- 172.21.15.77:0/4023684883 wait complete. 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 Processor -- start 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 -- start start 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88100e20 0x7f2a8810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a88102ef0 0x7f2a8810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88112060 con 0x7f2a88102570 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88111ee0 con 0x7f2a88100e20 2024-09-15T22:10:02.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8ceff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881121e0 con 0x7f2a88102ef0 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52932/0 (socket says 172.21.15.77:52932) 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 -- 172.21.15.77:0/207059831 learned_addr learned my addr 172.21.15.77:0/207059831 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a8659c640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88100e20 0x7f2a8810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 -- 172.21.15.77:0/207059831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a88102ef0 msgr2=0x7f2a8810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a88102ef0 0x7f2a8810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 -- 172.21.15.77:0/207059831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88100e20 msgr2=0x7f2a8810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:02.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88100e20 0x7f2a8810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.605+0000 7f2a7ffff640 1 -- 172.21.15.77:0/207059831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a881a6980 con 0x7f2a88102570 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a7ffff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d950 tx=0x7f2a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70017070 con 0x7f2a88102570 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70009c30 con 0x7f2a88102570 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a881a6c70 con 0x7f2a88102570 2024-09-15T22:10:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000f6b0 con 0x7f2a88102570 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.606+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881a71b0 con 0x7f2a88102570 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.607+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a88101200 con 0x7f2a88102570 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.608+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2a700109d0 con 0x7f2a88102570 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.609+0000 7f2a7f7fe640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 0x7f2a740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.609+0000 7f2a8659c640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 0x7f2a740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.609+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f2a700cf750 con 0x7f2a88102570 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.609+0000 7f2a8659c640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 0x7f2a740a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a780096c0 tx=0x7f2a78000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:02.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.612+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a70009de0 con 0x7f2a88102570 2024-09-15T22:10:02.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:02.779+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f2a8810fa00 con 0x7f2a88102570 2024-09-15T22:10:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.227+0000 7f2a7f7fe640 1 -- 172.21.15.77:0/207059831 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7f2a8810fa00 con 0x7f2a88102570 2024-09-15T22:10:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:set pool 4 allow_ec_overwrites to true 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 msgr2=0x7f2a740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 0x7f2a740a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a780096c0 tx=0x7f2a78000bc0 comp rx=0 tx=0).stop 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 msgr2=0x7f2a8810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d950 tx=0x7f2a7000de10 comp rx=0 tx=0).stop 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 shutdown_connections 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2a740a32c0 0x7f2a740a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a88102ef0 0x7f2a8810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:03.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a88102570 0x7f2a8810ea10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 --2- 172.21.15.77:0/207059831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a88100e20 0x7f2a8810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.229+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 >> 172.21.15.77:0/207059831 conn(0x7f2a880fc7d0 msgr2=0x7f2a880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.230+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 shutdown_connections 2024-09-15T22:10:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:03.230+0000 7f2a8ceff640 1 -- 172.21.15.77:0/207059831 wait complete. 2024-09-15T22:10:03.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:03 smithi077 ceph-mon[30980]: pgmap v284: 129 pgs: 18 creating+peering, 14 unknown, 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:03.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:03 smithi077 ceph-mon[30980]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:10:03.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:03 smithi077 ceph-mon[30980]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:10:03.550 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:10:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:03 smithi196 ceph-mon[35109]: pgmap v284: 129 pgs: 18 creating+peering, 14 unknown, 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:03 smithi196 ceph-mon[35109]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:10:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:03 smithi196 ceph-mon[35109]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:10:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:10:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:03 smithi117 ceph-mon[35021]: pgmap v284: 129 pgs: 18 creating+peering, 14 unknown, 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-15T22:10:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:03 smithi117 ceph-mon[35021]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:10:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:03 smithi117 ceph-mon[35021]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:10:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:10:03.889 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-15T22:10:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:04.539 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:04 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:10:04.539 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:04 smithi077 ceph-mon[30980]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:10:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:04 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:10:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:04 smithi196 ceph-mon[35109]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:10:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:04 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/207059831' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:10:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:04 smithi117 ceph-mon[35021]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:10:04.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 -- 172.21.15.77:0/3997759921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968100a80 msgr2=0x7f7968100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:04.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 --2- 172.21.15.77:0/3997759921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968100a80 0x7f7968100ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7958009930 tx=0x7f795802a420 comp rx=0 tx=0).stop 2024-09-15T22:10:04.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 -- 172.21.15.77:0/3997759921 shutdown_connections 2024-09-15T22:10:04.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 --2- 172.21.15.77:0/3997759921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968101420 0x7f796810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 --2- 172.21.15.77:0/3997759921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968100a80 0x7f7968100ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 --2- 172.21.15.77:0/3997759921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968106a80 0x7f7968106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 -- 172.21.15.77:0/3997759921 >> 172.21.15.77:0/3997759921 conn(0x7f79680fc7a0 msgr2=0x7f79680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 -- 172.21.15.77:0/3997759921 shutdown_connections 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.618+0000 7f796fd79640 1 -- 172.21.15.77:0/3997759921 wait complete. 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 Processor -- start 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 -- start start 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968100a80 0x7f79681972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968106a80 0x7f796819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7968110e80 con 0x7f7968101420 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7968110d00 con 0x7f7968100a80 2024-09-15T22:10:04.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796fd79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7968111000 con 0x7f7968106a80 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52960/0 (socket says 172.21.15.77:52960) 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 -- 172.21.15.77:0/163016400 learned_addr learned my addr 172.21.15.77:0/163016400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796db15640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968100a80 0x7f79681972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796e316640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968106a80 0x7f796819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 -- 172.21.15.77:0/163016400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968106a80 msgr2=0x7f796819bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968106a80 0x7f796819bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 -- 172.21.15.77:0/163016400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968100a80 msgr2=0x7f79681972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968100a80 0x7f79681972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 -- 172.21.15.77:0/163016400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796819c270 con 0x7f7968101420 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.619+0000 7f796d314640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7958009230 tx=0x7f7958004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.620+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795803f030 con 0x7f7968101420 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.620+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958003ad0 con 0x7f7968101420 2024-09-15T22:10:04.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.620+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796819c500 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.620+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958035070 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.620+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79681a3e40 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.621+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79681021c0 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.622+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7958042030 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.622+0000 7f795effd640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 0x7f79540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.622+0000 7f796db15640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 0x7f79540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.622+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f79580eff50 con 0x7f7968101420 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.623+0000 7f796db15640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 0x7f79540a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f796400ab60 tx=0x7f796400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:04.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.626+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79580bca50 con 0x7f7968101420 2024-09-15T22:10:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:04.831+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f79681a4010 con 0x7f7968101420 2024-09-15T22:10:05.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.233+0000 7f795effd640 1 -- 172.21.15.77:0/163016400 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f79580bc350 con 0x7f7968101420 2024-09-15T22:10:05.235 INFO:teuthology.orchestra.run.smithi077.stderr:added data pool 4 to fsmap 2024-09-15T22:10:05.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 msgr2=0x7f79540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:05.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 0x7f79540a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f796400ab60 tx=0x7f796400a2d0 comp rx=0 tx=0).stop 2024-09-15T22:10:05.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 msgr2=0x7f7968197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:05.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7958009230 tx=0x7f7958004270 comp rx=0 tx=0).stop 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 shutdown_connections 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f79540a3240 0x7f79540a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7968106a80 0x7f796819bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7968101420 0x7f7968197800 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 --2- 172.21.15.77:0/163016400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7968100a80 0x7f79681972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 >> 172.21.15.77:0/163016400 conn(0x7f79680fc7a0 msgr2=0x7f79680fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 shutdown_connections 2024-09-15T22:10:05.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:05.236+0000 7f796fd79640 1 -- 172.21.15.77:0/163016400 wait complete. 2024-09-15T22:10:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:05 smithi077 ceph-mon[30980]: pgmap v287: 129 pgs: 19 creating+peering, 5 unknown, 105 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:05 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:10:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:05 smithi196 ceph-mon[35109]: pgmap v287: 129 pgs: 19 creating+peering, 5 unknown, 105 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:05 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:10:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:05 smithi117 ceph-mon[35021]: pgmap v287: 129 pgs: 19 creating+peering, 5 unknown, 105 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:05 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:10:05.954 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:10:06.200 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:06.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:06 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:10:06.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:06 smithi077 ceph-mon[30980]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:10:06.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:06 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:06 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:10:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:06 smithi196 ceph-mon[35109]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:10:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:06 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:06 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/163016400' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:10:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:06 smithi117 ceph-mon[35021]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:10:06.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:06 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:06.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 -- 172.21.15.77:0/4227995686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc101de0 msgr2=0x7f19fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 --2- 172.21.15.77:0/4227995686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc101de0 0x7f19fc1021c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ca0 tx=0x7f19ec02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 -- 172.21.15.77:0/4227995686 shutdown_connections 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 --2- 172.21.15.77:0/4227995686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc106f10 0x7f19fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 --2- 172.21.15.77:0/4227995686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc102790 0x7f19fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 --2- 172.21.15.77:0/4227995686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc101de0 0x7f19fc1021c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 -- 172.21.15.77:0/4227995686 >> 172.21.15.77:0/4227995686 conn(0x7f19fc0fd8c0 msgr2=0x7f19fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 -- 172.21.15.77:0/4227995686 shutdown_connections 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.675+0000 7f1a02b20640 1 -- 172.21.15.77:0/4227995686 wait complete. 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 Processor -- start 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 -- start start 2024-09-15T22:10:06.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 0x7f19fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc102790 0x7f19fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc111f90 con 0x7f19fc106f10 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc111e10 con 0x7f19fc101de0 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a02b20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc112110 con 0x7f19fc102790 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a010bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a010bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52982/0 (socket says 172.21.15.77:52982) 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.676+0000 7f1a010bd640 1 -- 172.21.15.77:0/3300294215 learned_addr learned my addr 172.21.15.77:0/3300294215 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f19fbfff640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc102790 0x7f19fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a008bc640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 0x7f19fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 -- 172.21.15.77:0/3300294215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc102790 msgr2=0x7f19fc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc102790 0x7f19fc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 -- 172.21.15.77:0/3300294215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 msgr2=0x7f19fc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 0x7f19fc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 -- 172.21.15.77:0/3300294215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc19d380 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a008bc640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 0x7f19fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a010bd640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009fd0 tx=0x7f19f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4011830 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4016070 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc19d670 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f400f2e0 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.677+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc1a4f50 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.678+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19fc1021c0 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.679+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f19f40119d0 con 0x7f19fc106f10 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.680+0000 7f19f9ffb640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 0x7f19e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.680+0000 7f1a008bc640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 0x7f19e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:06.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.680+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f19f40cf240 con 0x7f19fc106f10 2024-09-15T22:10:06.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.680+0000 7f1a008bc640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 0x7f19e40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ca0 tx=0x7f19ec000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:06.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.684+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19f409bd20 con 0x7f19fc106f10 2024-09-15T22:10:06.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.886+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19fc10a150 con 0x7f19fc106f10 2024-09-15T22:10:06.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.887+0000 7f19f9ffb640 1 -- 172.21.15.77:0/3300294215 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11854 (secure 0 0 0) 0x7f19f409b620 con 0x7f19fc106f10 2024-09-15T22:10:06.889 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:06.889 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"btime":"2024-09-15T22:10:05:232844+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:04.832565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24458,"qdb_cluster":[24458]},"id":1}]} 2024-09-15T22:10:06.889 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 17 2024-09-15T22:10:06.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 msgr2=0x7f19e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 0x7f19e40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ca0 tx=0x7f19ec000f40 comp rx=0 tx=0).stop 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 msgr2=0x7f19fc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009fd0 tx=0x7f19f400b8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 shutdown_connections 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f19e40a3240 0x7f19e40a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19fc106f10 0x7f19fc19cca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc102790 0x7f19fc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 --2- 172.21.15.77:0/3300294215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19fc101de0 0x7f19fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:06.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 >> 172.21.15.77:0/3300294215 conn(0x7f19fc0fd8c0 msgr2=0x7f19fc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:06.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 shutdown_connections 2024-09-15T22:10:06.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:06.889+0000 7f1a02b20640 1 -- 172.21.15.77:0/3300294215 wait complete. 2024-09-15T22:10:07.561 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:07 smithi077 ceph-mon[30980]: pgmap v289: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:07.561 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:07 smithi077 ceph-mon[30980]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:10:07.561 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:07 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:10:07.561 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:07 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3300294215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:07.592 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:07 smithi196 ceph-mon[35109]: pgmap v289: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:07 smithi196 ceph-mon[35109]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:10:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:07 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:10:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:07 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3300294215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:07 smithi117 ceph-mon[35021]: pgmap v289: 129 pgs: 19 creating+peering, 110 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:07 smithi117 ceph-mon[35021]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:10:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:07 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:10:07.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:07 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3300294215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:07.836 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:08.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 -- 172.21.15.77:0/2860227507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 msgr2=0x7f620c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:08.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 --2- 172.21.15.77:0/2860227507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c1067e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009900 tx=0x7f61f802a410 comp rx=0 tx=0).stop 2024-09-15T22:10:08.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 -- 172.21.15.77:0/2860227507 shutdown_connections 2024-09-15T22:10:08.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 --2- 172.21.15.77:0/2860227507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 --2- 172.21.15.77:0/2860227507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c1067e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 --2- 172.21.15.77:0/2860227507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 -- 172.21.15.77:0/2860227507 >> 172.21.15.77:0/2860227507 conn(0x7f620c0fd8c0 msgr2=0x7f620c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 -- 172.21.15.77:0/2860227507 shutdown_connections 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.393+0000 7f6212ed3640 1 -- 172.21.15.77:0/2860227507 wait complete. 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 Processor -- start 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 -- start start 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f620c112010 con 0x7f620c102790 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f620c111e90 con 0x7f620c101de0 2024-09-15T22:10:08.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6212ed3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f620c112190 con 0x7f620c106f10 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53006/0 (socket says 172.21.15.77:53006) 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 -- 172.21.15.77:0/3529930170 learned_addr learned my addr 172.21.15.77:0/3529930170 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6210c6f640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6211470640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 -- 172.21.15.77:0/3529930170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 msgr2=0x7f620c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 -- 172.21.15.77:0/3529930170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 msgr2=0x7f620c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 -- 172.21.15.77:0/3529930170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c19d560 con 0x7f620c102790 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6211470640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f6210c6f640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:08.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.394+0000 7f620bfff640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005ab0 tx=0x7f61f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.395+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f803f030 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.395+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f8003ce0 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.395+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c19d7f0 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.395+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8035070 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.395+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1a5130 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.396+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c1032c0 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.397+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f61f8042030 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.398+0000 7f6209ffb640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 0x7f61fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.398+0000 7f6210c6f640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 0x7f61fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.398+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f61f80f0740 con 0x7f620c102790 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.398+0000 7f6210c6f640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 0x7f61fc0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6200009930 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:08.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.401+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61f80bd220 con 0x7f620c102790 2024-09-15T22:10:08.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.572+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f620c10a0f0 con 0x7f620c102790 2024-09-15T22:10:08.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.573+0000 7f6209ffb640 1 -- 172.21.15.77:0/3529930170 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f61f80bcb20 con 0x7f620c102790 2024-09-15T22:10:08.575 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:08.575 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 msgr2=0x7f61fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 0x7f61fc0a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6200009930 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 msgr2=0x7f620c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005ab0 tx=0x7f61f80042f0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 shutdown_connections 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f61fc0a3240 0x7f61fc0a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f620c106f10 0x7f620c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f620c102790 0x7f620c198af0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 --2- 172.21.15.77:0/3529930170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c101de0 0x7f620c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.575+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 >> 172.21.15.77:0/3529930170 conn(0x7f620c0fd8c0 msgr2=0x7f620c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.576+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 shutdown_connections 2024-09-15T22:10:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:08.576+0000 7f6212ed3640 1 -- 172.21.15.77:0/3529930170 wait complete. 2024-09-15T22:10:09.290 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:09.533 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:09 smithi077 ceph-mon[30980]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:09 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3529930170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:09 smithi196 ceph-mon[35109]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:09 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3529930170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:09 smithi117 ceph-mon[35021]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:09 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3529930170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 -- 172.21.15.77:0/2411728369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c102d40 msgr2=0x7fc72c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2411728369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c102d40 0x7fc72c103120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009900 tx=0x7fc71c02a480 comp rx=0 tx=0).stop 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 -- 172.21.15.77:0/2411728369 shutdown_connections 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2411728369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 0x7fc72c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2411728369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c103660 0x7fc72c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2411728369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c102d40 0x7fc72c103120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 -- 172.21.15.77:0/2411728369 >> 172.21.15.77:0/2411728369 conn(0x7fc72c0fc7a0 msgr2=0x7fc72c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:10.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 -- 172.21.15.77:0/2411728369 shutdown_connections 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.020+0000 7fc7312ca640 1 -- 172.21.15.77:0/2411728369 wait complete. 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 Processor -- start 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 -- start start 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c102d40 0x7fc72c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 0x7fc72c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c111f70 con 0x7fc72c103660 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c111df0 con 0x7fc72c10c170 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc7312ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c1120f0 con 0x7fc72c102d40 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58404/0 (socket says 172.21.15.77:58404) 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 -- 172.21.15.77:0/2610184405 learned_addr learned my addr 172.21.15.77:0/2610184405 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc72affd640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c102d40 0x7fc72c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc72b7fe640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 0x7fc72c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 -- 172.21.15.77:0/2610184405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c102d40 msgr2=0x7fc72c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c102d40 0x7fc72c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 -- 172.21.15.77:0/2610184405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 msgr2=0x7fc72c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 0x7fc72c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.021+0000 7fc723fff640 1 -- 172.21.15.77:0/2610184405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c19d3e0 con 0x7fc72c103660 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc723fff640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc71400d740 tx=0x7fc71400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714017070 con 0x7fc72c103660 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71400de50 con 0x7fc72c103660 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c19d6d0 con 0x7fc72c103660 2024-09-15T22:10:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71400f620 con 0x7fc72c103660 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.022+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c1a4fb0 con 0x7fc72c103660 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.023+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c103120 con 0x7fc72c103660 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.024+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc714009c30 con 0x7fc72c103660 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.024+0000 7fc728ff9640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 0x7fc7180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.024+0000 7fc72affd640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 0x7fc7180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.024+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc7140cf620 con 0x7fc72c103660 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.025+0000 7fc72affd640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 0x7fc7180a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009230 tx=0x7fc71c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:10.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.029+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc71409bab0 con 0x7fc72c103660 2024-09-15T22:10:10.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.196+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc72c107240 con 0x7fc72c103660 2024-09-15T22:10:10.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.197+0000 7fc728ff9640 1 -- 172.21.15.77:0/2610184405 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7fc71409fdb0 con 0x7fc72c103660 2024-09-15T22:10:10.199 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:10.199 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:10.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.199+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 msgr2=0x7fc7180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.199+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 0x7fc7180a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009230 tx=0x7fc71c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.199+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 msgr2=0x7fc72c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.199+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc71400d740 tx=0x7fc71400dc00 comp rx=0 tx=0).stop 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 shutdown_connections 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc7180a32c0 0x7fc7180a5780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72c10c170 0x7fc72c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc72c103660 0x7fc72c198970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 --2- 172.21.15.77:0/2610184405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc72c102d40 0x7fc72c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 >> 172.21.15.77:0/2610184405 conn(0x7fc72c0fc7a0 msgr2=0x7fc72c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 shutdown_connections 2024-09-15T22:10:10.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:10.200+0000 7fc7312ca640 1 -- 172.21.15.77:0/2610184405 wait complete. 2024-09-15T22:10:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:10 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2610184405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:10 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2610184405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:10 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2610184405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:10.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:10:11.195 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:11 smithi077 ceph-mon[30980]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:11 smithi196 ceph-mon[35109]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:11 smithi117 ceph-mon[35021]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:11.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 -- 172.21.15.77:0/937408434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0106f10 msgr2=0x7f34b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:11.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 --2- 172.21.15.77:0/937408434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0106f10 0x7f34b010ea90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ca0 tx=0x7f34a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 -- 172.21.15.77:0/937408434 shutdown_connections 2024-09-15T22:10:11.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 --2- 172.21.15.77:0/937408434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0106f10 0x7f34b010ea90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 --2- 172.21.15.77:0/937408434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b0102790 0x7f34b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 --2- 172.21.15.77:0/937408434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 0x7f34b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 -- 172.21.15.77:0/937408434 >> 172.21.15.77:0/937408434 conn(0x7f34b00fd8c0 msgr2=0x7f34b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 -- 172.21.15.77:0/937408434 shutdown_connections 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.687+0000 7f34b67da640 1 -- 172.21.15.77:0/937408434 wait complete. 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 Processor -- start 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 -- start start 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 0x7f34b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b0106f10 0x7f34b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0111f90 con 0x7f34b0102790 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0111e10 con 0x7f34b0101de0 2024-09-15T22:10:11.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34b67da640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112110 con 0x7f34b0106f10 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58428/0 (socket says 172.21.15.77:58428) 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.688+0000 7f34a7fff640 1 -- 172.21.15.77:0/1146961515 learned_addr learned my addr 172.21.15.77:0/1146961515 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 -- 172.21.15.77:0/1146961515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b0106f10 msgr2=0x7f34b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b0106f10 0x7f34b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 -- 172.21.15.77:0/1146961515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 msgr2=0x7f34b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34affff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 0x7f34b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 0x7f34b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 -- 172.21.15.77:0/1146961515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b01a68f0 con 0x7f34b0102790 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34a7fff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349800c890 tx=0x7f349800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:11.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004220 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34980043c0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b01a6be0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980049e0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.689+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a7120 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.690+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b01021c0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.691+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f34980079b0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.692+0000 7f34adffb640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 0x7f349c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.692+0000 7f34affff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 0x7f349c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.692+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f34980cfcc0 con 0x7f34b0102790 2024-09-15T22:10:11.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.692+0000 7f34affff640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 0x7f349c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34a000a810 tx=0x7f34a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:11.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.696+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34980d6080 con 0x7f34b0102790 2024-09-15T22:10:11.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.897+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34b002d7c0 con 0x7f34b0102790 2024-09-15T22:10:11.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.898+0000 7f34adffb640 1 -- 172.21.15.77:0/1146961515 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11854 (secure 0 0 0) 0x7f3498014030 con 0x7f34b0102790 2024-09-15T22:10:11.900 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:11.900 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"btime":"2024-09-15T22:10:05:232844+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24439,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:04.832565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24458,"qdb_cluster":[24458]},"id":1}]} 2024-09-15T22:10:11.900 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 17 2024-09-15T22:10:11.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 msgr2=0x7f349c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 0x7f349c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34a000a810 tx=0x7f34a0008040 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 msgr2=0x7f34b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f349800c890 tx=0x7f349800cd50 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 shutdown_connections 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f349c0a3240 0x7f349c0a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34b0106f10 0x7f34b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b0102790 0x7f34b0074a30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 --2- 172.21.15.77:0/1146961515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34b0101de0 0x7f34b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 >> 172.21.15.77:0/1146961515 conn(0x7f34b00fd8c0 msgr2=0x7f34b00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 shutdown_connections 2024-09-15T22:10:11.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:11.900+0000 7f34b67da640 1 -- 172.21.15.77:0/1146961515 wait complete. 2024-09-15T22:10:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:12 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1146961515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:12 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1146961515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:12 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1146961515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:12.659 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:13.290 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:13 smithi077 ceph-mon[30980]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- 172.21.15.77:0/1365025156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 msgr2=0x7f44a4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 --2- 172.21.15.77:0/1365025156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4107b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4494009cd0 tx=0x7f449402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- 172.21.15.77:0/1365025156 shutdown_connections 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 --2- 172.21.15.77:0/1365025156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a41082a0 0x7f44a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 --2- 172.21.15.77:0/1365025156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4107b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 --2- 172.21.15.77:0/1365025156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a4102d30 0x7f44a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- 172.21.15.77:0/1365025156 >> 172.21.15.77:0/1365025156 conn(0x7f44a40fc7d0 msgr2=0x7f44a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- 172.21.15.77:0/1365025156 shutdown_connections 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- 172.21.15.77:0/1365025156 wait complete. 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 Processor -- start 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.385+0000 7f44aae97640 1 -- start start 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a4102d30 0x7f44a4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 0x7f44a419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44a41120c0 con 0x7f44a41036e0 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4111f40 con 0x7f44a4102d30 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4112240 con 0x7f44a41082a0 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58440/0 (socket says 172.21.15.77:58440) 2024-09-15T22:10:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 -- 172.21.15.77:0/351396835 learned_addr learned my addr 172.21.15.77:0/351396835 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a8c33640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a4102d30 0x7f44a4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a9434640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 0x7f44a419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 -- 172.21.15.77:0/351396835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 msgr2=0x7f44a419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 0x7f44a419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 -- 172.21.15.77:0/351396835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a4102d30 msgr2=0x7f44a4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a4102d30 0x7f44a4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 -- 172.21.15.77:0/351396835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a419d5f0 con 0x7f44a41036e0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a9434640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 0x7f44a419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a3fff640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4494000f80 tx=0x7f4494003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494004040 con 0x7f44a41036e0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44940046c0 con 0x7f44a41036e0 2024-09-15T22:10:13.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a419d880 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.386+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494030e80 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.387+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a41a51c0 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.388+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a4103110 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.389+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f44940305a0 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.390+0000 7f44a1ffb640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 0x7f44900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.390+0000 7f44a8c33640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 0x7f44900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.390+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f449402e030 con 0x7f44a41036e0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.390+0000 7f44a8c33640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 0x7f44900a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f449c009990 tx=0x7f449c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:13.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.393+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f449404b060 con 0x7f44a41036e0 2024-09-15T22:10:13.513 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:13 smithi117 ceph-mon[35021]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:13.519 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:13 smithi196 ceph-mon[35109]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:13.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.560+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44a410a2b0 con 0x7f44a41036e0 2024-09-15T22:10:13.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.562+0000 7f44a1ffb640 1 -- 172.21.15.77:0/351396835 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f44940bdec0 con 0x7f44a41036e0 2024-09-15T22:10:13.564 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:13.564 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 msgr2=0x7f44900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 0x7f44900a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f449c009990 tx=0x7f449c009110 comp rx=0 tx=0).stop 2024-09-15T22:10:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 msgr2=0x7f44a4198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4494000f80 tx=0x7f4494003970 comp rx=0 tx=0).stop 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 shutdown_connections 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f44900a32c0 0x7f44900a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44a41082a0 0x7f44a419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44a41036e0 0x7f44a4198b80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 --2- 172.21.15.77:0/351396835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44a4102d30 0x7f44a4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 >> 172.21.15.77:0/351396835 conn(0x7f44a40fc7d0 msgr2=0x7f44a4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 shutdown_connections 2024-09-15T22:10:13.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:13.564+0000 7f44aae97640 1 -- 172.21.15.77:0/351396835 wait complete. 2024-09-15T22:10:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:14 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/351396835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:14.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T22:09:55.819962+0000', 'modified': '2024-09-15T22:10:04.832565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/1252344498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1252344498}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1252344498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24458, 'qdb_cluster': [24458]} 2024-09-15T22:10:14.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T22:10:14.437 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:10:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:14 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/351396835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:14 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/351396835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:14.681 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 -- 172.21.15.77:0/963113142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 msgr2=0x7fb9481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 --2- 172.21.15.77:0/963113142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb9481029b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb94402a480 comp rx=0 tx=0).stop 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 -- 172.21.15.77:0/963113142 shutdown_connections 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 --2- 172.21.15.77:0/963113142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948102ef0 0x7fb94810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 --2- 172.21.15.77:0/963113142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb9481029b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 --2- 172.21.15.77:0/963113142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948100e20 0x7fb948101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 -- 172.21.15.77:0/963113142 >> 172.21.15.77:0/963113142 conn(0x7fb9480fc7d0 msgr2=0x7fb9480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 -- 172.21.15.77:0/963113142 shutdown_connections 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.181+0000 7fb94f406640 1 -- 172.21.15.77:0/963113142 wait complete. 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 Processor -- start 2024-09-15T22:10:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 -- start start 2024-09-15T22:10:15.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 0x7fb948199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948102ef0 0x7fb9481a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb948110f80 con 0x7fb948102570 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb948110e00 con 0x7fb948100e20 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb948111100 con 0x7fb948102ef0 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58456/0 (socket says 172.21.15.77:58456) 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 -- 172.21.15.77:0/4108479332 learned_addr learned my addr 172.21.15.77:0/4108479332 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948102ef0 0x7fb9481a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 0x7fb948199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 -- 172.21.15.77:0/4108479332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948102ef0 msgr2=0x7fb9481a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948102ef0 0x7fb9481a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 -- 172.21.15.77:0/4108479332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 msgr2=0x7fb948199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 0x7fb948199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 -- 172.21.15.77:0/4108479332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481a4910 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 0x7fb948199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb944003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb93a7fc640 1 -- 172.21.15.77:0/4108479332 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94403f070 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.182+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481a4ae0 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.183+0000 7fb93a7fc640 1 -- 172.21.15.77:0/4108479332 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944031c30 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.183+0000 7fb93a7fc640 1 -- 172.21.15.77:0/4108479332 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944035030 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.183+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a4f90 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.185+0000 7fb93a7fc640 1 -- 172.21.15.77:0/4108479332 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb944042070 con 0x7fb948102570 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.185+0000 7fb9517df740 1 Processor -- start 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.185+0000 7fb9517df740 1 -- start start 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6278db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f6283670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f61df090 con 0x55e9f6283bb0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6186a70 con 0x55e9f62792f0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb9517df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9f61db050 con 0x55e9f62789d0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58468/0 (socket says 172.21.15.77:58468) 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb93a7fc640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 0x7fb9340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94c9a1640 1 -- 172.21.15.77:0/2331993965 learned_addr learned my addr 172.21.15.77:0/2331993965 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f6283670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6278db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 0x7fb9340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 -- 172.21.15.77:0/2331993965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 msgr2=0x55e9f6278db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6278db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 -- 172.21.15.77:0/2331993965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 msgr2=0x55e9f6285fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb93a7fc640 1 -- 172.21.15.77:0/4108479332 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb9440f0420 con 0x7fb948102570 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 -- 172.21.15.77:0/2331993965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f6193130 con 0x55e9f62792f0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6278db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.186+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f6283670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00bc80 tx=0x7fb93c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d5c7a66af6031ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 0x7fb9340a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb940005fd0 tx=0x7fb940009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb9337fe640 1 -- 172.21.15.77:0/2331993965 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c015070 con 0x55e9f62792f0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb9337fe640 1 -- 172.21.15.77:0/2331993965 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93c011920 con 0x55e9f62792f0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb9337fe640 1 -- 172.21.15.77:0/2331993965 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0108e0 con 0x55e9f62792f0 2024-09-15T22:10:15.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb9517df740 1 -- 172.21.15.77:0/2331993965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 msgr2=0x55e9f6283670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.187+0000 7fb9517df740 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f6283670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00bc80 tx=0x7fb93c00f730 comp rx=0 tx=0).stop 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 -- 172.21.15.77:0/2331993965 shutdown_connections 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6285fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f6283670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 --2- 172.21.15.77:0/2331993965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6278db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 -- 172.21.15.77:0/2331993965 >> 172.21.15.77:0/2331993965 conn(0x55e9f6274f60 msgr2=0x55e9f6275340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 -- 172.21.15.77:0/2331993965 shutdown_connections 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 -- 172.21.15.77:0/2331993965 wait complete. 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 -- start start 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.188+0000 7fb9517df740 1 Processor -- start 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6283390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 0x55e9f6280c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6286780 con 0x55e9f62792f0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6199f20 con 0x55e9f6283bb0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6286900 con 0x55e9f62789d0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 10 client.0 _ll_register_callbacks cb 0x55e9f6163030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58470/0 (socket says 172.21.15.77:58470) 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 -- 172.21.15.77:0/2080996482 learned_addr learned my addr 172.21.15.77:0/2080996482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6283390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 0x55e9f6280c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 -- 172.21.15.77:0/2080996482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 msgr2=0x55e9f6283390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6283390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 -- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 msgr2=0x55e9f6280c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 0x55e9f6280c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 -- 172.21.15.77:0/2080996482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f5f502b0 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 0x55e9f6280c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb944005940 tx=0x7fb9440b8440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3350cf16af5517f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944049030 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb9440b8e10 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93c011920 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb9440bcea0 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 10 client.14709 fetch_fsmap learned FSMap version 17 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.189+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e9f6292010 con 0x55e9f62792f0 2024-09-15T22:10:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93c011920 con 0x55e9f62792f0 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb94403b460 con 0x55e9f62792f0 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 20 client.14709 populate_metadata read hostname 'smithi077' 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6294420 0x55e9f6295890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 4 client.14709 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f61ec600 con 0x55e9f6294420 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.190+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6294420 0x55e9f6295890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb930ff9640 10 client.14709 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6294420 0x55e9f6295890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5edb38b59964ee9 server_cookie=ff5d7620a47ae09e in_seq=0 out_seq=0 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb930ff9640 1 -- 172.21.15.77:0/2080996482 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9f61ec600 con 0x55e9f6294420 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb930ff9640 10 client.14709 handle_command_reply: tid=0 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb9517df740 1 client.14709 shutdown 2024-09-15T22:10:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 msgr2=0x55e9f62806e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.191+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb944005940 tx=0x7fb9440b8440 comp rx=0 tx=0).stop 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 shutdown_connections 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6294420 0x55e9f6295890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f6283bb0 0x55e9f6280c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f62806e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb930ff9640 0 client.14709 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 --2- 172.21.15.77:0/2080996482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f6283390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 >> 172.21.15.77:0/2080996482 conn(0x55e9f6274f60 msgr2=0x55e9f627ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 shutdown_connections 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 1 -- 172.21.15.77:0/2080996482 wait complete. 2024-09-15T22:10:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.192+0000 7fb9517df740 20 client.14709 trim_cache size 0 max 16384 2024-09-15T22:10:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 Processor -- start 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 -- start start 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62789d0 0x55e9f6293d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f6283bb0 0x55e9f627efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6293bd0 con 0x55e9f62792f0 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9f62942a0 con 0x55e9f62789d0 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb9517df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6294970 con 0x55e9f6283bb0 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb94c9a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58472/0 (socket says 172.21.15.77:58472) 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.210+0000 7fb94c9a1640 1 -- 172.21.15.77:0/3628959936 learned_addr learned my addr 172.21.15.77:0/3628959936 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62789d0 0x55e9f6293d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 -- 172.21.15.77:0/3628959936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f6283bb0 msgr2=0x55e9f627efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f6283bb0 0x55e9f627efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 -- 172.21.15.77:0/3628959936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62789d0 msgr2=0x55e9f6293d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62789d0 0x55e9f6293d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 -- 172.21.15.77:0/3628959936 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f6279120 con 0x55e9f62792f0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb944005ab0 tx=0x7fb944009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0f89d6bc09a931 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb932ffd640 1 -- 172.21.15.77:0/3628959936 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944049040 con 0x55e9f62792f0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb932ffd640 1 -- 172.21.15.77:0/3628959936 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944031de0 con 0x55e9f62792f0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.211+0000 7fb932ffd640 1 -- 172.21.15.77:0/3628959936 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440bbbf0 con 0x55e9f62792f0 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- 172.21.15.77:0/3628959936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 msgr2=0x55e9f626fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb944005ab0 tx=0x7fb944009290 comp rx=0 tx=0).stop 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- 172.21.15.77:0/3628959936 shutdown_connections 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f6283bb0 0x55e9f627efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f62792f0 0x55e9f626fce0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 --2- 172.21.15.77:0/3628959936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62789d0 0x55e9f6293d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- 172.21.15.77:0/3628959936 >> 172.21.15.77:0/3628959936 conn(0x55e9f6274f60 msgr2=0x55e9f6275340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- 172.21.15.77:0/3628959936 shutdown_connections 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- 172.21.15.77:0/3628959936 wait complete. 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 -- start start 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.212+0000 7fb9517df740 1 Processor -- start 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f629ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f629e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6295710 con 0x55e9f6283bb0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6295040 con 0x55e9f62792f0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6295de0 con 0x55e9f62789d0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb9517df740 10 client.0 _ll_register_callbacks cb 0x7fb944005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d1a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f629ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58484/0 (socket says 172.21.15.77:58484) 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 -- 172.21.15.77:0/2974071375 learned_addr learned my addr 172.21.15.77:0/2974071375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f629e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 -- 172.21.15.77:0/2974071375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 msgr2=0x55e9f629ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f629ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 -- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 msgr2=0x55e9f629e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f629e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 -- 172.21.15.77:0/2974071375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f6297a90 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94c9a1640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f629e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb94d9a3640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb940010b70 tx=0x7fb940009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bbce475adbaf34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.213+0000 7fb930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940004d20 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb940021920 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb944031de0 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb940021b20 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 10 client.14715 fetch_fsmap learned FSMap version 17 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e9f629c550 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb944031de0 con 0x55e9f6283bb0 2024-09-15T22:10:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb940028070 con 0x55e9f6283bb0 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 20 client.14715 populate_metadata read hostname 'smithi077' 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6303390 0x55e9f63020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 4 client.14715 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.214+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "status"}) -- 0x55e9f61e6b10 con 0x55e9f6303390 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6303390 0x55e9f63020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb94d1a2640 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6303390 0x55e9f63020a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f7dc3f175cb328 server_cookie=a8150db15c1a41c3 in_seq=0 out_seq=0 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb930ff9640 10 client.14715 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb930ff9640 1 -- 172.21.15.77:0/2974071375 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55e9f61e6b10 con 0x55e9f6303390 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb930ff9640 10 client.14715 handle_command_reply: tid=0 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 client.14715 shutdown 2024-09-15T22:10:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 msgr2=0x55e9f6297260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb940010b70 tx=0x7fb940009d10 comp rx=0 tx=0).stop 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 shutdown_connections 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9f6303390 0x55e9f63020a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f6283bb0 0x55e9f6297260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb930ff9640 0 client.14715 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9f62792f0 0x55e9f629e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 --2- 172.21.15.77:0/2974071375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9f62789d0 0x55e9f629ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.215+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 >> 172.21.15.77:0/2974071375 conn(0x55e9f6274f60 msgr2=0x55e9f626fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 shutdown_connections 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb9517df740 1 -- 172.21.15.77:0/2974071375 wait complete. 2024-09-15T22:10:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb9517df740 20 client.14715 trim_cache size 0 max 16384 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24458, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 19.017838069, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 17, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 88, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 87, 2024-09-15T22:10:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 29.821678141 2024-09-15T22:10:15.233 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:10:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 msgr2=0x7fb9340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 0x7fb9340a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb940005fd0 tx=0x7fb940009600 comp rx=0 tx=0).stop 2024-09-15T22:10:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 msgr2=0x7fb948199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb944003c10 comp rx=0 tx=0).stop 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 shutdown_connections 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb9340a3240 0x7fb9340a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948102ef0 0x7fb9481a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb948102570 0x7fb948199890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 --2- 172.21.15.77:0/4108479332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb948100e20 0x7fb948199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 >> 172.21.15.77:0/4108479332 conn(0x7fb9480fc7d0 msgr2=0x7fb9480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 shutdown_connections 2024-09-15T22:10:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:15.216+0000 7fb94f406640 1 -- 172.21.15.77:0/4108479332 wait complete. 2024-09-15T22:10:15.536 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:15 smithi077 ceph-mon[30980]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:15 smithi196 ceph-mon[35109]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:15 smithi117 ceph-mon[35021]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:15.982 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:10:15.982 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:15.983 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:10:16.226 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:16.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 -- 172.21.15.77:0/1418616267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c101420 msgr2=0x7ff75c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:16.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 --2- 172.21.15.77:0/1418616267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c101420 0x7ff75c10d950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff754009cd0 tx=0x7ff75402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 -- 172.21.15.77:0/1418616267 shutdown_connections 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 --2- 172.21.15.77:0/1418616267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c101420 0x7ff75c10d950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 --2- 172.21.15.77:0/1418616267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c100a80 0x7ff75c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 --2- 172.21.15.77:0/1418616267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c106a80 0x7ff75c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 -- 172.21.15.77:0/1418616267 >> 172.21.15.77:0/1418616267 conn(0x7ff75c0fc7a0 msgr2=0x7ff75c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 -- 172.21.15.77:0/1418616267 shutdown_connections 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.734+0000 7ff762a88640 1 -- 172.21.15.77:0/1418616267 wait complete. 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 Processor -- start 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 -- start start 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 0x7ff75c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c101420 0x7ff75c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:16.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c110e80 con 0x7ff75c106a80 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c110d00 con 0x7ff75c100a80 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff762a88640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c111000 con 0x7ff75c101420 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58502/0 (socket says 172.21.15.77:58502) 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 -- 172.21.15.77:0/2022064293 learned_addr learned my addr 172.21.15.77:0/2022064293 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff760824640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 0x7ff75c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 -- 172.21.15.77:0/2022064293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c101420 msgr2=0x7ff75c198970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c101420 0x7ff75c198970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 -- 172.21.15.77:0/2022064293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 msgr2=0x7ff75c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 0x7ff75c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 -- 172.21.15.77:0/2022064293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c19d3e0 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff760824640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 0x7ff75c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff761025640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff754005cc0 tx=0x7ff754003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.735+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754004040 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.736+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c19d670 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.736+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7540046c0 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.736+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754046480 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.736+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1a4fb0 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.737+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c100ee0 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.738+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff7540305a0 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.738+0000 7ff759ffb640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 0x7ff74c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.739+0000 7ff760824640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 0x7ff74c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.739+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff75402e030 con 0x7ff75c106a80 2024-09-15T22:10:16.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.739+0000 7ff760824640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 0x7ff74c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff750005fd0 tx=0x7ff750005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:16.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.743+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff75404b060 con 0x7ff75c106a80 2024-09-15T22:10:16.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.942+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff75c1a5180 con 0x7ff75c106a80 2024-09-15T22:10:16.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.942+0000 7ff759ffb640 1 -- 172.21.15.77:0/2022064293 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff7540bdf40 con 0x7ff75c106a80 2024-09-15T22:10:16.944 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.944+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 msgr2=0x7ff74c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.944+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 0x7ff74c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff750005fd0 tx=0x7ff750005850 comp rx=0 tx=0).stop 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.944+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 msgr2=0x7ff75c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.944+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff754005cc0 tx=0x7ff754003970 comp rx=0 tx=0).stop 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 shutdown_connections 2024-09-15T22:10:16.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff74c0a3240 0x7ff74c0a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff75c106a80 0x7ff75c19cd00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c101420 0x7ff75c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 --2- 172.21.15.77:0/2022064293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75c100a80 0x7ff75c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 >> 172.21.15.77:0/2022064293 conn(0x7ff75c0fc7a0 msgr2=0x7ff75c104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 shutdown_connections 2024-09-15T22:10:16.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:16.945+0000 7ff762a88640 1 -- 172.21.15.77:0/2022064293 wait complete. 2024-09-15T22:10:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:17 smithi077 ceph-mon[30980]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:17 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2022064293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:10:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:17 smithi196 ceph-mon[35109]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:17 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2022064293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:10:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:17 smithi117 ceph-mon[35021]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:17 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2022064293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:10:17.734 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:10:17.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:17.735 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-09-15T22:10:17.752 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42318sec preferred_lft 42318sec 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-15T22:10:17.753 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:10:17.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:10:17.754 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:10:17.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:10:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:17.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:17.963 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:10:17.987 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-15T22:10:17.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:17.990 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-09-15T22:10:18.061 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-09-15T22:10:18.061 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:10:18.061 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-15T22:10:18.061 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-15T22:10:18.061 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-15T22:10:18.062 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:10:18.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:10:18.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:18.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:18.368 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-15T22:10:18.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:18.384 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-09-15T22:10:18.440 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:10:18.440 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:10:18.440 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:10:18.440 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-15T22:10:18.440 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:10:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:10:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:18.557 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-15T22:10:18.557 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:10:18.557 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:10:18.557 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-15T22:10:18.558 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:10:18.558 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-15T22:10:18.558 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-15T22:10:18.558 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-15T22:10:18.558 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:10:18.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:10:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:18.679 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:10:18.679 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:10:18.680 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-09-15T22:10:18.680 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:10:18.680 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:10:18.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:10:18.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:18.755 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-15T22:10:18.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:10:18.756 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:18.771 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:10:18.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:10:18.771 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:18.827 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-09-15T22:10:18.892 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-09-15T22:10:18.950 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:10:18.951 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/cdc568ad2c1767b524cac21382230d8779d34d0fcf0330af21a905009bafd12d/merged 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8801e02c35cdbf0885f2a13be2272572d27e4967d43a5c8037f17b8fc3e66052/merged 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/49ffaa62f42d4db8c284a7902f59bce2470cf8a38c9cf5d85524f79aaa7b870a/merged 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/212681802f5f431cc775c043870d826d9d0e7eb6248995330f295dd1898697aa/merged 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/01cde458dbabf8bd9f0f4c889b9e3656869af31f0bb0cab8be3e3ed910e3e075/merged 2024-09-15T22:10:18.952 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1e712cae653b5b2ebe6e935c293cd7063e8ef5ba2414b114d41fb12ecf04550c/merged 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/168ce4494edb39088f3d53c5e815bf1c88fcd7a859cff6920f35fc0b862d8073/merged 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/938d89a3508328989b5041e9eddf0b14e391988876bc4a2cf1de7ce1513e4bae/merged 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/9ca0cdf2d1cc079661865f972a8a14792c20040a5dd8a3156f62af0e07c7ff86/merged 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/a1c9e2be082da7c0bab17bdba1476cfdff05fda314b91cc25a179e7cd01877b2/merged 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:10:18.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:18.953 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-09-15T22:10:19.005 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-15T22:10:19.006 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-15T22:10:19.049 DEBUG:teuthology.orchestra.run.smithi077:> sudo modprobe fuse 2024-09-15T22:10:19.074 DEBUG:teuthology.orchestra.run.smithi077:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:10:19.132 INFO:teuthology.orchestra.run.smithi077.stdout:/proc 2024-09-15T22:10:19.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys 2024-09-15T22:10:19.132 INFO:teuthology.orchestra.run.smithi077.stdout:/dev 2024-09-15T22:10:19.132 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/security 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/shm 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/pts 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/run 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/cgroup 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/pstore 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/bpf 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/ 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/selinux 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/mqueue 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/dev/hugepages 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/debug 2024-09-15T22:10:19.133 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/tracing 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/fs/fuse/connections 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/sys/kernel/config 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/1000 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/ceph 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/cdc568ad2c1767b524cac21382230d8779d34d0fcf0330af21a905009bafd12d/merged 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/8801e02c35cdbf0885f2a13be2272572d27e4967d43a5c8037f17b8fc3e66052/merged 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/run/user/0 2024-09-15T22:10:19.134 INFO:teuthology.orchestra.run.smithi077.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/49ffaa62f42d4db8c284a7902f59bce2470cf8a38c9cf5d85524f79aaa7b870a/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/212681802f5f431cc775c043870d826d9d0e7eb6248995330f295dd1898697aa/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/01cde458dbabf8bd9f0f4c889b9e3656869af31f0bb0cab8be3e3ed910e3e075/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/1e712cae653b5b2ebe6e935c293cd7063e8ef5ba2414b114d41fb12ecf04550c/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/168ce4494edb39088f3d53c5e815bf1c88fcd7a859cff6920f35fc0b862d8073/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/938d89a3508328989b5041e9eddf0b14e391988876bc4a2cf1de7ce1513e4bae/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/9ca0cdf2d1cc079661865f972a8a14792c20040a5dd8a3156f62af0e07c7ff86/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/var/lib/containers/storage/overlay/a1c9e2be082da7c0bab17bdba1476cfdff05fda314b91cc25a179e7cd01877b2/merged 2024-09-15T22:10:19.135 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns 2024-09-15T22:10:19.136 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:10:19.136 INFO:teuthology.orchestra.run.smithi077.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:10:19.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:19.136 DEBUG:teuthology.orchestra.run.smithi077:> ls /sys/fs/fuse/connections 2024-09-15T22:10:19.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:2024-09-15T22:10:19.152+0000 7f79e3956480 -1 init, newargv = 0x564a22337b30 newargc=15 2024-09-15T22:10:19.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74624]: starting ceph client 2024-09-15T22:10:19.160 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74624]: starting fuse 2024-09-15T22:10:19.169 INFO:teuthology.orchestra.run.smithi077.stdout:120 2024-09-15T22:10:19.170 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> sudo stdin-killer -- python3 -c ' 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> import glob 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> import re 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> import os 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> import subprocess 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> def _find_admin_socket(client_name): 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> files = glob.glob(asok_path) 2024-09-15T22:10:19.170 DEBUG:teuthology.orchestra.run.smithi077:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> # Given a non-glob path, it better be there 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> if "*" not in asok_path: 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> assert(len(files) == 1) 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> return files[0] 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> for f in files: 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> contents = proc_f.read() 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> if mountpoint in contents: 2024-09-15T22:10:19.171 DEBUG:teuthology.orchestra.run.smithi077:> return f 2024-09-15T22:10:19.172 DEBUG:teuthology.orchestra.run.smithi077:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-15T22:10:19.172 DEBUG:teuthology.orchestra.run.smithi077:> 2024-09-15T22:10:19.172 DEBUG:teuthology.orchestra.run.smithi077:> print(_find_admin_socket("client.admin")) 2024-09-15T22:10:19.172 DEBUG:teuthology.orchestra.run.smithi077:> ' 2024-09-15T22:10:19.289 INFO:teuthology.orchestra.run.smithi077.stdout:/var/run/ceph/ceph-client.admin.74624.asok 2024-09-15T22:10:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:19.300 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74624.asok 2024-09-15T22:10:19.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:19.301 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74624.asok status 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout: "metadata": { 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout: "entity_id": "admin", 2024-09-15T22:10:19.434 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": "smithi077", 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "pid": "74624", 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "root": "/" 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_count": 0, 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "dentry_pinned_count": 0, 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14727, 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "inst": { 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": { 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "client", 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: "num": 14727 2024-09-15T22:10:19.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2553994558 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": { 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2553994558 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "inst_str": "client.14727 192.168.0.1:0/2553994558", 2024-09-15T22:10:19.436 INFO:teuthology.orchestra.run.smithi077.stdout: "addr_str": "192.168.0.1:0/2553994558", 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "inode_count": 1, 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_epoch": 17, 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch": 88, 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_epoch_barrier": 0, 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "blocklisted": false, 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs" 2024-09-15T22:10:19.437 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:10:19.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:19.443 DEBUG:teuthology.orchestra.run.smithi077:> stat --file-system '--printf=%T 2024-09-15T22:10:19.443 DEBUG:teuthology.orchestra.run.smithi077:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:19.458 INFO:teuthology.orchestra.run.smithi077.stdout:fuseblk 2024-09-15T22:10:19.459 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:19.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:19.459 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:19.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:19 smithi077 ceph-mon[30980]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:19.525 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-15T22:10:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:19 smithi196 ceph-mon[35109]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:19 smithi117 ceph-mon[35021]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:10:19.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:10:19.636 INFO:teuthology.orchestra.run.smithi077.stdout:# file: . 2024-09-15T22:10:19.636 INFO:teuthology.orchestra.run.smithi077.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-15T22:10:19.636 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:10:19.642 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:10:19.642 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:10:19.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:10:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:20.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.446+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/1579960386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0108290 msgr2=0x7ff3c010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.446+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/1579960386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0108290 0x7ff3c010ec30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009930 tx=0x7ff3b802a420 comp rx=0 tx=0).stop 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/1579960386 shutdown_connections 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/1579960386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0108290 0x7ff3c010ec30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/1579960386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0103740 0x7ff3c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/1579960386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/1579960386 >> 172.21.15.77:0/1579960386 conn(0x7ff3c00fc7d0 msgr2=0x7ff3c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/1579960386 shutdown_connections 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/1579960386 wait complete. 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 Processor -- start 2024-09-15T22:10:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- start start 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 0x7ff3c019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0112090 con 0x7ff3c0103740 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0111f10 con 0x7ff3c0102d90 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3c6cdb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0112210 con 0x7ff3c0108290 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43222/0 (socket says 172.21.15.77:43222) 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.447+0000 7ff3bffff640 1 -- 172.21.15.77:0/2019589315 learned_addr learned my addr 172.21.15.77:0/2019589315 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c4a77640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c5278640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 0x7ff3c019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 -- 172.21.15.77:0/2019589315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 msgr2=0x7ff3c019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 0x7ff3c019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 -- 172.21.15.77:0/2019589315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 msgr2=0x7ff3c0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 -- 172.21.15.77:0/2019589315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c019d510 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c5278640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 0x7ff3c019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c4a77640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bffff640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005fd0 tx=0x7ff3b400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4017070 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c019d800 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4009c30 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4010460 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.448+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01a50e0 con 0x7ff3c0103740 2024-09-15T22:10:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.449+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c0103170 con 0x7ff3c0103740 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.450+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff3b4003e70 con 0x7ff3c0103740 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.451+0000 7ff3bdffb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 0x7ff3a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.451+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff3b40cfff0 con 0x7ff3c0103740 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.451+0000 7ff3c4a77640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 0x7ff3a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.451+0000 7ff3c4a77640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 0x7ff3a80a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0005fd0 tx=0x7ff3b00093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.454+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3b4020030 con 0x7ff3c0103740 2024-09-15T22:10:20.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.631+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff3c010a2d0 con 0x7ff3c0103740 2024-09-15T22:10:20.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.631+0000 7ff3bdffb640 1 -- 172.21.15.77:0/2019589315 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+591 (secure 0 0 0) 0x7ff3b409cad0 con 0x7ff3c0103740 2024-09-15T22:10:20.633 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1261342424 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:20.633 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1397043946 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3468864668 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3493953936 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/573117065 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2559994639 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:20.634 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 msgr2=0x7ff3a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 0x7ff3a80a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0005fd0 tx=0x7ff3b00093e0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 msgr2=0x7ff3c0198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005fd0 tx=0x7ff3b400dde0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 shutdown_connections 2024-09-15T22:10:20.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff3a80a3240 0x7ff3a80a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3c0108290 0x7ff3c019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3c0103740 0x7ff3c0198aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 --2- 172.21.15.77:0/2019589315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3c0102d90 0x7ff3c0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 >> 172.21.15.77:0/2019589315 conn(0x7ff3c00fc7d0 msgr2=0x7ff3c0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 shutdown_connections 2024-09-15T22:10:20.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:20.633+0000 7ff3c6cdb640 1 -- 172.21.15.77:0/2019589315 wait complete. 2024-09-15T22:10:21.343 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:21 smithi077 ceph-mon[30980]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:21.343 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:21 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2019589315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:21.345 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:10:21.345 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:10:21.364 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:10:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:21 smithi196 ceph-mon[35109]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:21.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:21 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2019589315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:21 smithi117 ceph-mon[35021]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:21 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2019589315' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:22.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- 172.21.15.77:0/215453631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 msgr2=0x7f9a8c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:22.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 --2- 172.21.15.77:0/215453631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c10eaa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009ca0 tx=0x7f9a8402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- 172.21.15.77:0/215453631 shutdown_connections 2024-09-15T22:10:22.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 --2- 172.21.15.77:0/215453631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c10eaa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 --2- 172.21.15.77:0/215453631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c1027a0 0x7f9a8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 --2- 172.21.15.77:0/215453631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c101df0 0x7f9a8c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- 172.21.15.77:0/215453631 >> 172.21.15.77:0/215453631 conn(0x7f9a8c0fd8b0 msgr2=0x7f9a8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- 172.21.15.77:0/215453631 shutdown_connections 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- 172.21.15.77:0/215453631 wait complete. 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 Processor -- start 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.163+0000 7f9a926d9640 1 -- start start 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c101df0 0x7f9a8c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:22.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c1027a0 0x7f9a8c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c111f80 con 0x7f9a8c106f20 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c111e00 con 0x7f9a8c1027a0 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a926d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c112100 con 0x7f9a8c101df0 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:22.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43236/0 (socket says 172.21.15.77:43236) 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 -- 172.21.15.77:0/3680339398 learned_addr learned my addr 172.21.15.77:0/3680339398 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a8bfff640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c101df0 0x7f9a8c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a8b7fe640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c1027a0 0x7f9a8c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 -- 172.21.15.77:0/3680339398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c101df0 msgr2=0x7f9a8c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c101df0 0x7f9a8c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 -- 172.21.15.77:0/3680339398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c1027a0 msgr2=0x7f9a8c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c1027a0 0x7f9a8c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 -- 172.21.15.77:0/3680339398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c19d350 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.164+0000 7f9a90c76640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a84031a60 tx=0x7f9a84031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.165+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84003c30 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.165+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a84003dd0 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.165+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840047e0 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.165+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c19d5e0 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.165+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1a4f20 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.166+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c1021d0 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.167+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9a8402f790 con 0x7f9a8c106f20 2024-09-15T22:10:22.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.168+0000 7f9a897fa640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 0x7f9a740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:22.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.168+0000 7f9a8bfff640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 0x7f9a740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:22.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.168+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f9a840f10f0 con 0x7f9a8c106f20 2024-09-15T22:10:22.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.168+0000 7f9a8bfff640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 0x7f9a740a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0058c0 tx=0x7f9a7c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:22.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.171+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a840bdbd0 con 0x7f9a8c106f20 2024-09-15T22:10:22.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.349+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a8c10a100 con 0x7f9a8c106f20 2024-09-15T22:10:22.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.350+0000 7f9a897fa640 1 -- 172.21.15.77:0/3680339398 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+591 (secure 0 0 0) 0x7f9a840bd4d0 con 0x7f9a8c106f20 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1261342424 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1397043946 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3468864668 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3493953936 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/573117065 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2559994639 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:22.352 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:22.353 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:10:22.353 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2024-09-15T22:10:22.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 msgr2=0x7f9a740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 0x7f9a740a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0058c0 tx=0x7f9a7c00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 msgr2=0x7f9a8c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a84031a60 tx=0x7f9a84031a90 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 shutdown_connections 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a740a3240 0x7f9a740a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a8c106f20 0x7f9a8c19cc70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a8c1027a0 0x7f9a8c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 --2- 172.21.15.77:0/3680339398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a8c101df0 0x7f9a8c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 >> 172.21.15.77:0/3680339398 conn(0x7f9a8c0fd8b0 msgr2=0x7f9a8c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 shutdown_connections 2024-09-15T22:10:22.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:22.352+0000 7f9a926d9640 1 -- 172.21.15.77:0/3680339398 wait complete. 2024-09-15T22:10:23.003 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi077.front.sepia.ceph.com... 2024-09-15T22:10:23.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:23.004 DEBUG:teuthology.orchestra.run.smithi077:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:10:23.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi077.stderr:ceph-fuse[74624]: fuse finished with error 0 and tester_r 0 2024-09-15T22:10:23.032 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T22:10:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:23 smithi077 ceph-mon[30980]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:23.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:23 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3680339398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:23.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:23 smithi196 ceph-mon[35109]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:23.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:23 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3680339398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:23 smithi117 ceph-mon[35021]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:23 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3680339398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:10:25.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:25 smithi077 ceph-mon[30980]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:25 smithi196 ceph-mon[35109]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:25 smithi117 ceph-mon[35021]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:10:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:27 smithi077 ceph-mon[30980]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:27 smithi196 ceph-mon[35109]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:27 smithi117 ceph-mon[35021]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:29.134 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:10:29.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:10:29.135 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-15T22:10:29.165 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:29.473 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:29.518 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:29 smithi077 ceph-mon[30980]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:29.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:29 smithi196 ceph-mon[35109]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:29.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:29 smithi117 ceph-mon[35021]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.959+0000 7f186f716640 1 -- 172.21.15.77:0/778375841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868102ef0 msgr2=0x7f186810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.959+0000 7f186f716640 1 --2- 172.21.15.77:0/778375841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868102ef0 0x7f186810da40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ca0 tx=0x7f186002a950 comp rx=0 tx=0).stop 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- 172.21.15.77:0/778375841 shutdown_connections 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- 172.21.15.77:0/778375841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868102ef0 0x7f186810da40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- 172.21.15.77:0/778375841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102570 0x7f18681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- 172.21.15.77:0/778375841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868100e20 0x7f1868101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- 172.21.15.77:0/778375841 >> 172.21.15.77:0/778375841 conn(0x7f18680fc7d0 msgr2=0x7f18680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- 172.21.15.77:0/778375841 shutdown_connections 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- 172.21.15.77:0/778375841 wait complete. 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 Processor -- start 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- start start 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 0x7f18680fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 0x7f18680fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1868110f80 con 0x7f1868100e20 2024-09-15T22:10:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1868110e00 con 0x7f1868102ef0 2024-09-15T22:10:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.960+0000 7f186f716640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111100 con 0x7f1868102570 2024-09-15T22:10:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41342/0 (socket says 172.21.15.77:41342) 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 -- 172.21.15.77:0/1704146651 learned_addr learned my addr 172.21.15.77:0/1704146651 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186ccb1640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 0x7f18680fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186dcb3640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 0x7f18680fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 -- 172.21.15.77:0/1704146651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 msgr2=0x7f18680fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 0x7f18680fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 -- 172.21.15.77:0/1704146651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 msgr2=0x7f18680fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:29.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 0x7f18680fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:29.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 -- 172.21.15.77:0/1704146651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18681a67f0 con 0x7f1868100e20 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186dcb3640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 0x7f18680fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186ccb1640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 0x7f18680fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186d4b2640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f186400c920 tx=0x7f186400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864007c70 con 0x7f1868100e20 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681a6ae0 con 0x7f1868100e20 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1864007e10 con 0x7f1868100e20 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864018890 con 0x7f1868100e20 2024-09-15T22:10:29.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.961+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a7070 con 0x7f1868100e20 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.962+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1868101200 con 0x7f1868100e20 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.964+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1864018a50 con 0x7f1868100e20 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.964+0000 7f185e7fc640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 0x7f18540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.964+0000 7f186ccb1640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 0x7f18540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.964+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f18640cff80 con 0x7f1868100e20 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.965+0000 7f186ccb1640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 0x7f18540a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1858005e10 tx=0x7f1858005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:29.968+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1864014030 con 0x7f1868100e20 2024-09-15T22:10:30.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.139+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18680fe1b0 con 0x7f1868100e20 2024-09-15T22:10:30.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.140+0000 7f185e7fc640 1 -- 172.21.15.77:0/1704146651 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f186409caf0 con 0x7f1868100e20 2024-09-15T22:10:30.142 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:30.142 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:30.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 msgr2=0x7f18540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 0x7f18540a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1858005e10 tx=0x7f1858005da0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 msgr2=0x7f18680fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f186400c920 tx=0x7f186400cde0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 shutdown_connections 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f18540a3240 0x7f18540a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1868102ef0 0x7f18680fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1868102570 0x7f18680fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 --2- 172.21.15.77:0/1704146651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1868100e20 0x7f18680fcc80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 >> 172.21.15.77:0/1704146651 conn(0x7f18680fc7d0 msgr2=0x7f186806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 shutdown_connections 2024-09-15T22:10:30.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:30.143+0000 7f186f716640 1 -- 172.21.15.77:0/1704146651 wait complete. 2024-09-15T22:10:30.509 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:30 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1704146651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:30 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1704146651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:30 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1704146651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:31.147 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:31.511 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:31 smithi077 ceph-mon[30980]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:31 smithi196 ceph-mon[35109]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:31 smithi117 ceph-mon[35021]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:10:31.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.857+0000 7f86c5a93640 1 -- 172.21.15.77:0/2356650873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0102790 msgr2=0x7f86c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:31.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.857+0000 7f86c5a93640 1 --2- 172.21.15.77:0/2356650873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0102790 0x7f86c01067e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009900 tx=0x7f86a802a410 comp rx=0 tx=0).stop 2024-09-15T22:10:31.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- 172.21.15.77:0/2356650873 shutdown_connections 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- 172.21.15.77:0/2356650873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86c0106f10 0x7f86c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- 172.21.15.77:0/2356650873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0102790 0x7f86c01067e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- 172.21.15.77:0/2356650873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 0x7f86c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- 172.21.15.77:0/2356650873 >> 172.21.15.77:0/2356650873 conn(0x7f86c00fd8c0 msgr2=0x7f86c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- 172.21.15.77:0/2356650873 shutdown_connections 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- 172.21.15.77:0/2356650873 wait complete. 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 Processor -- start 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- start start 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 0x7f86c01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86c0102790 0x7f86c01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111f70 con 0x7f86c0106f10 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0111df0 con 0x7f86c0101de0 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.858+0000 7f86c5a93640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86c01120f0 con 0x7f86c0102790 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41362/0 (socket says 172.21.15.77:41362) 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 -- 172.21.15.77:0/4091244284 learned_addr learned my addr 172.21.15.77:0/4091244284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bf7fe640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 0x7f86c01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 -- 172.21.15.77:0/4091244284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86c0102790 msgr2=0x7f86c01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86c0102790 0x7f86c01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 -- 172.21.15.77:0/4091244284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 msgr2=0x7f86c01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 0x7f86c01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 -- 172.21.15.77:0/4091244284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c019d360 con 0x7f86c0106f10 2024-09-15T22:10:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bffff640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86b800cce0 tx=0x7f86b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b800f450 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b800fad0 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c019d650 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8018c10 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.859+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a4f30 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.860+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c01021c0 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.861+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f86b800f5f0 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.862+0000 7f86bd7fa640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.862+0000 7f86bf7fe640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.862+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f86b80d0250 con 0x7f86c0106f10 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.862+0000 7f86bf7fe640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 0x7f86ac0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86b000bbe0 tx=0x7f86b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:31.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:31.865+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86b8014030 con 0x7f86c0106f10 2024-09-15T22:10:32.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.033+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86c0199710 con 0x7f86c0106f10 2024-09-15T22:10:32.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.035+0000 7f86bd7fa640 1 -- 172.21.15.77:0/4091244284 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f86b809cdc0 con 0x7f86c0106f10 2024-09-15T22:10:32.037 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:32.038 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:32.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 msgr2=0x7f86ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:32.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 0x7f86ac0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86b000bbe0 tx=0x7f86b000b390 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 msgr2=0x7f86c019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86b800cce0 tx=0x7f86b800a620 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 shutdown_connections 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f86ac0a3240 0x7f86ac0a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86c0106f10 0x7f86c019cc80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86c0102790 0x7f86c01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 --2- 172.21.15.77:0/4091244284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86c0101de0 0x7f86c01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 >> 172.21.15.77:0/4091244284 conn(0x7f86c00fd8c0 msgr2=0x7f86c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 shutdown_connections 2024-09-15T22:10:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:32.037+0000 7f86c5a93640 1 -- 172.21.15.77:0/4091244284 wait complete. 2024-09-15T22:10:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:32 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4091244284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:32 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4091244284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:32 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4091244284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:32.843 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-15T22:10:33.091 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:33.499 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:33 smithi077 ceph-mon[30980]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 -- 172.21.15.77:0/4161225797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8100a80 msgr2=0x7f01b8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 --2- 172.21.15.77:0/4161225797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8100a80 0x7f01b8100ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009cd0 tx=0x7f01ac02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 -- 172.21.15.77:0/4161225797 shutdown_connections 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 --2- 172.21.15.77:0/4161225797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8101420 0x7f01b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 --2- 172.21.15.77:0/4161225797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8100a80 0x7f01b8100ee0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 --2- 172.21.15.77:0/4161225797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8106a80 0x7f01b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 -- 172.21.15.77:0/4161225797 >> 172.21.15.77:0/4161225797 conn(0x7f01b80fc7a0 msgr2=0x7f01b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 -- 172.21.15.77:0/4161225797 shutdown_connections 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.579+0000 7f01bf49d640 1 -- 172.21.15.77:0/4161225797 wait complete. 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 Processor -- start 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 -- start start 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8100a80 0x7f01b819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8106a80 0x7f01b819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8110e40 con 0x7f01b8101420 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8110cc0 con 0x7f01b8100a80 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bf49d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01b8110fc0 con 0x7f01b8106a80 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bca38640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.580+0000 7f01bca38640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41376/0 (socket says 172.21.15.77:41376) 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 -- 172.21.15.77:0/2246370210 learned_addr learned my addr 172.21.15.77:0/2246370210 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bda3a640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8106a80 0x7f01b819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bd239640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8100a80 0x7f01b819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 -- 172.21.15.77:0/2246370210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8106a80 msgr2=0x7f01b819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8106a80 0x7f01b819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 -- 172.21.15.77:0/2246370210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8100a80 msgr2=0x7f01b819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8100a80 0x7f01b819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 -- 172.21.15.77:0/2246370210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b819f5f0 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bca38640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02ae00 tx=0x7f01ac003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac004240 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ac0305a0 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b819f880 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac03a660 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.581+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b81a71c0 con 0x7f01b8101420 2024-09-15T22:10:33.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.582+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b8100ee0 con 0x7f01b8101420 2024-09-15T22:10:33.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.584+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f01ac030ac0 con 0x7f01b8101420 2024-09-15T22:10:33.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.585+0000 7f01b67fc640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 0x7f01a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.585+0000 7f01bd239640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 0x7f01a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.585+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f01ac02e030 con 0x7f01b8101420 2024-09-15T22:10:33.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.585+0000 7f01bd239640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 0x7f01a00a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005d90 tx=0x7f01a8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.588+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01ac0098b0 con 0x7f01b8101420 2024-09-15T22:10:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:33 smithi196 ceph-mon[35109]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:33 smithi117 ceph-mon[35021]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:33.792+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f01b81a73f0 con 0x7f01b8101420 2024-09-15T22:10:34.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.302+0000 7f01b67fc640 1 -- 172.21.15.77:0/2246370210 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f01ac0bdf30 con 0x7f01b8101420 2024-09-15T22:10:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 msgr2=0x7f01a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 0x7f01a00a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005d90 tx=0x7f01a8005d20 comp rx=0 tx=0).stop 2024-09-15T22:10:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 msgr2=0x7f01b819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01ac02ae00 tx=0x7f01ac003ba0 comp rx=0 tx=0).stop 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 shutdown_connections 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01a00a3240 0x7f01a00a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01b8106a80 0x7f01b819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01b8101420 0x7f01b819ab80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 --2- 172.21.15.77:0/2246370210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01b8100a80 0x7f01b819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 >> 172.21.15.77:0/2246370210 conn(0x7f01b80fc7a0 msgr2=0x7f01b806a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 shutdown_connections 2024-09-15T22:10:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:34.304+0000 7f01bf49d640 1 -- 172.21.15.77:0/2246370210 wait complete. 2024-09-15T22:10:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:10:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:10:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:10:35.067 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T22:10:35.067 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-15T22:10:35.319 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:35.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:35 smithi077 ceph-mon[30980]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:35.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:35 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:10:35.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:35 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:35.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:35 smithi196 ceph-mon[35109]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:10:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:35 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:35 smithi117 ceph-mon[35021]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:35 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2246370210' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:10:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:35 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 -- 172.21.15.77:0/2077330563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 msgr2=0x7f750810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 --2- 172.21.15.77:0/2077330563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f750810d940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7500009cd0 tx=0x7f750002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 -- 172.21.15.77:0/2077330563 shutdown_connections 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 --2- 172.21.15.77:0/2077330563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f750810d940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 --2- 172.21.15.77:0/2077330563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 0x7f7508100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 --2- 172.21.15.77:0/2077330563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 0x7f7508106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 -- 172.21.15.77:0/2077330563 >> 172.21.15.77:0/2077330563 conn(0x7f75080fc790 msgr2=0x7f75080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 -- 172.21.15.77:0/2077330563 shutdown_connections 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.780+0000 7f750d437640 1 -- 172.21.15.77:0/2077330563 wait complete. 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 Processor -- start 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 -- start start 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 0x7f75081992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 0x7f75081a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7508110e70 con 0x7f7508101410 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7508110cf0 con 0x7f7508100a70 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.781+0000 7f750d437640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7508110ff0 con 0x7f7508106a70 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41386/0 (socket says 172.21.15.77:41386) 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 -- 172.21.15.77:0/3972832139 learned_addr learned my addr 172.21.15.77:0/3972832139 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:35.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f7506ffd640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 0x7f75081992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f75077fe640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 0x7f75081a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 -- 172.21.15.77:0/3972832139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 msgr2=0x7f75081a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 0x7f75081a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 -- 172.21.15.77:0/3972832139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 msgr2=0x7f75081992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 0x7f75081992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 -- 172.21.15.77:0/3972832139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75081a4860 con 0x7f7508101410 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f74fe7fc640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74f000d950 tx=0x7f74f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0017030 con 0x7f7508101410 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f0004020 con 0x7f7508101410 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081a4a30 con 0x7f7508101410 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f00047a0 con 0x7f7508101410 2024-09-15T22:10:35.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.782+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75081a4f70 con 0x7f7508101410 2024-09-15T22:10:35.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.783+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7508100ed0 con 0x7f7508101410 2024-09-15T22:10:35.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.784+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f74f00041d0 con 0x7f7508101410 2024-09-15T22:10:35.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.785+0000 7f7504ff9640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 0x7f74f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.785+0000 7f7506ffd640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 0x7f74f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.785+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f74f00cf4c0 con 0x7f7508101410 2024-09-15T22:10:35.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.786+0000 7f7506ffd640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 0x7f74f40a5780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005fd0 tx=0x7f74f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.789+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74f009bfa0 con 0x7f7508101410 2024-09-15T22:10:35.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:35.990+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f750802d7c0 con 0x7f7508101410 2024-09-15T22:10:36.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.313+0000 7f7504ff9640 1 -- 172.21.15.77:0/3972832139 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f74f009b8a0 con 0x7f7508101410 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 msgr2=0x7f74f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 0x7f74f40a5780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005fd0 tx=0x7f74f800b390 comp rx=0 tx=0).stop 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 msgr2=0x7f75081997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74f000d950 tx=0x7f74f000de10 comp rx=0 tx=0).stop 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 shutdown_connections 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74f40a32c0 0x7f74f40a5780 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:36.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7508106a70 0x7f75081a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:36.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7508101410 0x7f75081997e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:36.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.315+0000 7f750d437640 1 --2- 172.21.15.77:0/3972832139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7508100a70 0x7f75081992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:36.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.316+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 >> 172.21.15.77:0/3972832139 conn(0x7f75080fc790 msgr2=0x7f75080fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:36.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.316+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 shutdown_connections 2024-09-15T22:10:36.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:36.316+0000 7f750d437640 1 -- 172.21.15.77:0/3972832139 wait complete. 2024-09-15T22:10:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:36 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T22:10:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:36 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T22:10:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:36 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-15T22:10:37.090 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-15T22:10:37.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:37.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:10:37.344 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:37.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T22:10:37.575 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:10:37.575 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:37.575 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:10:37.575 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:37 smithi077 ceph-mon[30980]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:10:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:10:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:37 smithi196 ceph-mon[35109]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:10:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-15T22:10:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:10:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3972832139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-15T22:10:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:10:37.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:10:37.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:10:37.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:37 smithi117 ceph-mon[35021]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3603746403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc101420 msgr2=0x7fb5cc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3603746403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc101420 0x7fb5cc10d950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009cd0 tx=0x7fb5c402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3603746403 shutdown_connections 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3603746403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc101420 0x7fb5cc10d950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3603746403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5cc100a80 0x7fb5cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3603746403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc106a80 0x7fb5cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3603746403 >> 172.21.15.77:0/3603746403 conn(0x7fb5cc0fc7a0 msgr2=0x7fb5cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3603746403 shutdown_connections 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.791+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3603746403 wait complete. 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 Processor -- start 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 -- start start 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc101420 0x7fb5cc1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5cc106a80 0x7fb5cc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc110e80 con 0x7fb5cc100a80 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc110d00 con 0x7fb5cc106a80 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d2be4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc111000 con 0x7fb5cc101420 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d0980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d0980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41402/0 (socket says 172.21.15.77:41402) 2024-09-15T22:10:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5d0980640 1 -- 172.21.15.77:0/3610330385 learned_addr learned my addr 172.21.15.77:0/3610330385 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.792+0000 7fb5c3fff640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc101420 0x7fb5cc1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 -- 172.21.15.77:0/3610330385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc101420 msgr2=0x7fb5cc1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc101420 0x7fb5cc1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 -- 172.21.15.77:0/3610330385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5cc106a80 msgr2=0x7fb5cc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5cc106a80 0x7fb5cc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 -- 172.21.15.77:0/3610330385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc19c240 con 0x7fb5cc100a80 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d0980640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d770 tx=0x7fb5c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8017070 con 0x7fb5cc100a80 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c8005500 con 0x7fb5cc100a80 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004e00 con 0x7fb5cc100a80 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc19c530 con 0x7fb5cc100a80 2024-09-15T22:10:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.793+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc1a3e10 con 0x7fb5cc100a80 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.794+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc100ee0 con 0x7fb5cc100a80 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.795+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb5c8004fa0 con 0x7fb5cc100a80 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.796+0000 7fb5c1ffb640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 0x7fb5b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.796+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb5c80cf620 con 0x7fb5cc100a80 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.796+0000 7fb5c3fff640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 0x7fb5b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.796+0000 7fb5c3fff640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 0x7fb5b40a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc005e10 tx=0x7fb5bc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:37.800+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5c809c100 con 0x7fb5cc100a80 2024-09-15T22:10:38.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.002+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb5cc1a3fe0 con 0x7fb5cc100a80 2024-09-15T22:10:38.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.329+0000 7fb5c1ffb640 1 -- 172.21.15.77:0/3610330385 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fb5c809ba00 con 0x7fb5cc100a80 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 msgr2=0x7fb5b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 0x7fb5b40a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc005e10 tx=0x7fb5bc005da0 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 msgr2=0x7fb5cc197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d770 tx=0x7fb5c800dc30 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 shutdown_connections 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5b40a3240 0x7fb5b40a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5cc106a80 0x7fb5cc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5cc101420 0x7fb5cc1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 --2- 172.21.15.77:0/3610330385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5cc100a80 0x7fb5cc197290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.331+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 >> 172.21.15.77:0/3610330385 conn(0x7fb5cc0fc7a0 msgr2=0x7fb5cc0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:38.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.332+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 shutdown_connections 2024-09-15T22:10:38.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:38.332+0000 7fb5d2be4640 1 -- 172.21.15.77:0/3610330385 wait complete. 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:active 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:10:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:38 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:active 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:10:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:38 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] up:active 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:10:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:38 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:10:39.140 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-15T22:10:39.388 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:39.507 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:10:39.507 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:10:39.507 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:active 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:active 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-15T22:10:39.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:39 smithi077 ceph-mon[30980]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:active 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T22:10:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:active 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-15T22:10:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:39 smithi196 ceph-mon[35109]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-15T22:10:39.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: pgmap v305: 129 pgs: 129 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3610330385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] up:active 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] up:active 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:10:39.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:10:39.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-15T22:10:39.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-15T22:10:39.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:39 smithi117 ceph-mon[35021]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-15T22:10:39.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- 172.21.15.77:0/38150692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494106f40 msgr2=0x7f549410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 --2- 172.21.15.77:0/38150692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494106f40 0x7f549410eac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f548c009ca0 tx=0x7f548c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- 172.21.15.77:0/38150692 shutdown_connections 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 --2- 172.21.15.77:0/38150692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494106f40 0x7f549410eac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 --2- 172.21.15.77:0/38150692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54941027c0 0x7f5494106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 --2- 172.21.15.77:0/38150692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5494101e10 0x7f54941021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- 172.21.15.77:0/38150692 >> 172.21.15.77:0/38150692 conn(0x7f54940fd8d0 msgr2=0x7f54940ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:39.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- 172.21.15.77:0/38150692 shutdown_connections 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- 172.21.15.77:0/38150692 wait complete. 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 Processor -- start 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.877+0000 7f549a4eb640 1 -- start start 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54941027c0 0x7f549419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5494106f40 0x7f549419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5494111fa0 con 0x7f5494101e10 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5494111e20 con 0x7f5494106f40 2024-09-15T22:10:39.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f549a4eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5494112120 con 0x7f54941027c0 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50300/0 (socket says 172.21.15.77:50300) 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 -- 172.21.15.77:0/2357409393 learned_addr learned my addr 172.21.15.77:0/2357409393 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f54937fe640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54941027c0 0x7f549419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5498a88640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5494106f40 0x7f549419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 -- 172.21.15.77:0/2357409393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54941027c0 msgr2=0x7f549419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54941027c0 0x7f549419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 -- 172.21.15.77:0/2357409393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5494106f40 msgr2=0x7f549419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5494106f40 0x7f549419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 -- 172.21.15.77:0/2357409393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549419f630 con 0x7f5494101e10 2024-09-15T22:10:39.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.878+0000 7f5493fff640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f548400d770 tx=0x7f548400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.879+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484017070 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.879+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549419f920 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.879+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5484005500 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.879+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484004e00 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.879+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54941a7200 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.880+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54941021f0 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.881+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5484004fa0 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.882+0000 7f54917fa640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 0x7f547c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.882+0000 7f54937fe640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 0x7f547c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.882+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f54840cf4d0 con 0x7f5494101e10 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.882+0000 7f54937fe640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 0x7f547c0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f54940fddd0 tx=0x7f548800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:39.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:39.886+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f548409b960 con 0x7f5494101e10 2024-09-15T22:10:40.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.088+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f54941a73d0 con 0x7f5494101e10 2024-09-15T22:10:40.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.748+0000 7f54917fa640 1 -- 172.21.15.77:0/2357409393 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f548409fc60 con 0x7f5494101e10 2024-09-15T22:10:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.750+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 msgr2=0x7f547c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.750+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 0x7f547c0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f54940fddd0 tx=0x7f548800b390 comp rx=0 tx=0).stop 2024-09-15T22:10:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 msgr2=0x7f549419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f548400d770 tx=0x7f548400dc30 comp rx=0 tx=0).stop 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 shutdown_connections 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f547c0a3240 0x7f547c0a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5494106f40 0x7f549419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54941027c0 0x7f549419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 --2- 172.21.15.77:0/2357409393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5494101e10 0x7f549419a680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 >> 172.21.15.77:0/2357409393 conn(0x7f54940fd8d0 msgr2=0x7f54940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 shutdown_connections 2024-09-15T22:10:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:40.751+0000 7f549a4eb640 1 -- 172.21.15.77:0/2357409393 wait complete. 2024-09-15T22:10:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:40 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:active 2024-09-15T22:10:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:40 smithi077 ceph-mon[30980]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-15T22:10:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:40 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:40 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:10:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:40 smithi077 ceph-mon[30980]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:10:41.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:40 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:active 2024-09-15T22:10:41.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:40 smithi196 ceph-mon[35109]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-15T22:10:41.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:40 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:41.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:40 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:10:41.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:40 smithi196 ceph-mon[35109]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:10:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:40 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] up:active 2024-09-15T22:10:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:40 smithi117 ceph-mon[35021]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-15T22:10:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:40 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:40 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:10:41.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:40 smithi117 ceph-mon[35021]: pgmap v306: 129 pgs: 129 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:10:41.460 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-15T22:10:41.460 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-15T22:10:41.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:10:41.252+0000 7f48be595640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-15T22:10:41.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:10:41.252+0000 7f48be595640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-15T22:10:41.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:10:41.252+0000 7f48be595640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-15T22:10:41.612 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:10:41.252+0000 7f48be595640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-15T22:10:41.709 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:42.008 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:41 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:10:42.008 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:41 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:42.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:41 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:10:42.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:41 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2357409393' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:10:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:41 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:10:42.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 -- 172.21.15.77:0/3513213211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 msgr2=0x7f5c48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:42.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/3513213211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c48101200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009900 tx=0x7f5c3802a480 comp rx=0 tx=0).stop 2024-09-15T22:10:42.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 -- 172.21.15.77:0/3513213211 shutdown_connections 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/3513213211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c48102ef0 0x7f5c4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/3513213211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102570 0x7f5c481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/3513213211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c48101200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 -- 172.21.15.77:0/3513213211 >> 172.21.15.77:0/3513213211 conn(0x7f5c480fc7d0 msgr2=0x7f5c480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 -- 172.21.15.77:0/3513213211 shutdown_connections 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.197+0000 7f5c4cac8640 1 -- 172.21.15.77:0/3513213211 wait complete. 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 Processor -- start 2024-09-15T22:10:42.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 -- start start 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c48102570 0x7f5c481999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 0x7f5c481a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48111020 con 0x7f5c48100e20 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48110ea0 con 0x7f5c48102ef0 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4cac8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481111a0 con 0x7f5c48102570 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50324/0 (socket says 172.21.15.77:50324) 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 -- 172.21.15.77:0/1988234229 learned_addr learned my addr 172.21.15.77:0/1988234229 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 -- 172.21.15.77:0/1988234229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c48102570 msgr2=0x7f5c481999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c48102570 0x7f5c481999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 -- 172.21.15.77:0/1988234229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 msgr2=0x7f5c481a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:42.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c46d9d640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 0x7f5c481a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 0x7f5c481a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.198+0000 7f5c4659c640 1 -- 172.21.15.77:0/1988234229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c481a4a70 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c46d9d640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 0x7f5c481a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c4659c640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c38006010 tx=0x7f5c38003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3803f070 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c38031e30 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c481a4c40 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38035030 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.199+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a5120 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.200+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c48101280 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.201+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5c38042070 con 0x7f5c48100e20 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.202+0000 7f5c3f7fe640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 0x7f5c340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:42.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.202+0000 7f5c45d9b640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 0x7f5c340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:42.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.202+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f5c380f04d0 con 0x7f5c48100e20 2024-09-15T22:10:42.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.202+0000 7f5c45d9b640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 0x7f5c340a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c4806ab00 tx=0x7f5c30006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:42.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.206+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c380bcfb0 con 0x7f5c48100e20 2024-09-15T22:10:42.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.397+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5c48062630 con 0x7f5c340a32c0 2024-09-15T22:10:42.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.427+0000 7f5c3f7fe640 1 -- 172.21.15.77:0/1988234229 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c48062630 con 0x7f5c340a32c0 2024-09-15T22:10:42.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 msgr2=0x7f5c340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 0x7f5c340a5780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c4806ab00 tx=0x7f5c30006c90 comp rx=0 tx=0).stop 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 msgr2=0x7f5c481994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5c38006010 tx=0x7f5c38003c80 comp rx=0 tx=0).stop 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 shutdown_connections 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5c340a32c0 0x7f5c340a5780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c48102ef0 0x7f5c481a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c48102570 0x7f5c481999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 --2- 172.21.15.77:0/1988234229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c48100e20 0x7f5c481994b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 >> 172.21.15.77:0/1988234229 conn(0x7f5c480fc7d0 msgr2=0x7f5c480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 shutdown_connections 2024-09-15T22:10:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:42.429+0000 7f5c4cac8640 1 -- 172.21.15.77:0/1988234229 wait complete. 2024-09-15T22:10:42.763 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:10:42.401+0000 7f2d632b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:42.763 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:10:42.401+0000 7f2d632b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:42.763 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:10:42.401+0000 7f2d632b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:42.763 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:10:42.401+0000 7f2d632b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:42.764 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:10:42.401+0000 7f2d632b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:43.014 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:42 smithi117 ceph-mon[35021]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-15T22:10:43.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:42 smithi077 ceph-mon[30980]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-15T22:10:43.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:42 smithi196 ceph-mon[35109]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-09-15T22:10:43.230 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-15T22:10:43.474 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:43.877 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:43 smithi077 ceph-mon[30980]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.961+0000 7f9a90559640 1 -- 172.21.15.77:0/2274110914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 msgr2=0x7f9a8810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.961+0000 7f9a90559640 1 --2- 172.21.15.77:0/2274110914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8810eb00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009930 tx=0x7f9a8002a440 comp rx=0 tx=0).stop 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.961+0000 7f9a90559640 1 -- 172.21.15.77:0/2274110914 shutdown_connections 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.961+0000 7f9a90559640 1 --2- 172.21.15.77:0/2274110914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8810eb00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 --2- 172.21.15.77:0/2274110914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 --2- 172.21.15.77:0/2274110914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 0x7f9a88100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- 172.21.15.77:0/2274110914 >> 172.21.15.77:0/2274110914 conn(0x7f9a88077210 msgr2=0x7f9a88077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- 172.21.15.77:0/2274110914 shutdown_connections 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- 172.21.15.77:0/2274110914 wait complete. 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 Processor -- start 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- start start 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 0x7f9a881984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a88112010 con 0x7f9a88108160 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a88111e90 con 0x7f9a88101290 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a90559640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a88112190 con 0x7f9a881008e0 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a8eaf6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.962+0000 7f9a8eaf6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50364/0 (socket says 172.21.15.77:50364) 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 -- 172.21.15.77:0/4065123781 learned_addr learned my addr 172.21.15.77:0/4065123781 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8daf4640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 -- 172.21.15.77:0/4065123781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 msgr2=0x7f9a881984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8e2f5640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 0x7f9a881984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 0x7f9a881984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 -- 172.21.15.77:0/4065123781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 msgr2=0x7f9a88198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 -- 172.21.15.77:0/4065123781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8819d4a0 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8daf4640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a8eaf6640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005ab0 tx=0x7f9a800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8003f030 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a80003ce0 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8819d730 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80035070 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.963+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a881a5070 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.964+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a88100cc0 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.965+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9a80042030 con 0x7f9a88108160 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.966+0000 7f9a877fe640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 0x7f9a700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.966+0000 7f9a8e2f5640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 0x7f9a700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.966+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f9a800f0550 con 0x7f9a88108160 2024-09-15T22:10:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.967+0000 7f9a8e2f5640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 0x7f9a700a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005c40 tx=0x7f9a78008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:43.969+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a800bd030 con 0x7f9a88108160 2024-09-15T22:10:44.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:43 smithi196 ceph-mon[35109]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:44.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:43 smithi117 ceph-mon[35021]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:44.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.160+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9a8810a1e0 con 0x7f9a700a32c0 2024-09-15T22:10:44.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.184+0000 7f9a877fe640 1 -- 172.21.15.77:0/4065123781 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a78008cc0 con 0x7f9a700a32c0 2024-09-15T22:10:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.186+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 msgr2=0x7f9a700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:44.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 0x7f9a700a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a78005c40 tx=0x7f9a78008230 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 msgr2=0x7f9a8819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005ab0 tx=0x7f9a800042f0 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 shutdown_connections 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9a700a32c0 0x7f9a700a5780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a88108160 0x7f9a8819cdc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a88101290 0x7f9a88198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 --2- 172.21.15.77:0/4065123781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a881008e0 0x7f9a881984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 >> 172.21.15.77:0/4065123781 conn(0x7f9a88077210 msgr2=0x7f9a88105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 shutdown_connections 2024-09-15T22:10:44.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:44.187+0000 7f9a90559640 1 -- 172.21.15.77:0/4065123781 wait complete. 2024-09-15T22:10:44.950 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:44 smithi077 ceph-mon[30980]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:44.950 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:44 smithi077 ceph-mon[30980]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-15T22:10:44.978 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-15T22:10:45.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:44 smithi196 ceph-mon[35109]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:45.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:44 smithi196 ceph-mon[35109]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-15T22:10:45.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:44 smithi117 ceph-mon[35021]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:45.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:44 smithi117 ceph-mon[35021]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-15T22:10:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 -- 172.21.15.77:0/2864019115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50103740 msgr2=0x7f4e50107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 --2- 172.21.15.77:0/2864019115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50103740 0x7f4e50107b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009cd0 tx=0x7f4e4402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 -- 172.21.15.77:0/2864019115 shutdown_connections 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 --2- 172.21.15.77:0/2864019115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50108290 0x7f4e5010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 --2- 172.21.15.77:0/2864019115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50103740 0x7f4e50107b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 --2- 172.21.15.77:0/2864019115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50102d90 0x7f4e50103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 -- 172.21.15.77:0/2864019115 >> 172.21.15.77:0/2864019115 conn(0x7f4e500fc7d0 msgr2=0x7f4e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 -- 172.21.15.77:0/2864019115 shutdown_connections 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 -- 172.21.15.77:0/2864019115 wait complete. 2024-09-15T22:10:45.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.703+0000 7f4e56089640 1 Processor -- start 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 -- start start 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50103740 0x7f4e50198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50108290 0x7f4e5019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112090 con 0x7f4e50102d90 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50111f10 con 0x7f4e50103740 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e56089640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50112210 con 0x7f4e50108290 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50388/0 (socket says 172.21.15.77:50388) 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 -- 172.21.15.77:0/1874907135 learned_addr learned my addr 172.21.15.77:0/1874907135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4effd640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50103740 0x7f4e50198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:45.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4ffff640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50108290 0x7f4e5019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 -- 172.21.15.77:0/1874907135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50108290 msgr2=0x7f4e5019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50108290 0x7f4e5019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 -- 172.21.15.77:0/1874907135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50103740 msgr2=0x7f4e50198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50103740 0x7f4e50198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 -- 172.21.15.77:0/1874907135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5019d580 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4f7fe640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000d770 tx=0x7f4e4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.704+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40017070 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.705+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e40005500 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.705+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5019d870 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.705+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004e00 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.705+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a5150 con 0x7f4e50102d90 2024-09-15T22:10:45.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.706+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50103170 con 0x7f4e50102d90 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.707+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4e40004fa0 con 0x7f4e50102d90 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.707+0000 7f4e4cff9640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 0x7f4e380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.707+0000 7f4e4effd640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 0x7f4e380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.707+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f4e400cf600 con 0x7f4e50102d90 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.708+0000 7f4e4effd640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 0x7f4e380a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009ca0 tx=0x7f4e44009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.712+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e4009b930 con 0x7f4e50102d90 2024-09-15T22:10:45.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.905+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4e50062630 con 0x7f4e380a3350 2024-09-15T22:10:45.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:45 smithi077 ceph-mon[30980]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:10:45.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.910+0000 7f4e4cff9640 1 -- 172.21.15.77:0/1874907135 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4e50062630 con 0x7f4e380a3350 2024-09-15T22:10:45.911 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_0/75106a1d-c7fe-4bd2-bd2b-f609d7ddde40 2024-09-15T22:10:45.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 msgr2=0x7f4e380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:45.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 0x7f4e380a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009ca0 tx=0x7f4e44009360 comp rx=0 tx=0).stop 2024-09-15T22:10:45.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 msgr2=0x7f4e501985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000d770 tx=0x7f4e4000dc30 comp rx=0 tx=0).stop 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 shutdown_connections 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4e380a3350 0x7f4e380a5810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e50108290 0x7f4e5019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e50103740 0x7f4e50198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 --2- 172.21.15.77:0/1874907135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e50102d90 0x7f4e501985d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 >> 172.21.15.77:0/1874907135 conn(0x7f4e500fc7d0 msgr2=0x7f4e50105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 shutdown_connections 2024-09-15T22:10:45.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:45.912+0000 7f4e56089640 1 -- 172.21.15.77:0/1874907135 wait complete. 2024-09-15T22:10:46.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:45 smithi196 ceph-mon[35109]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:10:46.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:45 smithi117 ceph-mon[35021]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:10:46.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-15T22:10:46.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:46 smithi077 ceph-mon[30980]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:46 smithi077 ceph-mon[30980]: pgmap v309: 129 pgs: 129 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-15T22:10:46.850 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:47.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:46 smithi196 ceph-mon[35109]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:47.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:46 smithi196 ceph-mon[35109]: pgmap v309: 129 pgs: 129 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-15T22:10:47.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:46 smithi117 ceph-mon[35021]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:47.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:46 smithi117 ceph-mon[35021]: pgmap v309: 129 pgs: 129 active+clean; 607 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 -- 172.21.15.77:0/3357500052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c101420 msgr2=0x7f8e9c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/3357500052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c101420 0x7f8e9c10d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009cd0 tx=0x7f8e9402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 -- 172.21.15.77:0/3357500052 shutdown_connections 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/3357500052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c101420 0x7f8e9c10d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/3357500052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c100a80 0x7f8e9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/3357500052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 0x7f8e9c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 -- 172.21.15.77:0/3357500052 >> 172.21.15.77:0/3357500052 conn(0x7f8e9c0fc7a0 msgr2=0x7f8e9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 -- 172.21.15.77:0/3357500052 shutdown_connections 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.345+0000 7f8ea15ee640 1 -- 172.21.15.77:0/3357500052 wait complete. 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 Processor -- start 2024-09-15T22:10:47.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 -- start start 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101420 0x7f8e9c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 0x7f8e9c1a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c110f20 con 0x7f8e9c100a80 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c110da0 con 0x7f8e9c106a80 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8ea15ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1110a0 con 0x7f8e9c101420 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50410/0 (socket says 172.21.15.77:50410) 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 -- 172.21.15.77:0/2543019114 learned_addr learned my addr 172.21.15.77:0/2543019114 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e93fff640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101420 0x7f8e9c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9b7fe640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 0x7f8e9c1a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 -- 172.21.15.77:0/2543019114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101420 msgr2=0x7f8e9c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101420 0x7f8e9c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 -- 172.21.15.77:0/2543019114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 msgr2=0x7f8e9c1a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 0x7f8e9c1a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 -- 172.21.15.77:0/2543019114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1a4990 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.346+0000 7f8e9affd640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00b4a0 tx=0x7f8e8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.347+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c014070 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.347+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8c010af0 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.347+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c1a4b60 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.347+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c017d10 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.347+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a50f0 con 0x7f8e9c100a80 2024-09-15T22:10:47.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.348+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c100f60 con 0x7f8e9c100a80 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.349+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8e8c017070 con 0x7f8e9c100a80 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.349+0000 7f8e98ff9640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 0x7f8e880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.349+0000 7f8e93fff640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 0x7f8e880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.349+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f8e8c0cef70 con 0x7f8e9c100a80 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.350+0000 7f8e93fff640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 0x7f8e880a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c06b320 tx=0x7f8e84008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:47.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.354+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e8c09ba00 con 0x7f8e9c100a80 2024-09-15T22:10:47.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.544+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8e9c062630 con 0x7f8e880a3350 2024-09-15T22:10:47.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.569+0000 7f8e98ff9640 1 -- 172.21.15.77:0/2543019114 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8e9c062630 con 0x7f8e880a3350 2024-09-15T22:10:47.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 msgr2=0x7f8e880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 0x7f8e880a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c06b320 tx=0x7f8e84008040 comp rx=0 tx=0).stop 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 msgr2=0x7f8e9c1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00b4a0 tx=0x7f8e8c00b960 comp rx=0 tx=0).stop 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 shutdown_connections 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e880a3350 0x7f8e880a5810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e9c106a80 0x7f8e9c1a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e9c101420 0x7f8e9c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 --2- 172.21.15.77:0/2543019114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e9c100a80 0x7f8e9c1993d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:47.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 >> 172.21.15.77:0/2543019114 conn(0x7f8e9c0fc7a0 msgr2=0x7f8e9c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:47.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 shutdown_connections 2024-09-15T22:10:47.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:47.571+0000 7f8ea15ee640 1 -- 172.21.15.77:0/2543019114 wait complete. 2024-09-15T22:10:48.303 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-15T22:10:48.545 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.047+0000 7f8ebd98f640 1 -- 172.21.15.77:0/2827354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8102790 msgr2=0x7f8eb81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.047+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/2827354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8102790 0x7f8eb81067e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009ca0 tx=0x7f8ea802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 -- 172.21.15.77:0/2827354681 shutdown_connections 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/2827354681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8106f10 0x7f8eb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/2827354681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8102790 0x7f8eb81067e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/2827354681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 0x7f8eb81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 -- 172.21.15.77:0/2827354681 >> 172.21.15.77:0/2827354681 conn(0x7f8eb80fd8c0 msgr2=0x7f8eb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 -- 172.21.15.77:0/2827354681 shutdown_connections 2024-09-15T22:10:49.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 -- 172.21.15.77:0/2827354681 wait complete. 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 Processor -- start 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.048+0000 7f8ebd98f640 1 -- start start 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 0x7f8eb8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 0x7f8eb81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111f70 con 0x7f8eb8106f10 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111df0 con 0x7f8eb8102790 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81120f0 con 0x7f8eb8101de0 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41552/0 (socket says 172.21.15.77:41552) 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 -- 172.21.15.77:0/1303426447 learned_addr learned my addr 172.21.15.77:0/1303426447 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb6ffd640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 0x7f8eb8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb67fc640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 0x7f8eb81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 -- 172.21.15.77:0/1303426447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 msgr2=0x7f8eb8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 0x7f8eb8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 -- 172.21.15.77:0/1303426447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 msgr2=0x7f8eb81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 0x7f8eb81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 -- 172.21.15.77:0/1303426447 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb819d440 con 0x7f8eb8106f10 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb67fc640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 0x7f8eb81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8eb77fe640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c970 tx=0x7f8eb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0007bb0 con 0x7f8eb8106f10 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb0007d50 con 0x7f8eb8106f10 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb819d730 con 0x7f8eb8106f10 2024-09-15T22:10:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000f450 con 0x7f8eb8106f10 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.049+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a5010 con 0x7f8eb8106f10 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.050+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb81021c0 con 0x7f8eb8106f10 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.051+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8eb0004f60 con 0x7f8eb8106f10 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.052+0000 7f8ebc98d640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 0x7f8ea00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.052+0000 7f8eb6ffd640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 0x7f8ea00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.052+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f8eb00cf990 con 0x7f8eb8106f10 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.052+0000 7f8eb6ffd640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 0x7f8ea00a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00bbe0 tx=0x7f8eac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:49.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.056+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eb0014030 con 0x7f8eb8106f10 2024-09-15T22:10:49.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.253+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8eb8062630 con 0x7f8ea00a32d0 2024-09-15T22:10:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.258+0000 7f8ebc98d640 1 -- 172.21.15.77:0/1303426447 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8eb8062630 con 0x7f8ea00a32d0 2024-09-15T22:10:49.260 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_1/c85cddab-bddb-43c6-8f43-ac1d628fc105 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 msgr2=0x7f8ea00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 0x7f8ea00a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00bbe0 tx=0x7f8eac00b390 comp rx=0 tx=0).stop 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 msgr2=0x7f8eb819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000c970 tx=0x7f8eb000ce30 comp rx=0 tx=0).stop 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 shutdown_connections 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ea00a32d0 0x7f8ea00a5790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8eb8106f10 0x7f8eb819cd60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8102790 0x7f8eb81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 --2- 172.21.15.77:0/1303426447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb8101de0 0x7f8eb8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:49.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.260+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 >> 172.21.15.77:0/1303426447 conn(0x7f8eb80fd8c0 msgr2=0x7f8eb80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:49.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.261+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 shutdown_connections 2024-09-15T22:10:49.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:49.261+0000 7f8ebd98f640 1 -- 172.21.15.77:0/1303426447 wait complete. 2024-09-15T22:10:49.567 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:49 smithi077 ceph-mon[30980]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:49.567 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:49 smithi077 ceph-mon[30980]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-15T22:10:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:49 smithi196 ceph-mon[35109]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:49 smithi196 ceph-mon[35109]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-15T22:10:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:49 smithi117 ceph-mon[35021]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:49 smithi117 ceph-mon[35021]: pgmap v310: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-15T22:10:49.923 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:10:50.167 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:50.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:50 smithi077 ceph-mon[30980]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:50 smithi196 ceph-mon[35109]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:50 smithi117 ceph-mon[35021]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:10:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 -- 172.21.15.77:0/2369385862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4618103660 msgr2=0x7f461810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 --2- 172.21.15.77:0/2369385862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4618103660 0x7f461810bb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f460c009900 tx=0x7f460c02a480 comp rx=0 tx=0).stop 2024-09-15T22:10:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 -- 172.21.15.77:0/2369385862 shutdown_connections 2024-09-15T22:10:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 --2- 172.21.15.77:0/2369385862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f461810c170 0x7f461810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 --2- 172.21.15.77:0/2369385862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4618103660 0x7f461810bb50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 --2- 172.21.15.77:0/2369385862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618102d40 0x7f4618103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 -- 172.21.15.77:0/2369385862 >> 172.21.15.77:0/2369385862 conn(0x7f46180fc7a0 msgr2=0x7f46180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 -- 172.21.15.77:0/2369385862 shutdown_connections 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.641+0000 7f461e127640 1 -- 172.21.15.77:0/2369385862 wait complete. 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 Processor -- start 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 -- start start 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 0x7f4618198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 0x7f4618198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111fb0 con 0x7f461810c170 2024-09-15T22:10:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111e30 con 0x7f4618102d40 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.642+0000 7f461e127640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4618112130 con 0x7f4618103660 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41564/0 (socket says 172.21.15.77:41564) 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 -- 172.21.15.77:0/2849252760 learned_addr learned my addr 172.21.15.77:0/2849252760 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4616ffd640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 0x7f4618198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f46177fe640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 0x7f4618198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 -- 172.21.15.77:0/2849252760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 msgr2=0x7f4618198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 0x7f4618198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 -- 172.21.15.77:0/2849252760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 msgr2=0x7f4618198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 0x7f4618198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 -- 172.21.15.77:0/2849252760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461819d520 con 0x7f461810c170 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f46177fe640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 0x7f4618198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4616ffd640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 0x7f4618198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4617fff640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f461000cad0 tx=0x7f461000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000ae20 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4610005020 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461819d810 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000f6f0 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.643+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a50f0 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.644+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4618103120 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.646+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f46100051d0 con 0x7f461810c170 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.647+0000 7f4614ff9640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 0x7f46000a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.647+0000 7f46177fe640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 0x7f46000a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:50.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.647+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f46100cfe00 con 0x7f461810c170 2024-09-15T22:10:50.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.647+0000 7f46177fe640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 0x7f46000a5890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:50.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.651+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4610014030 con 0x7f461810c170 2024-09-15T22:10:50.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.853+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4618106c50 con 0x7f461810c170 2024-09-15T22:10:50.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.853+0000 7f4614ff9640 1 -- 172.21.15.77:0/2849252760 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f461009c720 con 0x7f461810c170 2024-09-15T22:10:50.855 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:50.855 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:10:50.855 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 msgr2=0x7f46000a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 0x7f46000a5890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608008040 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 msgr2=0x7f461819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f461000cad0 tx=0x7f461000cf90 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 shutdown_connections 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f46000a33d0 0x7f46000a5890 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f461810c170 0x7f461819ce40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4618103660 0x7f4618198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 --2- 172.21.15.77:0/2849252760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4618102d40 0x7f4618198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:50.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.855+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 >> 172.21.15.77:0/2849252760 conn(0x7f46180fc7a0 msgr2=0x7f46180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:50.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.856+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 shutdown_connections 2024-09-15T22:10:50.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:50.856+0000 7f461e127640 1 -- 172.21.15.77:0/2849252760 wait complete. 2024-09-15T22:10:51.519 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:51 smithi077 ceph-mon[30980]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-09-15T22:10:51.519 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:51 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2849252760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:51.520 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:51 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:51.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:51 smithi196 ceph-mon[35109]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-09-15T22:10:51.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:51 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2849252760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:51.541 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:51 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:51.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:10:51.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:51 smithi117 ceph-mon[35021]: pgmap v311: 129 pgs: 129 active+clean; 626 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 24 op/s 2024-09-15T22:10:51.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:51 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2849252760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:51.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:51 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:52.480 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:53.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/53727711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 msgr2=0x7f2ca0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/53727711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca0106e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009930 tx=0x7f2c9802a420 comp rx=0 tx=0).stop 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/53727711 shutdown_connections 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/53727711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 0x7f2ca010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/53727711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 0x7f2ca0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/53727711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca0106e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/53727711 >> 172.21.15.77:0/53727711 conn(0x7f2ca00fc7a0 msgr2=0x7f2ca00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/53727711 shutdown_connections 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.996+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/53727711 wait complete. 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 Processor -- start 2024-09-15T22:10:53.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 -- start start 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 0x7f2ca0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 0x7f2ca01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0110e80 con 0x7f2ca0106a80 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0110d00 con 0x7f2ca0100a80 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca6cbb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0111000 con 0x7f2ca0101420 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41584/0 (socket says 172.21.15.77:41584) 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 -- 172.21.15.77:0/3490415371 learned_addr learned my addr 172.21.15.77:0/3490415371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2c9ffff640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 0x7f2ca01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca4a57640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 0x7f2ca0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 -- 172.21.15.77:0/3490415371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 msgr2=0x7f2ca01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:53.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 0x7f2ca01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 -- 172.21.15.77:0/3490415371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 msgr2=0x7f2ca0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 0x7f2ca0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.997+0000 7f2ca5258640 1 -- 172.21.15.77:0/3490415371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca019d340 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2ca5258640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800d750 tx=0x7f2c8800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88017070 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c88005500 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca019d630 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88004d90 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.998+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01a4f10 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:52.999+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca0100ee0 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.000+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2c88004f30 con 0x7f2ca0106a80 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.000+0000 7f2c9dffb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 0x7f2c8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:53.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.001+0000 7f2ca4a57640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 0x7f2c8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:53.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.001+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f2c880cf590 con 0x7f2ca0106a80 2024-09-15T22:10:53.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.001+0000 7f2ca4a57640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 0x7f2c8c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:53.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.004+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c8809b860 con 0x7f2ca0106a80 2024-09-15T22:10:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.190+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ca010a060 con 0x7f2ca0106a80 2024-09-15T22:10:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.193+0000 7f2c9dffb640 1 -- 172.21.15.77:0/3490415371 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f2c8809fb60 con 0x7f2ca0106a80 2024-09-15T22:10:53.194 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:53.195 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 msgr2=0x7f2c8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 0x7f2c8c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c980058d0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 msgr2=0x7f2ca019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800d750 tx=0x7f2c8800dc10 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 shutdown_connections 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c8c0a32d0 0x7f2c8c0a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ca0106a80 0x7f2ca019cc60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ca0101420 0x7f2ca01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 --2- 172.21.15.77:0/3490415371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ca0100a80 0x7f2ca0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 >> 172.21.15.77:0/3490415371 conn(0x7f2ca00fc7a0 msgr2=0x7f2ca006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.195+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 shutdown_connections 2024-09-15T22:10:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:53.196+0000 7f2ca6cbb640 1 -- 172.21.15.77:0/3490415371 wait complete. 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 22 op/s 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:53 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3490415371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 22 op/s 2024-09-15T22:10:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:53 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3490415371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: pgmap v312: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 22 op/s 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:53 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3490415371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:10:53.970 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:10:53.970 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:10:54.230 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:54.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:54.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:54.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:54 smithi077 ceph-mon[30980]: pgmap v313: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/2231972223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0100b30 msgr2=0x7fc0c0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/2231972223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0100b30 0x7fc0c0100f10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009900 tx=0x7fc0bc02a470 comp rx=0 tx=0).stop 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/2231972223 shutdown_connections 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/2231972223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/2231972223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0101450 0x7fc0c0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/2231972223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0100b30 0x7fc0c0100f10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/2231972223 >> 172.21.15.77:0/2231972223 conn(0x7fc0c00fc7a0 msgr2=0x7fc0c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/2231972223 shutdown_connections 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.733+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/2231972223 wait complete. 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 Processor -- start 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 -- start start 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0100b30 0x7fc0c01a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c01185d0 con 0x7fc0c0101450 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0118450 con 0x7fc0c0100b30 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c7dcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0118750 con 0x7fc0c0105f40 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c536a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c536a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41610/0 (socket says 172.21.15.77:41610) 2024-09-15T22:10:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.734+0000 7fc0c536a640 1 -- 172.21.15.77:0/384547563 learned_addr learned my addr 172.21.15.77:0/384547563 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c636c640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 -- 172.21.15.77:0/384547563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 msgr2=0x7fc0c019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c5b6b640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0100b30 0x7fc0c01a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 -- 172.21.15.77:0/384547563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0100b30 msgr2=0x7fc0c01a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0100b30 0x7fc0c01a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 -- 172.21.15.77:0/384547563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c019f070 con 0x7fc0c0101450 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c636c640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c536a640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000c940 tx=0x7fc0b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004220 con 0x7fc0c0101450 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b00043c0 con 0x7fc0c0101450 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c019f360 con 0x7fc0c0101450 2024-09-15T22:10:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004a00 con 0x7fc0c0101450 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.735+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01ab650 con 0x7fc0c0101450 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.736+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0105920 con 0x7fc0c0101450 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.737+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc0b00079b0 con 0x7fc0c0101450 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.738+0000 7fc0b6ffd640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 0x7fc0ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.738+0000 7fc0c5b6b640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 0x7fc0ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.738+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc0b00cfd90 con 0x7fc0c0101450 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.738+0000 7fc0c5b6b640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 0x7fc0ac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0096c0 tx=0x7fc0bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.742+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0b00d3030 con 0x7fc0c0101450 2024-09-15T22:10:54.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:54 smithi196 ceph-mon[35109]: pgmap v313: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:10:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:54 smithi117 ceph-mon[35021]: pgmap v313: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-15T22:10:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.905+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0c010fe10 con 0x7fc0ac0a32d0 2024-09-15T22:10:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.915+0000 7fc0b6ffd640 1 -- 172.21.15.77:0/384547563 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340660 (secure 0 0 0) 0x7fc0c010fe10 con 0x7fc0ac0a32d0 2024-09-15T22:10:54.920 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:10:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 msgr2=0x7fc0ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 0x7fc0ac0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0096c0 tx=0x7fc0bc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 msgr2=0x7fc0c01a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000c940 tx=0x7fc0b000ce00 comp rx=0 tx=0).stop 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 shutdown_connections 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fc0ac0a32d0 0x7fc0ac0a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c0105f40 0x7fc0c019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c0101450 0x7fc0c01a4cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 --2- 172.21.15.77:0/384547563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c0100b30 0x7fc0c01a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 >> 172.21.15.77:0/384547563 conn(0x7fc0c00fc7a0 msgr2=0x7fc0c010f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 shutdown_connections 2024-09-15T22:10:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:54.920+0000 7fc0c7dcf640 1 -- 172.21.15.77:0/384547563 wait complete. 2024-09-15T22:10:55.658 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-15T22:10:54.195395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641548,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":302,"num_read_kb":259,"num_write":422,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373608,"kb_used_data":11880,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568216,"statfs":{"total":1151940427776,"available":1151557853184,"internally_reserved":0,"allocated":12165120,"data_stored":7743284,"data_compressed":317160,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41614,"num_objects":13,"num_object_clones":0,"num_object_copies":41,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":122,"num_read_kb":105,"num_write":58,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003503"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.763273+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:10:51.763273+0000","last_peered":"2024-09-15T22:10:51.763273+0000","last_clean":"2024-09-15T22:10:51.763273+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:10:51.763273+0000","last_undegraded":"2024-09-15T22:10:51.763273+0000","last_fullsized":"2024-09-15T22:10:51.763273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502511+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:10:06.502511+0000","last_peered":"2024-09-15T22:10:06.502511+0000","last_clean":"2024-09-15T22:10:06.502511+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:10:06.502511+0000","last_undegraded":"2024-09-15T22:10:06.502511+0000","last_fullsized":"2024-09-15T22:10:06.502511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.250909+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:10:06.250909+0000","last_peered":"2024-09-15T22:10:06.250909+0000","last_clean":"2024-09-15T22:10:06.250909+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:10:06.250909+0000","last_undegraded":"2024-09-15T22:10:06.250909+0000","last_fullsized":"2024-09-15T22:10:06.250909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502690+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:10:06.502690+0000","last_peered":"2024-09-15T22:10:06.502690+0000","last_clean":"2024-09-15T22:10:06.502690+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:10:06.502690+0000","last_undegraded":"2024-09-15T22:10:06.502690+0000","last_fullsized":"2024-09-15T22:10:06.502690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252495+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:10:06.252495+0000","last_peered":"2024-09-15T22:10:06.252495+0000","last_clean":"2024-09-15T22:10:06.252495+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:10:06.252495+0000","last_undegraded":"2024-09-15T22:10:06.252495+0000","last_fullsized":"2024-09-15T22:10:06.252495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251317+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:10:06.251317+0000","last_peered":"2024-09-15T22:10:06.251317+0000","last_clean":"2024-09-15T22:10:06.251317+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:10:06.251317+0000","last_undegraded":"2024-09-15T22:10:06.251317+0000","last_fullsized":"2024-09-15T22:10:06.251317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692565+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:10:38.692565+0000","last_peered":"2024-09-15T22:10:38.692565+0000","last_clean":"2024-09-15T22:10:38.692565+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:10:38.692565+0000","last_undegraded":"2024-09-15T22:10:38.692565+0000","last_fullsized":"2024-09-15T22:10:38.692565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251855+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:10:06.251855+0000","last_peered":"2024-09-15T22:10:06.251855+0000","last_clean":"2024-09-15T22:10:06.251855+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:10:06.251855+0000","last_undegraded":"2024-09-15T22:10:06.251855+0000","last_fullsized":"2024-09-15T22:10:06.251855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252466+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:10:06.252466+0000","last_peered":"2024-09-15T22:10:06.252466+0000","last_clean":"2024-09-15T22:10:06.252466+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:10:06.252466+0000","last_undegraded":"2024-09-15T22:10:06.252466+0000","last_fullsized":"2024-09-15T22:10:06.252466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692316+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:10:38.692316+0000","last_peered":"2024-09-15T22:10:38.692316+0000","last_clean":"2024-09-15T22:10:38.692316+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:10:38.692316+0000","last_undegraded":"2024-09-15T22:10:38.692316+0000","last_fullsized":"2024-09-15T22:10:38.692316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252563+0000","last_change":"2024-09-15T22:09:55.219247+0000","last_active":"2024-09-15T22:10:06.252563+0000","last_peered":"2024-09-15T22:10:06.252563+0000","last_clean":"2024-09-15T22:10:06.252563+0000","last_became_active":"2024-09-15T22:09:55.218925+0000","last_became_peered":"2024-09-15T22:09:55.218925+0000","last_unstale":"2024-09-15T22:10:06.252563+0000","last_undegraded":"2024-09-15T22:10:06.252563+0000","last_fullsized":"2024-09-15T22:10:06.252563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:29.567713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252098+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:10:06.252098+0000","last_peered":"2024-09-15T22:10:06.252098+0000","last_clean":"2024-09-15T22:10:06.252098+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:10:06.252098+0000","last_undegraded":"2024-09-15T22:10:06.252098+0000","last_fullsized":"2024-09-15T22:10:06.252098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084413+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:10:06.084413+0000","last_peered":"2024-09-15T22:10:06.084413+0000","last_clean":"2024-09-15T22:10:06.084413+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:10:06.084413+0000","last_undegraded":"2024-09-15T22:10:06.084413+0000","last_fullsized":"2024-09-15T22:10:06.084413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:36.334153+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:10:36.334153+0000","last_peered":"2024-09-15T22:10:36.334153+0000","last_clean":"2024-09-15T22:10:36.334153+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:10:36.334153+0000","last_undegraded":"2024-09-15T22:10:36.334153+0000","last_fullsized":"2024-09-15T22:10:36.334153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251772+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:10:06.251772+0000","last_peered":"2024-09-15T22:10:06.251772+0000","last_clean":"2024-09-15T22:10:06.251772+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:10:06.251772+0000","last_undegraded":"2024-09-15T22:10:06.251772+0000","last_fullsized":"2024-09-15T22:10:06.251772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502307+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:10:06.502307+0000","last_peered":"2024-09-15T22:10:06.502307+0000","last_clean":"2024-09-15T22:10:06.502307+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:10:06.502307+0000","last_undegraded":"2024-09-15T22:10:06.502307+0000","last_fullsized":"2024-09-15T22:10:06.502307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.344593+0000","last_change":"2024-09-15T22:09:53.191315+0000","last_active":"2024-09-15T22:10:38.344593+0000","last_peered":"2024-09-15T22:10:38.344593+0000","last_clean":"2024-09-15T22:10:38.344593+0000","last_became_active":"2024-09-15T22:09:53.190950+0000","last_became_peered":"2024-09-15T22:09:53.190950+0000","last_unstale":"2024-09-15T22:10:38.344593+0000","last_undegraded":"2024-09-15T22:10:38.344593+0000","last_fullsized":"2024-09-15T22:10:38.344593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:05.113961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251645+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:10:06.251645+0000","last_peered":"2024-09-15T22:10:06.251645+0000","last_clean":"2024-09-15T22:10:06.251645+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:10:06.251645+0000","last_undegraded":"2024-09-15T22:10:06.251645+0000","last_fullsized":"2024-09-15T22:10:06.251645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237179+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:10:05.237179+0000","last_peered":"2024-09-15T22:10:05.237179+0000","last_clean":"2024-09-15T22:10:05.237179+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:10:05.237179+0000","last_undegraded":"2024-09-15T22:10:05.237179+0000","last_fullsized":"2024-09-15T22:10:05.237179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.689533+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:10:38.689533+0000","last_peered":"2024-09-15T22:10:38.689533+0000","last_clean":"2024-09-15T22:10:38.689533+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:10:38.689533+0000","last_undegraded":"2024-09-15T22:10:38.689533+0000","last_fullsized":"2024-09-15T22:10:38.689533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252674+0000","last_change":"2024-09-15T22:10:02.237508+0000","last_active":"2024-09-15T22:10:06.252674+0000","last_peered":"2024-09-15T22:10:06.252674+0000","last_clean":"2024-09-15T22:10:06.252674+0000","last_became_active":"2024-09-15T22:10:02.237111+0000","last_became_peered":"2024-09-15T22:10:02.237111+0000","last_unstale":"2024-09-15T22:10:06.252674+0000","last_undegraded":"2024-09-15T22:10:06.252674+0000","last_fullsized":"2024-09-15T22:10:06.252674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:52.924409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237294+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:10:05.237294+0000","last_peered":"2024-09-15T22:10:05.237294+0000","last_clean":"2024-09-15T22:10:05.237294+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:10:05.237294+0000","last_undegraded":"2024-09-15T22:10:05.237294+0000","last_fullsized":"2024-09-15T22:10:05.237294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:36.333281+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:10:36.333281+0000","last_peered":"2024-09-15T22:10:36.333281+0000","last_clean":"2024-09-15T22:10:36.333281+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:10:36.333281+0000","last_undegraded":"2024-09-15T22:10:36.333281+0000","last_fullsized":"2024-09-15T22:10:36.333281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252043+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:10:06.252043+0000","last_peered":"2024-09-15T22:10:06.252043+0000","last_clean":"2024-09-15T22:10:06.252043+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:10:06.252043+0000","last_undegraded":"2024-09-15T22:10:06.252043+0000","last_fullsized":"2024-09-15T22:10:06.252043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252211+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:10:06.252211+0000","last_peered":"2024-09-15T22:10:06.252211+0000","last_clean":"2024-09-15T22:10:06.252211+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:10:06.252211+0000","last_undegraded":"2024-09-15T22:10:06.252211+0000","last_fullsized":"2024-09-15T22:10:06.252211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690893+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:10:38.690893+0000","last_peered":"2024-09-15T22:10:38.690893+0000","last_clean":"2024-09-15T22:10:38.690893+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:10:38.690893+0000","last_undegraded":"2024-09-15T22:10:38.690893+0000","last_fullsized":"2024-09-15T22:10:38.690893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252162+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:10:06.252162+0000","last_peered":"2024-09-15T22:10:06.252162+0000","last_clean":"2024-09-15T22:10:06.252162+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:10:06.252162+0000","last_undegraded":"2024-09-15T22:10:06.252162+0000","last_fullsized":"2024-09-15T22:10:06.252162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502648+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:10:06.502648+0000","last_peered":"2024-09-15T22:10:06.502648+0000","last_clean":"2024-09-15T22:10:06.502648+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:10:06.502648+0000","last_undegraded":"2024-09-15T22:10:06.502648+0000","last_fullsized":"2024-09-15T22:10:06.502648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.210969+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:10:51.210969+0000","last_peered":"2024-09-15T22:10:51.210969+0000","last_clean":"2024-09-15T22:10:51.210969+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:10:51.210969+0000","last_undegraded":"2024-09-15T22:10:51.210969+0000","last_fullsized":"2024-09-15T22:10:51.210969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502039+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:10:06.502039+0000","last_peered":"2024-09-15T22:10:06.502039+0000","last_clean":"2024-09-15T22:10:06.502039+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:10:06.502039+0000","last_undegraded":"2024-09-15T22:10:06.502039+0000","last_fullsized":"2024-09-15T22:10:06.502039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251913+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:10:06.251913+0000","last_peered":"2024-09-15T22:10:06.251913+0000","last_clean":"2024-09-15T22:10:06.251913+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:10:06.251913+0000","last_undegraded":"2024-09-15T22:10:06.251913+0000","last_fullsized":"2024-09-15T22:10:06.251913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.683550+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:10:38.683550+0000","last_peered":"2024-09-15T22:10:38.683550+0000","last_clean":"2024-09-15T22:10:38.683550+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:10:38.683550+0000","last_undegraded":"2024-09-15T22:10:38.683550+0000","last_fullsized":"2024-09-15T22:10:38.683550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237477+0000","last_change":"2024-09-15T22:10:02.240073+0000","last_active":"2024-09-15T22:10:05.237477+0000","last_peered":"2024-09-15T22:10:05.237477+0000","last_clean":"2024-09-15T22:10:05.237477+0000","last_became_active":"2024-09-15T22:10:02.239655+0000","last_became_peered":"2024-09-15T22:10:02.239655+0000","last_unstale":"2024-09-15T22:10:05.237477+0000","last_undegraded":"2024-09-15T22:10:05.237477+0000","last_fullsized":"2024-09-15T22:10:05.237477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:59.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238063+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:10:05.238063+0000","last_peered":"2024-09-15T22:10:05.238063+0000","last_clean":"2024-09-15T22:10:05.238063+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:10:05.238063+0000","last_undegraded":"2024-09-15T22:10:05.238063+0000","last_fullsized":"2024-09-15T22:10:05.238063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252461+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:10:06.252461+0000","last_peered":"2024-09-15T22:10:06.252461+0000","last_clean":"2024-09-15T22:10:06.252461+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:10:06.252461+0000","last_undegraded":"2024-09-15T22:10:06.252461+0000","last_fullsized":"2024-09-15T22:10:06.252461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237761+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:10:05.237761+0000","last_peered":"2024-09-15T22:10:05.237761+0000","last_clean":"2024-09-15T22:10:05.237761+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:10:05.237761+0000","last_undegraded":"2024-09-15T22:10:05.237761+0000","last_fullsized":"2024-09-15T22:10:05.237761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237989+0000","last_change":"2024-09-15T22:09:55.203424+0000","last_active":"2024-09-15T22:10:05.237989+0000","last_peered":"2024-09-15T22:10:05.237989+0000","last_clean":"2024-09-15T22:10:05.237989+0000","last_became_active":"2024-09-15T22:09:55.203120+0000","last_became_peered":"2024-09-15T22:09:55.203120+0000","last_unstale":"2024-09-15T22:10:05.237989+0000","last_undegraded":"2024-09-15T22:10:05.237989+0000","last_fullsized":"2024-09-15T22:10:05.237989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:44.090660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.762694+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:10:50.762694+0000","last_peered":"2024-09-15T22:10:50.762694+0000","last_clean":"2024-09-15T22:10:50.762694+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:10:50.762694+0000","last_undegraded":"2024-09-15T22:10:50.762694+0000","last_fullsized":"2024-09-15T22:10:50.762694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251839+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:10:06.251839+0000","last_peered":"2024-09-15T22:10:06.251839+0000","last_clean":"2024-09-15T22:10:06.251839+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:10:06.251839+0000","last_undegraded":"2024-09-15T22:10:06.251839+0000","last_fullsized":"2024-09-15T22:10:06.251839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251488+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:10:06.251488+0000","last_peered":"2024-09-15T22:10:06.251488+0000","last_clean":"2024-09-15T22:10:06.251488+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:10:06.251488+0000","last_undegraded":"2024-09-15T22:10:06.251488+0000","last_fullsized":"2024-09-15T22:10:06.251488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.708671+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:10:51.708671+0000","last_peered":"2024-09-15T22:10:51.708671+0000","last_clean":"2024-09-15T22:10:51.708671+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:10:51.708671+0000","last_undegraded":"2024-09-15T22:10:51.708671+0000","last_fullsized":"2024-09-15T22:10:51.708671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251513+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:10:06.251513+0000","last_peered":"2024-09-15T22:10:06.251513+0000","last_clean":"2024-09-15T22:10:06.251513+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:10:06.251513+0000","last_undegraded":"2024-09-15T22:10:06.251513+0000","last_fullsized":"2024-09-15T22:10:06.251513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251032+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:10:06.251032+0000","last_peered":"2024-09-15T22:10:06.251032+0000","last_clean":"2024-09-15T22:10:06.251032+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:10:06.251032+0000","last_undegraded":"2024-09-15T22:10:06.251032+0000","last_fullsized":"2024-09-15T22:10:06.251032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:37.337466+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:10:37.337466+0000","last_peered":"2024-09-15T22:10:37.337466+0000","last_clean":"2024-09-15T22:10:37.337466+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:10:37.337466+0000","last_undegraded":"2024-09-15T22:10:37.337466+0000","last_fullsized":"2024-09-15T22:10:37.337466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252475+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:10:06.252475+0000","last_peered":"2024-09-15T22:10:06.252475+0000","last_clean":"2024-09-15T22:10:06.252475+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:10:06.252475+0000","last_undegraded":"2024-09-15T22:10:06.252475+0000","last_fullsized":"2024-09-15T22:10:06.252475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502232+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:10:06.502232+0000","last_peered":"2024-09-15T22:10:06.502232+0000","last_clean":"2024-09-15T22:10:06.502232+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:10:06.502232+0000","last_undegraded":"2024-09-15T22:10:06.502232+0000","last_fullsized":"2024-09-15T22:10:06.502232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690275+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:10:38.690275+0000","last_peered":"2024-09-15T22:10:38.690275+0000","last_clean":"2024-09-15T22:10:38.690275+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:10:38.690275+0000","last_undegraded":"2024-09-15T22:10:38.690275+0000","last_fullsized":"2024-09-15T22:10:38.690275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084238+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:10:06.084238+0000","last_peered":"2024-09-15T22:10:06.084238+0000","last_clean":"2024-09-15T22:10:06.084238+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:10:06.084238+0000","last_undegraded":"2024-09-15T22:10:06.084238+0000","last_fullsized":"2024-09-15T22:10:06.084238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084569+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:10:06.084569+0000","last_peered":"2024-09-15T22:10:06.084569+0000","last_clean":"2024-09-15T22:10:06.084569+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:10:06.084569+0000","last_undegraded":"2024-09-15T22:10:06.084569+0000","last_fullsized":"2024-09-15T22:10:06.084569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084673+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:10:06.084673+0000","last_peered":"2024-09-15T22:10:06.084673+0000","last_clean":"2024-09-15T22:10:06.084673+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:10:06.084673+0000","last_undegraded":"2024-09-15T22:10:06.084673+0000","last_fullsized":"2024-09-15T22:10:06.084673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:49.707476+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:10:49.707476+0000","last_peered":"2024-09-15T22:10:49.707476+0000","last_clean":"2024-09-15T22:10:49.707476+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:10:49.707476+0000","last_undegraded":"2024-09-15T22:10:49.707476+0000","last_fullsized":"2024-09-15T22:10:49.707476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251657+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:10:06.251657+0000","last_peered":"2024-09-15T22:10:06.251657+0000","last_clean":"2024-09-15T22:10:06.251657+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:10:06.251657+0000","last_undegraded":"2024-09-15T22:10:06.251657+0000","last_fullsized":"2024-09-15T22:10:06.251657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084410+0000","last_change":"2024-09-15T22:09:55.216216+0000","last_active":"2024-09-15T22:10:06.084410+0000","last_peered":"2024-09-15T22:10:06.084410+0000","last_clean":"2024-09-15T22:10:06.084410+0000","last_became_active":"2024-09-15T22:09:55.215653+0000","last_became_peered":"2024-09-15T22:09:55.215653+0000","last_unstale":"2024-09-15T22:10:06.084410+0000","last_undegraded":"2024-09-15T22:10:06.084410+0000","last_fullsized":"2024-09-15T22:10:06.084410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:58.820562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.683563+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:10:38.683563+0000","last_peered":"2024-09-15T22:10:38.683563+0000","last_clean":"2024-09-15T22:10:38.683563+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:10:38.683563+0000","last_undegraded":"2024-09-15T22:10:38.683563+0000","last_fullsized":"2024-09-15T22:10:38.683563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251778+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:10:06.251778+0000","last_peered":"2024-09-15T22:10:06.251778+0000","last_clean":"2024-09-15T22:10:06.251778+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:10:06.251778+0000","last_undegraded":"2024-09-15T22:10:06.251778+0000","last_fullsized":"2024-09-15T22:10:06.251778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084588+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:10:06.084588+0000","last_peered":"2024-09-15T22:10:06.084588+0000","last_clean":"2024-09-15T22:10:06.084588+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:10:06.084588+0000","last_undegraded":"2024-09-15T22:10:06.084588+0000","last_fullsized":"2024-09-15T22:10:06.084588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.688915+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:10:38.688915+0000","last_peered":"2024-09-15T22:10:38.688915+0000","last_clean":"2024-09-15T22:10:38.688915+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:10:38.688915+0000","last_undegraded":"2024-09-15T22:10:38.688915+0000","last_fullsized":"2024-09-15T22:10:38.688915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251244+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:10:06.251244+0000","last_peered":"2024-09-15T22:10:06.251244+0000","last_clean":"2024-09-15T22:10:06.251244+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:10:06.251244+0000","last_undegraded":"2024-09-15T22:10:06.251244+0000","last_fullsized":"2024-09-15T22:10:06.251244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237745+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:10:05.237745+0000","last_peered":"2024-09-15T22:10:05.237745+0000","last_clean":"2024-09-15T22:10:05.237745+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:10:05.237745+0000","last_undegraded":"2024-09-15T22:10:05.237745+0000","last_fullsized":"2024-09-15T22:10:05.237745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.682746+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:10:38.682746+0000","last_peered":"2024-09-15T22:10:38.682746+0000","last_clean":"2024-09-15T22:10:38.682746+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:10:38.682746+0000","last_undegraded":"2024-09-15T22:10:38.682746+0000","last_fullsized":"2024-09-15T22:10:38.682746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251871+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:10:06.251871+0000","last_peered":"2024-09-15T22:10:06.251871+0000","last_clean":"2024-09-15T22:10:06.251871+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:10:06.251871+0000","last_undegraded":"2024-09-15T22:10:06.251871+0000","last_fullsized":"2024-09-15T22:10:06.251871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251925+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:10:06.251925+0000","last_peered":"2024-09-15T22:10:06.251925+0000","last_clean":"2024-09-15T22:10:06.251925+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:10:06.251925+0000","last_undegraded":"2024-09-15T22:10:06.251925+0000","last_fullsized":"2024-09-15T22:10:06.251925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.682464+0000","last_change":"2024-09-15T22:09:53.778216+0000","last_active":"2024-09-15T22:10:38.682464+0000","last_peered":"2024-09-15T22:10:38.682464+0000","last_clean":"2024-09-15T22:10:38.682464+0000","last_became_active":"2024-09-15T22:09:53.777799+0000","last_became_peered":"2024-09-15T22:09:53.777799+0000","last_unstale":"2024-09-15T22:10:38.682464+0000","last_undegraded":"2024-09-15T22:10:38.682464+0000","last_fullsized":"2024-09-15T22:10:38.682464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:39.945378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251376+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:10:06.251376+0000","last_peered":"2024-09-15T22:10:06.251376+0000","last_clean":"2024-09-15T22:10:06.251376+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:10:06.251376+0000","last_undegraded":"2024-09-15T22:10:06.251376+0000","last_fullsized":"2024-09-15T22:10:06.251376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084049+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:10:06.084049+0000","last_peered":"2024-09-15T22:10:06.084049+0000","last_clean":"2024-09-15T22:10:06.084049+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:10:06.084049+0000","last_undegraded":"2024-09-15T22:10:06.084049+0000","last_fullsized":"2024-09-15T22:10:06.084049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.708100+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:10:50.708100+0000","last_peered":"2024-09-15T22:10:50.708100+0000","last_clean":"2024-09-15T22:10:50.708100+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:10:50.708100+0000","last_undegraded":"2024-09-15T22:10:50.708100+0000","last_fullsized":"2024-09-15T22:10:50.708100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252820+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:10:06.252820+0000","last_peered":"2024-09-15T22:10:06.252820+0000","last_clean":"2024-09-15T22:10:06.252820+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:10:06.252820+0000","last_undegraded":"2024-09-15T22:10:06.252820+0000","last_fullsized":"2024-09-15T22:10:06.252820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084404+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:10:06.084404+0000","last_peered":"2024-09-15T22:10:06.084404+0000","last_clean":"2024-09-15T22:10:06.084404+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:10:06.084404+0000","last_undegraded":"2024-09-15T22:10:06.084404+0000","last_fullsized":"2024-09-15T22:10:06.084404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692213+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:10:38.692213+0000","last_peered":"2024-09-15T22:10:38.692213+0000","last_clean":"2024-09-15T22:10:38.692213+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:10:38.692213+0000","last_undegraded":"2024-09-15T22:10:38.692213+0000","last_fullsized":"2024-09-15T22:10:38.692213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237544+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:10:05.237544+0000","last_peered":"2024-09-15T22:10:05.237544+0000","last_clean":"2024-09-15T22:10:05.237544+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:10:05.237544+0000","last_undegraded":"2024-09-15T22:10:05.237544+0000","last_fullsized":"2024-09-15T22:10:05.237544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251418+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:10:06.251418+0000","last_peered":"2024-09-15T22:10:06.251418+0000","last_clean":"2024-09-15T22:10:06.251418+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:10:06.251418+0000","last_undegraded":"2024-09-15T22:10:06.251418+0000","last_fullsized":"2024-09-15T22:10:06.251418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.691925+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:10:38.691925+0000","last_peered":"2024-09-15T22:10:38.691925+0000","last_clean":"2024-09-15T22:10:38.691925+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:10:38.691925+0000","last_undegraded":"2024-09-15T22:10:38.691925+0000","last_fullsized":"2024-09-15T22:10:38.691925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252576+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:10:06.252576+0000","last_peered":"2024-09-15T22:10:06.252576+0000","last_clean":"2024-09-15T22:10:06.252576+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:10:06.252576+0000","last_undegraded":"2024-09-15T22:10:06.252576+0000","last_fullsized":"2024-09-15T22:10:06.252576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237740+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:10:05.237740+0000","last_peered":"2024-09-15T22:10:05.237740+0000","last_clean":"2024-09-15T22:10:05.237740+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:10:05.237740+0000","last_undegraded":"2024-09-15T22:10:05.237740+0000","last_fullsized":"2024-09-15T22:10:05.237740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'27","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:48.706298+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:10:48.706298+0000","last_peered":"2024-09-15T22:10:48.706298+0000","last_clean":"2024-09-15T22:10:48.706298+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:10:48.706298+0000","last_undegraded":"2024-09-15T22:10:48.706298+0000","last_fullsized":"2024-09-15T22:10:48.706298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":56,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252170+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:10:06.252170+0000","last_peered":"2024-09-15T22:10:06.252170+0000","last_clean":"2024-09-15T22:10:06.252170+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:10:06.252170+0000","last_undegraded":"2024-09-15T22:10:06.252170+0000","last_fullsized":"2024-09-15T22:10:06.252170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251977+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:10:06.251977+0000","last_peered":"2024-09-15T22:10:06.251977+0000","last_clean":"2024-09-15T22:10:06.251977+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:10:06.251977+0000","last_undegraded":"2024-09-15T22:10:06.251977+0000","last_fullsized":"2024-09-15T22:10:06.251977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.345095+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:10:38.345095+0000","last_peered":"2024-09-15T22:10:38.345095+0000","last_clean":"2024-09-15T22:10:38.345095+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:10:38.345095+0000","last_undegraded":"2024-09-15T22:10:38.345095+0000","last_fullsized":"2024-09-15T22:10:38.345095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252212+0000","last_change":"2024-09-15T22:10:02.238807+0000","last_active":"2024-09-15T22:10:06.252212+0000","last_peered":"2024-09-15T22:10:06.252212+0000","last_clean":"2024-09-15T22:10:06.252212+0000","last_became_active":"2024-09-15T22:10:02.237763+0000","last_became_peered":"2024-09-15T22:10:02.237763+0000","last_unstale":"2024-09-15T22:10:06.252212+0000","last_undegraded":"2024-09-15T22:10:06.252212+0000","last_fullsized":"2024-09-15T22:10:06.252212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:36.366180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252500+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:10:06.252500+0000","last_peered":"2024-09-15T22:10:06.252500+0000","last_clean":"2024-09-15T22:10:06.252500+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:10:06.252500+0000","last_undegraded":"2024-09-15T22:10:06.252500+0000","last_fullsized":"2024-09-15T22:10:06.252500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.688950+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:10:38.688950+0000","last_peered":"2024-09-15T22:10:38.688950+0000","last_clean":"2024-09-15T22:10:38.688950+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:10:38.688950+0000","last_undegraded":"2024-09-15T22:10:38.688950+0000","last_fullsized":"2024-09-15T22:10:38.688950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251078+0000","last_change":"2024-09-15T22:10:02.241604+0000","last_active":"2024-09-15T22:10:06.251078+0000","last_peered":"2024-09-15T22:10:06.251078+0000","last_clean":"2024-09-15T22:10:06.251078+0000","last_became_active":"2024-09-15T22:10:02.241249+0000","last_became_peered":"2024-09-15T22:10:02.241249+0000","last_unstale":"2024-09-15T22:10:06.251078+0000","last_undegraded":"2024-09-15T22:10:06.251078+0000","last_fullsized":"2024-09-15T22:10:06.251078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:48.407593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252147+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:10:06.252147+0000","last_peered":"2024-09-15T22:10:06.252147+0000","last_clean":"2024-09-15T22:10:06.252147+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:10:06.252147+0000","last_undegraded":"2024-09-15T22:10:06.252147+0000","last_fullsized":"2024-09-15T22:10:06.252147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:48.705105+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:10:48.705105+0000","last_peered":"2024-09-15T22:10:48.705105+0000","last_clean":"2024-09-15T22:10:48.705105+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:10:48.705105+0000","last_undegraded":"2024-09-15T22:10:48.705105+0000","last_fullsized":"2024-09-15T22:10:48.705105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502157+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:10:06.502157+0000","last_peered":"2024-09-15T22:10:06.502157+0000","last_clean":"2024-09-15T22:10:06.502157+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:10:06.502157+0000","last_undegraded":"2024-09-15T22:10:06.502157+0000","last_fullsized":"2024-09-15T22:10:06.502157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252131+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:10:06.252131+0000","last_peered":"2024-09-15T22:10:06.252131+0000","last_clean":"2024-09-15T22:10:06.252131+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:10:06.252131+0000","last_undegraded":"2024-09-15T22:10:06.252131+0000","last_fullsized":"2024-09-15T22:10:06.252131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251596+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:10:06.251596+0000","last_peered":"2024-09-15T22:10:06.251596+0000","last_clean":"2024-09-15T22:10:06.251596+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:10:06.251596+0000","last_undegraded":"2024-09-15T22:10:06.251596+0000","last_fullsized":"2024-09-15T22:10:06.251596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690184+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:10:38.690184+0000","last_peered":"2024-09-15T22:10:38.690184+0000","last_clean":"2024-09-15T22:10:38.690184+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:10:38.690184+0000","last_undegraded":"2024-09-15T22:10:38.690184+0000","last_fullsized":"2024-09-15T22:10:38.690184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237924+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:10:05.237924+0000","last_peered":"2024-09-15T22:10:05.237924+0000","last_clean":"2024-09-15T22:10:05.237924+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:10:05.237924+0000","last_undegraded":"2024-09-15T22:10:05.237924+0000","last_fullsized":"2024-09-15T22:10:05.237924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502138+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:10:06.502138+0000","last_peered":"2024-09-15T22:10:06.502138+0000","last_clean":"2024-09-15T22:10:06.502138+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:10:06.502138+0000","last_undegraded":"2024-09-15T22:10:06.502138+0000","last_fullsized":"2024-09-15T22:10:06.502138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:37.336669+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:10:37.336669+0000","last_peered":"2024-09-15T22:10:37.336669+0000","last_clean":"2024-09-15T22:10:37.336669+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:10:37.336669+0000","last_undegraded":"2024-09-15T22:10:37.336669+0000","last_fullsized":"2024-09-15T22:10:37.336669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.345471+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:10:38.345471+0000","last_peered":"2024-09-15T22:10:38.345471+0000","last_clean":"2024-09-15T22:10:38.345471+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:10:38.345471+0000","last_undegraded":"2024-09-15T22:10:38.345471+0000","last_fullsized":"2024-09-15T22:10:38.345471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251463+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:10:06.251463+0000","last_peered":"2024-09-15T22:10:06.251463+0000","last_clean":"2024-09-15T22:10:06.251463+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:10:06.251463+0000","last_undegraded":"2024-09-15T22:10:06.251463+0000","last_fullsized":"2024-09-15T22:10:06.251463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251257+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:10:06.251257+0000","last_peered":"2024-09-15T22:10:06.251257+0000","last_clean":"2024-09-15T22:10:06.251257+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:10:06.251257+0000","last_undegraded":"2024-09-15T22:10:06.251257+0000","last_fullsized":"2024-09-15T22:10:06.251257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.708979+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:10:50.708979+0000","last_peered":"2024-09-15T22:10:50.708979+0000","last_clean":"2024-09-15T22:10:50.708979+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:10:50.708979+0000","last_undegraded":"2024-09-15T22:10:50.708979+0000","last_fullsized":"2024-09-15T22:10:50.708979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.236894+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:10:05.236894+0000","last_peered":"2024-09-15T22:10:05.236894+0000","last_clean":"2024-09-15T22:10:05.236894+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:10:05.236894+0000","last_undegraded":"2024-09-15T22:10:05.236894+0000","last_fullsized":"2024-09-15T22:10:05.236894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252144+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:10:06.252144+0000","last_peered":"2024-09-15T22:10:06.252144+0000","last_clean":"2024-09-15T22:10:06.252144+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:10:06.252144+0000","last_undegraded":"2024-09-15T22:10:06.252144+0000","last_fullsized":"2024-09-15T22:10:06.252144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237058+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:10:05.237058+0000","last_peered":"2024-09-15T22:10:05.237058+0000","last_clean":"2024-09-15T22:10:05.237058+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:10:05.237058+0000","last_undegraded":"2024-09-15T22:10:05.237058+0000","last_fullsized":"2024-09-15T22:10:05.237058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252221+0000","last_change":"2024-09-15T22:09:55.211701+0000","last_active":"2024-09-15T22:10:06.252221+0000","last_peered":"2024-09-15T22:10:06.252221+0000","last_clean":"2024-09-15T22:10:06.252221+0000","last_became_active":"2024-09-15T22:09:55.211251+0000","last_became_peered":"2024-09-15T22:09:55.211251+0000","last_unstale":"2024-09-15T22:10:06.252221+0000","last_undegraded":"2024-09-15T22:10:06.252221+0000","last_fullsized":"2024-09-15T22:10:06.252221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:09.758924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.250882+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:10:06.250882+0000","last_peered":"2024-09-15T22:10:06.250882+0000","last_clean":"2024-09-15T22:10:06.250882+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:10:06.250882+0000","last_undegraded":"2024-09-15T22:10:06.250882+0000","last_fullsized":"2024-09-15T22:10:06.250882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251733+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:10:06.251733+0000","last_peered":"2024-09-15T22:10:06.251733+0000","last_clean":"2024-09-15T22:10:06.251733+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:10:06.251733+0000","last_undegraded":"2024-09-15T22:10:06.251733+0000","last_fullsized":"2024-09-15T22:10:06.251733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251888+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:10:06.251888+0000","last_peered":"2024-09-15T22:10:06.251888+0000","last_clean":"2024-09-15T22:10:06.251888+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:10:06.251888+0000","last_undegraded":"2024-09-15T22:10:06.251888+0000","last_fullsized":"2024-09-15T22:10:06.251888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237522+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:10:05.237522+0000","last_peered":"2024-09-15T22:10:05.237522+0000","last_clean":"2024-09-15T22:10:05.237522+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:10:05.237522+0000","last_undegraded":"2024-09-15T22:10:05.237522+0000","last_fullsized":"2024-09-15T22:10:05.237522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251556+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:10:06.251556+0000","last_peered":"2024-09-15T22:10:06.251556+0000","last_clean":"2024-09-15T22:10:06.251556+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:10:06.251556+0000","last_undegraded":"2024-09-15T22:10:06.251556+0000","last_fullsized":"2024-09-15T22:10:06.251556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502645+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:10:06.502645+0000","last_peered":"2024-09-15T22:10:06.502645+0000","last_clean":"2024-09-15T22:10:06.502645+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:10:06.502645+0000","last_undegraded":"2024-09-15T22:10:06.502645+0000","last_fullsized":"2024-09-15T22:10:06.502645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251452+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:10:06.251452+0000","last_peered":"2024-09-15T22:10:06.251452+0000","last_clean":"2024-09-15T22:10:06.251452+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:10:06.251452+0000","last_undegraded":"2024-09-15T22:10:06.251452+0000","last_fullsized":"2024-09-15T22:10:06.251452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502180+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:10:06.502180+0000","last_peered":"2024-09-15T22:10:06.502180+0000","last_clean":"2024-09-15T22:10:06.502180+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:10:06.502180+0000","last_undegraded":"2024-09-15T22:10:06.502180+0000","last_fullsized":"2024-09-15T22:10:06.502180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251786+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:10:06.251786+0000","last_peered":"2024-09-15T22:10:06.251786+0000","last_clean":"2024-09-15T22:10:06.251786+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:10:06.251786+0000","last_undegraded":"2024-09-15T22:10:06.251786+0000","last_fullsized":"2024-09-15T22:10:06.251786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251772+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:10:06.251772+0000","last_peered":"2024-09-15T22:10:06.251772+0000","last_clean":"2024-09-15T22:10:06.251772+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:10:06.251772+0000","last_undegraded":"2024-09-15T22:10:06.251772+0000","last_fullsized":"2024-09-15T22:10:06.251772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237636+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:10:05.237636+0000","last_peered":"2024-09-15T22:10:05.237636+0000","last_clean":"2024-09-15T22:10:05.237636+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:10:05.237636+0000","last_undegraded":"2024-09-15T22:10:05.237636+0000","last_fullsized":"2024-09-15T22:10:05.237636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252255+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:10:06.252255+0000","last_peered":"2024-09-15T22:10:06.252255+0000","last_clean":"2024-09-15T22:10:06.252255+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:10:06.252255+0000","last_undegraded":"2024-09-15T22:10:06.252255+0000","last_fullsized":"2024-09-15T22:10:06.252255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252644+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:10:06.252644+0000","last_peered":"2024-09-15T22:10:06.252644+0000","last_clean":"2024-09-15T22:10:06.252644+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:10:06.252644+0000","last_undegraded":"2024-09-15T22:10:06.252644+0000","last_fullsized":"2024-09-15T22:10:06.252644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238101+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:10:05.238101+0000","last_peered":"2024-09-15T22:10:05.238101+0000","last_clean":"2024-09-15T22:10:05.238101+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:10:05.238101+0000","last_undegraded":"2024-09-15T22:10:05.238101+0000","last_fullsized":"2024-09-15T22:10:05.238101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251314+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:10:06.251314+0000","last_peered":"2024-09-15T22:10:06.251314+0000","last_clean":"2024-09-15T22:10:06.251314+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:10:06.251314+0000","last_undegraded":"2024-09-15T22:10:06.251314+0000","last_fullsized":"2024-09-15T22:10:06.251314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251555+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:10:06.251555+0000","last_peered":"2024-09-15T22:10:06.251555+0000","last_clean":"2024-09-15T22:10:06.251555+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:10:06.251555+0000","last_undegraded":"2024-09-15T22:10:06.251555+0000","last_fullsized":"2024-09-15T22:10:06.251555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251539+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:10:06.251539+0000","last_peered":"2024-09-15T22:10:06.251539+0000","last_clean":"2024-09-15T22:10:06.251539+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:10:06.251539+0000","last_undegraded":"2024-09-15T22:10:06.251539+0000","last_fullsized":"2024-09-15T22:10:06.251539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251688+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:10:06.251688+0000","last_peered":"2024-09-15T22:10:06.251688+0000","last_clean":"2024-09-15T22:10:06.251688+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:10:06.251688+0000","last_undegraded":"2024-09-15T22:10:06.251688+0000","last_fullsized":"2024-09-15T22:10:06.251688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252062+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:10:06.252062+0000","last_peered":"2024-09-15T22:10:06.252062+0000","last_clean":"2024-09-15T22:10:06.252062+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:10:06.252062+0000","last_undegraded":"2024-09-15T22:10:06.252062+0000","last_fullsized":"2024-09-15T22:10:06.252062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084635+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:10:06.084635+0000","last_peered":"2024-09-15T22:10:06.084635+0000","last_clean":"2024-09-15T22:10:06.084635+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:10:06.084635+0000","last_undegraded":"2024-09-15T22:10:06.084635+0000","last_fullsized":"2024-09-15T22:10:06.084635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238382+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:10:05.238382+0000","last_peered":"2024-09-15T22:10:05.238382+0000","last_clean":"2024-09-15T22:10:05.238382+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:10:05.238382+0000","last_undegraded":"2024-09-15T22:10:05.238382+0000","last_fullsized":"2024-09-15T22:10:05.238382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252338+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:10:06.252338+0000","last_peered":"2024-09-15T22:10:06.252338+0000","last_clean":"2024-09-15T22:10:06.252338+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:10:06.252338+0000","last_undegraded":"2024-09-15T22:10:06.252338+0000","last_fullsized":"2024-09-15T22:10:06.252338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502096+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:10:06.502096+0000","last_peered":"2024-09-15T22:10:06.502096+0000","last_clean":"2024-09-15T22:10:06.502096+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:10:06.502096+0000","last_undegraded":"2024-09-15T22:10:06.502096+0000","last_fullsized":"2024-09-15T22:10:06.502096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084152+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:10:06.084152+0000","last_peered":"2024-09-15T22:10:06.084152+0000","last_clean":"2024-09-15T22:10:06.084152+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:10:06.084152+0000","last_undegraded":"2024-09-15T22:10:06.084152+0000","last_fullsized":"2024-09-15T22:10:06.084152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237887+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:10:05.237887+0000","last_peered":"2024-09-15T22:10:05.237887+0000","last_clean":"2024-09-15T22:10:05.237887+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:10:05.237887+0000","last_undegraded":"2024-09-15T22:10:05.237887+0000","last_fullsized":"2024-09-15T22:10:05.237887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.236799+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:10:05.236799+0000","last_peered":"2024-09-15T22:10:05.236799+0000","last_clean":"2024-09-15T22:10:05.236799+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:10:05.236799+0000","last_undegraded":"2024-09-15T22:10:05.236799+0000","last_fullsized":"2024-09-15T22:10:05.236799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251964+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:10:06.251964+0000","last_peered":"2024-09-15T22:10:06.251964+0000","last_clean":"2024-09-15T22:10:06.251964+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:10:06.251964+0000","last_undegraded":"2024-09-15T22:10:06.251964+0000","last_fullsized":"2024-09-15T22:10:06.251964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251935+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:10:06.251935+0000","last_peered":"2024-09-15T22:10:06.251935+0000","last_clean":"2024-09-15T22:10:06.251935+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:10:06.251935+0000","last_undegraded":"2024-09-15T22:10:06.251935+0000","last_fullsized":"2024-09-15T22:10:06.251935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252129+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:10:06.252129+0000","last_peered":"2024-09-15T22:10:06.252129+0000","last_clean":"2024-09-15T22:10:06.252129+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:10:06.252129+0000","last_undegraded":"2024-09-15T22:10:06.252129+0000","last_fullsized":"2024-09-15T22:10:06.252129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238066+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:10:05.238066+0000","last_peered":"2024-09-15T22:10:05.238066+0000","last_clean":"2024-09-15T22:10:05.238066+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:10:05.238066+0000","last_undegraded":"2024-09-15T22:10:05.238066+0000","last_fullsized":"2024-09-15T22:10:05.238066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50946,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":166,"num_read_kb":140,"num_write":239,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":170964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280881,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":486385,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743452,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":1116546,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":490018,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":486300,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1083192,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":481426,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1078381,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":485893,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":481134,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":535870,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":534864,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":483275,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:10:55.664 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:10:55.909 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:56.307 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:56 smithi077 ceph-mon[30980]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:56.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/2219411895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0101de0 msgr2=0x7f7cd01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:56.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/2219411895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0101de0 0x7f7cd01021c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009900 tx=0x7f7cc002a410 comp rx=0 tx=0).stop 2024-09-15T22:10:56.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/2219411895 shutdown_connections 2024-09-15T22:10:56.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/2219411895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0106f10 0x7f7cd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/2219411895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0102790 0x7f7cd01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/2219411895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0101de0 0x7f7cd01021c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/2219411895 >> 172.21.15.77:0/2219411895 conn(0x7f7cd00fd8c0 msgr2=0x7f7cd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/2219411895 shutdown_connections 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.383+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/2219411895 wait complete. 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 Processor -- start 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 -- start start 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 0x7f7cd01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 0x7f7cd0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0112010 con 0x7f7cd0106f10 2024-09-15T22:10:56.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0111e90 con 0x7f7cd0102790 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cd4a6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0112190 con 0x7f7cd0101de0 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41632/0 (socket says 172.21.15.77:41632) 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.384+0000 7f7cced9d640 1 -- 172.21.15.77:0/1537012040 learned_addr learned my addr 172.21.15.77:0/1537012040 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cce59c640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 0x7f7cd01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cc5d9b640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 0x7f7cd0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 -- 172.21.15.77:0/1537012040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 msgr2=0x7f7cd01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 0x7f7cd01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 -- 172.21.15.77:0/1537012040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 msgr2=0x7f7cd0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 0x7f7cd0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 -- 172.21.15.77:0/1537012040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd019d560 con 0x7f7cd0106f10 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cc5d9b640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 0x7f7cd0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:56.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cce59c640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 0x7f7cd01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cced9d640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800b970 tx=0x7f7cc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8019070 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc800cb40 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd019d850 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007650 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.385+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd01a5130 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.386+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd0102240 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.387+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7cc801c070 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.388+0000 7f7cc7fff640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 0x7f7cbc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.388+0000 7f7cce59c640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 0x7f7cbc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.388+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f7cc80cf780 con 0x7f7cd0106f10 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.389+0000 7f7cce59c640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 0x7f7cbc0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005ab0 tx=0x7f7cc0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.391+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cc809c1b0 con 0x7f7cd0106f10 2024-09-15T22:10:56.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.555+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cd0062630 con 0x7f7cbc0a32d0 2024-09-15T22:10:56.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.563+0000 7f7cc7fff640 1 -- 172.21.15.77:0/1537012040 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340659 (secure 0 0 0) 0x7f7cd0062630 con 0x7f7cbc0a32d0 2024-09-15T22:10:56.567 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:10:56.569 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 msgr2=0x7f7cbc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 0x7f7cbc0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005ab0 tx=0x7f7cc0005a20 comp rx=0 tx=0).stop 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 msgr2=0x7f7cd019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800b970 tx=0x7f7cc800be30 comp rx=0 tx=0).stop 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 shutdown_connections 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7cbc0a32d0 0x7f7cbc0a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cd0106f10 0x7f7cd019ce80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd0102790 0x7f7cd0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 --2- 172.21.15.77:0/1537012040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cd0101de0 0x7f7cd01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:56.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 >> 172.21.15.77:0/1537012040 conn(0x7f7cd00fd8c0 msgr2=0x7f7cd00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:56.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 shutdown_connections 2024-09-15T22:10:56.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:56.567+0000 7f7cd4a6f640 1 -- 172.21.15.77:0/1537012040 wait complete. 2024-09-15T22:10:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:56 smithi196 ceph-mon[35109]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:56 smithi117 ceph-mon[35021]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:57.242 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-15T22:10:56.195932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650611,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":323,"num_read_kb":283,"num_write":426,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373616,"kb_used_data":11888,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568208,"statfs":{"total":1151940427776,"available":1151557844992,"internally_reserved":0,"allocated":12173312,"data_stored":7752347,"data_compressed":317160,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49977,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":117,"num_write":34,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003586"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.763273+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:10:51.763273+0000","last_peered":"2024-09-15T22:10:51.763273+0000","last_clean":"2024-09-15T22:10:51.763273+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:10:51.763273+0000","last_undegraded":"2024-09-15T22:10:51.763273+0000","last_fullsized":"2024-09-15T22:10:51.763273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502511+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:10:06.502511+0000","last_peered":"2024-09-15T22:10:06.502511+0000","last_clean":"2024-09-15T22:10:06.502511+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:10:06.502511+0000","last_undegraded":"2024-09-15T22:10:06.502511+0000","last_fullsized":"2024-09-15T22:10:06.502511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.250909+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:10:06.250909+0000","last_peered":"2024-09-15T22:10:06.250909+0000","last_clean":"2024-09-15T22:10:06.250909+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:10:06.250909+0000","last_undegraded":"2024-09-15T22:10:06.250909+0000","last_fullsized":"2024-09-15T22:10:06.250909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502690+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:10:06.502690+0000","last_peered":"2024-09-15T22:10:06.502690+0000","last_clean":"2024-09-15T22:10:06.502690+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:10:06.502690+0000","last_undegraded":"2024-09-15T22:10:06.502690+0000","last_fullsized":"2024-09-15T22:10:06.502690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252495+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:10:06.252495+0000","last_peered":"2024-09-15T22:10:06.252495+0000","last_clean":"2024-09-15T22:10:06.252495+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:10:06.252495+0000","last_undegraded":"2024-09-15T22:10:06.252495+0000","last_fullsized":"2024-09-15T22:10:06.252495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251317+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:10:06.251317+0000","last_peered":"2024-09-15T22:10:06.251317+0000","last_clean":"2024-09-15T22:10:06.251317+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:10:06.251317+0000","last_undegraded":"2024-09-15T22:10:06.251317+0000","last_fullsized":"2024-09-15T22:10:06.251317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692565+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:10:38.692565+0000","last_peered":"2024-09-15T22:10:38.692565+0000","last_clean":"2024-09-15T22:10:38.692565+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:10:38.692565+0000","last_undegraded":"2024-09-15T22:10:38.692565+0000","last_fullsized":"2024-09-15T22:10:38.692565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251855+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:10:06.251855+0000","last_peered":"2024-09-15T22:10:06.251855+0000","last_clean":"2024-09-15T22:10:06.251855+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:10:06.251855+0000","last_undegraded":"2024-09-15T22:10:06.251855+0000","last_fullsized":"2024-09-15T22:10:06.251855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252466+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:10:06.252466+0000","last_peered":"2024-09-15T22:10:06.252466+0000","last_clean":"2024-09-15T22:10:06.252466+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:10:06.252466+0000","last_undegraded":"2024-09-15T22:10:06.252466+0000","last_fullsized":"2024-09-15T22:10:06.252466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692316+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:10:38.692316+0000","last_peered":"2024-09-15T22:10:38.692316+0000","last_clean":"2024-09-15T22:10:38.692316+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:10:38.692316+0000","last_undegraded":"2024-09-15T22:10:38.692316+0000","last_fullsized":"2024-09-15T22:10:38.692316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252563+0000","last_change":"2024-09-15T22:09:55.219247+0000","last_active":"2024-09-15T22:10:06.252563+0000","last_peered":"2024-09-15T22:10:06.252563+0000","last_clean":"2024-09-15T22:10:06.252563+0000","last_became_active":"2024-09-15T22:09:55.218925+0000","last_became_peered":"2024-09-15T22:09:55.218925+0000","last_unstale":"2024-09-15T22:10:06.252563+0000","last_undegraded":"2024-09-15T22:10:06.252563+0000","last_fullsized":"2024-09-15T22:10:06.252563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:29.567713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252098+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:10:06.252098+0000","last_peered":"2024-09-15T22:10:06.252098+0000","last_clean":"2024-09-15T22:10:06.252098+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:10:06.252098+0000","last_undegraded":"2024-09-15T22:10:06.252098+0000","last_fullsized":"2024-09-15T22:10:06.252098+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084413+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:10:06.084413+0000","last_peered":"2024-09-15T22:10:06.084413+0000","last_clean":"2024-09-15T22:10:06.084413+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:10:06.084413+0000","last_undegraded":"2024-09-15T22:10:06.084413+0000","last_fullsized":"2024-09-15T22:10:06.084413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:36.334153+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:10:36.334153+0000","last_peered":"2024-09-15T22:10:36.334153+0000","last_clean":"2024-09-15T22:10:36.334153+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:10:36.334153+0000","last_undegraded":"2024-09-15T22:10:36.334153+0000","last_fullsized":"2024-09-15T22:10:36.334153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251772+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:10:06.251772+0000","last_peered":"2024-09-15T22:10:06.251772+0000","last_clean":"2024-09-15T22:10:06.251772+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:10:06.251772+0000","last_undegraded":"2024-09-15T22:10:06.251772+0000","last_fullsized":"2024-09-15T22:10:06.251772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502307+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:10:06.502307+0000","last_peered":"2024-09-15T22:10:06.502307+0000","last_clean":"2024-09-15T22:10:06.502307+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:10:06.502307+0000","last_undegraded":"2024-09-15T22:10:06.502307+0000","last_fullsized":"2024-09-15T22:10:06.502307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.344593+0000","last_change":"2024-09-15T22:09:53.191315+0000","last_active":"2024-09-15T22:10:38.344593+0000","last_peered":"2024-09-15T22:10:38.344593+0000","last_clean":"2024-09-15T22:10:38.344593+0000","last_became_active":"2024-09-15T22:09:53.190950+0000","last_became_peered":"2024-09-15T22:09:53.190950+0000","last_unstale":"2024-09-15T22:10:38.344593+0000","last_undegraded":"2024-09-15T22:10:38.344593+0000","last_fullsized":"2024-09-15T22:10:38.344593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:05.113961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251645+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:10:06.251645+0000","last_peered":"2024-09-15T22:10:06.251645+0000","last_clean":"2024-09-15T22:10:06.251645+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:10:06.251645+0000","last_undegraded":"2024-09-15T22:10:06.251645+0000","last_fullsized":"2024-09-15T22:10:06.251645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237179+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:10:05.237179+0000","last_peered":"2024-09-15T22:10:05.237179+0000","last_clean":"2024-09-15T22:10:05.237179+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:10:05.237179+0000","last_undegraded":"2024-09-15T22:10:05.237179+0000","last_fullsized":"2024-09-15T22:10:05.237179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.689533+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:10:38.689533+0000","last_peered":"2024-09-15T22:10:38.689533+0000","last_clean":"2024-09-15T22:10:38.689533+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:10:38.689533+0000","last_undegraded":"2024-09-15T22:10:38.689533+0000","last_fullsized":"2024-09-15T22:10:38.689533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252674+0000","last_change":"2024-09-15T22:10:02.237508+0000","last_active":"2024-09-15T22:10:06.252674+0000","last_peered":"2024-09-15T22:10:06.252674+0000","last_clean":"2024-09-15T22:10:06.252674+0000","last_became_active":"2024-09-15T22:10:02.237111+0000","last_became_peered":"2024-09-15T22:10:02.237111+0000","last_unstale":"2024-09-15T22:10:06.252674+0000","last_undegraded":"2024-09-15T22:10:06.252674+0000","last_fullsized":"2024-09-15T22:10:06.252674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:27:52.924409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237294+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:10:05.237294+0000","last_peered":"2024-09-15T22:10:05.237294+0000","last_clean":"2024-09-15T22:10:05.237294+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:10:05.237294+0000","last_undegraded":"2024-09-15T22:10:05.237294+0000","last_fullsized":"2024-09-15T22:10:05.237294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:36.333281+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:10:36.333281+0000","last_peered":"2024-09-15T22:10:36.333281+0000","last_clean":"2024-09-15T22:10:36.333281+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:10:36.333281+0000","last_undegraded":"2024-09-15T22:10:36.333281+0000","last_fullsized":"2024-09-15T22:10:36.333281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252043+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:10:06.252043+0000","last_peered":"2024-09-15T22:10:06.252043+0000","last_clean":"2024-09-15T22:10:06.252043+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:10:06.252043+0000","last_undegraded":"2024-09-15T22:10:06.252043+0000","last_fullsized":"2024-09-15T22:10:06.252043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252211+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:10:06.252211+0000","last_peered":"2024-09-15T22:10:06.252211+0000","last_clean":"2024-09-15T22:10:06.252211+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:10:06.252211+0000","last_undegraded":"2024-09-15T22:10:06.252211+0000","last_fullsized":"2024-09-15T22:10:06.252211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690893+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:10:38.690893+0000","last_peered":"2024-09-15T22:10:38.690893+0000","last_clean":"2024-09-15T22:10:38.690893+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:10:38.690893+0000","last_undegraded":"2024-09-15T22:10:38.690893+0000","last_fullsized":"2024-09-15T22:10:38.690893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252162+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:10:06.252162+0000","last_peered":"2024-09-15T22:10:06.252162+0000","last_clean":"2024-09-15T22:10:06.252162+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:10:06.252162+0000","last_undegraded":"2024-09-15T22:10:06.252162+0000","last_fullsized":"2024-09-15T22:10:06.252162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502648+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:10:06.502648+0000","last_peered":"2024-09-15T22:10:06.502648+0000","last_clean":"2024-09-15T22:10:06.502648+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:10:06.502648+0000","last_undegraded":"2024-09-15T22:10:06.502648+0000","last_fullsized":"2024-09-15T22:10:06.502648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.210969+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:10:51.210969+0000","last_peered":"2024-09-15T22:10:51.210969+0000","last_clean":"2024-09-15T22:10:51.210969+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:10:51.210969+0000","last_undegraded":"2024-09-15T22:10:51.210969+0000","last_fullsized":"2024-09-15T22:10:51.210969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502039+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:10:06.502039+0000","last_peered":"2024-09-15T22:10:06.502039+0000","last_clean":"2024-09-15T22:10:06.502039+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:10:06.502039+0000","last_undegraded":"2024-09-15T22:10:06.502039+0000","last_fullsized":"2024-09-15T22:10:06.502039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251913+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:10:06.251913+0000","last_peered":"2024-09-15T22:10:06.251913+0000","last_clean":"2024-09-15T22:10:06.251913+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:10:06.251913+0000","last_undegraded":"2024-09-15T22:10:06.251913+0000","last_fullsized":"2024-09-15T22:10:06.251913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.683550+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:10:38.683550+0000","last_peered":"2024-09-15T22:10:38.683550+0000","last_clean":"2024-09-15T22:10:38.683550+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:10:38.683550+0000","last_undegraded":"2024-09-15T22:10:38.683550+0000","last_fullsized":"2024-09-15T22:10:38.683550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237477+0000","last_change":"2024-09-15T22:10:02.240073+0000","last_active":"2024-09-15T22:10:05.237477+0000","last_peered":"2024-09-15T22:10:05.237477+0000","last_clean":"2024-09-15T22:10:05.237477+0000","last_became_active":"2024-09-15T22:10:02.239655+0000","last_became_peered":"2024-09-15T22:10:02.239655+0000","last_unstale":"2024-09-15T22:10:05.237477+0000","last_undegraded":"2024-09-15T22:10:05.237477+0000","last_fullsized":"2024-09-15T22:10:05.237477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:59.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238063+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:10:05.238063+0000","last_peered":"2024-09-15T22:10:05.238063+0000","last_clean":"2024-09-15T22:10:05.238063+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:10:05.238063+0000","last_undegraded":"2024-09-15T22:10:05.238063+0000","last_fullsized":"2024-09-15T22:10:05.238063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252461+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:10:06.252461+0000","last_peered":"2024-09-15T22:10:06.252461+0000","last_clean":"2024-09-15T22:10:06.252461+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:10:06.252461+0000","last_undegraded":"2024-09-15T22:10:06.252461+0000","last_fullsized":"2024-09-15T22:10:06.252461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237761+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:10:05.237761+0000","last_peered":"2024-09-15T22:10:05.237761+0000","last_clean":"2024-09-15T22:10:05.237761+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:10:05.237761+0000","last_undegraded":"2024-09-15T22:10:05.237761+0000","last_fullsized":"2024-09-15T22:10:05.237761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237989+0000","last_change":"2024-09-15T22:09:55.203424+0000","last_active":"2024-09-15T22:10:05.237989+0000","last_peered":"2024-09-15T22:10:05.237989+0000","last_clean":"2024-09-15T22:10:05.237989+0000","last_became_active":"2024-09-15T22:09:55.203120+0000","last_became_peered":"2024-09-15T22:09:55.203120+0000","last_unstale":"2024-09-15T22:10:05.237989+0000","last_undegraded":"2024-09-15T22:10:05.237989+0000","last_fullsized":"2024-09-15T22:10:05.237989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:44.090660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.762694+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:10:50.762694+0000","last_peered":"2024-09-15T22:10:50.762694+0000","last_clean":"2024-09-15T22:10:50.762694+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:10:50.762694+0000","last_undegraded":"2024-09-15T22:10:50.762694+0000","last_fullsized":"2024-09-15T22:10:50.762694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251839+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:10:06.251839+0000","last_peered":"2024-09-15T22:10:06.251839+0000","last_clean":"2024-09-15T22:10:06.251839+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:10:06.251839+0000","last_undegraded":"2024-09-15T22:10:06.251839+0000","last_fullsized":"2024-09-15T22:10:06.251839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251488+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:10:06.251488+0000","last_peered":"2024-09-15T22:10:06.251488+0000","last_clean":"2024-09-15T22:10:06.251488+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:10:06.251488+0000","last_undegraded":"2024-09-15T22:10:06.251488+0000","last_fullsized":"2024-09-15T22:10:06.251488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:51.708671+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:10:51.708671+0000","last_peered":"2024-09-15T22:10:51.708671+0000","last_clean":"2024-09-15T22:10:51.708671+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:10:51.708671+0000","last_undegraded":"2024-09-15T22:10:51.708671+0000","last_fullsized":"2024-09-15T22:10:51.708671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251513+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:10:06.251513+0000","last_peered":"2024-09-15T22:10:06.251513+0000","last_clean":"2024-09-15T22:10:06.251513+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:10:06.251513+0000","last_undegraded":"2024-09-15T22:10:06.251513+0000","last_fullsized":"2024-09-15T22:10:06.251513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251032+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:10:06.251032+0000","last_peered":"2024-09-15T22:10:06.251032+0000","last_clean":"2024-09-15T22:10:06.251032+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:10:06.251032+0000","last_undegraded":"2024-09-15T22:10:06.251032+0000","last_fullsized":"2024-09-15T22:10:06.251032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:37.337466+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:10:37.337466+0000","last_peered":"2024-09-15T22:10:37.337466+0000","last_clean":"2024-09-15T22:10:37.337466+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:10:37.337466+0000","last_undegraded":"2024-09-15T22:10:37.337466+0000","last_fullsized":"2024-09-15T22:10:37.337466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252475+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:10:06.252475+0000","last_peered":"2024-09-15T22:10:06.252475+0000","last_clean":"2024-09-15T22:10:06.252475+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:10:06.252475+0000","last_undegraded":"2024-09-15T22:10:06.252475+0000","last_fullsized":"2024-09-15T22:10:06.252475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502232+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:10:06.502232+0000","last_peered":"2024-09-15T22:10:06.502232+0000","last_clean":"2024-09-15T22:10:06.502232+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:10:06.502232+0000","last_undegraded":"2024-09-15T22:10:06.502232+0000","last_fullsized":"2024-09-15T22:10:06.502232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690275+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:10:38.690275+0000","last_peered":"2024-09-15T22:10:38.690275+0000","last_clean":"2024-09-15T22:10:38.690275+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:10:38.690275+0000","last_undegraded":"2024-09-15T22:10:38.690275+0000","last_fullsized":"2024-09-15T22:10:38.690275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084238+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:10:06.084238+0000","last_peered":"2024-09-15T22:10:06.084238+0000","last_clean":"2024-09-15T22:10:06.084238+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:10:06.084238+0000","last_undegraded":"2024-09-15T22:10:06.084238+0000","last_fullsized":"2024-09-15T22:10:06.084238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084569+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:10:06.084569+0000","last_peered":"2024-09-15T22:10:06.084569+0000","last_clean":"2024-09-15T22:10:06.084569+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:10:06.084569+0000","last_undegraded":"2024-09-15T22:10:06.084569+0000","last_fullsized":"2024-09-15T22:10:06.084569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084673+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:10:06.084673+0000","last_peered":"2024-09-15T22:10:06.084673+0000","last_clean":"2024-09-15T22:10:06.084673+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:10:06.084673+0000","last_undegraded":"2024-09-15T22:10:06.084673+0000","last_fullsized":"2024-09-15T22:10:06.084673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:49.707476+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:10:49.707476+0000","last_peered":"2024-09-15T22:10:49.707476+0000","last_clean":"2024-09-15T22:10:49.707476+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:10:49.707476+0000","last_undegraded":"2024-09-15T22:10:49.707476+0000","last_fullsized":"2024-09-15T22:10:49.707476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251657+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:10:06.251657+0000","last_peered":"2024-09-15T22:10:06.251657+0000","last_clean":"2024-09-15T22:10:06.251657+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:10:06.251657+0000","last_undegraded":"2024-09-15T22:10:06.251657+0000","last_fullsized":"2024-09-15T22:10:06.251657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084410+0000","last_change":"2024-09-15T22:09:55.216216+0000","last_active":"2024-09-15T22:10:06.084410+0000","last_peered":"2024-09-15T22:10:06.084410+0000","last_clean":"2024-09-15T22:10:06.084410+0000","last_became_active":"2024-09-15T22:09:55.215653+0000","last_became_peered":"2024-09-15T22:09:55.215653+0000","last_unstale":"2024-09-15T22:10:06.084410+0000","last_undegraded":"2024-09-15T22:10:06.084410+0000","last_fullsized":"2024-09-15T22:10:06.084410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:58.820562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.683563+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:10:38.683563+0000","last_peered":"2024-09-15T22:10:38.683563+0000","last_clean":"2024-09-15T22:10:38.683563+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:10:38.683563+0000","last_undegraded":"2024-09-15T22:10:38.683563+0000","last_fullsized":"2024-09-15T22:10:38.683563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251778+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:10:06.251778+0000","last_peered":"2024-09-15T22:10:06.251778+0000","last_clean":"2024-09-15T22:10:06.251778+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:10:06.251778+0000","last_undegraded":"2024-09-15T22:10:06.251778+0000","last_fullsized":"2024-09-15T22:10:06.251778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084588+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:10:06.084588+0000","last_peered":"2024-09-15T22:10:06.084588+0000","last_clean":"2024-09-15T22:10:06.084588+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:10:06.084588+0000","last_undegraded":"2024-09-15T22:10:06.084588+0000","last_fullsized":"2024-09-15T22:10:06.084588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.688915+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:10:38.688915+0000","last_peered":"2024-09-15T22:10:38.688915+0000","last_clean":"2024-09-15T22:10:38.688915+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:10:38.688915+0000","last_undegraded":"2024-09-15T22:10:38.688915+0000","last_fullsized":"2024-09-15T22:10:38.688915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251244+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:10:06.251244+0000","last_peered":"2024-09-15T22:10:06.251244+0000","last_clean":"2024-09-15T22:10:06.251244+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:10:06.251244+0000","last_undegraded":"2024-09-15T22:10:06.251244+0000","last_fullsized":"2024-09-15T22:10:06.251244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237745+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:10:05.237745+0000","last_peered":"2024-09-15T22:10:05.237745+0000","last_clean":"2024-09-15T22:10:05.237745+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:10:05.237745+0000","last_undegraded":"2024-09-15T22:10:05.237745+0000","last_fullsized":"2024-09-15T22:10:05.237745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.682746+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:10:38.682746+0000","last_peered":"2024-09-15T22:10:38.682746+0000","last_clean":"2024-09-15T22:10:38.682746+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:10:38.682746+0000","last_undegraded":"2024-09-15T22:10:38.682746+0000","last_fullsized":"2024-09-15T22:10:38.682746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251871+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:10:06.251871+0000","last_peered":"2024-09-15T22:10:06.251871+0000","last_clean":"2024-09-15T22:10:06.251871+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:10:06.251871+0000","last_undegraded":"2024-09-15T22:10:06.251871+0000","last_fullsized":"2024-09-15T22:10:06.251871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251925+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:10:06.251925+0000","last_peered":"2024-09-15T22:10:06.251925+0000","last_clean":"2024-09-15T22:10:06.251925+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:10:06.251925+0000","last_undegraded":"2024-09-15T22:10:06.251925+0000","last_fullsized":"2024-09-15T22:10:06.251925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.682464+0000","last_change":"2024-09-15T22:09:53.778216+0000","last_active":"2024-09-15T22:10:38.682464+0000","last_peered":"2024-09-15T22:10:38.682464+0000","last_clean":"2024-09-15T22:10:38.682464+0000","last_became_active":"2024-09-15T22:09:53.777799+0000","last_became_peered":"2024-09-15T22:09:53.777799+0000","last_unstale":"2024-09-15T22:10:38.682464+0000","last_undegraded":"2024-09-15T22:10:38.682464+0000","last_fullsized":"2024-09-15T22:10:38.682464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:39.945378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251376+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:10:06.251376+0000","last_peered":"2024-09-15T22:10:06.251376+0000","last_clean":"2024-09-15T22:10:06.251376+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:10:06.251376+0000","last_undegraded":"2024-09-15T22:10:06.251376+0000","last_fullsized":"2024-09-15T22:10:06.251376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084049+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:10:06.084049+0000","last_peered":"2024-09-15T22:10:06.084049+0000","last_clean":"2024-09-15T22:10:06.084049+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:10:06.084049+0000","last_undegraded":"2024-09-15T22:10:06.084049+0000","last_fullsized":"2024-09-15T22:10:06.084049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.708100+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:10:50.708100+0000","last_peered":"2024-09-15T22:10:50.708100+0000","last_clean":"2024-09-15T22:10:50.708100+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:10:50.708100+0000","last_undegraded":"2024-09-15T22:10:50.708100+0000","last_fullsized":"2024-09-15T22:10:50.708100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252820+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:10:06.252820+0000","last_peered":"2024-09-15T22:10:06.252820+0000","last_clean":"2024-09-15T22:10:06.252820+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:10:06.252820+0000","last_undegraded":"2024-09-15T22:10:06.252820+0000","last_fullsized":"2024-09-15T22:10:06.252820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084404+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:10:06.084404+0000","last_peered":"2024-09-15T22:10:06.084404+0000","last_clean":"2024-09-15T22:10:06.084404+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:10:06.084404+0000","last_undegraded":"2024-09-15T22:10:06.084404+0000","last_fullsized":"2024-09-15T22:10:06.084404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.692213+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:10:38.692213+0000","last_peered":"2024-09-15T22:10:38.692213+0000","last_clean":"2024-09-15T22:10:38.692213+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:10:38.692213+0000","last_undegraded":"2024-09-15T22:10:38.692213+0000","last_fullsized":"2024-09-15T22:10:38.692213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237544+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:10:05.237544+0000","last_peered":"2024-09-15T22:10:05.237544+0000","last_clean":"2024-09-15T22:10:05.237544+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:10:05.237544+0000","last_undegraded":"2024-09-15T22:10:05.237544+0000","last_fullsized":"2024-09-15T22:10:05.237544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251418+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:10:06.251418+0000","last_peered":"2024-09-15T22:10:06.251418+0000","last_clean":"2024-09-15T22:10:06.251418+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:10:06.251418+0000","last_undegraded":"2024-09-15T22:10:06.251418+0000","last_fullsized":"2024-09-15T22:10:06.251418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.691925+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:10:38.691925+0000","last_peered":"2024-09-15T22:10:38.691925+0000","last_clean":"2024-09-15T22:10:38.691925+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:10:38.691925+0000","last_undegraded":"2024-09-15T22:10:38.691925+0000","last_fullsized":"2024-09-15T22:10:38.691925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252576+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:10:06.252576+0000","last_peered":"2024-09-15T22:10:06.252576+0000","last_clean":"2024-09-15T22:10:06.252576+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:10:06.252576+0000","last_undegraded":"2024-09-15T22:10:06.252576+0000","last_fullsized":"2024-09-15T22:10:06.252576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237740+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:10:05.237740+0000","last_peered":"2024-09-15T22:10:05.237740+0000","last_clean":"2024-09-15T22:10:05.237740+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:10:05.237740+0000","last_undegraded":"2024-09-15T22:10:05.237740+0000","last_fullsized":"2024-09-15T22:10:05.237740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:53.712090+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:10:53.712090+0000","last_peered":"2024-09-15T22:10:53.712090+0000","last_clean":"2024-09-15T22:10:53.712090+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:10:53.712090+0000","last_undegraded":"2024-09-15T22:10:53.712090+0000","last_fullsized":"2024-09-15T22:10:53.712090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":70,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252170+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:10:06.252170+0000","last_peered":"2024-09-15T22:10:06.252170+0000","last_clean":"2024-09-15T22:10:06.252170+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:10:06.252170+0000","last_undegraded":"2024-09-15T22:10:06.252170+0000","last_fullsized":"2024-09-15T22:10:06.252170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251977+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:10:06.251977+0000","last_peered":"2024-09-15T22:10:06.251977+0000","last_clean":"2024-09-15T22:10:06.251977+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:10:06.251977+0000","last_undegraded":"2024-09-15T22:10:06.251977+0000","last_fullsized":"2024-09-15T22:10:06.251977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.345095+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:10:38.345095+0000","last_peered":"2024-09-15T22:10:38.345095+0000","last_clean":"2024-09-15T22:10:38.345095+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:10:38.345095+0000","last_undegraded":"2024-09-15T22:10:38.345095+0000","last_fullsized":"2024-09-15T22:10:38.345095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252212+0000","last_change":"2024-09-15T22:10:02.238807+0000","last_active":"2024-09-15T22:10:06.252212+0000","last_peered":"2024-09-15T22:10:06.252212+0000","last_clean":"2024-09-15T22:10:06.252212+0000","last_became_active":"2024-09-15T22:10:02.237763+0000","last_became_peered":"2024-09-15T22:10:02.237763+0000","last_unstale":"2024-09-15T22:10:06.252212+0000","last_undegraded":"2024-09-15T22:10:06.252212+0000","last_fullsized":"2024-09-15T22:10:06.252212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:36.366180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252500+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:10:06.252500+0000","last_peered":"2024-09-15T22:10:06.252500+0000","last_clean":"2024-09-15T22:10:06.252500+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:10:06.252500+0000","last_undegraded":"2024-09-15T22:10:06.252500+0000","last_fullsized":"2024-09-15T22:10:06.252500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.688950+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:10:38.688950+0000","last_peered":"2024-09-15T22:10:38.688950+0000","last_clean":"2024-09-15T22:10:38.688950+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:10:38.688950+0000","last_undegraded":"2024-09-15T22:10:38.688950+0000","last_fullsized":"2024-09-15T22:10:38.688950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251078+0000","last_change":"2024-09-15T22:10:02.241604+0000","last_active":"2024-09-15T22:10:06.251078+0000","last_peered":"2024-09-15T22:10:06.251078+0000","last_clean":"2024-09-15T22:10:06.251078+0000","last_became_active":"2024-09-15T22:10:02.241249+0000","last_became_peered":"2024-09-15T22:10:02.241249+0000","last_unstale":"2024-09-15T22:10:06.251078+0000","last_undegraded":"2024-09-15T22:10:06.251078+0000","last_fullsized":"2024-09-15T22:10:06.251078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:48.407593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252147+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:10:06.252147+0000","last_peered":"2024-09-15T22:10:06.252147+0000","last_clean":"2024-09-15T22:10:06.252147+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:10:06.252147+0000","last_undegraded":"2024-09-15T22:10:06.252147+0000","last_fullsized":"2024-09-15T22:10:06.252147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:53.712012+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:10:53.712012+0000","last_peered":"2024-09-15T22:10:53.712012+0000","last_clean":"2024-09-15T22:10:53.712012+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:10:53.712012+0000","last_undegraded":"2024-09-15T22:10:53.712012+0000","last_fullsized":"2024-09-15T22:10:53.712012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502157+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:10:06.502157+0000","last_peered":"2024-09-15T22:10:06.502157+0000","last_clean":"2024-09-15T22:10:06.502157+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:10:06.502157+0000","last_undegraded":"2024-09-15T22:10:06.502157+0000","last_fullsized":"2024-09-15T22:10:06.502157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252131+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:10:06.252131+0000","last_peered":"2024-09-15T22:10:06.252131+0000","last_clean":"2024-09-15T22:10:06.252131+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:10:06.252131+0000","last_undegraded":"2024-09-15T22:10:06.252131+0000","last_fullsized":"2024-09-15T22:10:06.252131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251596+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:10:06.251596+0000","last_peered":"2024-09-15T22:10:06.251596+0000","last_clean":"2024-09-15T22:10:06.251596+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:10:06.251596+0000","last_undegraded":"2024-09-15T22:10:06.251596+0000","last_fullsized":"2024-09-15T22:10:06.251596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.690184+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:10:38.690184+0000","last_peered":"2024-09-15T22:10:38.690184+0000","last_clean":"2024-09-15T22:10:38.690184+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:10:38.690184+0000","last_undegraded":"2024-09-15T22:10:38.690184+0000","last_fullsized":"2024-09-15T22:10:38.690184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237924+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:10:05.237924+0000","last_peered":"2024-09-15T22:10:05.237924+0000","last_clean":"2024-09-15T22:10:05.237924+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:10:05.237924+0000","last_undegraded":"2024-09-15T22:10:05.237924+0000","last_fullsized":"2024-09-15T22:10:05.237924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502138+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:10:06.502138+0000","last_peered":"2024-09-15T22:10:06.502138+0000","last_clean":"2024-09-15T22:10:06.502138+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:10:06.502138+0000","last_undegraded":"2024-09-15T22:10:06.502138+0000","last_fullsized":"2024-09-15T22:10:06.502138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:37.336669+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:10:37.336669+0000","last_peered":"2024-09-15T22:10:37.336669+0000","last_clean":"2024-09-15T22:10:37.336669+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:10:37.336669+0000","last_undegraded":"2024-09-15T22:10:37.336669+0000","last_fullsized":"2024-09-15T22:10:37.336669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:38.345471+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:10:38.345471+0000","last_peered":"2024-09-15T22:10:38.345471+0000","last_clean":"2024-09-15T22:10:38.345471+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:10:38.345471+0000","last_undegraded":"2024-09-15T22:10:38.345471+0000","last_fullsized":"2024-09-15T22:10:38.345471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251463+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:10:06.251463+0000","last_peered":"2024-09-15T22:10:06.251463+0000","last_clean":"2024-09-15T22:10:06.251463+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:10:06.251463+0000","last_undegraded":"2024-09-15T22:10:06.251463+0000","last_fullsized":"2024-09-15T22:10:06.251463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251257+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:10:06.251257+0000","last_peered":"2024-09-15T22:10:06.251257+0000","last_clean":"2024-09-15T22:10:06.251257+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:10:06.251257+0000","last_undegraded":"2024-09-15T22:10:06.251257+0000","last_fullsized":"2024-09-15T22:10:06.251257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:50.708979+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:10:50.708979+0000","last_peered":"2024-09-15T22:10:50.708979+0000","last_clean":"2024-09-15T22:10:50.708979+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:10:50.708979+0000","last_undegraded":"2024-09-15T22:10:50.708979+0000","last_fullsized":"2024-09-15T22:10:50.708979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.236894+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:10:05.236894+0000","last_peered":"2024-09-15T22:10:05.236894+0000","last_clean":"2024-09-15T22:10:05.236894+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:10:05.236894+0000","last_undegraded":"2024-09-15T22:10:05.236894+0000","last_fullsized":"2024-09-15T22:10:05.236894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252144+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:10:06.252144+0000","last_peered":"2024-09-15T22:10:06.252144+0000","last_clean":"2024-09-15T22:10:06.252144+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:10:06.252144+0000","last_undegraded":"2024-09-15T22:10:06.252144+0000","last_fullsized":"2024-09-15T22:10:06.252144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237058+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:10:05.237058+0000","last_peered":"2024-09-15T22:10:05.237058+0000","last_clean":"2024-09-15T22:10:05.237058+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:10:05.237058+0000","last_undegraded":"2024-09-15T22:10:05.237058+0000","last_fullsized":"2024-09-15T22:10:05.237058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252221+0000","last_change":"2024-09-15T22:09:55.211701+0000","last_active":"2024-09-15T22:10:06.252221+0000","last_peered":"2024-09-15T22:10:06.252221+0000","last_clean":"2024-09-15T22:10:06.252221+0000","last_became_active":"2024-09-15T22:09:55.211251+0000","last_became_peered":"2024-09-15T22:09:55.211251+0000","last_unstale":"2024-09-15T22:10:06.252221+0000","last_undegraded":"2024-09-15T22:10:06.252221+0000","last_fullsized":"2024-09-15T22:10:06.252221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:09.758924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.250882+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:10:06.250882+0000","last_peered":"2024-09-15T22:10:06.250882+0000","last_clean":"2024-09-15T22:10:06.250882+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:10:06.250882+0000","last_undegraded":"2024-09-15T22:10:06.250882+0000","last_fullsized":"2024-09-15T22:10:06.250882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251733+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:10:06.251733+0000","last_peered":"2024-09-15T22:10:06.251733+0000","last_clean":"2024-09-15T22:10:06.251733+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:10:06.251733+0000","last_undegraded":"2024-09-15T22:10:06.251733+0000","last_fullsized":"2024-09-15T22:10:06.251733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251888+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:10:06.251888+0000","last_peered":"2024-09-15T22:10:06.251888+0000","last_clean":"2024-09-15T22:10:06.251888+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:10:06.251888+0000","last_undegraded":"2024-09-15T22:10:06.251888+0000","last_fullsized":"2024-09-15T22:10:06.251888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237522+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:10:05.237522+0000","last_peered":"2024-09-15T22:10:05.237522+0000","last_clean":"2024-09-15T22:10:05.237522+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:10:05.237522+0000","last_undegraded":"2024-09-15T22:10:05.237522+0000","last_fullsized":"2024-09-15T22:10:05.237522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251556+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:10:06.251556+0000","last_peered":"2024-09-15T22:10:06.251556+0000","last_clean":"2024-09-15T22:10:06.251556+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:10:06.251556+0000","last_undegraded":"2024-09-15T22:10:06.251556+0000","last_fullsized":"2024-09-15T22:10:06.251556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502645+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:10:06.502645+0000","last_peered":"2024-09-15T22:10:06.502645+0000","last_clean":"2024-09-15T22:10:06.502645+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:10:06.502645+0000","last_undegraded":"2024-09-15T22:10:06.502645+0000","last_fullsized":"2024-09-15T22:10:06.502645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251452+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:10:06.251452+0000","last_peered":"2024-09-15T22:10:06.251452+0000","last_clean":"2024-09-15T22:10:06.251452+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:10:06.251452+0000","last_undegraded":"2024-09-15T22:10:06.251452+0000","last_fullsized":"2024-09-15T22:10:06.251452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502180+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:10:06.502180+0000","last_peered":"2024-09-15T22:10:06.502180+0000","last_clean":"2024-09-15T22:10:06.502180+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:10:06.502180+0000","last_undegraded":"2024-09-15T22:10:06.502180+0000","last_fullsized":"2024-09-15T22:10:06.502180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251786+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:10:06.251786+0000","last_peered":"2024-09-15T22:10:06.251786+0000","last_clean":"2024-09-15T22:10:06.251786+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:10:06.251786+0000","last_undegraded":"2024-09-15T22:10:06.251786+0000","last_fullsized":"2024-09-15T22:10:06.251786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251772+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:10:06.251772+0000","last_peered":"2024-09-15T22:10:06.251772+0000","last_clean":"2024-09-15T22:10:06.251772+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:10:06.251772+0000","last_undegraded":"2024-09-15T22:10:06.251772+0000","last_fullsized":"2024-09-15T22:10:06.251772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237636+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:10:05.237636+0000","last_peered":"2024-09-15T22:10:05.237636+0000","last_clean":"2024-09-15T22:10:05.237636+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:10:05.237636+0000","last_undegraded":"2024-09-15T22:10:05.237636+0000","last_fullsized":"2024-09-15T22:10:05.237636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252255+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:10:06.252255+0000","last_peered":"2024-09-15T22:10:06.252255+0000","last_clean":"2024-09-15T22:10:06.252255+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:10:06.252255+0000","last_undegraded":"2024-09-15T22:10:06.252255+0000","last_fullsized":"2024-09-15T22:10:06.252255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252644+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:10:06.252644+0000","last_peered":"2024-09-15T22:10:06.252644+0000","last_clean":"2024-09-15T22:10:06.252644+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:10:06.252644+0000","last_undegraded":"2024-09-15T22:10:06.252644+0000","last_fullsized":"2024-09-15T22:10:06.252644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238101+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:10:05.238101+0000","last_peered":"2024-09-15T22:10:05.238101+0000","last_clean":"2024-09-15T22:10:05.238101+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:10:05.238101+0000","last_undegraded":"2024-09-15T22:10:05.238101+0000","last_fullsized":"2024-09-15T22:10:05.238101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251314+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:10:06.251314+0000","last_peered":"2024-09-15T22:10:06.251314+0000","last_clean":"2024-09-15T22:10:06.251314+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:10:06.251314+0000","last_undegraded":"2024-09-15T22:10:06.251314+0000","last_fullsized":"2024-09-15T22:10:06.251314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251555+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:10:06.251555+0000","last_peered":"2024-09-15T22:10:06.251555+0000","last_clean":"2024-09-15T22:10:06.251555+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:10:06.251555+0000","last_undegraded":"2024-09-15T22:10:06.251555+0000","last_fullsized":"2024-09-15T22:10:06.251555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251539+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:10:06.251539+0000","last_peered":"2024-09-15T22:10:06.251539+0000","last_clean":"2024-09-15T22:10:06.251539+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:10:06.251539+0000","last_undegraded":"2024-09-15T22:10:06.251539+0000","last_fullsized":"2024-09-15T22:10:06.251539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251688+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:10:06.251688+0000","last_peered":"2024-09-15T22:10:06.251688+0000","last_clean":"2024-09-15T22:10:06.251688+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:10:06.251688+0000","last_undegraded":"2024-09-15T22:10:06.251688+0000","last_fullsized":"2024-09-15T22:10:06.251688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252062+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:10:06.252062+0000","last_peered":"2024-09-15T22:10:06.252062+0000","last_clean":"2024-09-15T22:10:06.252062+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:10:06.252062+0000","last_undegraded":"2024-09-15T22:10:06.252062+0000","last_fullsized":"2024-09-15T22:10:06.252062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084635+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:10:06.084635+0000","last_peered":"2024-09-15T22:10:06.084635+0000","last_clean":"2024-09-15T22:10:06.084635+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:10:06.084635+0000","last_undegraded":"2024-09-15T22:10:06.084635+0000","last_fullsized":"2024-09-15T22:10:06.084635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238382+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:10:05.238382+0000","last_peered":"2024-09-15T22:10:05.238382+0000","last_clean":"2024-09-15T22:10:05.238382+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:10:05.238382+0000","last_undegraded":"2024-09-15T22:10:05.238382+0000","last_fullsized":"2024-09-15T22:10:05.238382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252338+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:10:06.252338+0000","last_peered":"2024-09-15T22:10:06.252338+0000","last_clean":"2024-09-15T22:10:06.252338+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:10:06.252338+0000","last_undegraded":"2024-09-15T22:10:06.252338+0000","last_fullsized":"2024-09-15T22:10:06.252338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.502096+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:10:06.502096+0000","last_peered":"2024-09-15T22:10:06.502096+0000","last_clean":"2024-09-15T22:10:06.502096+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:10:06.502096+0000","last_undegraded":"2024-09-15T22:10:06.502096+0000","last_fullsized":"2024-09-15T22:10:06.502096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.084152+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:10:06.084152+0000","last_peered":"2024-09-15T22:10:06.084152+0000","last_clean":"2024-09-15T22:10:06.084152+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:10:06.084152+0000","last_undegraded":"2024-09-15T22:10:06.084152+0000","last_fullsized":"2024-09-15T22:10:06.084152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.237887+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:10:05.237887+0000","last_peered":"2024-09-15T22:10:05.237887+0000","last_clean":"2024-09-15T22:10:05.237887+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:10:05.237887+0000","last_undegraded":"2024-09-15T22:10:05.237887+0000","last_fullsized":"2024-09-15T22:10:05.237887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.236799+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:10:05.236799+0000","last_peered":"2024-09-15T22:10:05.236799+0000","last_clean":"2024-09-15T22:10:05.236799+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:10:05.236799+0000","last_undegraded":"2024-09-15T22:10:05.236799+0000","last_fullsized":"2024-09-15T22:10:05.236799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251964+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:10:06.251964+0000","last_peered":"2024-09-15T22:10:06.251964+0000","last_clean":"2024-09-15T22:10:06.251964+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:10:06.251964+0000","last_undegraded":"2024-09-15T22:10:06.251964+0000","last_fullsized":"2024-09-15T22:10:06.251964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.251935+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:10:06.251935+0000","last_peered":"2024-09-15T22:10:06.251935+0000","last_clean":"2024-09-15T22:10:06.251935+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:10:06.251935+0000","last_undegraded":"2024-09-15T22:10:06.251935+0000","last_fullsized":"2024-09-15T22:10:06.251935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:06.252129+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:10:06.252129+0000","last_peered":"2024-09-15T22:10:06.252129+0000","last_clean":"2024-09-15T22:10:06.252129+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:10:06.252129+0000","last_undegraded":"2024-09-15T22:10:06.252129+0000","last_fullsized":"2024-09-15T22:10:06.252129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:10:05.238066+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:10:05.238066+0000","last_peered":"2024-09-15T22:10:05.238066+0000","last_clean":"2024-09-15T22:10:05.238066+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:10:05.238066+0000","last_undegraded":"2024-09-15T22:10:05.238066+0000","last_fullsized":"2024-09-15T22:10:05.238066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60009,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":187,"num_read_kb":164,"num_write":243,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":180027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280881,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":486385,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1125609,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":490018,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":486300,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1083192,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":481426,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1078381,"data_compressed":53223,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":485893,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":481134,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":535870,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":534864,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":483275,"data_compressed":17499,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:10:57.247 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:10:57.248 INFO:tasks.cephadm:Setting up client nodes... 2024-09-15T22:10:57.248 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-15T22:10:57.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:57 smithi077 ceph-mon[30980]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-15T22:10:57.492 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:57.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:57 smithi196 ceph-mon[35109]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-15T22:10:57.619 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:57 smithi117 ceph-mon[35021]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-15T22:10:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 -- 172.21.15.77:0/1396307110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d0104fd0 msgr2=0x7f74d01073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 --2- 172.21.15.77:0/1396307110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d0104fd0 0x7f74d01073c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009900 tx=0x7f74bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 -- 172.21.15.77:0/1396307110 shutdown_connections 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 --2- 172.21.15.77:0/1396307110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 --2- 172.21.15.77:0/1396307110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d0104fd0 0x7f74d01073c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 --2- 172.21.15.77:0/1396307110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d01026a0 0x7f74d0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 -- 172.21.15.77:0/1396307110 >> 172.21.15.77:0/1396307110 conn(0x7f74d00fc2a0 msgr2=0x7f74d00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 -- 172.21.15.77:0/1396307110 shutdown_connections 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.963+0000 7f74d71a9640 1 -- 172.21.15.77:0/1396307110 wait complete. 2024-09-15T22:10:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 Processor -- start 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 -- start start 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 0x7f74d0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d010d8f0 con 0x7f74d01026a0 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74d010d770 con 0x7f74d0104fd0 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d71a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74d010da70 con 0x7f74d0107900 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41652/0 (socket says 172.21.15.77:41652) 2024-09-15T22:10:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 -- 172.21.15.77:0/528275638 learned_addr learned my addr 172.21.15.77:0/528275638 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74cffff640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 0x7f74d0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d5746640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 -- 172.21.15.77:0/528275638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 msgr2=0x7f74d019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 -- 172.21.15.77:0/528275638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 msgr2=0x7f74d0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 0x7f74d0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 -- 172.21.15.77:0/528275638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d019fb40 con 0x7f74d01026a0 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74cffff640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 0x7f74d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d5746640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.964+0000 7f74d4f45640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c920 tx=0x7f74c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.965+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4007c70 con 0x7f74d01026a0 2024-09-15T22:10:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.965+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d019fe30 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.965+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c4007e10 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.965+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4018890 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.965+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a0300 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.966+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d00fdd90 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.967+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f74c4018a30 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.967+0000 7f74cdffb640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 0x7f74c00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.967+0000 7f74cffff640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 0x7f74c00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.967+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f74c40cfea0 con 0x7f74d01026a0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.968+0000 7f74cffff640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 0x7f74c00a57c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000c00 tx=0x7f74bc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:57.971+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74c4014030 con 0x7f74d01026a0 2024-09-15T22:10:58.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.203+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f74d002d9b0 con 0x7f74d01026a0 2024-09-15T22:10:58.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.210+0000 7f74cdffb640 1 -- 172.21.15.77:0/528275638 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f74c401f300 con 0x7f74d01026a0 2024-09-15T22:10:58.212 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2024-09-15T22:10:58.212 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQByW+dma8FUDBAAJLNXbVsWTZ9TJF82ZDGrIg== 2024-09-15T22:10:58.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 msgr2=0x7f74c00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:58.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 0x7f74c00a57c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000c00 tx=0x7f74bc031400 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 msgr2=0x7f74d0197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c920 tx=0x7f74c400cde0 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 shutdown_connections 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f74c00a3300 0x7f74c00a57c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d0107900 0x7f74d019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74d0104fd0 0x7f74d0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 --2- 172.21.15.77:0/528275638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d01026a0 0x7f74d0197f10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 >> 172.21.15.77:0/528275638 conn(0x7f74d00fc2a0 msgr2=0x7f74d0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 shutdown_connections 2024-09-15T22:10:58.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:58.213+0000 7f74d71a9640 1 -- 172.21.15.77:0/528275638 wait complete. 2024-09-15T22:10:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:58 smithi077 ceph-mon[30980]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:58 smithi196 ceph-mon[35109]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:58 smithi117 ceph-mon[35021]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:10:58.961 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:10:58.961 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-15T22:10:58.961 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T22:10:59.000 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T22:10:59.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-15T22:10:59.000 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-15T22:10:59.286 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:59.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:59 smithi077 ceph-mon[30980]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-15T22:10:59.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:10:59.562 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:10:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:10:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:59 smithi196 ceph-mon[35109]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-15T22:10:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:10:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:10:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:10:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:59 smithi117 ceph-mon[35021]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-15T22:10:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:10:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:10:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/528275638' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:10:59.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 -- 172.21.15.77:0/841387698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100100 msgr2=0x7f1be0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:59.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/841387698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100100 0x7f1be0100560 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009900 tx=0x7f1bd002a410 comp rx=0 tx=0).stop 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 -- 172.21.15.77:0/841387698 shutdown_connections 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/841387698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100aa0 0x7f1be010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/841387698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100100 0x7f1be0100560 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/841387698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 0x7f1be0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 -- 172.21.15.77:0/841387698 >> 172.21.15.77:0/841387698 conn(0x7f1be0077230 msgr2=0x7f1be00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 -- 172.21.15.77:0/841387698 shutdown_connections 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.779+0000 7f1be7cd4640 1 -- 172.21.15.77:0/841387698 wait complete. 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 Processor -- start 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 -- start start 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 0x7f1be019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:59.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 0x7f1be019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0112090 con 0x7f1be0100aa0 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0111f10 con 0x7f1be0106b80 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be7cd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0112210 con 0x7f1be0100100 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43590/0 (socket says 172.21.15.77:43590) 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 -- 172.21.15.77:0/2420249882 learned_addr learned my addr 172.21.15.77:0/2420249882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be5a70640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 0x7f1be019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be6271640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 0x7f1be019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 -- 172.21.15.77:0/2420249882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 msgr2=0x7f1be019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 0x7f1be019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 -- 172.21.15.77:0/2420249882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 msgr2=0x7f1be019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 0x7f1be019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:59.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 -- 172.21.15.77:0/2420249882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be019f6c0 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be5a70640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 0x7f1be019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.780+0000 7f1be526f640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006010 tx=0x7f1bd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.781+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd003f030 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.781+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be019f950 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.781+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0003ce0 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.781+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0035070 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.781+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01a7290 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.782+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be0103420 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.783+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1bd0042030 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.783+0000 7f1bd6ffd640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 0x7f1bcc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.783+0000 7f1be5a70640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 0x7f1bcc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.783+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f1bd00f06f0 con 0x7f1be0100aa0 2024-09-15T22:10:59.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.784+0000 7f1be5a70640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 0x7f1bcc0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005d00 tx=0x7f1bdc005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:59.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.787+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1bd00bd130 con 0x7f1be0100aa0 2024-09-15T22:10:59.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:10:59.997+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1be0100560 con 0x7f1be0100aa0 2024-09-15T22:11:00.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.002+0000 7f1bd6ffd640 1 -- 172.21.15.77:0/2420249882 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264049 (secure 0 0 0) 0x7f1bd00bca30 con 0x7f1be0100aa0 2024-09-15T22:11:00.007 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 msgr2=0x7f1bcc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 0x7f1bcc0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005d00 tx=0x7f1bdc005c70 comp rx=0 tx=0).stop 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 msgr2=0x7f1be019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006010 tx=0x7f1bd00042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 shutdown_connections 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1bcc0a32d0 0x7f1bcc0a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be0106b80 0x7f1be019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1be0100aa0 0x7f1be019ac50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 --2- 172.21.15.77:0/2420249882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1be0100100 0x7f1be019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 >> 172.21.15.77:0/2420249882 conn(0x7f1be0077230 msgr2=0x7f1be0104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 shutdown_connections 2024-09-15T22:11:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:00.005+0000 7f1be7cd4640 1 -- 172.21.15.77:0/2420249882 wait complete. 2024-09-15T22:11:00.285 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2420249882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:11:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2420249882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:11:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2420249882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:11:00.735 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1809193257},{"type":"v1","addr":"172.21.15.77:6801","nonce":1809193257}]},"active_addr":"172.21.15.77:6801/1809193257","active_change":"2024-09-15T22:03:04.102085+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.77:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3279424814}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1437248041}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3785009540}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":999829703}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1208699031}]}]} 2024-09-15T22:11:00.741 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-15T22:11:00.742 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-15T22:11:00.742 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:11:00.999 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:01.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:01 smithi077 ceph-mon[30980]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T22:11:01.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 -- 172.21.15.77:0/285774518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90106a80 msgr2=0x7f2c90106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 --2- 172.21.15.77:0/285774518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90106a80 0x7f2c90106e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009930 tx=0x7f2c8c02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 -- 172.21.15.77:0/285774518 shutdown_connections 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 --2- 172.21.15.77:0/285774518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90101420 0x7f2c9010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 --2- 172.21.15.77:0/285774518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90100a80 0x7f2c90100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 --2- 172.21.15.77:0/285774518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90106a80 0x7f2c90106e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 -- 172.21.15.77:0/285774518 >> 172.21.15.77:0/285774518 conn(0x7f2c900fc7a0 msgr2=0x7f2c900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 -- 172.21.15.77:0/285774518 shutdown_connections 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.500+0000 7f2c97680640 1 -- 172.21.15.77:0/285774518 wait complete. 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 Processor -- start 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 -- start start 2024-09-15T22:11:01.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90100a80 0x7f2c90102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 0x7f2c901034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90110e80 con 0x7f2c90101420 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90110d00 con 0x7f2c90106a80 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c97680640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90111000 con 0x7f2c90100a80 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43602/0 (socket says 172.21.15.77:43602) 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 -- 172.21.15.77:0/1987820676 learned_addr learned my addr 172.21.15.77:0/1987820676 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c9541c640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90100a80 0x7f2c90102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c95c1d640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 0x7f2c901034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:01.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 -- 172.21.15.77:0/1987820676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90100a80 msgr2=0x7f2c90102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90100a80 0x7f2c90102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 -- 172.21.15.77:0/1987820676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 msgr2=0x7f2c901034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 0x7f2c901034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 -- 172.21.15.77:0/1987820676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c901a68c0 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c95c1d640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 0x7f2c901034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.501+0000 7f2c94c1b640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.502+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80017070 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.502+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80009c30 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.502+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901a6bb0 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.502+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000f6b0 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.502+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a7140 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.503+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c90100ee0 con 0x7f2c90101420 2024-09-15T22:11:01.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.505+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2c800109d0 con 0x7f2c90101420 2024-09-15T22:11:01.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.505+0000 7f2c867fc640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 0x7f2c7c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:01.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.505+0000 7f2c9541c640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 0x7f2c7c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:01.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.505+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f2c800cf920 con 0x7f2c90101420 2024-09-15T22:11:01.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.506+0000 7f2c9541c640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 0x7f2c7c0a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c000c00 tx=0x7f2c8c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:01.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.508+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c80009de0 con 0x7f2c90101420 2024-09-15T22:11:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:01 smithi196 ceph-mon[35109]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T22:11:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:01 smithi117 ceph-mon[35021]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-15T22:11:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.677+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c90103f60 con 0x7f2c90101420 2024-09-15T22:11:01.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.679+0000 7f2c867fc640 1 -- 172.21.15.77:0/1987820676 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f2c800b4e20 con 0x7f2c90101420 2024-09-15T22:11:01.681 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:01.681 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:11:01.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.681+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 msgr2=0x7f2c7c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:01.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.681+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 0x7f2c7c0a5810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c000c00 tx=0x7f2c8c005a20 comp rx=0 tx=0).stop 2024-09-15T22:11:01.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.681+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 msgr2=0x7f2c90102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:01.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.681+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000de10 comp rx=0 tx=0).stop 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.681+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 shutdown_connections 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2c7c0a3350 0x7f2c7c0a5810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c90106a80 0x7f2c901034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c90101420 0x7f2c90102f70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 --2- 172.21.15.77:0/1987820676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c90100a80 0x7f2c90102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 >> 172.21.15.77:0/1987820676 conn(0x7f2c900fc7a0 msgr2=0x7f2c90104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 shutdown_connections 2024-09-15T22:11:01.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:01.682+0000 7f2c97680640 1 -- 172.21.15.77:0/1987820676 wait complete. 2024-09-15T22:11:02.378 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-15T22:11:02.378 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:11:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:02 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1987820676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:02 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1987820676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:02 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1987820676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:02.624 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:03.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 -- 172.21.15.77:0/318661755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e41002f0 msgr2=0x7f09e41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:03.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/318661755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e41002f0 0x7f09e41006d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009930 tx=0x7f09d402a420 comp rx=0 tx=0).stop 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 -- 172.21.15.77:0/318661755 shutdown_connections 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/318661755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4106f50 0x7f09e410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/318661755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e4100ca0 0x7f09e4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/318661755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e41002f0 0x7f09e41006d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 -- 172.21.15.77:0/318661755 >> 172.21.15.77:0/318661755 conn(0x7f09e4077210 msgr2=0x7f09e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 -- 172.21.15.77:0/318661755 shutdown_connections 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 -- 172.21.15.77:0/318661755 wait complete. 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.106+0000 7f09eb2b9640 1 Processor -- start 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 -- start start 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e41002f0 0x7f09e41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4100ca0 0x7f09e4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4112000 con 0x7f09e4106f50 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4111e80 con 0x7f09e4100ca0 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09eb2b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4112180 con 0x7f09e41002f0 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43620/0 (socket says 172.21.15.77:43620) 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 -- 172.21.15.77:0/1077943421 learned_addr learned my addr 172.21.15.77:0/1077943421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9055640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e41002f0 0x7f09e41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e8854640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4100ca0 0x7f09e4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 -- 172.21.15.77:0/1077943421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e41002f0 msgr2=0x7f09e41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e41002f0 0x7f09e41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 -- 172.21.15.77:0/1077943421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4100ca0 msgr2=0x7f09e4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4100ca0 0x7f09e4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 -- 172.21.15.77:0/1077943421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e419d490 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.107+0000 7f09e9856640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cce0 tx=0x7f09dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.108+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00f450 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.108+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09dc00fad0 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.108+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e419d780 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.108+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc018b60 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.108+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41a5060 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.109+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e41006d0 con 0x7f09e4106f50 2024-09-15T22:11:03.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.110+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f09dc00fc80 con 0x7f09e4106f50 2024-09-15T22:11:03.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.110+0000 7f09da7fc640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:03.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.111+0000 7f09e9055640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:03.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.111+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f09dc0d0540 con 0x7f09e4106f50 2024-09-15T22:11:03.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.111+0000 7f09e9055640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 0x7f09d00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009900 tx=0x7f09d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:03.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.114+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09dc014030 con 0x7f09e4106f50 2024-09-15T22:11:03.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.282+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e4199840 con 0x7f09e4106f50 2024-09-15T22:11:03.286 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:03 smithi077 ceph-mon[30980]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-15T22:11:03.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.284+0000 7f09da7fc640 1 -- 172.21.15.77:0/1077943421 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24249 (secure 0 0 0) 0x7f09e4199840 con 0x7f09e4106f50 2024-09-15T22:11:03.287 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:03.288 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":88,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:10:04.832434+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:11:03.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 msgr2=0x7f09d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 0x7f09d00a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009900 tx=0x7f09d40058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 msgr2=0x7f09e419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cce0 tx=0x7f09dc00a620 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 shutdown_connections 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09d00a32d0 0x7f09d00a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09e4106f50 0x7f09e419cdb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09e4100ca0 0x7f09e4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 --2- 172.21.15.77:0/1077943421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09e41002f0 0x7f09e41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 >> 172.21.15.77:0/1077943421 conn(0x7f09e4077210 msgr2=0x7f09e4105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 shutdown_connections 2024-09-15T22:11:03.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:03.287+0000 7f09eb2b9640 1 -- 172.21.15.77:0/1077943421 wait complete. 2024-09-15T22:11:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:03 smithi196 ceph-mon[35109]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-15T22:11:03.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:03 smithi117 ceph-mon[35021]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-15T22:11:03.963 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-15T22:11:03.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-15T22:11:03.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-15T22:11:03.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-15T22:11:03.965 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-15T22:11:03.965 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-15T22:11:03.965 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-15T22:11:03.966 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-15T22:11:03.966 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-15T22:11:03.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-15T22:11:03.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-15T22:11:03.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-15T22:11:04.232 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1077943421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:04.232 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-15T22:11:04.232 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:04 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1077943421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:04 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1077943421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:11:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-15T22:11:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:11:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-09-15T22:11:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-15T22:11:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:04 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-09-15T22:11:04.656 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:04.682 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:04.694 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:04.694 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:04.703 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-09-15T22:11:05.422 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-15T22:11:05.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-15T22:11:05.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-15T22:11:05.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:05 smithi077 ceph-mon[30980]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:11:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 -- 172.21.15.77:0/3419201346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0102790 msgr2=0x7f4cc01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/3419201346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0102790 0x7f4cc01067e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009900 tx=0x7f4cb402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 -- 172.21.15.77:0/3419201346 shutdown_connections 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/3419201346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0106f10 0x7f4cc010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/3419201346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0102790 0x7f4cc01067e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/3419201346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0101de0 0x7f4cc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 -- 172.21.15.77:0/3419201346 >> 172.21.15.77:0/3419201346 conn(0x7f4cc00fd8c0 msgr2=0x7f4cc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 -- 172.21.15.77:0/3419201346 shutdown_connections 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.420+0000 7f4cc63a8640 1 -- 172.21.15.77:0/3419201346 wait complete. 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 Processor -- start 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 -- start start 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0101de0 0x7f4cc0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0102790 0x7f4cc0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 0x7f4cc019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0111f70 con 0x7f4cc0106f10 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0111df0 con 0x7f4cc0101de0 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc63a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01120f0 con 0x7f4cc0102790 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 0x7f4cc019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cbffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0101de0 0x7f4cc0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 0x7f4cc019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43646/0 (socket says 172.21.15.77:43646) 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 -- 172.21.15.77:0/2968601392 learned_addr learned my addr 172.21.15.77:0/2968601392 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cbf7fe640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0102790 0x7f4cc0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 -- 172.21.15.77:0/2968601392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0102790 msgr2=0x7f4cc0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0102790 0x7f4cc0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 -- 172.21.15.77:0/2968601392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0101de0 msgr2=0x7f4cc0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0101de0 0x7f4cc0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cc4945640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc019d3c0 con 0x7f4cc0106f10 2024-09-15T22:11:05.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.421+0000 7f4cbf7fe640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc0102790 0x7f4cc0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cc4945640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 0x7f4cc019cce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800cce0 tx=0x7f4cb800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800f450 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb800fad0 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc019d6b0 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8018c10 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.422+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01a4f90 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.423+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cc0062980 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.424+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4cb800f5f0 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.424+0000 7f4cbd7fa640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4ca80a32d0 0x7f4ca80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbffff640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4ca80a32d0 0x7f4ca80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4cb80d06c0 con 0x7f4cc0106f10 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbd7fa640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] conn(0x7f4ca80ae4a0 0x7f4ca80b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cb8015e70 con 0x7f4ca80ae4a0 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4cb80d0b10 con 0x7f4cc0106f10 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbf7fe640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] conn(0x7f4ca80ae4a0 0x7f4ca80b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbffff640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4ca80a32d0 0x7f4ca80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000bbe0 tx=0x7f4cb000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.425+0000 7f4cbf7fe640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] conn(0x7f4ca80ae4a0 0x7f4ca80b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.426+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== osd.0 v2:172.21.15.77:6802/4041675784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cb8015e70 con 0x7f4ca80ae4a0 2024-09-15T22:11:05.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.440+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 --> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4cc0062e00 con 0x7f4ca80ae4a0 2024-09-15T22:11:05.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.440+0000 7f4cbd7fa640 1 -- 172.21.15.77:0/2968601392 <== osd.0 v2:172.21.15.77:6802/4041675784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4cc0062e00 con 0x7f4ca80ae4a0 2024-09-15T22:11:05.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] conn(0x7f4ca80ae4a0 msgr2=0x7f4ca80b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] conn(0x7f4ca80ae4a0 0x7f4ca80b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4ca80a32d0 msgr2=0x7f4ca80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4ca80a32d0 0x7f4ca80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000bbe0 tx=0x7f4cb000b390 comp rx=0 tx=0).stop 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 msgr2=0x7f4cc019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 --2- 172.21.15.77:0/2968601392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cc0106f10 0x7f4cc019cce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800cce0 tx=0x7f4cb800a620 comp rx=0 tx=0).stop 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc4945640 1 -- 172.21.15.77:0/2968601392 reap_dead start 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 shutdown_connections 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 >> 172.21.15.77:0/2968601392 conn(0x7f4cc00fd8c0 msgr2=0x7f4cc00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 shutdown_connections 2024-09-15T22:11:05.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.441+0000 7f4cc63a8640 1 -- 172.21.15.77:0/2968601392 wait complete. 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-15T22:11:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:11:05.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-09-15T22:11:05.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-15T22:11:05.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-15T22:11:05.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-15T22:11:05.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:05 smithi196 ceph-mon[35109]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:11:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-15T22:11:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-15T22:11:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-15T22:11:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-15T22:11:05.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:05 smithi117 ceph-mon[35021]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 -- 172.21.15.77:0/1353929000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738106f10 msgr2=0x7f573810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 --2- 172.21.15.77:0/1353929000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738106f10 0x7f573810ea90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f573002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 -- 172.21.15.77:0/1353929000 shutdown_connections 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 --2- 172.21.15.77:0/1353929000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738106f10 0x7f573810ea90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 --2- 172.21.15.77:0/1353929000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738102790 0x7f57381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 --2- 172.21.15.77:0/1353929000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5738101de0 0x7f57381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 -- 172.21.15.77:0/1353929000 >> 172.21.15.77:0/1353929000 conn(0x7f57380fd8c0 msgr2=0x7f57380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 -- 172.21.15.77:0/1353929000 shutdown_connections 2024-09-15T22:11:05.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.636+0000 7f573daa7640 1 -- 172.21.15.77:0/1353929000 wait complete. 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 Processor -- start 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 -- start start 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 0x7f5738198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5738102790 0x7f5738198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738106f10 0x7f573819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111f70 con 0x7f5738101de0 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5738111df0 con 0x7f5738102790 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f573daa7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57381120f0 con 0x7f5738106f10 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 0x7f5738198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 0x7f5738198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43670/0 (socket says 172.21.15.77:43670) 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 -- 172.21.15.77:0/3685547579 learned_addr learned my addr 172.21.15.77:0/3685547579 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f5736ffd640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5738102790 0x7f5738198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f5737fff640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738106f10 0x7f573819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 -- 172.21.15.77:0/3685547579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738106f10 msgr2=0x7f573819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5738106f10 0x7f573819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 -- 172.21.15.77:0/3685547579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5738102790 msgr2=0x7f5738198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5738102790 0x7f5738198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.637+0000 7f57377fe640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573819d3e0 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f57377fe640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 0x7f5738198430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f572800c920 tx=0x7f572800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728007c70 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573819d6d0 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728007e10 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728005260 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.638+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381a4fb0 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.639+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57380fee30 con 0x7f5738101de0 2024-09-15T22:11:05.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.640+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5728005400 con 0x7f5738101de0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5734ff9640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f57240a32d0 0x7f57240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5736ffd640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f57240a32d0 0x7f57240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f57280d0160 con 0x7f5738101de0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5734ff9640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] conn(0x7f57240ae4a0 0x7f57240b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5728004470 con 0x7f57240ae4a0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f57280d05b0 con 0x7f5738101de0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5737fff640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] conn(0x7f57240ae4a0 0x7f57240b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5736ffd640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f57240a32d0 0x7f57240a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57381087b0 tx=0x7f572000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.641+0000 7f5737fff640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] conn(0x7f57240ae4a0 0x7f57240b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:05.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.642+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== osd.9 v2:172.21.15.77:6826/3515252544 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5728004470 con 0x7f57240ae4a0 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.655+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 --> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f57381032c0 con 0x7f57240ae4a0 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f5734ff9640 1 -- 172.21.15.77:0/3685547579 <== osd.9 v2:172.21.15.77:6826/3515252544 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f57381032c0 con 0x7f57240ae4a0 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] conn(0x7f57240ae4a0 msgr2=0x7f57240b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] conn(0x7f57240ae4a0 0x7f57240b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f57240a32d0 msgr2=0x7f57240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f57240a32d0 0x7f57240a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57381087b0 tx=0x7f572000a250 comp rx=0 tx=0).stop 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 msgr2=0x7f5738198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:05.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 --2- 172.21.15.77:0/3685547579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5738101de0 0x7f5738198430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f572800c920 tx=0x7f572800cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:05.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f5737fff640 1 -- 172.21.15.77:0/3685547579 reap_dead start 2024-09-15T22:11:05.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 shutdown_connections 2024-09-15T22:11:05.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 >> 172.21.15.77:0/3685547579 conn(0x7f57380fd8c0 msgr2=0x7f57380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:05.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 shutdown_connections 2024-09-15T22:11:05.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:05.656+0000 7f573daa7640 1 -- 172.21.15.77:0/3685547579 wait complete. 2024-09-15T22:11:06.284 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:06.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:06 smithi077 ceph-mon[30980]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:11:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:06 smithi196 ceph-mon[35109]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:11:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:06 smithi117 ceph-mon[35021]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:11:06.980 INFO:teuthology.orchestra.run.smithi077.stdout:34359738449 2024-09-15T22:11:06.980 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-15T22:11:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 -- 172.21.15.77:0/1733887743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 msgr2=0x7fa7cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/1733887743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc100ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009930 tx=0x7fa7c002a420 comp rx=0 tx=0).stop 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 -- 172.21.15.77:0/1733887743 shutdown_connections 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/1733887743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101420 0x7fa7cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/1733887743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc100ee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/1733887743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc106a80 0x7fa7cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 -- 172.21.15.77:0/1733887743 >> 172.21.15.77:0/1733887743 conn(0x7fa7cc0fc7a0 msgr2=0x7fa7cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 -- 172.21.15.77:0/1733887743 shutdown_connections 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.159+0000 7fa7d22f0640 1 -- 172.21.15.77:0/1733887743 wait complete. 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 Processor -- start 2024-09-15T22:11:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 -- start start 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101420 0x7fa7cc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc106a80 0x7fa7cc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc110e40 con 0x7fa7cc100a80 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc110cc0 con 0x7fa7cc101420 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7d22f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc110fc0 con 0x7fa7cc106a80 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.160+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43680/0 (socket says 172.21.15.77:43680) 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 -- 172.21.15.77:0/881105484 learned_addr learned my addr 172.21.15.77:0/881105484 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cb7fe640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101420 0x7fa7cc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7d088d640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc106a80 0x7fa7cc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 -- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc106a80 msgr2=0x7fa7cc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc106a80 0x7fa7cc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 -- 172.21.15.77:0/881105484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101420 msgr2=0x7fa7cc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7cc101420 0x7fa7cc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc19d3b0 con 0x7fa7cc100a80 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7cbfff640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc198400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c920 tx=0x7fa7bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc007c70 con 0x7fa7cc100a80 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc007e10 con 0x7fa7cc100a80 2024-09-15T22:11:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc19d6a0 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.161+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc018890 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.162+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc1a4f80 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.162+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7cc06aaa0 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.164+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa7bc018a70 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7c97fa640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7b40a3350 0x7fa7b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa7bc0d01c0 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7cb7fe640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7b40a3350 0x7fa7b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7c97fa640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] conn(0x7fa7b40ae520 0x7fa7b40b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7b40b1090 con 0x7fa7b40ae520 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.165+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc0d0610 con 0x7fa7cc100a80 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.166+0000 7fa7cb7fe640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7b40a3350 0x7fa7b40a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009900 tx=0x7fa7c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.166+0000 7fa7d088d640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] conn(0x7fa7b40ae520 0x7fa7b40b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:07.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.166+0000 7fa7d088d640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] conn(0x7fa7b40ae520 0x7fa7b40b09c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.167+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== osd.2 v2:172.21.15.196:6800/3745852928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b40b1090 con 0x7fa7b40ae520 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.180+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 --> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7cc06ad70 con 0x7fa7b40ae520 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.181+0000 7fa7c97fa640 1 -- 172.21.15.77:0/881105484 <== osd.2 v2:172.21.15.196:6800/3745852928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa7cc06ad70 con 0x7fa7b40ae520 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] conn(0x7fa7b40ae520 msgr2=0x7fa7b40b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] conn(0x7fa7b40ae520 0x7fa7b40b09c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7b40a3350 msgr2=0x7fa7b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7b40a3350 0x7fa7b40a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009900 tx=0x7fa7c00058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 msgr2=0x7fa7cc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 --2- 172.21.15.77:0/881105484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7cc100a80 0x7fa7cc198400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c920 tx=0x7fa7bc00cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d088d640 1 -- 172.21.15.77:0/881105484 reap_dead start 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 shutdown_connections 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 >> 172.21.15.77:0/881105484 conn(0x7fa7cc0fc7a0 msgr2=0x7fa7cc1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 shutdown_connections 2024-09-15T22:11:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:07.182+0000 7fa7d22f0640 1 -- 172.21.15.77:0/881105484 wait complete. 2024-09-15T22:11:07.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:07.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:07.544 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:07 smithi077 ceph-mon[30980]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:11:07.544 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:07 smithi077 ceph-mon[30980]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-15T22:11:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:07 smithi196 ceph-mon[35109]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:11:07.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:07 smithi196 ceph-mon[35109]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-15T22:11:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:07 smithi117 ceph-mon[35021]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:11:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:07 smithi117 ceph-mon[35021]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-15T22:11:07.911 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:07.930 INFO:teuthology.orchestra.run.smithi077.stdout:77309411402 2024-09-15T22:11:07.930 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-15T22:11:08.333 INFO:teuthology.orchestra.run.smithi077.stdout:274877906982 2024-09-15T22:11:08.334 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-15T22:11:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- 172.21.15.77:0/2065155861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c106f10 msgr2=0x7fde9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 --2- 172.21.15.77:0/2065155861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c106f10 0x7fde9c10ea90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ca0 tx=0x7fde9402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- 172.21.15.77:0/2065155861 shutdown_connections 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 --2- 172.21.15.77:0/2065155861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c106f10 0x7fde9c10ea90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 --2- 172.21.15.77:0/2065155861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c102790 0x7fde9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 --2- 172.21.15.77:0/2065155861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c101de0 0x7fde9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- 172.21.15.77:0/2065155861 >> 172.21.15.77:0/2065155861 conn(0x7fde9c0fd8c0 msgr2=0x7fde9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- 172.21.15.77:0/2065155861 shutdown_connections 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- 172.21.15.77:0/2065155861 wait complete. 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 Processor -- start 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.049+0000 7fdea28d9640 1 -- start start 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 0x7fde9c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c102790 0x7fde9c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c106f10 0x7fde9c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112010 con 0x7fde9c101de0 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c111e90 con 0x7fde9c102790 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea28d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112190 con 0x7fde9c106f10 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 0x7fde9c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 0x7fde9c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40530/0 (socket says 172.21.15.77:40530) 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 -- 172.21.15.77:0/927237098 learned_addr learned my addr 172.21.15.77:0/927237098 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fdea0e76640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c106f10 0x7fde9c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9b7fe640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c102790 0x7fde9c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 -- 172.21.15.77:0/927237098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c106f10 msgr2=0x7fde9c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c106f10 0x7fde9c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 -- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c102790 msgr2=0x7fde9c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c102790 0x7fde9c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c19d520 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde9bfff640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 0x7fde9c198570 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00b4a0 tx=0x7fde8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.050+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c014070 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.051+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c010af0 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.051+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c017d10 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.051+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c19d810 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.051+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a50f0 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.051+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde9c062980 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.053+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fde8c017070 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.053+0000 7fde997fa640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fde840a32d0 0x7fde840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.053+0000 7fde9b7fe640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fde840a32d0 0x7fde840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.053+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fde8c0cf540 con 0x7fde9c101de0 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fde997fa640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] conn(0x7fde840ae4a0 0x7fde840b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde840b1010 con 0x7fde840ae4a0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fde8c0cf990 con 0x7fde9c101de0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fde9b7fe640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fde840a32d0 0x7fde840a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0fdce0 tx=0x7fde90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fdea0e76640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] conn(0x7fde840ae4a0 0x7fde840b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.054+0000 7fdea0e76640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] conn(0x7fde840ae4a0 0x7fde840b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.055+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== osd.1 v2:172.21.15.117:6800/3227694760 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde840b1010 con 0x7fde840ae4a0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.068+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 --> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde9c062e00 con 0x7fde840ae4a0 2024-09-15T22:11:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fde997fa640 1 -- 172.21.15.77:0/927237098 <== osd.1 v2:172.21.15.117:6800/3227694760 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fde9c062e00 con 0x7fde840ae4a0 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] conn(0x7fde840ae4a0 msgr2=0x7fde840b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] conn(0x7fde840ae4a0 0x7fde840b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fde840a32d0 msgr2=0x7fde840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fde840a32d0 0x7fde840a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0fdce0 tx=0x7fde90008040 comp rx=0 tx=0).stop 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 msgr2=0x7fde9c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 --2- 172.21.15.77:0/927237098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde9c101de0 0x7fde9c198570 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00b4a0 tx=0x7fde8c00b960 comp rx=0 tx=0).stop 2024-09-15T22:11:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea0e76640 1 -- 172.21.15.77:0/927237098 reap_dead start 2024-09-15T22:11:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 shutdown_connections 2024-09-15T22:11:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 >> 172.21.15.77:0/927237098 conn(0x7fde9c0fd8c0 msgr2=0x7fde9c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 shutdown_connections 2024-09-15T22:11:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.069+0000 7fdea28d9640 1 -- 172.21.15.77:0/927237098 wait complete. 2024-09-15T22:11:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 -- 172.21.15.77:0/547097413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 msgr2=0x7f589410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 --2- 172.21.15.77:0/547097413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f589410da40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f588c009ca0 tx=0x7f588c02a950 comp rx=0 tx=0).stop 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 -- 172.21.15.77:0/547097413 shutdown_connections 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 --2- 172.21.15.77:0/547097413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f589410da40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 --2- 172.21.15.77:0/547097413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 0x7f58941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 --2- 172.21.15.77:0/547097413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5894100e20 0x7f5894101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 -- 172.21.15.77:0/547097413 >> 172.21.15.77:0/547097413 conn(0x7f58940fc7d0 msgr2=0x7f58940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 -- 172.21.15.77:0/547097413 shutdown_connections 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.278+0000 7f5899c3c640 1 -- 172.21.15.77:0/547097413 wait complete. 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 Processor -- start 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 -- start start 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5894100e20 0x7f5894199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 0x7f5894199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f58941a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5894110f80 con 0x7f5894102ef0 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5894110e00 con 0x7f5894102570 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5899c3c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5894111100 con 0x7f5894100e20 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5893fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f58941a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5893fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f58941a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40544/0 (socket says 172.21.15.77:40544) 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.279+0000 7f5893fff640 1 -- 172.21.15.77:0/3099805996 learned_addr learned my addr 172.21.15.77:0/3099805996 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5892ffd640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 0x7f5894199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 -- 172.21.15.77:0/3099805996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5894100e20 msgr2=0x7f5894199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f58937fe640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5894100e20 0x7f5894199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5894100e20 0x7f5894199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 -- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 msgr2=0x7f5894199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 0x7f5894199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941a47d0 con 0x7f5894102ef0 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5892ffd640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5894102570 0x7f5894199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5893fff640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f58941a4290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f588c02ae70 tx=0x7f588c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c034040 con 0x7f5894102ef0 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58941a49a0 con 0x7f5894102ef0 2024-09-15T22:11:09.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f588c030920 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c045480 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.280+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58941a4e80 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.281+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5894107290 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.282+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f588c037020 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f5890ff9640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f587c0a32d0 0x7f587c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f58937fe640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f587c0a32d0 0x7f587c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f588c0f1050 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f5890ff9640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] conn(0x7f587c0ae4a0 0x7f587c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f587c0b1010 con 0x7f587c0ae4a0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f588c0f14a0 con 0x7f5894102ef0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.283+0000 7f58937fe640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f587c0a32d0 0x7f587c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5884005fd0 tx=0x7f5884005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.284+0000 7f5892ffd640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] conn(0x7f587c0ae4a0 0x7f587c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:09.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.284+0000 7f5892ffd640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] conn(0x7f587c0ae4a0 0x7f587c0b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.285+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== osd.7 v2:172.21.15.117:6816/159988175 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f587c0b1010 con 0x7f587c0ae4a0 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.298+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 --> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58941021b0 con 0x7f587c0ae4a0 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5890ff9640 1 -- 172.21.15.77:0/3099805996 <== osd.7 v2:172.21.15.117:6816/159988175 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58941021b0 con 0x7f587c0ae4a0 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] conn(0x7f587c0ae4a0 msgr2=0x7f587c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] conn(0x7f587c0ae4a0 0x7f587c0b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f587c0a32d0 msgr2=0x7f587c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f587c0a32d0 0x7f587c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5884005fd0 tx=0x7f5884005850 comp rx=0 tx=0).stop 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 msgr2=0x7f58941a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 --2- 172.21.15.77:0/3099805996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5894102ef0 0x7f58941a4290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f588c02ae70 tx=0x7f588c02f5d0 comp rx=0 tx=0).stop 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5893fff640 1 -- 172.21.15.77:0/3099805996 reap_dead start 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 shutdown_connections 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 >> 172.21.15.77:0/3099805996 conn(0x7f58940fc7d0 msgr2=0x7f5894106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:09.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 shutdown_connections 2024-09-15T22:11:09.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:09.299+0000 7f5899c3c640 1 -- 172.21.15.77:0/3099805996 wait complete. 2024-09-15T22:11:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:09 smithi077 ceph-mon[30980]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 9 op/s 2024-09-15T22:11:09.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:09 smithi196 ceph-mon[35109]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 9 op/s 2024-09-15T22:11:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:09 smithi117 ceph-mon[35021]: pgmap v322: 129 pgs: 4 peering, 125 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 9 op/s 2024-09-15T22:11:10.034 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:10.034 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:10.490 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:10 smithi077 ceph-mon[30980]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-15T22:11:10.490 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:10 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:11:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:10 smithi196 ceph-mon[35109]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-15T22:11:10.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:10 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:11:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:10 smithi117 ceph-mon[35021]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-15T22:11:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:10 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:11:10.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 -- 172.21.15.77:0/6537778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 msgr2=0x7f56b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 --2- 172.21.15.77:0/6537778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b410da40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f569c009900 tx=0x7f569c02a480 comp rx=0 tx=0).stop 2024-09-15T22:11:10.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 -- 172.21.15.77:0/6537778 shutdown_connections 2024-09-15T22:11:10.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 --2- 172.21.15.77:0/6537778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b410da40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 --2- 172.21.15.77:0/6537778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4102570 0x7f56b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 --2- 172.21.15.77:0/6537778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100e20 0x7f56b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 -- 172.21.15.77:0/6537778 >> 172.21.15.77:0/6537778 conn(0x7f56b40fc7d0 msgr2=0x7f56b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 -- 172.21.15.77:0/6537778 shutdown_connections 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.568+0000 7f56bac76640 1 -- 172.21.15.77:0/6537778 wait complete. 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 Processor -- start 2024-09-15T22:11:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 -- start start 2024-09-15T22:11:10.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100e20 0x7f56b41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4102570 0x7f56b41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4110fa0 con 0x7f56b4102ef0 2024-09-15T22:11:10.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4110e20 con 0x7f56b4100e20 2024-09-15T22:11:10.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56bac76640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4111120 con 0x7f56b4102570 2024-09-15T22:11:10.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56b9213640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56b9213640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b41a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40570/0 (socket says 172.21.15.77:40570) 2024-09-15T22:11:10.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.569+0000 7f56b9213640 1 -- 172.21.15.77:0/732117884 learned_addr learned my addr 172.21.15.77:0/732117884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:10.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b8a12640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100e20 0x7f56b41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 -- 172.21.15.77:0/732117884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4102570 msgr2=0x7f56b41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4102570 0x7f56b41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 -- 172.21.15.77:0/732117884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100e20 msgr2=0x7f56b41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100e20 0x7f56b41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41a4860 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b9213640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b41a4320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f569c005ab0 tx=0x7f569c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c03f070 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569c031e30 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41a4a30 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c035030 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.570+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41a4e80 con 0x7f56b4102ef0 2024-09-15T22:11:10.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.571+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56b4107220 con 0x7f56b4102ef0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.572+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f569c042070 con 0x7f56b4102ef0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b1ffb640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b8a12640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f569c0f0f00 con 0x7f56b4102ef0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b1ffb640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] conn(0x7f56a00ae4a0 0x7f56a00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f569c038e90 con 0x7f56a00ae4a0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f569c0f1350 con 0x7f56b4102ef0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b3fff640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] conn(0x7f56a00ae4a0 0x7f56a00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.573+0000 7f56b8a12640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00ba40 tx=0x7f56ac005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.574+0000 7f56b3fff640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] conn(0x7f56a00ae4a0 0x7f56a00b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.574+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== osd.6 v2:172.21.15.77:6818/3314328155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f569c038e90 con 0x7f56a00ae4a0 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 --> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f56b41021b0 con 0x7f56a00ae4a0 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56b1ffb640 1 -- 172.21.15.77:0/732117884 <== osd.6 v2:172.21.15.77:6818/3314328155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f56b41021b0 con 0x7f56a00ae4a0 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] conn(0x7f56a00ae4a0 msgr2=0x7f56a00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] conn(0x7f56a00ae4a0 0x7f56a00b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 msgr2=0x7f56a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00ba40 tx=0x7f56ac005cc0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 msgr2=0x7f56b41a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 --2- 172.21.15.77:0/732117884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4102ef0 0x7f56b41a4320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f569c005ab0 tx=0x7f569c003c30 comp rx=0 tx=0).stop 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56b9213640 1 -- 172.21.15.77:0/732117884 reap_dead start 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 shutdown_connections 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 >> 172.21.15.77:0/732117884 conn(0x7f56b40fc7d0 msgr2=0x7f56b4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 shutdown_connections 2024-09-15T22:11:10.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:10.588+0000 7f56bac76640 1 -- 172.21.15.77:0/732117884 wait complete. 2024-09-15T22:11:11.454 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:11 smithi077 ceph-mon[30980]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:11.487 INFO:teuthology.orchestra.run.smithi077.stdout:193273528373 2024-09-15T22:11:11.487 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-15T22:11:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:11 smithi196 ceph-mon[35109]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:11 smithi117 ceph-mon[35021]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:11.944 INFO:teuthology.orchestra.run.smithi077.stdout:55834574927 2024-09-15T22:11:11.945 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-15T22:11:12.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 -- 172.21.15.77:0/221700495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70106a70 msgr2=0x7fcd70106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 --2- 172.21.15.77:0/221700495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70106a70 0x7fcd70106e50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009930 tx=0x7fcd6c02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:12.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 -- 172.21.15.77:0/221700495 shutdown_connections 2024-09-15T22:11:12.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 --2- 172.21.15.77:0/221700495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd70101410 0x7fcd7010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 --2- 172.21.15.77:0/221700495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd70100a70 0x7fcd70100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 --2- 172.21.15.77:0/221700495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70106a70 0x7fcd70106e50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 -- 172.21.15.77:0/221700495 >> 172.21.15.77:0/221700495 conn(0x7fcd700fc7d0 msgr2=0x7fcd700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 -- 172.21.15.77:0/221700495 shutdown_connections 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.127+0000 7fcd7746f640 1 -- 172.21.15.77:0/221700495 wait complete. 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 Processor -- start 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 -- start start 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd70100a70 0x7fcd70197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 0x7fcd70197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd70106a70 0x7fcd7019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70110eb0 con 0x7fcd70101410 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70110d30 con 0x7fcd70100a70 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd74a0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 0x7fcd70197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd74a0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 0x7fcd70197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40594/0 (socket says 172.21.15.77:40594) 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7746f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111030 con 0x7fcd70106a70 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd74a0a640 1 -- 172.21.15.77:0/905468068 learned_addr learned my addr 172.21.15.77:0/905468068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:12.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd7520b640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd70100a70 0x7fcd70197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.128+0000 7fcd75a0c640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd70106a70 0x7fcd7019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 -- 172.21.15.77:0/905468068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd70106a70 msgr2=0x7fcd7019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd70106a70 0x7fcd7019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 -- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd70100a70 msgr2=0x7fcd70197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd70100a70 0x7fcd70197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019c3c0 con 0x7fcd70101410 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd74a0a640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 0x7fcd70197950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c970 tx=0x7fcd6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60007bb0 con 0x7fcd70101410 2024-09-15T22:11:12.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60007d50 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019c6b0 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000f450 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.129+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7006a790 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.130+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd7006ada0 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.131+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcd60004fa0 con 0x7fcd70101410 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd667fc640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcd5c0a32d0 0x7fcd5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd7520b640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcd5c0a32d0 0x7fcd5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcd600cfef0 con 0x7fcd70101410 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd667fc640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] conn(0x7fcd5c0ae4a0 0x7fcd5c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd5c0b1010 con 0x7fcd5c0ae4a0 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.132+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcd600d0340 con 0x7fcd70101410 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.133+0000 7fcd7520b640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcd5c0a32d0 0x7fcd5c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009900 tx=0x7fcd6c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.133+0000 7fcd75a0c640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] conn(0x7fcd5c0ae4a0 0x7fcd5c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.133+0000 7fcd75a0c640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] conn(0x7fcd5c0ae4a0 0x7fcd5c0b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.134+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== osd.4 v2:172.21.15.117:6808/1062550497 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd5c0b1010 con 0x7fcd5c0ae4a0 2024-09-15T22:11:12.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.147+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 --> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd7006b220 con 0x7fcd5c0ae4a0 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.148+0000 7fcd667fc640 1 -- 172.21.15.77:0/905468068 <== osd.4 v2:172.21.15.117:6808/1062550497 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcd7006b220 con 0x7fcd5c0ae4a0 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.148+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] conn(0x7fcd5c0ae4a0 msgr2=0x7fcd5c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.148+0000 7fcd7746f640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] conn(0x7fcd5c0ae4a0 0x7fcd5c0b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcd5c0a32d0 msgr2=0x7fcd5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fcd5c0a32d0 0x7fcd5c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009900 tx=0x7fcd6c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 msgr2=0x7fcd70197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 --2- 172.21.15.77:0/905468068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd70101410 0x7fcd70197950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c970 tx=0x7fcd6000ce30 comp rx=0 tx=0).stop 2024-09-15T22:11:12.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd75a0c640 1 -- 172.21.15.77:0/905468068 reap_dead start 2024-09-15T22:11:12.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 shutdown_connections 2024-09-15T22:11:12.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 >> 172.21.15.77:0/905468068 conn(0x7fcd700fc7d0 msgr2=0x7fcd700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:12.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 shutdown_connections 2024-09-15T22:11:12.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.149+0000 7fcd7746f640 1 -- 172.21.15.77:0/905468068 wait complete. 2024-09-15T22:11:12.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 -- 172.21.15.77:0/4010228878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c106f10 msgr2=0x7f842c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 --2- 172.21.15.77:0/4010228878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c106f10 0x7f842c10ea90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8424009ca0 tx=0x7f842402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 -- 172.21.15.77:0/4010228878 shutdown_connections 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 --2- 172.21.15.77:0/4010228878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c106f10 0x7f842c10ea90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 --2- 172.21.15.77:0/4010228878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c102790 0x7f842c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 --2- 172.21.15.77:0/4010228878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c101de0 0x7f842c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 -- 172.21.15.77:0/4010228878 >> 172.21.15.77:0/4010228878 conn(0x7f842c0fd8c0 msgr2=0x7f842c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 -- 172.21.15.77:0/4010228878 shutdown_connections 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.567+0000 7f8433d68640 1 -- 172.21.15.77:0/4010228878 wait complete. 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 Processor -- start 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 -- start start 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 0x7f842c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c102790 0x7f842c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c106f10 0x7f842c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f842c111f70 con 0x7f842c101de0 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f842c111df0 con 0x7f842c102790 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8433d68640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1120f0 con 0x7f842c106f10 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 0x7f842c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 0x7f842c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40602/0 (socket says 172.21.15.77:40602) 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 -- 172.21.15.77:0/2416688028 learned_addr learned my addr 172.21.15.77:0/2416688028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:12.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431303640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c102790 0x7f842c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8432305640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c106f10 0x7f842c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 -- 172.21.15.77:0/2416688028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c106f10 msgr2=0x7f842c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f842c106f10 0x7f842c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 -- 172.21.15.77:0/2416688028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c102790 msgr2=0x7f842c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c102790 0x7f842c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.568+0000 7f8431b04640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a6870 con 0x7f842c101de0 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8431b04640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 0x7f842c0744f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f842800d950 tx=0x7f842800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428017070 con 0x7f842c101de0 2024-09-15T22:11:12.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8428009c30 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a6b60 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800f6b0 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.569+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a70f0 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.570+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f842c0628c0 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.571+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f84280109d0 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8422ffd640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f84180a3350 0x7f84180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8431303640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f84180a3350 0x7f84180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f84280cfdd0 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8422ffd640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] conn(0x7f84180ae520 0x7f84180b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f842801ce50 con 0x7f84180ae520 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f84280d0220 con 0x7f842c101de0 2024-09-15T22:11:12.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8432305640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] conn(0x7f84180ae520 0x7f84180b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8431303640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f84180a3350 0x7f84180a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f842c075f80 tx=0x7f841c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.572+0000 7f8432305640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] conn(0x7f84180ae520 0x7f84180b09c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.573+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== osd.3 v2:172.21.15.77:6810/2407517217 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f842801ce50 con 0x7f84180ae520 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.586+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 --> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f842c062cc0 con 0x7f84180ae520 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8422ffd640 1 -- 172.21.15.77:0/2416688028 <== osd.3 v2:172.21.15.77:6810/2407517217 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f842c062cc0 con 0x7f84180ae520 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] conn(0x7f84180ae520 msgr2=0x7f84180b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] conn(0x7f84180ae520 0x7f84180b09c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f84180a3350 msgr2=0x7f84180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f84180a3350 0x7f84180a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f842c075f80 tx=0x7f841c00b390 comp rx=0 tx=0).stop 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 msgr2=0x7f842c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 --2- 172.21.15.77:0/2416688028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f842c101de0 0x7f842c0744f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f842800d950 tx=0x7f842800de10 comp rx=0 tx=0).stop 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8432305640 1 -- 172.21.15.77:0/2416688028 reap_dead start 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 shutdown_connections 2024-09-15T22:11:12.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 >> 172.21.15.77:0/2416688028 conn(0x7f842c0fd8c0 msgr2=0x7f842c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:12.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 shutdown_connections 2024-09-15T22:11:12.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:12.587+0000 7f8433d68640 1 -- 172.21.15.77:0/2416688028 wait complete. 2024-09-15T22:11:13.339 INFO:teuthology.orchestra.run.smithi077.stdout:214748364848 2024-09-15T22:11:13.340 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-15T22:11:13.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:13 smithi077 ceph-mon[30980]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:13 smithi196 ceph-mon[35109]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:13 smithi117 ceph-mon[35021]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 383 B/s wr, 11 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:11:13.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 -- 172.21.15.77:0/1915417417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 msgr2=0x7f6100112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 --2- 172.21.15.77:0/1915417417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f6100112650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009900 tx=0x7f60f002a480 comp rx=0 tx=0).stop 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 -- 172.21.15.77:0/1915417417 shutdown_connections 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 --2- 172.21.15.77:0/1915417417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6100112b90 0x7f6100114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 --2- 172.21.15.77:0/1915417417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f6100112650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 --2- 172.21.15.77:0/1915417417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 0x7f6100074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 -- 172.21.15.77:0/1915417417 >> 172.21.15.77:0/1915417417 conn(0x7f61000fd8c0 msgr2=0x7f61000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.739+0000 7f61074db640 1 -- 172.21.15.77:0/1915417417 shutdown_connections 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 -- 172.21.15.77:0/1915417417 wait complete. 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 Processor -- start 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 -- start start 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 0x7f61001a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f61001a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6100112b90 0x7f610019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6100118560 con 0x7f6100074eb0 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61001183e0 con 0x7f6100112b90 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.740+0000 7f61074db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61001186e0 con 0x7f6100074590 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f61001a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6105277640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 0x7f61001a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f61001a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40620/0 (socket says 172.21.15.77:40620) 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 -- 172.21.15.77:0/2097299935 learned_addr learned my addr 172.21.15.77:0/2097299935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6105a78640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6100112b90 0x7f610019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 -- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 msgr2=0x7f61001a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 0x7f61001a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 -- 172.21.15.77:0/2097299935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6100112b90 msgr2=0x7f610019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6100112b90 0x7f610019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019f000 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6105277640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6100074590 0x7f61001a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f6104a76640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f61001a4c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006010 tx=0x7f60f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f003f030 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019f290 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f0003ce0 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0035070 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.741+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001ab5e0 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.742+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6100062980 con 0x7f6100074eb0 2024-09-15T22:11:13.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.743+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f60f0042030 con 0x7f6100074eb0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f60f67fc640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f60ec0a32d0 0x7f60ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f6105277640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f60ec0a32d0 0x7f60ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f60f00f08b0 con 0x7f6100074eb0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f60f67fc640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] conn(0x7f60ec0ae4a0 0x7f60ec0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60f0001d30 con 0x7f60ec0ae4a0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.744+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f60f00f0d00 con 0x7f6100074eb0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.745+0000 7f6105277640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f60ec0a32d0 0x7f60ec0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f610019fa10 tx=0x7f60fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.745+0000 7f6105a78640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] conn(0x7f60ec0ae4a0 0x7f60ec0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.745+0000 7f6105a78640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] conn(0x7f60ec0ae4a0 0x7f60ec0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.746+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== osd.8 v2:172.21.15.196:6816/3896356979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60f0001d30 con 0x7f60ec0ae4a0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.759+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 --> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6100062d80 con 0x7f60ec0ae4a0 2024-09-15T22:11:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.759+0000 7f60f67fc640 1 -- 172.21.15.77:0/2097299935 <== osd.8 v2:172.21.15.196:6816/3896356979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6100062d80 con 0x7f60ec0ae4a0 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] conn(0x7f60ec0ae4a0 msgr2=0x7f60ec0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] conn(0x7f60ec0ae4a0 0x7f60ec0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f60ec0a32d0 msgr2=0x7f60ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f60ec0a32d0 0x7f60ec0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f610019fa10 tx=0x7f60fc005f90 comp rx=0 tx=0).stop 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 msgr2=0x7f61001a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 --2- 172.21.15.77:0/2097299935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6100074eb0 0x7f61001a4c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006010 tx=0x7f60f00042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f6105a78640 1 -- 172.21.15.77:0/2097299935 reap_dead start 2024-09-15T22:11:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 shutdown_connections 2024-09-15T22:11:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 >> 172.21.15.77:0/2097299935 conn(0x7f61000fd8c0 msgr2=0x7f61000ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 shutdown_connections 2024-09-15T22:11:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:13.760+0000 7f61074db640 1 -- 172.21.15.77:0/2097299935 wait complete. 2024-09-15T22:11:14.529 INFO:teuthology.orchestra.run.smithi077.stdout:137438953535 2024-09-15T22:11:14.529 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-15T22:11:14.984 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:15.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 -- 172.21.15.77:0/754079091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8101420 msgr2=0x7f65b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 --2- 172.21.15.77:0/754079091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8101420 0x7f65b810d950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009930 tx=0x7f65a802a420 comp rx=0 tx=0).stop 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 -- 172.21.15.77:0/754079091 shutdown_connections 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 --2- 172.21.15.77:0/754079091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8101420 0x7f65b810d950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 --2- 172.21.15.77:0/754079091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b8100a80 0x7f65b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 --2- 172.21.15.77:0/754079091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8106a80 0x7f65b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 -- 172.21.15.77:0/754079091 >> 172.21.15.77:0/754079091 conn(0x7f65b80fc7a0 msgr2=0x7f65b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 -- 172.21.15.77:0/754079091 shutdown_connections 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 -- 172.21.15.77:0/754079091 wait complete. 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.193+0000 7f65bd880640 1 Processor -- start 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 -- start start 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8100a80 0x7f65b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:15.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b8101420 0x7f65b81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 0x7f65b819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8110e80 con 0x7f65b8106a80 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8110d00 con 0x7f65b8100a80 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8111000 con 0x7f65b8101420 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 0x7f65b819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 0x7f65b819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40638/0 (socket says 172.21.15.77:40638) 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 -- 172.21.15.77:0/4159929814 learned_addr learned my addr 172.21.15.77:0/4159929814 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b6ffd640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8100a80 0x7f65b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b67fc640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b8101420 0x7f65b81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 -- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b8101420 msgr2=0x7f65b81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65b8101420 0x7f65b81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 -- 172.21.15.77:0/4159929814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8100a80 msgr2=0x7f65b81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8100a80 0x7f65b81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b819d360 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b6ffd640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65b8100a80 0x7f65b81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65b77fe640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 0x7f65b819cc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005ab0 tx=0x7f65a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a803f030 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a80044c0 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.194+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b819d5f0 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.195+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8035070 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.195+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a4f30 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.195+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65b8062980 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f65a8042030 con 0x7f65b8106a80 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f65a00a32d0 0x7f65a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65b6ffd640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f65a00a32d0 0x7f65a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:15.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65a80f06d0 con 0x7f65b8106a80 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] conn(0x7f65a00ae4a0 0x7f65a00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65a00b1010 con 0x7f65a00ae4a0 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.197+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f65a80f0b20 con 0x7f65b8106a80 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.198+0000 7f65b6ffd640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f65a00a32d0 0x7f65a00a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009f40 tx=0x7f65b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.198+0000 7f65b67fc640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] conn(0x7f65a00ae4a0 0x7f65a00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.198+0000 7f65b67fc640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] conn(0x7f65a00ae4a0 0x7f65a00b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:15.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.199+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== osd.5 v2:172.21.15.196:6808/2355584153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65a00b1010 con 0x7f65a00ae4a0 2024-09-15T22:11:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.215+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 --> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65b8062e00 con 0x7f65a00ae4a0 2024-09-15T22:11:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bc87e640 1 -- 172.21.15.77:0/4159929814 <== osd.5 v2:172.21.15.196:6808/2355584153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65b8062e00 con 0x7f65a00ae4a0 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] conn(0x7f65a00ae4a0 msgr2=0x7f65a00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] conn(0x7f65a00ae4a0 0x7f65a00b0940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f65a00a32d0 msgr2=0x7f65a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f65a00a32d0 0x7f65a00a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009f40 tx=0x7f65b0005f90 comp rx=0 tx=0).stop 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 msgr2=0x7f65b819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 --2- 172.21.15.77:0/4159929814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b8106a80 0x7f65b819cc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005ab0 tx=0x7f65a80040c0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65b77fe640 1 -- 172.21.15.77:0/4159929814 reap_dead start 2024-09-15T22:11:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 shutdown_connections 2024-09-15T22:11:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.216+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 >> 172.21.15.77:0/4159929814 conn(0x7f65b80fc7a0 msgr2=0x7f65b806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.217+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 shutdown_connections 2024-09-15T22:11:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:15.217+0000 7f65bd880640 1 -- 172.21.15.77:0/4159929814 wait complete. 2024-09-15T22:11:15.576 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:15 smithi077 ceph-mon[30980]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-09-15T22:11:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:15 smithi196 ceph-mon[35109]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-09-15T22:11:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:15 smithi117 ceph-mon[35021]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 B/s wr, 10 op/s; 11 B/s, 1 objects/s recovering 2024-09-15T22:11:15.906 INFO:teuthology.orchestra.run.smithi077.stdout:103079215173 2024-09-15T22:11:15.906 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-15T22:11:16.365 INFO:teuthology.orchestra.run.smithi077.stdout:244813135916 2024-09-15T22:11:16.366 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-15T22:11:16.862 INFO:teuthology.orchestra.run.smithi077.stdout:158913790011 2024-09-15T22:11:16.863 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- 172.21.15.77:0/2236190968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 msgr2=0x7f231410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 --2- 172.21.15.77:0/2236190968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231410ec40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f230c009cd0 tx=0x7f230c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- 172.21.15.77:0/2236190968 shutdown_connections 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 --2- 172.21.15.77:0/2236190968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231410ec40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 --2- 172.21.15.77:0/2236190968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 0x7f2314107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 --2- 172.21.15.77:0/2236190968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 0x7f2314103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- 172.21.15.77:0/2236190968 >> 172.21.15.77:0/2236190968 conn(0x7f23140fc7d0 msgr2=0x7f23140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:17.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- 172.21.15.77:0/2236190968 shutdown_connections 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- 172.21.15.77:0/2236190968 wait complete. 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 Processor -- start 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.038+0000 7f231bdc7640 1 -- start start 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 0x7f2314198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 0x7f2314198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23141120a0 con 0x7f23141082a0 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2314111f20 con 0x7f23141036e0 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231bdc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2314112220 con 0x7f2314102d30 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231a364640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231a364640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40660/0 (socket says 172.21.15.77:40660) 2024-09-15T22:11:17.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 0x7f2314198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231a364640 1 -- 172.21.15.77:0/1285155742 learned_addr learned my addr 172.21.15.77:0/1285155742 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319b63640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 0x7f2314198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 -- 172.21.15.77:0/1285155742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 msgr2=0x7f2314198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 0x7f2314198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 -- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 msgr2=0x7f231419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231419d520 con 0x7f23141036e0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f231a364640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23141082a0 0x7f231419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319b63640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2314102d30 0x7f2314198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.039+0000 7f2319362640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 0x7f2314198ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23140697d0 tx=0x7f230400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230400cc40 con 0x7f23141036e0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231419d810 con 0x7f23141036e0 2024-09-15T22:11:17.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230400cde0 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040188e0 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23141a50f0 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.040+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2314062980 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.042+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2304018a80 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f230affd640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f23000a32d0 0x7f23000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f23040d0590 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f2319b63640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f23000a32d0 0x7f23000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f230affd640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] conn(0x7f23000ae4a0 0x7f23000b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23000b1010 con 0x7f23000ae4a0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f23040d0980 con 0x7f23141036e0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f2319b63640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f23000a32d0 0x7f23000a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2310009c20 tx=0x7f2310005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.043+0000 7f231a364640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] conn(0x7f23000ae4a0 0x7f23000b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.044+0000 7f231a364640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] conn(0x7f23000ae4a0 0x7f23000b0940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:17.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.045+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== osd.10 v2:172.21.15.117:6824/2316398411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23000b1010 con 0x7f23000ae4a0 2024-09-15T22:11:17.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.060+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 --> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2314062d80 con 0x7f23000ae4a0 2024-09-15T22:11:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f230affd640 1 -- 172.21.15.77:0/1285155742 <== osd.10 v2:172.21.15.117:6824/2316398411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2314062d80 con 0x7f23000ae4a0 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] conn(0x7f23000ae4a0 msgr2=0x7f23000b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] conn(0x7f23000ae4a0 0x7f23000b0940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f23000a32d0 msgr2=0x7f23000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f23000a32d0 0x7f23000a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2310009c20 tx=0x7f2310005f90 comp rx=0 tx=0).stop 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 msgr2=0x7f2314198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 --2- 172.21.15.77:0/1285155742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23141036e0 0x7f2314198ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23140697d0 tx=0x7f230400ee50 comp rx=0 tx=0).stop 2024-09-15T22:11:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231a364640 1 -- 172.21.15.77:0/1285155742 reap_dead start 2024-09-15T22:11:17.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 shutdown_connections 2024-09-15T22:11:17.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.061+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 >> 172.21.15.77:0/1285155742 conn(0x7f23140fc7d0 msgr2=0x7f2314105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:17.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.062+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 shutdown_connections 2024-09-15T22:11:17.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:17.062+0000 7f231bdc7640 1 -- 172.21.15.77:0/1285155742 wait complete. 2024-09-15T22:11:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:17.310 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:17 smithi077 ceph-mon[30980]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-09-15T22:11:17.565 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:17 smithi196 ceph-mon[35109]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-09-15T22:11:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:17 smithi117 ceph-mon[35021]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-09-15T22:11:18.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 -- 172.21.15.77:0/1851310583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044106a80 msgr2=0x7f8044106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 --2- 172.21.15.77:0/1851310583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044106a80 0x7f8044106e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8038009930 tx=0x7f803802a420 comp rx=0 tx=0).stop 2024-09-15T22:11:18.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 -- 172.21.15.77:0/1851310583 shutdown_connections 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 --2- 172.21.15.77:0/1851310583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044101420 0x7f804410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 --2- 172.21.15.77:0/1851310583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100a80 0x7f8044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 --2- 172.21.15.77:0/1851310583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044106a80 0x7f8044106e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 -- 172.21.15.77:0/1851310583 >> 172.21.15.77:0/1851310583 conn(0x7f80440fc7a0 msgr2=0x7f80440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 -- 172.21.15.77:0/1851310583 shutdown_connections 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.041+0000 7f804bbb9640 1 -- 172.21.15.77:0/1851310583 wait complete. 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 Processor -- start 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 -- start start 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100a80 0x7f8044198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:18.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 0x7f8044198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044106a80 0x7f804419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8044110e40 con 0x7f8044101420 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8044110cc0 con 0x7f8044100a80 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804bbb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8044110fc0 con 0x7f8044106a80 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 0x7f8044198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 0x7f8044198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40692/0 (socket says 172.21.15.77:40692) 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 -- 172.21.15.77:0/2757312832 learned_addr learned my addr 172.21.15.77:0/2757312832 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:18.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049955640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100a80 0x7f8044198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:18.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804a156640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044106a80 0x7f804419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 -- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044106a80 msgr2=0x7f804419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044106a80 0x7f804419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 -- 172.21.15.77:0/2757312832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100a80 msgr2=0x7f8044198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100a80 0x7f8044198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804419d400 con 0x7f8044101420 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f804a156640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8044106a80 0x7f804419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.042+0000 7f8049154640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 0x7f8044198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f804000c890 tx=0x7f804000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.043+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040004220 con 0x7f8044101420 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.043+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80400043c0 con 0x7f8044101420 2024-09-15T22:11:18.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.043+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804419d6f0 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.043+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80400049e0 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.043+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441a4fd0 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.044+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8044062940 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.045+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f80400079b0 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.045+0000 7f8036ffd640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80300a32d0 0x7f80300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8049955640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80300a32d0 0x7f80300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f80400d0040 con 0x7f8044101420 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8036ffd640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] conn(0x7f80300ae4a0 0x7f80300b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80300b1010 con 0x7f80300ae4a0 2024-09-15T22:11:18.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f80400d0490 con 0x7f8044101420 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f8049955640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80300a32d0 0x7f80300a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8038009900 tx=0x7f80380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.046+0000 7f804a156640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] conn(0x7f80300ae4a0 0x7f80300b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.047+0000 7f804a156640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] conn(0x7f80300ae4a0 0x7f80300b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.047+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== osd.11 v2:172.21.15.196:6824/2953911305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80300b1010 con 0x7f80300ae4a0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.061+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 --> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8044062dc0 con 0x7f80300ae4a0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.061+0000 7f8036ffd640 1 -- 172.21.15.77:0/2757312832 <== osd.11 v2:172.21.15.196:6824/2953911305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8044062dc0 con 0x7f80300ae4a0 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] conn(0x7f80300ae4a0 msgr2=0x7f80300b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] conn(0x7f80300ae4a0 0x7f80300b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80300a32d0 msgr2=0x7f80300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f80300a32d0 0x7f80300a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8038009900 tx=0x7f80380058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:18.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 msgr2=0x7f8044198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 --2- 172.21.15.77:0/2757312832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8044101420 0x7f8044198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f804000c890 tx=0x7f804000cd50 comp rx=0 tx=0).stop 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804a156640 1 -- 172.21.15.77:0/2757312832 reap_dead start 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 shutdown_connections 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 >> 172.21.15.77:0/2757312832 conn(0x7f80440fc7a0 msgr2=0x7f804406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 shutdown_connections 2024-09-15T22:11:18.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:18.062+0000 7f804bbb9640 1 -- 172.21.15.77:0/2757312832 wait complete. 2024-09-15T22:11:18.301 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:18.301 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:18.301 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:18.999 INFO:teuthology.orchestra.run.smithi077.stdout:326417514526 2024-09-15T22:11:19.000 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-15T22:11:19.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 -- 172.21.15.77:0/3399618217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 msgr2=0x7fe68010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 --2- 172.21.15.77:0/3399618217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe68010bb40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe670009900 tx=0x7fe67002a480 comp rx=0 tx=0).stop 2024-09-15T22:11:19.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 -- 172.21.15.77:0/3399618217 shutdown_connections 2024-09-15T22:11:19.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 --2- 172.21.15.77:0/3399618217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe68010c160 0x7fe68010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 --2- 172.21.15.77:0/3399618217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe68010bb40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 --2- 172.21.15.77:0/3399618217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe680102d30 0x7fe680103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.160+0000 7fe68740a640 1 -- 172.21.15.77:0/3399618217 >> 172.21.15.77:0/3399618217 conn(0x7fe6800fc790 msgr2=0x7fe6800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- 172.21.15.77:0/3399618217 shutdown_connections 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- 172.21.15.77:0/3399618217 wait complete. 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 Processor -- start 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- start start 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe680102d30 0x7fe6801a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe68010c160 0x7fe68019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe680111fa0 con 0x7fe680103650 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe680111e20 con 0x7fe680102d30 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.161+0000 7fe68740a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe680112120 con 0x7fe68010c160 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44496/0 (socket says 172.21.15.77:44496) 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 -- 172.21.15.77:0/4290250904 learned_addr learned my addr 172.21.15.77:0/4290250904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6851a6640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe680102d30 0x7fe6801a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6859a7640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe68010c160 0x7fe68019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 -- 172.21.15.77:0/4290250904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe68010c160 msgr2=0x7fe68019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe68010c160 0x7fe68019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 -- 172.21.15.77:0/4290250904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe680102d30 msgr2=0x7fe6801a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe680102d30 0x7fe6801a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 -- 172.21.15.77:0/4290250904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68019acc0 con 0x7fe680103650 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6849a5640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe6700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67003f030 con 0x7fe680103650 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670003ce0 con 0x7fe680103650 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68019af50 con 0x7fe680103650 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670035070 con 0x7fe680103650 2024-09-15T22:11:19.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.162+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a72a0 con 0x7fe680103650 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.164+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe680103110 con 0x7fe680103650 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.166+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe670042030 con 0x7fe680103650 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.166+0000 7fe6767fc640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 0x7fe66c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.167+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe6700f0780 con 0x7fe680103650 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.167+0000 7fe6851a6640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 0x7fe66c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.167+0000 7fe6851a6640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 0x7fe66c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009990 tx=0x7fe67c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:19.171 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:19.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.170+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6700bcfe0 con 0x7fe680103650 2024-09-15T22:11:19.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.346+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe6801a7470 con 0x7fe680103650 2024-09-15T22:11:19.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.346+0000 7fe6767fc640 1 -- 172.21.15.77:0/4290250904 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6700bc8e0 con 0x7fe680103650 2024-09-15T22:11:19.349 INFO:teuthology.orchestra.run.smithi077.stdout:34359738452 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 msgr2=0x7fe66c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 0x7fe66c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009990 tx=0x7fe67c009110 comp rx=0 tx=0).stop 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 msgr2=0x7fe6801a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe6700042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 shutdown_connections 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fe66c0a32d0 0x7fe66c0a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe68010c160 0x7fe68019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe680103650 0x7fe6801a0910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 --2- 172.21.15.77:0/4290250904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe680102d30 0x7fe6801a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 >> 172.21.15.77:0/4290250904 conn(0x7fe6800fc790 msgr2=0x7fe6800fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 shutdown_connections 2024-09-15T22:11:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.348+0000 7fe68740a640 1 -- 172.21.15.77:0/4290250904 wait complete. 2024-09-15T22:11:19.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 -- 172.21.15.77:0/3149071038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 msgr2=0x7f56b4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 --2- 172.21.15.77:0/3149071038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b4106f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009900 tx=0x7f56a402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:19.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 -- 172.21.15.77:0/3149071038 shutdown_connections 2024-09-15T22:11:19.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 --2- 172.21.15.77:0/3149071038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 0x7f56b410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 --2- 172.21.15.77:0/3149071038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 --2- 172.21.15.77:0/3149071038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b4106f60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 -- 172.21.15.77:0/3149071038 >> 172.21.15.77:0/3149071038 conn(0x7f56b4077230 msgr2=0x7f56b40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.372+0000 7f56b9058640 1 -- 172.21.15.77:0/3149071038 shutdown_connections 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.373+0000 7f56b9058640 1 -- 172.21.15.77:0/3149071038 wait complete. 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.373+0000 7f56b9058640 1 Processor -- start 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.373+0000 7f56b9058640 1 -- start start 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 0x7f56b4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4112050 con 0x7f56b4106b80 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4111ed0 con 0x7f56b4100aa0 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b9058640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41121d0 con 0x7f56b4100100 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44524/0 (socket says 172.21.15.77:44524) 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 -- 172.21.15.77:0/1513037984 learned_addr learned my addr 172.21.15.77:0/1513037984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b2d9d640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56aa59c640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 0x7f56b4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 -- 172.21.15.77:0/1513037984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 msgr2=0x7f56b4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 -- 172.21.15.77:0/1513037984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 msgr2=0x7f56b4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 0x7f56b4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b359e640 1 -- 172.21.15.77:0/1513037984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b419d520 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.374+0000 7f56b2d9d640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b359e640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00e9d0 tx=0x7f56ac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00cd80 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac005020 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0116d0 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b419d810 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.375+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41a50f0 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.376+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b4100560 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.377+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f56ac0051d0 con 0x7f56b4106b80 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.378+0000 7f56b0d99640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.378+0000 7f56b2d9d640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:19.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.378+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f56ac0d0340 con 0x7f56b4106b80 2024-09-15T22:11:19.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.378+0000 7f56b2d9d640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009230 tx=0x7f56a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:19.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.382+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56ac016030 con 0x7f56b4106b80 2024-09-15T22:11:19.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.555+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f56b410a0c0 con 0x7f56b4106b80 2024-09-15T22:11:19.558 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:19 smithi077 ceph-mon[30980]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:19.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.555+0000 7f56b0d99640 1 -- 172.21.15.77:0/1513037984 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56ac09c4f0 con 0x7f56b4106b80 2024-09-15T22:11:19.559 INFO:teuthology.orchestra.run.smithi077.stdout:77309411404 2024-09-15T22:11:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 msgr2=0x7f56a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009230 tx=0x7f56a40058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 msgr2=0x7f56b419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00e9d0 tx=0x7f56ac00ee90 comp rx=0 tx=0).stop 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 shutdown_connections 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f56a00a32d0 0x7f56a00a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56b4106b80 0x7f56b419ce40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b4100aa0 0x7f56b4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 --2- 172.21.15.77:0/1513037984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56b4100100 0x7f56b4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 >> 172.21.15.77:0/1513037984 conn(0x7f56b4077230 msgr2=0x7f56b4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 shutdown_connections 2024-09-15T22:11:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:19.558+0000 7f56b9058640 1 -- 172.21.15.77:0/1513037984 wait complete. 2024-09-15T22:11:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:19 smithi196 ceph-mon[35109]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:19 smithi117 ceph-mon[35021]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:20.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738452 for osd.0 2024-09-15T22:11:20.103 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:20.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:20 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4290250904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:11:20.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:20 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1513037984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:11:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:20 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4290250904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:11:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:20 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1513037984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:11:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:20 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4290250904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:11:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:20 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1513037984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:11:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:21 smithi077 ceph-mon[30980]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:21.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:21 smithi196 ceph-mon[35109]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:21 smithi117 ceph-mon[35021]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:11:21.634 INFO:teuthology.orchestra.run.smithi077.stdout:296352743458 2024-09-15T22:11:21.635 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-15T22:11:21.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411404 for osd.2 2024-09-15T22:11:21.758 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 -- 172.21.15.77:0/1367897144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010100a60 msgr2=0x7f2010100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 --2- 172.21.15.77:0/1367897144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010100a60 0x7f2010100ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2000009cd0 tx=0x7f200002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 -- 172.21.15.77:0/1367897144 shutdown_connections 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 --2- 172.21.15.77:0/1367897144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010101400 0x7f201010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 --2- 172.21.15.77:0/1367897144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010100a60 0x7f2010100ec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 --2- 172.21.15.77:0/1367897144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f2010106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 -- 172.21.15.77:0/1367897144 >> 172.21.15.77:0/1367897144 conn(0x7f20100fc780 msgr2=0x7f20100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 -- 172.21.15.77:0/1367897144 shutdown_connections 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.228+0000 7f20171ef640 1 -- 172.21.15.77:0/1367897144 wait complete. 2024-09-15T22:11:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 Processor -- start 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 -- start start 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 0x7f201010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f201010ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2010112000 con 0x7f2010101400 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111e80 con 0x7f2010106a60 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f20171ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2010112180 con 0x7f2010100a60 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f200ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f200ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44546/0 (socket says 172.21.15.77:44546) 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f200ffff640 1 -- 172.21.15.77:0/2459306555 learned_addr learned my addr 172.21.15.77:0/2459306555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.229+0000 7f201578c640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f201010ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f2014f8b640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 0x7f201010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 -- 172.21.15.77:0/2459306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 msgr2=0x7f201010e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 0x7f201010e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 -- 172.21.15.77:0/2459306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 msgr2=0x7f201010ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f201010ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 -- 172.21.15.77:0/2459306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101a6a80 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f201578c640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f201010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f2014f8b640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 0x7f201010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200ffff640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f200002ae00 tx=0x7f2000003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004240 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20000305a0 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003a660 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101a6d10 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.230+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a7240 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.231+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010100f40 con 0x7f2010101400 2024-09-15T22:11:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.232+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2000030ac0 con 0x7f2010101400 2024-09-15T22:11:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.233+0000 7f200dffb640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 0x7f1ffc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.233+0000 7f2014f8b640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 0x7f1ffc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.233+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f200002e030 con 0x7f2010101400 2024-09-15T22:11:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.234+0000 7f2014f8b640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 0x7f1ffc0a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2008006fd0 tx=0x7f2008008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.238+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20000098b0 con 0x7f2010101400 2024-09-15T22:11:22.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.413+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f2010108a60 con 0x7f2010101400 2024-09-15T22:11:22.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.413+0000 7f200dffb640 1 -- 172.21.15.77:0/2459306555 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20000bd980 con 0x7f2010101400 2024-09-15T22:11:22.416 INFO:teuthology.orchestra.run.smithi077.stdout:274877906985 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.415+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 msgr2=0x7f1ffc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.415+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 0x7f1ffc0a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2008006fd0 tx=0x7f2008008040 comp rx=0 tx=0).stop 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.415+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 msgr2=0x7f201010e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.415+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f200002ae00 tx=0x7f2000003ba0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 shutdown_connections 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f1ffc0a3350 0x7f1ffc0a5810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2010106a60 0x7f201010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2010101400 0x7f201010e900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 --2- 172.21.15.77:0/2459306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2010100a60 0x7f201010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 >> 172.21.15.77:0/2459306555 conn(0x7f20100fc780 msgr2=0x7f20100fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:22.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 shutdown_connections 2024-09-15T22:11:22.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.416+0000 7f20171ef640 1 -- 172.21.15.77:0/2459306555 wait complete. 2024-09-15T22:11:22.484 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:22.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 -- 172.21.15.77:0/912594911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 msgr2=0x7fd5bc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/912594911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc103180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009900 tx=0x7fd5ac02a480 comp rx=0 tx=0).stop 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 -- 172.21.15.77:0/912594911 shutdown_connections 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/912594911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5bc10c180 0x7fd5bc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/912594911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5bc1036c0 0x7fd5bc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/912594911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc103180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 -- 172.21.15.77:0/912594911 >> 172.21.15.77:0/912594911 conn(0x7fd5bc0fc7a0 msgr2=0x7fd5bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 -- 172.21.15.77:0/912594911 shutdown_connections 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.945+0000 7fd5c15dc640 1 -- 172.21.15.77:0/912594911 wait complete. 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 Processor -- start 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 -- start start 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5bc1036c0 0x7fd5bc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5bc10c180 0x7fd5bc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc112020 con 0x7fd5bc102da0 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc111ea0 con 0x7fd5bc1036c0 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5c15dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1121a0 con 0x7fd5bc10c180 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44562/0 (socket says 172.21.15.77:44562) 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 -- 172.21.15.77:0/389184515 learned_addr learned my addr 172.21.15.77:0/389184515 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 -- 172.21.15.77:0/389184515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5bc10c180 msgr2=0x7fd5bc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5bc10c180 0x7fd5bc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 -- 172.21.15.77:0/389184515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5bc1036c0 msgr2=0x7fd5bc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5bc1036c0 0x7fd5bc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.946+0000 7fd5baffd640 1 -- 172.21.15.77:0/389184515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc19d530 con 0x7fd5bc102da0 2024-09-15T22:11:22.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5baffd640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005ab0 tx=0x7fd5ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac03f030 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac003ce0 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc19d7c0 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac035070 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.947+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1a5100 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.948+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc06b010 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.949+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd5ac042030 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.950+0000 7fd5b8ff9640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.950+0000 7fd5b27fc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.950+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd5ac0f09f0 con 0x7fd5bc102da0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.950+0000 7fd5b27fc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 0x7fd5a80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0fef00 tx=0x7fd5a4006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:22.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:22.953+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5ac0bd250 con 0x7fd5bc102da0 2024-09-15T22:11:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.128+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd5bc103200 con 0x7fd5bc102da0 2024-09-15T22:11:23.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.128+0000 7fd5b8ff9640 1 -- 172.21.15.77:0/389184515 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5bc103200 con 0x7fd5bc102da0 2024-09-15T22:11:23.131 INFO:teuthology.orchestra.run.smithi077.stdout:55834574929 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 msgr2=0x7fd5a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 0x7fd5a80a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0fef00 tx=0x7fd5a4006c90 comp rx=0 tx=0).stop 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 msgr2=0x7fd5bc198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005ab0 tx=0x7fd5ac0042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 shutdown_connections 2024-09-15T22:11:23.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd5a80a32d0 0x7fd5a80a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5bc10c180 0x7fd5bc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5bc1036c0 0x7fd5bc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 --2- 172.21.15.77:0/389184515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5bc102da0 0x7fd5bc198580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.130+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 >> 172.21.15.77:0/389184515 conn(0x7fd5bc0fc7a0 msgr2=0x7fd5bc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.131+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 shutdown_connections 2024-09-15T22:11:23.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.131+0000 7fd5c15dc640 1 -- 172.21.15.77:0/389184515 wait complete. 2024-09-15T22:11:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 -- 172.21.15.77:0/966377883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2814106f40 msgr2=0x7f281410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 --2- 172.21.15.77:0/966377883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2814106f40 0x7f281410eac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c02a410 comp rx=0 tx=0).stop 2024-09-15T22:11:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 -- 172.21.15.77:0/966377883 shutdown_connections 2024-09-15T22:11:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 --2- 172.21.15.77:0/966377883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2814106f40 0x7f281410eac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 --2- 172.21.15.77:0/966377883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28141027c0 0x7f2814106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 --2- 172.21.15.77:0/966377883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814101e10 0x7f28141021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 -- 172.21.15.77:0/966377883 >> 172.21.15.77:0/966377883 conn(0x7f28140fd8d0 msgr2=0x7f28140ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 -- 172.21.15.77:0/966377883 shutdown_connections 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.181+0000 7f2819525640 1 -- 172.21.15.77:0/966377883 wait complete. 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 Processor -- start 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 -- start start 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2814101e10 0x7f28140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 0x7f2814074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2814111fc0 con 0x7f28141027c0 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2814111e40 con 0x7f2814106f40 2024-09-15T22:11:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f2819525640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2814112140 con 0x7f2814101e10 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.182+0000 7f28127fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44586/0 (socket says 172.21.15.77:44586) 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 -- 172.21.15.77:0/3057649028 learned_addr learned my addr 172.21.15.77:0/3057649028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28137fe640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 0x7f2814074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f2812ffd640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2814101e10 0x7f28140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 -- 172.21.15.77:0/3057649028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2814101e10 msgr2=0x7f28140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2814101e10 0x7f28140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 -- 172.21.15.77:0/3057649028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 msgr2=0x7f2814074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 0x7f2814074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 -- 172.21.15.77:0/3057649028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141a6910 con 0x7f28141027c0 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28137fe640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 0x7f2814074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f28127fc640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b4a0 tx=0x7f27fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc014070 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc010af0 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a6c00 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc017d10 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.183+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a70f0 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.184+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28141021f0 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.186+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f27fc017070 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.186+0000 7f280bfff640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 0x7f28000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.186+0000 7f2812ffd640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 0x7f28000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.186+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f27fc0cf510 con 0x7f28141027c0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.187+0000 7f2812ffd640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 0x7f28000a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2804005c40 tx=0x7f280400a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.189+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27fc09be80 con 0x7f28141027c0 2024-09-15T22:11:23.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.363+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f2814075a20 con 0x7f28141027c0 2024-09-15T22:11:23.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:23 smithi077 ceph-mon[30980]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:23.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:23 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2459306555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:11:23.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:23 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/389184515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:11:23.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.363+0000 7f280bfff640 1 -- 172.21.15.77:0/3057649028 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2814075a20 con 0x7f28141027c0 2024-09-15T22:11:23.367 INFO:teuthology.orchestra.run.smithi077.stdout:244813135918 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 msgr2=0x7f28000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 0x7f28000a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2804005c40 tx=0x7f280400a780 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 msgr2=0x7f2814074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b4a0 tx=0x7f27fc00b960 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 shutdown_connections 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f28000a32d0 0x7f28000a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2814106f40 0x7f2814074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28141027c0 0x7f2814074a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 --2- 172.21.15.77:0/3057649028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2814101e10 0x7f28140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 >> 172.21.15.77:0/3057649028 conn(0x7f28140fd8d0 msgr2=0x7f28140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 shutdown_connections 2024-09-15T22:11:23.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:23.365+0000 7f2819525640 1 -- 172.21.15.77:0/3057649028 wait complete. 2024-09-15T22:11:23.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:23 smithi196 ceph-mon[35109]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:23.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:23 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2459306555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:11:23.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:23 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/389184515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:11:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:23 smithi117 ceph-mon[35021]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:23 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2459306555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:11:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:23 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/389184515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:11:23.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574929 for osd.1 2024-09-15T22:11:23.884 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:24.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3057649028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:11:24.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906982 got 274877906985 for osd.9 2024-09-15T22:11:24.507 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:24.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3057649028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:11:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3057649028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:11:25.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.269+0000 7f58c826f640 1 -- 172.21.15.77:0/3461806839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 msgr2=0x7f58c01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.269+0000 7f58c826f640 1 --2- 172.21.15.77:0/3461806839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c01067d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009900 tx=0x7f58b002a410 comp rx=0 tx=0).stop 2024-09-15T22:11:25.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- 172.21.15.77:0/3461806839 shutdown_connections 2024-09-15T22:11:25.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- 172.21.15.77:0/3461806839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 0x7f58c010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- 172.21.15.77:0/3461806839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c01067d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- 172.21.15.77:0/3461806839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 0x7f58c01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- 172.21.15.77:0/3461806839 >> 172.21.15.77:0/3461806839 conn(0x7f58c00fd890 msgr2=0x7f58c00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- 172.21.15.77:0/3461806839 shutdown_connections 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- 172.21.15.77:0/3461806839 wait complete. 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 Processor -- start 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- start start 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 0x7f58c01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 0x7f58c019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0111fa0 con 0x7f58c0102780 2024-09-15T22:11:25.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0111e20 con 0x7f58c0106f00 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.270+0000 7f58c826f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0112120 con 0x7f58c0101dd0 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44628/0 (socket says 172.21.15.77:44628) 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 -- 172.21.15.77:0/1357097103 learned_addr learned my addr 172.21.15.77:0/1357097103 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c600b640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 0x7f58c01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c680c640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 0x7f58c019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 -- 172.21.15.77:0/1357097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 msgr2=0x7f58c01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 0x7f58c01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 -- 172.21.15.77:0/1357097103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 msgr2=0x7f58c019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 0x7f58c019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 -- 172.21.15.77:0/1357097103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c019d550 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c580a640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58b0006010 tx=0x7f58b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b003f030 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b0003ce0 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c019d7e0 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0035070 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.271+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01a5120 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.272+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c01032b0 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.273+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f58b0042030 con 0x7f58c0102780 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.274+0000 7f58b77fe640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 0x7f58ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.274+0000 7f58c600b640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 0x7f58ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.274+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f58b00f0760 con 0x7f58c0102780 2024-09-15T22:11:25.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.275+0000 7f58c600b640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 0x7f58ac0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005fd0 tx=0x7f58bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.277+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58b00bd080 con 0x7f58c0102780 2024-09-15T22:11:25.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.449+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f58c010a140 con 0x7f58c0102780 2024-09-15T22:11:25.452 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:25 smithi077 ceph-mon[30980]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-15T22:11:25.452 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3562699310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:11:25.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.449+0000 7f58b77fe640 1 -- 172.21.15.77:0/1357097103 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58b00bc980 con 0x7f58c0102780 2024-09-15T22:11:25.452 INFO:teuthology.orchestra.run.smithi077.stdout:158913790013 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 msgr2=0x7f58ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 0x7f58ac0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005fd0 tx=0x7f58bc00b390 comp rx=0 tx=0).stop 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 msgr2=0x7f58c0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58b0006010 tx=0x7f58b00042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 shutdown_connections 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f58ac0a32d0 0x7f58ac0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c0106f00 0x7f58c019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58c0102780 0x7f58c0198ae0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 --2- 172.21.15.77:0/1357097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0101dd0 0x7f58c01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.451+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 >> 172.21.15.77:0/1357097103 conn(0x7f58c00fd890 msgr2=0x7f58c00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.452+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 shutdown_connections 2024-09-15T22:11:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.452+0000 7f58c826f640 1 -- 172.21.15.77:0/1357097103 wait complete. 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 -- 172.21.15.77:0/4233565904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 msgr2=0x7f183410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 --2- 172.21.15.77:0/4233565904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f183410bb50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1828009900 tx=0x7f182802a480 comp rx=0 tx=0).stop 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 -- 172.21.15.77:0/4233565904 shutdown_connections 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 --2- 172.21.15.77:0/4233565904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f183410c170 0x7f183410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 --2- 172.21.15.77:0/4233565904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f183410bb50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 --2- 172.21.15.77:0/4233565904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1834102d40 0x7f1834103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.676+0000 7f1839ede640 1 -- 172.21.15.77:0/4233565904 >> 172.21.15.77:0/4233565904 conn(0x7f18340fc7a0 msgr2=0x7f18340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 -- 172.21.15.77:0/4233565904 shutdown_connections 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 -- 172.21.15.77:0/4233565904 wait complete. 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 Processor -- start 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 -- start start 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1834102d40 0x7f18341983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.677+0000 7f1839ede640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 0x7f183419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1834111f70 con 0x7f1834103660 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1834111df0 con 0x7f183410c170 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18341120f0 con 0x7f1834102d40 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44618/0 (socket says 172.21.15.77:44618) 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 -- 172.21.15.77:0/3562699310 learned_addr learned my addr 172.21.15.77:0/3562699310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:25.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1833fff640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 0x7f183419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 -- 172.21.15.77:0/3562699310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1834102d40 msgr2=0x7f18341983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1834102d40 0x7f18341983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:137438953538 2024-09-15T22:11:25.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 -- 172.21.15.77:0/3562699310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 msgr2=0x7f183419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 0x7f183419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 -- 172.21.15.77:0/3562699310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183419d380 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1833fff640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 0x7f183419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1832ffd640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1828005ab0 tx=0x7f18280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182803f030 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183419d610 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1828003ce0 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828035070 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.678+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18341a4f50 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.679+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183406b070 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.680+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1828042030 con 0x7f1834103660 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.681+0000 7f1830ff9640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 0x7f181c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.682+0000 7f18337fe640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 0x7f181c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.682+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18280f09e0 con 0x7f1834103660 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.682+0000 7f18337fe640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 0x7f181c0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1824005fd0 tx=0x7f18240071c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.685+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18280bd1e0 con 0x7f1834103660 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.862+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f18341071e0 con 0x7f1834103660 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.863+0000 7f1830ff9640 1 -- 172.21.15.77:0/3562699310 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f18280bcae0 con 0x7f1834103660 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 msgr2=0x7f181c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 0x7f181c0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1824005fd0 tx=0x7f18240071c0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 msgr2=0x7f1834198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1828005ab0 tx=0x7f18280042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 shutdown_connections 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f181c0a32d0 0x7f181c0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f183410c170 0x7f183419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1834103660 0x7f1834198910 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 --2- 172.21.15.77:0/3562699310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1834102d40 0x7f18341983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 >> 172.21.15.77:0/3562699310 conn(0x7f18340fc7a0 msgr2=0x7f18340fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 shutdown_connections 2024-09-15T22:11:25.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:24.865+0000 7f1839ede640 1 -- 172.21.15.77:0/3562699310 wait complete. 2024-09-15T22:11:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:25 smithi196 ceph-mon[35109]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-15T22:11:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3562699310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:11:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:25 smithi117 ceph-mon[35021]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-15T22:11:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3562699310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 -- 172.21.15.77:0/4175314463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8103780 msgr2=0x7fd1b8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 --2- 172.21.15.77:0/4175314463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8103780 0x7fd1b8107bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009930 tx=0x7fd1ac02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 -- 172.21.15.77:0/4175314463 shutdown_connections 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 --2- 172.21.15.77:0/4175314463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8108300 0x7fd1b810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 --2- 172.21.15.77:0/4175314463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8103780 0x7fd1b8107bc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 --2- 172.21.15.77:0/4175314463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8102dd0 0x7fd1b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 -- 172.21.15.77:0/4175314463 >> 172.21.15.77:0/4175314463 conn(0x7fd1b80fc7d0 msgr2=0x7fd1b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 -- 172.21.15.77:0/4175314463 shutdown_connections 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 -- 172.21.15.77:0/4175314463 wait complete. 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.510+0000 7fd1be70d640 1 Processor -- start 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 -- start start 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8103780 0x7fd1b8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 0x7fd1b819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8112050 con 0x7fd1b8102dd0 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8111ed0 con 0x7fd1b8108300 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1be70d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81121d0 con 0x7fd1b8103780 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44650/0 (socket says 172.21.15.77:44650) 2024-09-15T22:11:25.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3847130818 learned_addr learned my addr 172.21.15.77:0/3847130818 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b77fe640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8103780 0x7fd1b8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1bccaa640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 0x7fd1b819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3847130818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8103780 msgr2=0x7fd1b8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8103780 0x7fd1b8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3847130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 msgr2=0x7fd1b819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 0x7fd1b819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3847130818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b819d540 con 0x7fd1b8102dd0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.511+0000 7fd1bccaa640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 0x7fd1b819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c920 tx=0x7fd1a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8007c70 con 0x7fd1b8102dd0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8007e10 con 0x7fd1b8102dd0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b819d830 con 0x7fd1b8102dd0 2024-09-15T22:11:25.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8018890 con 0x7fd1b8102dd0 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.512+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b81a5110 con 0x7fd1b8102dd0 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.513+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b81031b0 con 0x7fd1b8102dd0 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.514+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd1a8018a50 con 0x7fd1b8102dd0 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.515+0000 7fd1b57fa640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 0x7fd1a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.515+0000 7fd1b77fe640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 0x7fd1a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:25.858 INFO:teuthology.orchestra.run.smithi077.stdout:214748364851 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.515+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd1a80d02a0 con 0x7fd1b8102dd0 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.515+0000 7fd1b77fe640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 0x7fd1a00a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.518+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1a8014030 con 0x7fd1b8102dd0 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.688+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd1b81998f0 con 0x7fd1b8102dd0 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.688+0000 7fd1b57fa640 1 -- 172.21.15.77:0/3847130818 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1a809cb90 con 0x7fd1b8102dd0 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 msgr2=0x7fd1a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 0x7fd1a00a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 msgr2=0x7fd1b8198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c920 tx=0x7fd1a800cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 shutdown_connections 2024-09-15T22:11:25.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1a00a32d0 0x7fd1a00a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8108300 0x7fd1b819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8103780 0x7fd1b8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 --2- 172.21.15.77:0/3847130818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8102dd0 0x7fd1b8198590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 >> 172.21.15.77:0/3847130818 conn(0x7fd1b80fc7d0 msgr2=0x7fd1b8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 shutdown_connections 2024-09-15T22:11:25.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:25.690+0000 7fd1be70d640 1 -- 172.21.15.77:0/3847130818 wait complete. 2024-09-15T22:11:26.501 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1357097103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:11:26.501 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3847130818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:11:26.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364848 got 214748364851 for osd.7 2024-09-15T22:11:26.534 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:26 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1357097103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:11:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:26 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3847130818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:11:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:26 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1357097103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:11:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:26 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3847130818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:11:27.002 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953535 got 137438953538 for osd.4 2024-09-15T22:11:27.002 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 -- 172.21.15.77:0/2938102778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 msgr2=0x7f671010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 --2- 172.21.15.77:0/2938102778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671010da40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6708009900 tx=0x7f670802a480 comp rx=0 tx=0).stop 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 -- 172.21.15.77:0/2938102778 shutdown_connections 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 --2- 172.21.15.77:0/2938102778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671010da40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 --2- 172.21.15.77:0/2938102778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710102570 0x7f67101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 --2- 172.21.15.77:0/2938102778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710100e20 0x7f6710101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 -- 172.21.15.77:0/2938102778 >> 172.21.15.77:0/2938102778 conn(0x7f67100fc7d0 msgr2=0x7f67100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 -- 172.21.15.77:0/2938102778 shutdown_connections 2024-09-15T22:11:27.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.444+0000 7f6715361640 1 -- 172.21.15.77:0/2938102778 wait complete. 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 Processor -- start 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 -- start start 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 0x7f67101984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 0x7f6710198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6710110fa0 con 0x7f6710102ef0 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6710110e20 con 0x7f6710100e20 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.445+0000 7f6715361640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6710111120 con 0x7f6710102570 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44676/0 (socket says 172.21.15.77:44676) 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 -- 172.21.15.77:0/4178995067 learned_addr learned my addr 172.21.15.77:0/4178995067 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670effd640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 0x7f67101984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670e7fc640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 0x7f6710198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 -- 172.21.15.77:0/4178995067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 msgr2=0x7f6710198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 0x7f6710198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 -- 172.21.15.77:0/4178995067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 msgr2=0x7f67101984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 0x7f67101984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 -- 172.21.15.77:0/4178995067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671019d4a0 con 0x7f6710102ef0 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670effd640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 0x7f67101984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670e7fc640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 0x7f6710198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f670f7fe640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6708004770 tx=0x7f6708003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670803f070 con 0x7f6710102ef0 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6708031c30 con 0x7f6710102ef0 2024-09-15T22:11:27.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671019d730 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708035030 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.446+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101a5070 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.447+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710101200 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.448+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6708042070 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.449+0000 7f66f7fff640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 0x7f66f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.449+0000 7f670effd640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 0x7f66f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.449+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f67080f06f0 con 0x7f6710102ef0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.449+0000 7f670effd640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 0x7f66f80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6700007eb0 tx=0x7f67000073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.453+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67080bcf50 con 0x7f6710102ef0 2024-09-15T22:11:27.495 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:27 smithi077 ceph-mon[30980]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:27 smithi196 ceph-mon[35109]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:27 smithi117 ceph-mon[35021]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:27.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.626+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f671010a0a0 con 0x7f6710102ef0 2024-09-15T22:11:27.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.626+0000 7f66f7fff640 1 -- 172.21.15.77:0/4178995067 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67080bc850 con 0x7f6710102ef0 2024-09-15T22:11:27.628 INFO:teuthology.orchestra.run.smithi077.stdout:193273528376 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 msgr2=0x7f66f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 0x7f66f80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6700007eb0 tx=0x7f67000073c0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 msgr2=0x7f671019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6708004770 tx=0x7f6708003c10 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 shutdown_connections 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f66f80a32d0 0x7f66f80a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6710102ef0 0x7f671019cdc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6710102570 0x7f6710198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 --2- 172.21.15.77:0/4178995067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710100e20 0x7f67101984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 >> 172.21.15.77:0/4178995067 conn(0x7f67100fc7d0 msgr2=0x7f671006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 shutdown_connections 2024-09-15T22:11:27.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:27.628+0000 7f6715361640 1 -- 172.21.15.77:0/4178995067 wait complete. 2024-09-15T22:11:28.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790011 got 158913790013 for osd.5 2024-09-15T22:11:28.495 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:28 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4178995067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:11:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:28 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4178995067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:11:28.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:28 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4178995067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:11:28.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528373 got 193273528376 for osd.6 2024-09-15T22:11:28.975 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:29.547 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:29.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:29 smithi077 ceph-mon[30980]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:29.586 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135916 got 244813135918 for osd.8 2024-09-15T22:11:29.586 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:29.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:29 smithi196 ceph-mon[35109]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:29.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:29 smithi117 ceph-mon[35021]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:30.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 -- 172.21.15.77:0/3609456992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 msgr2=0x7f005c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 --2- 172.21.15.77:0/3609456992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c10ec40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f004c009930 tx=0x7f004c02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 -- 172.21.15.77:0/3609456992 shutdown_connections 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 --2- 172.21.15.77:0/3609456992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c10ec40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 --2- 172.21.15.77:0/3609456992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c1036e0 0x7f005c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 --2- 172.21.15.77:0/3609456992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c102d30 0x7f005c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 -- 172.21.15.77:0/3609456992 >> 172.21.15.77:0/3609456992 conn(0x7f005c0fc7d0 msgr2=0x7f005c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 -- 172.21.15.77:0/3609456992 shutdown_connections 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.062+0000 7f0060984640 1 -- 172.21.15.77:0/3609456992 wait complete. 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 Processor -- start 2024-09-15T22:11:30.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 -- start start 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c102d30 0x7f005c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c1036e0 0x7f005c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1120a0 con 0x7f005c1082a0 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f005c111f20 con 0x7f005c1036e0 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0060984640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f005c112220 con 0x7f005c102d30 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37386/0 (socket says 172.21.15.77:37386) 2024-09-15T22:11:30.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f005ad9d640 1 -- 172.21.15.77:0/2314164739 learned_addr learned my addr 172.21.15.77:0/2314164739 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f0059d9b640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c1036e0 0x7f005c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f005a59c640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c102d30 0x7f005c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.063+0000 7f005ad9d640 1 -- 172.21.15.77:0/2314164739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c102d30 msgr2=0x7f005c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f005ad9d640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c102d30 0x7f005c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f005ad9d640 1 -- 172.21.15.77:0/2314164739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c1036e0 msgr2=0x7f005c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f005ad9d640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c1036e0 0x7f005c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f005ad9d640 1 -- 172.21.15.77:0/2314164739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c19d510 con 0x7f005c1082a0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f005ad9d640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c03f030 con 0x7f005c1082a0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c003ce0 con 0x7f005c1082a0 2024-09-15T22:11:30.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c19d7a0 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c035070 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.064+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a50e0 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.065+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c06a850 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.066+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f004c042030 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.067+0000 7f004b7fe640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 0x7f00440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.067+0000 7f005a59c640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 0x7f00440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.067+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f004c0f0740 con 0x7f005c1082a0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.068+0000 7f005a59c640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 0x7f00440a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0054005fd0 tx=0x7f005400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.071+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f004c0bcfa0 con 0x7f005c1082a0 2024-09-15T22:11:30.118 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:30.120 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:30.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.257+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f005c10a2b0 con 0x7f005c1082a0 2024-09-15T22:11:30.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.257+0000 7f004b7fe640 1 -- 172.21.15.77:0/2314164739 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f004c0bc8a0 con 0x7f005c1082a0 2024-09-15T22:11:30.259 INFO:teuthology.orchestra.run.smithi077.stdout:103079215176 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 msgr2=0x7f00440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 0x7f00440a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0054005fd0 tx=0x7f005400b590 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 msgr2=0x7f005c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 shutdown_connections 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f00440a32d0 0x7f00440a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f005c1082a0 0x7f005c19ce30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f005c1036e0 0x7f005c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 --2- 172.21.15.77:0/2314164739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f005c102d30 0x7f005c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.259+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 >> 172.21.15.77:0/2314164739 conn(0x7f005c0fc7d0 msgr2=0x7f005c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:30.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.260+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 shutdown_connections 2024-09-15T22:11:30.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:30.260+0000 7f0060984640 1 -- 172.21.15.77:0/2314164739 wait complete. 2024-09-15T22:11:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:30 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2314164739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:11:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:30 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2314164739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:11:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:30 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2314164739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:11:31.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215176 for osd.3 2024-09-15T22:11:31.041 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 -- 172.21.15.77:0/1687419898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c106a80 msgr2=0x7fa71c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 --2- 172.21.15.77:0/1687419898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c106a80 0x7fa71c106e60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009930 tx=0x7fa70c02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 -- 172.21.15.77:0/1687419898 shutdown_connections 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 --2- 172.21.15.77:0/1687419898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c101420 0x7fa71c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 --2- 172.21.15.77:0/1687419898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c100a80 0x7fa71c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 --2- 172.21.15.77:0/1687419898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c106a80 0x7fa71c106e60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 -- 172.21.15.77:0/1687419898 >> 172.21.15.77:0/1687419898 conn(0x7fa71c0fc7a0 msgr2=0x7fa71c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 -- 172.21.15.77:0/1687419898 shutdown_connections 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.255+0000 7fa7221bb640 1 -- 172.21.15.77:0/1687419898 wait complete. 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 Processor -- start 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 -- start start 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c101420 0x7fa71c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c106a80 0x7fa71c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c110f20 con 0x7fa71c100a80 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c110da0 con 0x7fa71c101420 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa7221bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1110a0 con 0x7fa71c106a80 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37414/0 (socket says 172.21.15.77:37414) 2024-09-15T22:11:31.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa71b7fe640 1 -- 172.21.15.77:0/2567079321 learned_addr learned my addr 172.21.15.77:0/2567079321 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa71affd640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c101420 0x7fa71c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.256+0000 7fa71bfff640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c106a80 0x7fa71c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 -- 172.21.15.77:0/2567079321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c106a80 msgr2=0x7fa71c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c106a80 0x7fa71c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 -- 172.21.15.77:0/2567079321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c101420 msgr2=0x7fa71c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c101420 0x7fa71c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 -- 172.21.15.77:0/2567079321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c19d5a0 con 0x7fa71c100a80 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa71b7fe640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009a50 tx=0x7fa70c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c03f030 con 0x7fa71c100a80 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c19d830 con 0x7fa71c100a80 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70c003ad0 con 0x7fa71c100a80 2024-09-15T22:11:31.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c035070 con 0x7fa71c100a80 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.257+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c1a5170 con 0x7fa71c100a80 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.258+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c100f60 con 0x7fa71c100a80 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.259+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa70c042030 con 0x7fa71c100a80 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.260+0000 7fa718ff9640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 0x7fa7040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.260+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa70c0f0a20 con 0x7fa71c100a80 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.260+0000 7fa71affd640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 0x7fa7040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.261+0000 7fa71affd640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 0x7fa7040a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa71c06b7e0 tx=0x7fa710009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.263+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa70c0bd280 con 0x7fa71c100a80 2024-09-15T22:11:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.436+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa71c1a5340 con 0x7fa71c100a80 2024-09-15T22:11:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.436+0000 7fa718ff9640 1 -- 172.21.15.77:0/2567079321 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa70c0bcb80 con 0x7fa71c100a80 2024-09-15T22:11:31.438 INFO:teuthology.orchestra.run.smithi077.stdout:296352743461 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 msgr2=0x7fa7040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 0x7fa7040a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa71c06b7e0 tx=0x7fa710009110 comp rx=0 tx=0).stop 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 msgr2=0x7fa71c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009a50 tx=0x7fa70c004270 comp rx=0 tx=0).stop 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 shutdown_connections 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa7040a32d0 0x7fa7040a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa71c106a80 0x7fa71c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa71c101420 0x7fa71c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 --2- 172.21.15.77:0/2567079321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa71c100a80 0x7fa71c1985f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.438+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 >> 172.21.15.77:0/2567079321 conn(0x7fa71c0fc7a0 msgr2=0x7fa71c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.439+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 shutdown_connections 2024-09-15T22:11:31.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:31.439+0000 7fa7221bb640 1 -- 172.21.15.77:0/2567079321 wait complete. 2024-09-15T22:11:31.525 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:31 smithi077 ceph-mon[30980]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-15T22:11:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:31 smithi117 ceph-mon[35021]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-15T22:11:31.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:31 smithi196 ceph-mon[35109]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-09-15T22:11:32.175 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743458 got 296352743461 for osd.10 2024-09-15T22:11:32.176 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 -- 172.21.15.77:0/3950064786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 msgr2=0x7fb22010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 --2- 172.21.15.77:0/3950064786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb22010e5f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb218009ca0 tx=0x7fb21802a950 comp rx=0 tx=0).stop 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 -- 172.21.15.77:0/3950064786 shutdown_connections 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 --2- 172.21.15.77:0/3950064786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb22010e5f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 --2- 172.21.15.77:0/3950064786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 0x7fb22010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 --2- 172.21.15.77:0/3950064786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 -- 172.21.15.77:0/3950064786 >> 172.21.15.77:0/3950064786 conn(0x7fb2200fc780 msgr2=0x7fb2200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 -- 172.21.15.77:0/3950064786 shutdown_connections 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.396+0000 7fb2249ec640 1 -- 172.21.15.77:0/3950064786 wait complete. 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 Processor -- start 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 -- start start 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 0x7fb220100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb220112010 con 0x7fb22010c180 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb220111e90 con 0x7fb220102da0 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb2249ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb220112190 con 0x7fb2201036c0 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:32.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37426/0 (socket says 172.21.15.77:37426) 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21ed9d640 1 -- 172.21.15.77:0/4257826877 learned_addr learned my addr 172.21.15.77:0/4257826877 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21dd9b640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 0x7fb220100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21e59c640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.397+0000 7fb21ed9d640 1 -- 172.21.15.77:0/4257826877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 msgr2=0x7fb220100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21ed9d640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 0x7fb220100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21ed9d640 1 -- 172.21.15.77:0/4257826877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 msgr2=0x7fb220100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21ed9d640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21ed9d640 1 -- 172.21.15.77:0/4257826877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2201a6a10 con 0x7fb22010c180 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21e59c640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb21ed9d640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb21802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218034040 con 0x7fb22010c180 2024-09-15T22:11:32.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201a6ca0 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb218003e00 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218045480 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.398+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a71d0 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.399+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220103200 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.400+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb218037020 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.400+0000 7fb20f7fe640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.401+0000 7fb21e59c640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.401+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb2180f0e30 con 0x7fb22010c180 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.401+0000 7fb21e59c640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb210009930 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:32.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.404+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2180bd7a0 con 0x7fb22010c180 2024-09-15T22:11:32.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.571+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb220107200 con 0x7fb22010c180 2024-09-15T22:11:32.572 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:32 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2567079321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:11:32.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.571+0000 7fb20f7fe640 1 -- 172.21.15.77:0/4257826877 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb2180bd0a0 con 0x7fb22010c180 2024-09-15T22:11:32.573 INFO:teuthology.orchestra.run.smithi077.stdout:326417514529 2024-09-15T22:11:32.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 msgr2=0x7fb2080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb210009930 tx=0x7fb210008040 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 msgr2=0x7fb220101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb218006010 tx=0x7fb21802f710 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 shutdown_connections 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb22010c180 0x7fb220101480 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2201036c0 0x7fb220100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 --2- 172.21.15.77:0/4257826877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb220102da0 0x7fb220100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 >> 172.21.15.77:0/4257826877 conn(0x7fb2200fc780 msgr2=0x7fb2200fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 shutdown_connections 2024-09-15T22:11:32.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:32.573+0000 7fb2249ec640 1 -- 172.21.15.77:0/4257826877 wait complete. 2024-09-15T22:11:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:32 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2567079321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:11:32.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:32 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2567079321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:11:33.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514526 got 326417514529 for osd.11 2024-09-15T22:11:33.405 DEBUG:teuthology.parallel:result is None 2024-09-15T22:11:33.405 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:11:33.405 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:11:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:33.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:33 smithi077 ceph-mon[30980]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:33.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4257826877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:11:33.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:33 smithi196 ceph-mon[35109]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:33.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:33 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4257826877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:11:33.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:33 smithi117 ceph-mon[35021]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:33.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:33 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4257826877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 -- 172.21.15.77:0/1528469473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 msgr2=0x7f4a8010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 --2- 172.21.15.77:0/1528469473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a8010ea90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009ca0 tx=0x7f4a7802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 -- 172.21.15.77:0/1528469473 shutdown_connections 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 --2- 172.21.15.77:0/1528469473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a8010ea90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 --2- 172.21.15.77:0/1528469473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80102790 0x7f4a801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 --2- 172.21.15.77:0/1528469473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80101de0 0x7f4a801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 -- 172.21.15.77:0/1528469473 >> 172.21.15.77:0/1528469473 conn(0x7f4a800fd8c0 msgr2=0x7f4a800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 -- 172.21.15.77:0/1528469473 shutdown_connections 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.139+0000 7f4a86136640 1 -- 172.21.15.77:0/1528469473 wait complete. 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 Processor -- start 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 -- start start 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 0x7f4a800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 0x7f4a80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80111f70 con 0x7f4a80106f10 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80111df0 con 0x7f4a80102790 2024-09-15T22:11:34.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a86136640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a801120f0 con 0x7f4a80101de0 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37438/0 (socket says 172.21.15.77:37438) 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.140+0000 7f4a7ffff640 1 -- 172.21.15.77:0/1837885910 learned_addr learned my addr 172.21.15.77:0/1837885910 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7f7fe640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 0x7f4a800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 -- 172.21.15.77:0/1837885910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 msgr2=0x7f4a800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 0x7f4a800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 -- 172.21.15.77:0/1837885910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 msgr2=0x7f4a80074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7effd640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 0x7f4a80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 0x7f4a80074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 -- 172.21.15.77:0/1837885910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a801a68b0 con 0x7f4a80106f10 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7f7fe640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 0x7f4a800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7effd640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 0x7f4a80074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:34.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7ffff640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a78031a20 tx=0x7f4a78031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78003c30 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a78003dd0 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801a6b40 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7802fcf0 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.141+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801a7080 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.142+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a801021c0 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.143+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a78004b40 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.144+0000 7f4a7cff9640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 0x7f4a680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.144+0000 7f4a7f7fe640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 0x7f4a680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.144+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4a780f1270 con 0x7f4a80106f10 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.144+0000 7f4a7f7fe640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 0x7f4a680a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a700058c0 tx=0x7f4a70005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:34.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.147+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a780bdc70 con 0x7f4a80106f10 2024-09-15T22:11:34.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.311+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a80062630 con 0x7f4a680a32d0 2024-09-15T22:11:34.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.319+0000 7f4a7cff9640 1 -- 172.21.15.77:0/1837885910 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340684 (secure 0 0 0) 0x7f4a80062630 con 0x7f4a680a32d0 2024-09-15T22:11:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:34.326 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:11:34.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 msgr2=0x7f4a680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:34.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 0x7f4a680a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a700058c0 tx=0x7f4a70005d30 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 msgr2=0x7f4a80074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a78031a20 tx=0x7f4a78031a50 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 shutdown_connections 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4a680a32d0 0x7f4a680a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80106f10 0x7f4a80074f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a80102790 0x7f4a80074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 --2- 172.21.15.77:0/1837885910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a80101de0 0x7f4a800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.322+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 >> 172.21.15.77:0/1837885910 conn(0x7f4a800fd8c0 msgr2=0x7f4a800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.323+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 shutdown_connections 2024-09-15T22:11:34.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:34.323+0000 7f4a86136640 1 -- 172.21.15.77:0/1837885910 wait complete. 2024-09-15T22:11:35.066 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-15T22:11:34.206496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660279,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":704,"num_read_kb":478,"num_write":433,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378160,"kb_used_data":12200,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563664,"statfs":{"total":1151940427776,"available":1151553191936,"internally_reserved":0,"allocated":12492800,"data_stored":8008511,"data_compressed":388296,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003062"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":102,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:31.820630+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:11:31.820630+0000","last_peered":"2024-09-15T22:11:31.820630+0000","last_clean":"2024-09-15T22:11:31.820630+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:11:31.820630+0000","last_undegraded":"2024-09-15T22:11:31.820630+0000","last_fullsized":"2024-09-15T22:11:31.820630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234594+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:11:05.234594+0000","last_peered":"2024-09-15T22:11:05.234594+0000","last_clean":"2024-09-15T22:11:05.234594+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:11:05.234594+0000","last_undegraded":"2024-09-15T22:11:05.234594+0000","last_fullsized":"2024-09-15T22:11:05.234594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237735+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:11:05.237735+0000","last_peered":"2024-09-15T22:11:05.237735+0000","last_clean":"2024-09-15T22:11:05.237735+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:11:05.237735+0000","last_undegraded":"2024-09-15T22:11:05.237735+0000","last_fullsized":"2024-09-15T22:11:05.237735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234804+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:11:05.234804+0000","last_peered":"2024-09-15T22:11:05.234804+0000","last_clean":"2024-09-15T22:11:05.234804+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:11:05.234804+0000","last_undegraded":"2024-09-15T22:11:05.234804+0000","last_fullsized":"2024-09-15T22:11:05.234804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233657+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:11:05.233657+0000","last_peered":"2024-09-15T22:11:05.233657+0000","last_clean":"2024-09-15T22:11:05.233657+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:11:05.233657+0000","last_undegraded":"2024-09-15T22:11:05.233657+0000","last_fullsized":"2024-09-15T22:11:05.233657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231120+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:11:05.231120+0000","last_peered":"2024-09-15T22:11:05.231120+0000","last_clean":"2024-09-15T22:11:05.231120+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:11:05.231120+0000","last_undegraded":"2024-09-15T22:11:05.231120+0000","last_fullsized":"2024-09-15T22:11:05.231120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236230+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:11:05.236230+0000","last_peered":"2024-09-15T22:11:05.236230+0000","last_clean":"2024-09-15T22:11:05.236230+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:11:05.236230+0000","last_undegraded":"2024-09-15T22:11:05.236230+0000","last_fullsized":"2024-09-15T22:11:05.236230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229647+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:11:05.229647+0000","last_peered":"2024-09-15T22:11:05.229647+0000","last_clean":"2024-09-15T22:11:05.229647+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:11:05.229647+0000","last_undegraded":"2024-09-15T22:11:05.229647+0000","last_fullsized":"2024-09-15T22:11:05.229647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232331+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:11:05.232331+0000","last_peered":"2024-09-15T22:11:05.232331+0000","last_clean":"2024-09-15T22:11:05.232331+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:11:05.232331+0000","last_undegraded":"2024-09-15T22:11:05.232331+0000","last_fullsized":"2024-09-15T22:11:05.232331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236447+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:11:05.236447+0000","last_peered":"2024-09-15T22:11:05.236447+0000","last_clean":"2024-09-15T22:11:05.236447+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:11:05.236447+0000","last_undegraded":"2024-09-15T22:11:05.236447+0000","last_fullsized":"2024-09-15T22:11:05.236447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.239018+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:11:05.239018+0000","last_peered":"2024-09-15T22:11:05.239018+0000","last_clean":"2024-09-15T22:11:05.239018+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:11:05.239018+0000","last_undegraded":"2024-09-15T22:11:05.239018+0000","last_fullsized":"2024-09-15T22:11:05.239018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231265+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:11:05.231265+0000","last_peered":"2024-09-15T22:11:05.231265+0000","last_clean":"2024-09-15T22:11:05.231265+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:11:05.231265+0000","last_undegraded":"2024-09-15T22:11:05.231265+0000","last_fullsized":"2024-09-15T22:11:05.231265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233818+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:11:05.233818+0000","last_peered":"2024-09-15T22:11:05.233818+0000","last_clean":"2024-09-15T22:11:05.233818+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:11:05.233818+0000","last_undegraded":"2024-09-15T22:11:05.233818+0000","last_fullsized":"2024-09-15T22:11:05.233818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235563+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:11:05.235563+0000","last_peered":"2024-09-15T22:11:05.235563+0000","last_clean":"2024-09-15T22:11:05.235563+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:11:05.235563+0000","last_undegraded":"2024-09-15T22:11:05.235563+0000","last_fullsized":"2024-09-15T22:11:05.235563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231916+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:11:05.231916+0000","last_peered":"2024-09-15T22:11:05.231916+0000","last_clean":"2024-09-15T22:11:05.231916+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:11:05.231916+0000","last_undegraded":"2024-09-15T22:11:05.231916+0000","last_fullsized":"2024-09-15T22:11:05.231916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234342+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:11:05.234342+0000","last_peered":"2024-09-15T22:11:05.234342+0000","last_clean":"2024-09-15T22:11:05.234342+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:11:05.234342+0000","last_undegraded":"2024-09-15T22:11:05.234342+0000","last_fullsized":"2024-09-15T22:11:05.234342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.246212+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:11:05.246212+0000","last_peered":"2024-09-15T22:11:05.246212+0000","last_clean":"2024-09-15T22:11:05.246212+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:11:05.246212+0000","last_undegraded":"2024-09-15T22:11:05.246212+0000","last_fullsized":"2024-09-15T22:11:05.246212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231703+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:11:05.231703+0000","last_peered":"2024-09-15T22:11:05.231703+0000","last_clean":"2024-09-15T22:11:05.231703+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:11:05.231703+0000","last_undegraded":"2024-09-15T22:11:05.231703+0000","last_fullsized":"2024-09-15T22:11:05.231703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231059+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:11:05.231059+0000","last_peered":"2024-09-15T22:11:05.231059+0000","last_clean":"2024-09-15T22:11:05.231059+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:11:05.231059+0000","last_undegraded":"2024-09-15T22:11:05.231059+0000","last_fullsized":"2024-09-15T22:11:05.231059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234467+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:11:05.234467+0000","last_peered":"2024-09-15T22:11:05.234467+0000","last_clean":"2024-09-15T22:11:05.234467+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:11:05.234467+0000","last_undegraded":"2024-09-15T22:11:05.234467+0000","last_fullsized":"2024-09-15T22:11:05.234467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.244091+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:11:05.244091+0000","last_peered":"2024-09-15T22:11:05.244091+0000","last_clean":"2024-09-15T22:11:05.244091+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:11:05.244091+0000","last_undegraded":"2024-09-15T22:11:05.244091+0000","last_fullsized":"2024-09-15T22:11:05.244091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234631+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:11:05.234631+0000","last_peered":"2024-09-15T22:11:05.234631+0000","last_clean":"2024-09-15T22:11:05.234631+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:11:05.234631+0000","last_undegraded":"2024-09-15T22:11:05.234631+0000","last_fullsized":"2024-09-15T22:11:05.234631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229452+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:11:05.229452+0000","last_peered":"2024-09-15T22:11:05.229452+0000","last_clean":"2024-09-15T22:11:05.229452+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:11:05.229452+0000","last_undegraded":"2024-09-15T22:11:05.229452+0000","last_fullsized":"2024-09-15T22:11:05.229452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231973+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:11:05.231973+0000","last_peered":"2024-09-15T22:11:05.231973+0000","last_clean":"2024-09-15T22:11:05.231973+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:11:05.231973+0000","last_undegraded":"2024-09-15T22:11:05.231973+0000","last_fullsized":"2024-09-15T22:11:05.231973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233518+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:11:05.233518+0000","last_peered":"2024-09-15T22:11:05.233518+0000","last_clean":"2024-09-15T22:11:05.233518+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:11:05.233518+0000","last_undegraded":"2024-09-15T22:11:05.233518+0000","last_fullsized":"2024-09-15T22:11:05.233518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232463+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:11:05.232463+0000","last_peered":"2024-09-15T22:11:05.232463+0000","last_clean":"2024-09-15T22:11:05.232463+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:11:05.232463+0000","last_undegraded":"2024-09-15T22:11:05.232463+0000","last_fullsized":"2024-09-15T22:11:05.232463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237933+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:11:05.237933+0000","last_peered":"2024-09-15T22:11:05.237933+0000","last_clean":"2024-09-15T22:11:05.237933+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:11:05.237933+0000","last_undegraded":"2024-09-15T22:11:05.237933+0000","last_fullsized":"2024-09-15T22:11:05.237933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234774+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:11:05.234774+0000","last_peered":"2024-09-15T22:11:05.234774+0000","last_clean":"2024-09-15T22:11:05.234774+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:11:05.234774+0000","last_undegraded":"2024-09-15T22:11:05.234774+0000","last_fullsized":"2024-09-15T22:11:05.234774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293199+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:11:05.293199+0000","last_peered":"2024-09-15T22:11:05.293199+0000","last_clean":"2024-09-15T22:11:05.293199+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:11:05.293199+0000","last_undegraded":"2024-09-15T22:11:05.293199+0000","last_fullsized":"2024-09-15T22:11:05.293199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234136+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:11:05.234136+0000","last_peered":"2024-09-15T22:11:05.234136+0000","last_clean":"2024-09-15T22:11:05.234136+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:11:05.234136+0000","last_undegraded":"2024-09-15T22:11:05.234136+0000","last_fullsized":"2024-09-15T22:11:05.234136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229318+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:11:05.229318+0000","last_peered":"2024-09-15T22:11:05.229318+0000","last_clean":"2024-09-15T22:11:05.229318+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:11:05.229318+0000","last_undegraded":"2024-09-15T22:11:05.229318+0000","last_fullsized":"2024-09-15T22:11:05.229318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236291+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:11:05.236291+0000","last_peered":"2024-09-15T22:11:05.236291+0000","last_clean":"2024-09-15T22:11:05.236291+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:11:05.236291+0000","last_undegraded":"2024-09-15T22:11:05.236291+0000","last_fullsized":"2024-09-15T22:11:05.236291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.238076+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:11:05.238076+0000","last_peered":"2024-09-15T22:11:05.238076+0000","last_clean":"2024-09-15T22:11:05.238076+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:11:05.238076+0000","last_undegraded":"2024-09-15T22:11:05.238076+0000","last_fullsized":"2024-09-15T22:11:05.238076+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234836+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:11:05.234836+0000","last_peered":"2024-09-15T22:11:05.234836+0000","last_clean":"2024-09-15T22:11:05.234836+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:11:05.234836+0000","last_undegraded":"2024-09-15T22:11:05.234836+0000","last_fullsized":"2024-09-15T22:11:05.234836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292980+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:11:05.292980+0000","last_peered":"2024-09-15T22:11:05.292980+0000","last_clean":"2024-09-15T22:11:05.292980+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:11:05.292980+0000","last_undegraded":"2024-09-15T22:11:05.292980+0000","last_fullsized":"2024-09-15T22:11:05.292980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234798+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:11:05.234798+0000","last_peered":"2024-09-15T22:11:05.234798+0000","last_clean":"2024-09-15T22:11:05.234798+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:11:05.234798+0000","last_undegraded":"2024-09-15T22:11:05.234798+0000","last_fullsized":"2024-09-15T22:11:05.234798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294432+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:11:05.294432+0000","last_peered":"2024-09-15T22:11:05.294432+0000","last_clean":"2024-09-15T22:11:05.294432+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:11:05.294432+0000","last_undegraded":"2024-09-15T22:11:05.294432+0000","last_fullsized":"2024-09-15T22:11:05.294432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.819740+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:11:30.819740+0000","last_peered":"2024-09-15T22:11:30.819740+0000","last_clean":"2024-09-15T22:11:30.819740+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:11:30.819740+0000","last_undegraded":"2024-09-15T22:11:30.819740+0000","last_fullsized":"2024-09-15T22:11:30.819740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229240+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:11:05.229240+0000","last_peered":"2024-09-15T22:11:05.229240+0000","last_clean":"2024-09-15T22:11:05.229240+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:11:05.229240+0000","last_undegraded":"2024-09-15T22:11:05.229240+0000","last_fullsized":"2024-09-15T22:11:05.229240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233447+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:11:05.233447+0000","last_peered":"2024-09-15T22:11:05.233447+0000","last_clean":"2024-09-15T22:11:05.233447+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:11:05.233447+0000","last_undegraded":"2024-09-15T22:11:05.233447+0000","last_fullsized":"2024-09-15T22:11:05.233447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:31.764100+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:11:31.764100+0000","last_peered":"2024-09-15T22:11:31.764100+0000","last_clean":"2024-09-15T22:11:31.764100+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:11:31.764100+0000","last_undegraded":"2024-09-15T22:11:31.764100+0000","last_fullsized":"2024-09-15T22:11:31.764100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229271+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:11:05.229271+0000","last_peered":"2024-09-15T22:11:05.229271+0000","last_clean":"2024-09-15T22:11:05.229271+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:11:05.229271+0000","last_undegraded":"2024-09-15T22:11:05.229271+0000","last_fullsized":"2024-09-15T22:11:05.229271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228542+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:11:05.228542+0000","last_peered":"2024-09-15T22:11:05.228542+0000","last_clean":"2024-09-15T22:11:05.228542+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:11:05.228542+0000","last_undegraded":"2024-09-15T22:11:05.228542+0000","last_fullsized":"2024-09-15T22:11:05.228542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233587+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:11:05.233587+0000","last_peered":"2024-09-15T22:11:05.233587+0000","last_clean":"2024-09-15T22:11:05.233587+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:11:05.233587+0000","last_undegraded":"2024-09-15T22:11:05.233587+0000","last_fullsized":"2024-09-15T22:11:05.233587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232889+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:11:05.232889+0000","last_peered":"2024-09-15T22:11:05.232889+0000","last_clean":"2024-09-15T22:11:05.232889+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:11:05.232889+0000","last_undegraded":"2024-09-15T22:11:05.232889+0000","last_fullsized":"2024-09-15T22:11:05.232889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234619+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:11:05.234619+0000","last_peered":"2024-09-15T22:11:05.234619+0000","last_clean":"2024-09-15T22:11:05.234619+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:11:05.234619+0000","last_undegraded":"2024-09-15T22:11:05.234619+0000","last_fullsized":"2024-09-15T22:11:05.234619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232689+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:11:05.232689+0000","last_peered":"2024-09-15T22:11:05.232689+0000","last_clean":"2024-09-15T22:11:05.232689+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:11:05.232689+0000","last_undegraded":"2024-09-15T22:11:05.232689+0000","last_fullsized":"2024-09-15T22:11:05.232689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233868+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:11:05.233868+0000","last_peered":"2024-09-15T22:11:05.233868+0000","last_clean":"2024-09-15T22:11:05.233868+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:11:05.233868+0000","last_undegraded":"2024-09-15T22:11:05.233868+0000","last_fullsized":"2024-09-15T22:11:05.233868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234395+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:11:05.234395+0000","last_peered":"2024-09-15T22:11:05.234395+0000","last_clean":"2024-09-15T22:11:05.234395+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:11:05.234395+0000","last_undegraded":"2024-09-15T22:11:05.234395+0000","last_fullsized":"2024-09-15T22:11:05.234395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234616+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:11:05.234616+0000","last_peered":"2024-09-15T22:11:05.234616+0000","last_clean":"2024-09-15T22:11:05.234616+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:11:05.234616+0000","last_undegraded":"2024-09-15T22:11:05.234616+0000","last_fullsized":"2024-09-15T22:11:05.234616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:29.762952+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:11:29.762952+0000","last_peered":"2024-09-15T22:11:29.762952+0000","last_clean":"2024-09-15T22:11:29.762952+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:11:29.762952+0000","last_undegraded":"2024-09-15T22:11:29.762952+0000","last_fullsized":"2024-09-15T22:11:29.762952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229375+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:11:05.229375+0000","last_peered":"2024-09-15T22:11:05.229375+0000","last_clean":"2024-09-15T22:11:05.229375+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:11:05.229375+0000","last_undegraded":"2024-09-15T22:11:05.229375+0000","last_fullsized":"2024-09-15T22:11:05.229375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294394+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:11:05.294394+0000","last_peered":"2024-09-15T22:11:05.294394+0000","last_clean":"2024-09-15T22:11:05.294394+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:11:05.294394+0000","last_undegraded":"2024-09-15T22:11:05.294394+0000","last_fullsized":"2024-09-15T22:11:05.294394+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234765+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:11:05.234765+0000","last_peered":"2024-09-15T22:11:05.234765+0000","last_clean":"2024-09-15T22:11:05.234765+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:11:05.234765+0000","last_undegraded":"2024-09-15T22:11:05.234765+0000","last_fullsized":"2024-09-15T22:11:05.234765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233017+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:11:05.233017+0000","last_peered":"2024-09-15T22:11:05.233017+0000","last_clean":"2024-09-15T22:11:05.233017+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:11:05.233017+0000","last_undegraded":"2024-09-15T22:11:05.233017+0000","last_fullsized":"2024-09-15T22:11:05.233017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234433+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:11:05.234433+0000","last_peered":"2024-09-15T22:11:05.234433+0000","last_clean":"2024-09-15T22:11:05.234433+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:11:05.234433+0000","last_undegraded":"2024-09-15T22:11:05.234433+0000","last_fullsized":"2024-09-15T22:11:05.234433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229428+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:11:05.229428+0000","last_peered":"2024-09-15T22:11:05.229428+0000","last_clean":"2024-09-15T22:11:05.229428+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:11:05.229428+0000","last_undegraded":"2024-09-15T22:11:05.229428+0000","last_fullsized":"2024-09-15T22:11:05.229428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232764+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:11:05.232764+0000","last_peered":"2024-09-15T22:11:05.232764+0000","last_clean":"2024-09-15T22:11:05.232764+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:11:05.232764+0000","last_undegraded":"2024-09-15T22:11:05.232764+0000","last_fullsized":"2024-09-15T22:11:05.232764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231617+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:11:05.231617+0000","last_peered":"2024-09-15T22:11:05.231617+0000","last_clean":"2024-09-15T22:11:05.231617+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:11:05.231617+0000","last_undegraded":"2024-09-15T22:11:05.231617+0000","last_fullsized":"2024-09-15T22:11:05.231617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236561+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:11:05.236561+0000","last_peered":"2024-09-15T22:11:05.236561+0000","last_clean":"2024-09-15T22:11:05.236561+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:11:05.236561+0000","last_undegraded":"2024-09-15T22:11:05.236561+0000","last_fullsized":"2024-09-15T22:11:05.236561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233407+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:11:05.233407+0000","last_peered":"2024-09-15T22:11:05.233407+0000","last_clean":"2024-09-15T22:11:05.233407+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:11:05.233407+0000","last_undegraded":"2024-09-15T22:11:05.233407+0000","last_fullsized":"2024-09-15T22:11:05.233407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234219+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:11:05.234219+0000","last_peered":"2024-09-15T22:11:05.234219+0000","last_clean":"2024-09-15T22:11:05.234219+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:11:05.234219+0000","last_undegraded":"2024-09-15T22:11:05.234219+0000","last_fullsized":"2024-09-15T22:11:05.234219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.256328+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:11:05.256328+0000","last_peered":"2024-09-15T22:11:05.256328+0000","last_clean":"2024-09-15T22:11:05.256328+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:11:05.256328+0000","last_undegraded":"2024-09-15T22:11:05.256328+0000","last_fullsized":"2024-09-15T22:11:05.256328+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229427+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:11:05.229427+0000","last_peered":"2024-09-15T22:11:05.229427+0000","last_clean":"2024-09-15T22:11:05.229427+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:11:05.229427+0000","last_undegraded":"2024-09-15T22:11:05.229427+0000","last_fullsized":"2024-09-15T22:11:05.229427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233526+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:11:05.233526+0000","last_peered":"2024-09-15T22:11:05.233526+0000","last_clean":"2024-09-15T22:11:05.233526+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:11:05.233526+0000","last_undegraded":"2024-09-15T22:11:05.233526+0000","last_fullsized":"2024-09-15T22:11:05.233526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.763316+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:11:30.763316+0000","last_peered":"2024-09-15T22:11:30.763316+0000","last_clean":"2024-09-15T22:11:30.763316+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:11:30.763316+0000","last_undegraded":"2024-09-15T22:11:30.763316+0000","last_fullsized":"2024-09-15T22:11:30.763316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293135+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:11:05.293135+0000","last_peered":"2024-09-15T22:11:05.293135+0000","last_clean":"2024-09-15T22:11:05.293135+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:11:05.293135+0000","last_undegraded":"2024-09-15T22:11:05.293135+0000","last_fullsized":"2024-09-15T22:11:05.293135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233971+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:11:05.233971+0000","last_peered":"2024-09-15T22:11:05.233971+0000","last_clean":"2024-09-15T22:11:05.233971+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:11:05.233971+0000","last_undegraded":"2024-09-15T22:11:05.233971+0000","last_fullsized":"2024-09-15T22:11:05.233971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234113+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:11:05.234113+0000","last_peered":"2024-09-15T22:11:05.234113+0000","last_clean":"2024-09-15T22:11:05.234113+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:11:05.234113+0000","last_undegraded":"2024-09-15T22:11:05.234113+0000","last_fullsized":"2024-09-15T22:11:05.234113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231197+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:11:05.231197+0000","last_peered":"2024-09-15T22:11:05.231197+0000","last_clean":"2024-09-15T22:11:05.231197+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:11:05.231197+0000","last_undegraded":"2024-09-15T22:11:05.231197+0000","last_fullsized":"2024-09-15T22:11:05.231197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228802+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:11:05.228802+0000","last_peered":"2024-09-15T22:11:05.228802+0000","last_clean":"2024-09-15T22:11:05.228802+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:11:05.228802+0000","last_undegraded":"2024-09-15T22:11:05.228802+0000","last_fullsized":"2024-09-15T22:11:05.228802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236710+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:11:05.236710+0000","last_peered":"2024-09-15T22:11:05.236710+0000","last_clean":"2024-09-15T22:11:05.236710+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:11:05.236710+0000","last_undegraded":"2024-09-15T22:11:05.236710+0000","last_fullsized":"2024-09-15T22:11:05.236710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233559+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:11:05.233559+0000","last_peered":"2024-09-15T22:11:05.233559+0000","last_clean":"2024-09-15T22:11:05.233559+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:11:05.233559+0000","last_undegraded":"2024-09-15T22:11:05.233559+0000","last_fullsized":"2024-09-15T22:11:05.233559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236034+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:11:05.236034+0000","last_peered":"2024-09-15T22:11:05.236034+0000","last_clean":"2024-09-15T22:11:05.236034+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:11:05.236034+0000","last_undegraded":"2024-09-15T22:11:05.236034+0000","last_fullsized":"2024-09-15T22:11:05.236034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:28.760549+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:11:28.760549+0000","last_peered":"2024-09-15T22:11:28.760549+0000","last_clean":"2024-09-15T22:11:28.760549+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:11:28.760549+0000","last_undegraded":"2024-09-15T22:11:28.760549+0000","last_fullsized":"2024-09-15T22:11:28.760549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":116,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292722+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:11:05.292722+0000","last_peered":"2024-09-15T22:11:05.292722+0000","last_clean":"2024-09-15T22:11:05.292722+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:11:05.292722+0000","last_undegraded":"2024-09-15T22:11:05.292722+0000","last_fullsized":"2024-09-15T22:11:05.292722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229836+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:11:05.229836+0000","last_peered":"2024-09-15T22:11:05.229836+0000","last_clean":"2024-09-15T22:11:05.229836+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:11:05.229836+0000","last_undegraded":"2024-09-15T22:11:05.229836+0000","last_fullsized":"2024-09-15T22:11:05.229836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234717+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:11:05.234717+0000","last_peered":"2024-09-15T22:11:05.234717+0000","last_clean":"2024-09-15T22:11:05.234717+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:11:05.234717+0000","last_undegraded":"2024-09-15T22:11:05.234717+0000","last_fullsized":"2024-09-15T22:11:05.234717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.239662+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:11:05.239662+0000","last_peered":"2024-09-15T22:11:05.239662+0000","last_clean":"2024-09-15T22:11:05.239662+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:11:05.239662+0000","last_undegraded":"2024-09-15T22:11:05.239662+0000","last_fullsized":"2024-09-15T22:11:05.239662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233686+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:11:05.233686+0000","last_peered":"2024-09-15T22:11:05.233686+0000","last_clean":"2024-09-15T22:11:05.233686+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:11:05.233686+0000","last_undegraded":"2024-09-15T22:11:05.233686+0000","last_fullsized":"2024-09-15T22:11:05.233686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234498+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:11:05.234498+0000","last_peered":"2024-09-15T22:11:05.234498+0000","last_clean":"2024-09-15T22:11:05.234498+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:11:05.234498+0000","last_undegraded":"2024-09-15T22:11:05.234498+0000","last_fullsized":"2024-09-15T22:11:05.234498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.245652+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:11:05.245652+0000","last_peered":"2024-09-15T22:11:05.245652+0000","last_clean":"2024-09-15T22:11:05.245652+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:11:05.245652+0000","last_undegraded":"2024-09-15T22:11:05.245652+0000","last_fullsized":"2024-09-15T22:11:05.245652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230107+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:11:05.230107+0000","last_peered":"2024-09-15T22:11:05.230107+0000","last_clean":"2024-09-15T22:11:05.230107+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:11:05.230107+0000","last_undegraded":"2024-09-15T22:11:05.230107+0000","last_fullsized":"2024-09-15T22:11:05.230107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:28.760629+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:11:28.760629+0000","last_peered":"2024-09-15T22:11:28.760629+0000","last_clean":"2024-09-15T22:11:28.760629+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:11:28.760629+0000","last_undegraded":"2024-09-15T22:11:28.760629+0000","last_fullsized":"2024-09-15T22:11:28.760629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234273+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:11:05.234273+0000","last_peered":"2024-09-15T22:11:05.234273+0000","last_clean":"2024-09-15T22:11:05.234273+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:11:05.234273+0000","last_undegraded":"2024-09-15T22:11:05.234273+0000","last_fullsized":"2024-09-15T22:11:05.234273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233416+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:11:05.233416+0000","last_peered":"2024-09-15T22:11:05.233416+0000","last_clean":"2024-09-15T22:11:05.233416+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:11:05.233416+0000","last_undegraded":"2024-09-15T22:11:05.233416+0000","last_fullsized":"2024-09-15T22:11:05.233416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292831+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:11:05.292831+0000","last_peered":"2024-09-15T22:11:05.292831+0000","last_clean":"2024-09-15T22:11:05.292831+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:11:05.292831+0000","last_undegraded":"2024-09-15T22:11:05.292831+0000","last_fullsized":"2024-09-15T22:11:05.292831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233600+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:11:05.233600+0000","last_peered":"2024-09-15T22:11:05.233600+0000","last_clean":"2024-09-15T22:11:05.233600+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:11:05.233600+0000","last_undegraded":"2024-09-15T22:11:05.233600+0000","last_fullsized":"2024-09-15T22:11:05.233600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231748+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:11:05.231748+0000","last_peered":"2024-09-15T22:11:05.231748+0000","last_clean":"2024-09-15T22:11:05.231748+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:11:05.231748+0000","last_undegraded":"2024-09-15T22:11:05.231748+0000","last_fullsized":"2024-09-15T22:11:05.231748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234250+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:11:05.234250+0000","last_peered":"2024-09-15T22:11:05.234250+0000","last_clean":"2024-09-15T22:11:05.234250+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:11:05.234250+0000","last_undegraded":"2024-09-15T22:11:05.234250+0000","last_fullsized":"2024-09-15T22:11:05.234250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228574+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:11:05.228574+0000","last_peered":"2024-09-15T22:11:05.228574+0000","last_clean":"2024-09-15T22:11:05.228574+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:11:05.228574+0000","last_undegraded":"2024-09-15T22:11:05.228574+0000","last_fullsized":"2024-09-15T22:11:05.228574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235257+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:11:05.235257+0000","last_peered":"2024-09-15T22:11:05.235257+0000","last_clean":"2024-09-15T22:11:05.235257+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:11:05.235257+0000","last_undegraded":"2024-09-15T22:11:05.235257+0000","last_fullsized":"2024-09-15T22:11:05.235257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234474+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:11:05.234474+0000","last_peered":"2024-09-15T22:11:05.234474+0000","last_clean":"2024-09-15T22:11:05.234474+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:11:05.234474+0000","last_undegraded":"2024-09-15T22:11:05.234474+0000","last_fullsized":"2024-09-15T22:11:05.234474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236033+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:11:05.236033+0000","last_peered":"2024-09-15T22:11:05.236033+0000","last_clean":"2024-09-15T22:11:05.236033+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:11:05.236033+0000","last_undegraded":"2024-09-15T22:11:05.236033+0000","last_fullsized":"2024-09-15T22:11:05.236033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.764312+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:11:30.764312+0000","last_peered":"2024-09-15T22:11:30.764312+0000","last_clean":"2024-09-15T22:11:30.764312+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:11:30.764312+0000","last_undegraded":"2024-09-15T22:11:30.764312+0000","last_fullsized":"2024-09-15T22:11:30.764312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234982+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:11:05.234982+0000","last_peered":"2024-09-15T22:11:05.234982+0000","last_clean":"2024-09-15T22:11:05.234982+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:11:05.234982+0000","last_undegraded":"2024-09-15T22:11:05.234982+0000","last_fullsized":"2024-09-15T22:11:05.234982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236649+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:11:05.236649+0000","last_peered":"2024-09-15T22:11:05.236649+0000","last_clean":"2024-09-15T22:11:05.236649+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:11:05.236649+0000","last_undegraded":"2024-09-15T22:11:05.236649+0000","last_fullsized":"2024-09-15T22:11:05.236649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230947+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:11:05.230947+0000","last_peered":"2024-09-15T22:11:05.230947+0000","last_clean":"2024-09-15T22:11:05.230947+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:11:05.230947+0000","last_undegraded":"2024-09-15T22:11:05.230947+0000","last_fullsized":"2024-09-15T22:11:05.230947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294107+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:11:05.294107+0000","last_peered":"2024-09-15T22:11:05.294107+0000","last_clean":"2024-09-15T22:11:05.294107+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:11:05.294107+0000","last_undegraded":"2024-09-15T22:11:05.294107+0000","last_fullsized":"2024-09-15T22:11:05.294107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237700+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:11:05.237700+0000","last_peered":"2024-09-15T22:11:05.237700+0000","last_clean":"2024-09-15T22:11:05.237700+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:11:05.237700+0000","last_undegraded":"2024-09-15T22:11:05.237700+0000","last_fullsized":"2024-09-15T22:11:05.237700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233137+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:11:05.233137+0000","last_peered":"2024-09-15T22:11:05.233137+0000","last_clean":"2024-09-15T22:11:05.233137+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:11:05.233137+0000","last_undegraded":"2024-09-15T22:11:05.233137+0000","last_fullsized":"2024-09-15T22:11:05.233137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229232+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:11:05.229232+0000","last_peered":"2024-09-15T22:11:05.229232+0000","last_clean":"2024-09-15T22:11:05.229232+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:11:05.229232+0000","last_undegraded":"2024-09-15T22:11:05.229232+0000","last_fullsized":"2024-09-15T22:11:05.229232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231708+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:11:05.231708+0000","last_peered":"2024-09-15T22:11:05.231708+0000","last_clean":"2024-09-15T22:11:05.231708+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:11:05.231708+0000","last_undegraded":"2024-09-15T22:11:05.231708+0000","last_fullsized":"2024-09-15T22:11:05.231708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236950+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:11:05.236950+0000","last_peered":"2024-09-15T22:11:05.236950+0000","last_clean":"2024-09-15T22:11:05.236950+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:11:05.236950+0000","last_undegraded":"2024-09-15T22:11:05.236950+0000","last_fullsized":"2024-09-15T22:11:05.236950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234733+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:11:05.234733+0000","last_peered":"2024-09-15T22:11:05.234733+0000","last_clean":"2024-09-15T22:11:05.234733+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:11:05.234733+0000","last_undegraded":"2024-09-15T22:11:05.234733+0000","last_fullsized":"2024-09-15T22:11:05.234733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233330+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:11:05.233330+0000","last_peered":"2024-09-15T22:11:05.233330+0000","last_clean":"2024-09-15T22:11:05.233330+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:11:05.233330+0000","last_undegraded":"2024-09-15T22:11:05.233330+0000","last_fullsized":"2024-09-15T22:11:05.233330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234371+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:11:05.234371+0000","last_peered":"2024-09-15T22:11:05.234371+0000","last_clean":"2024-09-15T22:11:05.234371+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:11:05.234371+0000","last_undegraded":"2024-09-15T22:11:05.234371+0000","last_fullsized":"2024-09-15T22:11:05.234371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229709+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:11:05.229709+0000","last_peered":"2024-09-15T22:11:05.229709+0000","last_clean":"2024-09-15T22:11:05.229709+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:11:05.229709+0000","last_undegraded":"2024-09-15T22:11:05.229709+0000","last_fullsized":"2024-09-15T22:11:05.229709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229024+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:11:05.229024+0000","last_peered":"2024-09-15T22:11:05.229024+0000","last_clean":"2024-09-15T22:11:05.229024+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:11:05.229024+0000","last_undegraded":"2024-09-15T22:11:05.229024+0000","last_fullsized":"2024-09-15T22:11:05.229024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231312+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:11:05.231312+0000","last_peered":"2024-09-15T22:11:05.231312+0000","last_clean":"2024-09-15T22:11:05.231312+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:11:05.231312+0000","last_undegraded":"2024-09-15T22:11:05.231312+0000","last_fullsized":"2024-09-15T22:11:05.231312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236840+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:11:05.236840+0000","last_peered":"2024-09-15T22:11:05.236840+0000","last_clean":"2024-09-15T22:11:05.236840+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:11:05.236840+0000","last_undegraded":"2024-09-15T22:11:05.236840+0000","last_fullsized":"2024-09-15T22:11:05.236840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293170+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:11:05.293170+0000","last_peered":"2024-09-15T22:11:05.293170+0000","last_clean":"2024-09-15T22:11:05.293170+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:11:05.293170+0000","last_undegraded":"2024-09-15T22:11:05.293170+0000","last_fullsized":"2024-09-15T22:11:05.293170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232141+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:11:05.232141+0000","last_peered":"2024-09-15T22:11:05.232141+0000","last_clean":"2024-09-15T22:11:05.232141+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:11:05.232141+0000","last_undegraded":"2024-09-15T22:11:05.232141+0000","last_fullsized":"2024-09-15T22:11:05.232141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229299+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:11:05.229299+0000","last_peered":"2024-09-15T22:11:05.229299+0000","last_clean":"2024-09-15T22:11:05.229299+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:11:05.229299+0000","last_undegraded":"2024-09-15T22:11:05.229299+0000","last_fullsized":"2024-09-15T22:11:05.229299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233247+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:11:05.233247+0000","last_peered":"2024-09-15T22:11:05.233247+0000","last_clean":"2024-09-15T22:11:05.233247+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:11:05.233247+0000","last_undegraded":"2024-09-15T22:11:05.233247+0000","last_fullsized":"2024-09-15T22:11:05.233247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232869+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:11:05.232869+0000","last_peered":"2024-09-15T22:11:05.232869+0000","last_clean":"2024-09-15T22:11:05.232869+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:11:05.232869+0000","last_undegraded":"2024-09-15T22:11:05.232869+0000","last_fullsized":"2024-09-15T22:11:05.232869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228937+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:11:05.228937+0000","last_peered":"2024-09-15T22:11:05.228937+0000","last_clean":"2024-09-15T22:11:05.228937+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:11:05.228937+0000","last_undegraded":"2024-09-15T22:11:05.228937+0000","last_fullsized":"2024-09-15T22:11:05.228937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232601+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:11:05.232601+0000","last_peered":"2024-09-15T22:11:05.232601+0000","last_clean":"2024-09-15T22:11:05.232601+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:11:05.232601+0000","last_undegraded":"2024-09-15T22:11:05.232601+0000","last_fullsized":"2024-09-15T22:11:05.232601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234512+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:11:05.234512+0000","last_peered":"2024-09-15T22:11:05.234512+0000","last_clean":"2024-09-15T22:11:05.234512+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:11:05.234512+0000","last_undegraded":"2024-09-15T22:11:05.234512+0000","last_fullsized":"2024-09-15T22:11:05.234512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235637+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:11:05.235637+0000","last_peered":"2024-09-15T22:11:05.235637+0000","last_clean":"2024-09-15T22:11:05.235637+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:11:05.235637+0000","last_undegraded":"2024-09-15T22:11:05.235637+0000","last_fullsized":"2024-09-15T22:11:05.235637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230140+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:11:05.230140+0000","last_peered":"2024-09-15T22:11:05.230140+0000","last_clean":"2024-09-15T22:11:05.230140+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:11:05.230140+0000","last_undegraded":"2024-09-15T22:11:05.230140+0000","last_fullsized":"2024-09-15T22:11:05.230140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234240+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:11:05.234240+0000","last_peered":"2024-09-15T22:11:05.234240+0000","last_clean":"2024-09-15T22:11:05.234240+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:11:05.234240+0000","last_undegraded":"2024-09-15T22:11:05.234240+0000","last_fullsized":"2024-09-15T22:11:05.234240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233732+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:11:05.233732+0000","last_peered":"2024-09-15T22:11:05.233732+0000","last_clean":"2024-09-15T22:11:05.233732+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:11:05.233732+0000","last_undegraded":"2024-09-15T22:11:05.233732+0000","last_fullsized":"2024-09-15T22:11:05.233732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235123+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:11:05.235123+0000","last_peered":"2024-09-15T22:11:05.235123+0000","last_clean":"2024-09-15T22:11:05.235123+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:11:05.235123+0000","last_undegraded":"2024-09-15T22:11:05.235123+0000","last_fullsized":"2024-09-15T22:11:05.235123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235367+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:11:05.235367+0000","last_peered":"2024-09-15T22:11:05.235367+0000","last_clean":"2024-09-15T22:11:05.235367+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:11:05.235367+0000","last_undegraded":"2024-09-15T22:11:05.235367+0000","last_fullsized":"2024-09-15T22:11:05.235367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231581+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:11:05.231581+0000","last_peered":"2024-09-15T22:11:05.231581+0000","last_clean":"2024-09-15T22:11:05.231581+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:11:05.231581+0000","last_undegraded":"2024-09-15T22:11:05.231581+0000","last_fullsized":"2024-09-15T22:11:05.231581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233328+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:11:05.233328+0000","last_peered":"2024-09-15T22:11:05.233328+0000","last_clean":"2024-09-15T22:11:05.233328+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:11:05.233328+0000","last_undegraded":"2024-09-15T22:11:05.233328+0000","last_fullsized":"2024-09-15T22:11:05.233328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233419+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:11:05.233419+0000","last_peered":"2024-09-15T22:11:05.233419+0000","last_clean":"2024-09-15T22:11:05.233419+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:11:05.233419+0000","last_undegraded":"2024-09-15T22:11:05.233419+0000","last_fullsized":"2024-09-15T22:11:05.233419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232052+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:11:05.232052+0000","last_peered":"2024-09-15T22:11:05.232052+0000","last_clean":"2024-09-15T22:11:05.232052+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:11:05.232052+0000","last_undegraded":"2024-09-15T22:11:05.232052+0000","last_fullsized":"2024-09-15T22:11:05.232052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69677,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":359,"num_write":250,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280881,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1003520,"data_stored":505427,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743461,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1142784,"data_stored":1154207,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1024000,"data_stored":508948,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":505118,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1102122,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528378,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":500356,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1077248,"data_stored":1097311,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":504823,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":500064,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":564468,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1060864,"data_stored":563462,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":502205,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:11:35.072 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:11:35.321 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:35.754 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:35 smithi077 ceph-mon[30980]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:35.754 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:35 smithi077 ceph-mon[30980]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:35.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:35 smithi196 ceph-mon[35109]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:35.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:35 smithi196 ceph-mon[35109]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 -- 172.21.15.77:0/377731075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 msgr2=0x7f5f5010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 --2- 172.21.15.77:0/377731075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f5010d940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009cd0 tx=0x7f5f4002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 -- 172.21.15.77:0/377731075 shutdown_connections 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 --2- 172.21.15.77:0/377731075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f5010d940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 --2- 172.21.15.77:0/377731075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 --2- 172.21.15.77:0/377731075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 0x7f5f50106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 -- 172.21.15.77:0/377731075 >> 172.21.15.77:0/377731075 conn(0x7f5f500fc790 msgr2=0x7f5f500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 -- 172.21.15.77:0/377731075 shutdown_connections 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.832+0000 7f5f5877e640 1 -- 172.21.15.77:0/377731075 wait complete. 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 Processor -- start 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 -- start start 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 0x7f5f5019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50110ee0 con 0x7f5f50101410 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50110d60 con 0x7f5f50100a70 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5877e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111060 con 0x7f5f50106a70 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37460/0 (socket says 172.21.15.77:37460) 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 -- 172.21.15.77:0/1778123691 learned_addr learned my addr 172.21.15.77:0/1778123691 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5651a640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f56d1b640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 0x7f5f5019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 -- 172.21.15.77:0/1778123691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 msgr2=0x7f5f5019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 0x7f5f5019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 -- 172.21.15.77:0/1778123691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 msgr2=0x7f5f50198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 -- 172.21.15.77:0/1778123691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5019d3c0 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f5651a640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.833+0000 7f5f55d19640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b4a0 tx=0x7f5f4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.834+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44014070 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.834+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5019d6b0 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.834+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f44010af0 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.834+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44017d10 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.834+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a4f90 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.835+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f50100ed0 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.836+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5f44017070 con 0x7f5f50101410 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.836+0000 7f5f4f7fe640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 0x7f5f380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.836+0000 7f5f5651a640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 0x7f5f380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.836+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5f440cf170 con 0x7f5f50101410 2024-09-15T22:11:35.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.837+0000 7f5f5651a640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 0x7f5f380a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800abe0 tx=0x7f5f4800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:35.840+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f4409bae0 con 0x7f5f50101410 2024-09-15T22:11:35.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:35 smithi117 ceph-mon[35021]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:35.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:35 smithi117 ceph-mon[35021]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:36.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.004+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f50062630 con 0x7f5f380a32d0 2024-09-15T22:11:36.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.012+0000 7f5f4f7fe640 1 -- 172.21.15.77:0/1778123691 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340684 (secure 0 0 0) 0x7f5f50062630 con 0x7f5f380a32d0 2024-09-15T22:11:36.017 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:36.018 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 msgr2=0x7f5f380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 0x7f5f380a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800abe0 tx=0x7f5f4800a5c0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 msgr2=0x7f5f50198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b4a0 tx=0x7f5f4400b960 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 shutdown_connections 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f5f380a32d0 0x7f5f380a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f50106a70 0x7f5f5019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f50101410 0x7f5f50198950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 --2- 172.21.15.77:0/1778123691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f50100a70 0x7f5f50198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.015+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 >> 172.21.15.77:0/1778123691 conn(0x7f5f500fc790 msgr2=0x7f5f5006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:36.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.016+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 shutdown_connections 2024-09-15T22:11:36.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:36.016+0000 7f5f5877e640 1 -- 172.21.15.77:0/1778123691 wait complete. 2024-09-15T22:11:36.689 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-15T22:11:34.206496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660279,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":704,"num_read_kb":478,"num_write":433,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378160,"kb_used_data":12200,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563664,"statfs":{"total":1151940427776,"available":1151553191936,"internally_reserved":0,"allocated":12492800,"data_stored":8008511,"data_compressed":388296,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003062"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":102,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:31.820630+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:11:31.820630+0000","last_peered":"2024-09-15T22:11:31.820630+0000","last_clean":"2024-09-15T22:11:31.820630+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:11:31.820630+0000","last_undegraded":"2024-09-15T22:11:31.820630+0000","last_fullsized":"2024-09-15T22:11:31.820630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234594+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:11:05.234594+0000","last_peered":"2024-09-15T22:11:05.234594+0000","last_clean":"2024-09-15T22:11:05.234594+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:11:05.234594+0000","last_undegraded":"2024-09-15T22:11:05.234594+0000","last_fullsized":"2024-09-15T22:11:05.234594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237735+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:11:05.237735+0000","last_peered":"2024-09-15T22:11:05.237735+0000","last_clean":"2024-09-15T22:11:05.237735+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:11:05.237735+0000","last_undegraded":"2024-09-15T22:11:05.237735+0000","last_fullsized":"2024-09-15T22:11:05.237735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234804+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:11:05.234804+0000","last_peered":"2024-09-15T22:11:05.234804+0000","last_clean":"2024-09-15T22:11:05.234804+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:11:05.234804+0000","last_undegraded":"2024-09-15T22:11:05.234804+0000","last_fullsized":"2024-09-15T22:11:05.234804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233657+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:11:05.233657+0000","last_peered":"2024-09-15T22:11:05.233657+0000","last_clean":"2024-09-15T22:11:05.233657+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:11:05.233657+0000","last_undegraded":"2024-09-15T22:11:05.233657+0000","last_fullsized":"2024-09-15T22:11:05.233657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231120+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:11:05.231120+0000","last_peered":"2024-09-15T22:11:05.231120+0000","last_clean":"2024-09-15T22:11:05.231120+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:11:05.231120+0000","last_undegraded":"2024-09-15T22:11:05.231120+0000","last_fullsized":"2024-09-15T22:11:05.231120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236230+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:11:05.236230+0000","last_peered":"2024-09-15T22:11:05.236230+0000","last_clean":"2024-09-15T22:11:05.236230+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:11:05.236230+0000","last_undegraded":"2024-09-15T22:11:05.236230+0000","last_fullsized":"2024-09-15T22:11:05.236230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229647+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:11:05.229647+0000","last_peered":"2024-09-15T22:11:05.229647+0000","last_clean":"2024-09-15T22:11:05.229647+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:11:05.229647+0000","last_undegraded":"2024-09-15T22:11:05.229647+0000","last_fullsized":"2024-09-15T22:11:05.229647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232331+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:11:05.232331+0000","last_peered":"2024-09-15T22:11:05.232331+0000","last_clean":"2024-09-15T22:11:05.232331+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:11:05.232331+0000","last_undegraded":"2024-09-15T22:11:05.232331+0000","last_fullsized":"2024-09-15T22:11:05.232331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236447+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:11:05.236447+0000","last_peered":"2024-09-15T22:11:05.236447+0000","last_clean":"2024-09-15T22:11:05.236447+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:11:05.236447+0000","last_undegraded":"2024-09-15T22:11:05.236447+0000","last_fullsized":"2024-09-15T22:11:05.236447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.239018+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:11:05.239018+0000","last_peered":"2024-09-15T22:11:05.239018+0000","last_clean":"2024-09-15T22:11:05.239018+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:11:05.239018+0000","last_undegraded":"2024-09-15T22:11:05.239018+0000","last_fullsized":"2024-09-15T22:11:05.239018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231265+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:11:05.231265+0000","last_peered":"2024-09-15T22:11:05.231265+0000","last_clean":"2024-09-15T22:11:05.231265+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:11:05.231265+0000","last_undegraded":"2024-09-15T22:11:05.231265+0000","last_fullsized":"2024-09-15T22:11:05.231265+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233818+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:11:05.233818+0000","last_peered":"2024-09-15T22:11:05.233818+0000","last_clean":"2024-09-15T22:11:05.233818+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:11:05.233818+0000","last_undegraded":"2024-09-15T22:11:05.233818+0000","last_fullsized":"2024-09-15T22:11:05.233818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235563+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:11:05.235563+0000","last_peered":"2024-09-15T22:11:05.235563+0000","last_clean":"2024-09-15T22:11:05.235563+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:11:05.235563+0000","last_undegraded":"2024-09-15T22:11:05.235563+0000","last_fullsized":"2024-09-15T22:11:05.235563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231916+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:11:05.231916+0000","last_peered":"2024-09-15T22:11:05.231916+0000","last_clean":"2024-09-15T22:11:05.231916+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:11:05.231916+0000","last_undegraded":"2024-09-15T22:11:05.231916+0000","last_fullsized":"2024-09-15T22:11:05.231916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234342+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:11:05.234342+0000","last_peered":"2024-09-15T22:11:05.234342+0000","last_clean":"2024-09-15T22:11:05.234342+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:11:05.234342+0000","last_undegraded":"2024-09-15T22:11:05.234342+0000","last_fullsized":"2024-09-15T22:11:05.234342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.246212+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:11:05.246212+0000","last_peered":"2024-09-15T22:11:05.246212+0000","last_clean":"2024-09-15T22:11:05.246212+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:11:05.246212+0000","last_undegraded":"2024-09-15T22:11:05.246212+0000","last_fullsized":"2024-09-15T22:11:05.246212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231703+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:11:05.231703+0000","last_peered":"2024-09-15T22:11:05.231703+0000","last_clean":"2024-09-15T22:11:05.231703+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:11:05.231703+0000","last_undegraded":"2024-09-15T22:11:05.231703+0000","last_fullsized":"2024-09-15T22:11:05.231703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231059+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:11:05.231059+0000","last_peered":"2024-09-15T22:11:05.231059+0000","last_clean":"2024-09-15T22:11:05.231059+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:11:05.231059+0000","last_undegraded":"2024-09-15T22:11:05.231059+0000","last_fullsized":"2024-09-15T22:11:05.231059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234467+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:11:05.234467+0000","last_peered":"2024-09-15T22:11:05.234467+0000","last_clean":"2024-09-15T22:11:05.234467+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:11:05.234467+0000","last_undegraded":"2024-09-15T22:11:05.234467+0000","last_fullsized":"2024-09-15T22:11:05.234467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.244091+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:11:05.244091+0000","last_peered":"2024-09-15T22:11:05.244091+0000","last_clean":"2024-09-15T22:11:05.244091+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:11:05.244091+0000","last_undegraded":"2024-09-15T22:11:05.244091+0000","last_fullsized":"2024-09-15T22:11:05.244091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234631+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:11:05.234631+0000","last_peered":"2024-09-15T22:11:05.234631+0000","last_clean":"2024-09-15T22:11:05.234631+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:11:05.234631+0000","last_undegraded":"2024-09-15T22:11:05.234631+0000","last_fullsized":"2024-09-15T22:11:05.234631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229452+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:11:05.229452+0000","last_peered":"2024-09-15T22:11:05.229452+0000","last_clean":"2024-09-15T22:11:05.229452+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:11:05.229452+0000","last_undegraded":"2024-09-15T22:11:05.229452+0000","last_fullsized":"2024-09-15T22:11:05.229452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231973+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:11:05.231973+0000","last_peered":"2024-09-15T22:11:05.231973+0000","last_clean":"2024-09-15T22:11:05.231973+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:11:05.231973+0000","last_undegraded":"2024-09-15T22:11:05.231973+0000","last_fullsized":"2024-09-15T22:11:05.231973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233518+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:11:05.233518+0000","last_peered":"2024-09-15T22:11:05.233518+0000","last_clean":"2024-09-15T22:11:05.233518+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:11:05.233518+0000","last_undegraded":"2024-09-15T22:11:05.233518+0000","last_fullsized":"2024-09-15T22:11:05.233518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232463+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:11:05.232463+0000","last_peered":"2024-09-15T22:11:05.232463+0000","last_clean":"2024-09-15T22:11:05.232463+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:11:05.232463+0000","last_undegraded":"2024-09-15T22:11:05.232463+0000","last_fullsized":"2024-09-15T22:11:05.232463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237933+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:11:05.237933+0000","last_peered":"2024-09-15T22:11:05.237933+0000","last_clean":"2024-09-15T22:11:05.237933+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:11:05.237933+0000","last_undegraded":"2024-09-15T22:11:05.237933+0000","last_fullsized":"2024-09-15T22:11:05.237933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234774+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:11:05.234774+0000","last_peered":"2024-09-15T22:11:05.234774+0000","last_clean":"2024-09-15T22:11:05.234774+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:11:05.234774+0000","last_undegraded":"2024-09-15T22:11:05.234774+0000","last_fullsized":"2024-09-15T22:11:05.234774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293199+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:11:05.293199+0000","last_peered":"2024-09-15T22:11:05.293199+0000","last_clean":"2024-09-15T22:11:05.293199+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:11:05.293199+0000","last_undegraded":"2024-09-15T22:11:05.293199+0000","last_fullsized":"2024-09-15T22:11:05.293199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234136+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:11:05.234136+0000","last_peered":"2024-09-15T22:11:05.234136+0000","last_clean":"2024-09-15T22:11:05.234136+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:11:05.234136+0000","last_undegraded":"2024-09-15T22:11:05.234136+0000","last_fullsized":"2024-09-15T22:11:05.234136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229318+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:11:05.229318+0000","last_peered":"2024-09-15T22:11:05.229318+0000","last_clean":"2024-09-15T22:11:05.229318+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:11:05.229318+0000","last_undegraded":"2024-09-15T22:11:05.229318+0000","last_fullsized":"2024-09-15T22:11:05.229318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236291+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:11:05.236291+0000","last_peered":"2024-09-15T22:11:05.236291+0000","last_clean":"2024-09-15T22:11:05.236291+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:11:05.236291+0000","last_undegraded":"2024-09-15T22:11:05.236291+0000","last_fullsized":"2024-09-15T22:11:05.236291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.238076+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:11:05.238076+0000","last_peered":"2024-09-15T22:11:05.238076+0000","last_clean":"2024-09-15T22:11:05.238076+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:11:05.238076+0000","last_undegraded":"2024-09-15T22:11:05.238076+0000","last_fullsized":"2024-09-15T22:11:05.238076+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234836+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:11:05.234836+0000","last_peered":"2024-09-15T22:11:05.234836+0000","last_clean":"2024-09-15T22:11:05.234836+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:11:05.234836+0000","last_undegraded":"2024-09-15T22:11:05.234836+0000","last_fullsized":"2024-09-15T22:11:05.234836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292980+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:11:05.292980+0000","last_peered":"2024-09-15T22:11:05.292980+0000","last_clean":"2024-09-15T22:11:05.292980+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:11:05.292980+0000","last_undegraded":"2024-09-15T22:11:05.292980+0000","last_fullsized":"2024-09-15T22:11:05.292980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234798+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:11:05.234798+0000","last_peered":"2024-09-15T22:11:05.234798+0000","last_clean":"2024-09-15T22:11:05.234798+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:11:05.234798+0000","last_undegraded":"2024-09-15T22:11:05.234798+0000","last_fullsized":"2024-09-15T22:11:05.234798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294432+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:11:05.294432+0000","last_peered":"2024-09-15T22:11:05.294432+0000","last_clean":"2024-09-15T22:11:05.294432+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:11:05.294432+0000","last_undegraded":"2024-09-15T22:11:05.294432+0000","last_fullsized":"2024-09-15T22:11:05.294432+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.819740+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:11:30.819740+0000","last_peered":"2024-09-15T22:11:30.819740+0000","last_clean":"2024-09-15T22:11:30.819740+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:11:30.819740+0000","last_undegraded":"2024-09-15T22:11:30.819740+0000","last_fullsized":"2024-09-15T22:11:30.819740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229240+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:11:05.229240+0000","last_peered":"2024-09-15T22:11:05.229240+0000","last_clean":"2024-09-15T22:11:05.229240+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:11:05.229240+0000","last_undegraded":"2024-09-15T22:11:05.229240+0000","last_fullsized":"2024-09-15T22:11:05.229240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233447+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:11:05.233447+0000","last_peered":"2024-09-15T22:11:05.233447+0000","last_clean":"2024-09-15T22:11:05.233447+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:11:05.233447+0000","last_undegraded":"2024-09-15T22:11:05.233447+0000","last_fullsized":"2024-09-15T22:11:05.233447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:31.764100+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:11:31.764100+0000","last_peered":"2024-09-15T22:11:31.764100+0000","last_clean":"2024-09-15T22:11:31.764100+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:11:31.764100+0000","last_undegraded":"2024-09-15T22:11:31.764100+0000","last_fullsized":"2024-09-15T22:11:31.764100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229271+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:11:05.229271+0000","last_peered":"2024-09-15T22:11:05.229271+0000","last_clean":"2024-09-15T22:11:05.229271+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:11:05.229271+0000","last_undegraded":"2024-09-15T22:11:05.229271+0000","last_fullsized":"2024-09-15T22:11:05.229271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228542+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:11:05.228542+0000","last_peered":"2024-09-15T22:11:05.228542+0000","last_clean":"2024-09-15T22:11:05.228542+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:11:05.228542+0000","last_undegraded":"2024-09-15T22:11:05.228542+0000","last_fullsized":"2024-09-15T22:11:05.228542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233587+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:11:05.233587+0000","last_peered":"2024-09-15T22:11:05.233587+0000","last_clean":"2024-09-15T22:11:05.233587+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:11:05.233587+0000","last_undegraded":"2024-09-15T22:11:05.233587+0000","last_fullsized":"2024-09-15T22:11:05.233587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232889+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:11:05.232889+0000","last_peered":"2024-09-15T22:11:05.232889+0000","last_clean":"2024-09-15T22:11:05.232889+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:11:05.232889+0000","last_undegraded":"2024-09-15T22:11:05.232889+0000","last_fullsized":"2024-09-15T22:11:05.232889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234619+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:11:05.234619+0000","last_peered":"2024-09-15T22:11:05.234619+0000","last_clean":"2024-09-15T22:11:05.234619+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:11:05.234619+0000","last_undegraded":"2024-09-15T22:11:05.234619+0000","last_fullsized":"2024-09-15T22:11:05.234619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232689+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:11:05.232689+0000","last_peered":"2024-09-15T22:11:05.232689+0000","last_clean":"2024-09-15T22:11:05.232689+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:11:05.232689+0000","last_undegraded":"2024-09-15T22:11:05.232689+0000","last_fullsized":"2024-09-15T22:11:05.232689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233868+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:11:05.233868+0000","last_peered":"2024-09-15T22:11:05.233868+0000","last_clean":"2024-09-15T22:11:05.233868+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:11:05.233868+0000","last_undegraded":"2024-09-15T22:11:05.233868+0000","last_fullsized":"2024-09-15T22:11:05.233868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234395+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:11:05.234395+0000","last_peered":"2024-09-15T22:11:05.234395+0000","last_clean":"2024-09-15T22:11:05.234395+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:11:05.234395+0000","last_undegraded":"2024-09-15T22:11:05.234395+0000","last_fullsized":"2024-09-15T22:11:05.234395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234616+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:11:05.234616+0000","last_peered":"2024-09-15T22:11:05.234616+0000","last_clean":"2024-09-15T22:11:05.234616+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:11:05.234616+0000","last_undegraded":"2024-09-15T22:11:05.234616+0000","last_fullsized":"2024-09-15T22:11:05.234616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:29.762952+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:11:29.762952+0000","last_peered":"2024-09-15T22:11:29.762952+0000","last_clean":"2024-09-15T22:11:29.762952+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:11:29.762952+0000","last_undegraded":"2024-09-15T22:11:29.762952+0000","last_fullsized":"2024-09-15T22:11:29.762952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229375+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:11:05.229375+0000","last_peered":"2024-09-15T22:11:05.229375+0000","last_clean":"2024-09-15T22:11:05.229375+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:11:05.229375+0000","last_undegraded":"2024-09-15T22:11:05.229375+0000","last_fullsized":"2024-09-15T22:11:05.229375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294394+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:11:05.294394+0000","last_peered":"2024-09-15T22:11:05.294394+0000","last_clean":"2024-09-15T22:11:05.294394+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:11:05.294394+0000","last_undegraded":"2024-09-15T22:11:05.294394+0000","last_fullsized":"2024-09-15T22:11:05.294394+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234765+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:11:05.234765+0000","last_peered":"2024-09-15T22:11:05.234765+0000","last_clean":"2024-09-15T22:11:05.234765+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:11:05.234765+0000","last_undegraded":"2024-09-15T22:11:05.234765+0000","last_fullsized":"2024-09-15T22:11:05.234765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233017+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:11:05.233017+0000","last_peered":"2024-09-15T22:11:05.233017+0000","last_clean":"2024-09-15T22:11:05.233017+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:11:05.233017+0000","last_undegraded":"2024-09-15T22:11:05.233017+0000","last_fullsized":"2024-09-15T22:11:05.233017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234433+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:11:05.234433+0000","last_peered":"2024-09-15T22:11:05.234433+0000","last_clean":"2024-09-15T22:11:05.234433+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:11:05.234433+0000","last_undegraded":"2024-09-15T22:11:05.234433+0000","last_fullsized":"2024-09-15T22:11:05.234433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229428+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:11:05.229428+0000","last_peered":"2024-09-15T22:11:05.229428+0000","last_clean":"2024-09-15T22:11:05.229428+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:11:05.229428+0000","last_undegraded":"2024-09-15T22:11:05.229428+0000","last_fullsized":"2024-09-15T22:11:05.229428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232764+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:11:05.232764+0000","last_peered":"2024-09-15T22:11:05.232764+0000","last_clean":"2024-09-15T22:11:05.232764+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:11:05.232764+0000","last_undegraded":"2024-09-15T22:11:05.232764+0000","last_fullsized":"2024-09-15T22:11:05.232764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231617+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:11:05.231617+0000","last_peered":"2024-09-15T22:11:05.231617+0000","last_clean":"2024-09-15T22:11:05.231617+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:11:05.231617+0000","last_undegraded":"2024-09-15T22:11:05.231617+0000","last_fullsized":"2024-09-15T22:11:05.231617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236561+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:11:05.236561+0000","last_peered":"2024-09-15T22:11:05.236561+0000","last_clean":"2024-09-15T22:11:05.236561+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:11:05.236561+0000","last_undegraded":"2024-09-15T22:11:05.236561+0000","last_fullsized":"2024-09-15T22:11:05.236561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233407+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:11:05.233407+0000","last_peered":"2024-09-15T22:11:05.233407+0000","last_clean":"2024-09-15T22:11:05.233407+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:11:05.233407+0000","last_undegraded":"2024-09-15T22:11:05.233407+0000","last_fullsized":"2024-09-15T22:11:05.233407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234219+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:11:05.234219+0000","last_peered":"2024-09-15T22:11:05.234219+0000","last_clean":"2024-09-15T22:11:05.234219+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:11:05.234219+0000","last_undegraded":"2024-09-15T22:11:05.234219+0000","last_fullsized":"2024-09-15T22:11:05.234219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.256328+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:11:05.256328+0000","last_peered":"2024-09-15T22:11:05.256328+0000","last_clean":"2024-09-15T22:11:05.256328+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:11:05.256328+0000","last_undegraded":"2024-09-15T22:11:05.256328+0000","last_fullsized":"2024-09-15T22:11:05.256328+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229427+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:11:05.229427+0000","last_peered":"2024-09-15T22:11:05.229427+0000","last_clean":"2024-09-15T22:11:05.229427+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:11:05.229427+0000","last_undegraded":"2024-09-15T22:11:05.229427+0000","last_fullsized":"2024-09-15T22:11:05.229427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233526+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:11:05.233526+0000","last_peered":"2024-09-15T22:11:05.233526+0000","last_clean":"2024-09-15T22:11:05.233526+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:11:05.233526+0000","last_undegraded":"2024-09-15T22:11:05.233526+0000","last_fullsized":"2024-09-15T22:11:05.233526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.763316+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:11:30.763316+0000","last_peered":"2024-09-15T22:11:30.763316+0000","last_clean":"2024-09-15T22:11:30.763316+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:11:30.763316+0000","last_undegraded":"2024-09-15T22:11:30.763316+0000","last_fullsized":"2024-09-15T22:11:30.763316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293135+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:11:05.293135+0000","last_peered":"2024-09-15T22:11:05.293135+0000","last_clean":"2024-09-15T22:11:05.293135+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:11:05.293135+0000","last_undegraded":"2024-09-15T22:11:05.293135+0000","last_fullsized":"2024-09-15T22:11:05.293135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233971+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:11:05.233971+0000","last_peered":"2024-09-15T22:11:05.233971+0000","last_clean":"2024-09-15T22:11:05.233971+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:11:05.233971+0000","last_undegraded":"2024-09-15T22:11:05.233971+0000","last_fullsized":"2024-09-15T22:11:05.233971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234113+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:11:05.234113+0000","last_peered":"2024-09-15T22:11:05.234113+0000","last_clean":"2024-09-15T22:11:05.234113+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:11:05.234113+0000","last_undegraded":"2024-09-15T22:11:05.234113+0000","last_fullsized":"2024-09-15T22:11:05.234113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231197+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:11:05.231197+0000","last_peered":"2024-09-15T22:11:05.231197+0000","last_clean":"2024-09-15T22:11:05.231197+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:11:05.231197+0000","last_undegraded":"2024-09-15T22:11:05.231197+0000","last_fullsized":"2024-09-15T22:11:05.231197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228802+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:11:05.228802+0000","last_peered":"2024-09-15T22:11:05.228802+0000","last_clean":"2024-09-15T22:11:05.228802+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:11:05.228802+0000","last_undegraded":"2024-09-15T22:11:05.228802+0000","last_fullsized":"2024-09-15T22:11:05.228802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236710+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:11:05.236710+0000","last_peered":"2024-09-15T22:11:05.236710+0000","last_clean":"2024-09-15T22:11:05.236710+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:11:05.236710+0000","last_undegraded":"2024-09-15T22:11:05.236710+0000","last_fullsized":"2024-09-15T22:11:05.236710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233559+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:11:05.233559+0000","last_peered":"2024-09-15T22:11:05.233559+0000","last_clean":"2024-09-15T22:11:05.233559+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:11:05.233559+0000","last_undegraded":"2024-09-15T22:11:05.233559+0000","last_fullsized":"2024-09-15T22:11:05.233559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236034+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:11:05.236034+0000","last_peered":"2024-09-15T22:11:05.236034+0000","last_clean":"2024-09-15T22:11:05.236034+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:11:05.236034+0000","last_undegraded":"2024-09-15T22:11:05.236034+0000","last_fullsized":"2024-09-15T22:11:05.236034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:28.760549+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:11:28.760549+0000","last_peered":"2024-09-15T22:11:28.760549+0000","last_clean":"2024-09-15T22:11:28.760549+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:11:28.760549+0000","last_undegraded":"2024-09-15T22:11:28.760549+0000","last_fullsized":"2024-09-15T22:11:28.760549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":116,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292722+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:11:05.292722+0000","last_peered":"2024-09-15T22:11:05.292722+0000","last_clean":"2024-09-15T22:11:05.292722+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:11:05.292722+0000","last_undegraded":"2024-09-15T22:11:05.292722+0000","last_fullsized":"2024-09-15T22:11:05.292722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229836+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:11:05.229836+0000","last_peered":"2024-09-15T22:11:05.229836+0000","last_clean":"2024-09-15T22:11:05.229836+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:11:05.229836+0000","last_undegraded":"2024-09-15T22:11:05.229836+0000","last_fullsized":"2024-09-15T22:11:05.229836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234717+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:11:05.234717+0000","last_peered":"2024-09-15T22:11:05.234717+0000","last_clean":"2024-09-15T22:11:05.234717+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:11:05.234717+0000","last_undegraded":"2024-09-15T22:11:05.234717+0000","last_fullsized":"2024-09-15T22:11:05.234717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.239662+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:11:05.239662+0000","last_peered":"2024-09-15T22:11:05.239662+0000","last_clean":"2024-09-15T22:11:05.239662+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:11:05.239662+0000","last_undegraded":"2024-09-15T22:11:05.239662+0000","last_fullsized":"2024-09-15T22:11:05.239662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233686+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:11:05.233686+0000","last_peered":"2024-09-15T22:11:05.233686+0000","last_clean":"2024-09-15T22:11:05.233686+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:11:05.233686+0000","last_undegraded":"2024-09-15T22:11:05.233686+0000","last_fullsized":"2024-09-15T22:11:05.233686+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234498+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:11:05.234498+0000","last_peered":"2024-09-15T22:11:05.234498+0000","last_clean":"2024-09-15T22:11:05.234498+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:11:05.234498+0000","last_undegraded":"2024-09-15T22:11:05.234498+0000","last_fullsized":"2024-09-15T22:11:05.234498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.245652+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:11:05.245652+0000","last_peered":"2024-09-15T22:11:05.245652+0000","last_clean":"2024-09-15T22:11:05.245652+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:11:05.245652+0000","last_undegraded":"2024-09-15T22:11:05.245652+0000","last_fullsized":"2024-09-15T22:11:05.245652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230107+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:11:05.230107+0000","last_peered":"2024-09-15T22:11:05.230107+0000","last_clean":"2024-09-15T22:11:05.230107+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:11:05.230107+0000","last_undegraded":"2024-09-15T22:11:05.230107+0000","last_fullsized":"2024-09-15T22:11:05.230107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:28.760629+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:11:28.760629+0000","last_peered":"2024-09-15T22:11:28.760629+0000","last_clean":"2024-09-15T22:11:28.760629+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:11:28.760629+0000","last_undegraded":"2024-09-15T22:11:28.760629+0000","last_fullsized":"2024-09-15T22:11:28.760629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234273+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:11:05.234273+0000","last_peered":"2024-09-15T22:11:05.234273+0000","last_clean":"2024-09-15T22:11:05.234273+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:11:05.234273+0000","last_undegraded":"2024-09-15T22:11:05.234273+0000","last_fullsized":"2024-09-15T22:11:05.234273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233416+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:11:05.233416+0000","last_peered":"2024-09-15T22:11:05.233416+0000","last_clean":"2024-09-15T22:11:05.233416+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:11:05.233416+0000","last_undegraded":"2024-09-15T22:11:05.233416+0000","last_fullsized":"2024-09-15T22:11:05.233416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.292831+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:11:05.292831+0000","last_peered":"2024-09-15T22:11:05.292831+0000","last_clean":"2024-09-15T22:11:05.292831+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:11:05.292831+0000","last_undegraded":"2024-09-15T22:11:05.292831+0000","last_fullsized":"2024-09-15T22:11:05.292831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233600+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:11:05.233600+0000","last_peered":"2024-09-15T22:11:05.233600+0000","last_clean":"2024-09-15T22:11:05.233600+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:11:05.233600+0000","last_undegraded":"2024-09-15T22:11:05.233600+0000","last_fullsized":"2024-09-15T22:11:05.233600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231748+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:11:05.231748+0000","last_peered":"2024-09-15T22:11:05.231748+0000","last_clean":"2024-09-15T22:11:05.231748+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:11:05.231748+0000","last_undegraded":"2024-09-15T22:11:05.231748+0000","last_fullsized":"2024-09-15T22:11:05.231748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234250+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:11:05.234250+0000","last_peered":"2024-09-15T22:11:05.234250+0000","last_clean":"2024-09-15T22:11:05.234250+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:11:05.234250+0000","last_undegraded":"2024-09-15T22:11:05.234250+0000","last_fullsized":"2024-09-15T22:11:05.234250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228574+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:11:05.228574+0000","last_peered":"2024-09-15T22:11:05.228574+0000","last_clean":"2024-09-15T22:11:05.228574+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:11:05.228574+0000","last_undegraded":"2024-09-15T22:11:05.228574+0000","last_fullsized":"2024-09-15T22:11:05.228574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235257+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:11:05.235257+0000","last_peered":"2024-09-15T22:11:05.235257+0000","last_clean":"2024-09-15T22:11:05.235257+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:11:05.235257+0000","last_undegraded":"2024-09-15T22:11:05.235257+0000","last_fullsized":"2024-09-15T22:11:05.235257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234474+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:11:05.234474+0000","last_peered":"2024-09-15T22:11:05.234474+0000","last_clean":"2024-09-15T22:11:05.234474+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:11:05.234474+0000","last_undegraded":"2024-09-15T22:11:05.234474+0000","last_fullsized":"2024-09-15T22:11:05.234474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236033+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:11:05.236033+0000","last_peered":"2024-09-15T22:11:05.236033+0000","last_clean":"2024-09-15T22:11:05.236033+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:11:05.236033+0000","last_undegraded":"2024-09-15T22:11:05.236033+0000","last_fullsized":"2024-09-15T22:11:05.236033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:30.764312+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:11:30.764312+0000","last_peered":"2024-09-15T22:11:30.764312+0000","last_clean":"2024-09-15T22:11:30.764312+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:11:30.764312+0000","last_undegraded":"2024-09-15T22:11:30.764312+0000","last_fullsized":"2024-09-15T22:11:30.764312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234982+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:11:05.234982+0000","last_peered":"2024-09-15T22:11:05.234982+0000","last_clean":"2024-09-15T22:11:05.234982+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:11:05.234982+0000","last_undegraded":"2024-09-15T22:11:05.234982+0000","last_fullsized":"2024-09-15T22:11:05.234982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236649+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:11:05.236649+0000","last_peered":"2024-09-15T22:11:05.236649+0000","last_clean":"2024-09-15T22:11:05.236649+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:11:05.236649+0000","last_undegraded":"2024-09-15T22:11:05.236649+0000","last_fullsized":"2024-09-15T22:11:05.236649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230947+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:11:05.230947+0000","last_peered":"2024-09-15T22:11:05.230947+0000","last_clean":"2024-09-15T22:11:05.230947+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:11:05.230947+0000","last_undegraded":"2024-09-15T22:11:05.230947+0000","last_fullsized":"2024-09-15T22:11:05.230947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.294107+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:11:05.294107+0000","last_peered":"2024-09-15T22:11:05.294107+0000","last_clean":"2024-09-15T22:11:05.294107+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:11:05.294107+0000","last_undegraded":"2024-09-15T22:11:05.294107+0000","last_fullsized":"2024-09-15T22:11:05.294107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.237700+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:11:05.237700+0000","last_peered":"2024-09-15T22:11:05.237700+0000","last_clean":"2024-09-15T22:11:05.237700+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:11:05.237700+0000","last_undegraded":"2024-09-15T22:11:05.237700+0000","last_fullsized":"2024-09-15T22:11:05.237700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233137+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:11:05.233137+0000","last_peered":"2024-09-15T22:11:05.233137+0000","last_clean":"2024-09-15T22:11:05.233137+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:11:05.233137+0000","last_undegraded":"2024-09-15T22:11:05.233137+0000","last_fullsized":"2024-09-15T22:11:05.233137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229232+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:11:05.229232+0000","last_peered":"2024-09-15T22:11:05.229232+0000","last_clean":"2024-09-15T22:11:05.229232+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:11:05.229232+0000","last_undegraded":"2024-09-15T22:11:05.229232+0000","last_fullsized":"2024-09-15T22:11:05.229232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231708+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:11:05.231708+0000","last_peered":"2024-09-15T22:11:05.231708+0000","last_clean":"2024-09-15T22:11:05.231708+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:11:05.231708+0000","last_undegraded":"2024-09-15T22:11:05.231708+0000","last_fullsized":"2024-09-15T22:11:05.231708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236950+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:11:05.236950+0000","last_peered":"2024-09-15T22:11:05.236950+0000","last_clean":"2024-09-15T22:11:05.236950+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:11:05.236950+0000","last_undegraded":"2024-09-15T22:11:05.236950+0000","last_fullsized":"2024-09-15T22:11:05.236950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234733+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:11:05.234733+0000","last_peered":"2024-09-15T22:11:05.234733+0000","last_clean":"2024-09-15T22:11:05.234733+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:11:05.234733+0000","last_undegraded":"2024-09-15T22:11:05.234733+0000","last_fullsized":"2024-09-15T22:11:05.234733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233330+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:11:05.233330+0000","last_peered":"2024-09-15T22:11:05.233330+0000","last_clean":"2024-09-15T22:11:05.233330+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:11:05.233330+0000","last_undegraded":"2024-09-15T22:11:05.233330+0000","last_fullsized":"2024-09-15T22:11:05.233330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234371+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:11:05.234371+0000","last_peered":"2024-09-15T22:11:05.234371+0000","last_clean":"2024-09-15T22:11:05.234371+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:11:05.234371+0000","last_undegraded":"2024-09-15T22:11:05.234371+0000","last_fullsized":"2024-09-15T22:11:05.234371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229709+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:11:05.229709+0000","last_peered":"2024-09-15T22:11:05.229709+0000","last_clean":"2024-09-15T22:11:05.229709+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:11:05.229709+0000","last_undegraded":"2024-09-15T22:11:05.229709+0000","last_fullsized":"2024-09-15T22:11:05.229709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229024+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:11:05.229024+0000","last_peered":"2024-09-15T22:11:05.229024+0000","last_clean":"2024-09-15T22:11:05.229024+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:11:05.229024+0000","last_undegraded":"2024-09-15T22:11:05.229024+0000","last_fullsized":"2024-09-15T22:11:05.229024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231312+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:11:05.231312+0000","last_peered":"2024-09-15T22:11:05.231312+0000","last_clean":"2024-09-15T22:11:05.231312+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:11:05.231312+0000","last_undegraded":"2024-09-15T22:11:05.231312+0000","last_fullsized":"2024-09-15T22:11:05.231312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.236840+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:11:05.236840+0000","last_peered":"2024-09-15T22:11:05.236840+0000","last_clean":"2024-09-15T22:11:05.236840+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:11:05.236840+0000","last_undegraded":"2024-09-15T22:11:05.236840+0000","last_fullsized":"2024-09-15T22:11:05.236840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.293170+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:11:05.293170+0000","last_peered":"2024-09-15T22:11:05.293170+0000","last_clean":"2024-09-15T22:11:05.293170+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:11:05.293170+0000","last_undegraded":"2024-09-15T22:11:05.293170+0000","last_fullsized":"2024-09-15T22:11:05.293170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232141+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:11:05.232141+0000","last_peered":"2024-09-15T22:11:05.232141+0000","last_clean":"2024-09-15T22:11:05.232141+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:11:05.232141+0000","last_undegraded":"2024-09-15T22:11:05.232141+0000","last_fullsized":"2024-09-15T22:11:05.232141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.229299+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:11:05.229299+0000","last_peered":"2024-09-15T22:11:05.229299+0000","last_clean":"2024-09-15T22:11:05.229299+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:11:05.229299+0000","last_undegraded":"2024-09-15T22:11:05.229299+0000","last_fullsized":"2024-09-15T22:11:05.229299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233247+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:11:05.233247+0000","last_peered":"2024-09-15T22:11:05.233247+0000","last_clean":"2024-09-15T22:11:05.233247+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:11:05.233247+0000","last_undegraded":"2024-09-15T22:11:05.233247+0000","last_fullsized":"2024-09-15T22:11:05.233247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232869+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:11:05.232869+0000","last_peered":"2024-09-15T22:11:05.232869+0000","last_clean":"2024-09-15T22:11:05.232869+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:11:05.232869+0000","last_undegraded":"2024-09-15T22:11:05.232869+0000","last_fullsized":"2024-09-15T22:11:05.232869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.228937+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:11:05.228937+0000","last_peered":"2024-09-15T22:11:05.228937+0000","last_clean":"2024-09-15T22:11:05.228937+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:11:05.228937+0000","last_undegraded":"2024-09-15T22:11:05.228937+0000","last_fullsized":"2024-09-15T22:11:05.228937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232601+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:11:05.232601+0000","last_peered":"2024-09-15T22:11:05.232601+0000","last_clean":"2024-09-15T22:11:05.232601+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:11:05.232601+0000","last_undegraded":"2024-09-15T22:11:05.232601+0000","last_fullsized":"2024-09-15T22:11:05.232601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234512+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:11:05.234512+0000","last_peered":"2024-09-15T22:11:05.234512+0000","last_clean":"2024-09-15T22:11:05.234512+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:11:05.234512+0000","last_undegraded":"2024-09-15T22:11:05.234512+0000","last_fullsized":"2024-09-15T22:11:05.234512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235637+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:11:05.235637+0000","last_peered":"2024-09-15T22:11:05.235637+0000","last_clean":"2024-09-15T22:11:05.235637+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:11:05.235637+0000","last_undegraded":"2024-09-15T22:11:05.235637+0000","last_fullsized":"2024-09-15T22:11:05.235637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.230140+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:11:05.230140+0000","last_peered":"2024-09-15T22:11:05.230140+0000","last_clean":"2024-09-15T22:11:05.230140+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:11:05.230140+0000","last_undegraded":"2024-09-15T22:11:05.230140+0000","last_fullsized":"2024-09-15T22:11:05.230140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.234240+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:11:05.234240+0000","last_peered":"2024-09-15T22:11:05.234240+0000","last_clean":"2024-09-15T22:11:05.234240+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:11:05.234240+0000","last_undegraded":"2024-09-15T22:11:05.234240+0000","last_fullsized":"2024-09-15T22:11:05.234240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233732+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:11:05.233732+0000","last_peered":"2024-09-15T22:11:05.233732+0000","last_clean":"2024-09-15T22:11:05.233732+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:11:05.233732+0000","last_undegraded":"2024-09-15T22:11:05.233732+0000","last_fullsized":"2024-09-15T22:11:05.233732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235123+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:11:05.235123+0000","last_peered":"2024-09-15T22:11:05.235123+0000","last_clean":"2024-09-15T22:11:05.235123+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:11:05.235123+0000","last_undegraded":"2024-09-15T22:11:05.235123+0000","last_fullsized":"2024-09-15T22:11:05.235123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.235367+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:11:05.235367+0000","last_peered":"2024-09-15T22:11:05.235367+0000","last_clean":"2024-09-15T22:11:05.235367+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:11:05.235367+0000","last_undegraded":"2024-09-15T22:11:05.235367+0000","last_fullsized":"2024-09-15T22:11:05.235367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.231581+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:11:05.231581+0000","last_peered":"2024-09-15T22:11:05.231581+0000","last_clean":"2024-09-15T22:11:05.231581+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:11:05.231581+0000","last_undegraded":"2024-09-15T22:11:05.231581+0000","last_fullsized":"2024-09-15T22:11:05.231581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233328+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:11:05.233328+0000","last_peered":"2024-09-15T22:11:05.233328+0000","last_clean":"2024-09-15T22:11:05.233328+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:11:05.233328+0000","last_undegraded":"2024-09-15T22:11:05.233328+0000","last_fullsized":"2024-09-15T22:11:05.233328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.233419+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:11:05.233419+0000","last_peered":"2024-09-15T22:11:05.233419+0000","last_clean":"2024-09-15T22:11:05.233419+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:11:05.233419+0000","last_undegraded":"2024-09-15T22:11:05.233419+0000","last_fullsized":"2024-09-15T22:11:05.233419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:11:05.232052+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:11:05.232052+0000","last_peered":"2024-09-15T22:11:05.232052+0000","last_clean":"2024-09-15T22:11:05.232052+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:11:05.232052+0000","last_undegraded":"2024-09-15T22:11:05.232052+0000","last_fullsized":"2024-09-15T22:11:05.232052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69677,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":359,"num_write":250,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280881,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1003520,"data_stored":505427,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743461,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1142784,"data_stored":1154207,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1024000,"data_stored":508948,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1007616,"data_stored":505118,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":1102122,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528378,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":995328,"data_stored":500356,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1077248,"data_stored":1097311,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":504823,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":500064,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":564468,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1060864,"data_stored":563462,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":502205,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34087,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:11:36.694 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:11:36.694 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T22:11:36.694 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-15T22:11:36.695 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-15T22:11:36.940 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:37.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 -- 172.21.15.77:0/2920810931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268102790 msgr2=0x7f72681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:37.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 --2- 172.21.15.77:0/2920810931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268102790 0x7f72681067e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f725c009900 tx=0x7f725c02a410 comp rx=0 tx=0).stop 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 -- 172.21.15.77:0/2920810931 shutdown_connections 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 --2- 172.21.15.77:0/2920810931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268106f10 0x7f726810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 --2- 172.21.15.77:0/2920810931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268102790 0x7f72681067e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 --2- 172.21.15.77:0/2920810931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268101de0 0x7f72681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.456+0000 7f726e02f640 1 -- 172.21.15.77:0/2920810931 >> 172.21.15.77:0/2920810931 conn(0x7f72680fd8c0 msgr2=0x7f72680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- 172.21.15.77:0/2920810931 shutdown_connections 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- 172.21.15.77:0/2920810931 wait complete. 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 Processor -- start 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- start start 2024-09-15T22:11:37.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268102790 0x7f72681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268106f10 0x7f726819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7268111f70 con 0x7f7268101de0 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7268111df0 con 0x7f7268106f10 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.457+0000 7f726e02f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72681120f0 con 0x7f7268102790 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37478/0 (socket says 172.21.15.77:37478) 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 -- 172.21.15.77:0/4257956180 learned_addr learned my addr 172.21.15.77:0/4257956180 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f7266ffd640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268102790 0x7f72681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:37.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f7267fff640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268106f10 0x7f726819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 -- 172.21.15.77:0/4257956180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268102790 msgr2=0x7f72681988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268102790 0x7f72681988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 -- 172.21.15.77:0/4257956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268106f10 msgr2=0x7f726819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268106f10 0x7f726819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 -- 172.21.15.77:0/4257956180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726819d310 con 0x7f7268101de0 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f72677fe640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f725800c920 tx=0x7f725800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258007c70 con 0x7f7268101de0 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258007e10 con 0x7f7268101de0 2024-09-15T22:11:37.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726819d600 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258018890 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.458+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681a4ee0 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.459+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72681021c0 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.460+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7258018a70 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.461+0000 7f7264ff9640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 0x7f72500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.461+0000 7f7266ffd640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 0x7f72500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.461+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f72580cff50 con 0x7f7268101de0 2024-09-15T22:11:37.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.462+0000 7f7266ffd640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 0x7f72500a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f725c009230 tx=0x7f725c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:37.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.466+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7258014030 con 0x7f7268101de0 2024-09-15T22:11:37.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.664+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f726802d7c0 con 0x7f7268101de0 2024-09-15T22:11:37.666 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:37 smithi077 ceph-mon[30980]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:37.666 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:37 smithi077 ceph-mon[30980]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:37.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.664+0000 7f7264ff9640 1 -- 172.21.15.77:0/4257956180 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f725809c7b0 con 0x7f7268101de0 2024-09-15T22:11:37.667 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:37.667 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 msgr2=0x7f72500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 0x7f72500a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f725c009230 tx=0x7f725c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 msgr2=0x7f7268198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f725800c920 tx=0x7f725800cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 shutdown_connections 2024-09-15T22:11:37.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f72500a32d0 0x7f72500a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7268106f10 0x7f726819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7268102790 0x7f72681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 --2- 172.21.15.77:0/4257956180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7268101de0 0x7f7268198360 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.666+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 >> 172.21.15.77:0/4257956180 conn(0x7f72680fd8c0 msgr2=0x7f72680fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.667+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 shutdown_connections 2024-09-15T22:11:37.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:37.667+0000 7f726e02f640 1 -- 172.21.15.77:0/4257956180 wait complete. 2024-09-15T22:11:37.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:37 smithi196 ceph-mon[35109]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:37.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:37 smithi196 ceph-mon[35109]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:37.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:37 smithi117 ceph-mon[35021]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:11:37.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:37 smithi117 ceph-mon[35021]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:38.306 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-15T22:11:38.306 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:38.552 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:38.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:38 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4257956180' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:11:38.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:38 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4257956180' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:11:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:38 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4257956180' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2664502854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 msgr2=0x7fd1f410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2664502854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f410e610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009ca0 tx=0x7fd1ec02a950 comp rx=0 tx=0).stop 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2664502854 shutdown_connections 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2664502854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f410e610 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2664502854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 0x7fd1f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2664502854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 0x7fd1f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2664502854 >> 172.21.15.77:0/2664502854 conn(0x7fd1f40fc7a0 msgr2=0x7fd1f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2664502854 shutdown_connections 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.026+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2664502854 wait complete. 2024-09-15T22:11:39.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 Processor -- start 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 -- start start 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 0x7fd1f41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 0x7fd1f4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111f30 con 0x7fd1f410c170 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111db0 con 0x7fd1f4103660 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.027+0000 7fd1fa2f6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41120b0 con 0x7fd1f4102d40 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36900/0 (socket says 172.21.15.77:36900) 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 -- 172.21.15.77:0/2522427933 learned_addr learned my addr 172.21.15.77:0/2522427933 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f37fe640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 0x7fd1f4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f3fff640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 0x7fd1f41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:39.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 -- 172.21.15.77:0/2522427933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 msgr2=0x7fd1f41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 0x7fd1f41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 -- 172.21.15.77:0/2522427933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 msgr2=0x7fd1f4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 0x7fd1f4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 -- 172.21.15.77:0/2522427933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f419d380 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f8893640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec02ae70 tx=0x7fd1ec02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec034040 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec030920 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f419d610 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec045480 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.028+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a4f50 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.029+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f406b070 con 0x7fd1f410c170 2024-09-15T22:11:39.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.030+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd1ec037020 con 0x7fd1f410c170 2024-09-15T22:11:39.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.031+0000 7fd1f17fa640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:39.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.031+0000 7fd1f3fff640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:39.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.031+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd1ec0f0d10 con 0x7fd1f410c170 2024-09-15T22:11:39.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.031+0000 7fd1f3fff640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40058c0 tx=0x7fd1e400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:39.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.035+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1ec0bd680 con 0x7fd1f410c170 2024-09-15T22:11:39.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.239+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1f4107210 con 0x7fd1f410c170 2024-09-15T22:11:39.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.240+0000 7fd1f17fa640 1 -- 172.21.15.77:0/2522427933 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7fd1ec0bcf80 con 0x7fd1f410c170 2024-09-15T22:11:39.242 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:39.242 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:11:39.242 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:11:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.242+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 msgr2=0x7fd1dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.242+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40058c0 tx=0x7fd1e400a4c0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.242+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 msgr2=0x7fd1f419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.242+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec02ae70 tx=0x7fd1ec02f5b0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 shutdown_connections 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fd1dc0a32d0 0x7fd1dc0a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1f410c170 0x7fd1f419cca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1f4103660 0x7fd1f4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 --2- 172.21.15.77:0/2522427933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102d40 0x7fd1f41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 >> 172.21.15.77:0/2522427933 conn(0x7fd1f40fc7a0 msgr2=0x7fd1f40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 shutdown_connections 2024-09-15T22:11:39.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:39.243+0000 7fd1fa2f6640 1 -- 172.21.15.77:0/2522427933 wait complete. 2024-09-15T22:11:39.553 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:39 smithi077 ceph-mon[30980]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:39.553 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:39 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2522427933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:39.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:39 smithi196 ceph-mon[35109]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:39.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:39 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2522427933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:39 smithi117 ceph-mon[35021]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:39 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2522427933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:39.993 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:40.242 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 -- 172.21.15.77:0/2495510258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc106a80 msgr2=0x7f01dc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/2495510258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc106a80 0x7f01dc106e60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009930 tx=0x7f01cc02a440 comp rx=0 tx=0).stop 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 -- 172.21.15.77:0/2495510258 shutdown_connections 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/2495510258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc101420 0x7f01dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/2495510258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc100a80 0x7f01dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/2495510258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc106a80 0x7f01dc106e60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 -- 172.21.15.77:0/2495510258 >> 172.21.15.77:0/2495510258 conn(0x7f01dc0fc7a0 msgr2=0x7f01dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 -- 172.21.15.77:0/2495510258 shutdown_connections 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.737+0000 7f01e0ff6640 1 -- 172.21.15.77:0/2495510258 wait complete. 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 Processor -- start 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 -- start start 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc100a80 0x7f01dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc106a80 0x7f01dc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc111f60 con 0x7f01dc101420 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc111de0 con 0x7f01dc100a80 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01e0ff6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc1120e0 con 0x7f01dc106a80 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36926/0 (socket says 172.21.15.77:36926) 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 -- 172.21.15.77:0/770987449 learned_addr learned my addr 172.21.15.77:0/770987449 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01dad9d640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc106a80 0x7f01dc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 -- 172.21.15.77:0/770987449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc106a80 msgr2=0x7f01dc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc106a80 0x7f01dc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 -- 172.21.15.77:0/770987449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc100a80 msgr2=0x7f01dc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc100a80 0x7f01dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.738+0000 7f01d9d9b640 1 -- 172.21.15.77:0/770987449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc19d320 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01d9d9b640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c920 tx=0x7f01c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4007c70 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c4007e10 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc19d610 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4018890 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.739+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1a4ef0 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.740+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc100ee0 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.741+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f01c4018a50 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.741+0000 7f01d37fe640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 0x7f01c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.742+0000 7f01da59c640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 0x7f01c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.742+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f01c40d02a0 con 0x7f01dc101420 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.742+0000 7f01da59c640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 0x7f01c80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02a960 tx=0x7f01cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.745+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01c4014030 con 0x7f01dc101420 2024-09-15T22:11:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.945+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01dc1996d0 con 0x7f01dc101420 2024-09-15T22:11:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.946+0000 7f01d37fe640 1 -- 172.21.15.77:0/770987449 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f01dc1996d0 con 0x7f01dc101420 2024-09-15T22:11:40.948 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:40.948 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:11:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 msgr2=0x7f01c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 0x7f01c80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01cc02a960 tx=0x7f01cc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 msgr2=0x7f01dc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c920 tx=0x7f01c400cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 shutdown_connections 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f01c80a32d0 0x7f01c80a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01dc106a80 0x7f01dc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01dc101420 0x7f01dc1988b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 --2- 172.21.15.77:0/770987449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01dc100a80 0x7f01dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.948+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 >> 172.21.15.77:0/770987449 conn(0x7f01dc0fc7a0 msgr2=0x7f01dc0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.949+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 shutdown_connections 2024-09-15T22:11:40.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:40.949+0000 7f01e0ff6640 1 -- 172.21.15.77:0/770987449 wait complete. 2024-09-15T22:11:41.731 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:41 smithi077 ceph-mon[30980]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:41.732 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:41 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/770987449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:41.760 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:11:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:41 smithi196 ceph-mon[35109]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:41.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:41 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/770987449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:41 smithi117 ceph-mon[35021]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:41 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/770987449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.521+0000 7f35e6e80640 1 -- 172.21.15.77:0/4060438467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 msgr2=0x7f35e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.521+0000 7f35e6e80640 1 --2- 172.21.15.77:0/4060438467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e0101200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009900 tx=0x7f35dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- 172.21.15.77:0/4060438467 shutdown_connections 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- 172.21.15.77:0/4060438467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- 172.21.15.77:0/4060438467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 0x7f35e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- 172.21.15.77:0/4060438467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e0101200 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- 172.21.15.77:0/4060438467 >> 172.21.15.77:0/4060438467 conn(0x7f35e00fc7d0 msgr2=0x7f35e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- 172.21.15.77:0/4060438467 shutdown_connections 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- 172.21.15.77:0/4060438467 wait complete. 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 Processor -- start 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- start start 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 0x7f35e0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:42.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0110f80 con 0x7f35e0100e20 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0110e00 con 0x7f35e0102ef0 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.522+0000 7f35e6e80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0111100 con 0x7f35e0102570 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36940/0 (socket says 172.21.15.77:36940) 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 -- 172.21.15.77:0/3440841158 learned_addr learned my addr 172.21.15.77:0/3440841158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35d7fff640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 0x7f35e0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e541d640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:42.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 -- 172.21.15.77:0/3440841158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 msgr2=0x7f35e0197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 0x7f35e0197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 -- 172.21.15.77:0/3440841158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 msgr2=0x7f35e0197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e0197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 -- 172.21.15.77:0/3440841158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01a6a30 con 0x7f35e0100e20 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e541d640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e4c1c640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f35dc006010 tx=0x7f35dc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc03f070 con 0x7f35e0100e20 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01a6cc0 con 0x7f35e0100e20 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35dc031c30 con 0x7f35e0100e20 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc035030 con 0x7f35e0100e20 2024-09-15T22:11:42.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.523+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01a71a0 con 0x7f35e0100e20 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.524+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0101200 con 0x7f35e0100e20 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.525+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f35dc042070 con 0x7f35e0100e20 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.526+0000 7f35d5ffb640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 0x7f35c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.526+0000 7f35d7fff640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 0x7f35c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.527+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f35dc0f0590 con 0x7f35e0100e20 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.527+0000 7f35d7fff640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 0x7f35c80a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005fd0 tx=0x7f35d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:42.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.531+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35dc0bcdf0 con 0x7f35e0100e20 2024-09-15T22:11:42.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.698+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35e0109090 con 0x7f35e0100e20 2024-09-15T22:11:42.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.700+0000 7f35d5ffb640 1 -- 172.21.15.77:0/3440841158 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24775 (secure 0 0 0) 0x7f35dc0bc6f0 con 0x7f35e0100e20 2024-09-15T22:11:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:42.702 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":90,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:11:05.224880+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:11:42.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 msgr2=0x7f35c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:42.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 0x7f35c80a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005fd0 tx=0x7f35d0005850 comp rx=0 tx=0).stop 2024-09-15T22:11:42.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 msgr2=0x7f35e01971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:42.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f35dc006010 tx=0x7f35dc003c10 comp rx=0 tx=0).stop 2024-09-15T22:11:42.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 shutdown_connections 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f35c80a32d0 0x7f35c80a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e0102ef0 0x7f35e0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35e0102570 0x7f35e0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 --2- 172.21.15.77:0/3440841158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e0100e20 0x7f35e01971c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 >> 172.21.15.77:0/3440841158 conn(0x7f35e00fc7d0 msgr2=0x7f35e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 shutdown_connections 2024-09-15T22:11:42.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:42.702+0000 7f35e6e80640 1 -- 172.21.15.77:0/3440841158 wait complete. 2024-09-15T22:11:43.516 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:43.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:43 smithi196 ceph-mon[35109]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:43.656 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:43 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3440841158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:43.664 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:43 smithi117 ceph-mon[35021]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:43.665 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:43 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3440841158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:43.763 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:43.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:43 smithi077 ceph-mon[30980]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:43.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:43 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3440841158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:11:44.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 -- 172.21.15.77:0/1060872376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 msgr2=0x7faa50100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 --2- 172.21.15.77:0/1060872376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50100ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faa40009930 tx=0x7faa4002a420 comp rx=0 tx=0).stop 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 -- 172.21.15.77:0/1060872376 shutdown_connections 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 --2- 172.21.15.77:0/1060872376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 --2- 172.21.15.77:0/1060872376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50100ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 --2- 172.21.15.77:0/1060872376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 -- 172.21.15.77:0/1060872376 >> 172.21.15.77:0/1060872376 conn(0x7faa500fc7a0 msgr2=0x7faa500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 -- 172.21.15.77:0/1060872376 shutdown_connections 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.274+0000 7faa57fe2640 1 -- 172.21.15.77:0/1060872376 wait complete. 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 Processor -- start 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 -- start start 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:44.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa50197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa50110ea0 con 0x7faa50100a80 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa50110d20 con 0x7faa50106a80 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa57fe2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa50111020 con 0x7faa50101420 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa55d7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa55d7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36954/0 (socket says 172.21.15.77:36954) 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa55d7e640 1 -- 172.21.15.77:0/4121530541 learned_addr learned my addr 172.21.15.77:0/4121530541 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.275+0000 7faa5657f640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa5557d640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa50197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 -- 172.21.15.77:0/4121530541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 msgr2=0x7faa50197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa50197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 -- 172.21.15.77:0/4121530541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 msgr2=0x7faa50197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 -- 172.21.15.77:0/4121530541 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa501a6980 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa5557d640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa50197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa5657f640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa55d7e640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d950 tx=0x7faa4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c017070 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa501a6c70 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4c009c30 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00f6b0 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.276+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a71b0 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.277+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50100ee0 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.278+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faa4c0109d0 con 0x7faa50100a80 2024-09-15T22:11:44.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.279+0000 7faa46ffd640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 0x7faa3c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:44.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.279+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7faa4c0cfb20 con 0x7faa50100a80 2024-09-15T22:11:44.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.279+0000 7faa5557d640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 0x7faa3c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:44.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.279+0000 7faa5557d640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 0x7faa3c0a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa40000c00 tx=0x7faa40032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:44.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.283+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa4c009de0 con 0x7faa50100a80 2024-09-15T22:11:44.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.485+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa5002d7c0 con 0x7faa50100a80 2024-09-15T22:11:44.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.486+0000 7faa46ffd640 1 -- 172.21.15.77:0/4121530541 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7faa4c09c320 con 0x7faa50100a80 2024-09-15T22:11:44.488 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:11:44.488 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:11:44.488 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 msgr2=0x7faa3c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 0x7faa3c0a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa40000c00 tx=0x7faa40032c70 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 msgr2=0x7faa50197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00d950 tx=0x7faa4c00de10 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 shutdown_connections 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7faa3c0a32d0 0x7faa3c0a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50106a80 0x7faa50197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa50101420 0x7faa50197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 --2- 172.21.15.77:0/4121530541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa50100a80 0x7faa50197110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 >> 172.21.15.77:0/4121530541 conn(0x7faa500fc7a0 msgr2=0x7faa500fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:44.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 shutdown_connections 2024-09-15T22:11:44.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:44.488+0000 7faa57fe2640 1 -- 172.21.15.77:0/4121530541 wait complete. 2024-09-15T22:11:45.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:09:55.819962+0000', 'modified': '2024-09-15T22:10:40.090210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24458, 'mds_1': 24440, 'mds_2': 24439, 'mds_3': 24424, 'mds_4': 14604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/2629133747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2629133747}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2629133747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.77:6841/3032989520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 3032989520}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 3032989520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/3132990127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3132990127}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3132990127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6833/2823149773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 2823149773}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 2823149773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/889715770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 889715770}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 889715770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/3112549785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3112549785}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3112549785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/3513024753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3513024753}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3513024753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6839/1624290607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1624290607}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1624290607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.117:6837/3263285576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3263285576}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3263285576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/1252344498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1252344498}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1252344498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24458, 'qdb_cluster': [14604, 24458, 24424, 24439, 24440]} 2024-09-15T22:11:45.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T22:11:45.302 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-15T22:11:45.550 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:45 smithi077 ceph-mon[30980]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:45 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4121530541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:45.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:45 smithi196 ceph-mon[35109]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:45.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:45 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4121530541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:45.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:45 smithi117 ceph-mon[35021]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:45.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:45 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4121530541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:46.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 -- 172.21.15.77:0/3105594454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 msgr2=0x7fea44101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 --2- 172.21.15.77:0/3105594454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea44101620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea34009900 tx=0x7fea3402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 -- 172.21.15.77:0/3105594454 shutdown_connections 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 --2- 172.21.15.77:0/3105594454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea4410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 --2- 172.21.15.77:0/3105594454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea44101620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 --2- 172.21.15.77:0/3105594454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 0x7fea44105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 -- 172.21.15.77:0/3105594454 >> 172.21.15.77:0/3105594454 conn(0x7fea44077210 msgr2=0x7fea440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 -- 172.21.15.77:0/3105594454 shutdown_connections 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.064+0000 7fea4c453640 1 -- 172.21.15.77:0/3105594454 wait complete. 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 Processor -- start 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 -- start start 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea44198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 0x7fea4419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea44110ef0 con 0x7fea441011c0 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea44110d70 con 0x7fea44105ad0 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4c453640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea44111070 con 0x7fea44101b60 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36964/0 (socket says 172.21.15.77:36964) 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 -- 172.21.15.77:0/1573119910 learned_addr learned my addr 172.21.15.77:0/1573119910 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea499ee640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea44198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 0x7fea4419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 -- 172.21.15.77:0/1573119910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 msgr2=0x7fea44198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea44198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 -- 172.21.15.77:0/1573119910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 msgr2=0x7fea4419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 0x7fea4419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea4a1ef640 1 -- 172.21.15.77:0/1573119910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4419d390 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.065+0000 7fea499ee640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea44198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d950 tx=0x7fea4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea3b7fe640 1 -- 172.21.15.77:0/1573119910 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40017030 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4419d680 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea3b7fe640 1 -- 172.21.15.77:0/1573119910 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40004020 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea3b7fe640 1 -- 172.21.15.77:0/1573119910 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400047a0 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.066+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441a4f60 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.068+0000 7fea3b7fe640 1 -- 172.21.15.77:0/1573119910 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fea400041d0 con 0x7fea441011c0 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.068+0000 7fea4e82c740 1 Processor -- start 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 -- start start 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea3b7fe640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 0x7fea300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587de439090 con 0x5587de4ddbb0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587de3e0a70 con 0x5587de4d32f0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4e82c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587de435050 con 0x5587de4d29d0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 0x7fea300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea499ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36968/0 (socket says 172.21.15.77:36968) 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea3b7fe640 1 -- 172.21.15.77:0/1573119910 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea400cf980 con 0x7fea441011c0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 -- 172.21.15.77:0/381025521 learned_addr learned my addr 172.21.15.77:0/381025521 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 -- 172.21.15.77:0/381025521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 msgr2=0x5587de4d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 -- 172.21.15.77:0/381025521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 msgr2=0x5587de4dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a9f0640 1 -- 172.21.15.77:0/381025521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de3ed130 con 0x5587de4ddbb0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.069+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 0x7fea300a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ac50 tx=0x7fea3c009e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fea3c016ad0 tx=0x7fea3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be5a569f4086317 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea38dea640 1 -- 172.21.15.77:0/381025521 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01f070 con 0x5587de4ddbb0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea38dea640 1 -- 172.21.15.77:0/381025521 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c018de0 con 0x5587de4ddbb0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea38dea640 1 -- 172.21.15.77:0/381025521 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c013040 con 0x5587de4ddbb0 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 -- 172.21.15.77:0/381025521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 msgr2=0x5587de4dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fea3c016ad0 tx=0x7fea3c019000 comp rx=0 tx=0).stop 2024-09-15T22:11:46.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 -- 172.21.15.77:0/381025521 shutdown_connections 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4dffa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 --2- 172.21.15.77:0/381025521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.070+0000 7fea4e82c740 1 -- 172.21.15.77:0/381025521 >> 172.21.15.77:0/381025521 conn(0x5587de4cef60 msgr2=0x5587de4cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- 172.21.15.77:0/381025521 shutdown_connections 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- 172.21.15.77:0/381025521 wait complete. 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- start start 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 Processor -- start 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587de4e0720 con 0x5587de4d32f0 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587de3f3f20 con 0x5587de4ddbb0 2024-09-15T22:11:46.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587de4e08a0 con 0x5587de4d29d0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea4e82c740 10 client.0 _ll_register_callbacks cb 0x5587de3bd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea499ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea499ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36984/0 (socket says 172.21.15.77:36984) 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.071+0000 7fea499ee640 1 -- 172.21.15.77:0/2251689774 learned_addr learned my addr 172.21.15.77:0/2251689774 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 -- 172.21.15.77:0/2251689774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 msgr2=0x5587de4dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 -- 172.21.15.77:0/2251689774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 msgr2=0x5587de4dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 -- 172.21.15.77:0/2251689774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587de1aa2b0 con 0x5587de4d32f0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea499ee640 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fea34005c40 tx=0x7fea34004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5275ed122e9b622 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3404a060 con 0x5587de4d32f0 2024-09-15T22:11:46.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c018de0 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea3403b8e0 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea34051050 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea4e82c740 10 client.15057 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587de4ec130 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c018de0 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fea340035f0 con 0x5587de4d32f0 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea4e82c740 10 client.15057 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.072+0000 7fea4e82c740 10 client.15057 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4e82c740 20 client.15057 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de4ee810 0x5587de4efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4e82c740 4 client.15057 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587de446600 con 0x5587de4ee810 2024-09-15T22:11:46.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de4ee810 0x5587de4efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de4ee810 0x5587de4efc80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6a8153a4f75daf6 server_cookie=2d81346b452c99e7 in_seq=0 out_seq=0 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.073+0000 7fea2dffb640 10 client.15057 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea2dffb640 1 -- 172.21.15.77:0/2251689774 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587de446600 con 0x5587de4ee810 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea2dffb640 10 client.15057 handle_command_reply: tid=0 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 client.15057 shutdown 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 msgr2=0x5587de4da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fea34005c40 tx=0x7fea34004610 comp rx=0 tx=0).stop 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 shutdown_connections 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de4ee810 0x5587de4efc80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d32f0 0x5587de4da7f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea2dffb640 0 client.15057 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:11:46.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 --2- 172.21.15.77:0/2251689774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 >> 172.21.15.77:0/2251689774 conn(0x5587de4cef60 msgr2=0x5587de4d4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 shutdown_connections 2024-09-15T22:11:46.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 1 -- 172.21.15.77:0/2251689774 wait complete. 2024-09-15T22:11:46.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.074+0000 7fea4e82c740 20 client.15057 trim_cache size 0 max 16384 2024-09-15T22:11:46.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.091+0000 7fea4e82c740 1 Processor -- start 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.091+0000 7fea4e82c740 1 -- start start 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4d7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587de4edfc0 con 0x5587de4ddbb0 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587de4ee690 con 0x5587de4d32f0 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4e82c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587de4eed60 con 0x5587de4d29d0 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36986/0 (socket says 172.21.15.77:36986) 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 -- 172.21.15.77:0/1310992230 learned_addr learned my addr 172.21.15.77:0/1310992230 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:46.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea499ee640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4d7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 -- 172.21.15.77:0/1310992230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 msgr2=0x5587de4cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 -- 172.21.15.77:0/1310992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 msgr2=0x5587de4d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4d7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 -- 172.21.15.77:0/1310992230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de4dd0c0 con 0x5587de4ddbb0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea499ee640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea4a9f0640 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea3c025040 tx=0x7fea3c00a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cd070cb7e31c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea2ffff640 1 -- 172.21.15.77:0/1310992230 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01f040 con 0x5587de4ddbb0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.092+0000 7fea2ffff640 1 -- 172.21.15.77:0/1310992230 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c032070 con 0x5587de4ddbb0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea2ffff640 1 -- 172.21.15.77:0/1310992230 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c02dd40 con 0x5587de4ddbb0 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 -- 172.21.15.77:0/1310992230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 msgr2=0x5587de4d7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea3c025040 tx=0x7fea3c00a520 comp rx=0 tx=0).stop 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 -- 172.21.15.77:0/1310992230 shutdown_connections 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4ddbb0 0x5587de4d7f90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4d32f0 0x5587de4d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 --2- 172.21.15.77:0/1310992230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d29d0 0x5587de4cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 -- 172.21.15.77:0/1310992230 >> 172.21.15.77:0/1310992230 conn(0x5587de4cef60 msgr2=0x5587de4cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 -- 172.21.15.77:0/1310992230 shutdown_connections 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.093+0000 7fea4e82c740 1 -- 172.21.15.77:0/1310992230 wait complete. 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 -- start start 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 Processor -- start 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d32f0 0x5587de4f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587de4efb00 con 0x5587de4d29d0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587de4ef430 con 0x5587de4ddbb0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587de4f01d0 con 0x5587de4d32f0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4e82c740 10 client.0 _ll_register_callbacks cb 0x7fea340044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36996/0 (socket says 172.21.15.77:36996) 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 -- 172.21.15.77:0/3411771236 learned_addr learned my addr 172.21.15.77:0/3411771236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea499ee640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d32f0 0x5587de4f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 -- 172.21.15.77:0/3411771236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d32f0 msgr2=0x5587de4f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d32f0 0x5587de4f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 -- 172.21.15.77:0/3411771236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 msgr2=0x5587de4f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 -- 172.21.15.77:0/3411771236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587de4f3e90 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea4a1ef640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea4009b780 tx=0x7fea4009b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90554fdde08a0fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.094+0000 7fea2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4009c180 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea400860b0 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c032070 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea40099b80 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 10 client.15063 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587de558ae0 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c032070 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fea400ba2d0 con 0x5587de4d29d0 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 10 client.15063 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 10 client.15063 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:11:46.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 20 client.15063 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de55d6a0 0x5587de55c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 4 client.15063 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "status"}) -- 0x5587de440b10 con 0x5587de55d6a0 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.095+0000 7fea499ee640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de55d6a0 0x5587de55c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea2dffb640 10 client.15063 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea499ee640 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de55d6a0 0x5587de55c3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32099b0ae11e28ce server_cookie=890023602c26b1d2 in_seq=0 out_seq=0 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea2dffb640 1 -- 172.21.15.77:0/3411771236 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5587de440b10 con 0x5587de55d6a0 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea2dffb640 10 client.15063 handle_command_reply: tid=0 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 client.15063 shutdown 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 msgr2=0x5587de4f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea4009b780 tx=0x7fea4009b6e0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 shutdown_connections 2024-09-15T22:11:46.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5587de55d6a0 0x5587de55c3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587de4ddbb0 0x5587de4f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea2dffb640 0 client.15063 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587de4d32f0 0x5587de4f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 --2- 172.21.15.77:0/3411771236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587de4d29d0 0x5587de4f8250 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 >> 172.21.15.77:0/3411771236 conn(0x5587de4cef60 msgr2=0x5587de4d8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 shutdown_connections 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 1 -- 172.21.15.77:0/3411771236 wait complete. 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.096+0000 7fea4e82c740 20 client.15063 trim_cache size 0 max 16384 2024-09-15T22:11:46.104 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14604, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 67.414714977000003, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 123.418208486 2024-09-15T22:11:46.105 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:11:46.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 msgr2=0x7fea300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 0x7fea300a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ac50 tx=0x7fea3c009e10 comp rx=0 tx=0).stop 2024-09-15T22:11:46.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 msgr2=0x7fea441983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:46.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d950 tx=0x7fea4000de10 comp rx=0 tx=0).stop 2024-09-15T22:11:46.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 shutdown_connections 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fea300a32d0 0x7fea300a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea44105ad0 0x7fea4419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea44101b60 0x7fea44198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 --2- 172.21.15.77:0/1573119910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea441011c0 0x7fea441983e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 >> 172.21.15.77:0/1573119910 conn(0x7fea44077210 msgr2=0x7fea441002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 shutdown_connections 2024-09-15T22:11:46.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:46.097+0000 7fea4c453640 1 -- 172.21.15.77:0/1573119910 wait complete. 2024-09-15T22:11:46.894 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-15T22:11:47.140 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:47.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:47 smithi077 ceph-mon[30980]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:47.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 -- 172.21.15.77:0/236504629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c100a80 msgr2=0x7f911c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 --2- 172.21.15.77:0/236504629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c100a80 0x7f911c100ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9110009930 tx=0x7f911002a420 comp rx=0 tx=0).stop 2024-09-15T22:11:47.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 -- 172.21.15.77:0/236504629 shutdown_connections 2024-09-15T22:11:47.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 --2- 172.21.15.77:0/236504629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 0x7f911c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 --2- 172.21.15.77:0/236504629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c100a80 0x7f911c100ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 --2- 172.21.15.77:0/236504629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c106a80 0x7f911c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 -- 172.21.15.77:0/236504629 >> 172.21.15.77:0/236504629 conn(0x7f911c0fc7a0 msgr2=0x7f911c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 -- 172.21.15.77:0/236504629 shutdown_connections 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 -- 172.21.15.77:0/236504629 wait complete. 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.605+0000 7f9123c41640 1 Processor -- start 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 -- start start 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c100a80 0x7f911c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 0x7f911c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f911c110f20 con 0x7f911c106a80 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f911c110da0 con 0x7f911c101420 2024-09-15T22:11:47.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f9123c41640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1110a0 con 0x7f911c100a80 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37008/0 (socket says 172.21.15.77:37008) 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 -- 172.21.15.77:0/3378620168 learned_addr learned my addr 172.21.15.77:0/3378620168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91219dd640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c100a80 0x7f911c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91211dc640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 0x7f911c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 -- 172.21.15.77:0/3378620168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c100a80 msgr2=0x7f911c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c100a80 0x7f911c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 -- 172.21.15.77:0/3378620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 msgr2=0x7f911c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 0x7f911c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 -- 172.21.15.77:0/3378620168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c1a4a20 con 0x7f911c106a80 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.606+0000 7f91221de640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f910c00b970 tx=0x7f910c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.607+0000 7f911affd640 1 -- 172.21.15.77:0/3378620168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c019070 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.607+0000 7f911affd640 1 -- 172.21.15.77:0/3378620168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c014440 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.607+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c1a4bf0 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.607+0000 7f911affd640 1 -- 172.21.15.77:0/3378620168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c0075d0 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.607+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1a50b0 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.609+0000 7f911affd640 1 -- 172.21.15.77:0/3378620168 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f910c01c070 con 0x7f911c106a80 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.609+0000 7f912601a740 1 Processor -- start 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.609+0000 7f912601a740 1 -- start start 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a6ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5633429c6090 con 0x563342a5f8b0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56334296da70 con 0x563342a601d0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f912601a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5633429c2050 con 0x563342a6efc0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37020/0 (socket says 172.21.15.77:37020) 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 -- 172.21.15.77:0/2861384455 learned_addr learned my addr 172.21.15.77:0/2861384455 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91219dd640 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a6ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f911affd640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 0x7f91040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 -- 172.21.15.77:0/2861384455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 msgr2=0x563342a713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 -- 172.21.15.77:0/2861384455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 msgr2=0x563342a6ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91221de640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 0x7f91040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a6ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 -- 172.21.15.77:0/2861384455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334297a130 con 0x563342a5f8b0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f911affd640 1 -- 172.21.15.77:0/3378620168 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f910c0cf270 con 0x7f911c106a80 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.610+0000 7f91211dc640 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ec0 tx=0x7f9110005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa5409895d40de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f9103fff640 1 -- 172.21.15.77:0/2861384455 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110035040 con 0x563342a5f8b0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f9103fff640 1 -- 172.21.15.77:0/2861384455 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110005020 con 0x563342a5f8b0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f91221de640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 0x7f91040a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f910c09b520 tx=0x7f910c090a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f9103fff640 1 -- 172.21.15.77:0/2861384455 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110004bc0 con 0x563342a5f8b0 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 -- 172.21.15.77:0/2861384455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 msgr2=0x563342a5fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9110005ec0 tx=0x7f9110005fd0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 -- 172.21.15.77:0/2861384455 shutdown_connections 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a6ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 --2- 172.21.15.77:0/2861384455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a5fc90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 -- 172.21.15.77:0/2861384455 >> 172.21.15.77:0/2861384455 conn(0x563342a5bf60 msgr2=0x563342a5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 -- 172.21.15.77:0/2861384455 shutdown_connections 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.611+0000 7f912601a740 1 -- 172.21.15.77:0/2861384455 wait complete. 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 -- start start 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 Processor -- start 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a5f8b0 0x563342a6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a6bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563342a71b90 con 0x563342a6efc0 2024-09-15T22:11:47.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563342980f20 con 0x563342a5f8b0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563342a71d10 con 0x563342a601d0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f912601a740 10 client.0 _ll_register_callbacks cb 0x56334294a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37036/0 (socket says 172.21.15.77:37036) 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.612+0000 7f91211dc640 1 -- 172.21.15.77:0/999997339 learned_addr learned my addr 172.21.15.77:0/999997339 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a5f8b0 0x563342a6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91219dd640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a6bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 -- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 msgr2=0x563342a6bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a6bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 -- 172.21.15.77:0/999997339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a5f8b0 msgr2=0x563342a6e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a5f8b0 0x563342a6e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 -- 172.21.15.77:0/999997339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633427372b0 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91211dc640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f91100059b0 tx=0x7f91100047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2d73b1515a04a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110052020 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9110005020 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f91100053e0 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9110035040 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f912601a740 10 client.15075 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563342a7d490 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110005020 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.613+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f91100039a0 con 0x563342a6efc0 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 10 client.15075 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:11:47.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 20 client.15075 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342a7e7e0 0x563342a80950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 4 client.15075 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633429d3600 con 0x563342a7e7e0 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f91219dd640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342a7e7e0 0x563342a80950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f91219dd640 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342a7e7e0 0x563342a80950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e50f4fa90c3503f server_cookie=d31fa80bba282085 in_seq=0 out_seq=0 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.614+0000 7f91017fa640 10 client.15075 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f91017fa640 1 -- 172.21.15.77:0/999997339 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633429d3600 con 0x563342a7e7e0 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f91017fa640 10 client.15075 handle_command_reply: tid=0 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 client.15075 shutdown 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 msgr2=0x563342a6c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f91100059b0 tx=0x7f91100047a0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 shutdown_connections 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342a7e7e0 0x563342a80950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a6efc0 0x563342a6c060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a6bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 --2- 172.21.15.77:0/999997339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a5f8b0 0x563342a6e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f91017fa640 0 client.15075 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 >> 172.21.15.77:0/999997339 conn(0x563342a5bf60 msgr2=0x563342a61cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 shutdown_connections 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 1 -- 172.21.15.77:0/999997339 wait complete. 2024-09-15T22:11:47.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.615+0000 7f912601a740 20 client.15075 trim_cache size 0 max 16384 2024-09-15T22:11:47.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.633+0000 7f912601a740 1 Processor -- start 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.633+0000 7f912601a740 1 -- start start 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a7ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563342a7ec90 con 0x563342a5f8b0 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563342a7f360 con 0x563342a601d0 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f912601a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563342a7fa30 con 0x563342a6efc0 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37052/0 (socket says 172.21.15.77:37052) 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 -- 172.21.15.77:0/2089968978 learned_addr learned my addr 172.21.15.77:0/2089968978 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:47.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a7ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91219dd640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 -- 172.21.15.77:0/2089968978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 msgr2=0x563342a7ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a7ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 -- 172.21.15.77:0/2089968978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 msgr2=0x563342a7f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a7f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 -- 172.21.15.77:0/2089968978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563342a6eb40 con 0x563342a5f8b0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91219dd640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91211dc640 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f91100317f0 tx=0x7f9110000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c161a24edc6e958 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91037fe640 1 -- 172.21.15.77:0/2089968978 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110052070 con 0x563342a5f8b0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91037fe640 1 -- 172.21.15.77:0/2089968978 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110003ad0 con 0x563342a5f8b0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.634+0000 7f91037fe640 1 -- 172.21.15.77:0/2089968978 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110005620 con 0x563342a5f8b0 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 -- 172.21.15.77:0/2089968978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 msgr2=0x563342a80280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f91100317f0 tx=0x7f9110000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 -- 172.21.15.77:0/2089968978 shutdown_connections 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a6efc0 0x563342a7ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a601d0 0x563342a7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 --2- 172.21.15.77:0/2089968978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a80280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 -- 172.21.15.77:0/2089968978 >> 172.21.15.77:0/2089968978 conn(0x563342a5bf60 msgr2=0x563342a5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 -- 172.21.15.77:0/2089968978 shutdown_connections 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.635+0000 7f912601a740 1 -- 172.21.15.77:0/2089968978 wait complete. 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 -- start start 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 Processor -- start 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a89360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a6efc0 0x563342a82170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563342a807d0 con 0x563342a5f8b0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563342a80100 con 0x563342a6efc0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563342a80ea0 con 0x563342a601d0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f912601a740 10 client.0 _ll_register_callbacks cb 0x7f91140093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37060/0 (socket says 172.21.15.77:37060) 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 -- 172.21.15.77:0/3195678299 learned_addr learned my addr 172.21.15.77:0/3195678299 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91219dd640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a89360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 -- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 msgr2=0x563342a89360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a89360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.636+0000 7f91211dc640 1 -- 172.21.15.77:0/3195678299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a6efc0 msgr2=0x563342a82170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91211dc640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a6efc0 0x563342a82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91211dc640 1 -- 172.21.15.77:0/3195678299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563342a829a0 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91211dc640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9110031460 tx=0x7f9110009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ab1e28a1e93a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110067060 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9110003800 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9110003ad0 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9110052070 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f912601a740 10 client.15081 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563342a873f0 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110003ad0 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f91100047a0 con 0x563342a5f8b0 2024-09-15T22:11:47.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.637+0000 7f912601a740 10 client.15081 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 20 client.15081 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342aede00 0x563342aecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 4 client.15081 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "status"}) -- 0x5633429cdb10 con 0x563342aede00 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f91219dd640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342aede00 0x563342aecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f91017fa640 10 client.15081 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f91219dd640 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342aede00 0x563342aecb10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51768f9037bd89c8 server_cookie=5e6e94cee21e584c in_seq=0 out_seq=0 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f91017fa640 1 -- 172.21.15.77:0/3195678299 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5633429cdb10 con 0x563342aede00 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f91017fa640 10 client.15081 handle_command_reply: tid=0 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.638+0000 7f912601a740 1 client.15081 shutdown 2024-09-15T22:11:47.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 msgr2=0x563342a88e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9110031460 tx=0x7f9110009290 comp rx=0 tx=0).stop 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 shutdown_connections 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563342aede00 0x563342aecb10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563342a6efc0 0x563342a82170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563342a601d0 0x563342a89360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f91017fa640 0 client.15081 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 --2- 172.21.15.77:0/3195678299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563342a5f8b0 0x563342a88e20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 >> 172.21.15.77:0/3195678299 conn(0x563342a5bf60 msgr2=0x563342a56ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 shutdown_connections 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 1 -- 172.21.15.77:0/3195678299 wait complete. 2024-09-15T22:11:47.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f912601a740 20 client.15081 trim_cache size 0 max 16384 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24424, 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 69.303822605999997, 2024-09-15T22:11:47.652 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:11:47.653 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:11:47.653 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:11:47.653 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 136.42493891699999 2024-09-15T22:11:47.653 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:11:47.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 msgr2=0x7f91040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 0x7f91040a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f910c09b520 tx=0x7f910c090a70 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 msgr2=0x7f911c1a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f910c00b970 tx=0x7f910c00bd70 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 shutdown_connections 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f91040a32d0 0x7f91040a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c106a80 0x7f911c1a44e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f911c101420 0x7f911c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 --2- 172.21.15.77:0/3378620168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f911c100a80 0x7f911c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.639+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 >> 172.21.15.77:0/3378620168 conn(0x7f911c0fc7a0 msgr2=0x7f911c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.640+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 shutdown_connections 2024-09-15T22:11:47.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:47.640+0000 7f9123c41640 1 -- 172.21.15.77:0/3378620168 wait complete. 2024-09-15T22:11:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:47 smithi196 ceph-mon[35109]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:47.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:47 smithi117 ceph-mon[35021]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:48.452 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:11:48.695 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:48.816 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:48 smithi077 ceph-mon[30980]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:48.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:48 smithi196 ceph-mon[35109]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:48.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:48 smithi117 ceph-mon[35021]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:49.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- 172.21.15.77:0/4068247203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec101de0 msgr2=0x7f10ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- 172.21.15.77:0/4068247203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec101de0 0x7f10ec1021c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009900 tx=0x7f10e002a410 comp rx=0 tx=0).stop 2024-09-15T22:11:49.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- 172.21.15.77:0/4068247203 shutdown_connections 2024-09-15T22:11:49.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- 172.21.15.77:0/4068247203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec106f10 0x7f10ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- 172.21.15.77:0/4068247203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec102790 0x7f10ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- 172.21.15.77:0/4068247203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec101de0 0x7f10ec1021c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- 172.21.15.77:0/4068247203 >> 172.21.15.77:0/4068247203 conn(0x7f10ec0fd8c0 msgr2=0x7f10ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- 172.21.15.77:0/4068247203 shutdown_connections 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- 172.21.15.77:0/4068247203 wait complete. 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 Processor -- start 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- start start 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec101de0 0x7f10ec1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec106f10 0x7f10ec19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec112010 con 0x7f10ec102790 2024-09-15T22:11:49.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec111e90 con 0x7f10ec101de0 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.190+0000 7f10f2bca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec112190 con 0x7f10ec106f10 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54696/0 (socket says 172.21.15.77:54696) 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 -- 172.21.15.77:0/2868103435 learned_addr learned my addr 172.21.15.77:0/2868103435 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10f1167640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec106f10 0x7f10ec19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10f0966640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec101de0 0x7f10ec1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 -- 172.21.15.77:0/2868103435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec106f10 msgr2=0x7f10ec19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec106f10 0x7f10ec19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 -- 172.21.15.77:0/2868103435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec101de0 msgr2=0x7f10ec1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec101de0 0x7f10ec1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 -- 172.21.15.77:0/2868103435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec19d560 con 0x7f10ec102790 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b4a0 tx=0x7f10d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10e9ffb640 1 -- 172.21.15.77:0/2868103435 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8014070 con 0x7f10ec102790 2024-09-15T22:11:49.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec19d850 con 0x7f10ec102790 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10e9ffb640 1 -- 172.21.15.77:0/2868103435 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8010af0 con 0x7f10ec102790 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10e9ffb640 1 -- 172.21.15.77:0/2868103435 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8017d10 con 0x7f10ec102790 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.191+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1a5130 con 0x7f10ec102790 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.194+0000 7f10e9ffb640 1 -- 172.21.15.77:0/2868103435 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f10d8017070 con 0x7f10ec102790 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.194+0000 7f10f4fa3740 1 Processor -- start 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.194+0000 7f10f4fa3740 1 -- start start 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e7a416090 con 0x556e7a4babb0 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e7a3bda70 con 0x556e7a4af9d0 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e7a412050 con 0x556e7a4b02f0 2024-09-15T22:11:49.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f1167640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54706/0 (socket says 172.21.15.77:54706) 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10e9ffb640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 0x7f10dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 -- 172.21.15.77:0/3091336031 learned_addr learned my addr 172.21.15.77:0/3091336031 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f1167640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 0x7f10dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10e9ffb640 1 -- 172.21.15.77:0/2868103435 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f10d80cf1e0 con 0x7f10ec102790 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 -- 172.21.15.77:0/3091336031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 msgr2=0x556e7a4ba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f0966640 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4ba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 -- 172.21.15.77:0/3091336031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 msgr2=0x556e7a4afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 -- 172.21.15.77:0/3091336031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7a3ca130 con 0x556e7a4babb0 2024-09-15T22:11:49.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10f1167640 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.195+0000 7f10ebfff640 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f10d809b3e0 tx=0x7f10d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b132ec2a47bb7fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f1167640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 0x7f10dc0a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009fd0 tx=0x7f10e4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10d6ffd640 1 -- 172.21.15.77:0/3091336031 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d801e070 con 0x556e7a4babb0 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10d6ffd640 1 -- 172.21.15.77:0/3091336031 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d809bb50 con 0x556e7a4babb0 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10d6ffd640 1 -- 172.21.15.77:0/3091336031 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8010ca0 con 0x556e7a4babb0 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3091336031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 msgr2=0x556e7a4bcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f10d809b3e0 tx=0x7f10d80046c0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3091336031 shutdown_connections 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4bcfa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4ba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3091336031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3091336031 >> 172.21.15.77:0/3091336031 conn(0x556e7a4abf60 msgr2=0x556e7a4ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3091336031 shutdown_connections 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.196+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3091336031 wait complete. 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 -- start start 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 Processor -- start 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4af9d0 0x556e7a4ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4bd720 con 0x556e7a4b02f0 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e7a3d0f20 con 0x556e7a4babb0 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4bd8a0 con 0x556e7a4af9d0 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10f4fa3740 10 client.0 _ll_register_callbacks cb 0x556e7a39a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:49.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54710/0 (socket says 172.21.15.77:54710) 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.197+0000 7f10ebfff640 1 -- 172.21.15.77:0/4191816121 learned_addr learned my addr 172.21.15.77:0/4191816121 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f1167640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f0966640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4af9d0 0x556e7a4ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 -- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4af9d0 msgr2=0x556e7a4ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4af9d0 0x556e7a4ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 -- 172.21.15.77:0/4191816121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 msgr2=0x556e7a4b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 -- 172.21.15.77:0/4191816121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7a1872b0 con 0x556e7a4b02f0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f1167640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10ebfff640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f10d8097e20 tx=0x7f10d8017590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7984434a4eac57ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:49.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d801e070 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d809bb50 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f10d8097ac0 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10d8010ca0 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f4fa3740 10 client.15093 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e7a4c9030 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d809bb50 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.198+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10d8097230 con 0x556e7a4b02f0 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 10 client.15093 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:11:49.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 20 client.15093 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a4cb1d0 0x556e7a4cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 4 client.15093 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e7a423600 con 0x556e7a4cb1d0 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f0966640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a4cb1d0 0x556e7a4cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10cbfff640 10 client.15093 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.199+0000 7f10f0966640 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a4cb1d0 0x556e7a4cc640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fed13b98ba14ac3 server_cookie=fe45f73d4dc1c73c in_seq=0 out_seq=0 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10cbfff640 1 -- 172.21.15.77:0/4191816121 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e7a423600 con 0x556e7a4cb1d0 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10cbfff640 10 client.15093 handle_command_reply: tid=0 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 client.15093 shutdown 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 msgr2=0x556e7a4b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f10d8097e20 tx=0x7f10d8017590 comp rx=0 tx=0).stop 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 shutdown_connections 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a4cb1d0 0x556e7a4cc640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4b02f0 0x556e7a4b7780 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10cbfff640 0 client.15093 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/4191816121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4af9d0 0x556e7a4ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 >> 172.21.15.77:0/4191816121 conn(0x556e7a4abf60 msgr2=0x556e7a4b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 shutdown_connections 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.200+0000 7f10f4fa3740 1 -- 172.21.15.77:0/4191816121 wait complete. 2024-09-15T22:11:49.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.201+0000 7f10f4fa3740 20 client.15093 trim_cache size 0 max 16384 2024-09-15T22:11:49.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 Processor -- start 2024-09-15T22:11:49.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 -- start start 2024-09-15T22:11:49.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4ca980 con 0x556e7a4babb0 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4cb050 con 0x556e7a4af9d0 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.218+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4cb720 con 0x556e7a4b02f0 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54714/0 (socket says 172.21.15.77:54714) 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 -- 172.21.15.77:0/3293073187 learned_addr learned my addr 172.21.15.77:0/3293073187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:49.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10ebfff640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f0966640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 -- 172.21.15.77:0/3293073187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 msgr2=0x556e7a4b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 -- 172.21.15.77:0/3293073187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 msgr2=0x556e7a4cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 -- 172.21.15.77:0/3293073187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7a4acc50 con 0x556e7a4babb0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f0966640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10f1167640 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009550 tx=0x7f10e40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e176c520733814a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10d67fc640 1 -- 172.21.15.77:0/3293073187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4015070 con 0x556e7a4babb0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10d67fc640 1 -- 172.21.15.77:0/3293073187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4016040 con 0x556e7a4babb0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.219+0000 7f10d67fc640 1 -- 172.21.15.77:0/3293073187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e400ad00 con 0x556e7a4babb0 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3293073187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 msgr2=0x556e7a4b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009550 tx=0x7f10e40098d0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3293073187 shutdown_connections 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4babb0 0x556e7a4b9410 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/3293073187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4af9d0 0x556e7a4cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3293073187 >> 172.21.15.77:0/3293073187 conn(0x556e7a4abf60 msgr2=0x556e7a4b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3293073187 shutdown_connections 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- 172.21.15.77:0/3293073187 wait complete. 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 -- start start 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.220+0000 7f10f4fa3740 1 Processor -- start 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4cc4c0 con 0x556e7a4af9d0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4cbdf0 con 0x556e7a4babb0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e7a4ccb90 con 0x556e7a4b02f0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 10 client.0 _ll_register_callbacks cb 0x7f10e4009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54722/0 (socket says 172.21.15.77:54722) 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 -- 172.21.15.77:0/2069635792 learned_addr learned my addr 172.21.15.77:0/2069635792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f1167640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 -- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 msgr2=0x556e7a4d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4d4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 -- 172.21.15.77:0/2069635792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 msgr2=0x556e7a4cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 -- 172.21.15.77:0/2069635792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7a4ce490 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f0966640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10e002a930 tx=0x7f10e0031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6e8240429a62b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0035040 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e4016040 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.221+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f10e0004250 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10e0052050 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 10 client.15099 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e7a4d2ee0 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4016040 con 0x556e7a4af9d0 2024-09-15T22:11:49.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10e00443a0 con 0x556e7a4af9d0 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 10 client.15099 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 20 client.15099 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a539e30 0x556e7a538b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 4 client.15099 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "status"}) -- 0x556e7a41db10 con 0x556e7a539e30 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.222+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a539e30 0x556e7a538b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10cbfff640 10 client.15099 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10ebfff640 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a539e30 0x556e7a538b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97c0aeaef71872ca server_cookie=4c751c9b077eead4 in_seq=0 out_seq=0 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10cbfff640 1 -- 172.21.15.77:0/2069635792 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556e7a41db10 con 0x556e7a539e30 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10cbfff640 10 client.15099 handle_command_reply: tid=0 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 client.15099 shutdown 2024-09-15T22:11:49.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 msgr2=0x556e7a4d4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10e002a930 tx=0x7f10e0031a90 comp rx=0 tx=0).stop 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 shutdown_connections 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556e7a539e30 0x556e7a538b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e7a4babb0 0x556e7a4cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e7a4b02f0 0x556e7a4d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 --2- 172.21.15.77:0/2069635792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e7a4af9d0 0x556e7a4d4a50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 >> 172.21.15.77:0/2069635792 conn(0x556e7a4abf60 msgr2=0x556e7a4aa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10cbfff640 0 client.15099 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 shutdown_connections 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 1 -- 172.21.15.77:0/2069635792 wait complete. 2024-09-15T22:11:49.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.223+0000 7f10f4fa3740 20 client.15099 trim_cache size 0 max 16384 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24439, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 71.894985571000007, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:11:49.235 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 132.870418806 2024-09-15T22:11:49.236 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:11:49.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 msgr2=0x7f10dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 0x7f10dc0a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009fd0 tx=0x7f10e4004a40 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 msgr2=0x7f10ec198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b4a0 tx=0x7f10d800b960 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 shutdown_connections 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f10dc0a32d0 0x7f10dc0a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ec106f10 0x7f10ec19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10ec102790 0x7f10ec198af0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 --2- 172.21.15.77:0/2868103435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10ec101de0 0x7f10ec1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 >> 172.21.15.77:0/2868103435 conn(0x7f10ec0fd8c0 msgr2=0x7f10ec0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 shutdown_connections 2024-09-15T22:11:49.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:49.224+0000 7f10f2bca640 1 -- 172.21.15.77:0/2868103435 wait complete. 2024-09-15T22:11:50.047 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:11:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:50.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 -- 172.21.15.77:0/1985690754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0101420 msgr2=0x7f09d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/1985690754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0101420 0x7f09d010d950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009cd0 tx=0x7f09c802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 -- 172.21.15.77:0/1985690754 shutdown_connections 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/1985690754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0101420 0x7f09d010d950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/1985690754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0100a80 0x7f09d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/1985690754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0106a80 0x7f09d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 -- 172.21.15.77:0/1985690754 >> 172.21.15.77:0/1985690754 conn(0x7f09d00fc7a0 msgr2=0x7f09d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 -- 172.21.15.77:0/1985690754 shutdown_connections 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.776+0000 7f09d4bb7640 1 -- 172.21.15.77:0/1985690754 wait complete. 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 Processor -- start 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 -- start start 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0100a80 0x7f09d0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 0x7f09d0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110e80 con 0x7f09d0106a80 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110d00 con 0x7f09d0100a80 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09d4bb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0111000 con 0x7f09d0101420 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54744/0 (socket says 172.21.15.77:54744) 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09ced9d640 1 -- 172.21.15.77:0/3227727403 learned_addr learned my addr 172.21.15.77:0/3227727403 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0100a80 0x7f09d0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.777+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 0x7f09d0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 -- 172.21.15.77:0/3227727403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 msgr2=0x7f09d0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 0x7f09d0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 -- 172.21.15.77:0/3227727403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0100a80 msgr2=0x7f09d0197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0100a80 0x7f09d0197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 -- 172.21.15.77:0/3227727403 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d019c2c0 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 0x7f09d0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09ced9d640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005cc0 tx=0x7f09c8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09bf7fe640 1 -- 172.21.15.77:0/3227727403 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8004040 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09bf7fe640 1 -- 172.21.15.77:0/3227727403 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c80046c0 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d019c550 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09bf7fe640 1 -- 172.21.15.77:0/3227727403 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8030e00 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.778+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a3e90 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.780+0000 7f09bf7fe640 1 -- 172.21.15.77:0/3227727403 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f09c80305a0 con 0x7f09d0106a80 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.780+0000 7f09d6f90740 1 Processor -- start 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 -- start start 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb018fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb0261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bfaf7d090 con 0x556bfb019500 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bfaf24a70 con 0x556bfb023dc0 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09d6f90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bfaf79050 con 0x556bfb018be0 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54748/0 (socket says 172.21.15.77:54748) 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 -- 172.21.15.77:0/3279103268 learned_addr learned my addr 172.21.15.77:0/3279103268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09bf7fe640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 0x7f09b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 -- 172.21.15.77:0/3279103268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 msgr2=0x556bfb018fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb018fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 -- 172.21.15.77:0/3279103268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 msgr2=0x556bfb0261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb0261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ce59c640 1 -- 172.21.15.77:0/3279103268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfaf31130 con 0x556bfb019500 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.781+0000 7f09ced9d640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 0x7f09b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09bf7fe640 1 -- 172.21.15.77:0/3227727403 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09c802e030 con 0x7f09d0106a80 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f09c000bc90 tx=0x7f09c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae1210af8fd91ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09bcff9640 1 -- 172.21.15.77:0/3279103268 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c001c480 con 0x556bfb019500 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09ced9d640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 0x7f09b80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09c80c5c50 tx=0x7f09c80c1040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09bcff9640 1 -- 172.21.15.77:0/3279103268 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c001cb00 con 0x556bfb019500 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.782+0000 7f09bcff9640 1 -- 172.21.15.77:0/3279103268 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0011c00 con 0x556bfb019500 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- 172.21.15.77:0/3279103268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 msgr2=0x556bfb023880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f09c000bc90 tx=0x7f09c00107a0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- 172.21.15.77:0/3279103268 shutdown_connections 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb0261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb023880 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3279103268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb018fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- 172.21.15.77:0/3279103268 >> 172.21.15.77:0/3279103268 conn(0x556bfb015170 msgr2=0x556bfb015550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- 172.21.15.77:0/3279103268 shutdown_connections 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- 172.21.15.77:0/3279103268 wait complete. 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- start start 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 Processor -- start 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 0x556bfb020990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb020ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bfb026990 con 0x556bfb018be0 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bfaf37f20 con 0x556bfb023dc0 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.783+0000 7f09d6f90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bfb026b10 con 0x556bfb019500 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09d6f90740 10 client.0 _ll_register_callbacks cb 0x556bfaf01030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54762/0 (socket says 172.21.15.77:54762) 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 -- 172.21.15.77:0/559768514 learned_addr learned my addr 172.21.15.77:0/559768514 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09ce59c640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 0x556bfb020990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb020ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 -- 172.21.15.77:0/559768514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 msgr2=0x556bfb020990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 0x556bfb020990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 -- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 msgr2=0x556bfb020ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb020ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 -- 172.21.15.77:0/559768514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfacee2b0 con 0x556bfb018be0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09ce59c640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 0x556bfb020990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09c40137c0 tx=0x7f09c4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944b281c53429165 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4013e60 con 0x556bfb018be0 2024-09-15T22:11:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09c400fa70 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c001cb00 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09c4024050 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09d6f90740 10 client.15111 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.784+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bfb032300 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c001cb00 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f09c4004990 con 0x556bfb018be0 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 10 client.15111 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 20 client.15111 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb034970 0x556bfb035de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 4 client.15111 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bfaf8a600 con 0x556bfb034970 2024-09-15T22:11:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.785+0000 7f09ce59c640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb034970 0x556bfb035de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.786+0000 7f09b5ffb640 10 client.15111 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.786+0000 7f09ce59c640 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb034970 0x556bfb035de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10000164a0b7269d server_cookie=3697d44f798ca19 in_seq=0 out_seq=0 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.786+0000 7f09b5ffb640 1 -- 172.21.15.77:0/559768514 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bfaf8a600 con 0x556bfb034970 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.786+0000 7f09b5ffb640 10 client.15111 handle_command_reply: tid=0 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.786+0000 7f09d6f90740 1 client.15111 shutdown 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 msgr2=0x556bfb023640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09c40137c0 tx=0x7f09c4009790 comp rx=0 tx=0).stop 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 shutdown_connections 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb034970 0x556bfb035de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb023dc0 0x556bfb020ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb019500 0x556bfb020990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 --2- 172.21.15.77:0/559768514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb018be0 0x556bfb023640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09b5ffb640 0 client.15111 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:11:50.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 >> 172.21.15.77:0/559768514 conn(0x556bfb015170 msgr2=0x556bfb012020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 shutdown_connections 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 1 -- 172.21.15.77:0/559768514 wait complete. 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.787+0000 7f09d6f90740 20 client.15111 trim_cache size 0 max 16384 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 Processor -- start 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 -- start start 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb018be0 0x556bfb0342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb023dc0 0x556bfb03e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bfb034120 con 0x556bfb019500 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bfb0347f0 con 0x556bfb018be0 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09d6f90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bfb034ec0 con 0x556bfb023dc0 2024-09-15T22:11:50.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54764/0 (socket says 172.21.15.77:54764) 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.804+0000 7f09ce59c640 1 -- 172.21.15.77:0/3907457622 learned_addr learned my addr 172.21.15.77:0/3907457622 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb018be0 0x556bfb0342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 -- 172.21.15.77:0/3907457622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb023dc0 msgr2=0x556bfb03e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb023dc0 0x556bfb03e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 -- 172.21.15.77:0/3907457622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb018be0 msgr2=0x556bfb0342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb018be0 0x556bfb0342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 -- 172.21.15.77:0/3907457622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfb019330 con 0x556bfb019500 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09ce59c640 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x556bfb015fd0 tx=0x7f09c000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190ff48e68c1f80d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09b7fff640 1 -- 172.21.15.77:0/3907457622 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c000bd80 con 0x556bfb019500 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09b7fff640 1 -- 172.21.15.77:0/3907457622 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c0011870 con 0x556bfb019500 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.805+0000 7f09b7fff640 1 -- 172.21.15.77:0/3907457622 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c001c8a0 con 0x556bfb019500 2024-09-15T22:11:50.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- 172.21.15.77:0/3907457622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 msgr2=0x556bfb03e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x556bfb015fd0 tx=0x7f09c000b390 comp rx=0 tx=0).stop 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- 172.21.15.77:0/3907457622 shutdown_connections 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb023dc0 0x556bfb03e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb019500 0x556bfb03e090 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- 172.21.15.77:0/3907457622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb018be0 0x556bfb0342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- 172.21.15.77:0/3907457622 >> 172.21.15.77:0/3907457622 conn(0x556bfb015170 msgr2=0x556bfb015550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- 172.21.15.77:0/3907457622 shutdown_connections 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- 172.21.15.77:0/3907457622 wait complete. 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- start start 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 Processor -- start 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb03e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb019500 0x556bfb03e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bfb035c60 con 0x556bfb023dc0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bfb035590 con 0x556bfb019500 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bfb036330 con 0x556bfb018be0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.806+0000 7f09d6f90740 10 client.0 _ll_register_callbacks cb 0x7f09c40096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54766/0 (socket says 172.21.15.77:54766) 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 -- 172.21.15.77:0/2508115709 learned_addr learned my addr 172.21.15.77:0/2508115709 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09ce59c640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb019500 0x556bfb03e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb03e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 -- 172.21.15.77:0/2508115709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 msgr2=0x556bfb03e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb03e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 -- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb019500 msgr2=0x556bfb03e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb019500 0x556bfb03e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 -- 172.21.15.77:0/2508115709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfb03a200 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09cdd9b640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09c4001eb0 tx=0x7f09c4009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1afac38e1f378d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c400fdc0 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f09c40046c0 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c0011870 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09c4039050 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09d6f90740 10 client.15117 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.807+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bfb09ecf0 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c0011870 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f09c4004f80 con 0x556bfb023dc0 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 10 client.15117 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:50.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 20 client.15117 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb0a3920 0x556bfb0a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 4 client.15117 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "status"}) -- 0x556bfaf84b10 con 0x556bfb0a3920 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09ce59c640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb0a3920 0x556bfb0a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09ce59c640 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb0a3920 0x556bfb0a2630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c5c3dae463c39d0 server_cookie=e3d8654318624a37 in_seq=0 out_seq=0 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.808+0000 7f09b5ffb640 10 client.15117 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09b5ffb640 1 -- 172.21.15.77:0/2508115709 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556bfaf84b10 con 0x556bfb0a3920 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09b5ffb640 10 client.15117 handle_command_reply: tid=0 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 client.15117 shutdown 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 msgr2=0x556bfb0399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09c4001eb0 tx=0x7f09c4009630 comp rx=0 tx=0).stop 2024-09-15T22:11:50.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 shutdown_connections 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bfb0a3920 0x556bfb0a2630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bfb023dc0 0x556bfb0399d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bfb019500 0x556bfb03e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 --2- 172.21.15.77:0/2508115709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bfb018be0 0x556bfb03e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 >> 172.21.15.77:0/2508115709 conn(0x556bfb015170 msgr2=0x556bfb0201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09b5ffb640 0 client.15117 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 shutdown_connections 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 1 -- 172.21.15.77:0/2508115709 wait complete. 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d6f90740 20 client.15117 trim_cache size 0 max 16384 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:11:50.827 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24440, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 74.485461717999996, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 136.99398527700001 2024-09-15T22:11:50.828 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 msgr2=0x7f09b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 0x7f09b80a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09c80c5c50 tx=0x7f09c80c1040 comp rx=0 tx=0).stop 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 msgr2=0x7f09d019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005cc0 tx=0x7f09c8032f40 comp rx=0 tx=0).stop 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 shutdown_connections 2024-09-15T22:11:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f09b80a32d0 0x7f09b80a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d0106a80 0x7f09d019bbe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d0101420 0x7f09d0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.809+0000 7f09d4bb7640 1 --2- 172.21.15.77:0/3227727403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d0100a80 0x7f09d0197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.810+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 >> 172.21.15.77:0/3227727403 conn(0x7f09d00fc7a0 msgr2=0x7f09d00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.810+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 shutdown_connections 2024-09-15T22:11:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:50.810+0000 7f09d4bb7640 1 -- 172.21.15.77:0/3227727403 wait complete. 2024-09-15T22:11:51.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:51 smithi077 ceph-mon[30980]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:51 smithi196 ceph-mon[35109]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:51 smithi117 ceph-mon[35021]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:11:51.701 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:11:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:52.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 -- 172.21.15.77:0/3897907553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8101de0 msgr2=0x7fbcd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 --2- 172.21.15.77:0/3897907553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8101de0 0x7fbcd81021c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009900 tx=0x7fbcd402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 -- 172.21.15.77:0/3897907553 shutdown_connections 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 --2- 172.21.15.77:0/3897907553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8106f10 0x7fbcd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 --2- 172.21.15.77:0/3897907553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8102790 0x7fbcd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 --2- 172.21.15.77:0/3897907553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8101de0 0x7fbcd81021c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 -- 172.21.15.77:0/3897907553 >> 172.21.15.77:0/3897907553 conn(0x7fbcd80fd8c0 msgr2=0x7fbcd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 -- 172.21.15.77:0/3897907553 shutdown_connections 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.414+0000 7fbce0715640 1 -- 172.21.15.77:0/3897907553 wait complete. 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 Processor -- start 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 -- start start 2024-09-15T22:11:52.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101de0 0x7fbcd8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 0x7fbcd819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8111f30 con 0x7fbcd8102790 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8111db0 con 0x7fbcd8106f10 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbce0715640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81120b0 con 0x7fbcd8101de0 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54778/0 (socket says 172.21.15.77:54778) 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 -- 172.21.15.77:0/377018209 learned_addr learned my addr 172.21.15.77:0/377018209 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101de0 0x7fbcd8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 -- 172.21.15.77:0/377018209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101de0 msgr2=0x7fbcd8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101de0 0x7fbcd8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcddcb0640 1 -- 172.21.15.77:0/377018209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 msgr2=0x7fbcd819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.415+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 0x7fbcd819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 0x7fbcd819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbcddcb0640 1 -- 172.21.15.77:0/377018209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd819d440 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 0x7fbcd819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800d950 tx=0x7fbcc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbccf7fe640 1 -- 172.21.15.77:0/377018209 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8017070 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbccf7fe640 1 -- 172.21.15.77:0/377018209 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc8009c30 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd819d730 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbccf7fe640 1 -- 172.21.15.77:0/377018209 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800f6b0 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.416+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81a5010 con 0x7fbcd8102790 2024-09-15T22:11:52.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.418+0000 7fbccf7fe640 1 -- 172.21.15.77:0/377018209 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbcc80109d0 con 0x7fbcd8102790 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.418+0000 7fbce2aee740 1 Processor -- start 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 -- start start 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbccf7fe640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 0x7fbcc40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd62f0 0x5592f0ce0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0ce2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592f0c3c090 con 0x5592f0cd59d0 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592f0be3a70 con 0x5592f0ce0bb0 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbce2aee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592f0c38050 con 0x5592f0cd62f0 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 0x7fbcc40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbccf7fe640 1 -- 172.21.15.77:0/377018209 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbcc80cfb00 con 0x7fbcd8102790 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54782/0 (socket says 172.21.15.77:54782) 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 -- 172.21.15.77:0/307562572 learned_addr learned my addr 172.21.15.77:0/307562572 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0ce2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd62f0 0x5592f0ce0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 -- 172.21.15.77:0/307562572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd62f0 msgr2=0x5592f0ce0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd62f0 0x5592f0ce0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 -- 172.21.15.77:0/307562572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 msgr2=0x5592f0ce2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0ce2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcde4b1640 1 -- 172.21.15.77:0/307562572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f0bf0130 con 0x5592f0cd59d0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.419+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0ce2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.420+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 0x7fbcc40a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd0014a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.420+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4006010 tx=0x7fbcd4031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c60196b60d711 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.420+0000 7fbcccff9640 1 -- 172.21.15.77:0/307562572 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4035030 con 0x5592f0cd59d0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.420+0000 7fbcccff9640 1 -- 172.21.15.77:0/307562572 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd40317b0 con 0x5592f0cd59d0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.420+0000 7fbcccff9640 1 -- 172.21.15.77:0/307562572 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40046c0 con 0x5592f0cd59d0 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- 172.21.15.77:0/307562572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 msgr2=0x5592f0cd5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4006010 tx=0x7fbcd4031510 comp rx=0 tx=0).stop 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- 172.21.15.77:0/307562572 shutdown_connections 2024-09-15T22:11:52.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0ce2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd62f0 0x5592f0ce0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- 172.21.15.77:0/307562572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd59d0 0x5592f0cd5db0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- 172.21.15.77:0/307562572 >> 172.21.15.77:0/307562572 conn(0x5592f0cd1f60 msgr2=0x5592f0cd2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- 172.21.15.77:0/307562572 shutdown_connections 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- 172.21.15.77:0/307562572 wait complete. 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 -- start start 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 Processor -- start 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0ce0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.421+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0cddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592f0ce3780 con 0x5592f0cd62f0 2024-09-15T22:11:52.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592f0bf6f20 con 0x5592f0ce0bb0 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592f0ce3900 con 0x5592f0cd59d0 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 10 client.0 _ll_register_callbacks cb 0x5592f0bc0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54784/0 (socket says 172.21.15.77:54784) 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3769134200 learned_addr learned my addr 172.21.15.77:0/3769134200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0ce0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3769134200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 msgr2=0x5592f0ce0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0ce0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 msgr2=0x5592f0cddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0cddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0cddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3769134200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f09ad2b0 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0cddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009610 tx=0x7fbcc8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deb76fdc3f4d02d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8020040 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbcc8086220 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd40317b0 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbcc809ffb0 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 10 client.15129 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.422+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592f0ceefb0 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd40317b0 con 0x5592f0cd62f0 2024-09-15T22:11:52.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbcc8090fd0 con 0x5592f0cd62f0 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 10 client.15129 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 20 client.15129 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0cf1620 0x5592f0cf2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 4 client.15129 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592f0c49600 con 0x5592f0cf1620 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0cf1620 0x5592f0cf2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbcc1ffb640 10 client.15129 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.423+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0cf1620 0x5592f0cf2a90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3832d673c270167a server_cookie=e01a6a70cc1e0e07 in_seq=0 out_seq=0 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3769134200 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592f0c49600 con 0x5592f0cf1620 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbcc1ffb640 10 client.15129 handle_command_reply: tid=0 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 client.15129 shutdown 2024-09-15T22:11:52.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 msgr2=0x5592f0cdd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009610 tx=0x7fbcc8009f20 comp rx=0 tx=0).stop 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 shutdown_connections 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0cf1620 0x5592f0cf2a90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0cddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cdd6d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbcc1ffb640 0 client.15129 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3769134200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0ce0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 >> 172.21.15.77:0/3769134200 conn(0x5592f0cd1f60 msgr2=0x5592f0cd7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 shutdown_connections 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.424+0000 7fbce2aee740 1 -- 172.21.15.77:0/3769134200 wait complete. 2024-09-15T22:11:52.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.425+0000 7fbce2aee740 20 client.15129 trim_cache size 0 max 16384 2024-09-15T22:11:52.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 Processor -- start 2024-09-15T22:11:52.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 -- start start 2024-09-15T22:11:52.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0cd59d0 0x5592f0cf2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0ce0bb0 0x5592f0cdbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf0dd0 con 0x5592f0cd62f0 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf14a0 con 0x5592f0cd59d0 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbce2aee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf1b70 con 0x5592f0ce0bb0 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.442+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54792/0 (socket says 172.21.15.77:54792) 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3374609050 learned_addr learned my addr 172.21.15.77:0/3374609050 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0cd59d0 0x5592f0cf2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3374609050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0ce0bb0 msgr2=0x5592f0cdbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0ce0bb0 0x5592f0cdbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3374609050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0cd59d0 msgr2=0x5592f0cf2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0cd59d0 0x5592f0cf2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3374609050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592f0cd1d90 con 0x5592f0cd62f0 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80a4340 tx=0x7fbcc809a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a3e1c787ec18e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcc3fff640 1 -- 172.21.15.77:0/3374609050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8020040 con 0x5592f0cd62f0 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcc3fff640 1 -- 172.21.15.77:0/3374609050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc8098270 con 0x5592f0cd62f0 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.443+0000 7fbcc3fff640 1 -- 172.21.15.77:0/3374609050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8010460 con 0x5592f0cd62f0 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- 172.21.15.77:0/3374609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 msgr2=0x5592f0cccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80a4340 tx=0x7fbcc809a110 comp rx=0 tx=0).stop 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- 172.21.15.77:0/3374609050 shutdown_connections 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0ce0bb0 0x5592f0cdbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cccdf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3374609050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0cd59d0 0x5592f0cf2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- 172.21.15.77:0/3374609050 >> 172.21.15.77:0/3374609050 conn(0x5592f0cd1f60 msgr2=0x5592f0cdd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- 172.21.15.77:0/3374609050 shutdown_connections 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- 172.21.15.77:0/3374609050 wait complete. 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- start start 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 Processor -- start 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0cf62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0d59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf2910 con 0x5592f0cd62f0 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf2240 con 0x5592f0ce0bb0 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592f0cf2fe0 con 0x5592f0cd59d0 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.444+0000 7fbce2aee740 10 client.0 _ll_register_callbacks cb 0x7fbcc8091410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:52.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54802/0 (socket says 172.21.15.77:54802) 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3189235932 learned_addr learned my addr 172.21.15.77:0/3189235932 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0cf62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0d59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3189235932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 msgr2=0x5592f0cf62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0cf62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 msgr2=0x5592f0d59d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0d59d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 -- 172.21.15.77:0/3189235932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592f0d5a330 con 0x5592f0cd62f0 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcdecb2640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0d59d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcddcb0640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcc809b900 tx=0x7fbcc8086450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79f2d6be84524f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80ba060 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbcc80b4050 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc8098270 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbcc8091050 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbce2aee740 10 client.15135 fetch_fsmap learned FSMap version 29 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592f0d5a330 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.445+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc8098270 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbcc809aac0 con 0x5592f0cd62f0 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 10 client.15135 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 20 client.15135 populate_metadata read hostname 'smithi077' 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0d5fb30 0x5592f0d5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 4 client.15135 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:11:52.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "status"}) -- 0x5592f0c43b10 con 0x5592f0d5fb30 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0d5fb30 0x5592f0d5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbcc1ffb640 10 client.15135 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.446+0000 7fbcde4b1640 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0d5fb30 0x5592f0d5e840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91634ebd58021bf4 server_cookie=f99aea3beca83b83 in_seq=0 out_seq=0 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbcc1ffb640 1 -- 172.21.15.77:0/3189235932 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5592f0c43b10 con 0x5592f0d5fb30 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbcc1ffb640 10 client.15135 handle_command_reply: tid=0 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 client.15135 shutdown 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 msgr2=0x5592f0cf67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcc809b900 tx=0x7fbcc8086450 comp rx=0 tx=0).stop 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 shutdown_connections 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592f0d5fb30 0x5592f0d5e840 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592f0ce0bb0 0x5592f0d59d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbcc1ffb640 0 client.15135 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:11:52.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592f0cd62f0 0x5592f0cf67f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 --2- 172.21.15.77:0/3189235932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592f0cd59d0 0x5592f0cf62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 >> 172.21.15.77:0/3189235932 conn(0x5592f0cd1f60 msgr2=0x5592f0cdbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 shutdown_connections 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 1 -- 172.21.15.77:0/3189235932 wait complete. 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce2aee740 20 client.15135 trim_cache size 0 max 16384 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:11:52.459 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24458, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 116.24932642, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 127.05316619600001 2024-09-15T22:11:52.460 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 msgr2=0x7fbcc40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 0x7fbcc40a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd0014a30 comp rx=0 tx=0).stop 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 msgr2=0x7fbcd81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.447+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800d950 tx=0x7fbcc800de10 comp rx=0 tx=0).stop 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 shutdown_connections 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3350 0x7fbcc40a5810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8106f10 0x7fbcd819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102790 0x7fbcd81989d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 --2- 172.21.15.77:0/377018209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101de0 0x7fbcd8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 >> 172.21.15.77:0/377018209 conn(0x7fbcd80fd8c0 msgr2=0x7fbcd80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 shutdown_connections 2024-09-15T22:11:52.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:52.448+0000 7fbce0715640 1 -- 172.21.15.77:0/377018209 wait complete. 2024-09-15T22:11:53.203 INFO:tasks.cephadm:Setup complete, yielding 2024-09-15T22:11:53.203 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:11:53.215 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:11:53.215 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-15T22:11:53.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:53 smithi077 ceph-mon[30980]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:53.461 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:53 smithi196 ceph-mon[35109]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:53 smithi117 ceph-mon[35021]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:53.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/1493304843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc102680 msgr2=0x7f2bbc104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/1493304843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc102680 0x7f2bbc104a70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb802a410 comp rx=0 tx=0).stop 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/1493304843 shutdown_connections 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/1493304843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/1493304843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 0x7f2bbc1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/1493304843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc102680 0x7f2bbc104a70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/1493304843 >> 172.21.15.77:0/1493304843 conn(0x7f2bbc0fc2a0 msgr2=0x7f2bbc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/1493304843 shutdown_connections 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/1493304843 wait complete. 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 Processor -- start 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- start start 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc102680 0x7f2bbc101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 0x7f2bbc100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10d900 con 0x7f2bbc1078e0 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10d780 con 0x7f2bbc102680 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.953+0000 7f2bc3cb4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc10da80 con 0x7f2bbc104fb0 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54824/0 (socket says 172.21.15.77:54824) 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 -- 172.21.15.77:0/3642859577 learned_addr learned my addr 172.21.15.77:0/3642859577 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc1a50640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc102680 0x7f2bbc101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc124f640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 0x7f2bbc100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 -- 172.21.15.77:0/3642859577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 msgr2=0x7f2bbc100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 0x7f2bbc100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 -- 172.21.15.77:0/3642859577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc102680 msgr2=0x7f2bbc101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc102680 0x7f2bbc101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 -- 172.21.15.77:0/3642859577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc101430 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc2251640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cce0 tx=0x7f2bb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f450 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb400fad0 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1a2100 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4018b60 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.954+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1a2690 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.955+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc0fdd90 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.956+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2bb400fc80 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.957+0000 7f2bb2ffd640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 0x7f2ba80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.957+0000 7f2bc1a50640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 0x7f2ba80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.957+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2bb40d0560 con 0x7f2bbc1078e0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.958+0000 7f2bc1a50640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 0x7f2ba80a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80314e0 tx=0x7f2bb8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:53.960+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bb4014030 con 0x7f2bbc1078e0 2024-09-15T22:11:54.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.143+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2bbc062630 con 0x7f2ba80a3350 2024-09-15T22:11:54.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.145+0000 7f2bb2ffd640 1 -- 172.21.15.77:0/3642859577 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2bbc062630 con 0x7f2ba80a3350 2024-09-15T22:11:54.146 INFO:teuthology.orchestra.run.smithi077.stdout:Backend: cephadm 2024-09-15T22:11:54.146 INFO:teuthology.orchestra.run.smithi077.stdout:Available: Yes 2024-09-15T22:11:54.146 INFO:teuthology.orchestra.run.smithi077.stdout:Paused: No 2024-09-15T22:11:54.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 msgr2=0x7f2ba80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 0x7f2ba80a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80314e0 tx=0x7f2bb8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 msgr2=0x7f2bbc100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cce0 tx=0x7f2bb400a620 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 shutdown_connections 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f2ba80a3350 0x7f2ba80a5810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bbc1078e0 0x7f2bbc100b70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bbc104fb0 0x7f2bbc100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 --2- 172.21.15.77:0/3642859577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bbc102680 0x7f2bbc101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 >> 172.21.15.77:0/3642859577 conn(0x7f2bbc0fc2a0 msgr2=0x7f2bbc102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 shutdown_connections 2024-09-15T22:11:54.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:54.147+0000 7f2bc3cb4640 1 -- 172.21.15.77:0/3642859577 wait complete. 2024-09-15T22:11:54.961 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-15T22:11:55.548 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:55 smithi077 ceph-mon[30980]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:55.548 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:55 smithi077 ceph-mon[30980]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:55.549 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:55 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:55 smithi196 ceph-mon[35109]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:55 smithi196 ceph-mon[35109]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:55.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:55 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:55 smithi117 ceph-mon[35021]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:55 smithi117 ceph-mon[35021]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:11:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:55 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:55.963 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:56.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 -- 172.21.15.77:0/2714902166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b4102ea0 msgr2=0x7f93b4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2714902166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b4102ea0 0x7f93b4103320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009930 tx=0x7f93a402a420 comp rx=0 tx=0).stop 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 -- 172.21.15.77:0/2714902166 shutdown_connections 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2714902166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4103860 0x7f93b410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2714902166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b4102ea0 0x7f93b4103320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2714902166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b40ffec0 0x7f93b41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 -- 172.21.15.77:0/2714902166 >> 172.21.15.77:0/2714902166 conn(0x7f93b40fba10 msgr2=0x7f93b40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 -- 172.21.15.77:0/2714902166 shutdown_connections 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.452+0000 7f93b8c74640 1 -- 172.21.15.77:0/2714902166 wait complete. 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 Processor -- start 2024-09-15T22:11:56.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 -- start start 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4102ea0 0x7f93b4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b4103860 0x7f93b4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93b410d900 con 0x7f93b40ffec0 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93b410d780 con 0x7f93b4103860 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b8c74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93b410da80 con 0x7f93b4102ea0 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54844/0 (socket says 172.21.15.77:54844) 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.453+0000 7f93b259c640 1 -- 172.21.15.77:0/2866443250 learned_addr learned my addr 172.21.15.77:0/2866443250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b1d9b640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4102ea0 0x7f93b4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b2d9d640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b4103860 0x7f93b4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 -- 172.21.15.77:0/2866443250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4102ea0 msgr2=0x7f93b4100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4102ea0 0x7f93b4100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 -- 172.21.15.77:0/2866443250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b4103860 msgr2=0x7f93b4101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b4103860 0x7f93b4101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 -- 172.21.15.77:0/2866443250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4101c50 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b259c640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f93a800d770 tx=0x7f93a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8017070 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a8009c30 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41a21a0 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8003dd0 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.454+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41a2730 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.455+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b4109190 con 0x7f93b40ffec0 2024-09-15T22:11:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.456+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f93a8003fb0 con 0x7f93b40ffec0 2024-09-15T22:11:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.457+0000 7f93a37fe640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 0x7f939c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.457+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f93a80cf840 con 0x7f93b40ffec0 2024-09-15T22:11:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.457+0000 7f93b1d9b640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 0x7f939c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.457+0000 7f93b1d9b640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 0x7f939c0a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000c00 tx=0x7f93a40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.461+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93a809c0a0 con 0x7f93b40ffec0 2024-09-15T22:11:56.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.671+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f93b4062630 con 0x7f939c0a32d0 2024-09-15T22:11:56.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.684+0000 7f93a37fe640 1 -- 172.21.15.77:0/2866443250 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f93b4062630 con 0x7f939c0a32d0 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.a smithi077 running (2m) 63s ago 2m 20.6M - 19.1.1-326-g8921c8ef f1455bd0219c 887f7da9ecf3 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.b smithi117 running (2m) 0s ago 2m 27.3M - 19.1.1-326-g8921c8ef f1455bd0219c 8b356d65b869 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.c smithi196 running (2m) 0s ago 2m 19.4M - 19.1.1-326-g8921c8ef f1455bd0219c c3b84b97d5af 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.d smithi077 running (2m) 63s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c 1afd74b55798 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.e smithi117 running (2m) 0s ago 2m 19.5M - 19.1.1-326-g8921c8ef f1455bd0219c 9fb8f1e2bfe3 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.f smithi196 running (2m) 0s ago 2m 20.0M - 19.1.1-326-g8921c8ef f1455bd0219c a14020065d39 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.g smithi077 running (2m) 63s ago 2m 15.5M - 19.1.1-326-g8921c8ef f1455bd0219c 3f91ebb6c0df 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.h smithi117 running (2m) 0s ago 2m 21.1M - 19.1.1-326-g8921c8ef f1455bd0219c 8b45cdfbec19 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.i smithi196 running (2m) 0s ago 2m 19.4M - 19.1.1-326-g8921c8ef f1455bd0219c 956abfeb375a 2024-09-15T22:11:56.686 INFO:teuthology.orchestra.run.smithi077.stdout:mds.j smithi077 running (2m) 63s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 966d312e222a 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mds.k smithi117 running (2m) 0s ago 2m 17.8M - 19.1.1-326-g8921c8ef f1455bd0219c a54f2434676d 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mds.l smithi196 running (2m) 0s ago 2m 20.0M - 19.1.1-326-g8921c8ef f1455bd0219c db40991e161a 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.x smithi077 *:9283,8765 running (9m) 63s ago 9m 508M - 19.1.1-326-g8921c8ef f1455bd0219c fcb0d844aa0a 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.y smithi117 *:8443,8765 running (7m) 0s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 3d9a106936dc 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.z smithi196 *:8443,8765 running (7m) 0s ago 7m 444M - 19.1.1-326-g8921c8ef f1455bd0219c 55933dc495ab 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mon.a smithi077 running (9m) 63s ago 9m 51.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ac1952b1d157 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mon.b smithi117 running (8m) 0s ago 8m 45.0M 2048M 19.1.1-326-g8921c8ef f1455bd0219c b5703de2a2f1 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:mon.c smithi196 running (8m) 0s ago 8m 41.2M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 2b2a1bd3652b 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (7m) 63s ago 7m 56.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 31c7c9eab21e 2024-09-15T22:11:56.687 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi117 running (7m) 0s ago 7m 59.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c faa5da505312 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi196 running (6m) 0s ago 6m 62.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c ff7fb82ba6e8 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi077 running (6m) 63s ago 6m 54.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c e3abf7208a3b 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi117 running (5m) 0s ago 5m 46.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 6dad0b316c4c 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi196 running (5m) 0s ago 5m 44.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 85a224fcbb61 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 smithi077 running (5m) 63s ago 5m 42.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c bc0e860bcefd 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 smithi117 running (4m) 0s ago 4m 55.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c f0925eb253da 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 smithi196 running (4m) 0s ago 4m 54.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4882d279b52b 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 smithi077 running (3m) 63s ago 3m 52.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 344ae49ca4e5 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 smithi117 running (3m) 0s ago 3m 58.4M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 7e5e31cd1815 2024-09-15T22:11:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 smithi196 running (3m) 0s ago 3m 51.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 621f829e8b04 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.686+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 msgr2=0x7f939c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.686+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 0x7f939c0a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000c00 tx=0x7f93a40315b0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.686+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 msgr2=0x7f93b4102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.686+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f93a800d770 tx=0x7f93a800dc30 comp rx=0 tx=0).stop 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 shutdown_connections 2024-09-15T22:11:56.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f939c0a32d0 0x7f939c0a5790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93b4103860 0x7f93b4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93b4102ea0 0x7f93b4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 --2- 172.21.15.77:0/2866443250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b40ffec0 0x7f93b4102800 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 >> 172.21.15.77:0/2866443250 conn(0x7f93b40fba10 msgr2=0x7f93b4106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 shutdown_connections 2024-09-15T22:11:56.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:56.687+0000 7f93b8c74640 1 -- 172.21.15.77:0/2866443250 wait complete. 2024-09-15T22:11:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:56 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:56 smithi077 ceph-mon[30980]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:57.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:56 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:56 smithi196 ceph-mon[35109]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:56 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:57.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:56 smithi117 ceph-mon[35021]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-15T22:11:57.452 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-15T22:11:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:57 smithi077 ceph-mon[30980]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 -- 172.21.15.77:0/1231393034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8102690 msgr2=0x7f7af8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/1231393034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8102690 0x7f7af8108f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009ca0 tx=0x7f7af002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 -- 172.21.15.77:0/1231393034 shutdown_connections 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/1231393034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8102690 0x7f7af8108f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/1231393034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8101cd0 0x7f7af8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/1231393034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 0x7f7af8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.207+0000 7f7afc8ee640 1 -- 172.21.15.77:0/1231393034 >> 172.21.15.77:0/1231393034 conn(0x7f7af80fc280 msgr2=0x7f7af80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.208+0000 7f7afc8ee640 1 -- 172.21.15.77:0/1231393034 shutdown_connections 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.208+0000 7f7afc8ee640 1 -- 172.21.15.77:0/1231393034 wait complete. 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.208+0000 7f7afc8ee640 1 Processor -- start 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.208+0000 7f7afc8ee640 1 -- start start 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 0x7f7af8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8102690 0x7f7af819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d8e0 con 0x7f7af8101cd0 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810d760 con 0x7f7af8102690 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7af810da60 con 0x7f7af8100ad0 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:58.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54860/0 (socket says 172.21.15.77:54860) 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 -- 172.21.15.77:0/2966577543 learned_addr learned my addr 172.21.15.77:0/2966577543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7af659c640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 0x7f7af8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7af6d9d640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8102690 0x7f7af819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 -- 172.21.15.77:0/2966577543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 msgr2=0x7f7af8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 0x7f7af8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 -- 172.21.15.77:0/2966577543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8102690 msgr2=0x7f7af819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8102690 0x7f7af819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 -- 172.21.15.77:0/2966577543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819fb60 con 0x7f7af8101cd0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aedd9b640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000d950 tx=0x7f7ae000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0017070 con 0x7f7af8101cd0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae000de80 con 0x7f7af8101cd0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819fe50 con 0x7f7af8101cd0 2024-09-15T22:11:58.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.209+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000f620 con 0x7f7af8101cd0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.210+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af81a0380 con 0x7f7af8101cd0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.211+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af8107ef0 con 0x7f7af8101cd0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.212+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7ae0009c30 con 0x7f7af8101cd0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.213+0000 7f7aeffff640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 0x7f7ae40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.213+0000 7f7af659c640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 0x7f7ae40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.213+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7ae00cf950 con 0x7f7af8101cd0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.213+0000 7f7af659c640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 0x7f7ae40a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7af8101b30 tx=0x7f7ae8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:58.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.216+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ae009c1b0 con 0x7f7af8101cd0 2024-09-15T22:11:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:57 smithi196 ceph-mon[35109]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:11:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:58.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:57 smithi117 ceph-mon[35021]: from='mgr.14150 172.21.15.77:0/3783172285' entity='mgr.x' 2024-09-15T22:11:58.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.391+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7af8062630 con 0x7f7ae40a3300 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.397+0000 7f7aeffff640 1 -- 172.21.15.77:0/2966577543 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7af8062630 con 0x7f7ae40a3300 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stdout:mds.all 12/12 2s ago 2m smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stdout:mgr 3/3 2s ago 8m smithi077=x;smithi117=y;smithi196=z;count:3 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stdout:mon 3/3 2s ago 8m smithi077:172.21.15.77=a;smithi117:172.21.15.117=b;smithi196:172.21.15.196=c;count:3 2024-09-15T22:11:58.398 INFO:teuthology.orchestra.run.smithi077.stdout:osd 12 2s ago - 2024-09-15T22:11:58.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 msgr2=0x7f7ae40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:58.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 0x7f7ae40a57c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7af8101b30 tx=0x7f7ae8009110 comp rx=0 tx=0).stop 2024-09-15T22:11:58.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 msgr2=0x7f7af8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000d950 tx=0x7f7ae000dd50 comp rx=0 tx=0).stop 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 shutdown_connections 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f7ae40a3300 0x7f7ae40a57c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7af8102690 0x7f7af819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7af8101cd0 0x7f7af8198470 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 --2- 172.21.15.77:0/2966577543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af8100ad0 0x7f7af8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 >> 172.21.15.77:0/2966577543 conn(0x7f7af80fc280 msgr2=0x7f7af81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 shutdown_connections 2024-09-15T22:11:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:58.399+0000 7f7afc8ee640 1 -- 172.21.15.77:0/2966577543 wait complete. 2024-09-15T22:11:59.061 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-15T22:11:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:59.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:58 smithi077 ceph-mon[30980]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:59.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:58 smithi196 ceph-mon[35109]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:59.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:58 smithi117 ceph-mon[35021]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:11:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/3462123284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0100b10 msgr2=0x7f9ba0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/3462123284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0100b10 0x7f9ba0100f10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009900 tx=0x7f9b9402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/3462123284 shutdown_connections 2024-09-15T22:11:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/3462123284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba01026d0 0x7f9ba0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/3462123284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0101d10 0x7f9ba0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/3462123284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0100b10 0x7f9ba0100f10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/3462123284 >> 172.21.15.77:0/3462123284 conn(0x7f9ba00fc2a0 msgr2=0x7f9ba00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/3462123284 shutdown_connections 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.792+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/3462123284 wait complete. 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 Processor -- start 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 -- start start 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 0x7f9ba0196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba01026d0 0x7f9ba019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010c7f0 con 0x7f9ba0101d10 2024-09-15T22:11:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010c670 con 0x7f9ba01026d0 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba6fb1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010c970 con 0x7f9ba0100b10 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58840/0 (socket says 172.21.15.77:58840) 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 -- 172.21.15.77:0/4143984827 learned_addr learned my addr 172.21.15.77:0/4143984827 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba4d4d640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 0x7f9ba0196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 -- 172.21.15.77:0/4143984827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 msgr2=0x7f9ba0196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 0x7f9ba0196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 -- 172.21.15.77:0/4143984827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba01026d0 msgr2=0x7f9ba019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba01026d0 0x7f9ba019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 -- 172.21.15.77:0/4143984827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba019ea10 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9ba4d4d640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 0x7f9ba0196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9ffff640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c920 tx=0x7f9b8c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.793+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c007c70 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.794+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba019ed00 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.794+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c007e10 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.794+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c018890 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.794+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba019f260 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.795+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba0107e40 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.796+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9b8c018a30 con 0x7f9ba0101d10 2024-09-15T22:11:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.796+0000 7f9b9dffb640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 0x7f9b900a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.796+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9b8c0cffa0 con 0x7f9ba0101d10 2024-09-15T22:11:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.796+0000 7f9ba4d4d640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 0x7f9b900a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.797+0000 7f9ba4d4d640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 0x7f9b900a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b94006010 tx=0x7f9b94000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.800+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b8c014030 con 0x7f9ba0101d10 2024-09-15T22:11:59.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.974+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9ba0062630 con 0x7f9b900a3300 2024-09-15T22:11:59.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.975+0000 7f9b9dffb640 1 -- 172.21.15.77:0/4143984827 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9ba0062630 con 0x7f9b900a3300 2024-09-15T22:11:59.977 INFO:teuthology.orchestra.run.smithi077.stdout:HOST ADDR LABELS STATUS 2024-09-15T22:11:59.977 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 172.21.15.77 2024-09-15T22:11:59.977 INFO:teuthology.orchestra.run.smithi077.stdout:smithi117 172.21.15.117 2024-09-15T22:11:59.977 INFO:teuthology.orchestra.run.smithi077.stdout:smithi196 172.21.15.196 2024-09-15T22:11:59.977 INFO:teuthology.orchestra.run.smithi077.stdout:3 hosts in cluster 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 msgr2=0x7f9b900a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 0x7f9b900a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b94006010 tx=0x7f9b94000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 msgr2=0x7f9ba0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c920 tx=0x7f9b8c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 shutdown_connections 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f9b900a3300 0x7f9b900a57c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ba01026d0 0x7f9ba019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba0101d10 0x7f9ba0197320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 --2- 172.21.15.77:0/4143984827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba0100b10 0x7f9ba0196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:59.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.977+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 >> 172.21.15.77:0/4143984827 conn(0x7f9ba00fc2a0 msgr2=0x7f9ba0105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:59.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.978+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 shutdown_connections 2024-09-15T22:11:59.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:11:59.978+0000 7f9ba6fb1640 1 -- 172.21.15.77:0/4143984827 wait complete. 2024-09-15T22:12:00.268 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:11:59 smithi077 ceph-mon[30980]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:11:59 smithi196 ceph-mon[35109]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:00.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:11:59 smithi117 ceph-mon[35021]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:00.710 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-15T22:12:00.965 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:01.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:00 smithi077 ceph-mon[30980]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:01.246 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:00 smithi077 ceph-mon[30980]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:12:01.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:00 smithi196 ceph-mon[35109]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:01.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:00 smithi196 ceph-mon[35109]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:12:01.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:00 smithi117 ceph-mon[35021]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:01.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:00 smithi117 ceph-mon[35021]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 -- 172.21.15.77:0/1169474468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 msgr2=0x7f4cd8074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 --2- 172.21.15.77:0/1169474468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd8074fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009900 tx=0x7f4cc002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 -- 172.21.15.77:0/1169474468 shutdown_connections 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 --2- 172.21.15.77:0/1169474468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 0x7f4cd810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 --2- 172.21.15.77:0/1169474468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd8074fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 --2- 172.21.15.77:0/1169474468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 0x7f4cd8074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 -- 172.21.15.77:0/1169474468 >> 172.21.15.77:0/1169474468 conn(0x7f4cd80fd3c0 msgr2=0x7f4cd80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 -- 172.21.15.77:0/1169474468 shutdown_connections 2024-09-15T22:12:01.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 -- 172.21.15.77:0/1169474468 wait complete. 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.492+0000 7f4cdde07640 1 Processor -- start 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 -- start start 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 0x7f4cd819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 0x7f4cd81a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810fa90 con 0x7f4cd8074b40 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810f910 con 0x7f4cd80761a0 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cdde07640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd810fc10 con 0x7f4cd8109a60 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58858/0 (socket says 172.21.15.77:58858) 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 -- 172.21.15.77:0/3513179869 learned_addr learned my addr 172.21.15.77:0/3513179869 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd7fff640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 0x7f4cd81a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:01.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4ccffff640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 0x7f4cd819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 -- 172.21.15.77:0/3513179869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 msgr2=0x7f4cd81a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 0x7f4cd81a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 -- 172.21.15.77:0/3513179869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 msgr2=0x7f4cd819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 0x7f4cd819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 -- 172.21.15.77:0/3513179869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd81a1c60 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd77fe640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800c920 tx=0x7f4cc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.493+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8007c70 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.494+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc8007e10 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.494+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81a1f50 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.494+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8018890 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.494+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a23a0 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.495+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd8108870 con 0x7f4cd8074b40 2024-09-15T22:12:01.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.496+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4cc8018a30 con 0x7f4cd8074b40 2024-09-15T22:12:01.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.497+0000 7f4cd57fa640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 0x7f4cc40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:01.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.497+0000 7f4ccffff640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 0x7f4cc40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:01.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.497+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4cc80d0160 con 0x7f4cd8074b40 2024-09-15T22:12:01.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.497+0000 7f4ccffff640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 0x7f4cc40a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0031440 tx=0x7f4cc00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:01.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.501+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cc8014030 con 0x7f4cd8074b40 2024-09-15T22:12:01.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.672+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4cd8062630 con 0x7f4cc40a3300 2024-09-15T22:12:01.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.674+0000 7f4cd57fa640 1 -- 172.21.15.77:0/3513179869 <== mgr.14150 v2:172.21.15.77:6800/1809193257 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4cd8062630 con 0x7f4cc40a3300 2024-09-15T22:12:01.675 INFO:teuthology.orchestra.run.smithi077.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-15T22:12:01.675 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:12:01.675 INFO:teuthology.orchestra.run.smithi077.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:12:01.675 INFO:teuthology.orchestra.run.smithi077.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T22:12:01.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 msgr2=0x7f4cc40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:01.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 0x7f4cc40a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0031440 tx=0x7f4cc00315b0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 msgr2=0x7f4cd819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800c920 tx=0x7f4cc800cde0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 shutdown_connections 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f4cc40a3300 0x7f4cc40a57c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8109a60 0x7f4cd81a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd80761a0 0x7f4cd819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 --2- 172.21.15.77:0/3513179869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8074b40 0x7f4cd819a030 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 >> 172.21.15.77:0/3513179869 conn(0x7f4cd80fd3c0 msgr2=0x7f4cd8105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 shutdown_connections 2024-09-15T22:12:01.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:01.676+0000 7f4cdde07640 1 -- 172.21.15.77:0/3513179869 wait complete. 2024-09-15T22:12:02.352 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:12:02.361 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:12:02.361 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-15T22:12:02.608 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 -- 172.21.15.77:0/2759693159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc1020a0 msgr2=0x7f52cc1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/2759693159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc1020a0 0x7f52cc1024a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009900 tx=0x7f52bc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 -- 172.21.15.77:0/2759693159 shutdown_connections 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/2759693159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1017a0 0x7f52cc109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/2759693159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc100de0 0x7f52cc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/2759693159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc1020a0 0x7f52cc1024a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 -- 172.21.15.77:0/2759693159 >> 172.21.15.77:0/2759693159 conn(0x7f52cc0fba40 msgr2=0x7f52cc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 -- 172.21.15.77:0/2759693159 shutdown_connections 2024-09-15T22:12:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.073+0000 7f52d0ddb640 1 -- 172.21.15.77:0/2759693159 wait complete. 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 Processor -- start 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 -- start start 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1017a0 0x7f52cc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1020a0 0x7f52cc1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10c780 con 0x7f52cc100de0 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10c600 con 0x7f52cc1020a0 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52d0ddb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10c900 con 0x7f52cc1017a0 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52ca59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52ca59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58874/0 (socket says 172.21.15.77:58874) 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52ca59c640 1 -- 172.21.15.77:0/545386796 learned_addr learned my addr 172.21.15.77:0/545386796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:03.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52c3fff640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1017a0 0x7f52cc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.074+0000 7f52cad9d640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1020a0 0x7f52cc1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 -- 172.21.15.77:0/545386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1017a0 msgr2=0x7f52cc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1017a0 0x7f52cc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 -- 172.21.15.77:0/545386796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1020a0 msgr2=0x7f52cc1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1020a0 0x7f52cc1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 -- 172.21.15.77:0/545386796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc1a1d40 con 0x7f52cc100de0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52ca59c640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc03f030 con 0x7f52cc100de0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52bc003ce0 con 0x7f52cc100de0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1a1f70 con 0x7f52cc100de0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035070 con 0x7f52cc100de0 2024-09-15T22:12:03.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.075+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a23c0 con 0x7f52cc100de0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.076+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc108010 con 0x7f52cc100de0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.077+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f52bc042030 con 0x7f52cc100de0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.078+0000 7f52c37fe640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 0x7f52b80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.078+0000 7f52c3fff640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 0x7f52b80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.078+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f52bc0f0700 con 0x7f52cc100de0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.078+0000 7f52c3fff640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 0x7f52b80a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005e10 tx=0x7f52b4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:03.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.081+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52bc0bcf60 con 0x7f52cc100de0 2024-09-15T22:12:03.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.291+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f52cc02d7c0 con 0x7f52cc100de0 2024-09-15T22:12:03.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.292+0000 7f52c37fe640 1 -- 172.21.15.77:0/545386796 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3140 (secure 0 0 0) 0x7f52bc0bc860 con 0x7f52cc100de0 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:e29 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:btime 2024-09-15T22:10:40:741597+0000 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 29 2024-09-15T22:12:03.294 INFO:teuthology.orchestra.run.smithi077.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-09-15T22:09:55.819962+0000 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-09-15T22:10:40.090210+0000 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 300 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 5 2024-09-15T22:12:03.295 INFO:teuthology.orchestra.run.smithi077.stdout:in 0,1,2,3,4 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=24458,1=24440,2=24439,3=24424,4=14604} 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3,4] 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data disabled 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 0 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 24458 members: 14604,24458,24424,24439,24440 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b{0:24458} state up:active seq 4 addr [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.296 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.e{0:24422} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/3112549785,v1:172.21.15.117:6835/3112549785] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.h{1:24440} state up:active seq 17 addr [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.f{1:24418} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/889715770,v1:172.21.15.196:6835/889715770] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.i{2:24439} state up:active seq 17 addr [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.k{2:24404} state up:standby-replay seq 1 addr [v2:172.21.15.117:6832/2823149773,v1:172.21.15.117:6833/2823149773] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c{3:24424} state up:active seq 18 addr [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.l{3:24385} state up:standby-replay seq 1 addr [v2:172.21.15.196:6832/3132990127,v1:172.21.15.196:6833/3132990127] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a{4:14604} state up:active seq 16 addr [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.g{4:14592} state up:standby-replay seq 1 addr [v2:172.21.15.77:6838/2629133747,v1:172.21.15.77:6839/2629133747] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.77:6834/1898309607,v1:172.21.15.77:6835/1898309607] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.297 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.77:6836/1847869314,v1:172.21.15.77:6837/1847869314] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:12:03.298 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:12:03.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 msgr2=0x7f52b80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 0x7f52b80a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005e10 tx=0x7f52b4005da0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 msgr2=0x7f52cc100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc004360 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 shutdown_connections 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f52b80a3300 0x7f52b80a57c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52cc1020a0 0x7f52cc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52cc1017a0 0x7f52cc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 --2- 172.21.15.77:0/545386796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52cc100de0 0x7f52cc100830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 >> 172.21.15.77:0/545386796 conn(0x7f52cc0fba40 msgr2=0x7f52cc1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 shutdown_connections 2024-09-15T22:12:03.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:03.294+0000 7f52d0ddb640 1 -- 172.21.15.77:0/545386796 wait complete. 2024-09-15T22:12:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:03 smithi077 ceph-mon[30980]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:03 smithi077 ceph-mon[30980]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:12:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:03 smithi196 ceph-mon[35109]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:03 smithi196 ceph-mon[35109]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:12:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:03 smithi117 ceph-mon[35021]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:03 smithi117 ceph-mon[35021]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:12:04.022 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-15T22:12:04.290 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:04.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:04 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/545386796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:12:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:04 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/545386796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:12:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:04 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/545386796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:12:04.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- 172.21.15.77:0/581397090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff110104f90 msgr2=0x7ff110107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:04.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- 172.21.15.77:0/581397090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff110104f90 0x7ff110107380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff100009900 tx=0x7ff10002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- 172.21.15.77:0/581397090 shutdown_connections 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- 172.21.15.77:0/581397090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1101078c0 0x7ff110109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- 172.21.15.77:0/581397090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff110104f90 0x7ff110107380 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- 172.21.15.77:0/581397090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 0x7ff110104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- 172.21.15.77:0/581397090 >> 172.21.15.77:0/581397090 conn(0x7ff1100fc280 msgr2=0x7ff1100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- 172.21.15.77:0/581397090 shutdown_connections 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- 172.21.15.77:0/581397090 wait complete. 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 Processor -- start 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 -- start start 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 0x7ff110101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.751+0000 7ff118702640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 0x7ff110100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:04.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff11010d860 con 0x7ff1101078c0 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff11010d6e0 con 0x7ff110104f90 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff11010d9e0 con 0x7ff110102660 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58882/0 (socket says 172.21.15.77:58882) 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 -- 172.21.15.77:0/1481410561 learned_addr learned my addr 172.21.15.77:0/1481410561 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff115c9d640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 0x7ff110100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff11649e640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 0x7ff110101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 -- 172.21.15.77:0/1481410561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 msgr2=0x7ff110101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 0x7ff110101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 -- 172.21.15.77:0/1481410561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 msgr2=0x7ff110100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 0x7ff110100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 -- 172.21.15.77:0/1481410561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff110101410 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff115c9d640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 0x7ff110100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff116c9f640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff10800eaa0 tx=0x7ff10800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10800cdb0 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff108005020 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101a20e0 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1080116b0 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.752+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101a25a0 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.753+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff110105370 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.754+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff108005560 con 0x7ff1101078c0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.755+0000 7ff1077fe640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 0x7ff0fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.755+0000 7ff11649e640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 0x7ff0fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:04.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.755+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff1080d04f0 con 0x7ff1101078c0 2024-09-15T22:12:04.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.756+0000 7ff11649e640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 0x7ff0fc0a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005fd0 tx=0x7ff10c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:04.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.758+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff108016030 con 0x7ff1101078c0 2024-09-15T22:12:04.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.930+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff1100fdd70 con 0x7ff1101078c0 2024-09-15T22:12:04.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.931+0000 7ff1077fe640 1 -- 172.21.15.77:0/1481410561 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5610 (secure 0 0 0) 0x7ff1100fdd70 con 0x7ff1101078c0 2024-09-15T22:12:04.933 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 90 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-09-15T22:02:08.800745+0000 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-09-15T22:11:05.224880+0000 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:crush_version 27 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:full_ratio 0.95 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:backfillfull_ratio 0.9 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:nearfull_ratio 0.85 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:require_min_compat_client luminous 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:min_compat_client luminous 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:require_osd_release squid 2024-09-15T22:12:04.934 INFO:teuthology.orchestra.run.smithi077.stdout:stretch_mode_enabled false 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:max_osd 12 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6802/4041675784,v1:172.21.15.77:6803/4041675784] [v2:172.21.15.77:6804/4041675784,v1:172.21.15.77:6805/4041675784] exists,up 286650d4-5c20-4659-a7ef-db7c243e0aba 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/3227694760,v1:172.21.15.117:6801/3227694760] [v2:172.21.15.117:6802/3227694760,v1:172.21.15.117:6803/3227694760] exists,up 5cddb21e-cf12-4806-9540-0b7c570af2aa 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/3745852928,v1:172.21.15.196:6801/3745852928] [v2:172.21.15.196:6802/3745852928,v1:172.21.15.196:6803/3745852928] exists,up 3e1d614a-34ca-45c9-b2c2-09d60c5c0821 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 up in weight 1 up_from 24 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6810/2407517217,v1:172.21.15.77:6811/2407517217] [v2:172.21.15.77:6812/2407517217,v1:172.21.15.77:6813/2407517217] exists,up 42be2980-1ab3-42e7-9256-eb759230d6e4 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/1062550497,v1:172.21.15.117:6809/1062550497] [v2:172.21.15.117:6810/1062550497,v1:172.21.15.117:6811/1062550497] exists,up 7db7bf42-1fed-4197-9bad-58e8a56f816a 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 up in weight 1 up_from 37 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/2355584153,v1:172.21.15.196:6809/2355584153] [v2:172.21.15.196:6810/2355584153,v1:172.21.15.196:6811/2355584153] exists,up d897eb1d-0f68-4ff4-832b-42deabfc6556 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 up in weight 1 up_from 45 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6818/3314328155,v1:172.21.15.77:6819/3314328155] [v2:172.21.15.77:6820/3314328155,v1:172.21.15.77:6821/3314328155] exists,up 916910cf-a42b-453d-be6b-c237a8305b43 2024-09-15T22:12:04.935 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 up in weight 1 up_from 50 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/159988175,v1:172.21.15.117:6817/159988175] [v2:172.21.15.117:6818/159988175,v1:172.21.15.117:6819/159988175] exists,up 5b4aa2ec-81c6-4365-82ec-6b470840358a 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 up in weight 1 up_from 57 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3896356979,v1:172.21.15.196:6817/3896356979] [v2:172.21.15.196:6818/3896356979,v1:172.21.15.196:6819/3896356979] exists,up 3dc4187f-569d-4da4-ba2e-3c708dd5b99c 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 up in weight 1 up_from 64 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6826/3515252544,v1:172.21.15.77:6827/3515252544] [v2:172.21.15.77:6828/3515252544,v1:172.21.15.77:6829/3515252544] exists,up 5097d6bd-7d2a-45fc-958c-da3902b80859 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/2316398411,v1:172.21.15.117:6825/2316398411] [v2:172.21.15.117:6826/2316398411,v1:172.21.15.117:6827/2316398411] exists,up 67e4b9ba-333a-40a1-a03a-9a31da950d8c 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 up in weight 1 up_from 76 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/2953911305,v1:172.21.15.196:6825/2953911305] [v2:172.21.15.196:6826/2953911305,v1:172.21.15.196:6827/2953911305] exists,up 35d9686a-188b-4bee-a88a-9c31fe9dc7b0 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.a [2,11] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.10 [8,11] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.3 [4,9] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.c [3,9] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.1d [4,8] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.21 [5,9] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.3 [5,8] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.f [9,8] 2024-09-15T22:12:04.936 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.10 [9,8] 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 4.13 [9,10] 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1261342424 expires 2024-09-16T22:03:04.101979+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/1757809354 expires 2024-09-16T22:03:04.101979+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1397043946 expires 2024-09-16T22:03:04.101979+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3468864668 expires 2024-09-16T22:03:04.101979+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/1757809354 expires 2024-09-16T22:03:04.101979+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3493953936 expires 2024-09-16T22:02:40.710885+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/573117065 expires 2024-09-16T22:02:40.710885+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/2559994639 expires 2024-09-16T22:02:40.710885+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/3382222599 expires 2024-09-16T22:02:40.710885+0000 2024-09-15T22:12:04.937 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/3382222599 expires 2024-09-16T22:02:40.710885+0000 2024-09-15T22:12:04.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 msgr2=0x7ff0fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:04.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 0x7ff0fc0a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff10c005fd0 tx=0x7ff10c005850 comp rx=0 tx=0).stop 2024-09-15T22:12:04.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 msgr2=0x7ff110100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:04.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff10800eaa0 tx=0x7ff10800ef60 comp rx=0 tx=0).stop 2024-09-15T22:12:04.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 shutdown_connections 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff0fc0a32d0 0x7ff0fc0a5790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1101078c0 0x7ff110100b50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff110104f90 0x7ff110100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 --2- 172.21.15.77:0/1481410561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff110102660 0x7ff110101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 >> 172.21.15.77:0/1481410561 conn(0x7ff1100fc280 msgr2=0x7ff110103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 shutdown_connections 2024-09-15T22:12:04.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:04.933+0000 7ff118702640 1 -- 172.21.15.77:0/1481410561 wait complete. 2024-09-15T22:12:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:05 smithi077 ceph-mon[30980]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:12:05.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:05 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1481410561' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:12:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:05 smithi196 ceph-mon[35109]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:12:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:05 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1481410561' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:12:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:05 smithi117 ceph-mon[35021]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-15T22:12:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:05 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1481410561' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:12:05.657 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-15T22:12:05.667 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:05.913 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:06.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:06 smithi077 ceph-mon[30980]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 -- 172.21.15.77:0/1441305726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 msgr2=0x7ff58c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 --2- 172.21.15.77:0/1441305726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c1029b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff580009900 tx=0x7ff58002a480 comp rx=0 tx=0).stop 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 -- 172.21.15.77:0/1441305726 shutdown_connections 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 --2- 172.21.15.77:0/1441305726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 --2- 172.21.15.77:0/1441305726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c1029b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 --2- 172.21.15.77:0/1441305726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 -- 172.21.15.77:0/1441305726 >> 172.21.15.77:0/1441305726 conn(0x7ff58c0fc7d0 msgr2=0x7ff58c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 -- 172.21.15.77:0/1441305726 shutdown_connections 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.392+0000 7ff5923cb640 1 -- 172.21.15.77:0/1441305726 wait complete. 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.393+0000 7ff5923cb640 1 Processor -- start 2024-09-15T22:12:06.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.393+0000 7ff5923cb640 1 -- start start 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.393+0000 7ff5923cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.393+0000 7ff5923cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.393+0000 7ff5923cb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5923cb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c110f80 con 0x7ff58c102570 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5923cb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c110e00 con 0x7ff58c102ef0 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5923cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111100 con 0x7ff58c100e20 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58904/0 (socket says 172.21.15.77:58904) 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 -- 172.21.15.77:0/2071388934 learned_addr learned my addr 172.21.15.77:0/2071388934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58bfff640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff590968640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 -- 172.21.15.77:0/2071388934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 msgr2=0x7ff58c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:06.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 -- 172.21.15.77:0/2071388934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 msgr2=0x7ff58c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 -- 172.21.15.77:0/2071388934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c19c390 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58bfff640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff590968640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff58b7fe640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff580006010 tx=0x7ff580003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58003f070 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff580031e30 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c19c620 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580035030 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.394+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a3f60 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.395+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c101200 con 0x7ff58c102570 2024-09-15T22:12:06.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.396+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff580042070 con 0x7ff58c102570 2024-09-15T22:12:06.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.397+0000 7ff5897fa640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 0x7ff5740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:06.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.397+0000 7ff58bfff640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 0x7ff5740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:06.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.397+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff5800f0ba0 con 0x7ff58c102570 2024-09-15T22:12:06.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.398+0000 7ff58bfff640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 0x7ff5740a5780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0058c0 tx=0x7ff57c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:06.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.400+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5800bd5f0 con 0x7ff58c102570 2024-09-15T22:12:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:06 smithi196 ceph-mon[35109]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:12:06.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.602+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff58c109090 con 0x7ff58c102570 2024-09-15T22:12:06.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.603+0000 7ff5897fa640 1 -- 172.21.15.77:0/2071388934 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7ff5800bcef0 con 0x7ff58c102570 2024-09-15T22:12:06.606 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:06.606 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:12:06.606 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:12:06.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 msgr2=0x7ff5740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 0x7ff5740a5780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0058c0 tx=0x7ff57c005d30 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 msgr2=0x7ff58c197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff580006010 tx=0x7ff580003c80 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 shutdown_connections 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff5740a32c0 0x7ff5740a5780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff58c102ef0 0x7ff58c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff58c102570 0x7ff58c197920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 --2- 172.21.15.77:0/2071388934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff58c100e20 0x7ff58c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:06.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 >> 172.21.15.77:0/2071388934 conn(0x7ff58c0fc7d0 msgr2=0x7ff58c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:06.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 shutdown_connections 2024-09-15T22:12:06.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:06.605+0000 7ff5923cb640 1 -- 172.21.15.77:0/2071388934 wait complete. 2024-09-15T22:12:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:06 smithi117 ceph-mon[35021]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:12:07.327 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:07.572 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:07.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:07 smithi077 ceph-mon[30980]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-15T22:12:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:07 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2071388934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:07 smithi196 ceph-mon[35109]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-15T22:12:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:07 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2071388934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:07 smithi117 ceph-mon[35021]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-15T22:12:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:07 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2071388934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- 172.21.15.77:0/2450097707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4102790 msgr2=0x7f8ef41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 --2- 172.21.15.77:0/2450097707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4102790 0x7f8ef41067e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009900 tx=0x7f8ee402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- 172.21.15.77:0/2450097707 shutdown_connections 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 --2- 172.21.15.77:0/2450097707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4106f10 0x7f8ef410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 --2- 172.21.15.77:0/2450097707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4102790 0x7f8ef41067e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 --2- 172.21.15.77:0/2450097707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 0x7f8ef41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- 172.21.15.77:0/2450097707 >> 172.21.15.77:0/2450097707 conn(0x7f8ef40fd8c0 msgr2=0x7f8ef40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- 172.21.15.77:0/2450097707 shutdown_connections 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- 172.21.15.77:0/2450097707 wait complete. 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 Processor -- start 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.062+0000 7f8efa019640 1 -- start start 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 0x7f8ef41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 0x7f8ef4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4112010 con 0x7f8ef4106f10 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4111e90 con 0x7f8ef4102790 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4112190 con 0x7f8ef4101de0 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58914/0 (socket says 172.21.15.77:58914) 2024-09-15T22:12:08.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 -- 172.21.15.77:0/2233879398 learned_addr learned my addr 172.21.15.77:0/2233879398 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef2ffd640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 0x7f8ef4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef37fe640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 0x7f8ef41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 -- 172.21.15.77:0/2233879398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 msgr2=0x7f8ef41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 0x7f8ef41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 -- 172.21.15.77:0/2233879398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 msgr2=0x7f8ef4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 0x7f8ef4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 -- 172.21.15.77:0/2233879398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef419d560 con 0x7f8ef4106f10 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef2ffd640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 0x7f8ef4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef3fff640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00b970 tx=0x7f8eec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec019070 con 0x7f8ef4106f10 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eec00cb40 con 0x7f8ef4106f10 2024-09-15T22:12:08.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef419d850 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.063+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec007650 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.064+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41a5130 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.064+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef4102240 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.065+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8eec01c070 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.066+0000 7f8ef0ff9640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 0x7f8ee00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.066+0000 7f8ef37fe640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 0x7f8ee00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.066+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8eec0cf330 con 0x7f8ef4106f10 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.067+0000 7f8ef37fe640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 0x7f8ee00a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009890 tx=0x7f8edc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:08.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.070+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8eec09bcf0 con 0x7f8ef4106f10 2024-09-15T22:12:08.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.270+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ef4109b60 con 0x7f8ef4106f10 2024-09-15T22:12:08.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.271+0000 7f8ef0ff9640 1 -- 172.21.15.77:0/2233879398 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f8eec09b5f0 con 0x7f8ef4106f10 2024-09-15T22:12:08.274 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:08.274 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"btime":"2024-09-15T22:10:40:741597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:10:40.090210+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:12:08.274 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 msgr2=0x7f8ee00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 0x7f8ee00a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009890 tx=0x7f8edc008040 comp rx=0 tx=0).stop 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 msgr2=0x7f8ef419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00b970 tx=0x7f8eec00be30 comp rx=0 tx=0).stop 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 shutdown_connections 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8ee00a3240 0x7f8ee00a5700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef4106f10 0x7f8ef419ce80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4102790 0x7f8ef4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 --2- 172.21.15.77:0/2233879398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef4101de0 0x7f8ef41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:08.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.273+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 >> 172.21.15.77:0/2233879398 conn(0x7f8ef40fd8c0 msgr2=0x7f8ef40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:08.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.274+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 shutdown_connections 2024-09-15T22:12:08.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:08.274+0000 7f8efa019640 1 -- 172.21.15.77:0/2233879398 wait complete. 2024-09-15T22:12:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:08 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2233879398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:08 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2233879398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:08 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2233879398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:08.969 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:12:09.212 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:09 smithi077 ceph-mon[30980]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-15T22:12:09.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:09 smithi196 ceph-mon[35109]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-15T22:12:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:09 smithi117 ceph-mon[35021]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.663+0000 7f331157a640 1 -- 172.21.15.77:0/154951802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c10c2f0 msgr2=0x7f330c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.663+0000 7f331157a640 1 --2- 172.21.15.77:0/154951802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c10c2f0 0x7f330c10e790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3304009900 tx=0x7f330402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 -- 172.21.15.77:0/154951802 shutdown_connections 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 --2- 172.21.15.77:0/154951802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c10c2f0 0x7f330c10e790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 --2- 172.21.15.77:0/154951802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c1037a0 0x7f330c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 --2- 172.21.15.77:0/154951802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c102df0 0x7f330c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 -- 172.21.15.77:0/154951802 >> 172.21.15.77:0/154951802 conn(0x7f330c0fc7d0 msgr2=0x7f330c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 -- 172.21.15.77:0/154951802 shutdown_connections 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 -- 172.21.15.77:0/154951802 wait complete. 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 Processor -- start 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.664+0000 7f331157a640 1 -- start start 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 0x7f330c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 0x7f330c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f330c112110 con 0x7f330c102df0 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f330c111f90 con 0x7f330c10c2f0 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f330c112290 con 0x7f330c1037a0 2024-09-15T22:12:09.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52082/0 (socket says 172.21.15.77:52082) 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 -- 172.21.15.77:0/3166385393 learned_addr learned my addr 172.21.15.77:0/3166385393 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330a7fc640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 0x7f330c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330b7fe640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 0x7f330c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 -- 172.21.15.77:0/3166385393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 msgr2=0x7f330c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 0x7f330c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 -- 172.21.15.77:0/3166385393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 msgr2=0x7f330c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 0x7f330c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 -- 172.21.15.77:0/3166385393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c19d560 con 0x7f330c102df0 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330b7fe640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 0x7f330c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330a7fc640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 0x7f330c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f330affd640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00ea80 tx=0x7f32fc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:09.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00ce40 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32fc005020 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c19d850 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc011760 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.665+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c1a5130 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.666+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c1031d0 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.667+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f32fc0051d0 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.668+0000 7f3303fff640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 0x7f32f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.668+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f32fc0d0310 con 0x7f330c102df0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.668+0000 7f330a7fc640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 0x7f32f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.668+0000 7f330a7fc640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 0x7f32f80a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005fd0 tx=0x7f32f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:09.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.672+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32fc016030 con 0x7f330c102df0 2024-09-15T22:12:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.842+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f330c1072c0 con 0x7f330c102df0 2024-09-15T22:12:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.844+0000 7f3303fff640 1 -- 172.21.15.77:0/3166385393 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24775 (secure 0 0 0) 0x7f32fc09cd70 con 0x7f330c102df0 2024-09-15T22:12:09.847 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:09.847 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":90,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:11:05.224880+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:12:09.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 msgr2=0x7f32f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:09.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 0x7f32f80a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005fd0 tx=0x7f32f4005850 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 msgr2=0x7f330c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00ea80 tx=0x7f32fc00ef40 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 shutdown_connections 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f32f80a3240 0x7f32f80a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f330c10c2f0 0x7f330c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f330c1037a0 0x7f330c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 --2- 172.21.15.77:0/3166385393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f330c102df0 0x7f330c1985b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 >> 172.21.15.77:0/3166385393 conn(0x7f330c0fc7d0 msgr2=0x7f330c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 shutdown_connections 2024-09-15T22:12:09.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:09.846+0000 7f331157a640 1 -- 172.21.15.77:0/3166385393 wait complete. 2024-09-15T22:12:10.506 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:09:55.819962+0000', 'modified': '2024-09-15T22:10:40.090210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24458, 'mds_1': 24440, 'mds_2': 24439, 'mds_3': 24424, 'mds_4': 14604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/2629133747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2629133747}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2629133747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.77:6841/3032989520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 3032989520}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 3032989520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/3132990127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3132990127}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3132990127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6833/2823149773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 2823149773}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 2823149773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/889715770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 889715770}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 889715770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/3112549785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3112549785}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3112549785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/3513024753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3513024753}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3513024753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6839/1624290607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1624290607}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1624290607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.117:6837/3263285576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3263285576}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3263285576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/1252344498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1252344498}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1252344498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24458, 'qdb_cluster': [14604, 24458, 24424, 24439, 24440]} 2024-09-15T22:12:10.506 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T22:12:10.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-15T22:12:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:10 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3166385393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:10 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3166385393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:10 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3166385393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:10.755 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- 172.21.15.77:0/567587405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102570 msgr2=0x7f59f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 --2- 172.21.15.77:0/567587405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102570 0x7f59f81029b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009900 tx=0x7f59e802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- 172.21.15.77:0/567587405 shutdown_connections 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 --2- 172.21.15.77:0/567587405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8102ef0 0x7f59f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 --2- 172.21.15.77:0/567587405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102570 0x7f59f81029b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 --2- 172.21.15.77:0/567587405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8100e20 0x7f59f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- 172.21.15.77:0/567587405 >> 172.21.15.77:0/567587405 conn(0x7f59f80fc7d0 msgr2=0x7f59f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- 172.21.15.77:0/567587405 shutdown_connections 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- 172.21.15.77:0/567587405 wait complete. 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 Processor -- start 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.252+0000 7f5a007ba640 1 -- start start 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8100e20 0x7f59f81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8102570 0x7f59f8197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8110f80 con 0x7f59f8102ef0 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8110e00 con 0x7f59f8102570 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8111100 con 0x7f59f8100e20 2024-09-15T22:12:11.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52102/0 (socket says 172.21.15.77:52102) 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 -- 172.21.15.77:0/490528031 learned_addr learned my addr 172.21.15.77:0/490528031 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fe556640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8100e20 0x7f59f81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fdd55640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8102570 0x7f59f8197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 -- 172.21.15.77:0/490528031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8100e20 msgr2=0x7f59f81973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8100e20 0x7f59f81973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 -- 172.21.15.77:0/490528031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8102570 msgr2=0x7f59f8197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8102570 0x7f59f8197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 -- 172.21.15.77:0/490528031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f819c370 con 0x7f59f8102ef0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59fed57640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f59f000cce0 tx=0x7f59f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59ef7fe640 1 -- 172.21.15.77:0/490528031 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f000f450 con 0x7f59f8102ef0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59ef7fe640 1 -- 172.21.15.77:0/490528031 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f000fad0 con 0x7f59f8102ef0 2024-09-15T22:12:11.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f819c660 con 0x7f59f8102ef0 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f59ef7fe640 1 -- 172.21.15.77:0/490528031 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0018c10 con 0x7f59f8102ef0 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.253+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f81a3f40 con 0x7f59f8102ef0 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59ef7fe640 1 -- 172.21.15.77:0/490528031 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f59f000f5f0 con 0x7f59f8102ef0 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 Processor -- start 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 -- start start 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a019cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a026dc0 0x55786a0291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557869f80090 con 0x55786a01a210 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557869f27a70 con 0x55786a026dc0 2024-09-15T22:12:11.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f5a02b93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557869f7c050 con 0x55786a0198f0 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59fe556640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59ef7fe640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 0x7f59e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59fe556640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52106/0 (socket says 172.21.15.77:52106) 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a019cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.256+0000 7f59fe556640 1 -- 172.21.15.77:0/1367967380 learned_addr learned my addr 172.21.15.77:0/1367967380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a026dc0 0x55786a0291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fed57640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 0x7f59e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 -- 172.21.15.77:0/1367967380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 msgr2=0x55786a019cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a019cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 -- 172.21.15.77:0/1367967380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a026dc0 msgr2=0x55786a0291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a026dc0 0x55786a0291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 -- 172.21.15.77:0/1367967380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557869f34130 con 0x55786a01a210 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a019cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59ef7fe640 1 -- 172.21.15.77:0/490528031 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f59f00d02a0 con 0x7f59f8102ef0 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fe556640 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f59f400bcb0 tx=0x7f59f400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca984e3f80147d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59fed57640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 0x7f59e40a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f59f00a4ca0 tx=0x7f59f0066c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59ecff9640 1 -- 172.21.15.77:0/1367967380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4015070 con 0x55786a01a210 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59ecff9640 1 -- 172.21.15.77:0/1367967380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f4011920 con 0x55786a01a210 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.257+0000 7f59ecff9640 1 -- 172.21.15.77:0/1367967380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f40108d0 con 0x55786a01a210 2024-09-15T22:12:11.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- 172.21.15.77:0/1367967380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 msgr2=0x55786a026880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f59f400bcb0 tx=0x7f59f400f730 comp rx=0 tx=0).stop 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- 172.21.15.77:0/1367967380 shutdown_connections 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a026dc0 0x55786a0291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a01a210 0x55786a026880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1367967380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a019cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- 172.21.15.77:0/1367967380 >> 172.21.15.77:0/1367967380 conn(0x55786a015f60 msgr2=0x55786a016340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- 172.21.15.77:0/1367967380 shutdown_connections 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- 172.21.15.77:0/1367967380 wait complete. 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.258+0000 7f5a02b93740 1 -- start start 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 Processor -- start 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a0266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a023a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55786a029930 con 0x55786a026dc0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557869f3af20 con 0x55786a01a210 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55786a029ab0 con 0x55786a0198f0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 10 client.0 _ll_register_callbacks cb 0x557869f04030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52108/0 (socket says 172.21.15.77:52108) 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 -- 172.21.15.77:0/3496013623 learned_addr learned my addr 172.21.15.77:0/3496013623 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fe556640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a023a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a0266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 -- 172.21.15.77:0/3496013623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 msgr2=0x55786a0266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a0266b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 -- 172.21.15.77:0/3496013623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 msgr2=0x55786a023a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a023a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 -- 172.21.15.77:0/3496013623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557869cf12b0 con 0x55786a026dc0 2024-09-15T22:12:11.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59fdd55640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005ab0 tx=0x7f59e8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e961ced4832983d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:11.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8035040 con 0x55786a026dc0 2024-09-15T22:12:11.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f59e8032a00 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.259+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f4011920 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59e8053050 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 10 client.15207 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55786a035320 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f4011920 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f59e803a040 con 0x55786a026dc0 2024-09-15T22:12:11.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 10 client.15207 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:11.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 10 client.15207 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:12:11.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 20 client.15207 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:11.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a037910 0x55786a038d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 4 client.15207 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:11.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557869f8d600 con 0x55786a037910 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59fe556640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a037910 0x55786a038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59fe556640 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a037910 0x55786a038d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b99aa509492a5cd2 server_cookie=bc8a0b10239372db in_seq=0 out_seq=0 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.260+0000 7f59e1ffb640 10 client.15207 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f59e1ffb640 1 -- 172.21.15.77:0/3496013623 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557869f8d600 con 0x55786a037910 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f59e1ffb640 10 client.15207 handle_command_reply: tid=0 2024-09-15T22:12:11.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 client.15207 shutdown 2024-09-15T22:12:11.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 msgr2=0x55786a023f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f59e8005ab0 tx=0x7f59e8005e50 comp rx=0 tx=0).stop 2024-09-15T22:12:11.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 shutdown_connections 2024-09-15T22:12:11.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a037910 0x55786a038d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a023f90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a023a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 --2- 172.21.15.77:0/3496013623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a0266b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 >> 172.21.15.77:0/3496013623 conn(0x55786a015f60 msgr2=0x55786a01bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f59e1ffb640 0 client.15207 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:11.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 shutdown_connections 2024-09-15T22:12:11.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.261+0000 7f5a02b93740 1 -- 172.21.15.77:0/3496013623 wait complete. 2024-09-15T22:12:11.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.262+0000 7f5a02b93740 20 client.15207 trim_cache size 0 max 16384 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 Processor -- start 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 -- start start 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a037240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a0386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55786a0370c0 con 0x55786a026dc0 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55786a037790 con 0x55786a01a210 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f5a02b93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55786a037e60 con 0x55786a0198f0 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52114/0 (socket says 172.21.15.77:52114) 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.279+0000 7f59fdd55640 1 -- 172.21.15.77:0/2913583555 learned_addr learned my addr 172.21.15.77:0/2913583555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a037240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fe556640 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a0386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 -- 172.21.15.77:0/2913583555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 msgr2=0x55786a037240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a037240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 -- 172.21.15.77:0/2913583555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 msgr2=0x55786a0386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a0386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 -- 172.21.15.77:0/2913583555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786a021fa0 con 0x55786a026dc0 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59fdd55640 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59e804d530 tx=0x7f59e802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7fe19c236e20da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59e3fff640 1 -- 172.21.15.77:0/2913583555 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8035030 con 0x55786a026dc0 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59e3fff640 1 -- 172.21.15.77:0/2913583555 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e804d730 con 0x55786a026dc0 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.280+0000 7f59e3fff640 1 -- 172.21.15.77:0/2913583555 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e804dd20 con 0x55786a026dc0 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- 172.21.15.77:0/2913583555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 msgr2=0x55786a010ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f59e804d530 tx=0x7f59e802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:11.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- 172.21.15.77:0/2913583555 shutdown_connections 2024-09-15T22:12:11.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a010ce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a01a210 0x55786a0386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- 172.21.15.77:0/2913583555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a0198f0 0x55786a037240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- 172.21.15.77:0/2913583555 >> 172.21.15.77:0/2913583555 conn(0x55786a015f60 msgr2=0x55786a016340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- 172.21.15.77:0/2913583555 shutdown_connections 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- 172.21.15.77:0/2913583555 wait complete. 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- start start 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 Processor -- start 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a0198f0 0x55786a041960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a01a210 0x55786a039f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55786a038c00 con 0x55786a026dc0 2024-09-15T22:12:11.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55786a038530 con 0x55786a0198f0 2024-09-15T22:12:11.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55786a0392d0 con 0x55786a01a210 2024-09-15T22:12:11.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.281+0000 7f5a02b93740 10 client.0 _ll_register_callbacks cb 0x7f59e8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52128/0 (socket says 172.21.15.77:52128) 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 -- 172.21.15.77:0/1039050262 learned_addr learned my addr 172.21.15.77:0/1039050262 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a0198f0 0x55786a041960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fe556640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a01a210 0x55786a039f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 -- 172.21.15.77:0/1039050262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a01a210 msgr2=0x55786a039f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a01a210 0x55786a039f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 -- 172.21.15.77:0/1039050262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a0198f0 msgr2=0x55786a041960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a0198f0 0x55786a041960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 -- 172.21.15.77:0/1039050262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786a03ad50 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59fdd55640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55786a032780 tx=0x7f59e804d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c845825775cdd1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e805f070 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f59e804dcb0 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e804d730 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59e8035070 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f5a02b93740 10 client.15213 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.282+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55786a0a10e0 con 0x55786a026dc0 2024-09-15T22:12:11.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e804d730 con 0x55786a026dc0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f59e8062020 con 0x55786a026dc0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 10 client.15213 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 20 client.15213 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a0a57d0 0x55786a0a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 4 client.15213 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "status"}) -- 0x557869f87b10 con 0x55786a0a57d0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f59fe556640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a0a57d0 0x55786a0a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f59fe556640 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a0a57d0 0x55786a0a44e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b253361f20c24611 server_cookie=fa92be9e85fa087a in_seq=0 out_seq=0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.283+0000 7f59e1ffb640 10 client.15213 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f59e1ffb640 1 -- 172.21.15.77:0/1039050262 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557869f87b10 con 0x55786a0a57d0 2024-09-15T22:12:11.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f59e1ffb640 10 client.15213 handle_command_reply: tid=0 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 client.15213 shutdown 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 msgr2=0x55786a03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55786a032780 tx=0x7f59e804d3d0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 shutdown_connections 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55786a0a57d0 0x55786a0a44e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55786a026dc0 0x55786a03a520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f59e1ffb640 0 client.15213 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55786a01a210 0x55786a039f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 --2- 172.21.15.77:0/1039050262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786a0198f0 0x55786a041960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 >> 172.21.15.77:0/1039050262 conn(0x55786a015f60 msgr2=0x55786a010ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 shutdown_connections 2024-09-15T22:12:11.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 1 -- 172.21.15.77:0/1039050262 wait complete. 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a02b93740 20 client.15213 trim_cache size 0 max 16384 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14604, 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:11.301 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 92.603467577000004, 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 148.60696089800001 2024-09-15T22:12:11.302 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 msgr2=0x7f59e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 0x7f59e40a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f59f00a4ca0 tx=0x7f59f0066c90 comp rx=0 tx=0).stop 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 msgr2=0x7f59f819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.284+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f59f000cce0 tx=0x7f59f000a620 comp rx=0 tx=0).stop 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 shutdown_connections 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f59e40a32c0 0x7f59e40a5780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f8102ef0 0x7f59f819bc90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59f8102570 0x7f59f8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 --2- 172.21.15.77:0/490528031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59f8100e20 0x7f59f81973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:11.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 >> 172.21.15.77:0/490528031 conn(0x7f59f80fc7d0 msgr2=0x7f59f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:11.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 shutdown_connections 2024-09-15T22:12:11.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:11.285+0000 7f5a007ba640 1 -- 172.21.15.77:0/490528031 wait complete. 2024-09-15T22:12:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:11 smithi077 ceph-mon[30980]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:12:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:11 smithi196 ceph-mon[35109]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:12:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:11 smithi117 ceph-mon[35021]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:12:12.056 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-15T22:12:12.300 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:12.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 -- 172.21.15.77:0/2044965730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784101de0 msgr2=0x7f37841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 --2- 172.21.15.77:0/2044965730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784101de0 0x7f37841021c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3780009900 tx=0x7f378002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 -- 172.21.15.77:0/2044965730 shutdown_connections 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 --2- 172.21.15.77:0/2044965730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784106f10 0x7f378410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 --2- 172.21.15.77:0/2044965730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784102790 0x7f37841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 --2- 172.21.15.77:0/2044965730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784101de0 0x7f37841021c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 -- 172.21.15.77:0/2044965730 >> 172.21.15.77:0/2044965730 conn(0x7f37840fd8c0 msgr2=0x7f37840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 -- 172.21.15.77:0/2044965730 shutdown_connections 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.785+0000 7f378b29b640 1 -- 172.21.15.77:0/2044965730 wait complete. 2024-09-15T22:12:12.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 Processor -- start 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 -- start start 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101de0 0x7f3784198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784102790 0x7f3784198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3784111f70 con 0x7f3784106f10 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3784111df0 con 0x7f3784101de0 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f378b29b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37841120f0 con 0x7f3784102790 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.786+0000 7f3789838640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52156/0 (socket says 172.21.15.77:52156) 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789037640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101de0 0x7f3784198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 -- 172.21.15.77:0/3829480209 learned_addr learned my addr 172.21.15.77:0/3829480209 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:12.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3788836640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784102790 0x7f3784198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 -- 172.21.15.77:0/3829480209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784102790 msgr2=0x7f3784198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784102790 0x7f3784198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 -- 172.21.15.77:0/3829480209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101de0 msgr2=0x7f3784198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101de0 0x7f3784198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 -- 172.21.15.77:0/3829480209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378419d3d0 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f3789838640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f377c00cad0 tx=0x7f377c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f377a7fc640 1 -- 172.21.15.77:0/3829480209 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c00ae20 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378419d6c0 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f377a7fc640 1 -- 172.21.15.77:0/3829480209 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c005020 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f377a7fc640 1 -- 172.21.15.77:0/3829480209 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c00f6f0 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.787+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37841a4fa0 con 0x7f3784106f10 2024-09-15T22:12:12.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.789+0000 7f377a7fc640 1 -- 172.21.15.77:0/3829480209 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f377c0051d0 con 0x7f3784106f10 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 Processor -- start 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 -- start start 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd83fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55804ccdd090 con 0x55804cd769d0 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55804cc84a70 con 0x55804cd772f0 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f378d674740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55804ccd9050 con 0x55804cd81bb0 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52162/0 (socket says 172.21.15.77:52162) 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f3788836640 1 -- 172.21.15.77:0/4147164800 learned_addr learned my addr 172.21.15.77:0/4147164800 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:12.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f377a7fc640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 0x7f37700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.790+0000 7f3789037640 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd83fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3789838640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 0x7f37700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 -- 172.21.15.77:0/4147164800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 msgr2=0x55804cd83fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd83fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 -- 172.21.15.77:0/4147164800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 msgr2=0x55804cd81670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd81670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 -- 172.21.15.77:0/4147164800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804cc91130 con 0x55804cd769d0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f377a7fc640 1 -- 172.21.15.77:0/3829480209 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f377c0cfea0 con 0x7f3784106f10 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3788836640 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37740134d0 tx=0x7f377400a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f4c6eba48e067c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f3789838640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 0x7f37700a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f377c06f060 tx=0x7f377c06f090 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f376f7fe640 1 -- 172.21.15.77:0/4147164800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774019030 con 0x55804cd769d0 2024-09-15T22:12:12.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f376f7fe640 1 -- 172.21.15.77:0/4147164800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3774005500 con 0x55804cd769d0 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.791+0000 7f376f7fe640 1 -- 172.21.15.77:0/4147164800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377400f830 con 0x55804cd769d0 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- 172.21.15.77:0/4147164800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 msgr2=0x55804cd76db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37740134d0 tx=0x7f377400a760 comp rx=0 tx=0).stop 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- 172.21.15.77:0/4147164800 shutdown_connections 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd83fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd81670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 --2- 172.21.15.77:0/4147164800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd76db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- 172.21.15.77:0/4147164800 >> 172.21.15.77:0/4147164800 conn(0x55804cd72f60 msgr2=0x55804cd73340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- 172.21.15.77:0/4147164800 shutdown_connections 2024-09-15T22:12:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- 172.21.15.77:0/4147164800 wait complete. 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 -- start start 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.792+0000 7f378d674740 1 Processor -- start 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd769d0 0x55804cd81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd772f0 0x55804cd7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55804cd84780 con 0x55804cd81bb0 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55804cc97f20 con 0x55804cd769d0 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55804cd84900 con 0x55804cd772f0 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 10 client.0 _ll_register_callbacks cb 0x55804cc61030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52172/0 (socket says 172.21.15.77:52172) 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 -- 172.21.15.77:0/1215090706 learned_addr learned my addr 172.21.15.77:0/1215090706 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3789037640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd772f0 0x55804cd7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd769d0 0x55804cd81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 -- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd772f0 msgr2=0x55804cd7e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd772f0 0x55804cd7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 -- 172.21.15.77:0/1215090706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd769d0 msgr2=0x55804cd81410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd769d0 0x55804cd81410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 -- 172.21.15.77:0/1215090706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804ca4e2b0 con 0x55804cd81bb0 2024-09-15T22:12:12.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f3788836640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3774004db0 tx=0x7f3774004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f593fc99b9506a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f376cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774019040 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3774005180 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.793+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3774005500 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f377400f630 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 10 client.15225 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55804cd90070 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3774005500 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f377401e5d0 con 0x55804cd81bb0 2024-09-15T22:12:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 10 client.15225 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 10 client.15225 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 20 client.15225 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804cd92290 0x55804cd93700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 4 client.15225 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55804ccea600 con 0x55804cd92290 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.794+0000 7f3789037640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804cd92290 0x55804cd93700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f376cff9640 10 client.15225 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f3789037640 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804cd92290 0x55804cd93700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9bd027568e4f0ddf server_cookie=75555773166ce0d7 in_seq=0 out_seq=0 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f376cff9640 1 -- 172.21.15.77:0/1215090706 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55804ccea600 con 0x55804cd92290 2024-09-15T22:12:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f376cff9640 10 client.15225 handle_command_reply: tid=0 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 client.15225 shutdown 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 msgr2=0x55804cd7eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3774004db0 tx=0x7f3774004cb0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 shutdown_connections 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804cd92290 0x55804cd93700 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd81bb0 0x55804cd7eca0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.795+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd772f0 0x55804cd7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f376cff9640 0 client.15225 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f378d674740 1 --2- 172.21.15.77:0/1215090706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd769d0 0x55804cd81410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 >> 172.21.15.77:0/1215090706 conn(0x55804cd72f60 msgr2=0x55804cd78c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 shutdown_connections 2024-09-15T22:12:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f378d674740 1 -- 172.21.15.77:0/1215090706 wait complete. 2024-09-15T22:12:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.796+0000 7f378d674740 20 client.15225 trim_cache size 0 max 16384 2024-09-15T22:12:12.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.813+0000 7f378d674740 1 Processor -- start 2024-09-15T22:12:12.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.813+0000 7f378d674740 1 -- start start 2024-09-15T22:12:12.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55804cd91a40 con 0x55804cd769d0 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55804cd92110 con 0x55804cd772f0 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f378d674740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55804cd927e0 con 0x55804cd81bb0 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52180/0 (socket says 172.21.15.77:52180) 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 -- 172.21.15.77:0/1235421227 learned_addr learned my addr 172.21.15.77:0/1235421227 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 -- 172.21.15.77:0/1235421227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 msgr2=0x55804cd7d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 -- 172.21.15.77:0/1235421227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 msgr2=0x55804cd7d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 -- 172.21.15.77:0/1235421227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804cd81780 con 0x55804cd769d0 2024-09-15T22:12:12.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f3788836640 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f377400adc0 tx=0x7f377400a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d500771e4f1a754b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f376effd640 1 -- 172.21.15.77:0/1235421227 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774019040 con 0x55804cd769d0 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f376effd640 1 -- 172.21.15.77:0/1235421227 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377400a590 con 0x55804cd769d0 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.814+0000 7f376effd640 1 -- 172.21.15.77:0/1235421227 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774005060 con 0x55804cd769d0 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- 172.21.15.77:0/1235421227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 msgr2=0x55804cd93700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f377400adc0 tx=0x7f377400a440 comp rx=0 tx=0).stop 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- 172.21.15.77:0/1235421227 shutdown_connections 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd81bb0 0x55804cd7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd772f0 0x55804cd7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 --2- 172.21.15.77:0/1235421227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd769d0 0x55804cd93700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- 172.21.15.77:0/1235421227 >> 172.21.15.77:0/1235421227 conn(0x55804cd72f60 msgr2=0x55804cd73340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- 172.21.15.77:0/1235421227 shutdown_connections 2024-09-15T22:12:12.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- 172.21.15.77:0/1235421227 wait complete. 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 -- start start 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.815+0000 7f378d674740 1 Processor -- start 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 0x55804cd96f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd81bb0 0x55804cdfa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55804cd93580 con 0x55804cd772f0 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55804cd92eb0 con 0x55804cd81bb0 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55804cd93c50 con 0x55804cd769d0 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f378d674740 10 client.0 _ll_register_callbacks cb 0x7f37800314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52182/0 (socket says 172.21.15.77:52182) 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 -- 172.21.15.77:0/3737236489 learned_addr learned my addr 172.21.15.77:0/3737236489 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:12.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3788836640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 0x55804cd96f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 -- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 msgr2=0x55804cd96f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 0x55804cd96f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 -- 172.21.15.77:0/3737236489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd81bb0 msgr2=0x55804cdfa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd81bb0 0x55804cdfa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 -- 172.21.15.77:0/3737236489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804cdfaf00 con 0x55804cd772f0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3788836640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 0x55804cd96f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f3789037640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3780033000 tx=0x7f3780031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25b42f8a1aa3fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.816+0000 7f376cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378004d070 con 0x55804cd772f0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3780004450 con 0x55804cd772f0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377400a590 con 0x55804cd772f0 2024-09-15T22:12:12.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3780059050 con 0x55804cd772f0 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 10 client.15231 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55804cdfc290 con 0x55804cd772f0 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377400a590 con 0x55804cd772f0 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3780048830 con 0x55804cd772f0 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 10 client.15231 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 20 client.15231 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804ce00e90 0x55804cdffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 4 client.15231 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "status"}) -- 0x55804cce4b10 con 0x55804ce00e90 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.817+0000 7f3788836640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804ce00e90 0x55804cdffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:12.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f3788836640 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804ce00e90 0x55804cdffba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38c356bddddb0a72 server_cookie=79c0f4a36ef826e5 in_seq=0 out_seq=0 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f376cff9640 10 client.15231 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f376cff9640 1 -- 172.21.15.77:0/3737236489 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55804cce4b10 con 0x55804ce00e90 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f376cff9640 10 client.15231 handle_command_reply: tid=0 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 client.15231 shutdown 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 msgr2=0x55804cd97480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3780033000 tx=0x7f3780031a20 comp rx=0 tx=0).stop 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 shutdown_connections 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55804ce00e90 0x55804cdffba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804cd81bb0 0x55804cdfa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804cd772f0 0x55804cd97480 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 --2- 172.21.15.77:0/3737236489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55804cd769d0 0x55804cd96f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 >> 172.21.15.77:0/3737236489 conn(0x55804cd72f60 msgr2=0x55804cd7cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.818+0000 7f376cff9640 0 client.15231 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 shutdown_connections 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378d674740 1 -- 172.21.15.77:0/3737236489 wait complete. 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378d674740 20 client.15231 trim_cache size 0 max 16384 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24424, 2024-09-15T22:12:12.832 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 94.484551162000002, 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 161.60566784299999 2024-09-15T22:12:12.833 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 msgr2=0x7f37700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 0x7f37700a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f377c06f060 tx=0x7f377c06f090 comp rx=0 tx=0).stop 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 msgr2=0x7f378419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f377c00cad0 tx=0x7f377c00cf90 comp rx=0 tx=0).stop 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 shutdown_connections 2024-09-15T22:12:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f37700a3240 0x7f37700a5700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3784106f10 0x7f378419ccf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3784102790 0x7f3784198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 --2- 172.21.15.77:0/3829480209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784101de0 0x7f3784198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 >> 172.21.15.77:0/3829480209 conn(0x7f37840fd8c0 msgr2=0x7f37840fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 shutdown_connections 2024-09-15T22:12:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:12.819+0000 7f378b29b640 1 -- 172.21.15.77:0/3829480209 wait complete. 2024-09-15T22:12:13.505 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:13 smithi077 ceph-mon[30980]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:13.533 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:12:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:13 smithi196 ceph-mon[35109]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:13 smithi117 ceph-mon[35021]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:14.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 -- 172.21.15.77:0/2227251433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484101410 msgr2=0x7fb48410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2227251433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484101410 0x7fb48410d940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009930 tx=0x7fb47c02a420 comp rx=0 tx=0).stop 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 -- 172.21.15.77:0/2227251433 shutdown_connections 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2227251433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484101410 0x7fb48410d940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2227251433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484100a70 0x7fb484100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2227251433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484106a70 0x7fb484106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.230+0000 7fb48be6d640 1 -- 172.21.15.77:0/2227251433 >> 172.21.15.77:0/2227251433 conn(0x7fb4840fc790 msgr2=0x7fb4840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- 172.21.15.77:0/2227251433 shutdown_connections 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- 172.21.15.77:0/2227251433 wait complete. 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 Processor -- start 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- start start 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 0x7fb484199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 0x7fb4841a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb484110eb0 con 0x7fb484100a70 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb484110d30 con 0x7fb484101410 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.231+0000 7fb48be6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb484111030 con 0x7fb484106a70 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52212/0 (socket says 172.21.15.77:52212) 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 -- 172.21.15.77:0/2807264996 learned_addr learned my addr 172.21.15.77:0/2807264996 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489408640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 0x7fb484199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb48a40a640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 0x7fb4841a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 -- 172.21.15.77:0/2807264996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 msgr2=0x7fb4841a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 0x7fb4841a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 -- 172.21.15.77:0/2807264996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 msgr2=0x7fb484199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 0x7fb484199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 -- 172.21.15.77:0/2807264996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4841a48f0 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489408640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 0x7fb484199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb48a40a640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 0x7fb4841a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb489c09640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb47400ba50 tx=0x7fb47400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb482ffd640 1 -- 172.21.15.77:0/2807264996 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474019070 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb482ffd640 1 -- 172.21.15.77:0/2807264996 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47400cbe0 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4841a4ac0 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb482ffd640 1 -- 172.21.15.77:0/2807264996 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740076e0 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.232+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841a5000 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.234+0000 7fb482ffd640 1 -- 172.21.15.77:0/2807264996 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb47401c070 con 0x7fb484100a70 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 Processor -- start 2024-09-15T22:12:14.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 -- start start 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19ecd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c19e29090 con 0x563c19ec29d0 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c19dd0a70 con 0x563c19ec32f0 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48e246740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c19e25050 con 0x563c19ecdbb0 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb482ffd640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 0x7fb46c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48a40a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48a40a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52228/0 (socket says 172.21.15.77:52228) 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48a40a640 1 -- 172.21.15.77:0/364313874 learned_addr learned my addr 172.21.15.77:0/364313874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb48a40a640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 0x7fb46c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb489408640 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb489c09640 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19ecd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.235+0000 7fb482ffd640 1 -- 172.21.15.77:0/2807264996 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb4740cf270 con 0x7fb484100a70 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 -- 172.21.15.77:0/364313874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 msgr2=0x563c19ecffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 -- 172.21.15.77:0/364313874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 msgr2=0x563c19ecd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19ecd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 -- 172.21.15.77:0/364313874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c19ddd130 con 0x563c19ec29d0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0096c0 tx=0x7fb47c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a21589c9e08e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb48a40a640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 0x7fb46c0a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02a940 tx=0x7fb47c04d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb46bfff640 1 -- 172.21.15.77:0/364313874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c057070 con 0x563c19ec29d0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb46bfff640 1 -- 172.21.15.77:0/364313874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47c04e070 con 0x563c19ec29d0 2024-09-15T22:12:14.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.236+0000 7fb46bfff640 1 -- 172.21.15.77:0/364313874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c035040 con 0x563c19ec29d0 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- 172.21.15.77:0/364313874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 msgr2=0x563c19ec2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0096c0 tx=0x7fb47c031b60 comp rx=0 tx=0).stop 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- 172.21.15.77:0/364313874 shutdown_connections 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19ecd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 --2- 172.21.15.77:0/364313874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ec2db0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- 172.21.15.77:0/364313874 >> 172.21.15.77:0/364313874 conn(0x563c19ebef60 msgr2=0x563c19ebf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- 172.21.15.77:0/364313874 shutdown_connections 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- 172.21.15.77:0/364313874 wait complete. 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 -- start start 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.237+0000 7fb48e246740 1 Processor -- start 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19eca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ecdbb0 0x563c19ecacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c19ed0780 con 0x563c19ec29d0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c19de3f20 con 0x563c19ecdbb0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c19ed0900 con 0x563c19ec32f0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48e246740 10 client.0 _ll_register_callbacks cb 0x563c19dad030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52238/0 (socket says 172.21.15.77:52238) 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 -- 172.21.15.77:0/1358670167 learned_addr learned my addr 172.21.15.77:0/1358670167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489408640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19eca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 -- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 msgr2=0x563c19eca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19eca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb48a40a640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ecdbb0 0x563c19ecacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 -- 172.21.15.77:0/1358670167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ecdbb0 msgr2=0x563c19ecacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ecdbb0 0x563c19ecacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 -- 172.21.15.77:0/1358670167 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c19b9a2b0 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489408640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19eca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb489c09640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb4740cca60 tx=0x7fb474004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae6f04393b4ad51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.238+0000 7fb4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740bb040 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb4740a3d10 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb47c04e070 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb47409f8e0 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 10 client.15243 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c19edc0f0 con 0x563c19ec29d0 2024-09-15T22:12:14.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47c04e070 con 0x563c19ec29d0 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb47409a980 con 0x563c19ec29d0 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 20 client.15243 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19ede770 0x563c19edfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 4 client.15243 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c19e36600 con 0x563c19ede770 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.239+0000 7fb489408640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19ede770 0x563c19edfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.240+0000 7fb489408640 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19ede770 0x563c19edfbe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbd384d0481e87bf server_cookie=bf37713324f6e4b8 in_seq=0 out_seq=0 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.240+0000 7fb4697fa640 10 client.15243 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb4697fa640 1 -- 172.21.15.77:0/1358670167 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c19e36600 con 0x563c19ede770 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb4697fa640 10 client.15243 handle_command_reply: tid=0 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 client.15243 shutdown 2024-09-15T22:12:14.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 msgr2=0x563c19ecd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb4740cca60 tx=0x7fb474004b50 comp rx=0 tx=0).stop 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 shutdown_connections 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19ede770 0x563c19edfbe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ecdbb0 0x563c19ecacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19eca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb4697fa640 0 client.15243 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 --2- 172.21.15.77:0/1358670167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19ecd440 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 >> 172.21.15.77:0/1358670167 conn(0x563c19ebef60 msgr2=0x563c19ec4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 shutdown_connections 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 1 -- 172.21.15.77:0/1358670167 wait complete. 2024-09-15T22:12:14.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.241+0000 7fb48e246740 20 client.15243 trim_cache size 0 max 16384 2024-09-15T22:12:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 Processor -- start 2024-09-15T22:12:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 -- start start 2024-09-15T22:12:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19eca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c19eddf20 con 0x563c19ec29d0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c19ede5f0 con 0x563c19ec32f0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb48e246740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c19edecc0 con 0x563c19ecdbb0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52242/0 (socket says 172.21.15.77:52242) 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 -- 172.21.15.77:0/3363992323 learned_addr learned my addr 172.21.15.77:0/3363992323 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 -- 172.21.15.77:0/3363992323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 msgr2=0x563c19ecac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 -- 172.21.15.77:0/3363992323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 msgr2=0x563c19eca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19eca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 -- 172.21.15.77:0/3363992323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c19ecb2d0 con 0x563c19ec29d0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb489c09640 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb47409b230 tx=0x7fb4740cf1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9605db35dce0650a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb46b7fe640 1 -- 172.21.15.77:0/3363992323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740bb040 con 0x563c19ec29d0 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb46b7fe640 1 -- 172.21.15.77:0/3363992323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4740a41a0 con 0x563c19ec29d0 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.259+0000 7fb46b7fe640 1 -- 172.21.15.77:0/3363992323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47409aaf0 con 0x563c19ec29d0 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- 172.21.15.77:0/3363992323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 msgr2=0x563c19edfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb47409b230 tx=0x7fb4740cf1e0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- 172.21.15.77:0/3363992323 shutdown_connections 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ecdbb0 0x563c19ecac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec32f0 0x563c19eca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 --2- 172.21.15.77:0/3363992323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ec29d0 0x563c19edfbe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- 172.21.15.77:0/3363992323 >> 172.21.15.77:0/3363992323 conn(0x563c19ebef60 msgr2=0x563c19eb9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- 172.21.15.77:0/3363992323 shutdown_connections 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- 172.21.15.77:0/3363992323 wait complete. 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 -- start start 2024-09-15T22:12:14.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.260+0000 7fb48e246740 1 Processor -- start 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec29d0 0x563c19ee3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19ee3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c19edfa60 con 0x563c19ecdbb0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c19edf390 con 0x563c19ec29d0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c19ee0130 con 0x563c19ec32f0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48e246740 10 client.0 _ll_register_callbacks cb 0x7fb47409b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52246/0 (socket says 172.21.15.77:52246) 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 -- 172.21.15.77:0/1654367164 learned_addr learned my addr 172.21.15.77:0/1654367164 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb489c09640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec29d0 0x563c19ee3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb489408640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19ee3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 -- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 msgr2=0x563c19ee3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19ee3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 -- 172.21.15.77:0/1654367164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec29d0 msgr2=0x563c19ee3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec29d0 0x563c19ee3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 -- 172.21.15.77:0/1654367164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c19f473b0 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb48a40a640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb47c04aca0 tx=0x7fb47c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86f02cbfe882ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.261+0000 7fb4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c060060 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb47c031dd0 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4740a41a0 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb47c035030 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 10 client.15249 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c19f48660 con 0x563c19ecdbb0 2024-09-15T22:12:14.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4740a41a0 con 0x563c19ecdbb0 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb47c03a070 con 0x563c19ecdbb0 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 20 client.15249 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19f4ce60 0x563c19f4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 4 client.15249 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.262+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "status"}) -- 0x563c19e30b10 con 0x563c19f4ce60 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb489c09640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19f4ce60 0x563c19f4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb489c09640 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19f4ce60 0x563c19f4bb70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea9e4f75d46fdb42 server_cookie=2538ba6fc9cd9c35 in_seq=0 out_seq=0 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb4697fa640 10 client.15249 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb4697fa640 1 -- 172.21.15.77:0/1654367164 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563c19e30b10 con 0x563c19f4ce60 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb4697fa640 10 client.15249 handle_command_reply: tid=0 2024-09-15T22:12:14.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb48e246740 1 client.15249 shutdown 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 msgr2=0x563c19f46e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.263+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb47c04aca0 tx=0x7fb47c0045b0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 shutdown_connections 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x563c19f4ce60 0x563c19f4bb70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c19ecdbb0 0x563c19f46e70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c19ec32f0 0x563c19ee3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb4697fa640 0 client.15249 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 --2- 172.21.15.77:0/1654367164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c19ec29d0 0x563c19ee3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 >> 172.21.15.77:0/1654367164 conn(0x563c19ebef60 msgr2=0x563c19eca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 shutdown_connections 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 1 -- 172.21.15.77:0/1654367164 wait complete. 2024-09-15T22:12:14.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48e246740 20 client.15249 trim_cache size 0 max 16384 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24439, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 96.936415948999993, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:12:14.274 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 157.91184908100001 2024-09-15T22:12:14.275 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:14.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 msgr2=0x7fb46c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 0x7fb46c0a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02a940 tx=0x7fb47c04d000 comp rx=0 tx=0).stop 2024-09-15T22:12:14.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 msgr2=0x7fb484199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:14.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb47400ba50 tx=0x7fb47400bf10 comp rx=0 tx=0).stop 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 shutdown_connections 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb46c0a32c0 0x7fb46c0a5780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb484106a70 0x7fb4841a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb484101410 0x7fb484199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 --2- 172.21.15.77:0/2807264996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb484100a70 0x7fb484199330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 >> 172.21.15.77:0/2807264996 conn(0x7fb4840fc790 msgr2=0x7fb4840fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 shutdown_connections 2024-09-15T22:12:14.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:14.264+0000 7fb48be6d640 1 -- 172.21.15.77:0/2807264996 wait complete. 2024-09-15T22:12:15.044 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:12:15.286 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:15.573 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:15 smithi077 ceph-mon[30980]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-15T22:12:15.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:15 smithi196 ceph-mon[35109]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-15T22:12:15.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:15 smithi117 ceph-mon[35021]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 -- 172.21.15.77:0/4073863985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c1055b0 msgr2=0x7fa65c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 --2- 172.21.15.77:0/4073863985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c1055b0 0x7fa65c10d130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa65400adb0 tx=0x7fa65402b7a0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 -- 172.21.15.77:0/4073863985 shutdown_connections 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 --2- 172.21.15.77:0/4073863985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c1055b0 0x7fa65c10d130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 --2- 172.21.15.77:0/4073863985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 --2- 172.21.15.77:0/4073863985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c100480 0x7fa65c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 -- 172.21.15.77:0/4073863985 >> 172.21.15.77:0/4073863985 conn(0x7fa65c077210 msgr2=0x7fa65c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 -- 172.21.15.77:0/4073863985 shutdown_connections 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.802+0000 7fa662fa5640 1 -- 172.21.15.77:0/4073863985 wait complete. 2024-09-15T22:12:15.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 Processor -- start 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 -- start start 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c1055b0 0x7fa65c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c10fd90 con 0x7fa65c100480 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c10fc10 con 0x7fa65c100e30 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa662fa5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c10ff10 con 0x7fa65c1055b0 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa660d41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa660d41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52270/0 (socket says 172.21.15.77:52270) 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa660d41640 1 -- 172.21.15.77:0/877491438 learned_addr learned my addr 172.21.15.77:0/877491438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa661542640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c1055b0 0x7fa65c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa65bfff640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa660d41640 1 -- 172.21.15.77:0/877491438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c1055b0 msgr2=0x7fa65c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.803+0000 7fa660d41640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c1055b0 0x7fa65c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa660d41640 1 -- 172.21.15.77:0/877491438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 msgr2=0x7fa65c196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa660d41640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa660d41640 1 -- 172.21.15.77:0/877491438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c19b080 con 0x7fa65c100480 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa65bfff640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa660d41640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa65000c920 tx=0x7fa65000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa659ffb640 1 -- 172.21.15.77:0/877491438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650007c70 con 0x7fa65c100480 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c19b370 con 0x7fa65c100480 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa659ffb640 1 -- 172.21.15.77:0/877491438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa650007e10 con 0x7fa65c100480 2024-09-15T22:12:15.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa659ffb640 1 -- 172.21.15.77:0/877491438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650018890 con 0x7fa65c100480 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.804+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c1a2c50 con 0x7fa65c100480 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.806+0000 7fa659ffb640 1 -- 172.21.15.77:0/877491438 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa650018a70 con 0x7fa65c100480 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 Processor -- start 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 -- start start 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa659ffb640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab6867a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6867adc0 0x55ab6867d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab685d4090 con 0x55ab6866d890 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab6857ba70 con 0x55ab6867adc0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa66537e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab685d0050 con 0x55ab6866e1b0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa660d41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa661542640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa660d41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52282/0 (socket says 172.21.15.77:52282) 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa660d41640 1 -- 172.21.15.77:0/3745176008 learned_addr learned my addr 172.21.15.77:0/3745176008 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa65bfff640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab6867a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa659ffb640 1 -- 172.21.15.77:0/877491438 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa6500cfe20 con 0x7fa65c100480 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.807+0000 7fa661542640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6867adc0 0x55ab6867d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 -- 172.21.15.77:0/3745176008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 msgr2=0x55ab6867a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab6867a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 -- 172.21.15.77:0/3745176008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6867adc0 msgr2=0x55ab6867d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6867adc0 0x55ab6867d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 -- 172.21.15.77:0/3745176008 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68588130 con 0x55ab6866d890 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa65bfff640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab6867a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa661542640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 0x7fa64c0a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa65400ab70 tx=0x7fa654003b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa660d41640 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa6500984e0 tx=0x7fa650020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159acdc9760df08f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa646ffd640 1 -- 172.21.15.77:0/3745176008 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650067630 con 0x55ab6866d890 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa646ffd640 1 -- 172.21.15.77:0/3745176008 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65009c080 con 0x55ab6866d890 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.808+0000 7fa646ffd640 1 -- 172.21.15.77:0/3745176008 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65000fb30 con 0x55ab6866d890 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- 172.21.15.77:0/3745176008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 msgr2=0x55ab6866dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa6500984e0 tx=0x7fa650020000 comp rx=0 tx=0).stop 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- 172.21.15.77:0/3745176008 shutdown_connections 2024-09-15T22:12:15.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6867adc0 0x55ab6867d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab6867a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 --2- 172.21.15.77:0/3745176008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866d890 0x55ab6866dc70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- 172.21.15.77:0/3745176008 >> 172.21.15.77:0/3745176008 conn(0x55ab68669f60 msgr2=0x55ab6866a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- 172.21.15.77:0/3745176008 shutdown_connections 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- 172.21.15.77:0/3745176008 wait complete. 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 -- start start 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.809+0000 7fa66537e740 1 Processor -- start 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab6867a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab68677a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab6867d930 con 0x55ab6867adc0 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab6858ef20 con 0x55ab6866d890 2024-09-15T22:12:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab6867dab0 con 0x55ab6866e1b0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa66537e740 10 client.0 _ll_register_callbacks cb 0x55ab68558030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52284/0 (socket says 172.21.15.77:52284) 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 -- 172.21.15.77:0/4011217088 learned_addr learned my addr 172.21.15.77:0/4011217088 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa660d41640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab6867a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa65bfff640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab68677a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 -- 172.21.15.77:0/4011217088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 msgr2=0x55ab68677a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab68677a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 -- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 msgr2=0x55ab6867a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab6867a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 -- 172.21.15.77:0/4011217088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab683452b0 con 0x55ab6867adc0 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa660d41640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab6867a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa661542640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa654035000 tx=0x7fa65402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f667746a2a94864 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.810+0000 7fa63bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654052070 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa654030510 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65009c080 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa65405e050 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 10 client.15261 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab686892f0 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65009c080 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa654040610 con 0x55ab6867adc0 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 10 client.15261 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 10 client.15261 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:12:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 20 client.15261 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab6868cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 4 client.15261 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:15.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.811+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab685e1600 con 0x55ab6868b980 2024-09-15T22:12:15.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.812+0000 7fa660d41640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab6868cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.813+0000 7fa660d41640 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab6868cdf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e35430cbcb6d276 server_cookie=85836e35d9659075 in_seq=0 out_seq=0 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.813+0000 7fa63bfff640 10 client.15261 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa63bfff640 1 -- 172.21.15.77:0/4011217088 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab685e1600 con 0x55ab6868b980 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa63bfff640 10 client.15261 handle_command_reply: tid=0 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 client.15261 shutdown 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 msgr2=0x55ab68677f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa654035000 tx=0x7fa65402b7a0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 shutdown_connections 2024-09-15T22:12:15.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab6868cdf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa63bfff640 0 client.15261 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab68677f60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866e1b0 0x55ab68677a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 --2- 172.21.15.77:0/4011217088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab6867a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 >> 172.21.15.77:0/4011217088 conn(0x55ab68669f60 msgr2=0x55ab6866fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 shutdown_connections 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 1 -- 172.21.15.77:0/4011217088 wait complete. 2024-09-15T22:12:15.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.814+0000 7fa66537e740 20 client.15261 trim_cache size 0 max 16384 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 Processor -- start 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 -- start start 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab686779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6867adc0 0x55ab686762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868b130 con 0x55ab6866e1b0 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868b800 con 0x55ab6866d890 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa66537e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868bed0 con 0x55ab6867adc0 2024-09-15T22:12:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa65bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52288/0 (socket says 172.21.15.77:52288) 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.832+0000 7fa65bfff640 1 -- 172.21.15.77:0/2325247614 learned_addr learned my addr 172.21.15.77:0/2325247614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa660d41640 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab686779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 -- 172.21.15.77:0/2325247614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6867adc0 msgr2=0x55ab686762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6867adc0 0x55ab686762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 -- 172.21.15.77:0/2325247614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 msgr2=0x55ab686779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab686779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 -- 172.21.15.77:0/2325247614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68675f70 con 0x55ab6866e1b0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa65bfff640 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa64800cba0 tx=0x7fa648012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf62d5d586054e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa6467fc640 1 -- 172.21.15.77:0/2325247614 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648004230 con 0x55ab6866e1b0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa6467fc640 1 -- 172.21.15.77:0/2325247614 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6480043d0 con 0x55ab6866e1b0 2024-09-15T22:12:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.833+0000 7fa6467fc640 1 -- 172.21.15.77:0/2325247614 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64801b9e0 con 0x55ab6866e1b0 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- 172.21.15.77:0/2325247614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 msgr2=0x55ab68677f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa64800cba0 tx=0x7fa648012820 comp rx=0 tx=0).stop 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- 172.21.15.77:0/2325247614 shutdown_connections 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6867adc0 0x55ab686762f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6866e1b0 0x55ab68677f20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 --2- 172.21.15.77:0/2325247614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866d890 0x55ab686779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- 172.21.15.77:0/2325247614 >> 172.21.15.77:0/2325247614 conn(0x55ab68669f60 msgr2=0x55ab68664ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- 172.21.15.77:0/2325247614 shutdown_connections 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- 172.21.15.77:0/2325247614 wait complete. 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 -- start start 2024-09-15T22:12:15.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.834+0000 7fa66537e740 1 Processor -- start 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866d890 0x55ab6868e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 0x55ab6868ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868cc70 con 0x55ab6867adc0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868c5a0 con 0x55ab6866e1b0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab6868d340 con 0x55ab6866d890 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa66537e740 10 client.0 _ll_register_callbacks cb 0x7fa6540328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52298/0 (socket says 172.21.15.77:52298) 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 -- 172.21.15.77:0/2728302824 learned_addr learned my addr 172.21.15.77:0/2728302824 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa660d41640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866d890 0x55ab6868e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa65bfff640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 0x55ab6868ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 -- 172.21.15.77:0/2728302824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866d890 msgr2=0x55ab6868e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866d890 0x55ab6868e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 -- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 msgr2=0x55ab6868ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 0x55ab6868ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa661542640 1 -- 172.21.15.77:0/2728302824 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab686f45f0 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.835+0000 7fa65bfff640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 0x55ab6868ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa661542640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa65402b7e0 tx=0x7fa654031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bbbc5521a49dad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654052070 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa654030510 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6480043d0 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa654031e60 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 10 client.15267 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab686f58f0 con 0x55ab6867adc0 2024-09-15T22:12:15.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6480043d0 con 0x55ab6867adc0 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa654057790 con 0x55ab6867adc0 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.836+0000 7fa66537e740 20 client.15267 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab686f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa66537e740 4 client.15267 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "status"}) -- 0x55ab685dbb10 con 0x55ab6868b980 2024-09-15T22:12:15.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa660d41640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab686f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:15.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa63bfff640 10 client.15267 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:15.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.837+0000 7fa660d41640 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab686f9250 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a130f34ea4aa32b9 server_cookie=849366363feb8124 in_seq=0 out_seq=0 2024-09-15T22:12:15.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa63bfff640 1 -- 172.21.15.77:0/2728302824 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ab685dbb10 con 0x55ab6868b980 2024-09-15T22:12:15.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa63bfff640 10 client.15267 handle_command_reply: tid=0 2024-09-15T22:12:15.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 client.15267 shutdown 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 msgr2=0x55ab686f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa65402b7e0 tx=0x7fa654031a70 comp rx=0 tx=0).stop 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 shutdown_connections 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ab6868b980 0x55ab686f9250 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab6867adc0 0x55ab686f4080 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa63bfff640 0 client.15267 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab6866e1b0 0x55ab6868ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 --2- 172.21.15.77:0/2728302824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab6866d890 0x55ab6868e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 >> 172.21.15.77:0/2728302824 conn(0x55ab68669f60 msgr2=0x55ab686779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 shutdown_connections 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 1 -- 172.21.15.77:0/2728302824 wait complete. 2024-09-15T22:12:15.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.838+0000 7fa66537e740 20 client.15267 trim_cache size 0 max 16384 2024-09-15T22:12:15.848 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:15.848 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:15.848 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-15T22:12:15.848 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24440, 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 99.514402059999995, 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:12:15.849 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 162.02293395000001 2024-09-15T22:12:15.850 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 msgr2=0x7fa64c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 0x7fa64c0a5700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa65400ab70 tx=0x7fa654003b80 comp rx=0 tx=0).stop 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 msgr2=0x7fa65c1960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa65000c920 tx=0x7fa65000cde0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 shutdown_connections 2024-09-15T22:12:15.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fa64c0a3240 0x7fa64c0a5700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c1055b0 0x7fa65c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c100e30 0x7fa65c196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 --2- 172.21.15.77:0/877491438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa65c100480 0x7fa65c1960d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 >> 172.21.15.77:0/877491438 conn(0x7fa65c077210 msgr2=0x7fa65c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:15.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 shutdown_connections 2024-09-15T22:12:15.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:15.839+0000 7fa662fa5640 1 -- 172.21.15.77:0/877491438 wait complete. 2024-09-15T22:12:16.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:12:16.813 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:17.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 -- 172.21.15.77:0/1710685152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8281027a0 msgr2=0x7ff8281067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 --2- 172.21.15.77:0/1710685152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8281027a0 0x7ff8281067f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009900 tx=0x7ff81c02a410 comp rx=0 tx=0).stop 2024-09-15T22:12:17.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 -- 172.21.15.77:0/1710685152 shutdown_connections 2024-09-15T22:12:17.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 --2- 172.21.15.77:0/1710685152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff828106f20 0x7ff82810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 --2- 172.21.15.77:0/1710685152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8281027a0 0x7ff8281067f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 --2- 172.21.15.77:0/1710685152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 0x7ff8281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 -- 172.21.15.77:0/1710685152 >> 172.21.15.77:0/1710685152 conn(0x7ff8280fd8f0 msgr2=0x7ff8280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 -- 172.21.15.77:0/1710685152 shutdown_connections 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.303+0000 7ff83063d640 1 -- 172.21.15.77:0/1710685152 wait complete. 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 Processor -- start 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 -- start start 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 0x7ff8280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8281027a0 0x7ff828074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111f80 con 0x7ff828106f20 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff828111e00 con 0x7ff8281027a0 2024-09-15T22:12:17.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff83063d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff828112100 con 0x7ff828101df0 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52326/0 (socket says 172.21.15.77:52326) 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 -- 172.21.15.77:0/1960793834 learned_addr learned my addr 172.21.15.77:0/1960793834 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8281027a0 0x7ff828074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 0x7ff8280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 -- 172.21.15.77:0/1960793834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 msgr2=0x7ff8280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 0x7ff8280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 -- 172.21.15.77:0/1960793834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8281027a0 msgr2=0x7ff828074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8281027a0 0x7ff828074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.304+0000 7ff82ebda640 1 -- 172.21.15.77:0/1960793834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8281a6880 con 0x7ff828106f20 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.305+0000 7ff82ebda640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff818009fd0 tx=0x7ff81800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.306+0000 7ff8277fe640 1 -- 172.21.15.77:0/1960793834 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818017070 con 0x7ff828106f20 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.306+0000 7ff8277fe640 1 -- 172.21.15.77:0/1960793834 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff818003e50 con 0x7ff828106f20 2024-09-15T22:12:17.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.306+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a6b70 con 0x7ff828106f20 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.306+0000 7ff8277fe640 1 -- 172.21.15.77:0/1960793834 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818010460 con 0x7ff828106f20 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.306+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a7030 con 0x7ff828106f20 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.308+0000 7ff8277fe640 1 -- 172.21.15.77:0/1960793834 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff818003970 con 0x7ff828106f20 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.308+0000 7ff832a16740 1 Processor -- start 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 -- start start 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c897db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff8277fe640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 0x7ff8100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b1c7fe090 con 0x559b1c8a2bb0 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b1c7a5a70 con 0x559b1c8979d0 2024-09-15T22:12:17.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff832a16740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b1c7fa050 con 0x559b1c8982f0 2024-09-15T22:12:17.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82ebda640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 0x7ff8100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52338/0 (socket says 172.21.15.77:52338) 2024-09-15T22:12:17.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82dbd8640 1 -- 172.21.15.77:0/276668785 learned_addr learned my addr 172.21.15.77:0/276668785 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:17.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c897db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.309+0000 7ff8277fe640 1 -- 172.21.15.77:0/1960793834 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8180cf860 con 0x7ff828106f20 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 -- 172.21.15.77:0/276668785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 msgr2=0x559b1c8a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 -- 172.21.15.77:0/276668785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 msgr2=0x559b1c897db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c897db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 -- 172.21.15.77:0/276668785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1c7b2130 con 0x559b1c8a2bb0 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82ebda640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 0x7ff8100a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b6f0 tx=0x7ff81806d8f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009900 tx=0x7ff81c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7822229c3195da36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff824ff9640 1 -- 172.21.15.77:0/276668785 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c035070 con 0x559b1c8a2bb0 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff824ff9640 1 -- 172.21.15.77:0/276668785 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c005020 con 0x559b1c8a2bb0 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.310+0000 7ff824ff9640 1 -- 172.21.15.77:0/276668785 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0049e0 con 0x559b1c8a2bb0 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- 172.21.15.77:0/276668785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 msgr2=0x559b1c8a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009900 tx=0x7ff81c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- 172.21.15.77:0/276668785 shutdown_connections 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8a2bb0 0x559b1c8a4fa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 --2- 172.21.15.77:0/276668785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c897db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- 172.21.15.77:0/276668785 >> 172.21.15.77:0/276668785 conn(0x559b1c893f60 msgr2=0x559b1c894340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- 172.21.15.77:0/276668785 shutdown_connections 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- 172.21.15.77:0/276668785 wait complete. 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 -- start start 2024-09-15T22:12:17.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.311+0000 7ff832a16740 1 Processor -- start 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 0x559b1c89f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c89fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8a5780 con 0x559b1c8979d0 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b1c7b8f20 con 0x559b1c8982f0 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8a5900 con 0x559b1c8a2bb0 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 10 client.0 _ll_register_callbacks cb 0x559b1c782030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52354/0 (socket says 172.21.15.77:52354) 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 -- 172.21.15.77:0/1927584422 learned_addr learned my addr 172.21.15.77:0/1927584422 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:17.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 0x559b1c89f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c89fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 -- 172.21.15.77:0/1927584422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 msgr2=0x559b1c89fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c89fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 -- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 msgr2=0x559b1c89f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 0x559b1c89f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 -- 172.21.15.77:0/1927584422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1c56f2b0 con 0x559b1c8979d0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 0x559b1c89f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff81c001eb0 tx=0x7ff81c032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2e58b908cd0f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff80dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c041070 con 0x559b1c8979d0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff81c004950 con 0x559b1c8979d0 2024-09-15T22:12:17.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c005020 con 0x559b1c8979d0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff81c05b050 con 0x559b1c8979d0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 10 client.15279 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.312+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b1c8b1070 con 0x559b1c8979d0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c005020 con 0x559b1c8979d0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff81c03db30 con 0x559b1c8979d0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 10 client.15279 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 20 client.15279 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c8b3170 0x559b1c8b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 4 client.15279 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1c80b600 con 0x559b1c8b3170 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c8b3170 0x559b1c8b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff80dffb640 10 client.15279 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.313+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c8b3170 0x559b1c8b45e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261d4692ff9449e3 server_cookie=ef8df3c162c8424b in_seq=0 out_seq=0 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff80dffb640 1 -- 172.21.15.77:0/1927584422 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b1c80b600 con 0x559b1c8b3170 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff80dffb640 10 client.15279 handle_command_reply: tid=0 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 client.15279 shutdown 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 msgr2=0x559b1c8a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff81c001eb0 tx=0x7ff81c032db0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 shutdown_connections 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c8b3170 0x559b1c8b45e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c89fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8982f0 0x559b1c89f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff80dffb640 0 client.15279 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 --2- 172.21.15.77:0/1927584422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8a23f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 >> 172.21.15.77:0/1927584422 conn(0x559b1c893f60 msgr2=0x559b1c899c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 shutdown_connections 2024-09-15T22:12:17.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.314+0000 7ff832a16740 1 -- 172.21.15.77:0/1927584422 wait complete. 2024-09-15T22:12:17.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.315+0000 7ff832a16740 20 client.15279 trim_cache size 0 max 16384 2024-09-15T22:12:17.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 Processor -- start 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 -- start start 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8a2bb0 0x559b1c89f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b2920 con 0x559b1c8979d0 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b2ff0 con 0x559b1c8a2bb0 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff832a16740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b36c0 con 0x559b1c8982f0 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff82dbd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52366/0 (socket says 172.21.15.77:52366) 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.332+0000 7ff82dbd8640 1 -- 172.21.15.77:0/2839270836 learned_addr learned my addr 172.21.15.77:0/2839270836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 -- 172.21.15.77:0/2839270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 msgr2=0x559b1c8b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 -- 172.21.15.77:0/2839270836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8a2bb0 msgr2=0x559b1c89f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8a2bb0 0x559b1c89f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 -- 172.21.15.77:0/2839270836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1c8a2760 con 0x559b1c8979d0 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0092f0 tx=0x7ff81c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dcf64fb9ed9e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff80ffff640 1 -- 172.21.15.77:0/2839270836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c063070 con 0x559b1c8979d0 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff80ffff640 1 -- 172.21.15.77:0/2839270836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c009520 con 0x559b1c8979d0 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.333+0000 7ff80ffff640 1 -- 172.21.15.77:0/2839270836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c041040 con 0x559b1c8979d0 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- 172.21.15.77:0/2839270836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 msgr2=0x559b1c8b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0092f0 tx=0x7ff81c005fd0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- 172.21.15.77:0/2839270836 shutdown_connections 2024-09-15T22:12:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8a2bb0 0x559b1c89f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8982f0 0x559b1c8b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- 172.21.15.77:0/2839270836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8979d0 0x559b1c8b45e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- 172.21.15.77:0/2839270836 >> 172.21.15.77:0/2839270836 conn(0x559b1c893f60 msgr2=0x559b1c894340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- 172.21.15.77:0/2839270836 shutdown_connections 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- 172.21.15.77:0/2839270836 wait complete. 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- start start 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 Processor -- start 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c8bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c8b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b4460 con 0x559b1c8982f0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b3d90 con 0x559b1c8979d0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559b1c8b4b30 con 0x559b1c8a2bb0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff832a16740 10 client.0 _ll_register_callbacks cb 0x7ff81c032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff82e3d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.334+0000 7ff82e3d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52376/0 (socket says 172.21.15.77:52376) 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 -- 172.21.15.77:0/1123311777 learned_addr learned my addr 172.21.15.77:0/1123311777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 -- 172.21.15.77:0/1123311777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 msgr2=0x559b1c8b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c8b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c8b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 -- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 msgr2=0x559b1c8bd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:17.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c8bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 -- 172.21.15.77:0/1123311777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1c8bac50 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c8b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff82e3d9640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff82000adf0 tx=0x7ff820010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c99d8e2deae173 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff80dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820005220 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8200053c0 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c009520 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff820030050 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff832a16740 10 client.15285 fetch_fsmap learned FSMap version 29 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b1c91daa0 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.335+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c009520 con 0x559b1c8982f0 2024-09-15T22:12:17.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff82001ea10 con 0x559b1c8982f0 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 10 client.15285 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 10 client.15285 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 20 client.15285 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c922650 0x559b1c921360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 4 client.15285 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "status"}) -- 0x559b1c805b10 con 0x559b1c922650 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c922650 0x559b1c921360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff82dbd8640 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c922650 0x559b1c921360 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987f0bbe97be4888 server_cookie=7608ef48b10b965 in_seq=0 out_seq=0 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff80dffb640 10 client.15285 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff80dffb640 1 -- 172.21.15.77:0/1123311777 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559b1c805b10 con 0x559b1c922650 2024-09-15T22:12:17.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff80dffb640 10 client.15285 handle_command_reply: tid=0 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.336+0000 7ff832a16740 1 client.15285 shutdown 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 msgr2=0x559b1c8b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff82000adf0 tx=0x7ff820010800 comp rx=0 tx=0).stop 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 shutdown_connections 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559b1c922650 0x559b1c921360 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b1c8a2bb0 0x559b1c8b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b1c8982f0 0x559b1c8b6460 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 --2- 172.21.15.77:0/1123311777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b1c8979d0 0x559b1c8bd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 >> 172.21.15.77:0/1123311777 conn(0x559b1c893f60 msgr2=0x559b1c8927e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff80dffb640 0 client.15285 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:17.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 shutdown_connections 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 1 -- 172.21.15.77:0/1123311777 wait complete. 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff832a16740 20 client.15285 trim_cache size 0 max 16384 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24458, 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:17.355 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 141.13913618300001, 2024-09-15T22:12:17.356 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-09-15T22:12:17.356 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 90, 2024-09-15T22:12:17.356 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:12:17.356 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 151.94297714000001 2024-09-15T22:12:17.356 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 msgr2=0x7ff8100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 0x7ff8100a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b6f0 tx=0x7ff81806d8f0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 msgr2=0x7ff828074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff818009fd0 tx=0x7ff81800def0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 shutdown_connections 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7ff8100a3240 0x7ff8100a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff828106f20 0x7ff828074f70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8281027a0 0x7ff828074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 --2- 172.21.15.77:0/1960793834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff828101df0 0x7ff8280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:17.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 >> 172.21.15.77:0/1960793834 conn(0x7ff8280fd8f0 msgr2=0x7ff8280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:17.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 shutdown_connections 2024-09-15T22:12:17.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:17.337+0000 7ff83063d640 1 -- 172.21.15.77:0/1960793834 wait complete. 2024-09-15T22:12:17.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:17 smithi077 ceph-mon[30980]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:17 smithi196 ceph-mon[35109]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:17 smithi117 ceph-mon[35021]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-15T22:12:18.050 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T22:12:18.060 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T22:12:18.060 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T22:12:18.060 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T22:12:18.060 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-15T22:12:18.061 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:12:18.061 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:12:18.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-15T22:12:18.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 -- 172.21.15.77:0/3511946677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4107830 msgr2=0x7f0ef4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:18.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 --2- 172.21.15.77:0/3511946677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4107830 0x7f0ef4109ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009ca0 tx=0x7f0eec02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 -- 172.21.15.77:0/3511946677 shutdown_connections 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 --2- 172.21.15.77:0/3511946677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4107830 0x7f0ef4109ca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 --2- 172.21.15.77:0/3511946677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4104f00 0x7f0ef41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 --2- 172.21.15.77:0/3511946677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef41025d0 0x7f0ef41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 -- 172.21.15.77:0/3511946677 >> 172.21.15.77:0/3511946677 conn(0x7f0ef40fc1b0 msgr2=0x7f0ef40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 -- 172.21.15.77:0/3511946677 shutdown_connections 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.162+0000 7f0ef9559640 1 -- 172.21.15.77:0/3511946677 wait complete. 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 Processor -- start 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 -- start start 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef41025d0 0x7f0ef4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:18.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4107830 0x7f0ef419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef410d870 con 0x7f0ef4104f00 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef410d6f0 con 0x7f0ef4107830 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0ef9559640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef410d9f0 con 0x7f0ef41025d0 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0eea7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0eea7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52400/0 (socket says 172.21.15.77:52400) 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.163+0000 7f0eea7fc640 1 -- 172.21.15.77:0/728206175 learned_addr learned my addr 172.21.15.77:0/728206175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef2ffd640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef41025d0 0x7f0ef4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 -- 172.21.15.77:0/728206175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef41025d0 msgr2=0x7f0ef4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef41025d0 0x7f0ef4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 -- 172.21.15.77:0/728206175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4107830 msgr2=0x7f0ef419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4107830 0x7f0ef419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 -- 172.21.15.77:0/728206175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419fa70 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0eea7fc640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00c970 tx=0x7f0edc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc007bb0 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0edc007d50 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419fd60 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00f450 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.164+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a02d0 con 0x7f0ef4104f00 2024-09-15T22:12:18.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.165+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40fdca0 con 0x7f0ef4104f00 2024-09-15T22:12:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.166+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0edc004ee0 con 0x7f0ef4104f00 2024-09-15T22:12:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.167+0000 7f0ef0ff9640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 0x7f0ee00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.167+0000 7f0ef2ffd640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 0x7f0ee00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.167+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0edc0cfca0 con 0x7f0ef4104f00 2024-09-15T22:12:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.167+0000 7f0ef2ffd640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 0x7f0ee00a57b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009990 tx=0x7f0ee4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.170+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0edc014030 con 0x7f0ef4104f00 2024-09-15T22:12:18.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:18.387+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0ef402da10 con 0x7f0ef4104f00 2024-09-15T22:12:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.308+0000 7f0ef0ff9640 1 -- 172.21.15.77:0/728206175 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f0edc09c660 con 0x7f0ef4104f00 2024-09-15T22:12:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 msgr2=0x7f0ee00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 0x7f0ee00a57b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009990 tx=0x7f0ee4009110 comp rx=0 tx=0).stop 2024-09-15T22:12:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 msgr2=0x7f0ef4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00c970 tx=0x7f0edc00cd70 comp rx=0 tx=0).stop 2024-09-15T22:12:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 shutdown_connections 2024-09-15T22:12:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f0ee00a32f0 0x7f0ee00a57b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ef4107830 0x7f0ef419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ef4104f00 0x7f0ef4198390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 --2- 172.21.15.77:0/728206175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef41025d0 0x7f0ef4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 >> 172.21.15.77:0/728206175 conn(0x7f0ef40fc1b0 msgr2=0x7f0ef4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.311+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 shutdown_connections 2024-09-15T22:12:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:19.312+0000 7f0ef9559640 1 -- 172.21.15.77:0/728206175 wait complete. 2024-09-15T22:12:19.331 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T22:12:19.350 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T22:12:19.351 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-15T22:12:19.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:19.351 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-15T22:12:19.368 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-15T22:12:19.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:19.369 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:12:19.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:19.435 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-09-15T22:12:19.532 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-15T22:12:19.532 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:12:19.532 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:19.532 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:12:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-mon[30980]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-15T22:12:19.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:12:19.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setuser ceph since I am not root 2024-09-15T22:12:19.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setgroup ceph since I am not root 2024-09-15T22:12:19.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:19.467+0000 7fda25ce9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:12:19.583 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:19.531+0000 7fda25ce9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-mon[35109]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-15T22:12:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:12:19.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: ignoring --setuser ceph since I am not root 2024-09-15T22:12:19.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: ignoring --setgroup ceph since I am not root 2024-09-15T22:12:19.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-mgr[36801]: -- 172.21.15.196:0/944681375 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0e94670e0 con 0x55d0e9445000 2024-09-15T22:12:19.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:19.463+0000 7f79dadee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:12:19.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:19.520+0000 7f79dadee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-mon[35021]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-15T22:12:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:12:19.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: ignoring --setuser ceph since I am not root 2024-09-15T22:12:19.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: ignoring --setgroup ceph since I am not root 2024-09-15T22:12:19.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-mgr[36396]: -- 172.21.15.117:0/504063431 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56363bf830e0 con 0x56363bf61400 2024-09-15T22:12:19.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:19.460+0000 7fd5dbb7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:12:19.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:19.516+0000 7fd5dbb7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:19.788 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:20.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:19.664+0000 7fd5dbb7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:12:20.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:19.732+0000 7fd5dbb7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:12:20.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:19.878+0000 7fd5dbb7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:20.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:20.019+0000 7fd5dbb7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:12:20.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:19.667+0000 7f79dadee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:12:20.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:19.734+0000 7f79dadee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:12:20.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:19.881+0000 7f79dadee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:20.022 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:20.020+0000 7f79dadee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:12:20.052 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:19.687+0000 7fda25ce9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:12:20.052 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:19.760+0000 7fda25ce9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:12:20.053 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:19.916+0000 7fda25ce9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 -- 172.21.15.77:0/2806209259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102ef0 msgr2=0x7f8e8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 --2- 172.21.15.77:0/2806209259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102ef0 0x7f8e8410da40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009ca0 tx=0x7f8e7c02a950 comp rx=0 tx=0).stop 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 -- 172.21.15.77:0/2806209259 shutdown_connections 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 --2- 172.21.15.77:0/2806209259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102ef0 0x7f8e8410da40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 --2- 172.21.15.77:0/2806209259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84102570 0x7f8e841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 --2- 172.21.15.77:0/2806209259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84100e20 0x7f8e84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 -- 172.21.15.77:0/2806209259 >> 172.21.15.77:0/2806209259 conn(0x7f8e840fc7d0 msgr2=0x7f8e840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 -- 172.21.15.77:0/2806209259 shutdown_connections 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.283+0000 7f8e890fb640 1 -- 172.21.15.77:0/2806209259 wait complete. 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 Processor -- start 2024-09-15T22:12:20.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 -- start start 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84100e20 0x7f8e841993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84102ef0 0x7f8e84199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84110f80 con 0x7f8e84102570 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84110e00 con 0x7f8e84100e20 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e890fb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e84111100 con 0x7f8e84102ef0 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49078/0 (socket says 172.21.15.77:49078) 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 -- 172.21.15.77:0/57046787 learned_addr learned my addr 172.21.15.77:0/57046787 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e82d9d640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84100e20 0x7f8e841993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e8359e640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84102ef0 0x7f8e84199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84102ef0 msgr2=0x7f8e84199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:20.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84102ef0 0x7f8e84199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84100e20 msgr2=0x7f8e841993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84100e20 0x7f8e841993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.284+0000 7f8e7bfff640 1 -- 172.21.15.77:0/57046787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e841a47d0 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e7bfff640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d950 tx=0x7f8e6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c017070 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c009c30 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e841a4ac0 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00f6b0 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.285+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a5050 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.286+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e84101200 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.287+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f8e6c0109d0 con 0x7f8e84102570 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.288+0000 7f8e80d99640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 0x7f8e700a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:20.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.288+0000 7f8e82d9d640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 msgr2=0x7f8e700a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:20.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.288+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8e6c0cf770 con 0x7f8e84102570 2024-09-15T22:12:20.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.288+0000 7f8e82d9d640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 0x7f8e700a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:12:20.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.292+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e6c009de0 con 0x7f8e84102570 2024-09-15T22:12:20.322 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:20.065+0000 7fda25ce9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:12:20.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.489+0000 7f8e82d9d640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 msgr2=0x7f8e700a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:20.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.489+0000 7f8e82d9d640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 0x7f8e700a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:12:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.532+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e84109090 con 0x7f8e84102570 2024-09-15T22:12:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.533+0000 7f8e80d99640 1 -- 172.21.15.77:0/57046787 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f8e6c09bae0 con 0x7f8e84102570 2024-09-15T22:12:20.534 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 msgr2=0x7f8e700a57d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 0x7f8e700a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 msgr2=0x7f8e84199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00d950 tx=0x7f8e6c00de10 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 shutdown_connections 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7f8e700a3310 0x7f8e700a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e84102ef0 0x7f8e84199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e84102570 0x7f8e84199910 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 --2- 172.21.15.77:0/57046787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e84100e20 0x7f8e841993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 >> 172.21.15.77:0/57046787 conn(0x7f8e840fc7d0 msgr2=0x7f8e841068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 shutdown_connections 2024-09-15T22:12:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:20.535+0000 7f8e890fb640 1 -- 172.21.15.77:0/57046787 wait complete. 2024-09-15T22:12:20.580 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:12:20.580 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-mon[35109]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:12:20.580 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:20.332+0000 7f79dadee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:12:20.580 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:20.394+0000 7f79dadee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:12:20.580 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:20.578+0000 7f79dadee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:12:20.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:12:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-mon[30980]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:12:20.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:20.420+0000 7fda25ce9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:12:20.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:20.482+0000 7fda25ce9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:12:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/728206175' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:12:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-mon[35021]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:12:20.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:20.331+0000 7fd5dbb7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:12:20.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:20.391+0000 7fd5dbb7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:12:20.613 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:20.575+0000 7fd5dbb7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:12:21.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:20.673+0000 7fda25ce9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:12:21.254 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T22:12:21.254 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:12:21.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:12:21.255 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:12:21.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:21.255 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-09-15T22:12:21.273 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:12:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:12:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:12:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-15T22:12:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42194sec preferred_lft 42194sec 2024-09-15T22:12:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link noprefixroute 2024-09-15T22:12:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:12:21.275 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:12:21.275 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:12:21.275 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:12:21.275 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:12:21.275 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-09-15T22:12:21.275 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-09-15T22:12:21.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:12:21.276 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:12:21.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:21.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:21.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:21.416 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:12:21.440 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-09-15T22:12:21.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:21.441 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-09-15T22:12:21.498 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-09-15T22:12:21.499 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:12:21.499 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-15T22:12:21.499 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-15T22:12:21.499 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:12:21.499 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:12:21.499 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:12:21.499 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-15T22:12:21.499 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-15T22:12:21.500 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-15T22:12:21.500 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:12:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:21 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/57046787' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:12:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:21.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:21 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/57046787' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:12:21.603 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:21.463+0000 7f79dadee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:12:21.604 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:21.530+0000 7f79dadee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:12:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:21 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/57046787' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:12:21.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:21.460+0000 7fd5dbb7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:12:21.612 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:21.526+0000 7fd5dbb7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:12:21.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:21.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:21.628 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-15T22:12:21.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:21.686 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-09-15T22:12:21.742 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:12:21.742 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:12:21.742 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T22:12:21.742 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-15T22:12:21.742 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:12:21.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:21.841 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:21.589+0000 7fda25ce9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:12:21.841 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:21.660+0000 7fda25ce9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:12:21.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:21.861 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-15T22:12:21.861 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:12:21.861 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:12:21.861 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-15T22:12:21.861 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:12:21.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-15T22:12:21.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-15T22:12:21.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-15T22:12:21.862 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:12:21.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:21.979 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:12:21.979 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-09-15T22:12:21.979 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-09-15T22:12:21.979 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:12:21.980 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-09-15T22:12:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:22.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:22.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:12:22.058 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:22.073 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:12:22.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:12:22.074 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:22.129 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-15T22:12:22.130 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-15T22:12:22.130 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c85cddab-bddb-43c6-8f43-ac1d628fc105 2024-09-15T22:12:22.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:22.130 DEBUG:teuthology.orchestra.run.smithi077:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c85cddab-bddb-43c6-8f43-ac1d628fc105 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-15T22:12:22.214 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-15T22:12:22.214 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-15T22:12:22.215 INFO:teuthology.orchestra.run.smithi077.stdout:invalid new device string format 2024-09-15T22:12:22.378 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:invalid new device string format 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: resolved to: "172.21.15.77:3300,172.21.15.117:3300,172.21.15.196:3300" 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.77:3300,172.21.15.117:3300,172.21.15.196:3300:/volumes/qa/sv_1/c85cddab-bddb-43c6-8f43-ac1d628fc105 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=0ddb746c-73ae-11ef-bceb-c7b262605968" will pass to kernel 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: You just mounted a file system that supports labels which does not 2024-09-15T22:12:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T22:12:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T22:12:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T22:12:22.382 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T22:12:22.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:12:22.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:22.410 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:12:22.495 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.232+0000 7fd5dbb7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:12:22.495 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.347+0000 7fd5dbb7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:12:22.499 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.237+0000 7f79dadee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:12:22.500 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.354+0000 7f79dadee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:12:22.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:12:22.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:22.508 INFO:teuthology.orchestra.run.smithi077.stdout:client24863 2024-09-15T22:12:22.509 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:12:22.509 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:12:22.526 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:12:22.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:12:22.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:22.623 INFO:teuthology.orchestra.run.smithi077.stdout:client24863 2024-09-15T22:12:22.624 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:12:22.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:12:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:22.681 INFO:teuthology.orchestra.run.smithi077.stdout:client24863 2024-09-15T22:12:22.681 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968.client24863/status') 2024-09-15T22:12:22.720 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:22.444+0000 7fda25ce9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:12:22.721 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:22.567+0000 7fda25ce9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:12:22.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:12:22.740 INFO:teuthology.orchestra.run.smithi077.stdout:instance: client.24863 (3)172.21.15.77:0/1819496512 2024-09-15T22:12:22.740 INFO:teuthology.orchestra.run.smithi077.stdout:blocklisted: false 2024-09-15T22:12:22.740 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records in 2024-09-15T22:12:22.741 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records out 2024-09-15T22:12:22.741 INFO:teuthology.orchestra.run.smithi077.stderr:71 bytes copied, 2.257e-05 s, 3.1 MB/s 2024-09-15T22:12:22.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:12:22.748 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:12:22.760 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:12:22.760 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:12:22.761 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-15T22:12:22.772 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.493+0000 7fd5dbb7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:22.773 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.554+0000 7fd5dbb7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:12:22.773 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.703+0000 7fd5dbb7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:12:22.773 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.771+0000 7fd5dbb7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:12:22.780 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.498+0000 7f79dadee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:22.780 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.559+0000 7f79dadee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:12:22.780 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.709+0000 7f79dadee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:12:22.780 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.778+0000 7f79dadee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- 172.21.15.77:0/241035675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 msgr2=0x7fb588103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/241035675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588103320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb584009900 tx=0x7fb58402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- 172.21.15.77:0/241035675 shutdown_connections 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/241035675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588103860 0x7fb58810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/241035675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588103320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/241035675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5880ffe30 0x7fb588100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:22.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- 172.21.15.77:0/241035675 >> 172.21.15.77:0/241035675 conn(0x7fb5880fb930 msgr2=0x7fb5880fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- 172.21.15.77:0/241035675 shutdown_connections 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- 172.21.15.77:0/241035675 wait complete. 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 Processor -- start 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.891+0000 7fb58f8e6640 1 -- start start 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880ffe30 0x7fb588197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb588103860 0x7fb58819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:22.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810d7c0 con 0x7fb588102ea0 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810d640 con 0x7fb588103860 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810d940 con 0x7fb5880ffe30 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49104/0 (socket says 172.21.15.77:49104) 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 -- 172.21.15.77:0/2198360277 learned_addr learned my addr 172.21.15.77:0/2198360277 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58d682640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880ffe30 0x7fb588197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58de83640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb588103860 0x7fb58819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880ffe30 msgr2=0x7fb588197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880ffe30 0x7fb588197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb588103860 msgr2=0x7fb58819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb588103860 0x7fb58819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:22.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 -- 172.21.15.77:0/2198360277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58819fa30 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58ce81640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb584031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58403f030 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584003ad0 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58819fcc0 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584035070 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.892+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a01b0 con 0x7fb588102ea0 2024-09-15T22:12:22.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.894+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb588108ff0 con 0x7fb588102ea0 2024-09-15T22:12:22.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.895+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fb584042030 con 0x7fb588102ea0 2024-09-15T22:12:22.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.895+0000 7fb57a7fc640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 0x7fb5740a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:22.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.896+0000 7fb58d682640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 msgr2=0x7fb5740a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:22.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.896+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb5840f01e0 con 0x7fb588102ea0 2024-09-15T22:12:22.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.896+0000 7fb58d682640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 0x7fb5740a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:12:22.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:22.900+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5840bc550 con 0x7fb588102ea0 2024-09-15T22:12:23.015 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:22.719+0000 7fda25ce9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:12:23.016 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:22.783+0000 7fda25ce9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:12:23.016 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:22.943+0000 7fda25ce9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:12:23.016 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.014+0000 7fda25ce9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:12:23.054 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.830+0000 7fd5dbb7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:12:23.054 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:22.988+0000 7fd5dbb7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:12:23.055 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:23.052+0000 7fd5dbb7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:12:23.063 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.838+0000 7f79dadee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:12:23.063 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:22.997+0000 7f79dadee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:12:23.064 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:23.062+0000 7f79dadee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:12:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.097+0000 7fb58d682640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 msgr2=0x7fb5740a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.097+0000 7fb58d682640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 0x7fb5740a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:12:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.110+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb588062df0 con 0x7fb588102ea0 2024-09-15T22:12:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.162+0000 7fb57a7fc640 1 -- 172.21.15.77:0/2198360277 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7fb5840c0850 con 0x7fb588102ea0 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 msgr2=0x7fb5740a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 0x7fb5740a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 msgr2=0x7fb588198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb584031b60 comp rx=0 tx=0).stop 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 shutdown_connections 2024-09-15T22:12:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fb5740a3340 0x7fb5740a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb588103860 0x7fb58819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb588102ea0 0x7fb588198350 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 --2- 172.21.15.77:0/2198360277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880ffe30 0x7fb588197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 >> 172.21.15.77:0/2198360277 conn(0x7fb5880fb930 msgr2=0x7fb5881061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 shutdown_connections 2024-09-15T22:12:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.164+0000 7fb58f8e6640 1 -- 172.21.15.77:0/2198360277 wait complete. 2024-09-15T22:12:23.178 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2737998786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 msgr2=0x7fbcd81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2737998786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd81031d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009900 tx=0x7fbcc802a410 comp rx=0 tx=0).stop 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2737998786 shutdown_connections 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2737998786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8103710 0x7fbcd8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2737998786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd81031d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2737998786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8101b60 0x7fbcd8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2737998786 >> 172.21.15.77:0/2737998786 conn(0x7fbcd80fd2e0 msgr2=0x7fbcd80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2737998786 shutdown_connections 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.281+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2737998786 wait complete. 2024-09-15T22:12:23.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 Processor -- start 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 -- start start 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8101b60 0x7fbcd8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8103710 0x7fbcd8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810d830 con 0x7fbcd8102d50 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810d6b0 con 0x7fbcd8101b60 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdf65e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810d9b0 con 0x7fbcd8103710 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49130/0 (socket says 172.21.15.77:49130) 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 -- 172.21.15.77:0/2811242100 learned_addr learned my addr 172.21.15.77:0/2811242100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8101b60 0x7fbcd8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8103710 msgr2=0x7fbcd8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:23.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcd8103710 0x7fbcd8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8101b60 msgr2=0x7fbcd8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd8101b60 0x7fbcd8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.282+0000 7fbcdcbf9640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd80753b0 con 0x7fbcd8102d50 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcdcbf9640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd80745e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8031440 tx=0x7fbcc8032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803f030 con 0x7fbcd8102d50 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc8003ce0 con 0x7fbcd8102d50 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8035070 con 0x7fbcd8102d50 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd81a1ee0 con 0x7fbcd8102d50 2024-09-15T22:12:23.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.283+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81a2340 con 0x7fbcd8102d50 2024-09-15T22:12:23.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.284+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd8108ea0 con 0x7fbcd8102d50 2024-09-15T22:12:23.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.285+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fbcc8042030 con 0x7fbcd8102d50 2024-09-15T22:12:23.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.286+0000 7fbcce7fc640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 0x7fbcc40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:23.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.286+0000 7fbcdd3fa640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 msgr2=0x7fbcc40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:23.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.286+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fbcc80f0360 con 0x7fbcd8102d50 2024-09-15T22:12:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.286+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 0x7fbcc40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:12:23.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.290+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcc80bccc0 con 0x7fbcd8102d50 2024-09-15T22:12:23.313 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.076+0000 7fda25ce9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:12:23.314 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.242+0000 7fda25ce9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:12:23.314 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.312+0000 7fda25ce9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:12:23.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:12:23.323 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:12:23.323 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:23.119+0000 7fd5dbb7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:12:23.323 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:23.203+0000 7fd5dbb7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:23.324 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:23.262+0000 7fd5dbb7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:12:23.324 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:23.321+0000 7fd5dbb7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:12:23.336 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:12:23.337 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:23.129+0000 7f79dadee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:12:23.337 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:23.214+0000 7f79dadee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:23.337 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:23.274+0000 7f79dadee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:12:23.337 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:23.334+0000 7f79dadee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:12:23.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.487+0000 7fbcdd3fa640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 msgr2=0x7fbcc40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:23.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.487+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 0x7fbcc40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:12:23.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.489+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbcd8062790 con 0x7fbcc40a3340 2024-09-15T22:12:23.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.382+0000 7fda25ce9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:12:23.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.468+0000 7fda25ce9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:12:23.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.528+0000 7fda25ce9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:12:23.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.888+0000 7fbcdd3fa640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 msgr2=0x7fbcc40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1809193257 2024-09-15T22:12:23.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:23.888+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 0x7fbcc40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:12:24.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:23.586+0000 7fda25ce9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:12:24.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:24.182+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 20) ==== 138372+0+0 (secure 0 0 0) 0x7fbcc80b8c30 con 0x7fbcd8102d50 2024-09-15T22:12:24.362 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y[36392]: 2024-09-15T22:12:24.106+0000 7fd5dbb7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:12:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: Standby manager daemon y restarted 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: Standby manager daemon y started 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: Standby manager daemon z restarted 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: Standby manager daemon z started 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:12:24.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:24 smithi117 ceph-mon[35021]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:24.395+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fbcc80edb20 con 0x7fbcd8102d50 2024-09-15T22:12:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:24.396+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 msgr2=0x7fbcc40a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:12:24.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:24.396+0000 7fbcce7fc640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/1809193257,v1:172.21.15.77:6801/1809193257] conn(0x7fbcc40a3340 0x7fbcc40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.383+0000 7fda25ce9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: Standby manager daemon y restarted 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: Standby manager daemon y started 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:12:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: Standby manager daemon z restarted 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: Standby manager daemon z started 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:12:24.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-mon[30980]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:24.603 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z[36797]: 2024-09-15T22:12:24.125+0000 7f79dadee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2198360277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: Standby manager daemon y restarted 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: Standby manager daemon y started 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.117:0/3599780153' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: Standby manager daemon z restarted 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: Standby manager daemon z started 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:12:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:12:24.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:12:24.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:24 smithi196 ceph-mon[35109]: from='mgr.? 172.21.15.196:0/466672612' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.636+0000 7fd98be38640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.636+0000 7fd98be38640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.636+0000 7fd98be38640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.636+0000 7fd98be38640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.636+0000 7fd98be38640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.082 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.637+0000 7fd98ee3e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.083 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.637+0000 7fd98ee3e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.083 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.637+0000 7fd98ee3e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.083 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.637+0000 7fd98ee3e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.083 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:24.637+0000 7fd98ee3e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.399+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mon.0 v2:172.21.15.77:3300/0 9 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fbcc8097a10 con 0x7fbcd8102d50 2024-09-15T22:12:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.400+0000 7fbcce7fc640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbcc4031da0 0x7fbcc4032180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.400+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbcc8003b30 con 0x7fbcc4031da0 2024-09-15T22:12:25.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.402+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbcc4031da0 0x7fbcc4032180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.407+0000 7fbcdd3fa640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbcc4031da0 0x7fbcc4032180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005fd0 tx=0x7fbcd400b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.423+0000 7fbcce7fc640 1 -- 172.21.15.77:0/2811242100 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fbcc8003b30 con 0x7fbcc4031da0 2024-09-15T22:12:25.424 INFO:teuthology.orchestra.run.smithi077.stdout:{} 2024-09-15T22:12:25.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbcc4031da0 msgr2=0x7fbcc4032180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbcc4031da0 0x7fbcc4032180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005fd0 tx=0x7fbcd400b390 comp rx=0 tx=0).stop 2024-09-15T22:12:25.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 msgr2=0x7fbcd80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 --2- 172.21.15.77:0/2811242100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbcd8102d50 0x7fbcd80745e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8031440 tx=0x7fbcc8032e20 comp rx=0 tx=0).stop 2024-09-15T22:12:25.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcddbfb640 1 -- 172.21.15.77:0/2811242100 reap_dead start 2024-09-15T22:12:25.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 shutdown_connections 2024-09-15T22:12:25.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 >> 172.21.15.77:0/2811242100 conn(0x7fbcd80fd2e0 msgr2=0x7fbcd8106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 shutdown_connections 2024-09-15T22:12:25.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.425+0000 7fbcdf65e640 1 -- 172.21.15.77:0/2811242100 wait complete. 2024-09-15T22:12:25.436 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.414+0000 7fd98a5f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.436 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.414+0000 7fd98a5f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.414+0000 7fd98a5f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.414+0000 7fd98a5f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.414+0000 7fd98a5f5640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: Active manager daemon x restarted 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: Activating manager daemon x 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: mgrmap e21: x(active, starting, since 0.0118197s), standbys: z, y 2024-09-15T22:12:25.437 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:12:25.438 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:12:25.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:12:25.440 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-15T22:12:25.451 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-15T22:12:25.461 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:12:25.461 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: Manager daemon x is now available 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:12:25.462 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:12:25.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:12:25.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: Active manager daemon x restarted 2024-09-15T22:12:25.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: Activating manager daemon x 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: mgrmap e21: x(active, starting, since 0.0118197s), standbys: z, y 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:12:25.537 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:12:25.538 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:12:25.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:12:25.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:12:25.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: Manager daemon x is now available 2024-09-15T22:12:25.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:25.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:12:25.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:25 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:12:25.557 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: Active manager daemon x restarted 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: Activating manager daemon x 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: mgrmap e21: x(active, starting, since 0.0118197s), standbys: z, y 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:12:25.558 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:12:25.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:12:25.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: Manager daemon x is now available 2024-09-15T22:12:25.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:25.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:12:25.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:25 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:12:25.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 -- 172.21.15.77:0/2441376774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0102590 msgr2=0x7f00b0104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 --2- 172.21.15.77:0/2441376774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0102590 0x7f00b0104980 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009900 tx=0x7f00a002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 -- 172.21.15.77:0/2441376774 shutdown_connections 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 --2- 172.21.15.77:0/2441376774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b01077f0 0x7f00b0109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 --2- 172.21.15.77:0/2441376774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b0104ec0 0x7f00b01072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 --2- 172.21.15.77:0/2441376774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0102590 0x7f00b0104980 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 -- 172.21.15.77:0/2441376774 >> 172.21.15.77:0/2441376774 conn(0x7f00b00fc170 msgr2=0x7f00b00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 -- 172.21.15.77:0/2441376774 shutdown_connections 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.564+0000 7f00b8131640 1 -- 172.21.15.77:0/2441376774 wait complete. 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 Processor -- start 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 -- start start 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b0102590 0x7f00b0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 0x7f00b019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00b010d850 con 0x7f00b0104ec0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00b010d6d0 con 0x7f00b01077f0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b8131640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00b010d9d0 con 0x7f00b0102590 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b56cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b56cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49286/0 (socket says 172.21.15.77:49286) 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b56cc640 1 -- 172.21.15.77:0/82767087 learned_addr learned my addr 172.21.15.77:0/82767087 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 0x7f00b019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b5ecd640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b0102590 0x7f00b0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 -- 172.21.15.77:0/82767087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b0102590 msgr2=0x7f00b0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b0102590 0x7f00b0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 -- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 msgr2=0x7f00b01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b66ce640 1 -- 172.21.15.77:0/82767087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b019faa0 con 0x7f00b01077f0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.565+0000 7f00b56cc640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00b66ce640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 0x7f00b019f430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00a8005fd0 tx=0x7f00a800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a800cdf0 con 0x7f00b01077f0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b019fd90 con 0x7f00b01077f0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a8005020 con 0x7f00b01077f0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8010690 con 0x7f00b01077f0 2024-09-15T22:12:25.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.566+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b01a0250 con 0x7f00b01077f0 2024-09-15T22:12:25.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.567+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00fdc60 con 0x7f00b01077f0 2024-09-15T22:12:25.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.568+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f00a80054d0 con 0x7f00b01077f0 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.568+0000 7f00a6ffd640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 0x7f009c0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.569+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f00a80d0430 con 0x7f00b01077f0 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.569+0000 7f00b5ecd640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 0x7f009c0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.569+0000 7f00b5ecd640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 0x7f009c0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f00a0006010 tx=0x7f00a0000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.572+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00a809cfa0 con 0x7f00b01077f0 2024-09-15T22:12:25.732 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.656+0000 7fd96f500640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:12:25] ENGINE Error in HTTPServer.serve 2024-09-15T22:12:25.732 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-15T22:12:25.732 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._connections.run(self.expiration_interval) 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._run(expiration_interval) 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: s = self.context.wrap_socket( 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: return self.sslsocket_class._create( 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:12:25.733 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.do_handshake() 2024-09-15T22:12:25.734 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:12:25.734 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self._sslobj.do_handshake() 2024-09-15T22:12:25.734 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:12:25.734 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:12:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:12:25.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.745+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00b0062e90 con 0x7f00b01077f0 2024-09-15T22:12:25.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.747+0000 7f00a6ffd640 1 -- 172.21.15.77:0/82767087 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25152 (secure 0 0 0) 0x7f00a809c8a0 con 0x7f00b01077f0 2024-09-15T22:12:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:25.749 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":91,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:12:24.385821+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/999829703":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/3279424814":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3785009540":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1809193257":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/1437248041":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1809193257":"2024-09-16T22:12:24.385790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:12:25.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 msgr2=0x7f009c0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 0x7f009c0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f00a0006010 tx=0x7f00a0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 msgr2=0x7f00b019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 0x7f00b019f430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00a8005fd0 tx=0x7f00a800ef10 comp rx=0 tx=0).stop 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 shutdown_connections 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f009c0a3050 0x7f009c0a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00b01077f0 0x7f00b019f430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00b0104ec0 0x7f00b01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 --2- 172.21.15.77:0/82767087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00b0102590 0x7f00b0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 >> 172.21.15.77:0/82767087 conn(0x7f00b00fc170 msgr2=0x7f00b0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 shutdown_connections 2024-09-15T22:12:25.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.750+0000 7f00b8131640 1 -- 172.21.15.77:0/82767087 wait complete. 2024-09-15T22:12:25.763 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:05:16.403426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:09:51.900322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:09:54.068598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:10:00.447354+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:12:25.764 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:12:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 -- 172.21.15.77:0/1864339702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 msgr2=0x7fdcec104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 --2- 172.21.15.77:0/1864339702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec104a20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ca0 tx=0x7fdcdc02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 -- 172.21.15.77:0/1864339702 shutdown_connections 2024-09-15T22:12:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 --2- 172.21.15.77:0/1864339702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 --2- 172.21.15.77:0/1864339702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 0x7fdcec107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 --2- 172.21.15.77:0/1864339702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec104a20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 -- 172.21.15.77:0/1864339702 >> 172.21.15.77:0/1864339702 conn(0x7fdcec0fc1b0 msgr2=0x7fdcec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 -- 172.21.15.77:0/1864339702 shutdown_connections 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 -- 172.21.15.77:0/1864339702 wait complete. 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.869+0000 7fdcf1817640 1 Processor -- start 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 -- start start 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 0x7fdcec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d7d0 con 0x7fdcec102630 2024-09-15T22:12:25.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d650 con 0x7fdcec107890 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf1817640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d950 con 0x7fdcec104f60 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49300/0 (socket says 172.21.15.77:49300) 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 -- 172.21.15.77:0/372857918 learned_addr learned my addr 172.21.15.77:0/372857918 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcea7fc640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 0x7fdcec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceb7fe640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 -- 172.21.15.77:0/372857918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 msgr2=0x7fdcec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 0x7fdcec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 -- 172.21.15.77:0/372857918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 msgr2=0x7fdcec19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 -- 172.21.15.77:0/372857918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec19fb30 con 0x7fdcec102630 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceb7fe640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdceaffd640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc031a40 tx=0x7fdcdc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.870+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02fa20 con 0x7fdcec102630 2024-09-15T22:12:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.871+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc02fbc0 con 0x7fdcec102630 2024-09-15T22:12:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.871+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec19fdc0 con 0x7fdcec102630 2024-09-15T22:12:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.871+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc03a780 con 0x7fdcec102630 2024-09-15T22:12:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.871+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a0230 con 0x7fdcec102630 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.871+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec0fdca0 con 0x7fdcec102630 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.873+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fdcdc03a920 con 0x7fdcec102630 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.873+0000 7fdcf0815640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 0x7fdcd40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.873+0000 7fdcea7fc640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 0x7fdcd40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.873+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdcdc0f0f50 con 0x7fdcec102630 2024-09-15T22:12:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.874+0000 7fdcea7fc640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 0x7fdcd40a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcec199420 tx=0x7fdce0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:25.877+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcdc0bdb00 con 0x7fdcec102630 2024-09-15T22:12:26.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.047+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcec062f30 con 0x7fdcec102630 2024-09-15T22:12:26.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.047+0000 7fdcf0815640 1 -- 172.21.15.77:0/372857918 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fdcdc0bd400 con 0x7fdcec102630 2024-09-15T22:12:26.049 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 msgr2=0x7fdcd40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 0x7fdcd40a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcec199420 tx=0x7fdce0005f90 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 msgr2=0x7fdcec197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc031a40 tx=0x7fdcdc031ee0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 shutdown_connections 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a3050 0x7fdcd40a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcec107890 0x7fdcec19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcec104f60 0x7fdcec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 --2- 172.21.15.77:0/372857918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcec102630 0x7fdcec197f10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 >> 172.21.15.77:0/372857918 conn(0x7fdcec0fc1b0 msgr2=0x7fdcec103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 shutdown_connections 2024-09-15T22:12:26.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.049+0000 7fdcf1817640 1 -- 172.21.15.77:0/372857918 wait complete. 2024-09-15T22:12:26.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:12:26.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/1586094491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b4101bf0 msgr2=0x7ff9b4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/1586094491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b4101bf0 0x7ff9b4102070 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009900 tx=0x7ff9a402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/1586094491 shutdown_connections 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/1586094491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b41025b0 0x7ff9b4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/1586094491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b4101bf0 0x7ff9b4102070 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/1586094491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4100a00 0x7ff9b4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/1586094491 >> 172.21.15.77:0/1586094491 conn(0x7ff9b40fc140 msgr2=0x7ff9b40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/1586094491 shutdown_connections 2024-09-15T22:12:26.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/1586094491 wait complete. 2024-09-15T22:12:26.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 Processor -- start 2024-09-15T22:12:26.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.177+0000 7ff9bc1a1640 1 -- start start 2024-09-15T22:12:26.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 0x7ff9b419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410c680 con 0x7ff9b41025b0 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410c500 con 0x7ff9b4100a00 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9bc1a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b410c800 con 0x7ff9b4101bf0 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9ba73e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 0x7ff9b419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b9f3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45772/0 (socket says 172.21.15.77:45772) 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b9f3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37742/0 (socket says 172.21.15.77:37742) 2024-09-15T22:12:26.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9ba73e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 0x7ff9b419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49316/0 (socket says 172.21.15.77:49316) 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 -- 172.21.15.77:0/698678907 learned_addr learned my addr 172.21.15.77:0/698678907 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 -- 172.21.15.77:0/698678907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 msgr2=0x7ff9b4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 -- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 msgr2=0x7ff9b419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 0x7ff9b419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 -- 172.21.15.77:0/698678907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b419faa0 con 0x7ff9b4101bf0 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b9f3d640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.178+0000 7ff9b973c640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009230 tx=0x7ff9a40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.179+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a403f030 con 0x7ff9b4101bf0 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.179+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b4102e10 con 0x7ff9b4101bf0 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.179+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b4103350 con 0x7ff9b4101bf0 2024-09-15T22:12:26.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.179+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a4003ce0 con 0x7ff9b4101bf0 2024-09-15T22:12:26.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.179+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4035070 con 0x7ff9b4101bf0 2024-09-15T22:12:26.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.180+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b4103670 con 0x7ff9b4101bf0 2024-09-15T22:12:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.182+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7ff9a4042030 con 0x7ff9b4101bf0 2024-09-15T22:12:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.182+0000 7ff9aaffd640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 0x7ff9a00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.183+0000 7ff9b9f3d640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 0x7ff9a00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.183+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff9a40efe00 con 0x7ff9b4101bf0 2024-09-15T22:12:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.183+0000 7ff9b9f3d640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 0x7ff9a00a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4101a50 tx=0x7ff9b0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.187+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9a40bc360 con 0x7ff9b4101bf0 2024-09-15T22:12:26.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.400+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9b4103a20 con 0x7ff9b4101bf0 2024-09-15T22:12:26.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.401+0000 7ff9aaffd640 1 -- 172.21.15.77:0/698678907 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff9a40c0660 con 0x7ff9b4101bf0 2024-09-15T22:12:26.402 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-15T22:12:26.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 msgr2=0x7ff9a00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 0x7ff9a00a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4101a50 tx=0x7ff9b0005f90 comp rx=0 tx=0).stop 2024-09-15T22:12:26.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 msgr2=0x7ff9b41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009230 tx=0x7ff9a40042f0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 shutdown_connections 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9a00a3050 0x7ff9a00a5510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.403+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9b41025b0 0x7ff9b419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.404+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b4101bf0 0x7ff9b41983c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.404+0000 7ff9bc1a1640 1 --2- 172.21.15.77:0/698678907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9b4100a00 0x7ff9b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.404+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 >> 172.21.15.77:0/698678907 conn(0x7ff9b40fc140 msgr2=0x7ff9b40693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.404+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 shutdown_connections 2024-09-15T22:12:26.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.404+0000 7ff9bc1a1640 1 -- 172.21.15.77:0/698678907 wait complete. 2024-09-15T22:12:26.419 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:12:26.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- 172.21.15.77:0/449699878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c104f50 msgr2=0x7f9c1c107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 --2- 172.21.15.77:0/449699878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c104f50 0x7f9c1c107340 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009900 tx=0x7f9c1002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- 172.21.15.77:0/449699878 shutdown_connections 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 --2- 172.21.15.77:0/449699878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c107880 0x7f9c1c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 --2- 172.21.15.77:0/449699878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c104f50 0x7f9c1c107340 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 --2- 172.21.15.77:0/449699878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c102620 0x7f9c1c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- 172.21.15.77:0/449699878 >> 172.21.15.77:0/449699878 conn(0x7f9c1c0fc140 msgr2=0x7f9c1c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- 172.21.15.77:0/449699878 shutdown_connections 2024-09-15T22:12:26.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- 172.21.15.77:0/449699878 wait complete. 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 Processor -- start 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.530+0000 7f9c2383c640 1 -- start start 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c104f50 0x7f9c1c1005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c107880 0x7f9c1c100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d770 con 0x7f9c1c102620 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d5f0 con 0x7f9c1c107880 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c2383c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10d8f0 con 0x7f9c1c104f50 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49340/0 (socket says 172.21.15.77:49340) 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 -- 172.21.15.77:0/1971432688 learned_addr learned my addr 172.21.15.77:0/1971432688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c20dd7640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c104f50 0x7f9c1c1005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c21dd9640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c107880 0x7f9c1c100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 -- 172.21.15.77:0/1971432688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c104f50 msgr2=0x7f9c1c1005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c104f50 0x7f9c1c1005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 -- 172.21.15.77:0/1971432688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c107880 msgr2=0x7f9c1c100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c107880 0x7f9c1c100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 -- 172.21.15.77:0/1971432688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c1013a0 con 0x7f9c1c102620 2024-09-15T22:12:26.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c215d8640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9f0 tx=0x7f9c1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400cd80 con 0x7f9c1c102620 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.531+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c14010910 con 0x7f9c1c102620 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.532+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c1a1fa0 con 0x7f9c1c102620 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.532+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14018990 con 0x7f9c1c102620 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.532+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1a2400 con 0x7f9c1c102620 2024-09-15T22:12:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.532+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c0fdc30 con 0x7f9c1c102620 2024-09-15T22:12:26.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.534+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f9c14010ac0 con 0x7f9c1c102620 2024-09-15T22:12:26.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.534+0000 7f9c1a7fc640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 0x7f9c040a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.535+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9c140d0040 con 0x7f9c1c102620 2024-09-15T22:12:26.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.535+0000 7f9c20dd7640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 0x7f9c040a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.535+0000 7f9c20dd7640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 0x7f9c040a5510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005ab0 tx=0x7f9c10000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.542+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c1409c5a0 con 0x7f9c1c102620 2024-09-15T22:12:26.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.714+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9c1c02da10 con 0x7f9c1c102620 2024-09-15T22:12:26.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.715+0000 7f9c1a7fc640 1 -- 172.21.15.77:0/1971432688 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f9c140a08a0 con 0x7f9c1c102620 2024-09-15T22:12:26.717 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 msgr2=0x7f9c040a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 0x7f9c040a5510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005ab0 tx=0x7f9c10000bc0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 msgr2=0x7f9c1c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9f0 tx=0x7f9c1400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 shutdown_connections 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9c040a3050 0x7f9c040a5510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c1c107880 0x7f9c1c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c1c104f50 0x7f9c1c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 --2- 172.21.15.77:0/1971432688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c1c102620 0x7f9c1c101f90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.717+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 >> 172.21.15.77:0/1971432688 conn(0x7f9c1c0fc140 msgr2=0x7f9c1c102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.718+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 shutdown_connections 2024-09-15T22:12:26.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.718+0000 7f9c2383c640 1 -- 172.21.15.77:0/1971432688 wait complete. 2024-09-15T22:12:26.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: mgrmap e22: x(active, since 1.01938s), standbys: z, y 2024-09-15T22:12:26.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: [15/Sep/2024:22:12:25] ENGINE Bus STARTING 2024-09-15T22:12:26.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: [15/Sep/2024:22:12:25] ENGINE Serving on http://172.21.15.77:8765 2024-09-15T22:12:26.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: [15/Sep/2024:22:12:25] ENGINE Serving on https://172.21.15.77:7150 2024-09-15T22:12:26.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: [15/Sep/2024:22:12:25] ENGINE Bus STARTED 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: [15/Sep/2024:22:12:25] ENGINE Error in HTTPServer.serve 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: Traceback (most recent call last): 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._run(expiration_interval) 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: s = self.context.wrap_socket( 2024-09-15T22:12:26.730 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: return self.sslsocket_class._create( 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: self.do_handshake() 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: self._sslobj.do_handshake() 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/82767087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/372857918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:26.731 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:26 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:26.732 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 -- 172.21.15.77:0/4050229815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0100a10 msgr2=0x7fd3d0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/4050229815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0100a10 0x7fd3d0100e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009900 tx=0x7fd3c402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 -- 172.21.15.77:0/4050229815 shutdown_connections 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/4050229815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 0x7fd3d0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/4050229815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0101c00 0x7fd3d0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/4050229815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0100a10 0x7fd3d0100e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 -- 172.21.15.77:0/4050229815 >> 172.21.15.77:0/4050229815 conn(0x7fd3d00fc190 msgr2=0x7fd3d00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:26.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 -- 172.21.15.77:0/4050229815 shutdown_connections 2024-09-15T22:12:26.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.838+0000 7fd3d6b59640 1 -- 172.21.15.77:0/4050229815 wait complete. 2024-09-15T22:12:26.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 Processor -- start 2024-09-15T22:12:26.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 -- start start 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0100a10 0x7fd3d0196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 0x7fd3d019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010c690 con 0x7fd3d0101c00 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010c510 con 0x7fd3d01025c0 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d6b59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010c810 con 0x7fd3d0100a10 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49366/0 (socket says 172.21.15.77:49366) 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d48f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0100a10 0x7fd3d0196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 -- 172.21.15.77:0/81105233 learned_addr learned my addr 172.21.15.77:0/81105233 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3d50f6640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 0x7fd3d019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 -- 172.21.15.77:0/81105233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0100a10 msgr2=0x7fd3d0196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0100a10 0x7fd3d0196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 -- 172.21.15.77:0/81105233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 msgr2=0x7fd3d019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.839+0000 7fd3cffff640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 0x7fd3d019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3cffff640 1 -- 172.21.15.77:0/81105233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d019e9d0 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3cffff640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00d770 tx=0x7fd3bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc017070 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3bc005500 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d019ecc0 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc004e00 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.840+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d019f190 con 0x7fd3d0101c00 2024-09-15T22:12:26.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.841+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0107ef0 con 0x7fd3d0101c00 2024-09-15T22:12:26.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.842+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fd3bc004fa0 con 0x7fd3d0101c00 2024-09-15T22:12:26.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.843+0000 7fd3cdffb640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 0x7fd3c00a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:26.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.843+0000 7fd3d48f5640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 0x7fd3c00a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:26.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.844+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd3bc0cf3d0 con 0x7fd3d0101c00 2024-09-15T22:12:26.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.844+0000 7fd3d48f5640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 0x7fd3c00a5510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4005ab0 tx=0x7fd3c4000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:26.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:26.847+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3bc09b930 con 0x7fd3d0101c00 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: mgrmap e22: x(active, since 1.01938s), standbys: z, y 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: [15/Sep/2024:22:12:25] ENGINE Bus STARTING 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: [15/Sep/2024:22:12:25] ENGINE Serving on http://172.21.15.77:8765 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: [15/Sep/2024:22:12:25] ENGINE Serving on https://172.21.15.77:7150 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: [15/Sep/2024:22:12:25] ENGINE Bus STARTED 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: [15/Sep/2024:22:12:25] ENGINE Error in HTTPServer.serve 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: Traceback (most recent call last): 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:12:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._run(expiration_interval) 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: s = self.context.wrap_socket( 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: return self.sslsocket_class._create( 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: self.do_handshake() 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: self._sslobj.do_handshake() 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/82767087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:26.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/372857918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:12:26.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:26.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:26 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: mgrmap e22: x(active, since 1.01938s), standbys: z, y 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: [15/Sep/2024:22:12:25] ENGINE Bus STARTING 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: [15/Sep/2024:22:12:25] ENGINE Serving on http://172.21.15.77:8765 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: [15/Sep/2024:22:12:25] ENGINE Serving on https://172.21.15.77:7150 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: [15/Sep/2024:22:12:25] ENGINE Bus STARTED 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: [15/Sep/2024:22:12:25] ENGINE Error in HTTPServer.serve 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout: Traceback (most recent call last): 2024-09-15T22:12:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._run(expiration_interval) 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: s = self.context.wrap_socket( 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: return self.sslsocket_class._create( 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: self.do_handshake() 2024-09-15T22:12:26.863 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._sslobj.do_handshake() 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/82767087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/372857918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:26.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:26 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.023+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd3d0062df0 con 0x7fd3d0101c00 2024-09-15T22:12:27.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.023+0000 7fd3cdffb640 1 -- 172.21.15.77:0/81105233 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fd3bc09fc30 con 0x7fd3d0101c00 2024-09-15T22:12:27.025 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-15T22:12:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 msgr2=0x7fd3c00a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 0x7fd3c00a5510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4005ab0 tx=0x7fd3c4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 msgr2=0x7fd3d0197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00d770 tx=0x7fd3bc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:12:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 shutdown_connections 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3c00a3050 0x7fd3c00a5510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d01025c0 0x7fd3d019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3d0101c00 0x7fd3d0197300 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 --2- 172.21.15.77:0/81105233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3d0100a10 0x7fd3d0196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 >> 172.21.15.77:0/81105233 conn(0x7fd3d00fc190 msgr2=0x7fd3d01050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 shutdown_connections 2024-09-15T22:12:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.026+0000 7fd3d6b59640 1 -- 172.21.15.77:0/81105233 wait complete. 2024-09-15T22:12:27.038 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-15T22:12:27.039 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:27.315 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/698678907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1971432688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/81105233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:12:27.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:12:27.770 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:12:27.770 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:27 smithi077 ceph-mon[30980]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/698678907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1971432688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/81105233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:12:27.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:27 smithi196 ceph-mon[35109]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-15T22:12:27.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/698678907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1971432688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/81105233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:12:27.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Updating smithi077:/etc/ceph/ceph.conf 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-15T22:12:27.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:27 smithi117 ceph-mon[35021]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 -- 172.21.15.77:0/4031052185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140101de0 msgr2=0x7f81401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 --2- 172.21.15.77:0/4031052185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140101de0 0x7f81401021c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8130009900 tx=0x7f813002a410 comp rx=0 tx=0).stop 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 -- 172.21.15.77:0/4031052185 shutdown_connections 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 --2- 172.21.15.77:0/4031052185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 --2- 172.21.15.77:0/4031052185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140102790 0x7f81401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 --2- 172.21.15.77:0/4031052185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140101de0 0x7f81401021c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 -- 172.21.15.77:0/4031052185 >> 172.21.15.77:0/4031052185 conn(0x7f81400fd8c0 msgr2=0x7f81400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 -- 172.21.15.77:0/4031052185 shutdown_connections 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.848+0000 7f8145d90640 1 -- 172.21.15.77:0/4031052185 wait complete. 2024-09-15T22:12:27.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 Processor -- start 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 -- start start 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140101de0 0x7f8140198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8140111f70 con 0x7f8140102790 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8140111df0 con 0x7f8140106f10 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f8145d90640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81401120f0 con 0x7f8140101de0 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f813effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f813effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49378/0 (socket says 172.21.15.77:49378) 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.849+0000 7f813effd640 1 -- 172.21.15.77:0/2167877020 learned_addr learned my addr 172.21.15.77:0/2167877020 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:27.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813ffff640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813f7fe640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140101de0 0x7f8140198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 -- 172.21.15.77:0/2167877020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140101de0 msgr2=0x7f8140198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140101de0 0x7f8140198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 -- 172.21.15.77:0/2167877020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 msgr2=0x7f814019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 -- 172.21.15.77:0/2167877020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814019d3c0 con 0x7f8140102790 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813ffff640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813effd640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f813400e9f0 tx=0x7f813400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813400cd80 con 0x7f8140102790 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814019d6b0 con 0x7f8140102790 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8134005500 con 0x7f8140102790 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.850+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134010690 con 0x7f8140102790 2024-09-15T22:12:27.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.851+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a4f90 con 0x7f8140102790 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.852+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81401021c0 con 0x7f8140102790 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.853+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f81340046c0 con 0x7f8140102790 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.853+0000 7f813cff9640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 0x7f81280a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.853+0000 7f813f7fe640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 0x7f81280a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.853+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f81340d05d0 con 0x7f8140102790 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.854+0000 7f813f7fe640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 0x7f81280a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8130009230 tx=0x7f81300058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:27.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:27.858+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f813409cb00 con 0x7f8140102790 2024-09-15T22:12:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.072+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f814010a0f0 con 0x7f8140102790 2024-09-15T22:12:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.073+0000 7f813cff9640 1 -- 172.21.15.77:0/2167877020 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f813409c400 con 0x7f8140102790 2024-09-15T22:12:28.074 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:28.075 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:12:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 msgr2=0x7f81280a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 0x7f81280a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8130009230 tx=0x7f81300058d0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 msgr2=0x7f8140198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f813400e9f0 tx=0x7f813400edf0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 shutdown_connections 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f81280a3440 0x7f81280a5900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140106f10 0x7f814019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8140102790 0x7f8140198950 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 --2- 172.21.15.77:0/2167877020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140101de0 0x7f8140198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 >> 172.21.15.77:0/2167877020 conn(0x7f81400fd8c0 msgr2=0x7f81400fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:28.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 shutdown_connections 2024-09-15T22:12:28.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.075+0000 7f8145d90640 1 -- 172.21.15.77:0/2167877020 wait complete. 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2167877020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:28.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:28 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.833 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-15T22:12:28.833 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-15T22:12:28.833 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2167877020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:28.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:28 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:12:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: Updating smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2167877020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:28.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:28.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:28 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- 172.21.15.77:0/421772285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894106990 msgr2=0x7fa894106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 --2- 172.21.15.77:0/421772285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894106990 0x7fa894106d70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa884009cd0 tx=0x7fa88402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- 172.21.15.77:0/421772285 shutdown_connections 2024-09-15T22:12:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 --2- 172.21.15.77:0/421772285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894101320 0x7fa89410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 --2- 172.21.15.77:0/421772285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 0x7fa894100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 --2- 172.21.15.77:0/421772285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894106990 0x7fa894106d70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- 172.21.15.77:0/421772285 >> 172.21.15.77:0/421772285 conn(0x7fa894077a10 msgr2=0x7fa8940fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- 172.21.15.77:0/421772285 shutdown_connections 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- 172.21.15.77:0/421772285 wait complete. 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 Processor -- start 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.934+0000 7fa899aaf640 1 -- start start 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 0x7fa8941983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 0x7fa89419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa894110dd0 con 0x7fa894101320 2024-09-15T22:12:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa894110c50 con 0x7fa894100980 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa894110f50 con 0x7fa894106990 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49398/0 (socket says 172.21.15.77:49398) 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 -- 172.21.15.77:0/1694686154 learned_addr learned my addr 172.21.15.77:0/1694686154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa8937fe640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 0x7fa8941983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa893fff640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 0x7fa89419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 -- 172.21.15.77:0/1694686154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 msgr2=0x7fa89419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 0x7fa89419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 -- 172.21.15.77:0/1694686154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 msgr2=0x7fa8941983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 0x7fa8941983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 -- 172.21.15.77:0/1694686154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89419d3d0 con 0x7fa894101320 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa893fff640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 0x7fa89419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa892ffd640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa88800e9f0 tx=0x7fa88800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88800cd80 con 0x7fa894101320 2024-09-15T22:12:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89419d6c0 con 0x7fa894101320 2024-09-15T22:12:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa888010910 con 0x7fa894101320 2024-09-15T22:12:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888018990 con 0x7fa894101320 2024-09-15T22:12:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.935+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8941a4f50 con 0x7fa894101320 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.936+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa894100de0 con 0x7fa894101320 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.937+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7fa888010ac0 con 0x7fa894101320 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.938+0000 7fa890ff9640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 0x7fa87c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.938+0000 7fa8937fe640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 0x7fa87c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.938+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa8880d0590 con 0x7fa894101320 2024-09-15T22:12:28.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.939+0000 7fa8937fe640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 0x7fa87c0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa884006010 tx=0x7fa884009360 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:28.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:28.942+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa88809cac0 con 0x7fa894101320 2024-09-15T22:12:29.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.106+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa894069380 con 0x7fa87c0a3450 2024-09-15T22:12:29.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.114+0000 7fa890ff9640 1 -- 172.21.15.77:0/1694686154 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7fa894069380 con 0x7fa87c0a3450 2024-09-15T22:12:29.120 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:29.123 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 msgr2=0x7fa87c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 0x7fa87c0a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa884006010 tx=0x7fa884009360 comp rx=0 tx=0).stop 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 msgr2=0x7fa894198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa88800e9f0 tx=0x7fa88800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 shutdown_connections 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa87c0a3450 0x7fa87c0a5910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa894106990 0x7fa89419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa894101320 0x7fa894198920 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 --2- 172.21.15.77:0/1694686154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa894100980 0x7fa8941983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 >> 172.21.15.77:0/1694686154 conn(0x7fa894077a10 msgr2=0x7fa894104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 shutdown_connections 2024-09-15T22:12:29.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.118+0000 7fa899aaf640 1 -- 172.21.15.77:0/1694686154 wait complete. 2024-09-15T22:12:29.130 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:12:28.402665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193721,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1346,"num_read_kb":844,"num_write":567,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378312,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563512,"statfs":{"total":1151940427776,"available":1151553036288,"internally_reserved":0,"allocated":12689408,"data_stored":9027196,"data_compressed":459456,"data_compressed_allocated":622592,"data_compressed_original":3538944,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988318"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642571+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:24.642571+0000","last_peered":"2024-09-15T22:12:24.642571+0000","last_clean":"2024-09-15T22:12:24.642571+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:24.642571+0000","last_undegraded":"2024-09-15T22:12:24.642571+0000","last_fullsized":"2024-09-15T22:12:24.642571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395034+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395034+0000","last_peered":"2024-09-15T22:12:24.395034+0000","last_clean":"2024-09-15T22:12:24.395034+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395034+0000","last_undegraded":"2024-09-15T22:12:24.395034+0000","last_fullsized":"2024-09-15T22:12:24.395034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393881+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393881+0000","last_peered":"2024-09-15T22:12:24.393881+0000","last_clean":"2024-09-15T22:12:24.393881+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393881+0000","last_undegraded":"2024-09-15T22:12:24.393881+0000","last_fullsized":"2024-09-15T22:12:24.393881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393962+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.393962+0000","last_peered":"2024-09-15T22:12:24.393962+0000","last_clean":"2024-09-15T22:12:24.393962+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.393962+0000","last_undegraded":"2024-09-15T22:12:24.393962+0000","last_fullsized":"2024-09-15T22:12:24.393962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648427+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648427+0000","last_peered":"2024-09-15T22:12:24.648427+0000","last_clean":"2024-09-15T22:12:24.648427+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648427+0000","last_undegraded":"2024-09-15T22:12:24.648427+0000","last_fullsized":"2024-09-15T22:12:24.648427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'52","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.610182+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:24.610182+0000","last_peered":"2024-09-15T22:12:24.610182+0000","last_clean":"2024-09-15T22:12:24.610182+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:24.610182+0000","last_undegraded":"2024-09-15T22:12:24.610182+0000","last_fullsized":"2024-09-15T22:12:24.610182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.889001+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:24.889001+0000","last_peered":"2024-09-15T22:12:24.889001+0000","last_clean":"2024-09-15T22:12:24.889001+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:24.889001+0000","last_undegraded":"2024-09-15T22:12:24.889001+0000","last_fullsized":"2024-09-15T22:12:24.889001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564288+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:24.564288+0000","last_peered":"2024-09-15T22:12:24.564288+0000","last_clean":"2024-09-15T22:12:24.564288+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:24.564288+0000","last_undegraded":"2024-09-15T22:12:24.564288+0000","last_fullsized":"2024-09-15T22:12:24.564288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642710+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:24.642710+0000","last_peered":"2024-09-15T22:12:24.642710+0000","last_clean":"2024-09-15T22:12:24.642710+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:24.642710+0000","last_undegraded":"2024-09-15T22:12:24.642710+0000","last_fullsized":"2024-09-15T22:12:24.642710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834687+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:24.834687+0000","last_peered":"2024-09-15T22:12:24.834687+0000","last_clean":"2024-09-15T22:12:24.834687+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:24.834687+0000","last_undegraded":"2024-09-15T22:12:24.834687+0000","last_fullsized":"2024-09-15T22:12:24.834687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.607356+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:24.607356+0000","last_peered":"2024-09-15T22:12:24.607356+0000","last_clean":"2024-09-15T22:12:24.607356+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:24.607356+0000","last_undegraded":"2024-09-15T22:12:24.607356+0000","last_fullsized":"2024-09-15T22:12:24.607356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478784+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478784+0000","last_peered":"2024-09-15T22:12:24.478784+0000","last_clean":"2024-09-15T22:12:24.478784+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478784+0000","last_undegraded":"2024-09-15T22:12:24.478784+0000","last_fullsized":"2024-09-15T22:12:24.478784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834654+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:24.834654+0000","last_peered":"2024-09-15T22:12:24.834654+0000","last_clean":"2024-09-15T22:12:24.834654+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:24.834654+0000","last_undegraded":"2024-09-15T22:12:24.834654+0000","last_fullsized":"2024-09-15T22:12:24.834654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":181,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834667+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:24.834667+0000","last_peered":"2024-09-15T22:12:24.834667+0000","last_clean":"2024-09-15T22:12:24.834667+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:24.834667+0000","last_undegraded":"2024-09-15T22:12:24.834667+0000","last_fullsized":"2024-09-15T22:12:24.834667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395286+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395286+0000","last_peered":"2024-09-15T22:12:24.395286+0000","last_clean":"2024-09-15T22:12:24.395286+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395286+0000","last_undegraded":"2024-09-15T22:12:24.395286+0000","last_fullsized":"2024-09-15T22:12:24.395286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834999+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:24.834999+0000","last_peered":"2024-09-15T22:12:24.834999+0000","last_clean":"2024-09-15T22:12:24.834999+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:24.834999+0000","last_undegraded":"2024-09-15T22:12:24.834999+0000","last_fullsized":"2024-09-15T22:12:24.834999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395677+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395677+0000","last_peered":"2024-09-15T22:12:24.395677+0000","last_clean":"2024-09-15T22:12:24.395677+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395677+0000","last_undegraded":"2024-09-15T22:12:24.395677+0000","last_fullsized":"2024-09-15T22:12:24.395677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603119,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1210,"num_read_kb":725,"num_write":384,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809357,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":515769,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1302472,"data_compressed":68275,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1097760,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":500064,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1097728,"data_stored":575423,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":564019,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512547,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135262,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202662,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:29.136 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- 172.21.15.77:0/832126890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430101320 msgr2=0x7fb43010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 --2- 172.21.15.77:0/832126890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430101320 0x7fb43010d850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb428009cd0 tx=0x7fb42802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- 172.21.15.77:0/832126890 shutdown_connections 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 --2- 172.21.15.77:0/832126890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430101320 0x7fb43010d850 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 --2- 172.21.15.77:0/832126890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430100980 0x7fb430100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 --2- 172.21.15.77:0/832126890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430106990 0x7fb430106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- 172.21.15.77:0/832126890 >> 172.21.15.77:0/832126890 conn(0x7fb430077a10 msgr2=0x7fb4300fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- 172.21.15.77:0/832126890 shutdown_connections 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- 172.21.15.77:0/832126890 wait complete. 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 Processor -- start 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.272+0000 7fb437ff8640 1 -- start start 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430100980 0x7fb430199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430101320 0x7fb4301996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb430110dd0 con 0x7fb430106990 2024-09-15T22:12:29.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb430110c50 con 0x7fb430101320 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb430110f50 con 0x7fb430100980 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56972/0 (socket says 172.21.15.77:56972) 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 -- 172.21.15.77:0/289393391 learned_addr learned my addr 172.21.15.77:0/289393391 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb435593640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430101320 0x7fb4301996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb435d94640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430100980 0x7fb430199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 -- 172.21.15.77:0/289393391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430100980 msgr2=0x7fb430199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430100980 0x7fb430199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 -- 172.21.15.77:0/289393391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430101320 msgr2=0x7fb4301996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430101320 0x7fb4301996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 -- 172.21.15.77:0/289393391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4301a4720 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb436595640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb428005cc0 tx=0x7fb428003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428004040 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4280046c0 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428030e70 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4301a48f0 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.273+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a4db0 con 0x7fb430106990 2024-09-15T22:12:29.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.274+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430100de0 con 0x7fb430106990 2024-09-15T22:12:29.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.276+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fb4280305a0 con 0x7fb430106990 2024-09-15T22:12:29.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.276+0000 7fb426ffd640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 0x7fb41c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.276+0000 7fb435d94640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 0x7fb41c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.276+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb42802e030 con 0x7fb430106990 2024-09-15T22:12:29.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.277+0000 7fb435d94640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 0x7fb41c0a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00ab60 tx=0x7fb42c00a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.280+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4280bdd80 con 0x7fb430106990 2024-09-15T22:12:29.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.443+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4300ff6c0 con 0x7fb41c0a34e0 2024-09-15T22:12:29.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.450+0000 7fb426ffd640 1 -- 172.21.15.77:0/289393391 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7fb4300ff6c0 con 0x7fb41c0a34e0 2024-09-15T22:12:29.456 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:29.457 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:29.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 msgr2=0x7fb41c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 0x7fb41c0a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00ab60 tx=0x7fb42c00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 msgr2=0x7fb4301a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb428005cc0 tx=0x7fb428003970 comp rx=0 tx=0).stop 2024-09-15T22:12:29.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 shutdown_connections 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb41c0a34e0 0x7fb41c0a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb430106990 0x7fb4301a41e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430101320 0x7fb4301996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 --2- 172.21.15.77:0/289393391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb430100980 0x7fb430199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 >> 172.21.15.77:0/289393391 conn(0x7fb430077a10 msgr2=0x7fb4300fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 shutdown_connections 2024-09-15T22:12:29.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.454+0000 7fb437ff8640 1 -- 172.21.15.77:0/289393391 wait complete. 2024-09-15T22:12:29.467 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:12:28.402665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193721,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1346,"num_read_kb":844,"num_write":567,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378312,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563512,"statfs":{"total":1151940427776,"available":1151553036288,"internally_reserved":0,"allocated":12689408,"data_stored":9027196,"data_compressed":459456,"data_compressed_allocated":622592,"data_compressed_original":3538944,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988318"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642571+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:24.642571+0000","last_peered":"2024-09-15T22:12:24.642571+0000","last_clean":"2024-09-15T22:12:24.642571+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:24.642571+0000","last_undegraded":"2024-09-15T22:12:24.642571+0000","last_fullsized":"2024-09-15T22:12:24.642571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395034+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395034+0000","last_peered":"2024-09-15T22:12:24.395034+0000","last_clean":"2024-09-15T22:12:24.395034+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395034+0000","last_undegraded":"2024-09-15T22:12:24.395034+0000","last_fullsized":"2024-09-15T22:12:24.395034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393881+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393881+0000","last_peered":"2024-09-15T22:12:24.393881+0000","last_clean":"2024-09-15T22:12:24.393881+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393881+0000","last_undegraded":"2024-09-15T22:12:24.393881+0000","last_fullsized":"2024-09-15T22:12:24.393881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393962+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.393962+0000","last_peered":"2024-09-15T22:12:24.393962+0000","last_clean":"2024-09-15T22:12:24.393962+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.393962+0000","last_undegraded":"2024-09-15T22:12:24.393962+0000","last_fullsized":"2024-09-15T22:12:24.393962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648427+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648427+0000","last_peered":"2024-09-15T22:12:24.648427+0000","last_clean":"2024-09-15T22:12:24.648427+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648427+0000","last_undegraded":"2024-09-15T22:12:24.648427+0000","last_fullsized":"2024-09-15T22:12:24.648427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'52","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.610182+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:24.610182+0000","last_peered":"2024-09-15T22:12:24.610182+0000","last_clean":"2024-09-15T22:12:24.610182+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:24.610182+0000","last_undegraded":"2024-09-15T22:12:24.610182+0000","last_fullsized":"2024-09-15T22:12:24.610182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.889001+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:24.889001+0000","last_peered":"2024-09-15T22:12:24.889001+0000","last_clean":"2024-09-15T22:12:24.889001+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:24.889001+0000","last_undegraded":"2024-09-15T22:12:24.889001+0000","last_fullsized":"2024-09-15T22:12:24.889001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564288+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:24.564288+0000","last_peered":"2024-09-15T22:12:24.564288+0000","last_clean":"2024-09-15T22:12:24.564288+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:24.564288+0000","last_undegraded":"2024-09-15T22:12:24.564288+0000","last_fullsized":"2024-09-15T22:12:24.564288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642710+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:24.642710+0000","last_peered":"2024-09-15T22:12:24.642710+0000","last_clean":"2024-09-15T22:12:24.642710+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:24.642710+0000","last_undegraded":"2024-09-15T22:12:24.642710+0000","last_fullsized":"2024-09-15T22:12:24.642710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834687+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:24.834687+0000","last_peered":"2024-09-15T22:12:24.834687+0000","last_clean":"2024-09-15T22:12:24.834687+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:24.834687+0000","last_undegraded":"2024-09-15T22:12:24.834687+0000","last_fullsized":"2024-09-15T22:12:24.834687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.607356+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:24.607356+0000","last_peered":"2024-09-15T22:12:24.607356+0000","last_clean":"2024-09-15T22:12:24.607356+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:24.607356+0000","last_undegraded":"2024-09-15T22:12:24.607356+0000","last_fullsized":"2024-09-15T22:12:24.607356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478784+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478784+0000","last_peered":"2024-09-15T22:12:24.478784+0000","last_clean":"2024-09-15T22:12:24.478784+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478784+0000","last_undegraded":"2024-09-15T22:12:24.478784+0000","last_fullsized":"2024-09-15T22:12:24.478784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834654+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:24.834654+0000","last_peered":"2024-09-15T22:12:24.834654+0000","last_clean":"2024-09-15T22:12:24.834654+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:24.834654+0000","last_undegraded":"2024-09-15T22:12:24.834654+0000","last_fullsized":"2024-09-15T22:12:24.834654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":181,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834667+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:24.834667+0000","last_peered":"2024-09-15T22:12:24.834667+0000","last_clean":"2024-09-15T22:12:24.834667+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:24.834667+0000","last_undegraded":"2024-09-15T22:12:24.834667+0000","last_fullsized":"2024-09-15T22:12:24.834667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395286+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395286+0000","last_peered":"2024-09-15T22:12:24.395286+0000","last_clean":"2024-09-15T22:12:24.395286+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395286+0000","last_undegraded":"2024-09-15T22:12:24.395286+0000","last_fullsized":"2024-09-15T22:12:24.395286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834999+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:24.834999+0000","last_peered":"2024-09-15T22:12:24.834999+0000","last_clean":"2024-09-15T22:12:24.834999+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:24.834999+0000","last_undegraded":"2024-09-15T22:12:24.834999+0000","last_fullsized":"2024-09-15T22:12:24.834999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395677+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395677+0000","last_peered":"2024-09-15T22:12:24.395677+0000","last_clean":"2024-09-15T22:12:24.395677+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395677+0000","last_undegraded":"2024-09-15T22:12:24.395677+0000","last_fullsized":"2024-09-15T22:12:24.395677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603119,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1210,"num_read_kb":725,"num_write":384,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809357,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":515769,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1302472,"data_compressed":68275,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1097760,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":500064,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1097728,"data_stored":575423,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":564019,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512547,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135262,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202662,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:29.472 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-15T22:12:29.472 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:12:29.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 -- 172.21.15.77:0/766071771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 msgr2=0x7f8aec1007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 --2- 172.21.15.77:0/766071771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1007c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009900 tx=0x7f8ae402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 -- 172.21.15.77:0/766071771 shutdown_connections 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 --2- 172.21.15.77:0/766071771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 --2- 172.21.15.77:0/766071771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec100d90 0x7f8aec104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 --2- 172.21.15.77:0/766071771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1007c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 -- 172.21.15.77:0/766071771 >> 172.21.15.77:0/766071771 conn(0x7f8aec0771a0 msgr2=0x7f8aec0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 -- 172.21.15.77:0/766071771 shutdown_connections 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.604+0000 7f8af10dd640 1 -- 172.21.15.77:0/766071771 wait complete. 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.605+0000 7f8af10dd640 1 Processor -- start 2024-09-15T22:12:29.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.605+0000 7f8af10dd640 1 -- start start 2024-09-15T22:12:29.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.605+0000 7f8af10dd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.605+0000 7f8af10dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec199d90 0x7f8aec1a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec110dd0 con 0x7f8aec1003e0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec110c50 con 0x7f8aec199d90 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec110f50 con 0x7f8aec1054e0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56980/0 (socket says 172.21.15.77:56980) 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 -- 172.21.15.77:0/4077028873 learned_addr learned my addr 172.21.15.77:0/4077028873 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aea59c640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aeb59e640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec199d90 0x7f8aec1a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 -- 172.21.15.77:0/4077028873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 msgr2=0x7f8aec199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 -- 172.21.15.77:0/4077028873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec199d90 msgr2=0x7f8aec1a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec199d90 0x7f8aec1a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 -- 172.21.15.77:0/4077028873 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1a4b10 con 0x7f8aec1003e0 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aea59c640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8aead9d640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4006010 tx=0x7f8ae40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae403f030 con 0x7f8aec1003e0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec1a4da0 con 0x7f8aec1003e0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae4003ce0 con 0x7f8aec1003e0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4035070 con 0x7f8aec1003e0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.606+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1a5230 con 0x7f8aec1003e0 2024-09-15T22:12:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.607+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec1007c0 con 0x7f8aec1003e0 2024-09-15T22:12:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.609+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f8ae4042030 con 0x7f8aec1003e0 2024-09-15T22:12:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.610+0000 7f8ae3fff640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 0x7f8ad80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.610+0000 7f8aea59c640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 0x7f8ad80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.610+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8ae40f0800 con 0x7f8aec1003e0 2024-09-15T22:12:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.610+0000 7f8aea59c640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 0x7f8ad80a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8aec10e0c0 tx=0x7f8ad400b5f0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.614+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ae40bcde0 con 0x7f8aec1003e0 2024-09-15T22:12:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.777+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8aec100d90 con 0x7f8ad80a34e0 2024-09-15T22:12:29.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.785+0000 7f8ae3fff640 1 -- 172.21.15.77:0/4077028873 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339716 (secure 0 0 0) 0x7f8aec100d90 con 0x7f8ad80a34e0 2024-09-15T22:12:29.790 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:29.792 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:29.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 msgr2=0x7f8ad80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 0x7f8ad80a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8aec10e0c0 tx=0x7f8ad400b5f0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 msgr2=0x7f8aec1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4006010 tx=0x7f8ae40042f0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 shutdown_connections 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ad80a34e0 0x7f8ad80a59a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec199d90 0x7f8aec1a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aec1054e0 0x7f8aec199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 --2- 172.21.15.77:0/4077028873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aec1003e0 0x7f8aec1992c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 >> 172.21.15.77:0/4077028873 conn(0x7f8aec0771a0 msgr2=0x7f8aec103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 shutdown_connections 2024-09-15T22:12:29.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:29.788+0000 7f8af10dd640 1 -- 172.21.15.77:0/4077028873 wait complete. 2024-09-15T22:12:29.801 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:12:28.402665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193721,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1346,"num_read_kb":844,"num_write":567,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378312,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563512,"statfs":{"total":1151940427776,"available":1151553036288,"internally_reserved":0,"allocated":12689408,"data_stored":9027196,"data_compressed":459456,"data_compressed_allocated":622592,"data_compressed_original":3538944,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988318"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642571+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:24.642571+0000","last_peered":"2024-09-15T22:12:24.642571+0000","last_clean":"2024-09-15T22:12:24.642571+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:24.642571+0000","last_undegraded":"2024-09-15T22:12:24.642571+0000","last_fullsized":"2024-09-15T22:12:24.642571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395034+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395034+0000","last_peered":"2024-09-15T22:12:24.395034+0000","last_clean":"2024-09-15T22:12:24.395034+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395034+0000","last_undegraded":"2024-09-15T22:12:24.395034+0000","last_fullsized":"2024-09-15T22:12:24.395034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393881+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393881+0000","last_peered":"2024-09-15T22:12:24.393881+0000","last_clean":"2024-09-15T22:12:24.393881+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393881+0000","last_undegraded":"2024-09-15T22:12:24.393881+0000","last_fullsized":"2024-09-15T22:12:24.393881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393962+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.393962+0000","last_peered":"2024-09-15T22:12:24.393962+0000","last_clean":"2024-09-15T22:12:24.393962+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.393962+0000","last_undegraded":"2024-09-15T22:12:24.393962+0000","last_fullsized":"2024-09-15T22:12:24.393962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648427+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648427+0000","last_peered":"2024-09-15T22:12:24.648427+0000","last_clean":"2024-09-15T22:12:24.648427+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648427+0000","last_undegraded":"2024-09-15T22:12:24.648427+0000","last_fullsized":"2024-09-15T22:12:24.648427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'52","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.610182+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:24.610182+0000","last_peered":"2024-09-15T22:12:24.610182+0000","last_clean":"2024-09-15T22:12:24.610182+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:24.610182+0000","last_undegraded":"2024-09-15T22:12:24.610182+0000","last_fullsized":"2024-09-15T22:12:24.610182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.889001+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:24.889001+0000","last_peered":"2024-09-15T22:12:24.889001+0000","last_clean":"2024-09-15T22:12:24.889001+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:24.889001+0000","last_undegraded":"2024-09-15T22:12:24.889001+0000","last_fullsized":"2024-09-15T22:12:24.889001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564288+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:24.564288+0000","last_peered":"2024-09-15T22:12:24.564288+0000","last_clean":"2024-09-15T22:12:24.564288+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:24.564288+0000","last_undegraded":"2024-09-15T22:12:24.564288+0000","last_fullsized":"2024-09-15T22:12:24.564288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642710+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:24.642710+0000","last_peered":"2024-09-15T22:12:24.642710+0000","last_clean":"2024-09-15T22:12:24.642710+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:24.642710+0000","last_undegraded":"2024-09-15T22:12:24.642710+0000","last_fullsized":"2024-09-15T22:12:24.642710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834687+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:24.834687+0000","last_peered":"2024-09-15T22:12:24.834687+0000","last_clean":"2024-09-15T22:12:24.834687+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:24.834687+0000","last_undegraded":"2024-09-15T22:12:24.834687+0000","last_fullsized":"2024-09-15T22:12:24.834687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.607356+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:24.607356+0000","last_peered":"2024-09-15T22:12:24.607356+0000","last_clean":"2024-09-15T22:12:24.607356+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:24.607356+0000","last_undegraded":"2024-09-15T22:12:24.607356+0000","last_fullsized":"2024-09-15T22:12:24.607356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478784+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478784+0000","last_peered":"2024-09-15T22:12:24.478784+0000","last_clean":"2024-09-15T22:12:24.478784+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478784+0000","last_undegraded":"2024-09-15T22:12:24.478784+0000","last_fullsized":"2024-09-15T22:12:24.478784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":304,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834654+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:24.834654+0000","last_peered":"2024-09-15T22:12:24.834654+0000","last_clean":"2024-09-15T22:12:24.834654+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:24.834654+0000","last_undegraded":"2024-09-15T22:12:24.834654+0000","last_fullsized":"2024-09-15T22:12:24.834654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":181,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":259,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834667+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:24.834667+0000","last_peered":"2024-09-15T22:12:24.834667+0000","last_clean":"2024-09-15T22:12:24.834667+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:24.834667+0000","last_undegraded":"2024-09-15T22:12:24.834667+0000","last_fullsized":"2024-09-15T22:12:24.834667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395286+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395286+0000","last_peered":"2024-09-15T22:12:24.395286+0000","last_clean":"2024-09-15T22:12:24.395286+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395286+0000","last_undegraded":"2024-09-15T22:12:24.395286+0000","last_fullsized":"2024-09-15T22:12:24.395286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.834999+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:24.834999+0000","last_peered":"2024-09-15T22:12:24.834999+0000","last_clean":"2024-09-15T22:12:24.834999+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:24.834999+0000","last_undegraded":"2024-09-15T22:12:24.834999+0000","last_fullsized":"2024-09-15T22:12:24.834999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395677+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395677+0000","last_peered":"2024-09-15T22:12:24.395677+0000","last_clean":"2024-09-15T22:12:24.395677+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395677+0000","last_undegraded":"2024-09-15T22:12:24.395677+0000","last_fullsized":"2024-09-15T22:12:24.395677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603119,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1210,"num_read_kb":725,"num_write":384,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1809357,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":515769,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1302472,"data_compressed":68275,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45541,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1097760,"data_compressed":59151,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":500064,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1097728,"data_stored":575423,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":564019,"data_compressed":23427,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512547,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135262,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202662,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:29.806 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:30.084 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:30.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:30 smithi077 ceph-mon[30980]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:30.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:30 smithi077 ceph-mon[30980]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:30.530 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:30 smithi077 ceph-mon[30980]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:12:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:30 smithi196 ceph-mon[35109]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:30 smithi196 ceph-mon[35109]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:30 smithi196 ceph-mon[35109]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:12:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:30 smithi117 ceph-mon[35021]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:30 smithi117 ceph-mon[35021]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:30 smithi117 ceph-mon[35021]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 -- 172.21.15.77:0/3446786635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4102570 msgr2=0x7f35d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 --2- 172.21.15.77:0/3446786635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4102570 0x7f35d41029b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009900 tx=0x7f35bc02a480 comp rx=0 tx=0).stop 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 -- 172.21.15.77:0/3446786635 shutdown_connections 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 --2- 172.21.15.77:0/3446786635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 0x7f35d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 --2- 172.21.15.77:0/3446786635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4102570 0x7f35d41029b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 --2- 172.21.15.77:0/3446786635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4100e20 0x7f35d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 -- 172.21.15.77:0/3446786635 >> 172.21.15.77:0/3446786635 conn(0x7f35d40fc7d0 msgr2=0x7f35d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:30.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 -- 172.21.15.77:0/3446786635 shutdown_connections 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.610+0000 7f35d8c65640 1 -- 172.21.15.77:0/3446786635 wait complete. 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 Processor -- start 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 -- start start 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4102570 0x7f35d4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 0x7f35d419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111070 con 0x7f35d4100e20 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4110ef0 con 0x7f35d4102ef0 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.611+0000 7f35d8c65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35d41111f0 con 0x7f35d4102570 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56990/0 (socket says 172.21.15.77:56990) 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 -- 172.21.15.77:0/1213437468 learned_addr learned my addr 172.21.15.77:0/1213437468 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:30.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d2d9d640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 0x7f35d419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d1d9b640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4102570 0x7f35d4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 -- 172.21.15.77:0/1213437468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4102570 msgr2=0x7f35d4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4102570 0x7f35d4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 -- 172.21.15.77:0/1213437468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 msgr2=0x7f35d419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 0x7f35d419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 -- 172.21.15.77:0/1213437468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419c500 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d259c640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00ba50 tx=0x7f35cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc019070 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419c7f0 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc00cbe0 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0076e0 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.612+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a40d0 con 0x7f35d4100e20 2024-09-15T22:12:30.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.613+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4101280 con 0x7f35d4100e20 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.615+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f35cc01c070 con 0x7f35d4100e20 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.615+0000 7f35cb7fe640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 0x7f35c00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.615+0000 7f35d1d9b640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 0x7f35c00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.616+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f35cc0cf800 con 0x7f35d4100e20 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.616+0000 7f35d1d9b640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 0x7f35c00a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c00 tx=0x7f35bc005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:30.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.619+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35cc09b710 con 0x7f35d4100e20 2024-09-15T22:12:30.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.825+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35d4109090 con 0x7f35d4100e20 2024-09-15T22:12:30.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.826+0000 7f35cb7fe640 1 -- 172.21.15.77:0/1213437468 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f35cc09fa10 con 0x7f35d4100e20 2024-09-15T22:12:30.828 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:30.828 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:12:30.828 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 msgr2=0x7f35c00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 0x7f35c00a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c00 tx=0x7f35bc005a20 comp rx=0 tx=0).stop 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 msgr2=0x7f35d4197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00ba50 tx=0x7f35cc00bf10 comp rx=0 tx=0).stop 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 shutdown_connections 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c00a34d0 0x7f35c00a5990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d4102ef0 0x7f35d419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35d4102570 0x7f35d4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 --2- 172.21.15.77:0/1213437468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35d4100e20 0x7f35d4197550 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:30.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 >> 172.21.15.77:0/1213437468 conn(0x7f35d40fc7d0 msgr2=0x7f35d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:30.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 shutdown_connections 2024-09-15T22:12:30.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:30.829+0000 7f35d8c65640 1 -- 172.21.15.77:0/1213437468 wait complete. 2024-09-15T22:12:31.509 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:31 smithi077 ceph-mon[30980]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.509 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:31 smithi077 ceph-mon[30980]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.509 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:31 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1213437468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:31.537 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:12:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:31 smithi196 ceph-mon[35109]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:31 smithi196 ceph-mon[35109]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:31 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1213437468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:31 smithi117 ceph-mon[35021]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:31 smithi117 ceph-mon[35021]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:31 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1213437468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:31.786 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 -- 172.21.15.77:0/1416155026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 msgr2=0x7f4968103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 --2- 172.21.15.77:0/1416155026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968103120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f495c009900 tx=0x7f495c02a480 comp rx=0 tx=0).stop 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 -- 172.21.15.77:0/1416155026 shutdown_connections 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 --2- 172.21.15.77:0/1416155026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 0x7f496810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 --2- 172.21.15.77:0/1416155026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 0x7f496810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 --2- 172.21.15.77:0/1416155026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968103120 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 -- 172.21.15.77:0/1416155026 >> 172.21.15.77:0/1416155026 conn(0x7f49680fc7a0 msgr2=0x7f49680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 -- 172.21.15.77:0/1416155026 shutdown_connections 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.277+0000 7f496edfb640 1 -- 172.21.15.77:0/1416155026 wait complete. 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.278+0000 7f496edfb640 1 Processor -- start 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.278+0000 7f496edfb640 1 -- start start 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.278+0000 7f496edfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.278+0000 7f496edfb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 0x7f4968198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496edfb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 0x7f496819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496edfb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111f70 con 0x7f4968102d40 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496edfb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111df0 con 0x7f4968103660 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496edfb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49681120f0 con 0x7f496810c170 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56996/0 (socket says 172.21.15.77:56996) 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 -- 172.21.15.77:0/2930812296 learned_addr learned my addr 172.21.15.77:0/2930812296 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496d398640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 0x7f496819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f4967fff640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 0x7f4968198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 -- 172.21.15.77:0/2930812296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 msgr2=0x7f496819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 0x7f496819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 -- 172.21.15.77:0/2930812296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 msgr2=0x7f4968198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 0x7f4968198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 -- 172.21.15.77:0/2930812296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496819d3e0 con 0x7f4968102d40 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496cb97640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f495c006010 tx=0x7f495c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c03f030 con 0x7f4968102d40 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c003ce0 con 0x7f4968102d40 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496819d670 con 0x7f4968102d40 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.279+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c035070 con 0x7f4968102d40 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.280+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a4fb0 con 0x7f4968102d40 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.281+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496806b070 con 0x7f4968102d40 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.282+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f495c042030 con 0x7f4968102d40 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.282+0000 7f4965ffb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 0x7f49580a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.282+0000 7f4967fff640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 0x7f49580a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.282+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f495c0f0900 con 0x7f4968102d40 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.283+0000 7f4967fff640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 0x7f49580a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49540058c0 tx=0x7f4954005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.286+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f495c0bcee0 con 0x7f4968102d40 2024-09-15T22:12:32.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.456+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4968107210 con 0x7f4968102d40 2024-09-15T22:12:32.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:32 smithi077 ceph-mon[30980]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:32.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.458+0000 7f4965ffb640 1 -- 172.21.15.77:0/2930812296 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25152 (secure 0 0 0) 0x7f495c0bc7e0 con 0x7f4968102d40 2024-09-15T22:12:32.461 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:32.461 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":91,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:12:24.385821+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/999829703":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/3279424814":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3785009540":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1809193257":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/1437248041":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1809193257":"2024-09-16T22:12:24.385790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 msgr2=0x7f49580a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 0x7f49580a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f49540058c0 tx=0x7f4954005850 comp rx=0 tx=0).stop 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 msgr2=0x7f4968198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f495c006010 tx=0x7f495c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 shutdown_connections 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49580a34d0 0x7f49580a5990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f496810c170 0x7f496819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968103660 0x7f4968198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 --2- 172.21.15.77:0/2930812296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968102d40 0x7f4968198430 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.460+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 >> 172.21.15.77:0/2930812296 conn(0x7f49680fc7a0 msgr2=0x7f49680fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.461+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 shutdown_connections 2024-09-15T22:12:32.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:32.461+0000 7f496edfb640 1 -- 172.21.15.77:0/2930812296 wait complete. 2024-09-15T22:12:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:32 smithi196 ceph-mon[35109]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:32 smithi117 ceph-mon[35021]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:12:33.174 DEBUG:tasks.fwd_scrub:Yielding 2024-09-15T22:12:33.174 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:12:33.184 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:12:33.184 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:12:33.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-15T22:12:33.185 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-15T22:12:33.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:12:33.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:12:33.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- 172.21.15.77:0/1708106745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28107b70 msgr2=0x7f9f28107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1708106745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28107b70 0x7f9f28107f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009930 tx=0x7f9f2402a420 comp rx=0 tx=0).stop 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- 172.21.15.77:0/1708106745 shutdown_connections 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1708106745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 0x7f9f2810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1708106745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28101ad0 0x7f9f28101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1708106745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28107b70 0x7f9f28107f50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- 172.21.15.77:0/1708106745 >> 172.21.15.77:0/1708106745 conn(0x7f9f280fd7d0 msgr2=0x7f9f280ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- 172.21.15.77:0/1708106745 shutdown_connections 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- 172.21.15.77:0/1708106745 wait complete. 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 Processor -- start 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.291+0000 7f9f3040e640 1 -- start start 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 0x7f9f28198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28107b70 0x7f9f2819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28111ee0 con 0x7f9f28101ad0 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28111d60 con 0x7f9f28107b70 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28112060 con 0x7f9f28102470 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57014/0 (socket says 172.21.15.77:57014) 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 -- 172.21.15.77:0/1762568586 learned_addr learned my addr 172.21.15.77:0/1762568586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2d9a9640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 0x7f9f28198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e9ab640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28107b70 0x7f9f2819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 -- 172.21.15.77:0/1762568586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 msgr2=0x7f9f28198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 0x7f9f28198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 -- 172.21.15.77:0/1762568586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28107b70 msgr2=0x7f9f2819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28107b70 0x7f9f2819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 -- 172.21.15.77:0/1762568586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2819d2e0 con 0x7f9f28101ad0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f2e1aa640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009a50 tx=0x7f9f24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2403f030 con 0x7f9f28101ad0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24003ad0 con 0x7f9f28101ad0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24035070 con 0x7f9f28101ad0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2819d570 con 0x7f9f28101ad0 2024-09-15T22:12:33.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.292+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a4e60 con 0x7f9f28101ad0 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.294+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f28101f30 con 0x7f9f28101ad0 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.295+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f9f24042030 con 0x7f9f28101ad0 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.296+0000 7f9f1f7fe640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 0x7f9f140a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.296+0000 7f9f2d9a9640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 0x7f9f140a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.296+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9f240f09c0 con 0x7f9f28101ad0 2024-09-15T22:12:33.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.296+0000 7f9f2d9a9640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 0x7f9f140a5ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f280ffac0 tx=0x7f9f1800a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.300+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f240bce80 con 0x7f9f28101ad0 2024-09-15T22:12:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:33.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.523+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f9f281a5030 con 0x7f9f28101ad0 2024-09-15T22:12:33.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.527+0000 7f9f1f7fe640 1 -- 172.21.15.77:0/1762568586 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7f9f240bc780 con 0x7f9f28101ad0 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 msgr2=0x7f9f140a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 0x7f9f140a5ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f280ffac0 tx=0x7f9f1800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 msgr2=0x7f9f281982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009a50 tx=0x7f9f24004270 comp rx=0 tx=0).stop 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 shutdown_connections 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f140a3600 0x7f9f140a5ac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f28107b70 0x7f9f2819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f28102470 0x7f9f28198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 --2- 172.21.15.77:0/1762568586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f28101ad0 0x7f9f281982f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 >> 172.21.15.77:0/1762568586 conn(0x7f9f280fd7d0 msgr2=0x7f9f280fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:33.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 shutdown_connections 2024-09-15T22:12:33.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.529+0000 7f9f3040e640 1 -- 172.21.15.77:0/1762568586 wait complete. 2024-09-15T22:12:33.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2930812296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:33.542 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-15T22:12:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:33 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2930812296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:33 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2930812296' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:33.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 -- 172.21.15.77:0/2799509022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee081080f0 msgr2=0x7fee0810ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 --2- 172.21.15.77:0/2799509022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee081080f0 0x7fee0810ea40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009cd0 tx=0x7fedf802a970 comp rx=0 tx=0).stop 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 -- 172.21.15.77:0/2799509022 shutdown_connections 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 --2- 172.21.15.77:0/2799509022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee081080f0 0x7fee0810ea40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 --2- 172.21.15.77:0/2799509022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee08101150 0x7fee08107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 --2- 172.21.15.77:0/2799509022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 0x7fee08100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 -- 172.21.15.77:0/2799509022 >> 172.21.15.77:0/2799509022 conn(0x7fee08075ec0 msgr2=0x7fee080762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 -- 172.21.15.77:0/2799509022 shutdown_connections 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.645+0000 7fee0f55f640 1 -- 172.21.15.77:0/2799509022 wait complete. 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 Processor -- start 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 -- start start 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 0x7fee081982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 0x7fee0819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee08111f20 con 0x7fee08101150 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee08111da0 con 0x7fee08100830 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0f55f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee081120a0 con 0x7fee081080f0 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57028/0 (socket says 172.21.15.77:57028) 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 -- 172.21.15.77:0/3078871559 learned_addr learned my addr 172.21.15.77:0/3078871559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0d2fb640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 0x7fee081982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0dafc640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 0x7fee0819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 -- 172.21.15.77:0/3078871559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 msgr2=0x7fee0819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 0x7fee0819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 -- 172.21.15.77:0/3078871559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 msgr2=0x7fee081982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 0x7fee081982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 -- 172.21.15.77:0/3078871559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0819d2b0 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0dafc640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 0x7fee0819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.646+0000 7fee0cafa640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d740 tx=0x7fedfc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.647+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc017070 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.647+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedfc00de50 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.647+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0819d5a0 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.647+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc004e00 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.647+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081a4e30 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.648+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee08100c10 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.649+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fedfc004fa0 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.649+0000 7fee067fc640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 0x7fedf00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.650+0000 7fee0d2fb640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 0x7fedf00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.650+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fedfc0cfc30 con 0x7fee08101150 2024-09-15T22:12:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.650+0000 7fee0d2fb640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 0x7fedf00a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee0000bbe0 tx=0x7fee0000b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.653+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedfc09c080 con 0x7fee08101150 2024-09-15T22:12:33.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.831+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fee081a5090 con 0x7fee08101150 2024-09-15T22:12:33.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.835+0000 7fee067fc640 1 -- 172.21.15.77:0/3078871559 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7fedfc09b980 con 0x7fee08101150 2024-09-15T22:12:33.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 msgr2=0x7fedf00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 0x7fedf00a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee0000bbe0 tx=0x7fee0000b390 comp rx=0 tx=0).stop 2024-09-15T22:12:33.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 msgr2=0x7fee08198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d740 tx=0x7fedfc00dc00 comp rx=0 tx=0).stop 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 shutdown_connections 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedf00a3580 0x7fedf00a5a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee081080f0 0x7fee0819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee08101150 0x7fee08198800 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 --2- 172.21.15.77:0/3078871559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee08100830 0x7fee081982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 >> 172.21.15.77:0/3078871559 conn(0x7fee08075ec0 msgr2=0x7fee080ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 shutdown_connections 2024-09-15T22:12:33.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.838+0000 7fee0f55f640 1 -- 172.21.15.77:0/3078871559 wait complete. 2024-09-15T22:12:33.852 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 -- 172.21.15.77:0/179909692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8101210 msgr2=0x7f37d810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 --2- 172.21.15.77:0/179909692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8101210 0x7f37d810ded0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009900 tx=0x7f37cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 -- 172.21.15.77:0/179909692 shutdown_connections 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 --2- 172.21.15.77:0/179909692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d810e410 0x7f37d8110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 --2- 172.21.15.77:0/179909692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8101210 0x7f37d810ded0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 --2- 172.21.15.77:0/179909692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d8100860 0x7f37d8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 -- 172.21.15.77:0/179909692 >> 172.21.15.77:0/179909692 conn(0x7f37d8077990 msgr2=0x7f37d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 -- 172.21.15.77:0/179909692 shutdown_connections 2024-09-15T22:12:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.967+0000 7f37dcf31640 1 -- 172.21.15.77:0/179909692 wait complete. 2024-09-15T22:12:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 Processor -- start 2024-09-15T22:12:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 -- start start 2024-09-15T22:12:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8101210 0x7f37d819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 0x7f37d819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81140c0 con 0x7f37d8100860 2024-09-15T22:12:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8113f40 con 0x7f37d8101210 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37dcf31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114240 con 0x7f37d810e410 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57052/0 (socket says 172.21.15.77:57052) 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 -- 172.21.15.77:0/4271820199 learned_addr learned my addr 172.21.15.77:0/4271820199 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d6d9d640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 0x7f37d819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d5d9b640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8101210 0x7f37d819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 -- 172.21.15.77:0/4271820199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 msgr2=0x7f37d819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 0x7f37d819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 -- 172.21.15.77:0/4271820199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8101210 msgr2=0x7f37d819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.968+0000 7f37d659c640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8101210 0x7f37d819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37d659c640 1 -- 172.21.15.77:0/4271820199 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819f580 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37d6d9d640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 0x7f37d819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37d659c640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f37c800e9f0 tx=0x7f37c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c800cd80 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c8010910 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819f870 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8018990 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.969+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a7100 con 0x7f37d8100860 2024-09-15T22:12:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.970+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8100c40 con 0x7f37d8100860 2024-09-15T22:12:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.971+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f37c8010ac0 con 0x7f37d8100860 2024-09-15T22:12:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.972+0000 7f37c77fe640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 0x7f37c00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.972+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f37c80d09f0 con 0x7f37d8100860 2024-09-15T22:12:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.972+0000 7f37d5d9b640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 0x7f37c00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.973+0000 7f37d5d9b640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 0x7f37c00a5a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37cc000c00 tx=0x7f37cc005b50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:33.975+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37c809c7f0 con 0x7f37d8100860 2024-09-15T22:12:34.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/932420460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc102df0 msgr2=0x7fc0bc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/932420460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc102df0 0x7fc0bc1031d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a410 comp rx=0 tx=0).stop 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/932420460 shutdown_connections 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/932420460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/932420460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 0x7fc0bc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/932420460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc102df0 0x7fc0bc1031d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/932420460 >> 172.21.15.77:0/932420460 conn(0x7fc0bc0fc7d0 msgr2=0x7fc0bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/932420460 shutdown_connections 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/932420460 wait complete. 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.027+0000 7fc0c1a2f640 1 Processor -- start 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 -- start start 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 0x7fc0bc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 0x7fc0bc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc112110 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc111f90 con 0x7fc0bc102df0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0c1a2f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc112290 con 0x7fc0bc1037a0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57076/0 (socket says 172.21.15.77:57076) 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/738596446 learned_addr learned my addr 172.21.15.77:0/738596446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0baffd640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 0x7fc0bc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 0x7fc0bc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/738596446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 msgr2=0x7fc0bc198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 0x7fc0bc198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/738596446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 msgr2=0x7fc0bc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 0x7fc0bc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/738596446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc19d5e0 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.028+0000 7fc0baffd640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 0x7fc0bc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0c0a2d640 1 -- 172.21.15.77:0/738596446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4017070 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc19d8d0 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0c0a2d640 1 -- 172.21.15.77:0/738596446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4003e50 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0c0a2d640 1 -- 172.21.15.77:0/738596446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40106c0 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.029+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a51b0 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.031+0000 7fc0c0a2d640 1 -- 172.21.15.77:0/738596446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fc0b4003970 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.031+0000 7fc0c3e08740 1 Processor -- start 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.031+0000 7fc0c3e08740 1 -- start start 2024-09-15T22:12:34.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.031+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a744e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.031+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a754150 0x55a18a756540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a18a6aa280 con 0x55a18a7453a0 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a18a651b10 con 0x55a18a754150 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a18a6a6240 con 0x55a18a744a80 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57090/0 (socket says 172.21.15.77:57090) 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 -- 172.21.15.77:0/1607879347 learned_addr learned my addr 172.21.15.77:0/1607879347 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a744e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a754150 0x55a18a756540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 -- 172.21.15.77:0/1607879347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 msgr2=0x55a18a744e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a744e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 -- 172.21.15.77:0/1607879347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a754150 msgr2=0x55a18a756540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a754150 0x55a18a756540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 -- 172.21.15.77:0/1607879347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18a65e1f0 con 0x55a18a7453a0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a744e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c0a2d640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 0x7fc0a80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 0x7fc0a80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac033000 tx=0x7fc0ac032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f668578e9efb0994 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0c0a2d640 1 -- 172.21.15.77:0/738596446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc0b40d0130 con 0x7fc0bc10c2f0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0b1ffb640 1 -- 172.21.15.77:0/1607879347 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac03f070 con 0x55a18a7453a0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.032+0000 7fc0b1ffb640 1 -- 172.21.15.77:0/1607879347 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac0317b0 con 0x55a18a7453a0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0b1ffb640 1 -- 172.21.15.77:0/1607879347 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac035070 con 0x55a18a7453a0 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1607879347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 msgr2=0x55a18a753c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac033000 tx=0x7fc0ac032cb0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 0x7fc0a80a5990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc10d7c0 tx=0x7fc0a40046c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1607879347 shutdown_connections 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a754150 0x55a18a756540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a753c10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1607879347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a744e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1607879347 >> 172.21.15.77:0/1607879347 conn(0x55a18a7410f0 msgr2=0x55a18a7414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1607879347 shutdown_connections 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.033+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1607879347 wait complete. 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 -- start start 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 Processor -- start 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a7510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a751620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a18a756c60 con 0x55a18a754150 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a18a664f10 con 0x55a18a7453a0 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a18a756de0 con 0x55a18a744a80 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0c3e08740 10 client.0 _ll_register_callbacks cb 0x55a18a62da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57098/0 (socket says 172.21.15.77:57098) 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/1517400777 learned_addr learned my addr 172.21.15.77:0/1517400777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.034+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a7510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a751620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/1517400777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 msgr2=0x55a18a7510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a7510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 msgr2=0x55a18a751620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a751620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/1517400777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18a41b6a0 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40982e0 tx=0x7fc0b409b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d745fe498b5e8b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4020040 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc0b40cd6a0 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0ac0317b0 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0b409be80 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0c3e08740 10 client.15471 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a18a7629a0 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac0317b0 con 0x55a18a754150 2024-09-15T22:12:34.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc0b409b2d0 con 0x55a18a754150 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0c3e08740 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.035+0000 7fc0c3e08740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0c3e08740 20 client.15471 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a763e60 0x55a18a7662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0c3e08740 4 client.15471 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18a6b77f0 con 0x55a18a763e60 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a763e60 0x55a18a7662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc09affd640 10 client.15471 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.036+0000 7fc0baffd640 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a763e60 0x55a18a7662e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8478d4c62144bc server_cookie=952a7e085cc92c82 in_seq=0 out_seq=0 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc09affd640 1 -- 172.21.15.77:0/1517400777 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a18a6b77f0 con 0x55a18a763e60 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc09affd640 10 client.15471 handle_command_reply: tid=0 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 client.15471 shutdown 2024-09-15T22:12:34.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 msgr2=0x55a18a751b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40982e0 tx=0x7fc0b409b7e0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 shutdown_connections 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a763e60 0x55a18a7662e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a754150 0x55a18a751b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a751620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/1517400777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a744a80 0x55a18a7510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc09affd640 0 client.15471 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 >> 172.21.15.77:0/1517400777 conn(0x55a18a7410f0 msgr2=0x55a18a746d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 shutdown_connections 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 1 -- 172.21.15.77:0/1517400777 wait complete. 2024-09-15T22:12:34.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.037+0000 7fc0c3e08740 20 client.15471 trim_cache size 0 max 16384 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.054+0000 7fc0c3e08740 1 Processor -- start 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.054+0000 7fc0c3e08740 1 -- start start 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a744a80 0x55a18a76bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a76c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1acb40 con 0x55a18a744a80 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ae680 con 0x55a18a7453a0 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ad210 con 0x55a18a754150 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35792/0 (socket says 172.21.15.77:35792) 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2681076313 learned_addr learned my addr 172.21.15.77:0/2681076313 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0ba7fc640 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a76c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0baffd640 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a744a80 0x55a18a76bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2681076313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 msgr2=0x55a18a76c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a76c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2681076313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a744a80 msgr2=0x55a18a76bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a744a80 0x55a18a76bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2681076313 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18a74f630 con 0x55a18a754150 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.055+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0b409c040 tx=0x7fc0b409b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44375d2e8cb10783 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.056+0000 7fc0b17fa640 1 -- 172.21.15.77:0/2681076313 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4014020 con 0x55a18a754150 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.056+0000 7fc0b17fa640 1 -- 172.21.15.77:0/2681076313 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4086040 con 0x55a18a754150 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.056+0000 7fc0b17fa640 1 -- 172.21.15.77:0/2681076313 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4020070 con 0x55a18a754150 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.056+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2681076313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 msgr2=0x55a18a74e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.056+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0b409c040 tx=0x7fc0b409b910 comp rx=0 tx=0).stop 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2681076313 shutdown_connections 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a74e360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a7453a0 0x55a18a76c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2681076313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a744a80 0x55a18a76bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2681076313 >> 172.21.15.77:0/2681076313 conn(0x55a18a7410f0 msgr2=0x55a18a753050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2681076313 shutdown_connections 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2681076313 wait complete. 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- start start 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 Processor -- start 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 0x55a18a75c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a75c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a759040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1adfb0 con 0x55a18a7453a0 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1ad8e0 con 0x55a18a744a80 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.057+0000 7fc0c3e08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc1aab20 con 0x55a18a754150 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0c3e08740 10 client.0 _ll_register_callbacks cb 0x7fc0b400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a75c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a759040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a75c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57128/0 (socket says 172.21.15.77:57128) 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0ba7fc640 1 -- 172.21.15.77:0/2676994892 learned_addr learned my addr 172.21.15.77:0/2676994892 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0baffd640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 0x55a18a75c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 msgr2=0x55a18a75c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 0x55a18a75c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2676994892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 msgr2=0x55a18a75c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a75c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 -- 172.21.15.77:0/2676994892 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18a7596d0 con 0x55a18a754150 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0baffd640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 0x55a18a75c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0bb7fe640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a759040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40982e0 tx=0x7fc0b40a03b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e070ab6d084e0cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc09affd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4020070 con 0x55a18a754150 2024-09-15T22:12:34.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b4086040 con 0x55a18a754150 2024-09-15T22:12:34.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.058+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc0b409b2d0 con 0x55a18a754150 2024-09-15T22:12:34.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0b40cd300 con 0x55a18a754150 2024-09-15T22:12:34.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 10 client.24967 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:34.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a18a7cf5a0 con 0x55a18a754150 2024-09-15T22:12:34.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4086040 con 0x55a18a754150 2024-09-15T22:12:34.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc0b409add0 con 0x55a18a754150 2024-09-15T22:12:34.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 10 client.24967 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:34.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 10 client.24967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 20 client.24967 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a7cfab0 0x55a18a7d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 4 client.24967 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a18a6b1d00 con 0x55a18a7cfab0 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.059+0000 7fc0baffd640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a7cfab0 0x55a18a7d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.060+0000 7fc09affd640 10 client.24967 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.060+0000 7fc0baffd640 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a7cfab0 0x55a18a7d1ed0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64157a0febd6f44 server_cookie=7f731929bd278f1e in_seq=0 out_seq=0 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc09affd640 1 -- 172.21.15.77:0/2676994892 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a18a6b1d00 con 0x55a18a7cfab0 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc09affd640 10 client.24967 handle_command_reply: tid=0 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 client.24967 shutdown 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 msgr2=0x55a18a759040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a759040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40982e0 tx=0x7fc0b40a03b0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 shutdown_connections 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a18a7cfab0 0x55a18a7d1ed0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a18a754150 0x55a18a759040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a18a7453a0 0x55a18a75c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 --2- 172.21.15.77:0/2676994892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a18a744a80 0x55a18a75c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 >> 172.21.15.77:0/2676994892 conn(0x55a18a7410f0 msgr2=0x55a18a7510a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc09affd640 0 client.24967 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.061+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 shutdown_connections 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c3e08740 1 -- 172.21.15.77:0/2676994892 wait complete. 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c3e08740 20 client.24967 trim_cache size 0 max 16384 2024-09-15T22:12:34.069 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:12:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "323a0f28-1f8b-479e-9784-09a1fd8582eb", 2024-09-15T22:12:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:12:34.070 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:34.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 msgr2=0x7fc0a80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 0x7fc0a80a5990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc10d7c0 tx=0x7fc0a40046c0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 msgr2=0x7fc0bc19cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009fd0 tx=0x7fc0b400b260 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 shutdown_connections 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc0a80a34d0 0x7fc0a80a5990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0bc10c2f0 0x7fc0bc19cf00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0bc1037a0 0x7fc0bc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 --2- 172.21.15.77:0/738596446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0bc102df0 0x7fc0bc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.062+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 >> 172.21.15.77:0/738596446 conn(0x7fc0bc0fc7d0 msgr2=0x7fc0bc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.063+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 shutdown_connections 2024-09-15T22:12:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.063+0000 7fc0c1a2f640 1 -- 172.21.15.77:0/738596446 wait complete. 2024-09-15T22:12:34.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.163+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f37d81a73a0 con 0x7f37d8100860 2024-09-15T22:12:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.167+0000 7f37c77fe640 1 -- 172.21.15.77:0/4271820199 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f37c80a0af0 con 0x7f37d8100860 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 msgr2=0x7f37c00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 0x7f37c00a5a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37cc000c00 tx=0x7f37cc005b50 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 msgr2=0x7f37d819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f37c800e9f0 tx=0x7f37c800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 shutdown_connections 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f37c00a3580 0x7f37c00a5a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37d810e410 0x7f37d819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d8101210 0x7f37d819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 --2- 172.21.15.77:0/4271820199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37d8100860 0x7f37d819a590 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 >> 172.21.15.77:0/4271820199 conn(0x7f37d8077990 msgr2=0x7f37d80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 shutdown_connections 2024-09-15T22:12:34.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.169+0000 7f37dcf31640 1 -- 172.21.15.77:0/4271820199 wait complete. 2024-09-15T22:12:34.182 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-15T22:12:34.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 -- 172.21.15.77:0/2778028816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c101350 msgr2=0x7fbc5c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 --2- 172.21.15.77:0/2778028816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c101350 0x7fbc5c10d880 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009930 tx=0x7fbc4c02a420 comp rx=0 tx=0).stop 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 -- 172.21.15.77:0/2778028816 shutdown_connections 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 --2- 172.21.15.77:0/2778028816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c101350 0x7fbc5c10d880 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 --2- 172.21.15.77:0/2778028816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 --2- 172.21.15.77:0/2778028816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.285+0000 7fbc612a4640 1 -- 172.21.15.77:0/2778028816 >> 172.21.15.77:0/2778028816 conn(0x7fbc5c077990 msgr2=0x7fbc5c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- 172.21.15.77:0/2778028816 shutdown_connections 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- 172.21.15.77:0/2778028816 wait complete. 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 Processor -- start 2024-09-15T22:12:34.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- start start 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c101350 0x7fbc5c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c1a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c110e00 con 0x7fbc5c1009b0 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c110c80 con 0x7fbc5c101350 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.286+0000 7fbc612a4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c110f80 con 0x7fbc5c1069c0 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57144/0 (socket says 172.21.15.77:57144) 2024-09-15T22:12:34.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 -- 172.21.15.77:0/3298371894 learned_addr learned my addr 172.21.15.77:0/3298371894 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5b7fe640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c1a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5a7fc640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c101350 0x7fbc5c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 -- 172.21.15.77:0/3298371894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 msgr2=0x7fbc5c1a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c1a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 -- 172.21.15.77:0/3298371894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c101350 msgr2=0x7fbc5c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c101350 0x7fbc5c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 -- 172.21.15.77:0/3298371894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c1a4710 con 0x7fbc5c1009b0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5b7fe640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc5affd640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d950 tx=0x7fbc5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54017030 con 0x7fbc5c1009b0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc54004020 con 0x7fbc5c1009b0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540047a0 con 0x7fbc5c1009b0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c1a48e0 con 0x7fbc5c1009b0 2024-09-15T22:12:34.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.287+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c1a4e00 con 0x7fbc5c1009b0 2024-09-15T22:12:34.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.288+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c100e10 con 0x7fbc5c1009b0 2024-09-15T22:12:34.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.289+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fbc540041d0 con 0x7fbc5c1009b0 2024-09-15T22:12:34.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.290+0000 7fbc43fff640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 0x7fbc440a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.290+0000 7fbc5a7fc640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 0x7fbc440a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.290+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fbc540cfc60 con 0x7fbc5c1009b0 2024-09-15T22:12:34.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.291+0000 7fbc5a7fc640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 0x7fbc440a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c06a3f0 tx=0x7fbc50009680 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.295+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc5409c0b0 con 0x7fbc5c1009b0 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1762568586' entity='client.admin' 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3078871559' entity='client.admin' 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4271820199' entity='client.admin' 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.488+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbc5c0ff690 con 0x7fbc440a3580 2024-09-15T22:12:34.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.555+0000 7fbc43fff640 1 -- 172.21.15.77:0/3298371894 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbc5c0ff690 con 0x7fbc440a3580 2024-09-15T22:12:34.557 INFO:teuthology.orchestra.run.smithi077.stdout:Schedule set for path / 2024-09-15T22:12:34.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 msgr2=0x7fbc440a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 0x7fbc440a5a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c06a3f0 tx=0x7fbc50009680 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 msgr2=0x7fbc5c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d950 tx=0x7fbc5400de10 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 shutdown_connections 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc440a3580 0x7fbc440a5a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc5c1069c0 0x7fbc5c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc5c101350 0x7fbc5c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 --2- 172.21.15.77:0/3298371894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc5c1009b0 0x7fbc5c199150 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 >> 172.21.15.77:0/3298371894 conn(0x7fbc5c077990 msgr2=0x7fbc5c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 shutdown_connections 2024-09-15T22:12:34.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.557+0000 7fbc612a4640 1 -- 172.21.15.77:0/3298371894 wait complete. 2024-09-15T22:12:34.569 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1762568586' entity='client.admin' 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3078871559' entity='client.admin' 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4271820199' entity='client.admin' 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 25 op/s 2024-09-15T22:12:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1762568586' entity='client.admin' 2024-09-15T22:12:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3078871559' entity='client.admin' 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4271820199' entity='client.admin' 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:34.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- 172.21.15.77:0/2280499437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644102470 msgr2=0x7f264410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 --2- 172.21.15.77:0/2280499437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644102470 0x7f264410e9a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f263c009cd0 tx=0x7f263c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- 172.21.15.77:0/2280499437 shutdown_connections 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 --2- 172.21.15.77:0/2280499437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644102470 0x7f264410e9a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 --2- 172.21.15.77:0/2280499437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644101ad0 0x7f2644101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 --2- 172.21.15.77:0/2280499437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 0x7f2644107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- 172.21.15.77:0/2280499437 >> 172.21.15.77:0/2280499437 conn(0x7f26440fd7d0 msgr2=0x7f26440ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- 172.21.15.77:0/2280499437 shutdown_connections 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- 172.21.15.77:0/2280499437 wait complete. 2024-09-15T22:12:34.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 Processor -- start 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.671+0000 7f264a4f1640 1 -- start start 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 0x7f2644198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 0x7f264419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2644111f20 con 0x7f2644101ad0 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2644111da0 con 0x7f2644102470 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26441120a0 con 0x7f2644107b70 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57162/0 (socket says 172.21.15.77:57162) 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 -- 172.21.15.77:0/700820443 learned_addr learned my addr 172.21.15.77:0/700820443 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2648a8e640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 0x7f264419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f26437fe640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 0x7f2644198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 -- 172.21.15.77:0/700820443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 msgr2=0x7f264419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 0x7f264419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 -- 172.21.15.77:0/700820443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 msgr2=0x7f2644198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 0x7f2644198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 -- 172.21.15.77:0/700820443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264419d3b0 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f26437fe640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 0x7f2644198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f2643fff640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f263400c970 tx=0x7f263400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634007bb0 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634007d50 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400f450 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.672+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264419d6a0 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.673+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441a4f30 con 0x7f2644101ad0 2024-09-15T22:12:34.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.673+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644101f30 con 0x7f2644101ad0 2024-09-15T22:12:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.675+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f2634005020 con 0x7f2644101ad0 2024-09-15T22:12:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.675+0000 7f26417fa640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 0x7f26300a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.675+0000 7f26437fe640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 0x7f26300a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.675+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f26340d0100 con 0x7f2644101ad0 2024-09-15T22:12:34.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.676+0000 7f26437fe640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 0x7f26300a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2644107170 tx=0x7f262c005e50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:34.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.679+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2634014030 con 0x7f2644101ad0 2024-09-15T22:12:34.849 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:12:34.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.875+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f2644062630 con 0x7f26300a3600 2024-09-15T22:12:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.937+0000 7f26417fa640 1 -- 172.21.15.77:0/700820443 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f2644062630 con 0x7f26300a3600 2024-09-15T22:12:34.939 INFO:teuthology.orchestra.run.smithi077.stdout:Retention added to path / 2024-09-15T22:12:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 msgr2=0x7f26300a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 0x7f26300a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2644107170 tx=0x7f262c005e50 comp rx=0 tx=0).stop 2024-09-15T22:12:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 msgr2=0x7f26441983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f263400c970 tx=0x7f263400ce30 comp rx=0 tx=0).stop 2024-09-15T22:12:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 shutdown_connections 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f26300a3600 0x7f26300a5ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2644107b70 0x7f264419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2644102470 0x7f2644198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 --2- 172.21.15.77:0/700820443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2644101ad0 0x7f26441983c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 >> 172.21.15.77:0/700820443 conn(0x7f26440fd7d0 msgr2=0x7f26440feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 shutdown_connections 2024-09-15T22:12:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:34.939+0000 7f264a4f1640 1 -- 172.21.15.77:0/700820443 wait complete. 2024-09-15T22:12:34.951 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 -- 172.21.15.77:0/4142596456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c101370 msgr2=0x7feb6c10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 --2- 172.21.15.77:0/4142596456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c101370 0x7feb6c10d8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7feb64009cd0 tx=0x7feb6402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 -- 172.21.15.77:0/4142596456 shutdown_connections 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 --2- 172.21.15.77:0/4142596456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c101370 0x7feb6c10d8a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 --2- 172.21.15.77:0/4142596456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c1009d0 0x7feb6c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 --2- 172.21.15.77:0/4142596456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 0x7feb6c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 -- 172.21.15.77:0/4142596456 >> 172.21.15.77:0/4142596456 conn(0x7feb6c0fc6b0 msgr2=0x7feb6c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 -- 172.21.15.77:0/4142596456 shutdown_connections 2024-09-15T22:12:35.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.100+0000 7feb745a6640 1 -- 172.21.15.77:0/4142596456 wait complete. 2024-09-15T22:12:35.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 Processor -- start 2024-09-15T22:12:35.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 -- start start 2024-09-15T22:12:35.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 0x7feb6c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 0x7feb6c1a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c110e20 con 0x7feb6c1009d0 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c110ca0 con 0x7feb6c1069e0 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb745a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c110fa0 con 0x7feb6c101370 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57176/0 (socket says 172.21.15.77:57176) 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 -- 172.21.15.77:0/3849619822 learned_addr learned my addr 172.21.15.77:0/3849619822 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb71b41640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 0x7feb6c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72b43640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 0x7feb6c1a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 -- 172.21.15.77:0/3849619822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 msgr2=0x7feb6c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 0x7feb6c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 -- 172.21.15.77:0/3849619822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 msgr2=0x7feb6c1a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 0x7feb6c1a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb72342640 1 -- 172.21.15.77:0/3849619822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1a4710 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.101+0000 7feb71b41640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 0x7feb6c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb72342640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017030 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68004020 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a48e0 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680047a0 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.102+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a4e00 con 0x7feb6c1009d0 2024-09-15T22:12:35.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.103+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c100e30 con 0x7feb6c1009d0 2024-09-15T22:12:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.104+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7feb680041d0 con 0x7feb6c1009d0 2024-09-15T22:12:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.105+0000 7feb637fe640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 0x7feb580a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.105+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7feb680cfd20 con 0x7feb6c1009d0 2024-09-15T22:12:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.105+0000 7feb71b41640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 0x7feb580a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.106+0000 7feb71b41640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 0x7feb580a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb6c06ac10 tx=0x7feb5c005e50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.109+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb6809c170 con 0x7feb6c1009d0 2024-09-15T22:12:35.112 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.306+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7feb6c0fe460 con 0x7feb580a3580 2024-09-15T22:12:35.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.312+0000 7feb637fe640 1 -- 172.21.15.77:0/3849619822 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7feb6c0fe460 con 0x7feb580a3580 2024-09-15T22:12:35.314 INFO:teuthology.orchestra.run.smithi077.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:12:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-15T22:12:35.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 msgr2=0x7feb580a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 0x7feb580a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb6c06ac10 tx=0x7feb5c005e50 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 msgr2=0x7feb6c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 shutdown_connections 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7feb580a3580 0x7feb580a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb6c1069e0 0x7feb6c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb6c101370 0x7feb6c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 --2- 172.21.15.77:0/3849619822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb6c1009d0 0x7feb6c199150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 >> 172.21.15.77:0/3849619822 conn(0x7feb6c0fc6b0 msgr2=0x7feb6c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 shutdown_connections 2024-09-15T22:12:35.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.315+0000 7feb745a6640 1 -- 172.21.15.77:0/3849619822 wait complete. 2024-09-15T22:12:35.327 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:12:35.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 -- 172.21.15.77:0/2794139447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b78101350 msgr2=0x7f7b7810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 --2- 172.21.15.77:0/2794139447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b78101350 0x7f7b7810d880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009cd0 tx=0x7f7b7002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 -- 172.21.15.77:0/2794139447 shutdown_connections 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 --2- 172.21.15.77:0/2794139447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b78101350 0x7f7b7810d880 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 --2- 172.21.15.77:0/2794139447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b781009b0 0x7f7b78100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 --2- 172.21.15.77:0/2794139447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781069c0 0x7f7b78106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.438+0000 7f7b7d053640 1 -- 172.21.15.77:0/2794139447 >> 172.21.15.77:0/2794139447 conn(0x7f7b78077990 msgr2=0x7f7b780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.439+0000 7f7b7d053640 1 -- 172.21.15.77:0/2794139447 shutdown_connections 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.439+0000 7f7b7d053640 1 -- 172.21.15.77:0/2794139447 wait complete. 2024-09-15T22:12:35.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.439+0000 7f7b7d053640 1 Processor -- start 2024-09-15T22:12:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.439+0000 7f7b7d053640 1 -- start start 2024-09-15T22:12:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781009b0 0x7f7b78198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b78101350 0x7f7b78198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78110e00 con 0x7f7b781069c0 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78110c80 con 0x7f7b781009b0 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78110f80 con 0x7f7b78101350 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57198/0 (socket says 172.21.15.77:57198) 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 -- 172.21.15.77:0/3954528872 learned_addr learned my addr 172.21.15.77:0/3954528872 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b76d9d640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781009b0 0x7f7b78198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 -- 172.21.15.77:0/3954528872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b78101350 msgr2=0x7f7b78198840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b78101350 0x7f7b78198840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 -- 172.21.15.77:0/3954528872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781009b0 msgr2=0x7f7b78198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781009b0 0x7f7b78198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 -- 172.21.15.77:0/3954528872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7819d2f0 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7759e640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005cc0 tx=0x7f7b70003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70004040 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b700046c0 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7819d580 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.440+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70046480 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.441+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b781a4e70 con 0x7f7b781069c0 2024-09-15T22:12:35.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.441+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b78100e10 con 0x7f7b781069c0 2024-09-15T22:12:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.443+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f7b700305a0 con 0x7f7b781069c0 2024-09-15T22:12:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.443+0000 7f7b5ffff640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 0x7f7b600a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.444+0000 7f7b76d9d640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 0x7f7b600a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.444+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7b7002e030 con 0x7f7b781069c0 2024-09-15T22:12:35.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.444+0000 7f7b76d9d640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 0x7f7b600a5ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bb60 tx=0x7f7b6800b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.446+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b7004b060 con 0x7f7b781069c0 2024-09-15T22:12:35.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:35 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:12:35.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:12:35.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:35 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:12:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:12:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:35 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:12:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:12:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:35.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.643+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7b78103640 con 0x7f7b600a3600 2024-09-15T22:12:35.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 -- 172.21.15.77:0/2709716180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85181082a0 msgr2=0x7f851810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 --2- 172.21.15.77:0/2709716180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85181082a0 0x7f851810ec40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8510009cd0 tx=0x7f851002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 -- 172.21.15.77:0/2709716180 shutdown_connections 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 --2- 172.21.15.77:0/2709716180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85181082a0 0x7f851810ec40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 --2- 172.21.15.77:0/2709716180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181036e0 0x7f8518107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 --2- 172.21.15.77:0/2709716180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8518102d30 0x7f8518103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 -- 172.21.15.77:0/2709716180 >> 172.21.15.77:0/2709716180 conn(0x7f85180fc7d0 msgr2=0x7f85180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 -- 172.21.15.77:0/2709716180 shutdown_connections 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.616+0000 7f851f920640 1 -- 172.21.15.77:0/2709716180 wait complete. 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 Processor -- start 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 -- start start 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85181036e0 0x7f851819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181082a0 0x7f851819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85181120a0 con 0x7f8518102d30 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8518111f20 con 0x7f85181082a0 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851f920640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8518112220 con 0x7f85181036e0 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851d6bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851d6bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57210/0 (socket says 172.21.15.77:57210) 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.617+0000 7f851d6bc640 1 -- 172.21.15.77:0/3580972240 learned_addr learned my addr 172.21.15.77:0/3580972240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851debd640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181082a0 0x7f851819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851cebb640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85181036e0 0x7f851819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 -- 172.21.15.77:0/3580972240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85181036e0 msgr2=0x7f851819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85181036e0 0x7f851819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 -- 172.21.15.77:0/3580972240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181082a0 msgr2=0x7f851819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181082a0 0x7f851819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 -- 172.21.15.77:0/3580972240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851819f6f0 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851d6bc640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f851400c920 tx=0x7f851400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f850a7fc640 1 -- 172.21.15.77:0/3580972240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514007c70 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f850a7fc640 1 -- 172.21.15.77:0/3580972240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8514007e10 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851819f9e0 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f850a7fc640 1 -- 172.21.15.77:0/3580972240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514018890 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.618+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181a72c0 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.621+0000 7f8521cf9740 1 Processor -- start 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.621+0000 7f850a7fc640 1 -- 172.21.15.77:0/3580972240 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f8514018a70 con 0x7f8518102d30 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.621+0000 7f8521cf9740 1 -- start start 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d0a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d0ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c671c652e0 con 0x55c671cff4a0 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c671c0cce0 con 0x55c671d0a680 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f8521cf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c671c612a0 con 0x55c671cffdc0 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57212/0 (socket says 172.21.15.77:57212) 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 -- 172.21.15.77:0/87416580 learned_addr learned my addr 172.21.15.77:0/87416580 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851cebb640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d0ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851d6bc640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d0a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f850a7fc640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 0x7f85040a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 -- 172.21.15.77:0/87416580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 msgr2=0x55c671d0a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d0a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 -- 172.21.15.77:0/87416580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 msgr2=0x55c671d0ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d0ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 -- 172.21.15.77:0/87416580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c671bf1440 con 0x55c671cff4a0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851cebb640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d0ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f850a7fc640 1 -- 172.21.15.77:0/3580972240 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f85140d0400 con 0x7f8518102d30 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 0x7f85040a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.622+0000 7f851debd640 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8510035000 tx=0x7f8510031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2455bd4ae0a67f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f85037fe640 1 -- 172.21.15.77:0/87416580 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510049070 con 0x55c671cff4a0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f85037fe640 1 -- 172.21.15.77:0/87416580 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510036070 con 0x55c671cff4a0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f85037fe640 1 -- 172.21.15.77:0/87416580 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100307e0 con 0x55c671cff4a0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 -- 172.21.15.77:0/87416580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 msgr2=0x55c671cff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f851debd640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 0x7f85040a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8510031860 tx=0x7f8510030e90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8510035000 tx=0x7f8510031700 comp rx=0 tx=0).stop 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 -- 172.21.15.77:0/87416580 shutdown_connections 2024-09-15T22:12:35.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d0ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d0a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 --2- 172.21.15.77:0/87416580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671cff880 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.623+0000 7f8521cf9740 1 -- 172.21.15.77:0/87416580 >> 172.21.15.77:0/87416580 conn(0x55c671cfba30 msgr2=0x55c671cfbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- 172.21.15.77:0/87416580 shutdown_connections 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- 172.21.15.77:0/87416580 wait complete. 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- start start 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 Processor -- start 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cff4a0 0x55c671d09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c671d0d250 con 0x55c671cffdc0 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c671c20050 con 0x55c671d0a680 2024-09-15T22:12:35.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c671d0d3d0 con 0x55c671cff4a0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.624+0000 7f8521cf9740 10 client.0 _ll_register_callbacks cb 0x55c671c60160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57224/0 (socket says 172.21.15.77:57224) 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 -- 172.21.15.77:0/2782365711 learned_addr learned my addr 172.21.15.77:0/2782365711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851d6bc640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cff4a0 0x55c671d09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 -- 172.21.15.77:0/2782365711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cff4a0 msgr2=0x55c671d09ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851debd640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cff4a0 0x55c671d09ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 -- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 msgr2=0x55c671d076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 -- 172.21.15.77:0/2782365711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6719d63c0 con 0x55c671cffdc0 2024-09-15T22:12:35.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851debd640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f851cebb640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f85180697d0 tx=0x7f850c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdcc1912edf82f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c01e060 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f850c00fd80 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8510036070 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f850c025050 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8521cf9740 10 client.15513 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.625+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c671d18b80 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510036070 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f850c018030 con 0x55c671cffdc0 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 10 client.15513 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:35.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 20 client.15513 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d1a090 0x55c671d1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 4 client.15513 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c671c72850 con 0x55c671d1a090 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.626+0000 7f851d6bc640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d1a090 0x55c671d1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f851d6bc640 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d1a090 0x55c671d1c4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d2534eaf704988 server_cookie=ec96f23c219a7e29 in_seq=0 out_seq=0 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8500ff9640 10 client.15513 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8500ff9640 1 -- 172.21.15.77:0/2782365711 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c671c72850 con 0x55c671d1a090 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8500ff9640 10 client.15513 handle_command_reply: tid=0 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 client.15513 shutdown 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 msgr2=0x55c671d07180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f85180697d0 tx=0x7f850c011bc0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 shutdown_connections 2024-09-15T22:12:35.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d1a090 0x55c671d1c4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8500ff9640 0 client.15513 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cffdc0 0x55c671d07180 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.627+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2782365711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cff4a0 0x55c671d09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.628+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 >> 172.21.15.77:0/2782365711 conn(0x55c671cfba30 msgr2=0x55c671d01710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.628+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 shutdown_connections 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.628+0000 7f8521cf9740 1 -- 172.21.15.77:0/2782365711 wait complete. 2024-09-15T22:12:35.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.628+0000 7f8521cf9740 20 client.15513 trim_cache size 0 max 16384 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 Processor -- start 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 -- start start 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671cff4a0 0x55c671d164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d21da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b10c0 con 0x55c671d0a680 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b2c00 con 0x55c671cff4a0 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.645+0000 7f8521cf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b1790 con 0x55c671cffdc0 2024-09-15T22:12:35.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57238/0 (socket says 172.21.15.77:57238) 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 -- 172.21.15.77:0/2701001299 learned_addr learned my addr 172.21.15.77:0/2701001299 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851cebb640 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d21da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851d6bc640 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671cff4a0 0x55c671d164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 -- 172.21.15.77:0/2701001299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 msgr2=0x55c671d21da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d21da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 -- 172.21.15.77:0/2701001299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671cff4a0 msgr2=0x55c671d164b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671cff4a0 0x55c671d164b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 -- 172.21.15.77:0/2701001299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c671cfb4d0 con 0x55c671d0a680 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f851debd640 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f851002ae00 tx=0x7f851004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe7e17f616ddcec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f8502ffd640 1 -- 172.21.15.77:0/2701001299 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510061070 con 0x55c671d0a680 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f8502ffd640 1 -- 172.21.15.77:0/2701001299 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510031980 con 0x55c671d0a680 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.646+0000 7f8502ffd640 1 -- 172.21.15.77:0/2701001299 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510049040 con 0x55c671d0a680 2024-09-15T22:12:35.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 -- 172.21.15.77:0/2701001299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 msgr2=0x55c671d222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f851002ae00 tx=0x7f851004f000 comp rx=0 tx=0).stop 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 -- 172.21.15.77:0/2701001299 shutdown_connections 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671d0a680 0x55c671d222e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d21da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 --2- 172.21.15.77:0/2701001299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671cff4a0 0x55c671d164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 -- 172.21.15.77:0/2701001299 >> 172.21.15.77:0/2701001299 conn(0x55c671cfba30 msgr2=0x55c671d06270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 -- 172.21.15.77:0/2701001299 shutdown_connections 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.647+0000 7f8521cf9740 1 -- 172.21.15.77:0/2701001299 wait complete. 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 -- start start 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 Processor -- start 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d10ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b2530 con 0x55c671cff4a0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b1e60 con 0x55c671d0a680 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85181af0a0 con 0x55c671cffdc0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f8521cf9740 10 client.0 _ll_register_callbacks cb 0x7f850c00c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f851d6bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f851d6bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57242/0 (socket says 172.21.15.77:57242) 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f851d6bc640 1 -- 172.21.15.77:0/1054116032 learned_addr learned my addr 172.21.15.77:0/1054116032 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f851cebb640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.648+0000 7f851debd640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d10ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 -- 172.21.15.77:0/1054116032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 msgr2=0x55c671d10980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d10980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 -- 172.21.15.77:0/1054116032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 msgr2=0x55c671d10440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:35.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 -- 172.21.15.77:0/1054116032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c671d83e30 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851cebb640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d10980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851d6bc640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f851debd640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d10ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85100491e0 tx=0x7f85100095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234705db0a8769ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f8500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510061070 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8510044c80 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.649+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8510031980 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8510049040 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 10 client.25043 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c671d85120 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510031980 con 0x55c671d0a680 2024-09-15T22:12:35.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8510042040 con 0x55c671d0a680 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f7b5ffff640 1 -- 172.21.15.77:0/3954528872 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7b78103640 con 0x7f7b600a3600 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 10 client.25043 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 10 client.25043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 20 client.25043 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d85400 0x55c671d877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 4 client.25043 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:12:35.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c671c6cd60 con 0x55c671d85400 2024-09-15T22:12:35.665 INFO:teuthology.orchestra.run.smithi077.stdout:/ 1m 6m3h 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.650+0000 7f851d6bc640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d85400 0x55c671d877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f851d6bc640 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d85400 0x55c671d877f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a3cc1b10ce9192 server_cookie=e64e907b46a872ed in_seq=0 out_seq=0 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8500ff9640 10 client.25043 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8500ff9640 1 -- 172.21.15.77:0/1054116032 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c671c6cd60 con 0x55c671d85400 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8500ff9640 10 client.25043 handle_command_reply: tid=0 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 client.25043 shutdown 2024-09-15T22:12:35.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 msgr2=0x55c671d10ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d10ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85100491e0 tx=0x7f85100095b0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 shutdown_connections 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c671d85400 0x55c671d877f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8500ff9640 0 client.25043 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c671d0a680 0x55c671d10ec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c671cffdc0 0x55c671d10980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 --2- 172.21.15.77:0/1054116032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c671cff4a0 0x55c671d10440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 >> 172.21.15.77:0/1054116032 conn(0x55c671cfba30 msgr2=0x55c671d089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 shutdown_connections 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 1 -- 172.21.15.77:0/1054116032 wait complete. 2024-09-15T22:12:35.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.651+0000 7f8521cf9740 20 client.25043 trim_cache size 0 max 16384 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "323a0f28-1f8b-479e-9784-09a1fd8582eb": { 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "323a0f28-1f8b-479e-9784-09a1fd8582eb", 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:12:35.668 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 msgr2=0x7f85040a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 0x7f85040a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8510031860 tx=0x7f8510030e90 comp rx=0 tx=0).stop 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 msgr2=0x7f851819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f851400c920 tx=0x7f851400cde0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 shutdown_connections 2024-09-15T22:12:35.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85040a3550 0x7f85040a5a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85181082a0 0x7f851819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85181036e0 0x7f851819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 --2- 172.21.15.77:0/3580972240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8518102d30 0x7f851819a740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 >> 172.21.15.77:0/3580972240 conn(0x7f85180fc7d0 msgr2=0x7f85180fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 shutdown_connections 2024-09-15T22:12:35.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f851f920640 1 -- 172.21.15.77:0/3580972240 wait complete. 2024-09-15T22:12:35.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 msgr2=0x7f7b600a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 0x7f7b600a5ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800bb60 tx=0x7f7b6800b390 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 msgr2=0x7f7b7819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.652+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005cc0 tx=0x7f7b70003970 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 shutdown_connections 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b600a3600 0x7f7b600a5ac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b781069c0 0x7f7b7819cbc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b78101350 0x7f7b78198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 --2- 172.21.15.77:0/3954528872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b781009b0 0x7f7b78198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 >> 172.21.15.77:0/3954528872 conn(0x7f7b78077990 msgr2=0x7f7b780ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 shutdown_connections 2024-09-15T22:12:35.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:35.653+0000 7f7b7d053640 1 -- 172.21.15.77:0/3954528872 wait complete. 2024-09-15T22:12:35.673 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-15T22:12:35.699 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-15T22:12:35.711 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-15T22:12:35.724 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:12:35.725 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:12:36.371 INFO:tasks.cephfs.filesystem:scrub status for tag:323a0f28-1f8b-479e-9784-09a1fd8582eb - {'path': '/', 'tag': '323a0f28-1f8b-479e-9784-09a1fd8582eb', 'options': 'recursive,force'} 2024-09-15T22:12:36.417 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:36.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:36 smithi077 ceph-mon[30980]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-15T22:12:36.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:36 smithi077 ceph-mon[30980]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:36 smithi077 ceph-mon[30980]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.539 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:36 smithi077 ceph-mon[30980]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:36 smithi196 ceph-mon[35109]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-15T22:12:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:36 smithi196 ceph-mon[35109]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:36 smithi196 ceph-mon[35109]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:36 smithi196 ceph-mon[35109]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:36 smithi117 ceph-mon[35021]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 20 op/s 2024-09-15T22:12:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:36 smithi117 ceph-mon[35021]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:36 smithi117 ceph-mon[35021]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:36 smithi117 ceph-mon[35021]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:36.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 -- 172.21.15.77:0/772683276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c101420 msgr2=0x7fa48c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 --2- 172.21.15.77:0/772683276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c101420 0x7fa48c10d950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa484009cd0 tx=0x7fa48402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 -- 172.21.15.77:0/772683276 shutdown_connections 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 --2- 172.21.15.77:0/772683276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c101420 0x7fa48c10d950 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 --2- 172.21.15.77:0/772683276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c100a80 0x7fa48c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 --2- 172.21.15.77:0/772683276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c106a80 0x7fa48c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 -- 172.21.15.77:0/772683276 >> 172.21.15.77:0/772683276 conn(0x7fa48c0fc7a0 msgr2=0x7fa48c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 -- 172.21.15.77:0/772683276 shutdown_connections 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.915+0000 7fa4928d7640 1 -- 172.21.15.77:0/772683276 wait complete. 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 Processor -- start 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 -- start start 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100a80 0x7fa48c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:36.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 0x7fa48c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c110ea0 con 0x7fa48c106a80 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c110d20 con 0x7fa48c100a80 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa4928d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c111020 con 0x7fa48c101420 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa490e74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa490e74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57258/0 (socket says 172.21.15.77:57258) 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.916+0000 7fa490e74640 1 -- 172.21.15.77:0/4115106705 learned_addr learned my addr 172.21.15.77:0/4115106705 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa48b7fe640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 0x7fa48c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa48bfff640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100a80 0x7fa48c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 -- 172.21.15.77:0/4115106705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 msgr2=0x7fa48c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 0x7fa48c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 -- 172.21.15.77:0/4115106705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100a80 msgr2=0x7fa48c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100a80 0x7fa48c197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 -- 172.21.15.77:0/4115106705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c19c310 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa48b7fe640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 0x7fa48c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa490e74640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa484006010 tx=0x7fa484003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484004450 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484003e10 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c19c5a0 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840466e0 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.917+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1a3ee0 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.918+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c100ee0 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.919+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fa4840305a0 con 0x7fa48c106a80 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.920+0000 7fa4897fa640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 0x7fa4740a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:36.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.920+0000 7fa48bfff640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 0x7fa4740a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:36.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.920+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa48402e030 con 0x7fa48c106a80 2024-09-15T22:12:36.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.921+0000 7fa48bfff640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 0x7fa4740a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0058c0 tx=0x7fa47c005c60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:36.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:36.924+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa48404b060 con 0x7fa48c106a80 2024-09-15T22:12:37.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.088+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa48c0fd670 con 0x7fa4740a3550 2024-09-15T22:12:37.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.098+0000 7fa4897fa640 1 -- 172.21.15.77:0/4115106705 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339754 (secure 0 0 0) 0x7fa48c0fd670 con 0x7fa4740a3550 2024-09-15T22:12:37.103 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:37.105 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:37.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 msgr2=0x7fa4740a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:37.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 0x7fa4740a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0058c0 tx=0x7fa47c005c60 comp rx=0 tx=0).stop 2024-09-15T22:12:37.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 msgr2=0x7fa48c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:37.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa484006010 tx=0x7fa484003c20 comp rx=0 tx=0).stop 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 shutdown_connections 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4740a3550 0x7fa4740a5a10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa48c106a80 0x7fa48c19bc30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa48c101420 0x7fa48c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 --2- 172.21.15.77:0/4115106705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48c100a80 0x7fa48c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 >> 172.21.15.77:0/4115106705 conn(0x7fa48c0fc7a0 msgr2=0x7fa48c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 shutdown_connections 2024-09-15T22:12:37.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:37.103+0000 7fa4928d7640 1 -- 172.21.15.77:0/4115106705 wait complete. 2024-09-15T22:12:37.475 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:37 smithi077 ceph-mon[30980]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:37 smithi196 ceph-mon[35109]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:37 smithi117 ceph-mon[35021]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:12:37.837 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-15T22:12:36.404526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393659,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1794,"num_read_kb":1332,"num_write":716,"num_write_kb":5336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1042,"ondisk_log_size":1042,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382632,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559192,"statfs":{"total":1151940427776,"available":1151548612608,"internally_reserved":0,"allocated":12861440,"data_stored":10129715,"data_compressed":517820,"data_compressed_allocated":700416,"data_compressed_original":4571136,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":448,"num_read_kb":488,"num_write":149,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.990179"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.061527+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:34.061527+0000","last_peered":"2024-09-15T22:12:34.061527+0000","last_clean":"2024-09-15T22:12:34.061527+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:34.061527+0000","last_undegraded":"2024-09-15T22:12:34.061527+0000","last_fullsized":"2024-09-15T22:12:34.061527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395110+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395110+0000","last_peered":"2024-09-15T22:12:24.395110+0000","last_clean":"2024-09-15T22:12:24.395110+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395110+0000","last_undegraded":"2024-09-15T22:12:24.395110+0000","last_fullsized":"2024-09-15T22:12:24.395110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393935+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393935+0000","last_peered":"2024-09-15T22:12:24.393935+0000","last_clean":"2024-09-15T22:12:24.393935+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393935+0000","last_undegraded":"2024-09-15T22:12:24.393935+0000","last_fullsized":"2024-09-15T22:12:24.393935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394229+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394229+0000","last_peered":"2024-09-15T22:12:24.394229+0000","last_clean":"2024-09-15T22:12:24.394229+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394229+0000","last_undegraded":"2024-09-15T22:12:24.394229+0000","last_fullsized":"2024-09-15T22:12:24.394229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493306+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493306+0000","last_peered":"2024-09-15T22:12:24.493306+0000","last_clean":"2024-09-15T22:12:24.493306+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493306+0000","last_undegraded":"2024-09-15T22:12:24.493306+0000","last_fullsized":"2024-09-15T22:12:24.493306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394012+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.394012+0000","last_peered":"2024-09-15T22:12:24.394012+0000","last_clean":"2024-09-15T22:12:24.394012+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.394012+0000","last_undegraded":"2024-09-15T22:12:24.394012+0000","last_fullsized":"2024-09-15T22:12:24.394012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648484+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648484+0000","last_peered":"2024-09-15T22:12:24.648484+0000","last_clean":"2024-09-15T22:12:24.648484+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648484+0000","last_undegraded":"2024-09-15T22:12:24.648484+0000","last_fullsized":"2024-09-15T22:12:24.648484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'104","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:35.311498+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:35.311498+0000","last_peered":"2024-09-15T22:12:35.311498+0000","last_clean":"2024-09-15T22:12:35.311498+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:35.311498+0000","last_undegraded":"2024-09-15T22:12:35.311498+0000","last_fullsized":"2024-09-15T22:12:35.311498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":189,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.901010+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:33.901010+0000","last_peered":"2024-09-15T22:12:33.901010+0000","last_clean":"2024-09-15T22:12:33.901010+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:33.901010+0000","last_undegraded":"2024-09-15T22:12:33.901010+0000","last_fullsized":"2024-09-15T22:12:33.901010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846185+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:33.846185+0000","last_peered":"2024-09-15T22:12:33.846185+0000","last_clean":"2024-09-15T22:12:33.846185+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:33.846185+0000","last_undegraded":"2024-09-15T22:12:33.846185+0000","last_fullsized":"2024-09-15T22:12:33.846185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:26.487136+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:26.487136+0000","last_peered":"2024-09-15T22:12:26.487136+0000","last_clean":"2024-09-15T22:12:26.487136+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:26.487136+0000","last_undegraded":"2024-09-15T22:12:26.487136+0000","last_fullsized":"2024-09-15T22:12:26.487136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847400+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:33.847400+0000","last_peered":"2024-09-15T22:12:33.847400+0000","last_clean":"2024-09-15T22:12:33.847400+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:33.847400+0000","last_undegraded":"2024-09-15T22:12:33.847400+0000","last_fullsized":"2024-09-15T22:12:33.847400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.935624+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:34.935624+0000","last_peered":"2024-09-15T22:12:34.935624+0000","last_clean":"2024-09-15T22:12:34.935624+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:34.935624+0000","last_undegraded":"2024-09-15T22:12:34.935624+0000","last_fullsized":"2024-09-15T22:12:34.935624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846673+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:33.846673+0000","last_peered":"2024-09-15T22:12:33.846673+0000","last_clean":"2024-09-15T22:12:33.846673+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:33.846673+0000","last_undegraded":"2024-09-15T22:12:33.846673+0000","last_fullsized":"2024-09-15T22:12:33.846673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478827+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478827+0000","last_peered":"2024-09-15T22:12:24.478827+0000","last_clean":"2024-09-15T22:12:24.478827+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478827+0000","last_undegraded":"2024-09-15T22:12:24.478827+0000","last_fullsized":"2024-09-15T22:12:24.478827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'43","reported_seq":329,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848165+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:34.848165+0000","last_peered":"2024-09-15T22:12:34.848165+0000","last_clean":"2024-09-15T22:12:34.848165+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:34.848165+0000","last_undegraded":"2024-09-15T22:12:34.848165+0000","last_fullsized":"2024-09-15T22:12:34.848165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":196,"num_write":45,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":279,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848263+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:34.848263+0000","last_peered":"2024-09-15T22:12:34.848263+0000","last_clean":"2024-09-15T22:12:34.848263+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:34.848263+0000","last_undegraded":"2024-09-15T22:12:34.848263+0000","last_fullsized":"2024-09-15T22:12:34.848263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395325+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395325+0000","last_peered":"2024-09-15T22:12:24.395325+0000","last_clean":"2024-09-15T22:12:24.395325+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395325+0000","last_undegraded":"2024-09-15T22:12:24.395325+0000","last_fullsized":"2024-09-15T22:12:24.395325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847490+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:33.847490+0000","last_peered":"2024-09-15T22:12:33.847490+0000","last_clean":"2024-09-15T22:12:33.847490+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:33.847490+0000","last_undegraded":"2024-09-15T22:12:33.847490+0000","last_fullsized":"2024-09-15T22:12:33.847490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395722+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395722+0000","last_peered":"2024-09-15T22:12:24.395722+0000","last_clean":"2024-09-15T22:12:24.395722+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395722+0000","last_undegraded":"2024-09-15T22:12:24.395722+0000","last_fullsized":"2024-09-15T22:12:24.395722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":803057,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1557,"num_read_kb":1002,"num_write":513,"num_write_kb":2396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2409171,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514542,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":516162,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1212416,"data_stored":1502017,"data_compressed":77321,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45684,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45670,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364865,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":1435800,"data_compressed":76959,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1034688,"data_compressed":51697,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":585281,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":583685,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512940,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331894,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":402207,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:37.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:12:38.088 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:38.498 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:38 smithi077 ceph-mon[30980]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-15T22:12:38.498 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:38 smithi077 ceph-mon[30980]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:38 smithi196 ceph-mon[35109]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-15T22:12:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:38 smithi196 ceph-mon[35109]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:38.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 -- 172.21.15.77:0/2505363319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a6810c170 msgr2=0x7f3a6810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/2505363319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a6810c170 0x7f3a6810e610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009ca0 tx=0x7f3a6002a950 comp rx=0 tx=0).stop 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 -- 172.21.15.77:0/2505363319 shutdown_connections 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/2505363319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a6810c170 0x7f3a6810e610 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/2505363319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a68103660 0x7f3a6810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/2505363319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 0x7f3a68103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 -- 172.21.15.77:0/2505363319 >> 172.21.15.77:0/2505363319 conn(0x7f3a680fc7a0 msgr2=0x7f3a680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 -- 172.21.15.77:0/2505363319 shutdown_connections 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.575+0000 7f3a6dec8640 1 -- 172.21.15.77:0/2505363319 wait complete. 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 Processor -- start 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 -- start start 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 0x7f3a68198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:38.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6810c170 0x7f3a6819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111f70 con 0x7f3a68103660 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111df0 con 0x7f3a6810c170 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a6dec8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a681120f0 con 0x7f3a68102d40 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a66ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a66ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57268/0 (socket says 172.21.15.77:57268) 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a66ffd640 1 -- 172.21.15.77:0/671216614 learned_addr learned my addr 172.21.15.77:0/671216614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a677fe640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 0x7f3a68198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.576+0000 7f3a67fff640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6810c170 0x7f3a6819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 -- 172.21.15.77:0/671216614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 msgr2=0x7f3a68198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 0x7f3a68198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 -- 172.21.15.77:0/671216614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6810c170 msgr2=0x7f3a6819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6810c170 0x7f3a6819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 -- 172.21.15.77:0/671216614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6819d3c0 con 0x7f3a68103660 2024-09-15T22:12:38.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a66ffd640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800d740 tx=0x7f3a5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a58017070 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6819d6b0 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5800de50 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800f620 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.577+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a4f90 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.578+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a68103120 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.579+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3a58009c30 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.580+0000 7f3a64ff9640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 0x7f3a500a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.580+0000 7f3a677fe640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 0x7f3a500a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.580+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f3a580cfe40 con 0x7f3a68103660 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.580+0000 7f3a677fe640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 0x7f3a500a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c005fd0 tx=0x7f3a5c00b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:38.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.583+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a5809bc40 con 0x7f3a68103660 2024-09-15T22:12:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:38 smithi117 ceph-mon[35021]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-15T22:12:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:38 smithi117 ceph-mon[35021]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:38.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.746+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a680feac0 con 0x7f3a500a34d0 2024-09-15T22:12:38.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.754+0000 7f3a64ff9640 1 -- 172.21.15.77:0/671216614 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339755 (secure 0 0 0) 0x7f3a680feac0 con 0x7f3a500a34d0 2024-09-15T22:12:38.758 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:38.759 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:38.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 msgr2=0x7f3a500a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:38.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 0x7f3a500a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c005fd0 tx=0x7f3a5c00b390 comp rx=0 tx=0).stop 2024-09-15T22:12:38.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 msgr2=0x7f3a68198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:38.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800d740 tx=0x7f3a5800dc00 comp rx=0 tx=0).stop 2024-09-15T22:12:38.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 shutdown_connections 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a500a34d0 0x7f3a500a5990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a6810c170 0x7f3a6819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a68103660 0x7f3a68198950 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 --2- 172.21.15.77:0/671216614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68102d40 0x7f3a68198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 >> 172.21.15.77:0/671216614 conn(0x7f3a680fc7a0 msgr2=0x7f3a680fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 shutdown_connections 2024-09-15T22:12:38.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:38.757+0000 7f3a6dec8640 1 -- 172.21.15.77:0/671216614 wait complete. 2024-09-15T22:12:39.527 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-15T22:12:38.405124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393659,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1794,"num_read_kb":1332,"num_write":716,"num_write_kb":5336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1042,"ondisk_log_size":1042,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382632,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559192,"statfs":{"total":1151940427776,"available":1151548612608,"internally_reserved":0,"allocated":12861440,"data_stored":10129715,"data_compressed":517820,"data_compressed_allocated":700416,"data_compressed_original":4571136,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":448,"num_read_kb":488,"num_write":149,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002823"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.061527+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:34.061527+0000","last_peered":"2024-09-15T22:12:34.061527+0000","last_clean":"2024-09-15T22:12:34.061527+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:34.061527+0000","last_undegraded":"2024-09-15T22:12:34.061527+0000","last_fullsized":"2024-09-15T22:12:34.061527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395110+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395110+0000","last_peered":"2024-09-15T22:12:24.395110+0000","last_clean":"2024-09-15T22:12:24.395110+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395110+0000","last_undegraded":"2024-09-15T22:12:24.395110+0000","last_fullsized":"2024-09-15T22:12:24.395110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393935+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393935+0000","last_peered":"2024-09-15T22:12:24.393935+0000","last_clean":"2024-09-15T22:12:24.393935+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393935+0000","last_undegraded":"2024-09-15T22:12:24.393935+0000","last_fullsized":"2024-09-15T22:12:24.393935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394229+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394229+0000","last_peered":"2024-09-15T22:12:24.394229+0000","last_clean":"2024-09-15T22:12:24.394229+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394229+0000","last_undegraded":"2024-09-15T22:12:24.394229+0000","last_fullsized":"2024-09-15T22:12:24.394229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493306+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493306+0000","last_peered":"2024-09-15T22:12:24.493306+0000","last_clean":"2024-09-15T22:12:24.493306+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493306+0000","last_undegraded":"2024-09-15T22:12:24.493306+0000","last_fullsized":"2024-09-15T22:12:24.493306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394012+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.394012+0000","last_peered":"2024-09-15T22:12:24.394012+0000","last_clean":"2024-09-15T22:12:24.394012+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.394012+0000","last_undegraded":"2024-09-15T22:12:24.394012+0000","last_fullsized":"2024-09-15T22:12:24.394012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648484+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648484+0000","last_peered":"2024-09-15T22:12:24.648484+0000","last_clean":"2024-09-15T22:12:24.648484+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648484+0000","last_undegraded":"2024-09-15T22:12:24.648484+0000","last_fullsized":"2024-09-15T22:12:24.648484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'104","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:35.311498+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:35.311498+0000","last_peered":"2024-09-15T22:12:35.311498+0000","last_clean":"2024-09-15T22:12:35.311498+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:35.311498+0000","last_undegraded":"2024-09-15T22:12:35.311498+0000","last_fullsized":"2024-09-15T22:12:35.311498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":189,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.901010+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:33.901010+0000","last_peered":"2024-09-15T22:12:33.901010+0000","last_clean":"2024-09-15T22:12:33.901010+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:33.901010+0000","last_undegraded":"2024-09-15T22:12:33.901010+0000","last_fullsized":"2024-09-15T22:12:33.901010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846185+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:33.846185+0000","last_peered":"2024-09-15T22:12:33.846185+0000","last_clean":"2024-09-15T22:12:33.846185+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:33.846185+0000","last_undegraded":"2024-09-15T22:12:33.846185+0000","last_fullsized":"2024-09-15T22:12:33.846185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:26.487136+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:26.487136+0000","last_peered":"2024-09-15T22:12:26.487136+0000","last_clean":"2024-09-15T22:12:26.487136+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:26.487136+0000","last_undegraded":"2024-09-15T22:12:26.487136+0000","last_fullsized":"2024-09-15T22:12:26.487136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847400+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:33.847400+0000","last_peered":"2024-09-15T22:12:33.847400+0000","last_clean":"2024-09-15T22:12:33.847400+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:33.847400+0000","last_undegraded":"2024-09-15T22:12:33.847400+0000","last_fullsized":"2024-09-15T22:12:33.847400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.935624+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:34.935624+0000","last_peered":"2024-09-15T22:12:34.935624+0000","last_clean":"2024-09-15T22:12:34.935624+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:34.935624+0000","last_undegraded":"2024-09-15T22:12:34.935624+0000","last_fullsized":"2024-09-15T22:12:34.935624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846673+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:33.846673+0000","last_peered":"2024-09-15T22:12:33.846673+0000","last_clean":"2024-09-15T22:12:33.846673+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:33.846673+0000","last_undegraded":"2024-09-15T22:12:33.846673+0000","last_fullsized":"2024-09-15T22:12:33.846673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478827+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478827+0000","last_peered":"2024-09-15T22:12:24.478827+0000","last_clean":"2024-09-15T22:12:24.478827+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478827+0000","last_undegraded":"2024-09-15T22:12:24.478827+0000","last_fullsized":"2024-09-15T22:12:24.478827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'43","reported_seq":329,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848165+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:34.848165+0000","last_peered":"2024-09-15T22:12:34.848165+0000","last_clean":"2024-09-15T22:12:34.848165+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:34.848165+0000","last_undegraded":"2024-09-15T22:12:34.848165+0000","last_fullsized":"2024-09-15T22:12:34.848165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":196,"num_write":45,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":279,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848263+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:34.848263+0000","last_peered":"2024-09-15T22:12:34.848263+0000","last_clean":"2024-09-15T22:12:34.848263+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:34.848263+0000","last_undegraded":"2024-09-15T22:12:34.848263+0000","last_fullsized":"2024-09-15T22:12:34.848263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395325+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395325+0000","last_peered":"2024-09-15T22:12:24.395325+0000","last_clean":"2024-09-15T22:12:24.395325+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395325+0000","last_undegraded":"2024-09-15T22:12:24.395325+0000","last_fullsized":"2024-09-15T22:12:24.395325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847490+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:33.847490+0000","last_peered":"2024-09-15T22:12:33.847490+0000","last_clean":"2024-09-15T22:12:33.847490+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:33.847490+0000","last_undegraded":"2024-09-15T22:12:33.847490+0000","last_fullsized":"2024-09-15T22:12:33.847490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395722+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395722+0000","last_peered":"2024-09-15T22:12:24.395722+0000","last_clean":"2024-09-15T22:12:24.395722+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395722+0000","last_undegraded":"2024-09-15T22:12:24.395722+0000","last_fullsized":"2024-09-15T22:12:24.395722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":803057,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1557,"num_read_kb":1002,"num_write":513,"num_write_kb":2396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2409171,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514542,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":516162,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1212416,"data_stored":1502017,"data_compressed":77321,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45684,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45670,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364865,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":1435800,"data_compressed":76959,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1034688,"data_compressed":51697,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":585281,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":583685,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512940,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331894,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":402207,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:39.532 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:12:39.533 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:12:39.777 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 -- 172.21.15.77:0/1482587817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074eb0 msgr2=0x7f0538112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 --2- 172.21.15.77:0/1482587817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074eb0 0x7f0538112650 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0528009900 tx=0x7f052802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 -- 172.21.15.77:0/1482587817 shutdown_connections 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 --2- 172.21.15.77:0/1482587817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f0538114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 --2- 172.21.15.77:0/1482587817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074eb0 0x7f0538112650 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 --2- 172.21.15.77:0/1482587817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074590 0x7f0538074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.289+0000 7f053f7a2640 1 -- 172.21.15.77:0/1482587817 >> 172.21.15.77:0/1482587817 conn(0x7f05380fd8c0 msgr2=0x7f05380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- 172.21.15.77:0/1482587817 shutdown_connections 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- 172.21.15.77:0/1482587817 wait complete. 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 Processor -- start 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- start start 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 0x7f05381a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f053819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05381185c0 con 0x7f0538074590 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0538118440 con 0x7f0538112b90 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053f7a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0538118740 con 0x7f0538074eb0 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053d53e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.290+0000 7f053d53e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56016/0 (socket says 172.21.15.77:56016) 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 -- 172.21.15.77:0/3892606457 learned_addr learned my addr 172.21.15.77:0/3892606457 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053cd3d640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 0x7f05381a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 -- 172.21.15.77:0/3892606457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 msgr2=0x7f05381a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053dd3f640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f053819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 0x7f05381a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 -- 172.21.15.77:0/3892606457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 msgr2=0x7f053819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f053819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 -- 172.21.15.77:0/3892606457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053819f060 con 0x7f0538074590 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053cd3d640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 0x7f05381a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053dd3f640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f053819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053d53e640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f053400b4a0 tx=0x7f053400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534014070 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053819f350 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0534010af0 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534017d10 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.291+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381ab640 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.292+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0538074970 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.293+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f0534017070 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.294+0000 7f052e7fc640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 0x7f05240a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.294+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f05340cf6e0 con 0x7f0538074590 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.294+0000 7f053cd3d640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 0x7f05240a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.294+0000 7f053cd3d640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 0x7f05240a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528032d50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.298+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f053409bbe0 con 0x7f0538074590 2024-09-15T22:12:40.468 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:40 smithi077 ceph-mon[30980]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:40.468 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:40 smithi077 ceph-mon[30980]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.466+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0538062630 con 0x7f05240a34d0 2024-09-15T22:12:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.474+0000 7f052e7fc640 1 -- 172.21.15.77:0/3892606457 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339755 (secure 0 0 0) 0x7f0538062630 con 0x7f05240a34d0 2024-09-15T22:12:40.478 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 msgr2=0x7f05240a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 0x7f05240a5990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528032d50 comp rx=0 tx=0).stop 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 msgr2=0x7f05381a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f053400b4a0 tx=0x7f053400b960 comp rx=0 tx=0).stop 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 shutdown_connections 2024-09-15T22:12:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f05240a34d0 0x7f05240a5990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538112b90 0x7f053819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0538074eb0 0x7f05381a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 --2- 172.21.15.77:0/3892606457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0538074590 0x7f05381a4770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 >> 172.21.15.77:0/3892606457 conn(0x7f05380fd8c0 msgr2=0x7f05380ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 shutdown_connections 2024-09-15T22:12:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:40.477+0000 7f053f7a2640 1 -- 172.21.15.77:0/3892606457 wait complete. 2024-09-15T22:12:40.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:40 smithi196 ceph-mon[35109]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:40.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:40 smithi196 ceph-mon[35109]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:40 smithi117 ceph-mon[35021]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:40 smithi117 ceph-mon[35021]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:41.243 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-15T22:12:40.405461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1393659,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1794,"num_read_kb":1332,"num_write":716,"num_write_kb":5336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1042,"ondisk_log_size":1042,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382632,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559192,"statfs":{"total":1151940427776,"available":1151548612608,"internally_reserved":0,"allocated":12861440,"data_stored":10129715,"data_compressed":517820,"data_compressed_allocated":700416,"data_compressed_original":4571136,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":448,"num_read_kb":488,"num_write":149,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002795"},"pg_stats":[{"pgid":"2.1f","version":"91'14","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.061527+0000","last_change":"2024-09-15T22:09:53.190106+0000","last_active":"2024-09-15T22:12:34.061527+0000","last_peered":"2024-09-15T22:12:34.061527+0000","last_clean":"2024-09-15T22:12:34.061527+0000","last_became_active":"2024-09-15T22:09:53.189802+0000","last_became_peered":"2024-09-15T22:09:53.189802+0000","last_unstale":"2024-09-15T22:12:34.061527+0000","last_undegraded":"2024-09-15T22:12:34.061527+0000","last_fullsized":"2024-09-15T22:12:34.061527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:31.908347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485375+0000","last_change":"2024-09-15T22:09:55.213036+0000","last_active":"2024-09-15T22:12:24.485375+0000","last_peered":"2024-09-15T22:12:24.485375+0000","last_clean":"2024-09-15T22:12:24.485375+0000","last_became_active":"2024-09-15T22:09:55.212702+0000","last_became_peered":"2024-09-15T22:09:55.212702+0000","last_unstale":"2024-09-15T22:12:24.485375+0000","last_undegraded":"2024-09-15T22:12:24.485375+0000","last_fullsized":"2024-09-15T22:12:24.485375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:28.496735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395110+0000","last_change":"2024-09-15T22:10:02.241074+0000","last_active":"2024-09-15T22:12:24.395110+0000","last_peered":"2024-09-15T22:12:24.395110+0000","last_clean":"2024-09-15T22:12:24.395110+0000","last_became_active":"2024-09-15T22:10:02.240676+0000","last_became_peered":"2024-09-15T22:10:02.240676+0000","last_unstale":"2024-09-15T22:12:24.395110+0000","last_undegraded":"2024-09-15T22:12:24.395110+0000","last_fullsized":"2024-09-15T22:12:24.395110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:42.500775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485745+0000","last_change":"2024-09-15T22:09:53.776236+0000","last_active":"2024-09-15T22:12:24.485745+0000","last_peered":"2024-09-15T22:12:24.485745+0000","last_clean":"2024-09-15T22:12:24.485745+0000","last_became_active":"2024-09-15T22:09:53.775845+0000","last_became_peered":"2024-09-15T22:09:53.775845+0000","last_unstale":"2024-09-15T22:12:24.485745+0000","last_undegraded":"2024-09-15T22:12:24.485745+0000","last_fullsized":"2024-09-15T22:12:24.485745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:39.128701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568332+0000","last_change":"2024-09-15T22:09:55.209335+0000","last_active":"2024-09-15T22:12:24.568332+0000","last_peered":"2024-09-15T22:12:24.568332+0000","last_clean":"2024-09-15T22:12:24.568332+0000","last_became_active":"2024-09-15T22:09:55.208848+0000","last_became_peered":"2024-09-15T22:09:55.208848+0000","last_unstale":"2024-09-15T22:12:24.568332+0000","last_undegraded":"2024-09-15T22:12:24.568332+0000","last_fullsized":"2024-09-15T22:12:24.568332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:05.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.393935+0000","last_change":"2024-09-15T22:10:02.237233+0000","last_active":"2024-09-15T22:12:24.393935+0000","last_peered":"2024-09-15T22:12:24.393935+0000","last_clean":"2024-09-15T22:12:24.393935+0000","last_became_active":"2024-09-15T22:10:02.236717+0000","last_became_peered":"2024-09-15T22:10:02.236717+0000","last_unstale":"2024-09-15T22:12:24.393935+0000","last_undegraded":"2024-09-15T22:12:24.393935+0000","last_fullsized":"2024-09-15T22:12:24.393935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:52.216902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'13","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.699151+0000","last_change":"2024-09-15T22:09:53.194459+0000","last_active":"2024-09-15T22:12:24.699151+0000","last_peered":"2024-09-15T22:12:24.699151+0000","last_clean":"2024-09-15T22:12:24.699151+0000","last_became_active":"2024-09-15T22:09:53.194154+0000","last_became_peered":"2024-09-15T22:09:53.194154+0000","last_unstale":"2024-09-15T22:12:24.699151+0000","last_undegraded":"2024-09-15T22:12:24.699151+0000","last_fullsized":"2024-09-15T22:12:24.699151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:51.586298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644481+0000","last_change":"2024-09-15T22:09:55.213081+0000","last_active":"2024-09-15T22:12:24.644481+0000","last_peered":"2024-09-15T22:12:24.644481+0000","last_clean":"2024-09-15T22:12:24.644481+0000","last_became_active":"2024-09-15T22:09:55.212741+0000","last_became_peered":"2024-09-15T22:09:55.212741+0000","last_unstale":"2024-09-15T22:12:24.644481+0000","last_undegraded":"2024-09-15T22:12:24.644481+0000","last_fullsized":"2024-09-15T22:12:24.644481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:19.886310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394229+0000","last_change":"2024-09-15T22:10:02.239019+0000","last_active":"2024-09-15T22:12:24.394229+0000","last_peered":"2024-09-15T22:12:24.394229+0000","last_clean":"2024-09-15T22:12:24.394229+0000","last_became_active":"2024-09-15T22:10:02.237557+0000","last_became_peered":"2024-09-15T22:10:02.237557+0000","last_unstale":"2024-09-15T22:12:24.394229+0000","last_undegraded":"2024-09-15T22:12:24.394229+0000","last_fullsized":"2024-09-15T22:12:24.394229+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:58.111531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391750+0000","last_change":"2024-09-15T22:09:53.191410+0000","last_active":"2024-09-15T22:12:24.391750+0000","last_peered":"2024-09-15T22:12:24.391750+0000","last_clean":"2024-09-15T22:12:24.391750+0000","last_became_active":"2024-09-15T22:09:53.190993+0000","last_became_peered":"2024-09-15T22:09:53.190993+0000","last_unstale":"2024-09-15T22:12:24.391750+0000","last_undegraded":"2024-09-15T22:12:24.391750+0000","last_fullsized":"2024-09-15T22:12:24.391750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:22.642717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493306+0000","last_change":"2024-09-15T22:11:05.239018+0000","last_active":"2024-09-15T22:12:24.493306+0000","last_peered":"2024-09-15T22:12:24.493306+0000","last_clean":"2024-09-15T22:12:24.493306+0000","last_became_active":"2024-09-15T22:11:05.238597+0000","last_became_peered":"2024-09-15T22:11:05.238597+0000","last_unstale":"2024-09-15T22:12:24.493306+0000","last_undegraded":"2024-09-15T22:12:24.493306+0000","last_fullsized":"2024-09-15T22:12:24.493306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:47.724829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394012+0000","last_change":"2024-09-15T22:10:02.238051+0000","last_active":"2024-09-15T22:12:24.394012+0000","last_peered":"2024-09-15T22:12:24.394012+0000","last_clean":"2024-09-15T22:12:24.394012+0000","last_became_active":"2024-09-15T22:10:02.236295+0000","last_became_peered":"2024-09-15T22:10:02.236295+0000","last_unstale":"2024-09-15T22:12:24.394012+0000","last_undegraded":"2024-09-15T22:12:24.394012+0000","last_fullsized":"2024-09-15T22:12:24.394012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:00.114581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642726+0000","last_change":"2024-09-15T22:09:55.213438+0000","last_active":"2024-09-15T22:12:24.642726+0000","last_peered":"2024-09-15T22:12:24.642726+0000","last_clean":"2024-09-15T22:12:24.642726+0000","last_became_active":"2024-09-15T22:09:55.213058+0000","last_became_peered":"2024-09-15T22:09:55.213058+0000","last_unstale":"2024-09-15T22:12:24.642726+0000","last_undegraded":"2024-09-15T22:12:24.642726+0000","last_fullsized":"2024-09-15T22:12:24.642726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:20.659781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391492+0000","last_change":"2024-09-15T22:09:53.193515+0000","last_active":"2024-09-15T22:12:24.391492+0000","last_peered":"2024-09-15T22:12:24.391492+0000","last_clean":"2024-09-15T22:12:24.391492+0000","last_became_active":"2024-09-15T22:09:53.192884+0000","last_became_peered":"2024-09-15T22:09:53.192884+0000","last_unstale":"2024-09-15T22:12:24.391492+0000","last_undegraded":"2024-09-15T22:12:24.391492+0000","last_fullsized":"2024-09-15T22:12:24.391492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:54.424546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394186+0000","last_change":"2024-09-15T22:10:02.758904+0000","last_active":"2024-09-15T22:12:24.394186+0000","last_peered":"2024-09-15T22:12:24.394186+0000","last_clean":"2024-09-15T22:12:24.394186+0000","last_became_active":"2024-09-15T22:10:02.758406+0000","last_became_peered":"2024-09-15T22:10:02.758406+0000","last_unstale":"2024-09-15T22:12:24.394186+0000","last_undegraded":"2024-09-15T22:12:24.394186+0000","last_fullsized":"2024-09-15T22:12:24.394186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:18.392458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484999+0000","last_change":"2024-09-15T22:09:55.201404+0000","last_active":"2024-09-15T22:12:24.484999+0000","last_peered":"2024-09-15T22:12:24.484999+0000","last_clean":"2024-09-15T22:12:24.484999+0000","last_became_active":"2024-09-15T22:09:55.201104+0000","last_became_peered":"2024-09-15T22:09:55.201104+0000","last_unstale":"2024-09-15T22:12:24.484999+0000","last_undegraded":"2024-09-15T22:12:24.484999+0000","last_fullsized":"2024-09-15T22:12:24.484999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:01.183401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648484+0000","last_change":"2024-09-15T22:11:05.246212+0000","last_active":"2024-09-15T22:12:24.648484+0000","last_peered":"2024-09-15T22:12:24.648484+0000","last_clean":"2024-09-15T22:12:24.648484+0000","last_became_active":"2024-09-15T22:11:05.239714+0000","last_became_peered":"2024-09-15T22:11:05.239714+0000","last_unstale":"2024-09-15T22:12:24.648484+0000","last_undegraded":"2024-09-15T22:12:24.648484+0000","last_fullsized":"2024-09-15T22:12:24.648484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:47.524177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394180+0000","last_change":"2024-09-15T22:10:02.236114+0000","last_active":"2024-09-15T22:12:24.394180+0000","last_peered":"2024-09-15T22:12:24.394180+0000","last_clean":"2024-09-15T22:12:24.394180+0000","last_became_active":"2024-09-15T22:10:02.235363+0000","last_became_peered":"2024-09-15T22:10:02.235363+0000","last_unstale":"2024-09-15T22:12:24.394180+0000","last_undegraded":"2024-09-15T22:12:24.394180+0000","last_fullsized":"2024-09-15T22:12:24.394180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.425222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648527+0000","last_change":"2024-09-15T22:09:55.200525+0000","last_active":"2024-09-15T22:12:24.648527+0000","last_peered":"2024-09-15T22:12:24.648527+0000","last_clean":"2024-09-15T22:12:24.648527+0000","last_became_active":"2024-09-15T22:09:55.200188+0000","last_became_peered":"2024-09-15T22:09:55.200188+0000","last_unstale":"2024-09-15T22:12:24.648527+0000","last_undegraded":"2024-09-15T22:12:24.648527+0000","last_fullsized":"2024-09-15T22:12:24.648527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:02.029797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484253+0000","last_change":"2024-09-15T22:09:53.188396+0000","last_active":"2024-09-15T22:12:24.484253+0000","last_peered":"2024-09-15T22:12:24.484253+0000","last_clean":"2024-09-15T22:12:24.484253+0000","last_became_active":"2024-09-15T22:09:53.187995+0000","last_became_peered":"2024-09-15T22:09:53.187995+0000","last_unstale":"2024-09-15T22:12:24.484253+0000","last_undegraded":"2024-09-15T22:12:24.484253+0000","last_fullsized":"2024-09-15T22:12:24.484253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:15.463475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391654+0000","last_change":"2024-09-15T22:11:05.242465+0000","last_active":"2024-09-15T22:12:24.391654+0000","last_peered":"2024-09-15T22:12:24.391654+0000","last_clean":"2024-09-15T22:12:24.391654+0000","last_became_active":"2024-09-15T22:11:05.241772+0000","last_became_peered":"2024-09-15T22:11:05.241772+0000","last_unstale":"2024-09-15T22:12:24.391654+0000","last_undegraded":"2024-09-15T22:12:24.391654+0000","last_fullsized":"2024-09-15T22:12:24.391654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:52.687479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391313+0000","last_change":"2024-09-15T22:09:55.199948+0000","last_active":"2024-09-15T22:12:24.391313+0000","last_peered":"2024-09-15T22:12:24.391313+0000","last_clean":"2024-09-15T22:12:24.391313+0000","last_became_active":"2024-09-15T22:09:55.199592+0000","last_became_peered":"2024-09-15T22:09:55.199592+0000","last_unstale":"2024-09-15T22:12:24.391313+0000","last_undegraded":"2024-09-15T22:12:24.391313+0000","last_fullsized":"2024-09-15T22:12:24.391313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:49.229703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644451+0000","last_change":"2024-09-15T22:09:53.776002+0000","last_active":"2024-09-15T22:12:24.644451+0000","last_peered":"2024-09-15T22:12:24.644451+0000","last_clean":"2024-09-15T22:12:24.644451+0000","last_became_active":"2024-09-15T22:09:53.775684+0000","last_became_peered":"2024-09-15T22:09:53.775684+0000","last_unstale":"2024-09-15T22:12:24.644451+0000","last_undegraded":"2024-09-15T22:12:24.644451+0000","last_fullsized":"2024-09-15T22:12:24.644451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:20.054868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394204+0000","last_change":"2024-09-15T22:10:02.759849+0000","last_active":"2024-09-15T22:12:24.394204+0000","last_peered":"2024-09-15T22:12:24.394204+0000","last_clean":"2024-09-15T22:12:24.394204+0000","last_became_active":"2024-09-15T22:10:02.759407+0000","last_became_peered":"2024-09-15T22:10:02.759407+0000","last_unstale":"2024-09-15T22:12:24.394204+0000","last_undegraded":"2024-09-15T22:12:24.394204+0000","last_fullsized":"2024-09-15T22:12:24.394204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:19.363399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567847+0000","last_change":"2024-09-15T22:09:55.210342+0000","last_active":"2024-09-15T22:12:24.567847+0000","last_peered":"2024-09-15T22:12:24.567847+0000","last_clean":"2024-09-15T22:12:24.567847+0000","last_became_active":"2024-09-15T22:09:55.210011+0000","last_became_peered":"2024-09-15T22:09:55.210011+0000","last_unstale":"2024-09-15T22:12:24.567847+0000","last_undegraded":"2024-09-15T22:12:24.567847+0000","last_fullsized":"2024-09-15T22:12:24.567847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:41.859135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394477+0000","last_change":"2024-09-15T22:09:53.188505+0000","last_active":"2024-09-15T22:12:24.394477+0000","last_peered":"2024-09-15T22:12:24.394477+0000","last_clean":"2024-09-15T22:12:24.394477+0000","last_became_active":"2024-09-15T22:09:53.188117+0000","last_became_peered":"2024-09-15T22:09:53.188117+0000","last_unstale":"2024-09-15T22:12:24.394477+0000","last_undegraded":"2024-09-15T22:12:24.394477+0000","last_fullsized":"2024-09-15T22:12:24.394477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:51.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395131+0000","last_change":"2024-09-15T22:10:02.236957+0000","last_active":"2024-09-15T22:12:24.395131+0000","last_peered":"2024-09-15T22:12:24.395131+0000","last_clean":"2024-09-15T22:12:24.395131+0000","last_became_active":"2024-09-15T22:10:02.236454+0000","last_became_peered":"2024-09-15T22:10:02.236454+0000","last_unstale":"2024-09-15T22:12:24.395131+0000","last_undegraded":"2024-09-15T22:12:24.395131+0000","last_fullsized":"2024-09-15T22:12:24.395131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:25.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485864+0000","last_change":"2024-09-15T22:09:55.205360+0000","last_active":"2024-09-15T22:12:24.485864+0000","last_peered":"2024-09-15T22:12:24.485864+0000","last_clean":"2024-09-15T22:12:24.485864+0000","last_became_active":"2024-09-15T22:09:55.205038+0000","last_became_peered":"2024-09-15T22:09:55.205038+0000","last_unstale":"2024-09-15T22:12:24.485864+0000","last_undegraded":"2024-09-15T22:12:24.485864+0000","last_fullsized":"2024-09-15T22:12:24.485864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:47.154443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'104","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:35.311498+0000","last_change":"2024-09-15T22:09:53.188822+0000","last_active":"2024-09-15T22:12:35.311498+0000","last_peered":"2024-09-15T22:12:35.311498+0000","last_clean":"2024-09-15T22:12:35.311498+0000","last_became_active":"2024-09-15T22:09:53.188503+0000","last_became_peered":"2024-09-15T22:09:53.188503+0000","last_unstale":"2024-09-15T22:12:35.311498+0000","last_undegraded":"2024-09-15T22:12:35.311498+0000","last_fullsized":"2024-09-15T22:12:35.311498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:40.708647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":189,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483614+0000","last_change":"2024-09-15T22:10:02.759787+0000","last_active":"2024-09-15T22:12:24.483614+0000","last_peered":"2024-09-15T22:12:24.483614+0000","last_clean":"2024-09-15T22:12:24.483614+0000","last_became_active":"2024-09-15T22:10:02.758207+0000","last_became_peered":"2024-09-15T22:10:02.758207+0000","last_unstale":"2024-09-15T22:12:24.483614+0000","last_undegraded":"2024-09-15T22:12:24.483614+0000","last_fullsized":"2024-09-15T22:12:24.483614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:39.602100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564710+0000","last_change":"2024-09-15T22:09:55.209617+0000","last_active":"2024-09-15T22:12:24.564710+0000","last_peered":"2024-09-15T22:12:24.564710+0000","last_clean":"2024-09-15T22:12:24.564710+0000","last_became_active":"2024-09-15T22:09:55.209103+0000","last_became_peered":"2024-09-15T22:09:55.209103+0000","last_unstale":"2024-09-15T22:12:24.564710+0000","last_undegraded":"2024-09-15T22:12:24.564710+0000","last_fullsized":"2024-09-15T22:12:24.564710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:49.153446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391839+0000","last_change":"2024-09-15T22:09:53.777578+0000","last_active":"2024-09-15T22:12:24.391839+0000","last_peered":"2024-09-15T22:12:24.391839+0000","last_clean":"2024-09-15T22:12:24.391839+0000","last_became_active":"2024-09-15T22:09:53.777130+0000","last_became_peered":"2024-09-15T22:09:53.777130+0000","last_unstale":"2024-09-15T22:12:24.391839+0000","last_undegraded":"2024-09-15T22:12:24.391839+0000","last_fullsized":"2024-09-15T22:12:24.391839+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:55.700341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648554+0000","last_change":"2024-09-15T22:11:05.238076+0000","last_active":"2024-09-15T22:12:24.648554+0000","last_peered":"2024-09-15T22:12:24.648554+0000","last_clean":"2024-09-15T22:12:24.648554+0000","last_became_active":"2024-09-15T22:11:05.237551+0000","last_became_peered":"2024-09-15T22:11:05.237551+0000","last_unstale":"2024-09-15T22:12:24.648554+0000","last_undegraded":"2024-09-15T22:12:24.648554+0000","last_fullsized":"2024-09-15T22:12:24.648554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:38.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495939+0000","last_change":"2024-09-15T22:09:55.201822+0000","last_active":"2024-09-15T22:12:24.495939+0000","last_peered":"2024-09-15T22:12:24.495939+0000","last_clean":"2024-09-15T22:12:24.495939+0000","last_became_active":"2024-09-15T22:09:55.200385+0000","last_became_peered":"2024-09-15T22:09:55.200385+0000","last_unstale":"2024-09-15T22:12:24.495939+0000","last_undegraded":"2024-09-15T22:12:24.495939+0000","last_fullsized":"2024-09-15T22:12:24.495939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:01.007486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478660+0000","last_change":"2024-09-15T22:09:53.776966+0000","last_active":"2024-09-15T22:12:24.478660+0000","last_peered":"2024-09-15T22:12:24.478660+0000","last_clean":"2024-09-15T22:12:24.478660+0000","last_became_active":"2024-09-15T22:09:53.776526+0000","last_became_peered":"2024-09-15T22:09:53.776526+0000","last_unstale":"2024-09-15T22:12:24.478660+0000","last_undegraded":"2024-09-15T22:12:24.478660+0000","last_fullsized":"2024-09-15T22:12:24.478660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:41.933757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391300+0000","last_change":"2024-09-15T22:10:02.758920+0000","last_active":"2024-09-15T22:12:24.391300+0000","last_peered":"2024-09-15T22:12:24.391300+0000","last_clean":"2024-09-15T22:12:24.391300+0000","last_became_active":"2024-09-15T22:10:02.758357+0000","last_became_peered":"2024-09-15T22:10:02.758357+0000","last_unstale":"2024-09-15T22:12:24.391300+0000","last_undegraded":"2024-09-15T22:12:24.391300+0000","last_fullsized":"2024-09-15T22:12:24.391300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:43.498678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495847+0000","last_change":"2024-09-15T22:11:05.294432+0000","last_active":"2024-09-15T22:12:24.495847+0000","last_peered":"2024-09-15T22:12:24.495847+0000","last_clean":"2024-09-15T22:12:24.495847+0000","last_became_active":"2024-09-15T22:11:05.293955+0000","last_became_peered":"2024-09-15T22:11:05.293955+0000","last_unstale":"2024-09-15T22:12:24.495847+0000","last_undegraded":"2024-09-15T22:12:24.495847+0000","last_fullsized":"2024-09-15T22:12:24.495847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:13.895827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'7","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.901010+0000","last_change":"2024-09-15T22:09:53.194429+0000","last_active":"2024-09-15T22:12:33.901010+0000","last_peered":"2024-09-15T22:12:33.901010+0000","last_clean":"2024-09-15T22:12:33.901010+0000","last_became_active":"2024-09-15T22:09:53.193870+0000","last_became_peered":"2024-09-15T22:09:53.193870+0000","last_unstale":"2024-09-15T22:12:33.901010+0000","last_undegraded":"2024-09-15T22:12:33.901010+0000","last_fullsized":"2024-09-15T22:12:33.901010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:39.643675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564433+0000","last_change":"2024-09-15T22:10:02.236551+0000","last_active":"2024-09-15T22:12:24.564433+0000","last_peered":"2024-09-15T22:12:24.564433+0000","last_clean":"2024-09-15T22:12:24.564433+0000","last_became_active":"2024-09-15T22:10:02.236155+0000","last_became_peered":"2024-09-15T22:10:02.236155+0000","last_unstale":"2024-09-15T22:12:24.564433+0000","last_undegraded":"2024-09-15T22:12:24.564433+0000","last_fullsized":"2024-09-15T22:12:24.564433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:41.512403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492888+0000","last_change":"2024-09-15T22:09:55.211716+0000","last_active":"2024-09-15T22:12:24.492888+0000","last_peered":"2024-09-15T22:12:24.492888+0000","last_clean":"2024-09-15T22:12:24.492888+0000","last_became_active":"2024-09-15T22:09:55.210908+0000","last_became_peered":"2024-09-15T22:09:55.210908+0000","last_unstale":"2024-09-15T22:12:24.492888+0000","last_undegraded":"2024-09-15T22:12:24.492888+0000","last_fullsized":"2024-09-15T22:12:24.492888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846185+0000","last_change":"2024-09-15T22:09:53.776282+0000","last_active":"2024-09-15T22:12:33.846185+0000","last_peered":"2024-09-15T22:12:33.846185+0000","last_clean":"2024-09-15T22:12:33.846185+0000","last_became_active":"2024-09-15T22:09:53.775894+0000","last_became_peered":"2024-09-15T22:09:53.775894+0000","last_unstale":"2024-09-15T22:12:33.846185+0000","last_undegraded":"2024-09-15T22:12:33.846185+0000","last_fullsized":"2024-09-15T22:12:33.846185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:15.501952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.565010+0000","last_change":"2024-09-15T22:10:02.759034+0000","last_active":"2024-09-15T22:12:24.565010+0000","last_peered":"2024-09-15T22:12:24.565010+0000","last_clean":"2024-09-15T22:12:24.565010+0000","last_became_active":"2024-09-15T22:10:02.758617+0000","last_became_peered":"2024-09-15T22:10:02.758617+0000","last_unstale":"2024-09-15T22:12:24.565010+0000","last_undegraded":"2024-09-15T22:12:24.565010+0000","last_fullsized":"2024-09-15T22:12:24.565010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:31.035125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564081+0000","last_change":"2024-09-15T22:09:55.208734+0000","last_active":"2024-09-15T22:12:24.564081+0000","last_peered":"2024-09-15T22:12:24.564081+0000","last_clean":"2024-09-15T22:12:24.564081+0000","last_became_active":"2024-09-15T22:09:55.208201+0000","last_became_peered":"2024-09-15T22:09:55.208201+0000","last_unstale":"2024-09-15T22:12:24.564081+0000","last_undegraded":"2024-09-15T22:12:24.564081+0000","last_fullsized":"2024-09-15T22:12:24.564081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:04.972652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642579+0000","last_change":"2024-09-15T22:09:53.191680+0000","last_active":"2024-09-15T22:12:24.642579+0000","last_peered":"2024-09-15T22:12:24.642579+0000","last_clean":"2024-09-15T22:12:24.642579+0000","last_became_active":"2024-09-15T22:09:53.191205+0000","last_became_peered":"2024-09-15T22:09:53.191205+0000","last_unstale":"2024-09-15T22:12:24.642579+0000","last_undegraded":"2024-09-15T22:12:24.642579+0000","last_fullsized":"2024-09-15T22:12:24.642579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:42:17.912948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493107+0000","last_change":"2024-09-15T22:10:02.758819+0000","last_active":"2024-09-15T22:12:24.493107+0000","last_peered":"2024-09-15T22:12:24.493107+0000","last_clean":"2024-09-15T22:12:24.493107+0000","last_became_active":"2024-09-15T22:10:02.758337+0000","last_became_peered":"2024-09-15T22:10:02.758337+0000","last_unstale":"2024-09-15T22:12:24.493107+0000","last_undegraded":"2024-09-15T22:12:24.493107+0000","last_fullsized":"2024-09-15T22:12:24.493107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:31.982826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484645+0000","last_change":"2024-09-15T22:09:55.212930+0000","last_active":"2024-09-15T22:12:24.484645+0000","last_peered":"2024-09-15T22:12:24.484645+0000","last_clean":"2024-09-15T22:12:24.484645+0000","last_became_active":"2024-09-15T22:09:55.212341+0000","last_became_peered":"2024-09-15T22:09:55.212341+0000","last_unstale":"2024-09-15T22:12:24.484645+0000","last_undegraded":"2024-09-15T22:12:24.484645+0000","last_fullsized":"2024-09-15T22:12:24.484645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:44.275113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394284+0000","last_change":"2024-09-15T22:09:53.190037+0000","last_active":"2024-09-15T22:12:24.394284+0000","last_peered":"2024-09-15T22:12:24.394284+0000","last_clean":"2024-09-15T22:12:24.394284+0000","last_became_active":"2024-09-15T22:09:53.189659+0000","last_became_peered":"2024-09-15T22:09:53.189659+0000","last_unstale":"2024-09-15T22:12:24.394284+0000","last_undegraded":"2024-09-15T22:12:24.394284+0000","last_fullsized":"2024-09-15T22:12:24.394284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:14.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:26.487136+0000","last_change":"2024-09-15T22:08:41.396410+0000","last_active":"2024-09-15T22:12:26.487136+0000","last_peered":"2024-09-15T22:12:26.487136+0000","last_clean":"2024-09-15T22:12:26.487136+0000","last_became_active":"2024-09-15T22:08:41.395994+0000","last_became_peered":"2024-09-15T22:08:41.395994+0000","last_unstale":"2024-09-15T22:12:26.487136+0000","last_undegraded":"2024-09-15T22:12:26.487136+0000","last_fullsized":"2024-09-15T22:12:26.487136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:05:16.920610+0000","last_clean_scrub_stamp":"2024-09-15T22:05:16.920610+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:53:39.692002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642970+0000","last_change":"2024-09-15T22:10:02.240072+0000","last_active":"2024-09-15T22:12:24.642970+0000","last_peered":"2024-09-15T22:12:24.642970+0000","last_clean":"2024-09-15T22:12:24.642970+0000","last_became_active":"2024-09-15T22:10:02.239420+0000","last_became_peered":"2024-09-15T22:10:02.239420+0000","last_unstale":"2024-09-15T22:12:24.642970+0000","last_undegraded":"2024-09-15T22:12:24.642970+0000","last_fullsized":"2024-09-15T22:12:24.642970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:18.883273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.213880+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.212980+0000","last_became_peered":"2024-09-15T22:09:55.212980+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:07.185485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847400+0000","last_change":"2024-09-15T22:09:53.777715+0000","last_active":"2024-09-15T22:12:33.847400+0000","last_peered":"2024-09-15T22:12:33.847400+0000","last_clean":"2024-09-15T22:12:33.847400+0000","last_became_active":"2024-09-15T22:09:53.777185+0000","last_became_peered":"2024-09-15T22:09:53.777185+0000","last_unstale":"2024-09-15T22:12:33.847400+0000","last_undegraded":"2024-09-15T22:12:33.847400+0000","last_fullsized":"2024-09-15T22:12:33.847400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:18.267832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644238+0000","last_change":"2024-09-15T22:10:02.235299+0000","last_active":"2024-09-15T22:12:24.644238+0000","last_peered":"2024-09-15T22:12:24.644238+0000","last_clean":"2024-09-15T22:12:24.644238+0000","last_became_active":"2024-09-15T22:10:02.234876+0000","last_became_peered":"2024-09-15T22:10:02.234876+0000","last_unstale":"2024-09-15T22:12:24.644238+0000","last_undegraded":"2024-09-15T22:12:24.644238+0000","last_fullsized":"2024-09-15T22:12:24.644238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:01.296171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642966+0000","last_change":"2024-09-15T22:11:05.294394+0000","last_active":"2024-09-15T22:12:24.642966+0000","last_peered":"2024-09-15T22:12:24.642966+0000","last_clean":"2024-09-15T22:12:24.642966+0000","last_became_active":"2024-09-15T22:11:05.293947+0000","last_became_peered":"2024-09-15T22:11:05.293947+0000","last_unstale":"2024-09-15T22:12:24.642966+0000","last_undegraded":"2024-09-15T22:12:24.642966+0000","last_fullsized":"2024-09-15T22:12:24.642966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:07.780986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.935624+0000","last_change":"2024-09-15T22:09:53.190186+0000","last_active":"2024-09-15T22:12:34.935624+0000","last_peered":"2024-09-15T22:12:34.935624+0000","last_clean":"2024-09-15T22:12:34.935624+0000","last_became_active":"2024-09-15T22:09:53.189773+0000","last_became_peered":"2024-09-15T22:09:53.189773+0000","last_unstale":"2024-09-15T22:12:34.935624+0000","last_undegraded":"2024-09-15T22:12:34.935624+0000","last_fullsized":"2024-09-15T22:12:34.935624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:39.899389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567071+0000","last_change":"2024-09-15T22:10:02.240595+0000","last_active":"2024-09-15T22:12:24.567071+0000","last_peered":"2024-09-15T22:12:24.567071+0000","last_clean":"2024-09-15T22:12:24.567071+0000","last_became_active":"2024-09-15T22:10:02.240229+0000","last_became_peered":"2024-09-15T22:10:02.240229+0000","last_unstale":"2024-09-15T22:12:24.567071+0000","last_undegraded":"2024-09-15T22:12:24.567071+0000","last_fullsized":"2024-09-15T22:12:24.567071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:27.462266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643063+0000","last_change":"2024-09-15T22:09:55.204801+0000","last_active":"2024-09-15T22:12:24.643063+0000","last_peered":"2024-09-15T22:12:24.643063+0000","last_clean":"2024-09-15T22:12:24.643063+0000","last_became_active":"2024-09-15T22:09:55.204439+0000","last_became_peered":"2024-09-15T22:09:55.204439+0000","last_unstale":"2024-09-15T22:12:24.643063+0000","last_undegraded":"2024-09-15T22:12:24.643063+0000","last_fullsized":"2024-09-15T22:12:24.643063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:15.568326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564885+0000","last_change":"2024-09-15T22:09:53.190703+0000","last_active":"2024-09-15T22:12:24.564885+0000","last_peered":"2024-09-15T22:12:24.564885+0000","last_clean":"2024-09-15T22:12:24.564885+0000","last_became_active":"2024-09-15T22:09:53.190407+0000","last_became_peered":"2024-09-15T22:09:53.190407+0000","last_unstale":"2024-09-15T22:12:24.564885+0000","last_undegraded":"2024-09-15T22:12:24.564885+0000","last_fullsized":"2024-09-15T22:12:24.564885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:27.069910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492671+0000","last_change":"2024-09-15T22:10:02.240325+0000","last_active":"2024-09-15T22:12:24.492671+0000","last_peered":"2024-09-15T22:12:24.492671+0000","last_clean":"2024-09-15T22:12:24.492671+0000","last_became_active":"2024-09-15T22:10:02.239868+0000","last_became_peered":"2024-09-15T22:10:02.239868+0000","last_unstale":"2024-09-15T22:12:24.492671+0000","last_undegraded":"2024-09-15T22:12:24.492671+0000","last_fullsized":"2024-09-15T22:12:24.492671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:12.410832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648775+0000","last_change":"2024-09-15T22:09:55.214768+0000","last_active":"2024-09-15T22:12:24.648775+0000","last_peered":"2024-09-15T22:12:24.648775+0000","last_clean":"2024-09-15T22:12:24.648775+0000","last_became_active":"2024-09-15T22:09:55.214479+0000","last_became_peered":"2024-09-15T22:09:55.214479+0000","last_unstale":"2024-09-15T22:12:24.648775+0000","last_undegraded":"2024-09-15T22:12:24.648775+0000","last_fullsized":"2024-09-15T22:12:24.648775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:53.225474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391758+0000","last_change":"2024-09-15T22:09:53.191376+0000","last_active":"2024-09-15T22:12:24.391758+0000","last_peered":"2024-09-15T22:12:24.391758+0000","last_clean":"2024-09-15T22:12:24.391758+0000","last_became_active":"2024-09-15T22:09:53.190959+0000","last_became_peered":"2024-09-15T22:09:53.190959+0000","last_unstale":"2024-09-15T22:12:24.391758+0000","last_undegraded":"2024-09-15T22:12:24.391758+0000","last_fullsized":"2024-09-15T22:12:24.391758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:33:44.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492781+0000","last_change":"2024-09-15T22:10:02.237126+0000","last_active":"2024-09-15T22:12:24.492781+0000","last_peered":"2024-09-15T22:12:24.492781+0000","last_clean":"2024-09-15T22:12:24.492781+0000","last_became_active":"2024-09-15T22:10:02.236308+0000","last_became_peered":"2024-09-15T22:10:02.236308+0000","last_unstale":"2024-09-15T22:12:24.492781+0000","last_undegraded":"2024-09-15T22:12:24.492781+0000","last_fullsized":"2024-09-15T22:12:24.492781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:57:31.607277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492803+0000","last_change":"2024-09-15T22:09:55.219144+0000","last_active":"2024-09-15T22:12:24.492803+0000","last_peered":"2024-09-15T22:12:24.492803+0000","last_clean":"2024-09-15T22:12:24.492803+0000","last_became_active":"2024-09-15T22:09:55.218807+0000","last_became_peered":"2024-09-15T22:09:55.218807+0000","last_unstale":"2024-09-15T22:12:24.492803+0000","last_undegraded":"2024-09-15T22:12:24.492803+0000","last_fullsized":"2024-09-15T22:12:24.492803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:13.599526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648365+0000","last_change":"2024-09-15T22:11:05.256328+0000","last_active":"2024-09-15T22:12:24.648365+0000","last_peered":"2024-09-15T22:12:24.648365+0000","last_clean":"2024-09-15T22:12:24.648365+0000","last_became_active":"2024-09-15T22:11:05.241950+0000","last_became_peered":"2024-09-15T22:11:05.241950+0000","last_unstale":"2024-09-15T22:12:24.648365+0000","last_undegraded":"2024-09-15T22:12:24.648365+0000","last_fullsized":"2024-09-15T22:12:24.648365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:36.917928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564834+0000","last_change":"2024-09-15T22:10:02.238543+0000","last_active":"2024-09-15T22:12:24.564834+0000","last_peered":"2024-09-15T22:12:24.564834+0000","last_clean":"2024-09-15T22:12:24.564834+0000","last_became_active":"2024-09-15T22:10:02.238152+0000","last_became_peered":"2024-09-15T22:10:02.238152+0000","last_unstale":"2024-09-15T22:12:24.564834+0000","last_undegraded":"2024-09-15T22:12:24.564834+0000","last_fullsized":"2024-09-15T22:12:24.564834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:42:50.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642469+0000","last_change":"2024-09-15T22:09:55.216986+0000","last_active":"2024-09-15T22:12:24.642469+0000","last_peered":"2024-09-15T22:12:24.642469+0000","last_clean":"2024-09-15T22:12:24.642469+0000","last_became_active":"2024-09-15T22:09:55.216671+0000","last_became_peered":"2024-09-15T22:09:55.216671+0000","last_unstale":"2024-09-15T22:12:24.642469+0000","last_undegraded":"2024-09-15T22:12:24.642469+0000","last_fullsized":"2024-09-15T22:12:24.642469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:49:56.151077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.846673+0000","last_change":"2024-09-15T22:09:53.190565+0000","last_active":"2024-09-15T22:12:33.846673+0000","last_peered":"2024-09-15T22:12:33.846673+0000","last_clean":"2024-09-15T22:12:33.846673+0000","last_became_active":"2024-09-15T22:09:53.190156+0000","last_became_peered":"2024-09-15T22:09:53.190156+0000","last_unstale":"2024-09-15T22:12:33.846673+0000","last_undegraded":"2024-09-15T22:12:33.846673+0000","last_fullsized":"2024-09-15T22:12:33.846673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:50.239078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478827+0000","last_change":"2024-09-15T22:10:02.238321+0000","last_active":"2024-09-15T22:12:24.478827+0000","last_peered":"2024-09-15T22:12:24.478827+0000","last_clean":"2024-09-15T22:12:24.478827+0000","last_became_active":"2024-09-15T22:10:02.237945+0000","last_became_peered":"2024-09-15T22:10:02.237945+0000","last_unstale":"2024-09-15T22:12:24.478827+0000","last_undegraded":"2024-09-15T22:12:24.478827+0000","last_fullsized":"2024-09-15T22:12:24.478827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:59.286389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642704+0000","last_change":"2024-09-15T22:09:55.210452+0000","last_active":"2024-09-15T22:12:24.642704+0000","last_peered":"2024-09-15T22:12:24.642704+0000","last_clean":"2024-09-15T22:12:24.642704+0000","last_became_active":"2024-09-15T22:09:55.209929+0000","last_became_peered":"2024-09-15T22:09:55.209929+0000","last_unstale":"2024-09-15T22:12:24.642704+0000","last_undegraded":"2024-09-15T22:12:24.642704+0000","last_fullsized":"2024-09-15T22:12:24.642704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:38.271463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492973+0000","last_change":"2024-09-15T22:09:53.193759+0000","last_active":"2024-09-15T22:12:24.492973+0000","last_peered":"2024-09-15T22:12:24.492973+0000","last_clean":"2024-09-15T22:12:24.492973+0000","last_became_active":"2024-09-15T22:09:53.193220+0000","last_became_peered":"2024-09-15T22:09:53.193220+0000","last_unstale":"2024-09-15T22:12:24.492973+0000","last_undegraded":"2024-09-15T22:12:24.492973+0000","last_fullsized":"2024-09-15T22:12:24.492973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:08.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648716+0000","last_change":"2024-09-15T22:10:02.243076+0000","last_active":"2024-09-15T22:12:24.648716+0000","last_peered":"2024-09-15T22:12:24.648716+0000","last_clean":"2024-09-15T22:12:24.648716+0000","last_became_active":"2024-09-15T22:10:02.242603+0000","last_became_peered":"2024-09-15T22:10:02.242603+0000","last_unstale":"2024-09-15T22:12:24.648716+0000","last_undegraded":"2024-09-15T22:12:24.648716+0000","last_fullsized":"2024-09-15T22:12:24.648716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:54.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564217+0000","last_change":"2024-09-15T22:09:55.203669+0000","last_active":"2024-09-15T22:12:24.564217+0000","last_peered":"2024-09-15T22:12:24.564217+0000","last_clean":"2024-09-15T22:12:24.564217+0000","last_became_active":"2024-09-15T22:09:55.203373+0000","last_became_peered":"2024-09-15T22:09:55.203373+0000","last_unstale":"2024-09-15T22:12:24.564217+0000","last_undegraded":"2024-09-15T22:12:24.564217+0000","last_fullsized":"2024-09-15T22:12:24.564217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:24.504990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395385+0000","last_change":"2024-09-15T22:09:53.192545+0000","last_active":"2024-09-15T22:12:24.395385+0000","last_peered":"2024-09-15T22:12:24.395385+0000","last_clean":"2024-09-15T22:12:24.395385+0000","last_became_active":"2024-09-15T22:09:53.192037+0000","last_became_peered":"2024-09-15T22:09:53.192037+0000","last_unstale":"2024-09-15T22:12:24.395385+0000","last_undegraded":"2024-09-15T22:12:24.395385+0000","last_fullsized":"2024-09-15T22:12:24.395385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:28.289861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493264+0000","last_change":"2024-09-15T22:10:02.235017+0000","last_active":"2024-09-15T22:12:24.493264+0000","last_peered":"2024-09-15T22:12:24.493264+0000","last_clean":"2024-09-15T22:12:24.493264+0000","last_became_active":"2024-09-15T22:10:02.234609+0000","last_became_peered":"2024-09-15T22:10:02.234609+0000","last_unstale":"2024-09-15T22:12:24.493264+0000","last_undegraded":"2024-09-15T22:12:24.493264+0000","last_fullsized":"2024-09-15T22:12:24.493264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:47.801421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391714+0000","last_change":"2024-09-15T22:09:55.207151+0000","last_active":"2024-09-15T22:12:24.391714+0000","last_peered":"2024-09-15T22:12:24.391714+0000","last_clean":"2024-09-15T22:12:24.391714+0000","last_became_active":"2024-09-15T22:09:55.206563+0000","last_became_peered":"2024-09-15T22:09:55.206563+0000","last_unstale":"2024-09-15T22:12:24.391714+0000","last_undegraded":"2024-09-15T22:12:24.391714+0000","last_fullsized":"2024-09-15T22:12:24.391714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:31.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'43","reported_seq":329,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848165+0000","last_change":"2024-09-15T22:09:53.191189+0000","last_active":"2024-09-15T22:12:34.848165+0000","last_peered":"2024-09-15T22:12:34.848165+0000","last_clean":"2024-09-15T22:12:34.848165+0000","last_became_active":"2024-09-15T22:09:53.190864+0000","last_became_peered":"2024-09-15T22:09:53.190864+0000","last_unstale":"2024-09-15T22:12:34.848165+0000","last_undegraded":"2024-09-15T22:12:34.848165+0000","last_fullsized":"2024-09-15T22:12:34.848165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:52.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":196,"num_write":45,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478254+0000","last_change":"2024-09-15T22:10:02.237355+0000","last_active":"2024-09-15T22:12:24.478254+0000","last_peered":"2024-09-15T22:12:24.478254+0000","last_clean":"2024-09-15T22:12:24.478254+0000","last_became_active":"2024-09-15T22:10:02.236995+0000","last_became_peered":"2024-09-15T22:10:02.236995+0000","last_unstale":"2024-09-15T22:12:24.478254+0000","last_undegraded":"2024-09-15T22:12:24.478254+0000","last_fullsized":"2024-09-15T22:12:24.478254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:14.715053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644633+0000","last_change":"2024-09-15T22:09:55.218960+0000","last_active":"2024-09-15T22:12:24.644633+0000","last_peered":"2024-09-15T22:12:24.644633+0000","last_clean":"2024-09-15T22:12:24.644633+0000","last_became_active":"2024-09-15T22:09:55.218623+0000","last_became_peered":"2024-09-15T22:09:55.218623+0000","last_unstale":"2024-09-15T22:12:24.644633+0000","last_undegraded":"2024-09-15T22:12:24.644633+0000","last_fullsized":"2024-09-15T22:12:24.644633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:10.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493414+0000","last_change":"2024-09-15T22:09:53.778321+0000","last_active":"2024-09-15T22:12:24.493414+0000","last_peered":"2024-09-15T22:12:24.493414+0000","last_clean":"2024-09-15T22:12:24.493414+0000","last_became_active":"2024-09-15T22:09:53.776717+0000","last_became_peered":"2024-09-15T22:09:53.776717+0000","last_unstale":"2024-09-15T22:12:24.493414+0000","last_undegraded":"2024-09-15T22:12:24.493414+0000","last_fullsized":"2024-09-15T22:12:24.493414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:15:58.489391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394121+0000","last_change":"2024-09-15T22:11:05.239662+0000","last_active":"2024-09-15T22:12:24.394121+0000","last_peered":"2024-09-15T22:12:24.394121+0000","last_clean":"2024-09-15T22:12:24.394121+0000","last_became_active":"2024-09-15T22:11:05.239230+0000","last_became_peered":"2024-09-15T22:11:05.239230+0000","last_unstale":"2024-09-15T22:12:24.394121+0000","last_undegraded":"2024-09-15T22:12:24.394121+0000","last_fullsized":"2024-09-15T22:12:24.394121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:46.897042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568502+0000","last_change":"2024-09-15T22:09:55.214779+0000","last_active":"2024-09-15T22:12:24.568502+0000","last_peered":"2024-09-15T22:12:24.568502+0000","last_clean":"2024-09-15T22:12:24.568502+0000","last_became_active":"2024-09-15T22:09:55.214495+0000","last_became_peered":"2024-09-15T22:09:55.214495+0000","last_unstale":"2024-09-15T22:12:24.568502+0000","last_undegraded":"2024-09-15T22:12:24.568502+0000","last_fullsized":"2024-09-15T22:12:24.568502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:57.529945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643115+0000","last_change":"2024-09-15T22:09:53.194397+0000","last_active":"2024-09-15T22:12:24.643115+0000","last_peered":"2024-09-15T22:12:24.643115+0000","last_clean":"2024-09-15T22:12:24.643115+0000","last_became_active":"2024-09-15T22:09:53.193972+0000","last_became_peered":"2024-09-15T22:09:53.193972+0000","last_unstale":"2024-09-15T22:12:24.643115+0000","last_undegraded":"2024-09-15T22:12:24.643115+0000","last_fullsized":"2024-09-15T22:12:24.643115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:05.834229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391274+0000","last_change":"2024-09-15T22:11:05.245091+0000","last_active":"2024-09-15T22:12:24.391274+0000","last_peered":"2024-09-15T22:12:24.391274+0000","last_clean":"2024-09-15T22:12:24.391274+0000","last_became_active":"2024-09-15T22:11:05.244579+0000","last_became_peered":"2024-09-15T22:11:05.244579+0000","last_unstale":"2024-09-15T22:12:24.391274+0000","last_undegraded":"2024-09-15T22:12:24.391274+0000","last_fullsized":"2024-09-15T22:12:24.391274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:37.937637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644802+0000","last_change":"2024-09-15T22:09:55.212109+0000","last_active":"2024-09-15T22:12:24.644802+0000","last_peered":"2024-09-15T22:12:24.644802+0000","last_clean":"2024-09-15T22:12:24.644802+0000","last_became_active":"2024-09-15T22:09:55.211747+0000","last_became_peered":"2024-09-15T22:09:55.211747+0000","last_unstale":"2024-09-15T22:12:24.644802+0000","last_undegraded":"2024-09-15T22:12:24.644802+0000","last_fullsized":"2024-09-15T22:12:24.644802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:46.027059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":279,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:34.848263+0000","last_change":"2024-09-15T22:09:53.192814+0000","last_active":"2024-09-15T22:12:34.848263+0000","last_peered":"2024-09-15T22:12:34.848263+0000","last_clean":"2024-09-15T22:12:34.848263+0000","last_became_active":"2024-09-15T22:09:53.192434+0000","last_became_peered":"2024-09-15T22:09:53.192434+0000","last_unstale":"2024-09-15T22:12:34.848263+0000","last_undegraded":"2024-09-15T22:12:34.848263+0000","last_fullsized":"2024-09-15T22:12:34.848263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:35.872473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483942+0000","last_change":"2024-09-15T22:10:02.237741+0000","last_active":"2024-09-15T22:12:24.483942+0000","last_peered":"2024-09-15T22:12:24.483942+0000","last_clean":"2024-09-15T22:12:24.483942+0000","last_became_active":"2024-09-15T22:10:02.237372+0000","last_became_peered":"2024-09-15T22:10:02.237372+0000","last_unstale":"2024-09-15T22:12:24.483942+0000","last_undegraded":"2024-09-15T22:12:24.483942+0000","last_fullsized":"2024-09-15T22:12:24.483942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:43.479714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567637+0000","last_change":"2024-09-15T22:10:02.237887+0000","last_active":"2024-09-15T22:12:24.567637+0000","last_peered":"2024-09-15T22:12:24.567637+0000","last_clean":"2024-09-15T22:12:24.567637+0000","last_became_active":"2024-09-15T22:10:02.237495+0000","last_became_peered":"2024-09-15T22:10:02.237495+0000","last_unstale":"2024-09-15T22:12:24.567637+0000","last_undegraded":"2024-09-15T22:12:24.567637+0000","last_fullsized":"2024-09-15T22:12:24.567637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.055731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.477852+0000","last_change":"2024-09-15T22:09:55.205729+0000","last_active":"2024-09-15T22:12:24.477852+0000","last_peered":"2024-09-15T22:12:24.477852+0000","last_clean":"2024-09-15T22:12:24.477852+0000","last_became_active":"2024-09-15T22:09:55.205232+0000","last_became_peered":"2024-09-15T22:09:55.205232+0000","last_unstale":"2024-09-15T22:12:24.477852+0000","last_undegraded":"2024-09-15T22:12:24.477852+0000","last_fullsized":"2024-09-15T22:12:24.477852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.945415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493105+0000","last_change":"2024-09-15T22:09:53.191616+0000","last_active":"2024-09-15T22:12:24.493105+0000","last_peered":"2024-09-15T22:12:24.493105+0000","last_clean":"2024-09-15T22:12:24.493105+0000","last_became_active":"2024-09-15T22:09:53.191200+0000","last_became_peered":"2024-09-15T22:09:53.191200+0000","last_unstale":"2024-09-15T22:12:24.493105+0000","last_undegraded":"2024-09-15T22:12:24.493105+0000","last_fullsized":"2024-09-15T22:12:24.493105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:01.805912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648827+0000","last_change":"2024-09-15T22:10:02.237507+0000","last_active":"2024-09-15T22:12:24.648827+0000","last_peered":"2024-09-15T22:12:24.648827+0000","last_clean":"2024-09-15T22:12:24.648827+0000","last_became_active":"2024-09-15T22:10:02.237120+0000","last_became_peered":"2024-09-15T22:10:02.237120+0000","last_unstale":"2024-09-15T22:12:24.648827+0000","last_undegraded":"2024-09-15T22:12:24.648827+0000","last_fullsized":"2024-09-15T22:12:24.648827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:35.273025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483803+0000","last_change":"2024-09-15T22:09:55.203422+0000","last_active":"2024-09-15T22:12:24.483803+0000","last_peered":"2024-09-15T22:12:24.483803+0000","last_clean":"2024-09-15T22:12:24.483803+0000","last_became_active":"2024-09-15T22:09:55.202321+0000","last_became_peered":"2024-09-15T22:09:55.202321+0000","last_unstale":"2024-09-15T22:12:24.483803+0000","last_undegraded":"2024-09-15T22:12:24.483803+0000","last_fullsized":"2024-09-15T22:12:24.483803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:54.165097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564110+0000","last_change":"2024-09-15T22:09:53.187739+0000","last_active":"2024-09-15T22:12:24.564110+0000","last_peered":"2024-09-15T22:12:24.564110+0000","last_clean":"2024-09-15T22:12:24.564110+0000","last_became_active":"2024-09-15T22:09:53.187453+0000","last_became_peered":"2024-09-15T22:09:53.187453+0000","last_unstale":"2024-09-15T22:12:24.564110+0000","last_undegraded":"2024-09-15T22:12:24.564110+0000","last_fullsized":"2024-09-15T22:12:24.564110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:20:51.164923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495883+0000","last_change":"2024-09-15T22:09:53.189792+0000","last_active":"2024-09-15T22:12:24.495883+0000","last_peered":"2024-09-15T22:12:24.495883+0000","last_clean":"2024-09-15T22:12:24.495883+0000","last_became_active":"2024-09-15T22:09:53.189425+0000","last_became_peered":"2024-09-15T22:09:53.189425+0000","last_unstale":"2024-09-15T22:12:24.495883+0000","last_undegraded":"2024-09-15T22:12:24.495883+0000","last_fullsized":"2024-09-15T22:12:24.495883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:05.811718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.493063+0000","last_change":"2024-09-15T22:09:55.216446+0000","last_active":"2024-09-15T22:12:24.493063+0000","last_peered":"2024-09-15T22:12:24.493063+0000","last_clean":"2024-09-15T22:12:24.493063+0000","last_became_active":"2024-09-15T22:09:55.215988+0000","last_became_peered":"2024-09-15T22:09:55.215988+0000","last_unstale":"2024-09-15T22:12:24.493063+0000","last_undegraded":"2024-09-15T22:12:24.493063+0000","last_fullsized":"2024-09-15T22:12:24.493063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:04.145450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395325+0000","last_change":"2024-09-15T22:10:02.244046+0000","last_active":"2024-09-15T22:12:24.395325+0000","last_peered":"2024-09-15T22:12:24.395325+0000","last_clean":"2024-09-15T22:12:24.395325+0000","last_became_active":"2024-09-15T22:10:02.243555+0000","last_became_peered":"2024-09-15T22:10:02.243555+0000","last_unstale":"2024-09-15T22:12:24.395325+0000","last_undegraded":"2024-09-15T22:12:24.395325+0000","last_fullsized":"2024-09-15T22:12:24.395325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:47.432930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:33.847490+0000","last_change":"2024-09-15T22:09:53.194932+0000","last_active":"2024-09-15T22:12:33.847490+0000","last_peered":"2024-09-15T22:12:33.847490+0000","last_clean":"2024-09-15T22:12:33.847490+0000","last_became_active":"2024-09-15T22:09:53.194595+0000","last_became_peered":"2024-09-15T22:09:53.194595+0000","last_unstale":"2024-09-15T22:12:33.847490+0000","last_undegraded":"2024-09-15T22:12:33.847490+0000","last_fullsized":"2024-09-15T22:12:33.847490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:52.165882+0000","last_clean_scrub_stamp":"2024-09-15T22:09:52.165882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.387158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1464,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391435+0000","last_change":"2024-09-15T22:09:55.211369+0000","last_active":"2024-09-15T22:12:24.391435+0000","last_peered":"2024-09-15T22:12:24.391435+0000","last_clean":"2024-09-15T22:12:24.391435+0000","last_became_active":"2024-09-15T22:09:55.211006+0000","last_became_peered":"2024-09-15T22:09:55.211006+0000","last_unstale":"2024-09-15T22:12:24.391435+0000","last_undegraded":"2024-09-15T22:12:24.391435+0000","last_fullsized":"2024-09-15T22:12:24.391435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.180199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395722+0000","last_change":"2024-09-15T22:10:02.240079+0000","last_active":"2024-09-15T22:12:24.395722+0000","last_peered":"2024-09-15T22:12:24.395722+0000","last_clean":"2024-09-15T22:12:24.395722+0000","last_became_active":"2024-09-15T22:10:02.239653+0000","last_became_peered":"2024-09-15T22:10:02.239653+0000","last_unstale":"2024-09-15T22:12:24.395722+0000","last_undegraded":"2024-09-15T22:12:24.395722+0000","last_fullsized":"2024-09-15T22:12:24.395722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:10:01.211285+0000","last_clean_scrub_stamp":"2024-09-15T22:10:01.211285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.345713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648467+0000","last_change":"2024-09-15T22:09:55.216327+0000","last_active":"2024-09-15T22:12:24.648467+0000","last_peered":"2024-09-15T22:12:24.648467+0000","last_clean":"2024-09-15T22:12:24.648467+0000","last_became_active":"2024-09-15T22:09:55.215978+0000","last_became_peered":"2024-09-15T22:09:55.215978+0000","last_unstale":"2024-09-15T22:12:24.648467+0000","last_undegraded":"2024-09-15T22:12:24.648467+0000","last_fullsized":"2024-09-15T22:12:24.648467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:24.934252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395292+0000","last_change":"2024-09-15T22:11:05.294107+0000","last_active":"2024-09-15T22:12:24.395292+0000","last_peered":"2024-09-15T22:12:24.395292+0000","last_clean":"2024-09-15T22:12:24.395292+0000","last_became_active":"2024-09-15T22:11:05.293722+0000","last_became_peered":"2024-09-15T22:11:05.293722+0000","last_unstale":"2024-09-15T22:12:24.395292+0000","last_undegraded":"2024-09-15T22:12:24.395292+0000","last_fullsized":"2024-09-15T22:12:24.395292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:09.457927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395092+0000","last_change":"2024-09-15T22:09:55.205839+0000","last_active":"2024-09-15T22:12:24.395092+0000","last_peered":"2024-09-15T22:12:24.395092+0000","last_clean":"2024-09-15T22:12:24.395092+0000","last_became_active":"2024-09-15T22:09:55.205379+0000","last_became_peered":"2024-09-15T22:09:55.205379+0000","last_unstale":"2024-09-15T22:12:24.395092+0000","last_undegraded":"2024-09-15T22:12:24.395092+0000","last_fullsized":"2024-09-15T22:12:24.395092+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:41.011878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567096+0000","last_change":"2024-09-15T22:09:55.210542+0000","last_active":"2024-09-15T22:12:24.567096+0000","last_peered":"2024-09-15T22:12:24.567096+0000","last_clean":"2024-09-15T22:12:24.567096+0000","last_became_active":"2024-09-15T22:09:55.210227+0000","last_became_peered":"2024-09-15T22:09:55.210227+0000","last_unstale":"2024-09-15T22:12:24.567096+0000","last_undegraded":"2024-09-15T22:12:24.567096+0000","last_fullsized":"2024-09-15T22:12:24.567096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:04.939249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564589+0000","last_change":"2024-09-15T22:09:55.206607+0000","last_active":"2024-09-15T22:12:24.564589+0000","last_peered":"2024-09-15T22:12:24.564589+0000","last_clean":"2024-09-15T22:12:24.564589+0000","last_became_active":"2024-09-15T22:09:55.206292+0000","last_became_peered":"2024-09-15T22:09:55.206292+0000","last_unstale":"2024-09-15T22:12:24.564589+0000","last_undegraded":"2024-09-15T22:12:24.564589+0000","last_fullsized":"2024-09-15T22:12:24.564589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:34.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648758+0000","last_change":"2024-09-15T22:09:55.212388+0000","last_active":"2024-09-15T22:12:24.648758+0000","last_peered":"2024-09-15T22:12:24.648758+0000","last_clean":"2024-09-15T22:12:24.648758+0000","last_became_active":"2024-09-15T22:09:55.211891+0000","last_became_peered":"2024-09-15T22:09:55.211891+0000","last_unstale":"2024-09-15T22:12:24.648758+0000","last_undegraded":"2024-09-15T22:12:24.648758+0000","last_fullsized":"2024-09-15T22:12:24.648758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:25:36.284614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.396006+0000","last_change":"2024-09-15T22:09:55.214787+0000","last_active":"2024-09-15T22:12:24.396006+0000","last_peered":"2024-09-15T22:12:24.396006+0000","last_clean":"2024-09-15T22:12:24.396006+0000","last_became_active":"2024-09-15T22:09:55.214254+0000","last_became_peered":"2024-09-15T22:09:55.214254+0000","last_unstale":"2024-09-15T22:12:24.396006+0000","last_undegraded":"2024-09-15T22:12:24.396006+0000","last_fullsized":"2024-09-15T22:12:24.396006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:29.300360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.485741+0000","last_change":"2024-09-15T22:09:55.208627+0000","last_active":"2024-09-15T22:12:24.485741+0000","last_peered":"2024-09-15T22:12:24.485741+0000","last_clean":"2024-09-15T22:12:24.485741+0000","last_became_active":"2024-09-15T22:09:55.208082+0000","last_became_peered":"2024-09-15T22:09:55.208082+0000","last_unstale":"2024-09-15T22:12:24.485741+0000","last_undegraded":"2024-09-15T22:12:24.485741+0000","last_fullsized":"2024-09-15T22:12:24.485741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:21.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.492810+0000","last_change":"2024-09-15T22:09:55.211987+0000","last_active":"2024-09-15T22:12:24.492810+0000","last_peered":"2024-09-15T22:12:24.492810+0000","last_clean":"2024-09-15T22:12:24.492810+0000","last_became_active":"2024-09-15T22:09:55.211144+0000","last_became_peered":"2024-09-15T22:09:55.211144+0000","last_unstale":"2024-09-15T22:12:24.492810+0000","last_undegraded":"2024-09-15T22:12:24.492810+0000","last_fullsized":"2024-09-15T22:12:24.492810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:29.926272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.484053+0000","last_change":"2024-09-15T22:09:55.208106+0000","last_active":"2024-09-15T22:12:24.484053+0000","last_peered":"2024-09-15T22:12:24.484053+0000","last_clean":"2024-09-15T22:12:24.484053+0000","last_became_active":"2024-09-15T22:09:55.207794+0000","last_became_peered":"2024-09-15T22:09:55.207794+0000","last_unstale":"2024-09-15T22:12:24.484053+0000","last_undegraded":"2024-09-15T22:12:24.484053+0000","last_fullsized":"2024-09-15T22:12:24.484053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:49.837551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644410+0000","last_change":"2024-09-15T22:09:55.215296+0000","last_active":"2024-09-15T22:12:24.644410+0000","last_peered":"2024-09-15T22:12:24.644410+0000","last_clean":"2024-09-15T22:12:24.644410+0000","last_became_active":"2024-09-15T22:09:55.214321+0000","last_became_peered":"2024-09-15T22:09:55.214321+0000","last_unstale":"2024-09-15T22:12:24.644410+0000","last_undegraded":"2024-09-15T22:12:24.644410+0000","last_fullsized":"2024-09-15T22:12:24.644410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:49.843862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564570+0000","last_change":"2024-09-15T22:09:55.204654+0000","last_active":"2024-09-15T22:12:24.564570+0000","last_peered":"2024-09-15T22:12:24.564570+0000","last_clean":"2024-09-15T22:12:24.564570+0000","last_became_active":"2024-09-15T22:09:55.203795+0000","last_became_peered":"2024-09-15T22:09:55.203795+0000","last_unstale":"2024-09-15T22:12:24.564570+0000","last_undegraded":"2024-09-15T22:12:24.564570+0000","last_fullsized":"2024-09-15T22:12:24.564570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:59.010367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648616+0000","last_change":"2024-09-15T22:09:55.204927+0000","last_active":"2024-09-15T22:12:24.648616+0000","last_peered":"2024-09-15T22:12:24.648616+0000","last_clean":"2024-09-15T22:12:24.648616+0000","last_became_active":"2024-09-15T22:09:55.204526+0000","last_became_peered":"2024-09-15T22:09:55.204526+0000","last_unstale":"2024-09-15T22:12:24.648616+0000","last_undegraded":"2024-09-15T22:12:24.648616+0000","last_fullsized":"2024-09-15T22:12:24.648616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.409310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.395761+0000","last_change":"2024-09-15T22:09:55.213511+0000","last_active":"2024-09-15T22:12:24.395761+0000","last_peered":"2024-09-15T22:12:24.395761+0000","last_clean":"2024-09-15T22:12:24.395761+0000","last_became_active":"2024-09-15T22:09:55.213175+0000","last_became_peered":"2024-09-15T22:09:55.213175+0000","last_unstale":"2024-09-15T22:12:24.395761+0000","last_undegraded":"2024-09-15T22:12:24.395761+0000","last_fullsized":"2024-09-15T22:12:24.395761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:02:26.706032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.478905+0000","last_change":"2024-09-15T22:09:55.205560+0000","last_active":"2024-09-15T22:12:24.478905+0000","last_peered":"2024-09-15T22:12:24.478905+0000","last_clean":"2024-09-15T22:12:24.478905+0000","last_became_active":"2024-09-15T22:09:55.205260+0000","last_became_peered":"2024-09-15T22:09:55.205260+0000","last_unstale":"2024-09-15T22:12:24.478905+0000","last_undegraded":"2024-09-15T22:12:24.478905+0000","last_fullsized":"2024-09-15T22:12:24.478905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:36.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.649013+0000","last_change":"2024-09-15T22:09:55.211031+0000","last_active":"2024-09-15T22:12:24.649013+0000","last_peered":"2024-09-15T22:12:24.649013+0000","last_clean":"2024-09-15T22:12:24.649013+0000","last_became_active":"2024-09-15T22:09:55.208882+0000","last_became_peered":"2024-09-15T22:09:55.208882+0000","last_unstale":"2024-09-15T22:12:24.649013+0000","last_undegraded":"2024-09-15T22:12:24.649013+0000","last_fullsized":"2024-09-15T22:12:24.649013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:42.662647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644319+0000","last_change":"2024-09-15T22:09:55.208649+0000","last_active":"2024-09-15T22:12:24.644319+0000","last_peered":"2024-09-15T22:12:24.644319+0000","last_clean":"2024-09-15T22:12:24.644319+0000","last_became_active":"2024-09-15T22:09:55.208312+0000","last_became_peered":"2024-09-15T22:09:55.208312+0000","last_unstale":"2024-09-15T22:12:24.644319+0000","last_undegraded":"2024-09-15T22:12:24.644319+0000","last_fullsized":"2024-09-15T22:12:24.644319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:24:53.287058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.566835+0000","last_change":"2024-09-15T22:09:55.209973+0000","last_active":"2024-09-15T22:12:24.566835+0000","last_peered":"2024-09-15T22:12:24.566835+0000","last_clean":"2024-09-15T22:12:24.566835+0000","last_became_active":"2024-09-15T22:09:55.209423+0000","last_became_peered":"2024-09-15T22:09:55.209423+0000","last_unstale":"2024-09-15T22:12:24.566835+0000","last_undegraded":"2024-09-15T22:12:24.566835+0000","last_fullsized":"2024-09-15T22:12:24.566835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:49.309980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.568608+0000","last_change":"2024-09-15T22:09:55.207098+0000","last_active":"2024-09-15T22:12:24.568608+0000","last_peered":"2024-09-15T22:12:24.568608+0000","last_clean":"2024-09-15T22:12:24.568608+0000","last_became_active":"2024-09-15T22:09:55.206779+0000","last_became_peered":"2024-09-15T22:09:55.206779+0000","last_unstale":"2024-09-15T22:12:24.568608+0000","last_undegraded":"2024-09-15T22:12:24.568608+0000","last_fullsized":"2024-09-15T22:12:24.568608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:02.417524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.564461+0000","last_change":"2024-09-15T22:09:55.214407+0000","last_active":"2024-09-15T22:12:24.564461+0000","last_peered":"2024-09-15T22:12:24.564461+0000","last_clean":"2024-09-15T22:12:24.564461+0000","last_became_active":"2024-09-15T22:09:55.213775+0000","last_became_peered":"2024-09-15T22:09:55.213775+0000","last_unstale":"2024-09-15T22:12:24.564461+0000","last_undegraded":"2024-09-15T22:12:24.564461+0000","last_fullsized":"2024-09-15T22:12:24.564461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.109087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394389+0000","last_change":"2024-09-15T22:09:55.211997+0000","last_active":"2024-09-15T22:12:24.394389+0000","last_peered":"2024-09-15T22:12:24.394389+0000","last_clean":"2024-09-15T22:12:24.394389+0000","last_became_active":"2024-09-15T22:09:55.211635+0000","last_became_peered":"2024-09-15T22:09:55.211635+0000","last_unstale":"2024-09-15T22:12:24.394389+0000","last_undegraded":"2024-09-15T22:12:24.394389+0000","last_fullsized":"2024-09-15T22:12:24.394389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:24:37.112967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.643143+0000","last_change":"2024-09-15T22:09:55.214692+0000","last_active":"2024-09-15T22:12:24.643143+0000","last_peered":"2024-09-15T22:12:24.643143+0000","last_clean":"2024-09-15T22:12:24.643143+0000","last_became_active":"2024-09-15T22:09:55.214222+0000","last_became_peered":"2024-09-15T22:09:55.214222+0000","last_unstale":"2024-09-15T22:12:24.643143+0000","last_undegraded":"2024-09-15T22:12:24.643143+0000","last_fullsized":"2024-09-15T22:12:24.643143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:08.217347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.496249+0000","last_change":"2024-09-15T22:09:55.208418+0000","last_active":"2024-09-15T22:12:24.496249+0000","last_peered":"2024-09-15T22:12:24.496249+0000","last_clean":"2024-09-15T22:12:24.496249+0000","last_became_active":"2024-09-15T22:09:55.208049+0000","last_became_peered":"2024-09-15T22:09:55.208049+0000","last_unstale":"2024-09-15T22:12:24.496249+0000","last_undegraded":"2024-09-15T22:12:24.496249+0000","last_fullsized":"2024-09-15T22:12:24.496249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:14.860978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.644899+0000","last_change":"2024-09-15T22:09:55.217838+0000","last_active":"2024-09-15T22:12:24.644899+0000","last_peered":"2024-09-15T22:12:24.644899+0000","last_clean":"2024-09-15T22:12:24.644899+0000","last_became_active":"2024-09-15T22:09:55.217479+0000","last_became_peered":"2024-09-15T22:09:55.217479+0000","last_unstale":"2024-09-15T22:12:24.644899+0000","last_undegraded":"2024-09-15T22:12:24.644899+0000","last_fullsized":"2024-09-15T22:12:24.644899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.460882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.483796+0000","last_change":"2024-09-15T22:09:55.213111+0000","last_active":"2024-09-15T22:12:24.483796+0000","last_peered":"2024-09-15T22:12:24.483796+0000","last_clean":"2024-09-15T22:12:24.483796+0000","last_became_active":"2024-09-15T22:09:55.212771+0000","last_became_peered":"2024-09-15T22:09:55.212771+0000","last_unstale":"2024-09-15T22:12:24.483796+0000","last_undegraded":"2024-09-15T22:12:24.483796+0000","last_fullsized":"2024-09-15T22:12:24.483796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:15.730577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.642618+0000","last_change":"2024-09-15T22:09:55.216047+0000","last_active":"2024-09-15T22:12:24.642618+0000","last_peered":"2024-09-15T22:12:24.642618+0000","last_clean":"2024-09-15T22:12:24.642618+0000","last_became_active":"2024-09-15T22:09:55.215688+0000","last_became_peered":"2024-09-15T22:09:55.215688+0000","last_unstale":"2024-09-15T22:12:24.642618+0000","last_undegraded":"2024-09-15T22:12:24.642618+0000","last_fullsized":"2024-09-15T22:12:24.642618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:10.997986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.495873+0000","last_change":"2024-09-15T22:09:55.208584+0000","last_active":"2024-09-15T22:12:24.495873+0000","last_peered":"2024-09-15T22:12:24.495873+0000","last_clean":"2024-09-15T22:12:24.495873+0000","last_became_active":"2024-09-15T22:09:55.208077+0000","last_became_peered":"2024-09-15T22:09:55.208077+0000","last_unstale":"2024-09-15T22:12:24.495873+0000","last_undegraded":"2024-09-15T22:12:24.495873+0000","last_fullsized":"2024-09-15T22:12:24.495873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:17.105607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.391534+0000","last_change":"2024-09-15T22:09:55.203125+0000","last_active":"2024-09-15T22:12:24.391534+0000","last_peered":"2024-09-15T22:12:24.391534+0000","last_clean":"2024-09-15T22:12:24.391534+0000","last_became_active":"2024-09-15T22:09:55.202776+0000","last_became_peered":"2024-09-15T22:09:55.202776+0000","last_unstale":"2024-09-15T22:12:24.391534+0000","last_undegraded":"2024-09-15T22:12:24.391534+0000","last_fullsized":"2024-09-15T22:12:24.391534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:40.787485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.394335+0000","last_change":"2024-09-15T22:09:55.209508+0000","last_active":"2024-09-15T22:12:24.394335+0000","last_peered":"2024-09-15T22:12:24.394335+0000","last_clean":"2024-09-15T22:12:24.394335+0000","last_became_active":"2024-09-15T22:09:55.209171+0000","last_became_peered":"2024-09-15T22:09:55.209171+0000","last_unstale":"2024-09-15T22:12:24.394335+0000","last_undegraded":"2024-09-15T22:12:24.394335+0000","last_fullsized":"2024-09-15T22:12:24.394335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.559684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567428+0000","last_change":"2024-09-15T22:09:55.208771+0000","last_active":"2024-09-15T22:12:24.567428+0000","last_peered":"2024-09-15T22:12:24.567428+0000","last_clean":"2024-09-15T22:12:24.567428+0000","last_became_active":"2024-09-15T22:09:55.208403+0000","last_became_peered":"2024-09-15T22:09:55.208403+0000","last_unstale":"2024-09-15T22:12:24.567428+0000","last_undegraded":"2024-09-15T22:12:24.567428+0000","last_fullsized":"2024-09-15T22:12:24.567428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:56.950273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.567727+0000","last_change":"2024-09-15T22:09:55.205216+0000","last_active":"2024-09-15T22:12:24.567727+0000","last_peered":"2024-09-15T22:12:24.567727+0000","last_clean":"2024-09-15T22:12:24.567727+0000","last_became_active":"2024-09-15T22:09:55.204883+0000","last_became_peered":"2024-09-15T22:09:55.204883+0000","last_unstale":"2024-09-15T22:12:24.567727+0000","last_undegraded":"2024-09-15T22:12:24.567727+0000","last_fullsized":"2024-09-15T22:12:24.567727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:50.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-15T22:12:24.648937+0000","last_change":"2024-09-15T22:09:55.210213+0000","last_active":"2024-09-15T22:12:24.648937+0000","last_peered":"2024-09-15T22:12:24.648937+0000","last_clean":"2024-09-15T22:12:24.648937+0000","last_became_active":"2024-09-15T22:09:55.208351+0000","last_became_peered":"2024-09-15T22:09:55.208351+0000","last_unstale":"2024-09-15T22:12:24.648937+0000","last_undegraded":"2024-09-15T22:12:24.648937+0000","last_fullsized":"2024-09-15T22:12:24.648937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:09:54.177124+0000","last_clean_scrub_stamp":"2024-09-15T22:09:54.177124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:00.697109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":803057,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1557,"num_read_kb":1002,"num_write":513,"num_write_kb":2396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2409171,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107172,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514542,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1015808,"data_stored":516162,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1212416,"data_stored":1502017,"data_compressed":77321,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877907000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":911972,"data_compressed":45684,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":908142,"data_compressed":45670,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364865,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1113306,"data_compressed":62119,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":510950,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":1435800,"data_compressed":76959,"data_compressed_allocated":102400,"data_compressed_original":970752,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":514772,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":1034688,"data_compressed":51697,"data_compressed_allocated":69632,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":585281,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":583685,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":512940,"data_compressed":26395,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35724,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331894,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":402207,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:12:41.249 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:41.495 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- 172.21.15.77:0/3571436577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38102ef0 msgr2=0x7fba3810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/3571436577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38102ef0 0x7fba3810da40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba30009900 tx=0x7fba3002a480 comp rx=0 tx=0).stop 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- 172.21.15.77:0/3571436577 shutdown_connections 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/3571436577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38102ef0 0x7fba3810da40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/3571436577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102570 0x7fba381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/3571436577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38100e20 0x7fba38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- 172.21.15.77:0/3571436577 >> 172.21.15.77:0/3571436577 conn(0x7fba380fc7d0 msgr2=0x7fba380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- 172.21.15.77:0/3571436577 shutdown_connections 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- 172.21.15.77:0/3571436577 wait complete. 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 Processor -- start 2024-09-15T22:12:41.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.965+0000 7fba3f1ba640 1 -- start start 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38102570 0x7fba38199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102ef0 0x7fba381a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba38110f80 con 0x7fba38100e20 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba38110e00 con 0x7fba38102ef0 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba38111100 con 0x7fba38102570 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56042/0 (socket says 172.21.15.77:56042) 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 -- 172.21.15.77:0/850291287 learned_addr learned my addr 172.21.15.77:0/850291287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3d757640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102ef0 0x7fba381a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba37fff640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38102570 0x7fba38199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 -- 172.21.15.77:0/850291287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38102570 msgr2=0x7fba38199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38102570 0x7fba38199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:41.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 -- 172.21.15.77:0/850291287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102ef0 msgr2=0x7fba381a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102ef0 0x7fba381a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 -- 172.21.15.77:0/850291287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba381a4810 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3cf56640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d770 tx=0x7fba2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20017070 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20005500 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.966+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba381a49e0 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.967+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20004e00 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.967+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a4ea0 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.968+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38101200 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.969+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fba20004fa0 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.969+0000 7fba35ffb640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 0x7fba240a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.969+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fba200cfd40 con 0x7fba38100e20 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.969+0000 7fba37fff640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 0x7fba240a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:41.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.970+0000 7fba37fff640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 0x7fba240a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba3806aac0 tx=0x7fba280075b0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:41.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:41.973+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba2009bae0 con 0x7fba38100e20 2024-09-15T22:12:42.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.173+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba381a50d0 con 0x7fba38100e20 2024-09-15T22:12:42.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.174+0000 7fba35ffb640 1 -- 172.21.15.77:0/850291287 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7fba2009fde0 con 0x7fba38100e20 2024-09-15T22:12:42.175 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:42.176 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:12:42.176 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 msgr2=0x7fba240a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 0x7fba240a5990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba3806aac0 tx=0x7fba280075b0 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 msgr2=0x7fba38199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d770 tx=0x7fba2000dc30 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 shutdown_connections 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba240a34d0 0x7fba240a5990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba38102ef0 0x7fba381a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba38102570 0x7fba38199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 --2- 172.21.15.77:0/850291287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba38100e20 0x7fba38199250 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 >> 172.21.15.77:0/850291287 conn(0x7fba380fc7d0 msgr2=0x7fba38106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 shutdown_connections 2024-09-15T22:12:42.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:42.176+0000 7fba3f1ba640 1 -- 172.21.15.77:0/850291287 wait complete. 2024-09-15T22:12:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:42 smithi077 ceph-mon[30980]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:42 smithi077 ceph-mon[30980]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:42 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/850291287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:42.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:42 smithi196 ceph-mon[35109]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:42.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:42 smithi196 ceph-mon[35109]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:42.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:42 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/850291287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:42 smithi117 ceph-mon[35021]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-15T22:12:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:42 smithi117 ceph-mon[35021]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:12:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:42 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/850291287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:42.895 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:43.137 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.622+0000 7f516382d640 1 -- 172.21.15.77:0/3279395301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c1008e0 msgr2=0x7f515c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.622+0000 7f516382d640 1 --2- 172.21.15.77:0/3279395301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c1008e0 0x7f515c100cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f514c009ca0 tx=0x7f514c02a950 comp rx=0 tx=0).stop 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- 172.21.15.77:0/3279395301 shutdown_connections 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- 172.21.15.77:0/3279395301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 0x7f515c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- 172.21.15.77:0/3279395301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c101200 0x7f515c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- 172.21.15.77:0/3279395301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c1008e0 0x7f515c100cc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- 172.21.15.77:0/3279395301 >> 172.21.15.77:0/3279395301 conn(0x7f515c077210 msgr2=0x7f515c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- 172.21.15.77:0/3279395301 shutdown_connections 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- 172.21.15.77:0/3279395301 wait complete. 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 Processor -- start 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- start start 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c1008e0 0x7f515c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 0x7f515c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f515c114180 con 0x7f515c101200 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f515c114000 con 0x7f515c1008e0 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.623+0000 7f516382d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f515c114300 con 0x7f515c10e3c0 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56046/0 (socket says 172.21.15.77:56046) 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 -- 172.21.15.77:0/1466523323 learned_addr learned my addr 172.21.15.77:0/1466523323 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5161dca640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 0x7f515c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f51615c9640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c1008e0 0x7f515c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 -- 172.21.15.77:0/1466523323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 msgr2=0x7f515c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 0x7f515c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 -- 172.21.15.77:0/1466523323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c1008e0 msgr2=0x7f515c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c1008e0 0x7f515c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 -- 172.21.15.77:0/1466523323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c19f550 con 0x7f515c101200 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f5160dc8640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f515000c970 tx=0x7f515000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150007c10 con 0x7f515c101200 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5150007db0 con 0x7f515c101200 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c19f840 con 0x7f515c101200 2024-09-15T22:12:43.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515000f940 con 0x7f515c101200 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.624+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a7120 con 0x7f515c101200 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.625+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c100cc0 con 0x7f515c101200 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.627+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f515000fae0 con 0x7f515c101200 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.627+0000 7f515a7fc640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 0x7f51440a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.627+0000 7f51615c9640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 0x7f51440a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.627+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f51500d0a70 con 0x7f515c101200 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.628+0000 7f51615c9640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 0x7f51440a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f514c005fc0 tx=0x7f514c009360 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.630+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f515009cfe0 con 0x7f515c101200 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.832+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f515c19b900 con 0x7f515c101200 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.833+0000 7f515a7fc640 1 -- 172.21.15.77:0/1466523323 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f515c19b900 con 0x7f515c101200 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 msgr2=0x7f51440a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 0x7f51440a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f514c005fc0 tx=0x7f514c009360 comp rx=0 tx=0).stop 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 msgr2=0x7f515c19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f515000c970 tx=0x7f515000ce30 comp rx=0 tx=0).stop 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 shutdown_connections 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51440a34d0 0x7f51440a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f515c10e3c0 0x7f515c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f515c101200 0x7f515c19aae0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 --2- 172.21.15.77:0/1466523323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c1008e0 0x7f515c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 >> 172.21.15.77:0/1466523323 conn(0x7f515c077210 msgr2=0x7f515c10e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 shutdown_connections 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:43.835+0000 7f516382d640 1 -- 172.21.15.77:0/1466523323 wait complete. 2024-09-15T22:12:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:44 smithi077 ceph-mon[30980]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-15T22:12:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:44 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1466523323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:44 smithi196 ceph-mon[35109]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-15T22:12:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:44 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1466523323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:44 smithi117 ceph-mon[35021]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-15T22:12:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:44 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1466523323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:44.646 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:12:44.890 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 -- 172.21.15.77:0/1101179921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 msgr2=0x7fe9f0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 --2- 172.21.15.77:0/1101179921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0100ee0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009930 tx=0x7fe9e002a420 comp rx=0 tx=0).stop 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 -- 172.21.15.77:0/1101179921 shutdown_connections 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 --2- 172.21.15.77:0/1101179921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0101420 0x7fe9f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 --2- 172.21.15.77:0/1101179921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0100ee0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 --2- 172.21.15.77:0/1101179921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0106a80 0x7fe9f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 -- 172.21.15.77:0/1101179921 >> 172.21.15.77:0/1101179921 conn(0x7fe9f00fc7a0 msgr2=0x7fe9f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 -- 172.21.15.77:0/1101179921 shutdown_connections 2024-09-15T22:12:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.375+0000 7fe9f758a640 1 -- 172.21.15.77:0/1101179921 wait complete. 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 Processor -- start 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 -- start start 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0101420 0x7fe9f0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 0x7fe9f01a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0110e80 con 0x7fe9f0100a80 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0110d00 con 0x7fe9f0106a80 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f758a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0111000 con 0x7fe9f0101420 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56060/0 (socket says 172.21.15.77:56060) 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 -- 172.21.15.77:0/3153095060 learned_addr learned my addr 172.21.15.77:0/3153095060 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5b27640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 0x7fe9f01a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f4b25640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0101420 0x7fe9f0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 -- 172.21.15.77:0/3153095060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0101420 msgr2=0x7fe9f0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0101420 0x7fe9f0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 -- 172.21.15.77:0/3153095060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 msgr2=0x7fe9f01a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 0x7fe9f01a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5326640 1 -- 172.21.15.77:0/3153095060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f01a48d0 con 0x7fe9f0100a80 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.376+0000 7fe9f5b27640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 0x7fe9f01a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9f5326640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9f0 tx=0x7fe9ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00cd80 con 0x7fe9f0100a80 2024-09-15T22:12:45.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9ec005500 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f01a4aa0 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec010690 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.377+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f01a4f60 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.378+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0100ee0 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.379+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fe9ec0046c0 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.380+0000 7fe9e67fc640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 0x7fe9dc0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.380+0000 7fe9f4b25640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 0x7fe9dc0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.380+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe9ec0d09c0 con 0x7fe9f0100a80 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.380+0000 7fe9f4b25640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 0x7fe9dc0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:45.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.383+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9ec0541b0 con 0x7fe9f0100a80 2024-09-15T22:12:45.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.553+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9f006ad80 con 0x7fe9f0100a80 2024-09-15T22:12:45.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.555+0000 7fe9e67fc640 1 -- 172.21.15.77:0/3153095060 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25152 (secure 0 0 0) 0x7fe9ec09ce10 con 0x7fe9f0100a80 2024-09-15T22:12:45.556 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:45.557 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":91,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:12:24.385821+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/999829703":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/3279424814":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3785009540":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1809193257":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/1437248041":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1809193257":"2024-09-16T22:12:24.385790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:12:45.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 msgr2=0x7fe9dc0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 0x7fe9dc0a5a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0000c00 tx=0x7fe9e0005a20 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 msgr2=0x7fe9f0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9f0 tx=0x7fe9ec00edf0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 shutdown_connections 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9dc0a3550 0x7fe9dc0a5a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f0106a80 0x7fe9f01a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9f0101420 0x7fe9f0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 --2- 172.21.15.77:0/3153095060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9f0100a80 0x7fe9f0199310 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 >> 172.21.15.77:0/3153095060 conn(0x7fe9f00fc7a0 msgr2=0x7fe9f00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 shutdown_connections 2024-09-15T22:12:45.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:45.557+0000 7fe9f758a640 1 -- 172.21.15.77:0/3153095060 wait complete. 2024-09-15T22:12:46.247 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-15T22:12:46.493 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:46.514 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:46 smithi077 ceph-mon[30980]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-15T22:12:46.514 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:12:46.514 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:46 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3153095060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:46 smithi196 ceph-mon[35109]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-15T22:12:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:12:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:46 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3153095060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:46 smithi117 ceph-mon[35021]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-15T22:12:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:12:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:46 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3153095060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:12:47.766 INFO:teuthology.orchestra.run.smithi077.stdout:{"run_dir":"/var/run/ceph"} 2024-09-15T22:12:47.766 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-15T22:12:47.766 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-15T22:12:47.767 DEBUG:tasks.quiescer:Yielding 2024-09-15T22:12:47.767 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T22:12:47.781 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:12:47.782 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T22:12:47.782 INFO:tasks.workunit:timeout=3h 2024-09-15T22:12:47.782 INFO:tasks.workunit:cleanup=True 2024-09-15T22:12:47.783 DEBUG:teuthology.orchestra.run.smithi077:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:47.783 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:47.799 INFO:teuthology.orchestra.run.smithi077.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:47.799 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-15T22:12:47.799 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-15T22:12:47.799 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:12:47.800 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-15T22:12:47.800 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-09-15 22:10:47.549133398 +0000 2024-09-15T22:12:47.800 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-09-15 22:10:47.549133398 +0000 2024-09-15T22:12:47.800 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-09-15 22:12:22.405592591 +0000 2024-09-15T22:12:47.800 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-09-15 22:10:47.549133398 +0000 2024-09-15T22:12:47.800 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-15T22:12:47.800 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-15T22:12:47.872 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:12:47.928 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T22:12:48.033 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:48.460 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:48 smithi077 ceph-mon[30980]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 -- 172.21.15.77:0/1198377553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0100a90 msgr2=0x7f4da0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 --2- 172.21.15.77:0/1198377553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0100a90 0x7f4da0100ef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009930 tx=0x7f4d8c02a420 comp rx=0 tx=0).stop 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 -- 172.21.15.77:0/1198377553 shutdown_connections 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 --2- 172.21.15.77:0/1198377553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 0x7f4da010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 --2- 172.21.15.77:0/1198377553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0100a90 0x7f4da0100ef0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 --2- 172.21.15.77:0/1198377553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0106a90 0x7f4da0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 -- 172.21.15.77:0/1198377553 >> 172.21.15.77:0/1198377553 conn(0x7f4da00fc7d0 msgr2=0x7f4da00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 -- 172.21.15.77:0/1198377553 shutdown_connections 2024-09-15T22:12:48.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.539+0000 7f4da6e18640 1 -- 172.21.15.77:0/1198377553 wait complete. 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 Processor -- start 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 -- start start 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0100a90 0x7f4da01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 0x7f4da01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0110e90 con 0x7f4da0106a90 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0110d10 con 0x7f4da0100a90 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.540+0000 7f4da6e18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111010 con 0x7f4da0101430 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56080/0 (socket says 172.21.15.77:56080) 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 -- 172.21.15.77:0/3151187444 learned_addr learned my addr 172.21.15.77:0/3151187444 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:48.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da4bb4640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0100a90 0x7f4da01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4d9ffff640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 0x7f4da01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 -- 172.21.15.77:0/3151187444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 msgr2=0x7f4da01977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 0x7f4da01977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 -- 172.21.15.77:0/3151187444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0100a90 msgr2=0x7f4da01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0100a90 0x7f4da01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 -- 172.21.15.77:0/3151187444 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da019c250 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da53b5640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800eaa0 tx=0x7f4d9800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004d20 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d98005020 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d980116d0 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da019c540 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.541+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01a3e20 con 0x7f4da0106a90 2024-09-15T22:12:48.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.543+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0100ef0 con 0x7f4da0106a90 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.543+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f4d980051d0 con 0x7f4da0106a90 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.544+0000 7f4d9dffb640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 0x7f4d900a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.544+0000 7f4da4bb4640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 0x7f4d900a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.544+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4d980d0490 con 0x7f4da0106a90 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.545+0000 7f4da4bb4640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 0x7f4d900a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400ab60 tx=0x7f4d9400a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:48.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.548+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d98016030 con 0x7f4da0106a90 2024-09-15T22:12:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:48 smithi196 ceph-mon[35109]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-15T22:12:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:48 smithi117 ceph-mon[35021]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-15T22:12:48.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.754+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da01a4080 con 0x7f4da0106a90 2024-09-15T22:12:48.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.755+0000 7f4d9dffb640 1 -- 172.21.15.77:0/3151187444 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f4d9809c920 con 0x7f4da0106a90 2024-09-15T22:12:48.756 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:12:48.757 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:12:48.757 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:12:48.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 msgr2=0x7f4d900a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:48.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 0x7f4d900a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400ab60 tx=0x7f4d9400a2d0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 msgr2=0x7f4da019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:48.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800eaa0 tx=0x7f4d9800eea0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 shutdown_connections 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4d900a3550 0x7f4d900a5a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4da0106a90 0x7f4da019bb70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4da0101430 0x7f4da01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 --2- 172.21.15.77:0/3151187444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4da0100a90 0x7f4da01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 >> 172.21.15.77:0/3151187444 conn(0x7f4da00fc7d0 msgr2=0x7f4da00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 shutdown_connections 2024-09-15T22:12:48.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:48.757+0000 7f4da6e18640 1 -- 172.21.15.77:0/3151187444 wait complete. 2024-09-15T22:12:49.455 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:49 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3151187444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:49.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:09:55.819962+0000', 'modified': '2024-09-15T22:12:23.111653+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24458, 'mds_1': 24440, 'mds_2': 24439, 'mds_3': 24424, 'mds_4': 14604}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14592': {'gid': 14592, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/2629133747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 2629133747}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 2629133747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14604': {'gid': 14604, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.77:6841/3032989520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 3032989520}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 3032989520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6833/3132990127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3132990127}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3132990127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6833/2823149773', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 2823149773}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 2823149773}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/889715770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 889715770}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 889715770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/3112549785', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3112549785}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3112549785}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.196:6837/3513024753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3513024753}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3513024753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6839/1624290607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1624290607}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1624290607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.117:6837/3263285576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 3263285576}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3263285576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6839/1252344498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1252344498}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1252344498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24458, 'qdb_cluster': [24440, 24439, 24424, 24458, 14604]} 2024-09-15T22:12:49.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T22:12:49.457 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.a status 2024-09-15T22:12:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:49 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3151187444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:49 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3151187444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:49.744 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:50.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 -- 172.21.15.77:0/3280756105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c10c170 msgr2=0x7fb33c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 --2- 172.21.15.77:0/3280756105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c10c170 0x7fb33c10e610 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb334009ca0 tx=0x7fb33402a950 comp rx=0 tx=0).stop 2024-09-15T22:12:50.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 -- 172.21.15.77:0/3280756105 shutdown_connections 2024-09-15T22:12:50.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 --2- 172.21.15.77:0/3280756105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c10c170 0x7fb33c10e610 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 --2- 172.21.15.77:0/3280756105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33c103660 0x7fb33c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 --2- 172.21.15.77:0/3280756105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 0x7fb33c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 -- 172.21.15.77:0/3280756105 >> 172.21.15.77:0/3280756105 conn(0x7fb33c0fc7a0 msgr2=0x7fb33c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 -- 172.21.15.77:0/3280756105 shutdown_connections 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.221+0000 7fb3437eb640 1 -- 172.21.15.77:0/3280756105 wait complete. 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 Processor -- start 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 -- start start 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 0x7fb33c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33c10c170 0x7fb33c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c112030 con 0x7fb33c103660 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c111eb0 con 0x7fb33c102d40 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb3437eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1121b0 con 0x7fb33c10c170 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54282/0 (socket says 172.21.15.77:54282) 2024-09-15T22:12:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 -- 172.21.15.77:0/2589422983 learned_addr learned my addr 172.21.15.77:0/2589422983 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb341587640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 0x7fb33c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 -- 172.21.15.77:0/2589422983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33c10c170 msgr2=0x7fb33c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33c10c170 0x7fb33c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 -- 172.21.15.77:0/2589422983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 msgr2=0x7fb33c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 0x7fb33c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 -- 172.21.15.77:0/2589422983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c19d500 con 0x7fb33c103660 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.222+0000 7fb340d86640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b3f0 tx=0x7fb33800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.223+0000 7fb3327fc640 1 -- 172.21.15.77:0/2589422983 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338015020 con 0x7fb33c103660 2024-09-15T22:12:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.223+0000 7fb3327fc640 1 -- 172.21.15.77:0/2589422983 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb338011d10 con 0x7fb33c103660 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.223+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c19d7f0 con 0x7fb33c103660 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.223+0000 7fb3327fc640 1 -- 172.21.15.77:0/2589422983 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33800f2e0 con 0x7fb33c103660 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.223+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c1a50d0 con 0x7fb33c103660 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb3327fc640 1 -- 172.21.15.77:0/2589422983 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fb338018020 con 0x7fb33c103660 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 Processor -- start 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 -- start start 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999073a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 0x5589990763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558998fcb090 con 0x558999068de0 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558998f72a70 con 0x558999073fc0 2024-09-15T22:12:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.225+0000 7fb345bc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558998fc7050 con 0x558999069700 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb3327fc640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 0x7fb3280a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54290/0 (socket says 172.21.15.77:54290) 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 -- 172.21.15.77:0/3019793054 learned_addr learned my addr 172.21.15.77:0/3019793054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb340d86640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 0x5589990763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 0x7fb3280a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341587640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb3327fc640 1 -- 172.21.15.77:0/2589422983 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb3380cfae0 con 0x7fb33c103660 2024-09-15T22:12:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 -- 172.21.15.77:0/3019793054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 msgr2=0x558999073a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999073a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 -- 172.21.15.77:0/3019793054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 msgr2=0x5589990763b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 0x5589990763b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 -- 172.21.15.77:0/3019793054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558998f7f130 con 0x558999068de0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb340d86640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 0x5589990763b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb33402ae70 tx=0x7fb33402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a82a6e48564766 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb3277fe640 1 -- 172.21.15.77:0/3019793054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334034040 con 0x558999068de0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb3277fe640 1 -- 172.21.15.77:0/3019793054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33404c070 con 0x558999068de0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb3277fe640 1 -- 172.21.15.77:0/3019793054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33403b6d0 con 0x558999068de0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.226+0000 7fb341d88640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 0x7fb3280a5a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb33404b000 tx=0x7fb33404c000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- 172.21.15.77:0/3019793054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 msgr2=0x5589990691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb33402ae70 tx=0x7fb33402f3a0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- 172.21.15.77:0/3019793054 shutdown_connections 2024-09-15T22:12:50.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999073fc0 0x5589990763b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 --2- 172.21.15.77:0/3019793054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999068de0 0x5589990691c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- 172.21.15.77:0/3019793054 >> 172.21.15.77:0/3019793054 conn(0x558999065370 msgr2=0x558999065750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- 172.21.15.77:0/3019793054 shutdown_connections 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- 172.21.15.77:0/3019793054 wait complete. 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.227+0000 7fb345bc4740 1 -- start start 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 Processor -- start 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558999076b90 con 0x558999073fc0 2024-09-15T22:12:50.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558998f85f20 con 0x558999068de0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558999076d10 con 0x558999069700 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb345bc4740 10 client.0 _ll_register_callbacks cb 0x558998f4f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54302/0 (socket says 172.21.15.77:54302) 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 -- 172.21.15.77:0/1869391119 learned_addr learned my addr 172.21.15.77:0/1869391119 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341587640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb340d86640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 -- 172.21.15.77:0/1869391119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 msgr2=0x558999070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 -- 172.21.15.77:0/1869391119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 msgr2=0x558999073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.228+0000 7fb341d88640 1 -- 172.21.15.77:0/1869391119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558998d3c2b0 con 0x558999073fc0 2024-09-15T22:12:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb341d88640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb33403b7e0 tx=0x7fb33402f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a56417bc089149 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334058070 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb334004040 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33404c070 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb334064050 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 10 client.15573 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589990824e0 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33404c070 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb334039530 con 0x558999073fc0 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 10 client.15573 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 10 client.15573 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 20 client.15573 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558999084670 0x558999085ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 4 client.15573 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.229+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558998fd8600 con 0x558999084670 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb341587640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558999084670 0x558999085ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb324ff9640 10 client.15573 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb341587640 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558999084670 0x558999085ae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5c09fab9b65b1b2 server_cookie=398c3c83e581c87c in_seq=0 out_seq=0 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb324ff9640 1 -- 172.21.15.77:0/1869391119 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558998fd8600 con 0x558999084670 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb324ff9640 10 client.15573 handle_command_reply: tid=0 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.230+0000 7fb345bc4740 1 client.15573 shutdown 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 msgr2=0x558999071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb33403b7e0 tx=0x7fb33402f7e0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 shutdown_connections 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558999084670 0x558999085ae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x558999071110 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1869391119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb324ff9640 0 client.15573 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 >> 172.21.15.77:0/1869391119 conn(0x558999065370 msgr2=0x55899906b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 shutdown_connections 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 1 -- 172.21.15.77:0/1869391119 wait complete. 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.231+0000 7fb345bc4740 20 client.15573 trim_cache size 0 max 16384 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 Processor -- start 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 -- start start 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999083fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999072a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558999083e20 con 0x558999073fc0 2024-09-15T22:12:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5589990844f0 con 0x558999068de0 2024-09-15T22:12:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb345bc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558999084bc0 con 0x558999069700 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54318/0 (socket says 172.21.15.77:54318) 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 -- 172.21.15.77:0/530206785 learned_addr learned my addr 172.21.15.77:0/530206785 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb340d86640 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341587640 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999083fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 -- 172.21.15.77:0/530206785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 msgr2=0x558999072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 -- 172.21.15.77:0/530206785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 msgr2=0x558999083fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.250+0000 7fb341d88640 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999083fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.251+0000 7fb341d88640 1 -- 172.21.15.77:0/530206785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558999069530 con 0x558999073fc0 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.251+0000 7fb341d88640 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb334009ca0 tx=0x7fb33402fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f0e57cdf14c668 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.251+0000 7fb326ffd640 1 -- 172.21.15.77:0/530206785 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334058040 con 0x558999073fc0 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.251+0000 7fb326ffd640 1 -- 172.21.15.77:0/530206785 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb334039ad0 con 0x558999073fc0 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.251+0000 7fb326ffd640 1 -- 172.21.15.77:0/530206785 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334034040 con 0x558999073fc0 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- 172.21.15.77:0/530206785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 msgr2=0x55899906de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb334009ca0 tx=0x7fb33402fda0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- 172.21.15.77:0/530206785 shutdown_connections 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999073fc0 0x55899906de50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999069700 0x558999072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- 172.21.15.77:0/530206785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x558999083fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- 172.21.15.77:0/530206785 >> 172.21.15.77:0/530206785 conn(0x558999065370 msgr2=0x558999072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- 172.21.15.77:0/530206785 shutdown_connections 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- 172.21.15.77:0/530206785 wait complete. 2024-09-15T22:12:50.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- start start 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 Processor -- start 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x55899908e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999073fc0 0x558999089f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558999085960 con 0x558999069700 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558999085290 con 0x558999068de0 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.252+0000 7fb345bc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558999086030 con 0x558999073fc0 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb345bc4740 10 client.0 _ll_register_callbacks cb 0x7fb33402fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54326/0 (socket says 172.21.15.77:54326) 2024-09-15T22:12:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 -- 172.21.15.77:0/1380610028 learned_addr learned my addr 172.21.15.77:0/1380610028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb341587640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x55899908e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb341d88640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999073fc0 0x558999089f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 -- 172.21.15.77:0/1380610028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999073fc0 msgr2=0x558999089f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999073fc0 0x558999089f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 -- 172.21.15.77:0/1380610028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 msgr2=0x55899908e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x55899908e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 -- 172.21.15.77:0/1380610028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899908a750 con 0x558999069700 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb340d86640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb338097da0 tx=0x7fb33801fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950eb3719c1afd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3380afe60 con 0x558999069700 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb338097fb0 con 0x558999069700 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb334039ad0 con 0x558999069700 2024-09-15T22:12:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb338009180 con 0x558999069700 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb345bc4740 10 client.15579 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.253+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589990eee90 con 0x558999069700 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb334039ad0 con 0x558999069700 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb33801ebf0 con 0x558999069700 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 10 client.15579 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 10 client.15579 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 20 client.15579 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5589990f3990 0x5589990f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 4 client.15579 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "status"}) -- 0x558998fd2b10 con 0x5589990f3990 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb341587640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5589990f3990 0x5589990f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb324ff9640 10 client.15579 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.254+0000 7fb341587640 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5589990f3990 0x5589990f26a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e6112b774a4b6f3 server_cookie=8f176a50d82e8cbc in_seq=0 out_seq=0 2024-09-15T22:12:50.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb324ff9640 1 -- 172.21.15.77:0/1380610028 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558998fd2b10 con 0x5589990f3990 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb324ff9640 10 client.15579 handle_command_reply: tid=0 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 client.15579 shutdown 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 msgr2=0x558999089930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb338097da0 tx=0x7fb33801fea0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 shutdown_connections 2024-09-15T22:12:50.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5589990f3990 0x5589990f26a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb324ff9640 0 client.15579 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558999073fc0 0x558999089f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558999069700 0x558999089930 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 --2- 172.21.15.77:0/1380610028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558999068de0 0x55899908e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 >> 172.21.15.77:0/1380610028 conn(0x558999065370 msgr2=0x558999060000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 shutdown_connections 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 1 -- 172.21.15.77:0/1380610028 wait complete. 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb345bc4740 20 client.15579 trim_cache size 0 max 16384 2024-09-15T22:12:50.279 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14604, 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:50.280 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 131.57444650599999, 2024-09-15T22:12:50.281 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 30, 2024-09-15T22:12:50.281 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 91, 2024-09-15T22:12:50.281 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 90, 2024-09-15T22:12:50.281 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 187.57793969100001 2024-09-15T22:12:50.281 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 msgr2=0x7fb3280a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 0x7fb3280a5a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb33404b000 tx=0x7fb33404c000 comp rx=0 tx=0).stop 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 msgr2=0x7fb33c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.255+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b3f0 tx=0x7fb33800b8b0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 shutdown_connections 2024-09-15T22:12:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb3280a3550 0x7fb3280a5a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb33c10c170 0x7fb33c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb33c103660 0x7fb33c198a90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 --2- 172.21.15.77:0/2589422983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb33c102d40 0x7fb33c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 >> 172.21.15.77:0/2589422983 conn(0x7fb33c0fc7a0 msgr2=0x7fb33c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 shutdown_connections 2024-09-15T22:12:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:50.256+0000 7fb3437eb640 1 -- 172.21.15.77:0/2589422983 wait complete. 2024-09-15T22:12:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:50 smithi077 ceph-mon[30980]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:12:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:50 smithi196 ceph-mon[35109]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:12:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:50 smithi117 ceph-mon[35021]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:12:51.020 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-15T22:12:51.263 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 -- 172.21.15.77:0/160516544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4106f10 msgr2=0x7ff9d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/160516544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4106f10 0x7ff9d410ea90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:12:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 -- 172.21.15.77:0/160516544 shutdown_connections 2024-09-15T22:12:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/160516544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4106f10 0x7ff9d410ea90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/160516544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 0x7ff9d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/160516544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4101de0 0x7ff9d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.742+0000 7ff9dc04e640 1 -- 172.21.15.77:0/160516544 >> 172.21.15.77:0/160516544 conn(0x7ff9d40fd8c0 msgr2=0x7ff9d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.743+0000 7ff9dc04e640 1 -- 172.21.15.77:0/160516544 shutdown_connections 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.743+0000 7ff9dc04e640 1 -- 172.21.15.77:0/160516544 wait complete. 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.743+0000 7ff9dc04e640 1 Processor -- start 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.743+0000 7ff9dc04e640 1 -- start start 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 0x7ff9d41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4106f10 0x7ff9d419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4111f70 con 0x7ff9d4101de0 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4111df0 con 0x7ff9d4102790 2024-09-15T22:12:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d41120f0 con 0x7ff9d4106f10 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54344/0 (socket says 172.21.15.77:54344) 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 -- 172.21.15.77:0/3463189751 learned_addr learned my addr 172.21.15.77:0/3463189751 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4106f10 0x7ff9d419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 0x7ff9d41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 -- 172.21.15.77:0/3463189751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4106f10 msgr2=0x7ff9d419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4106f10 0x7ff9d419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 -- 172.21.15.77:0/3463189751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 msgr2=0x7ff9d41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 0x7ff9d41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 -- 172.21.15.77:0/3463189751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d419d420 con 0x7ff9d4101de0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400cce0 tx=0x7ff9c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d2ffd640 1 -- 172.21.15.77:0/3463189751 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004b20 con 0x7ff9d4101de0 2024-09-15T22:12:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d2ffd640 1 -- 172.21.15.77:0/3463189751 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4004cc0 con 0x7ff9d4101de0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d419d710 con 0x7ff9d4101de0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.744+0000 7ff9d2ffd640 1 -- 172.21.15.77:0/3463189751 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400f7d0 con 0x7ff9d4101de0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.745+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d41a4ff0 con 0x7ff9d4101de0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.747+0000 7ff9d2ffd640 1 -- 172.21.15.77:0/3463189751 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7ff9c400f970 con 0x7ff9d4101de0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.747+0000 7ff9de427740 1 Processor -- start 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 -- start start 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f07c2f0 0x55585f086670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 0x55585f088fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55585efe2090 con 0x55585f07b9d0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55585ef89a70 con 0x55585f07c2f0 2024-09-15T22:12:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9de427740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55585efde050 con 0x55585f086bb0 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54352/0 (socket says 172.21.15.77:54352) 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9d2ffd640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 0x7ff9bc0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 -- 172.21.15.77:0/1413865731 learned_addr learned my addr 172.21.15.77:0/1413865731 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 0x55585f088fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f07c2f0 0x55585f086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 0x7ff9bc0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 -- 172.21.15.77:0/1413865731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 msgr2=0x55585f088fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 0x55585f088fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 -- 172.21.15.77:0/1413865731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f07c2f0 msgr2=0x55585f086670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f07c2f0 0x55585f086670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9d2ffd640 1 -- 172.21.15.77:0/3463189751 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff9c40d0390 con 0x7ff9d4101de0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9da5eb640 1 -- 172.21.15.77:0/1413865731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585ef96130 con 0x55585f07b9d0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.748+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 0x55585f088fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.749+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4941c4ffd4d30218 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.749+0000 7ff9bbfff640 1 -- 172.21.15.77:0/1413865731 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc035040 con 0x55585f07b9d0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.749+0000 7ff9bbfff640 1 -- 172.21.15.77:0/1413865731 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc0315f0 con 0x55585f07b9d0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.749+0000 7ff9bbfff640 1 -- 172.21.15.77:0/1413865731 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0486c0 con 0x55585f07b9d0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.749+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 0x7ff9bc0a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc04b000 tx=0x7ff9cc04a000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- 172.21.15.77:0/1413865731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 msgr2=0x55585f07bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc031b30 comp rx=0 tx=0).stop 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- 172.21.15.77:0/1413865731 shutdown_connections 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f086bb0 0x55585f088fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f07c2f0 0x55585f086670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 --2- 172.21.15.77:0/1413865731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f07bdb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- 172.21.15.77:0/1413865731 >> 172.21.15.77:0/1413865731 conn(0x55585f077f60 msgr2=0x55585f078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- 172.21.15.77:0/1413865731 shutdown_connections 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- 172.21.15.77:0/1413865731 wait complete. 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 -- start start 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.750+0000 7ff9de427740 1 Processor -- start 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 0x55585f086470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f083d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55585f089780 con 0x55585f07c2f0 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55585ef9cf20 con 0x55585f086bb0 2024-09-15T22:12:51.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55585f089900 con 0x55585f07b9d0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 10 client.0 _ll_register_callbacks cb 0x55585ef66030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54364/0 (socket says 172.21.15.77:54364) 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 -- 172.21.15.77:0/4034458945 learned_addr learned my addr 172.21.15.77:0/4034458945 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f083d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 0x55585f086470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 -- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 msgr2=0x55585f086470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 0x55585f086470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 -- 172.21.15.77:0/4034458945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 msgr2=0x55585f083d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f083d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 -- 172.21.15.77:0/4034458945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585ed532b0 con 0x55585f07c2f0 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 0x55585f086470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b9f0 tx=0x7ff9c800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ffeb7a517c2651 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8003e20 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff9c8005020 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc0315f0 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c8005220 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 10 client.15591 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.751+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55585f095130 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc0315f0 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff9c801b570 con 0x55585f07c2f0 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 10 client.15591 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 10 client.15591 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 20 client.15591 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f097790 0x55585f098c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 4 client.15591 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55585efef600 con 0x55585f097790 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.752+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f097790 0x55585f098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.753+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f097790 0x55585f098c00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14d0a3e5482e27d8 server_cookie=d3555189bf669928 in_seq=0 out_seq=0 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.753+0000 7ff9b97fa640 10 client.15591 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.753+0000 7ff9b97fa640 1 -- 172.21.15.77:0/4034458945 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55585efef600 con 0x55585f097790 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.753+0000 7ff9b97fa640 10 client.15591 handle_command_reply: tid=0 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 client.15591 shutdown 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 msgr2=0x55585f0837c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b9f0 tx=0x7ff9c800beb0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 shutdown_connections 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f097790 0x55585f098c00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f083d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07c2f0 0x55585f0837c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9b97fa640 0 client.15591 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 --2- 172.21.15.77:0/4034458945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07b9d0 0x55585f086470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 >> 172.21.15.77:0/4034458945 conn(0x55585f077f60 msgr2=0x55585f07dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 shutdown_connections 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 1 -- 172.21.15.77:0/4034458945 wait complete. 2024-09-15T22:12:51.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.754+0000 7ff9de427740 20 client.15591 trim_cache size 0 max 16384 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 Processor -- start 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 -- start start 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f085620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55585f096f40 con 0x55585f07b9d0 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55585f097610 con 0x55585f086bb0 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9de427740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55585f097ce0 con 0x55585f07c2f0 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54366/0 (socket says 172.21.15.77:54366) 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 -- 172.21.15.77:0/1616073520 learned_addr learned my addr 172.21.15.77:0/1616073520 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f085620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 -- 172.21.15.77:0/1616073520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 msgr2=0x55585f072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f072ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 -- 172.21.15.77:0/1616073520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 msgr2=0x55585f085620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f085620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 -- 172.21.15.77:0/1616073520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585f082770 con 0x55585f07b9d0 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f085620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff9c409c0c0 tx=0x7ff9c40d27e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba521329dbb2d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.772+0000 7ff9bb7fe640 1 -- 172.21.15.77:0/1616073520 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c401d040 con 0x55585f07b9d0 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9bb7fe640 1 -- 172.21.15.77:0/1616073520 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c409a1d0 con 0x55585f07b9d0 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9bb7fe640 1 -- 172.21.15.77:0/1616073520 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40987a0 con 0x55585f07b9d0 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 -- 172.21.15.77:0/1616073520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 msgr2=0x55585f098c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff9c409c0c0 tx=0x7ff9c40d27e0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 -- 172.21.15.77:0/1616073520 shutdown_connections 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f085620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 --2- 172.21.15.77:0/1616073520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f098c00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 -- 172.21.15.77:0/1616073520 >> 172.21.15.77:0/1616073520 conn(0x55585f077f60 msgr2=0x55585f084f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 -- 172.21.15.77:0/1616073520 shutdown_connections 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.773+0000 7ff9de427740 1 -- 172.21.15.77:0/1616073520 wait complete. 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 -- start start 2024-09-15T22:12:51.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 Processor -- start 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f09c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f09cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55585f098a80 con 0x55585f07b9d0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55585f0983b0 con 0x55585f086bb0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55585f099150 con 0x55585f07c2f0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9de427740 10 client.0 _ll_register_callbacks cb 0x7ff9c8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9d9dea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54372/0 (socket says 172.21.15.77:54372) 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.774+0000 7ff9d9dea640 1 -- 172.21.15.77:0/2607921876 learned_addr learned my addr 172.21.15.77:0/2607921876 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9da5eb640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f09cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f09c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 -- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 msgr2=0x55585f09c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f09c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 -- 172.21.15.77:0/2607921876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 msgr2=0x55585f09cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f09cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 -- 172.21.15.77:0/2607921876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585f09d260 con 0x55585f07b9d0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9d9dea640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff9c409be30 tx=0x7ff9c4004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a26abf2592d3543 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b4070 con 0x55585f07b9d0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff9c40bc040 con 0x55585f07b9d0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c409a1d0 con 0x55585f07b9d0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4084050 con 0x55585f07b9d0 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9de427740 10 client.15597 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:51.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55585f101be0 con 0x55585f07b9d0 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c409a1d0 con 0x55585f07b9d0 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff9c40cd8a0 con 0x55585f07b9d0 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.775+0000 7ff9de427740 10 client.15597 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9de427740 10 client.15597 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9de427740 20 client.15597 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f106270 0x55585f104f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9de427740 4 client.15597 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "status"}) -- 0x55585efe9b10 con 0x55585f106270 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f106270 0x55585f104f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9b97fa640 10 client.15597 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9d95e9640 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f106270 0x55585f104f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=209074dd558f8395 server_cookie=30674bead9b7f52 in_seq=0 out_seq=0 2024-09-15T22:12:51.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.776+0000 7ff9b97fa640 1 -- 172.21.15.77:0/2607921876 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55585efe9b10 con 0x55585f106270 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9b97fa640 10 client.15597 handle_command_reply: tid=0 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 client.15597 shutdown 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 msgr2=0x55585f0a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff9c409be30 tx=0x7ff9c4004f20 comp rx=0 tx=0).stop 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 shutdown_connections 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55585f106270 0x55585f104f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55585f086bb0 0x55585f09cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55585f07c2f0 0x55585f09c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9b97fa640 0 client.15597 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 --2- 172.21.15.77:0/2607921876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55585f07b9d0 0x55585f0a13e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 >> 172.21.15.77:0/2607921876 conn(0x55585f077f60 msgr2=0x55585f085650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 shutdown_connections 2024-09-15T22:12:51.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 1 -- 172.21.15.77:0/2607921876 wait complete. 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9de427740 20 client.15597 trim_cache size 0 max 16384 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24424, 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:51.794 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 133.44290002100001, 2024-09-15T22:12:51.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 30, 2024-09-15T22:12:51.795 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 91, 2024-09-15T22:12:51.795 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 90, 2024-09-15T22:12:51.795 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 200.56401529999999 2024-09-15T22:12:51.795 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 msgr2=0x7ff9bc0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 0x7ff9bc0a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc04b000 tx=0x7ff9cc04a000 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 msgr2=0x7ff9d4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.777+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400cce0 tx=0x7ff9c400a620 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 shutdown_connections 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff9bc0a34d0 0x7ff9bc0a5990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9d4106f10 0x7ff9d419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9d4102790 0x7ff9d41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 --2- 172.21.15.77:0/3463189751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9d4101de0 0x7ff9d4198470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 >> 172.21.15.77:0/3463189751 conn(0x7ff9d40fd8c0 msgr2=0x7ff9d40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 shutdown_connections 2024-09-15T22:12:51.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:51.778+0000 7ff9dc04e640 1 -- 172.21.15.77:0/3463189751 wait complete. 2024-09-15T22:12:52.544 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:52 smithi077 ceph-mon[30980]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-15T22:12:52.571 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:12:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:52 smithi196 ceph-mon[35109]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-15T22:12:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:52 smithi117 ceph-mon[35021]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-15T22:12:52.815 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.309+0000 7f121295d640 1 -- 172.21.15.77:0/1146754824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c101df0 msgr2=0x7f120c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.309+0000 7f121295d640 1 --2- 172.21.15.77:0/1146754824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c101df0 0x7f120c1021d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009900 tx=0x7f11fc02a410 comp rx=0 tx=0).stop 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 -- 172.21.15.77:0/1146754824 shutdown_connections 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 --2- 172.21.15.77:0/1146754824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c106f20 0x7f120c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 --2- 172.21.15.77:0/1146754824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 0x7f120c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 --2- 172.21.15.77:0/1146754824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c101df0 0x7f120c1021d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 -- 172.21.15.77:0/1146754824 >> 172.21.15.77:0/1146754824 conn(0x7f120c0fd8b0 msgr2=0x7f120c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 -- 172.21.15.77:0/1146754824 shutdown_connections 2024-09-15T22:12:53.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 -- 172.21.15.77:0/1146754824 wait complete. 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 Processor -- start 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.310+0000 7f121295d640 1 -- start start 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c101df0 0x7f120c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 0x7f120c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111f80 con 0x7f120c106f20 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111e00 con 0x7f120c1027a0 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f121295d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f120c112100 con 0x7f120c101df0 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54390/0 (socket says 172.21.15.77:54390) 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 -- 172.21.15.77:0/3246186332 learned_addr learned my addr 172.21.15.77:0/3246186332 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f120b7fe640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 0x7f120c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f120bfff640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c101df0 0x7f120c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 -- 172.21.15.77:0/3246186332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c101df0 msgr2=0x7f120c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c101df0 0x7f120c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 -- 172.21.15.77:0/3246186332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 msgr2=0x7f120c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 0x7f120c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.311+0000 7f1210efa640 1 -- 172.21.15.77:0/3246186332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19d350 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f1210efa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f120400cce0 tx=0x7f120400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f12097fa640 1 -- 172.21.15.77:0/3246186332 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400f450 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f12097fa640 1 -- 172.21.15.77:0/3246186332 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400fad0 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19d640 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f12097fa640 1 -- 172.21.15.77:0/3246186332 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204018c10 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.312+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a4f20 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.314+0000 7f12097fa640 1 -- 172.21.15.77:0/3246186332 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f120400f5f0 con 0x7f120c106f20 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 Processor -- start 2024-09-15T22:12:53.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 -- start start 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb964db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb8cb090 con 0x55e6bb96fbb0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb872a70 con 0x55e6bb9652f0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1214d36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb8c7050 con 0x55e6bb9649d0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f12097fa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 0x7f11f40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54398/0 (socket says 172.21.15.77:54398) 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 -- 172.21.15.77:0/431739382 learned_addr learned my addr 172.21.15.77:0/431739382 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb964db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120bfff640 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f1210efa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 0x7f11f40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f12097fa640 1 -- 172.21.15.77:0/3246186332 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f12040d08f0 con 0x7f120c106f20 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 -- 172.21.15.77:0/431739382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 msgr2=0x55e6bb964db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb964db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 -- 172.21.15.77:0/431739382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 msgr2=0x55e6bb96f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.315+0000 7f120b7fe640 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f120b7fe640 1 -- 172.21.15.77:0/431739382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bb87f130 con 0x55e6bb96fbb0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f120b7fe640 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1200011730 tx=0x7f120000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0b5e988cb318a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f1210efa640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 0x7f11f40a5990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdb80 tx=0x7f1204066fd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f11f27fc640 1 -- 172.21.15.77:0/431739382 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200004020 con 0x55e6bb96fbb0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f11f27fc640 1 -- 172.21.15.77:0/431739382 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1200005500 con 0x55e6bb96fbb0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.316+0000 7f11f27fc640 1 -- 172.21.15.77:0/431739382 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200004c30 con 0x55e6bb96fbb0 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- 172.21.15.77:0/431739382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 msgr2=0x55e6bb971fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1200011730 tx=0x7f120000a6a0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- 172.21.15.77:0/431739382 shutdown_connections 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb971fa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- 172.21.15.77:0/431739382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb964db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- 172.21.15.77:0/431739382 >> 172.21.15.77:0/431739382 conn(0x55e6bb960f60 msgr2=0x55e6bb961340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- 172.21.15.77:0/431739382 shutdown_connections 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- 172.21.15.77:0/431739382 wait complete. 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- start start 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 Processor -- start 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb972780 con 0x55e6bb9649d0 2024-09-15T22:12:53.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb885f20 con 0x55e6bb96fbb0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.317+0000 7f1214d36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb972900 con 0x55e6bb9652f0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f1214d36740 10 client.0 _ll_register_callbacks cb 0x55e6bb84f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54408/0 (socket says 172.21.15.77:54408) 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 -- 172.21.15.77:0/610618316 learned_addr learned my addr 172.21.15.77:0/610618316 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120bfff640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 -- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9652f0 msgr2=0x55e6bb96c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 -- 172.21.15.77:0/610618316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 msgr2=0x55e6bb96cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 -- 172.21.15.77:0/610618316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bb63c2b0 con 0x55e6bb9649d0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f120b7fe640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1200004be0 tx=0x7f1200004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ee4c4d09faeed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f11eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120000fba0 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f120000fd40 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1200005500 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12000046c0 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f1214d36740 10 client.15609 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.318+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6bb97e100 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1200005500 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1200025020 con 0x55e6bb9649d0 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 10 client.15609 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 20 client.15609 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9802e0 0x55e6bb981750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 4 client.15609 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:53.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6bb8d8600 con 0x55e6bb9802e0 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.319+0000 7f120bfff640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9802e0 0x55e6bb981750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f11eb7fe640 10 client.15609 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f120bfff640 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9802e0 0x55e6bb981750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64c425007fa3a340 server_cookie=3ce5148d53bd1b2a in_seq=0 out_seq=0 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f11eb7fe640 1 -- 172.21.15.77:0/610618316 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6bb8d8600 con 0x55e6bb9802e0 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f11eb7fe640 10 client.15609 handle_command_reply: tid=0 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 client.15609 shutdown 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 msgr2=0x55e6bb96f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1200004be0 tx=0x7f1200004c10 comp rx=0 tx=0).stop 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 shutdown_connections 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9802e0 0x55e6bb981750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f11eb7fe640 0 client.15609 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:53.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 --2- 172.21.15.77:0/610618316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9649d0 0x55e6bb96f4c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.320+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 >> 172.21.15.77:0/610618316 conn(0x55e6bb960f60 msgr2=0x55e6bb966c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.321+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 shutdown_connections 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.321+0000 7f1214d36740 1 -- 172.21.15.77:0/610618316 wait complete. 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.321+0000 7f1214d36740 20 client.15609 trim_cache size 0 max 16384 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 Processor -- start 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 -- start start 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb981750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb97fa90 con 0x55e6bb96fbb0 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb980160 con 0x55e6bb9652f0 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.338+0000 7f1214d36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb980830 con 0x55e6bb9649d0 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54414/0 (socket says 172.21.15.77:54414) 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 -- 172.21.15.77:0/3433656095 learned_addr learned my addr 172.21.15.77:0/3433656095 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb981750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120bfff640 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 -- 172.21.15.77:0/3433656095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 msgr2=0x55e6bb981750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb981750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 -- 172.21.15.77:0/3433656095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 msgr2=0x55e6bb96c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 -- 172.21.15.77:0/3433656095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bb96f830 con 0x55e6bb96fbb0 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f120b7fe640 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1200005360 tx=0x7f120000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d4b65d9ed04f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f11f1ffb640 1 -- 172.21.15.77:0/3433656095 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200027440 con 0x55e6bb96fbb0 2024-09-15T22:12:53.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f11f1ffb640 1 -- 172.21.15.77:0/3433656095 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120000a2d0 con 0x55e6bb96fbb0 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.339+0000 7f11f1ffb640 1 -- 172.21.15.77:0/3433656095 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120001e760 con 0x55e6bb96fbb0 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- 172.21.15.77:0/3433656095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 msgr2=0x55e6bb96e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1200005360 tx=0x7f120000a6a0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- 172.21.15.77:0/3433656095 shutdown_connections 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb96fbb0 0x55e6bb96e670 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb9652f0 0x55e6bb96c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 --2- 172.21.15.77:0/3433656095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb981750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- 172.21.15.77:0/3433656095 >> 172.21.15.77:0/3433656095 conn(0x55e6bb960f60 msgr2=0x55e6bb96df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- 172.21.15.77:0/3433656095 shutdown_connections 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- 172.21.15.77:0/3433656095 wait complete. 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.340+0000 7f1214d36740 1 -- start start 2024-09-15T22:12:53.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 Processor -- start 2024-09-15T22:12:53.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb982aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb9e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb9815d0 con 0x55e6bb9652f0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb980f00 con 0x55e6bb96fbb0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e6bb981ca0 con 0x55e6bb9649d0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f1214d36740 10 client.0 _ll_register_callbacks cb 0x7f11fc0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54430/0 (socket says 172.21.15.77:54430) 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 -- 172.21.15.77:0/206517628 learned_addr learned my addr 172.21.15.77:0/206517628 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120b7fe640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb982aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120b7fe640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb9e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 -- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 msgr2=0x55e6bb982aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb982aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 -- 172.21.15.77:0/206517628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 msgr2=0x55e6bb9e89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb9e89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 -- 172.21.15.77:0/206517628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bb983880 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120b7fe640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb982aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f120bfff640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8200eb9c0cd73ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.341+0000 7f11eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc04a070 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f11fc004450 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120000a2d0 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11fc057050 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 10 client.15615 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6bb9ea080 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120000a2d0 con 0x55e6bb9652f0 2024-09-15T22:12:53.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f11fc0035f0 con 0x55e6bb9652f0 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 10 client.15615 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 10 client.15615 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 20 client.15615 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9eec80 0x55e6bb9ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 4 client.15615 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.342+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "status"}) -- 0x55e6bb8d2b10 con 0x55e6bb9eec80 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f120b7fe640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9eec80 0x55e6bb9ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f120b7fe640 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9eec80 0x55e6bb9ed990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9be4601eed7047a server_cookie=57dedbf170f9890a in_seq=0 out_seq=0 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f11eb7fe640 10 client.15615 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f11eb7fe640 1 -- 172.21.15.77:0/206517628 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e6bb8d2b10 con 0x55e6bb9eec80 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f11eb7fe640 10 client.15615 handle_command_reply: tid=0 2024-09-15T22:12:53.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.343+0000 7f1214d36740 1 client.15615 shutdown 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 msgr2=0x55e6bb982fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc031af0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 shutdown_connections 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e6bb9eec80 0x55e6bb9ed990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6bb96fbb0 0x55e6bb9e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f11eb7fe640 0 client.15615 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e6bb9652f0 0x55e6bb982fe0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 --2- 172.21.15.77:0/206517628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e6bb9649d0 0x55e6bb982aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 >> 172.21.15.77:0/206517628 conn(0x55e6bb960f60 msgr2=0x55e6bb960000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 shutdown_connections 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 1 -- 172.21.15.77:0/206517628 wait complete. 2024-09-15T22:12:53.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f1214d36740 20 client.15615 trim_cache size 0 max 16384 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24439, 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 136.01648052799999, 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 30, 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 91, 2024-09-15T22:12:53.363 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 90, 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 196.99191422499999 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 msgr2=0x7f11f40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 0x7f11f40a5990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12040cdb80 tx=0x7f1204066fd0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 msgr2=0x7f120c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:53.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.344+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f120400cce0 tx=0x7f120400a620 comp rx=0 tx=0).stop 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 shutdown_connections 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f11f40a34d0 0x7f11f40a5990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f120c106f20 0x7f120c19cc70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f120c1027a0 0x7f120c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 --2- 172.21.15.77:0/3246186332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f120c101df0 0x7f120c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 >> 172.21.15.77:0/3246186332 conn(0x7f120c0fd8b0 msgr2=0x7f120c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 shutdown_connections 2024-09-15T22:12:53.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:53.345+0000 7f121295d640 1 -- 172.21.15.77:0/3246186332 wait complete. 2024-09-15T22:12:54.117 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:12:54.361 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:54 smithi077 ceph-mon[30980]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-15T22:12:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:54 smithi196 ceph-mon[35109]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-15T22:12:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:54 smithi117 ceph-mon[35021]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.819+0000 7ff58a2db640 1 -- 172.21.15.77:0/1467490522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 msgr2=0x7ff584106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.819+0000 7ff58a2db640 1 --2- 172.21.15.77:0/1467490522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff584106e60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff574009930 tx=0x7ff57402a420 comp rx=0 tx=0).stop 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 -- 172.21.15.77:0/1467490522 shutdown_connections 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 --2- 172.21.15.77:0/1467490522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff58410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 --2- 172.21.15.77:0/1467490522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 --2- 172.21.15.77:0/1467490522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff584106e60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 -- 172.21.15.77:0/1467490522 >> 172.21.15.77:0/1467490522 conn(0x7ff5840fc7a0 msgr2=0x7ff5840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 -- 172.21.15.77:0/1467490522 shutdown_connections 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 -- 172.21.15.77:0/1467490522 wait complete. 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 Processor -- start 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.820+0000 7ff58a2db640 1 -- start start 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff584198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff584111fa0 con 0x7ff584106a80 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff584111e20 con 0x7ff584101420 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff584112120 con 0x7ff584100a80 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54458/0 (socket says 172.21.15.77:54458) 2024-09-15T22:12:54.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 -- 172.21.15.77:0/392467365 learned_addr learned my addr 172.21.15.77:0/392467365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff583fff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff57bfff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff584198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 -- 172.21.15.77:0/392467365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 msgr2=0x7ff584198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 -- 172.21.15.77:0/392467365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 msgr2=0x7ff584198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff584198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 -- 172.21.15.77:0/392467365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58419d4d0 con 0x7ff584106a80 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff583fff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff57bfff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff584198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff588878640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00eaa0 tx=0x7ff57c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff581ffb640 1 -- 172.21.15.77:0/392467365 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00cdb0 con 0x7ff584106a80 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58419d7c0 con 0x7ff584106a80 2024-09-15T22:12:54.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff581ffb640 1 -- 172.21.15.77:0/392467365 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c005020 con 0x7ff584106a80 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff581ffb640 1 -- 172.21.15.77:0/392467365 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0116b0 con 0x7ff584106a80 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.821+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841a50a0 con 0x7ff584106a80 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.824+0000 7ff581ffb640 1 -- 172.21.15.77:0/392467365 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7ff57c005560 con 0x7ff584106a80 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 Processor -- start 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 -- start start 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55787035f8d0 0x55787035fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x5578703713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5578702c6090 con 0x5578703601f0 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787026da70 con 0x55787036efc0 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578702c2050 con 0x55787035f8d0 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54462/0 (socket says 172.21.15.77:54462) 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 -- 172.21.15.77:0/2952428497 learned_addr learned my addr 172.21.15.77:0/2952428497 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff57bfff640 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55787035f8d0 0x55787035fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 -- 172.21.15.77:0/2952428497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55787035f8d0 msgr2=0x55787035fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff57bfff640 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x5578703713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55787035f8d0 0x55787035fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 -- 172.21.15.77:0/2952428497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 msgr2=0x5578703713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x5578703713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 -- 172.21.15.77:0/2952428497 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787027a130 con 0x5578703601f0 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff581ffb640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 0x7ff5700a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.825+0000 7ff583fff640 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff574009930 tx=0x7ff574032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1661696bd11b15d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff57bfff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 0x7ff5700a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff57a7fc640 1 -- 172.21.15.77:0/2952428497 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57403f070 con 0x5578703601f0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff581ffb640 1 -- 172.21.15.77:0/392467365 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff57c0d0640 con 0x7ff584106a80 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff57a7fc640 1 -- 172.21.15.77:0/2952428497 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5740317b0 con 0x5578703601f0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff57a7fc640 1 -- 172.21.15.77:0/2952428497 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574035070 con 0x5578703601f0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2952428497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 msgr2=0x55787036ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff57bfff640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 0x7ff5700a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x557870361470 tx=0x7ff56c008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff574009930 tx=0x7ff574032d60 comp rx=0 tx=0).stop 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2952428497 shutdown_connections 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x5578703713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578703601f0 0x55787036ea80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2952428497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55787035f8d0 0x55787035fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2952428497 >> 172.21.15.77:0/2952428497 conn(0x55787035bf60 msgr2=0x55787035c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2952428497 shutdown_connections 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.826+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2952428497 wait complete. 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 -- start start 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 Processor -- start 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x55787036bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x55787036c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557870371b90 con 0x55787035f8d0 2024-09-15T22:12:54.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557870280f20 con 0x55787036efc0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557870371d10 con 0x5578703601f0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff58c6b4740 10 client.0 _ll_register_callbacks cb 0x55787024a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54470/0 (socket says 172.21.15.77:54470) 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.827+0000 7ff583fff640 1 -- 172.21.15.77:0/3530028332 learned_addr learned my addr 172.21.15.77:0/3530028332 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff57bfff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x55787036bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff588878640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x55787036c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 -- 172.21.15.77:0/3530028332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 msgr2=0x55787036bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x55787036bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 -- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 msgr2=0x55787036c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x55787036c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 -- 172.21.15.77:0/3530028332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578700372b0 con 0x55787035f8d0 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff57bfff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x55787036bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:54.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff588878640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x55787036c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff583fff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff57402a940 tx=0x7ff5740053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5243a4de6549f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff5637fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57404a020 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff5740037f0 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5740317b0 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff574055050 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff58c6b4740 10 client.15627 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55787037d4c0 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.828+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5740317b0 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff574004da0 con 0x55787035f8d0 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 10 client.15627 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 10 client.15627 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 20 client.15627 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:54.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55787037f7f0 0x557870380c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 4 client.15627 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578702d3600 con 0x55787037f7f0 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff57bfff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55787037f7f0 0x557870380c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff57bfff640 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55787037f7f0 0x557870380c60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad5fd47a27719ef server_cookie=49774f77ca2870ff in_seq=0 out_seq=0 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.829+0000 7ff5637fe640 10 client.15627 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff5637fe640 1 -- 172.21.15.77:0/3530028332 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578702d3600 con 0x55787037f7f0 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff5637fe640 10 client.15627 handle_command_reply: tid=0 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 client.15627 shutdown 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 msgr2=0x55787036e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff57402a940 tx=0x7ff5740053b0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 shutdown_connections 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55787037f7f0 0x557870380c60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x55787036c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x55787036bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/3530028332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787036e830 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff5637fe640 0 client.15627 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.830+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 >> 172.21.15.77:0/3530028332 conn(0x55787035bf60 msgr2=0x557870361cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.831+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 shutdown_connections 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.831+0000 7ff58c6b4740 1 -- 172.21.15.77:0/3530028332 wait complete. 2024-09-15T22:12:54.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.831+0000 7ff58c6b4740 20 client.15627 trim_cache size 0 max 16384 2024-09-15T22:12:54.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.848+0000 7ff58c6b4740 1 Processor -- start 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.848+0000 7ff58c6b4740 1 -- start start 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.848+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.848+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x557870380590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.848+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x557870356ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55787037efa0 con 0x55787035f8d0 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55787037f670 con 0x55787036efc0 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55787037fd40 con 0x5578703601f0 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54482/0 (socket says 172.21.15.77:54482) 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 -- 172.21.15.77:0/1778172304 learned_addr learned my addr 172.21.15.77:0/1778172304 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 -- 172.21.15.77:0/1778172304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 msgr2=0x557870380590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x557870380590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 -- 172.21.15.77:0/1778172304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 msgr2=0x557870356ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x557870356ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 -- 172.21.15.77:0/1778172304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787036a0d0 con 0x55787035f8d0 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff583fff640 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5740054c0 tx=0x7ff5740058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83baf733408ce068 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff579ffb640 1 -- 172.21.15.77:0/1778172304 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57404a070 con 0x55787035f8d0 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff579ffb640 1 -- 172.21.15.77:0/1778172304 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5740315b0 con 0x55787035f8d0 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.849+0000 7ff579ffb640 1 -- 172.21.15.77:0/1778172304 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57403f070 con 0x55787035f8d0 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- 172.21.15.77:0/1778172304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 msgr2=0x55787037f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5740054c0 tx=0x7ff5740058b0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- 172.21.15.77:0/1778172304 shutdown_connections 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787036efc0 0x557870356ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x557870380590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/1778172304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787035f8d0 0x55787037f120 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- 172.21.15.77:0/1778172304 >> 172.21.15.77:0/1778172304 conn(0x55787035bf60 msgr2=0x557870367240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- 172.21.15.77:0/1778172304 shutdown_connections 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- 172.21.15.77:0/1778172304 wait complete. 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 -- start start 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.850+0000 7ff58c6b4740 1 Processor -- start 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787035f8d0 0x557870381e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x5578703823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557870380ae0 con 0x55787036efc0 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557870380410 con 0x55787035f8d0 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578703811b0 con 0x5578703601f0 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff58c6b4740 10 client.0 _ll_register_callbacks cb 0x7ff574032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:54.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x5578703823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54496/0 (socket says 172.21.15.77:54496) 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 -- 172.21.15.77:0/2370242270 learned_addr learned my addr 172.21.15.77:0/2370242270 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff583fff640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787035f8d0 0x557870381e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 -- 172.21.15.77:0/2370242270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 msgr2=0x5578703823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x5578703823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 -- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787035f8d0 msgr2=0x557870381e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787035f8d0 0x557870381e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 -- 172.21.15.77:0/2370242270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578703e7f20 con 0x55787036efc0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff588878640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0cd9e0 tx=0x7ff57c09cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c27ec48717893f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff5637fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c01f040 con 0x55787036efc0 2024-09-15T22:12:54.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.851+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff57c0cdbb0 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5740315b0 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff57c0a5000 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 10 client.15633 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578703e9220 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5740315b0 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff57c00f040 con 0x55787036efc0 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 10 client.15633 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 10 client.15633 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 20 client.15633 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5578703ede10 0x5578703ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 4 client.15633 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "status"}) -- 0x5578702cdb10 con 0x5578703ede10 2024-09-15T22:12:54.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.852+0000 7ff583fff640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5578703ede10 0x5578703ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff5637fe640 10 client.15633 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff583fff640 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5578703ede10 0x5578703ecb20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e89186efb2344a7 server_cookie=6e567616236cdc3 in_seq=0 out_seq=0 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff5637fe640 1 -- 172.21.15.77:0/2370242270 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578702cdb10 con 0x5578703ede10 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff5637fe640 10 client.15633 handle_command_reply: tid=0 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 client.15633 shutdown 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 msgr2=0x5578703828e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0cd9e0 tx=0x7ff57c09cb80 comp rx=0 tx=0).stop 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 shutdown_connections 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5578703ede10 0x5578703ecb20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55787036efc0 0x5578703828e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578703601f0 0x5578703823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 --2- 172.21.15.77:0/2370242270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55787035f8d0 0x557870381e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 >> 172.21.15.77:0/2370242270 conn(0x55787035bf60 msgr2=0x557870367240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff5637fe640 0 client.15633 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 shutdown_connections 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 1 -- 172.21.15.77:0/2370242270 wait complete. 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.853+0000 7ff58c6b4740 20 client.15633 trim_cache size 0 max 16384 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24440, 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:54.864 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 138.53084029499999, 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 30, 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 91, 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 90, 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 201.03938010499999 2024-09-15T22:12:54.865 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 msgr2=0x7ff5700a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 0x7ff5700a5990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x557870361470 tx=0x7ff56c008040 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 msgr2=0x7ff58419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00eaa0 tx=0x7ff57c00ef60 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 shutdown_connections 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5700a34d0 0x7ff5700a5990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff584106a80 0x7ff58419cdf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff584101420 0x7ff584198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 --2- 172.21.15.77:0/392467365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff584100a80 0x7ff584198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 >> 172.21.15.77:0/392467365 conn(0x7ff5840fc7a0 msgr2=0x7ff5840fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 shutdown_connections 2024-09-15T22:12:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:54.854+0000 7ff58a2db640 1 -- 172.21.15.77:0/392467365 wait complete. 2024-09-15T22:12:55.879 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:12:56.122 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:56.504 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:56 smithi077 ceph-mon[30980]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:12:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:56 smithi196 ceph-mon[35109]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:12:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:56 smithi117 ceph-mon[35021]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 -- 172.21.15.77:0/1288233678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac106a80 msgr2=0x7f99ac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 --2- 172.21.15.77:0/1288233678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac106a80 0x7f99ac106e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009930 tx=0x7f99a402a420 comp rx=0 tx=0).stop 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 -- 172.21.15.77:0/1288233678 shutdown_connections 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 --2- 172.21.15.77:0/1288233678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac101420 0x7f99ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 --2- 172.21.15.77:0/1288233678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac100a80 0x7f99ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 --2- 172.21.15.77:0/1288233678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac106a80 0x7f99ac106e60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 -- 172.21.15.77:0/1288233678 >> 172.21.15.77:0/1288233678 conn(0x7f99ac0fc7a0 msgr2=0x7f99ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 -- 172.21.15.77:0/1288233678 shutdown_connections 2024-09-15T22:12:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.581+0000 7f99b3560640 1 -- 172.21.15.77:0/1288233678 wait complete. 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 Processor -- start 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 -- start start 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac101420 0x7f99ac1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 0x7f99ac1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac110e80 con 0x7f99ac100a80 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac110d00 con 0x7f99ac106a80 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b3560640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac111000 con 0x7f99ac101420 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b12fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b12fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54514/0 (socket says 172.21.15.77:54514) 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.582+0000 7f99b12fc640 1 -- 172.21.15.77:0/2278193942 learned_addr learned my addr 172.21.15.77:0/2278193942 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b0afb640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac101420 0x7f99ac1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 -- 172.21.15.77:0/2278193942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac101420 msgr2=0x7f99ac1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac101420 0x7f99ac1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 -- 172.21.15.77:0/2278193942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 msgr2=0x7f99ac1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 0x7f99ac1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 0x7f99ac1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 -- 172.21.15.77:0/2278193942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1a4850 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 0x7f99ac1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b12fc640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009230 tx=0x7f99a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99aa7fc640 1 -- 172.21.15.77:0/2278193942 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a403f030 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99aa7fc640 1 -- 172.21.15.77:0/2278193942 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a4003ad0 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac1a4a20 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99aa7fc640 1 -- 172.21.15.77:0/2278193942 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4035070 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.583+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a4f00 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99aa7fc640 1 -- 172.21.15.77:0/2278193942 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f99a4042030 con 0x7f99ac100a80 2024-09-15T22:12:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 Processor -- start 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 -- start start 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc98725090 con 0x55dc987be9d0 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc986cca70 con 0x55dc987bf2f0 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b5939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc98721050 con 0x55dc987c9bb0 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b1afd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b1afd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54520/0 (socket says 172.21.15.77:54520) 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.586+0000 7f99b1afd640 1 -- 172.21.15.77:0/2242836386 learned_addr learned my addr 172.21.15.77:0/2242836386 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b0afb640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99aa7fc640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 0x7f99940a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b12fc640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 -- 172.21.15.77:0/2242836386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 msgr2=0x55dc987cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 -- 172.21.15.77:0/2242836386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 msgr2=0x55dc987c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 -- 172.21.15.77:0/2242836386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc986d9130 con 0x55dc987be9d0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b0afb640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b12fc640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 0x7f99940a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99aa7fc640 1 -- 172.21.15.77:0/2278193942 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f99a40f0770 con 0x7f99ac100a80 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f999c00bbe0 tx=0x7f999c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb57a877828facfe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99937fe640 1 -- 172.21.15.77:0/2242836386 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c011de0 con 0x55dc987be9d0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99937fe640 1 -- 172.21.15.77:0/2242836386 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c0198f0 con 0x55dc987be9d0 2024-09-15T22:12:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99937fe640 1 -- 172.21.15.77:0/2242836386 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c00e810 con 0x55dc987be9d0 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.587+0000 7f99b1afd640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 0x7f99940a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f999c01f000 tx=0x7f999c020000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- 172.21.15.77:0/2242836386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 msgr2=0x55dc987bedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f999c00bbe0 tx=0x7f999c00d710 comp rx=0 tx=0).stop 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- 172.21.15.77:0/2242836386 shutdown_connections 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 --2- 172.21.15.77:0/2242836386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987bedb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- 172.21.15.77:0/2242836386 >> 172.21.15.77:0/2242836386 conn(0x55dc987baf60 msgr2=0x55dc987bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- 172.21.15.77:0/2242836386 shutdown_connections 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- 172.21.15.77:0/2242836386 wait complete. 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 -- start start 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.588+0000 7f99b5939740 1 Processor -- start 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc987cc780 con 0x55dc987be9d0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc986dff20 con 0x55dc987bf2f0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc987cc900 con 0x55dc987c9bb0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b5939740 10 client.0 _ll_register_callbacks cb 0x55dc986a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54524/0 (socket says 172.21.15.77:54524) 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 -- 172.21.15.77:0/300884336 learned_addr learned my addr 172.21.15.77:0/300884336 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b1afd640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b0afb640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 -- 172.21.15.77:0/300884336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 msgr2=0x55dc987c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 -- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 msgr2=0x55dc987c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 -- 172.21.15.77:0/300884336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc984962b0 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b1afd640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f99b12fc640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f99a40bc520 tx=0x7f99a40bc360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997733a1dbe33bca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.589+0000 7f9990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a40bcc90 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f99a4003c80 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c0198f0 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99a40bc590 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc987d7fe0 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c0198f0 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f99a40c4d70 con 0x55dc987be9d0 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 10 client.15645 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 20 client.15645 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc987da670 0x55dc987dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 4 client.15645 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc98732600 con 0x55dc987da670 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.590+0000 7f99b0afb640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc987da670 0x55dc987dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.591+0000 7f9990ff9640 10 client.15645 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.591+0000 7f99b0afb640 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc987da670 0x55dc987dbae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7459cf1916f167d1 server_cookie=6b8fba9da40c409a in_seq=0 out_seq=0 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.591+0000 7f9990ff9640 1 -- 172.21.15.77:0/300884336 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc98732600 con 0x55dc987da670 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.591+0000 7f9990ff9640 10 client.15645 handle_command_reply: tid=0 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 client.15645 shutdown 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 msgr2=0x55dc987c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f99a40bc520 tx=0x7f99a40bc360 comp rx=0 tx=0).stop 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 shutdown_connections 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc987da670 0x55dc987dbae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987c9bb0 0x55dc987c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f9990ff9640 0 client.15645 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987bf2f0 0x55dc987c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 --2- 172.21.15.77:0/300884336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987be9d0 0x55dc987c9420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 >> 172.21.15.77:0/300884336 conn(0x55dc987baf60 msgr2=0x55dc987c0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 shutdown_connections 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 1 -- 172.21.15.77:0/300884336 wait complete. 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.592+0000 7f99b5939740 20 client.15645 trim_cache size 0 max 16384 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.609+0000 7f99b5939740 1 Processor -- start 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 -- start start 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987be9d0 0x55dc987d6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 0x55dc987b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc987d9e20 con 0x55dc987c9bb0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc987da4f0 con 0x55dc987be9d0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b5939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc987dabc0 con 0x55dc987bf2f0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54532/0 (socket says 172.21.15.77:54532) 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 -- 172.21.15.77:0/308469966 learned_addr learned my addr 172.21.15.77:0/308469966 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b12fc640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987be9d0 0x55dc987d6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b0afb640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 0x55dc987b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 -- 172.21.15.77:0/308469966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 msgr2=0x55dc987b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 0x55dc987b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 -- 172.21.15.77:0/308469966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987be9d0 msgr2=0x55dc987d6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987be9d0 0x55dc987d6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 -- 172.21.15.77:0/308469966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc987cc890 con 0x55dc987c9bb0 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b0afb640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 0x55dc987b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f99b1afd640 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f999c005fd0 tx=0x7f999c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ecb6390bac31ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.610+0000 7f9992ffd640 1 -- 172.21.15.77:0/308469966 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c00eb30 con 0x55dc987c9bb0 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f9992ffd640 1 -- 172.21.15.77:0/308469966 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c032070 con 0x55dc987c9bb0 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f9992ffd640 1 -- 172.21.15.77:0/308469966 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c019950 con 0x55dc987c9bb0 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- 172.21.15.77:0/308469966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 msgr2=0x55dc987e41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f999c005fd0 tx=0x7f999c004730 comp rx=0 tx=0).stop 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- 172.21.15.77:0/308469966 shutdown_connections 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987c9bb0 0x55dc987e41b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987bf2f0 0x55dc987b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 --2- 172.21.15.77:0/308469966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987be9d0 0x55dc987d6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- 172.21.15.77:0/308469966 >> 172.21.15.77:0/308469966 conn(0x55dc987baf60 msgr2=0x55dc987d9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- 172.21.15.77:0/308469966 shutdown_connections 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- 172.21.15.77:0/308469966 wait complete. 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.611+0000 7f99b5939740 1 -- start start 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 Processor -- start 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987be9d0 0x55dc98841160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987c9bb0 0x55dc987c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc987db960 con 0x55dc987bf2f0 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc987db290 con 0x55dc987c9bb0 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc987dc030 con 0x55dc987be9d0 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b5939740 10 client.0 _ll_register_callbacks cb 0x7f99a40b8900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54536/0 (socket says 172.21.15.77:54536) 2024-09-15T22:12:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 -- 172.21.15.77:0/1675262690 learned_addr learned my addr 172.21.15.77:0/1675262690 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b1afd640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987c9bb0 0x55dc987c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b12fc640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987be9d0 0x55dc98841160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 -- 172.21.15.77:0/1675262690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987be9d0 msgr2=0x55dc98841160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987be9d0 0x55dc98841160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 -- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987c9bb0 msgr2=0x55dc987c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987c9bb0 0x55dc987c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.612+0000 7f99b0afb640 1 -- 172.21.15.77:0/1675262690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc987c3970 con 0x55dc987bf2f0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b0afb640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009470 tx=0x7f99a0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f97daa837053e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004cb0 con 0x55dc987bf2f0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f99a0005020 con 0x55dc987bf2f0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999c032070 con 0x55dc987bf2f0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f99a002b050 con 0x55dc987bf2f0 2024-09-15T22:12:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc988488e0 con 0x55dc987bf2f0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999c032070 con 0x55dc987bf2f0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f99a000f040 con 0x55dc987bf2f0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 10 client.15651 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 20 client.15651 populate_metadata read hostname 'smithi077' 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc9884d4e0 0x55dc9884c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 4 client.15651 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.613+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "status"}) -- 0x55dc9872cb10 con 0x55dc9884d4e0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b12fc640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc9884d4e0 0x55dc9884c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f9990ff9640 10 client.15651 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b12fc640 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc9884d4e0 0x55dc9884c1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5896937aa8fff406 server_cookie=b7ded64f2e176b47 in_seq=0 out_seq=0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f9990ff9640 1 -- 172.21.15.77:0/1675262690 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dc9872cb10 con 0x55dc9884d4e0 2024-09-15T22:12:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f9990ff9640 10 client.15651 handle_command_reply: tid=0 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 client.15651 shutdown 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 msgr2=0x55dc988416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009470 tx=0x7f99a0009950 comp rx=0 tx=0).stop 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 shutdown_connections 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dc9884d4e0 0x55dc9884c1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc987c9bb0 0x55dc987c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc987bf2f0 0x55dc988416a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f9990ff9640 0 client.15651 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 --2- 172.21.15.77:0/1675262690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc987be9d0 0x55dc98841160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 >> 172.21.15.77:0/1675262690 conn(0x55dc987baf60 msgr2=0x55dc987c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 shutdown_connections 2024-09-15T22:12:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 1 -- 172.21.15.77:0/1675262690 wait complete. 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "0ddb746c-73ae-11ef-bceb-c7b262605968", 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24458, 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 180.417817802, 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 30, 2024-09-15T22:12:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 91, 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 90, 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 191.22165758700001 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.614+0000 7f99b5939740 20 client.15651 trim_cache size 0 max 16384 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 msgr2=0x7f99940a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 0x7f99940a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f999c01f000 tx=0x7f999c020000 comp rx=0 tx=0).stop 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 msgr2=0x7f99ac199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009230 tx=0x7f99a4004270 comp rx=0 tx=0).stop 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 shutdown_connections 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f99940a3550 0x7f99940a5a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99ac106a80 0x7f99ac1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99ac101420 0x7f99ac1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 --2- 172.21.15.77:0/2278193942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99ac100a80 0x7f99ac199290 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 >> 172.21.15.77:0/2278193942 conn(0x7f99ac0fc7a0 msgr2=0x7f99ac0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 shutdown_connections 2024-09-15T22:12:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:12:56.615+0000 7f99b3560640 1 -- 172.21.15.77:0/2278193942 wait complete. 2024-09-15T22:12:57.421 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-15T22:12:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:12:58 smithi077 ceph-mon[30980]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-15T22:12:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:12:58 smithi196 ceph-mon[35109]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-15T22:12:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:12:58 smithi117 ceph-mon[35021]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-15T22:13:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-mon[30980]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:00 smithi196 ceph-mon[35109]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:00 smithi117 ceph-mon[35021]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:01.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:13:00.946+0000 7fd9653ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:13:01.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:13:00.946+0000 7fd9653ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:13:01.333 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:13:00.946+0000 7fd9653ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:13:01.333 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:13:00.946+0000 7fd9653ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:13:01.333 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:13:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:13:00.946+0000 7fd9653ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:13:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:02 smithi077 ceph-mon[30980]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:02 smithi196 ceph-mon[35109]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:02 smithi117 ceph-mon[35021]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-15T22:13:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:04 smithi077 ceph-mon[30980]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:04 smithi077 ceph-mon[30980]: mgrmap e25: x(active, since 38s), standbys: z, y 2024-09-15T22:13:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:04 smithi196 ceph-mon[35109]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:04 smithi196 ceph-mon[35109]: mgrmap e25: x(active, since 38s), standbys: z, y 2024-09-15T22:13:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:04 smithi117 ceph-mon[35021]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:04 smithi117 ceph-mon[35021]: mgrmap e25: x(active, since 38s), standbys: z, y 2024-09-15T22:13:06.372 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:06 smithi077 ceph-mon[30980]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:06 smithi196 ceph-mon[35109]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:06 smithi117 ceph-mon[35021]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:06.635 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 -- 172.21.15.77:0/3173812564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8106a70 msgr2=0x7f78c8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 --2- 172.21.15.77:0/3173812564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8106a70 0x7f78c8106e50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009930 tx=0x7f78bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:13:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 -- 172.21.15.77:0/3173812564 shutdown_connections 2024-09-15T22:13:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 --2- 172.21.15.77:0/3173812564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8101410 0x7f78c810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 --2- 172.21.15.77:0/3173812564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8100a70 0x7f78c8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 --2- 172.21.15.77:0/3173812564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8106a70 0x7f78c8106e50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 -- 172.21.15.77:0/3173812564 >> 172.21.15.77:0/3173812564 conn(0x7f78c80fc790 msgr2=0x7f78c80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 -- 172.21.15.77:0/3173812564 shutdown_connections 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.155+0000 7f78cee6a640 1 -- 172.21.15.77:0/3173812564 wait complete. 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 Processor -- start 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 -- start start 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 0x7f78c81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 0x7f78c819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8110e90 con 0x7f78c8101410 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8110d10 con 0x7f78c8100a70 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cee6a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8111010 con 0x7f78c8106a70 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46932/0 (socket says 172.21.15.77:46932) 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 -- 172.21.15.77:0/1018406724 learned_addr learned my addr 172.21.15.77:0/1018406724 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 -- 172.21.15.77:0/1018406724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 msgr2=0x7f78c819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 0x7f78c81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cd407640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 0x7f78c819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 0x7f78c819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 -- 172.21.15.77:0/1018406724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 msgr2=0x7f78c81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 0x7f78c81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 -- 172.21.15.77:0/1018406724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c819d460 con 0x7f78c8101410 2024-09-15T22:13:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 0x7f78c81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78cd407640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 0x7f78c819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.156+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.157+0000 7f78c5ffb640 1 -- 172.21.15.77:0/1018406724 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.157+0000 7f78c5ffb640 1 -- 172.21.15.77:0/1018406724 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4009c30 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.157+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c819d750 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.157+0000 7f78c5ffb640 1 -- 172.21.15.77:0/1018406724 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b400f6b0 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.157+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81a5030 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.159+0000 7f78c5ffb640 1 -- 172.21.15.77:0/1018406724 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f78b40109d0 con 0x7f78c8101410 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 Processor -- start 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 -- start start 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x56313732d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 0x56313732fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631372842e0 con 0x56313731e360 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56313722bce0 con 0x56313731ec80 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78d1243740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631372802a0 con 0x56313732da90 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78cd407640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78cd407640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46934/0 (socket says 172.21.15.77:46934) 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78cd407640 1 -- 172.21.15.77:0/1487923691 learned_addr learned my addr 172.21.15.77:0/1487923691 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78c5ffb640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 0x7f78b80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x56313732d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78cd407640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 0x7f78b80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78c5ffb640 1 -- 172.21.15.77:0/1018406724 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f78b40d00b0 con 0x7f78c8101410 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.160+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 0x56313732fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 -- 172.21.15.77:0/1487923691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 msgr2=0x56313732fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 0x56313732fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 -- 172.21.15.77:0/1487923691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 msgr2=0x56313732d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x56313732d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 -- 172.21.15.77:0/1487923691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563137210440 con 0x56313731e360 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 0x56313732fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78cd407640 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78c0010a30 tx=0x7f78c0010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0ea0316e3558de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78b2ffd640 1 -- 172.21.15.77:0/1487923691 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0020070 con 0x56313731e360 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78b2ffd640 1 -- 172.21.15.77:0/1487923691 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c000e830 con 0x56313731e360 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.161+0000 7f78b2ffd640 1 -- 172.21.15.77:0/1487923691 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00235c0 con 0x56313731e360 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- 172.21.15.77:0/1487923691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 msgr2=0x56313731e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78c0010a30 tx=0x7f78c0010ef0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78cd407640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 0x7f78b80a5aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c001d000 tx=0x7f78c0019000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- 172.21.15.77:0/1487923691 shutdown_connections 2024-09-15T22:13:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313732da90 0x56313732fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x56313732d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 --2- 172.21.15.77:0/1487923691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313731e740 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- 172.21.15.77:0/1487923691 >> 172.21.15.77:0/1487923691 conn(0x56313731aa30 msgr2=0x56313731ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- 172.21.15.77:0/1487923691 shutdown_connections 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- 172.21.15.77:0/1487923691 wait complete. 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 -- start start 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.162+0000 7f78d1243740 1 Processor -- start 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x56313732a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x56313732ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563137330660 con 0x56313731e360 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56313723f050 con 0x56313732da90 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631373307e0 con 0x56313731ec80 2024-09-15T22:13:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78d1243740 10 client.0 _ll_register_callbacks cb 0x56313727f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46936/0 (socket says 172.21.15.77:46936) 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 -- 172.21.15.77:0/395293211 learned_addr learned my addr 172.21.15.77:0/395293211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78c7fff640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x56313732a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78cd407640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x56313732ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 -- 172.21.15.77:0/395293211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 msgr2=0x56313732a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x56313732a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 -- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 msgr2=0x56313732ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x56313732ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 -- 172.21.15.77:0/395293211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563136ff53c0 con 0x56313731e360 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78ccc06640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009930 tx=0x7f78bc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc60a41267ff4af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.163+0000 7f78a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc040070 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f78bc004240 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78c000e830 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78bc035040 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56313733c000 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c000e830 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f78bc004ac0 con 0x56313731e360 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 20 client.15669 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56313733d4c0 0x56313733f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 4 client.15669 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.164+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563137291850 con 0x56313733d4c0 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.165+0000 7f78c7fff640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56313733d4c0 0x56313733f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.165+0000 7f78c7fff640 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56313733d4c0 0x56313733f8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0577f89bf913df7 server_cookie=6aff8772b02e7498 in_seq=0 out_seq=0 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.165+0000 7f78a7fff640 10 client.15669 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78a7fff640 1 -- 172.21.15.77:0/395293211 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563137291850 con 0x56313733d4c0 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78a7fff640 10 client.15669 handle_command_reply: tid=0 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 client.15669 shutdown 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 msgr2=0x56313732d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009930 tx=0x7f78bc0317e0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 shutdown_connections 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56313733d4c0 0x56313733f8d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x56313732ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x56313732a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 --2- 172.21.15.77:0/395293211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x56313732d420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 >> 172.21.15.77:0/395293211 conn(0x56313731aa30 msgr2=0x563137320790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78a7fff640 0 client.15669 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 shutdown_connections 2024-09-15T22:13:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 1 -- 172.21.15.77:0/395293211 wait complete. 2024-09-15T22:13:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.166+0000 7f78d1243740 20 client.15669 trim_cache size 0 max 16384 2024-09-15T22:13:07.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 Processor -- start 2024-09-15T22:13:07.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 -- start start 2024-09-15T22:13:07.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x563137328fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x563137325c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81acd30 con 0x56313731e360 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81ae870 con 0x56313732da90 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78d1243740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81ad400 con 0x56313731ec80 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78ccc06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78ccc06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46942/0 (socket says 172.21.15.77:46942) 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78ccc06640 1 -- 172.21.15.77:0/1032273363 learned_addr learned my addr 172.21.15.77:0/1032273363 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78c7fff640 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x563137328fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78ccc06640 1 -- 172.21.15.77:0/1032273363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 msgr2=0x563137328fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.186+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x563137328fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78ccc06640 1 -- 172.21.15.77:0/1032273363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 msgr2=0x563137325c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x563137325c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78ccc06640 1 -- 172.21.15.77:0/1032273363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563137329680 con 0x56313731e360 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78ccc06640 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009850 tx=0x7f78bc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f17b550beb813f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78b27fc640 1 -- 172.21.15.77:0/1032273363 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc040070 con 0x56313731e360 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78b27fc640 1 -- 172.21.15.77:0/1032273363 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78bc056070 con 0x56313731e360 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.187+0000 7f78b27fc640 1 -- 172.21.15.77:0/1032273363 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc005310 con 0x56313731e360 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- 172.21.15.77:0/1032273363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 msgr2=0x563137339900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009850 tx=0x7f78bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- 172.21.15.77:0/1032273363 shutdown_connections 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313732da90 0x563137325c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731ec80 0x563137328fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 --2- 172.21.15.77:0/1032273363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313731e360 0x563137339900 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- 172.21.15.77:0/1032273363 >> 172.21.15.77:0/1032273363 conn(0x56313731aa30 msgr2=0x5631373254f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- 172.21.15.77:0/1032273363 shutdown_connections 2024-09-15T22:13:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- 172.21.15.77:0/1032273363 wait complete. 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 -- start start 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.188+0000 7f78d1243740 1 Processor -- start 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731e360 0x563137336030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x563137331360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81ae1a0 con 0x56313732da90 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81adad0 con 0x56313731ec80 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78c81aad10 con 0x56313731e360 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 10 client.0 _ll_register_callbacks cb 0x7f78bc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46950/0 (socket says 172.21.15.77:46950) 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 -- 172.21.15.77:0/3814252070 learned_addr learned my addr 172.21.15.77:0/3814252070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:07.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78ccc06640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731e360 0x563137336030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78c7fff640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x563137331360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 -- 172.21.15.77:0/3814252070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731e360 msgr2=0x563137336030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731e360 0x563137336030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 -- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 msgr2=0x563137331360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x563137331360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 -- 172.21.15.77:0/3814252070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563137331f10 con 0x56313732da90 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78cd407640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78c00103c0 tx=0x7f78c0007280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652b28aa544c3ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0020040 con 0x56313732da90 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f78c0024440 con 0x56313732da90 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.189+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc056070 con 0x56313732da90 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78c003c050 con 0x56313732da90 2024-09-15T22:13:07.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5631373a7ff0 con 0x56313732da90 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78bc056070 con 0x56313732da90 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f78c0037030 con 0x56313732da90 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 20 client.15675 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5631373a85f0 0x5631373aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 4 client.15675 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56313728bd60 con 0x5631373a85f0 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.190+0000 7f78ccc06640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5631373a85f0 0x5631373aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78a7fff640 10 client.15675 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78ccc06640 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5631373a85f0 0x5631373aaa10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e3b604a1f9a641 server_cookie=39fdbb969a573d6f in_seq=0 out_seq=0 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78a7fff640 1 -- 172.21.15.77:0/3814252070 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56313728bd60 con 0x5631373a85f0 2024-09-15T22:13:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78a7fff640 10 client.15675 handle_command_reply: tid=0 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 client.15675 shutdown 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 msgr2=0x5631373318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78c00103c0 tx=0x7f78c0007280 comp rx=0 tx=0).stop 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 shutdown_connections 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5631373a85f0 0x5631373aaa10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78a7fff640 0 client.15675 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56313732da90 0x5631373318a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56313731ec80 0x563137331360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 --2- 172.21.15.77:0/3814252070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56313731e360 0x563137336030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 >> 172.21.15.77:0/3814252070 conn(0x56313731aa30 msgr2=0x5631373254f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 shutdown_connections 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 1 -- 172.21.15.77:0/3814252070 wait complete. 2024-09-15T22:13:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.191+0000 7f78d1243740 20 client.15675 trim_cache size 0 max 16384 2024-09-15T22:13:07.202 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:07.202 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:13:07.202 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:13:07.202 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:07.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 msgr2=0x7f78b80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 0x7f78b80a5aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c001d000 tx=0x7f78c0019000 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 msgr2=0x7f78c81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400de10 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 shutdown_connections 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78b80a35e0 0x7f78b80a5aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78c8106a70 0x7f78c819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78c8101410 0x7f78c81989f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 --2- 172.21.15.77:0/1018406724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8100a70 0x7f78c81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 >> 172.21.15.77:0/1018406724 conn(0x7f78c80fc790 msgr2=0x7f78c806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 shutdown_connections 2024-09-15T22:13:07.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:07.192+0000 7f78cee6a640 1 -- 172.21.15.77:0/1018406724 wait complete. 2024-09-15T22:13:08.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:13:08.040 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:08.292 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:08.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:08 smithi077 ceph-mon[30980]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:08 smithi196 ceph-mon[35109]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:08 smithi117 ceph-mon[35021]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 -- 172.21.15.77:0/4073327283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0100b50 msgr2=0x7f9de0106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 --2- 172.21.15.77:0/4073327283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0100b50 0x7f9de0106690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009930 tx=0x7f9dd002a420 comp rx=0 tx=0).stop 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 -- 172.21.15.77:0/4073327283 shutdown_connections 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 --2- 172.21.15.77:0/4073327283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de0106df0 0x7f9de010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 --2- 172.21.15.77:0/4073327283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0100b50 0x7f9de0106690 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 --2- 172.21.15.77:0/4073327283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de01001a0 0x7f9de0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 -- 172.21.15.77:0/4073327283 >> 172.21.15.77:0/4073327283 conn(0x7f9de0077070 msgr2=0x7f9de00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 -- 172.21.15.77:0/4073327283 shutdown_connections 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.774+0000 7f9de759d640 1 -- 172.21.15.77:0/4073327283 wait complete. 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 Processor -- start 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 -- start start 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de01001a0 0x7f9de0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0100b50 0x7f9de01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0111da0 con 0x7f9de0106df0 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0111c20 con 0x7f9de01001a0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de759d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0111f20 con 0x7f9de0100b50 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46974/0 (socket says 172.21.15.77:46974) 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 -- 172.21.15.77:0/1215097035 learned_addr learned my addr 172.21.15.77:0/1215097035 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de659b640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de01001a0 0x7f9de0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de5d9a640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0100b50 0x7f9de01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 -- 172.21.15.77:0/1215097035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0100b50 msgr2=0x7f9de01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0100b50 0x7f9de01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 -- 172.21.15.77:0/1215097035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de01001a0 msgr2=0x7f9de0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.775+0000 7f9de6d9c640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de01001a0 0x7f9de0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9de6d9c640 1 -- 172.21.15.77:0/1215097035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de019d210 con 0x7f9de0106df0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9de6d9c640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800eaa0 tx=0x7f9dd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800cdb0 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd8005020 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de019d500 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80116b0 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.776+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de01a4de0 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.777+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de0103260 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.779+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f9dd8005560 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.780+0000 7f9dd77fe640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 0x7f9dcc0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.780+0000 7f9de659b640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 0x7f9dcc0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.780+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9dd80d0a50 con 0x7f9de0106df0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.780+0000 7f9de659b640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 0x7f9dcc0a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005e10 tx=0x7f9ddc005da0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.783+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dd8016030 con 0x7f9de0106df0 2024-09-15T22:13:08.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.997+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9de0100580 con 0x7f9de0106df0 2024-09-15T22:13:08.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:08.998+0000 7f9dd77fe640 1 -- 172.21.15.77:0/1215097035 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f9de0100580 con 0x7f9de0106df0 2024-09-15T22:13:09.000 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:13:09.000 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:13:09.000 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 msgr2=0x7f9dcc0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 0x7f9dcc0a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005e10 tx=0x7f9ddc005da0 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 msgr2=0x7f9de019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800eaa0 tx=0x7f9dd800ef60 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 shutdown_connections 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9dcc0a3560 0x7f9dcc0a5a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de0106df0 0x7f9de019cb30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9de0100b50 0x7f9de01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 --2- 172.21.15.77:0/1215097035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de01001a0 0x7f9de0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 >> 172.21.15.77:0/1215097035 conn(0x7f9de0077070 msgr2=0x7f9de00fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 shutdown_connections 2024-09-15T22:13:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:09.000+0000 7f9de759d640 1 -- 172.21.15.77:0/1215097035 wait complete. 2024-09-15T22:13:09.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:09 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1215097035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:09.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:09 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1215097035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:09.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:09 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1215097035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:09.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:13:10.053 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:10.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:10 smithi077 ceph-mon[30980]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- 172.21.15.77:0/17588726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694108290 msgr2=0x7f569410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 --2- 172.21.15.77:0/17588726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694108290 0x7f569410ec30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f568c009cd0 tx=0x7f568c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- 172.21.15.77:0/17588726 shutdown_connections 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 --2- 172.21.15.77:0/17588726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694108290 0x7f569410ec30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 --2- 172.21.15.77:0/17588726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694103740 0x7f5694107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 --2- 172.21.15.77:0/17588726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694102d90 0x7f5694103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- 172.21.15.77:0/17588726 >> 172.21.15.77:0/17588726 conn(0x7f56940fc7d0 msgr2=0x7f56940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- 172.21.15.77:0/17588726 shutdown_connections 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- 172.21.15.77:0/17588726 wait complete. 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 Processor -- start 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.553+0000 7f569973a640 1 -- start start 2024-09-15T22:13:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 0x7f56941009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694108290 0x7f5694101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5694112090 con 0x7f5694103740 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5694111f10 con 0x7f5694108290 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f569973a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5694112210 con 0x7f5694102d90 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48868/0 (socket says 172.21.15.77:48868) 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 -- 172.21.15.77:0/2524973187 learned_addr learned my addr 172.21.15.77:0/2524973187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f56937fe640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694108290 0x7f5694101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f5692ffd640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 0x7f56941009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 -- 172.21.15.77:0/2524973187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 msgr2=0x7f56941009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 0x7f56941009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 -- 172.21.15.77:0/2524973187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694108290 msgr2=0x7f5694101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694108290 0x7f5694101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 -- 172.21.15.77:0/2524973187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56941a6a10 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f5692ffd640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 0x7f56941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f568bfff640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d740 tx=0x7f567c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.554+0000 7f5690ff9640 1 -- 172.21.15.77:0/2524973187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c017070 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.555+0000 7f5690ff9640 1 -- 172.21.15.77:0/2524973187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c00de50 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.555+0000 7f5690ff9640 1 -- 172.21.15.77:0/2524973187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c004e00 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.555+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56941a6d00 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.555+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a71e0 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.557+0000 7f5690ff9640 1 -- 172.21.15.77:0/2524973187 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f567c004fa0 con 0x7f5694103740 2024-09-15T22:13:10.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.557+0000 7f569bb13740 1 Processor -- start 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.557+0000 7f569bb13740 1 -- start start 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.557+0000 7f5690ff9640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 0x7f56800a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f56937fe640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 0x7f56800a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5690ff9640 1 -- 172.21.15.77:0/2524973187 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f567c0cfc10 con 0x7f5694103740 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a36370 0x55e592a38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5929912f0 con 0x55e592a2cb40 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e592938cf0 con 0x55e592a2b5d0 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f569bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e59298d2b0 con 0x55e592a36370 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48878/0 (socket says 172.21.15.77:48878) 2024-09-15T22:13:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 -- 172.21.15.77:0/3680485841 learned_addr learned my addr 172.21.15.77:0/3680485841 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f56937fe640 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a36370 0x55e592a38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f568bfff640 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 -- 172.21.15.77:0/3680485841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a36370 msgr2=0x55e592a38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a36370 0x55e592a38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f56937fe640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 0x7f56800a5aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f568c004ab0 tx=0x7f568c004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 -- 172.21.15.77:0/3680485841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 msgr2=0x55e592a2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 -- 172.21.15.77:0/3680485841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59291d3a0 con 0x55e592a2cb40 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.558+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f568400bd40 tx=0x7f568400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d395f3ece52e3308 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f56897fa640 1 -- 172.21.15.77:0/3680485841 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684015070 con 0x55e592a2cb40 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f56897fa640 1 -- 172.21.15.77:0/3680485841 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568400be30 con 0x55e592a2cb40 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f56897fa640 1 -- 172.21.15.77:0/3680485841 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684011a40 con 0x55e592a2cb40 2024-09-15T22:13:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 -- 172.21.15.77:0/3680485841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 msgr2=0x55e592a2b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f568400bd40 tx=0x7f568400f730 comp rx=0 tx=0).stop 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 -- 172.21.15.77:0/3680485841 shutdown_connections 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a36370 0x55e592a38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 --2- 172.21.15.77:0/3680485841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2cb40 0x55e592a2b090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.559+0000 7f569bb13740 1 -- 172.21.15.77:0/3680485841 >> 172.21.15.77:0/3680485841 conn(0x55e5929f4310 msgr2=0x55e592a27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- 172.21.15.77:0/3680485841 shutdown_connections 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- 172.21.15.77:0/3680485841 wait complete. 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- start start 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 Processor -- start 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a35d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a32fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e592a39070 con 0x55e592a36370 2024-09-15T22:13:10.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e59294c060 con 0x55e592a2b5d0 2024-09-15T22:13:10.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e592a391f0 con 0x55e592a2cb40 2024-09-15T22:13:10.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f569bb13740 10 client.0 _ll_register_callbacks cb 0x55e59269f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:10.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f56937fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f56937fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48894/0 (socket says 172.21.15.77:48894) 2024-09-15T22:13:10.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f56937fe640 1 -- 172.21.15.77:0/3127266340 learned_addr learned my addr 172.21.15.77:0/3127266340 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:10.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a35d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.560+0000 7f568bfff640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a32fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 -- 172.21.15.77:0/3127266340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 msgr2=0x55e592a32fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a32fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 -- 172.21.15.77:0/3127266340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 msgr2=0x55e592a35d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a35d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 -- 172.21.15.77:0/3127266340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5927023c0 con 0x55e592a36370 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f56937fe640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f568c02ae00 tx=0x7f568c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f2f0333324850d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f567a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c004240 con 0x55e592a36370 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f568c0043e0 con 0x55e592a36370 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568400be30 con 0x55e592a36370 2024-09-15T22:13:10.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f568c0045e0 con 0x55e592a36370 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f569bb13740 10 client.15693 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e592a44ac0 con 0x55e592a36370 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.561+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568400be30 con 0x55e592a36370 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f568c04e070 con 0x55e592a36370 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 10 client.15693 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 10 client.15693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 20 client.15693 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592a461f0 0x55e592a48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 4 client.15693 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e59299e860 con 0x55e592a461f0 2024-09-15T22:13:10.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592a461f0 0x55e592a48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f567a7fc640 10 client.15693 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.562+0000 7f5692ffd640 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592a461f0 0x55e592a48670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ce2cb17e77d62d0 server_cookie=ce82fa45b7f14ce8 in_seq=0 out_seq=0 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f567a7fc640 1 -- 172.21.15.77:0/3127266340 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e59299e860 con 0x55e592a461f0 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f567a7fc640 10 client.15693 handle_command_reply: tid=0 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 client.15693 shutdown 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 msgr2=0x55e592a33570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f568c02ae00 tx=0x7f568c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 shutdown_connections 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592a461f0 0x55e592a48670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a33570 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a32fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 --2- 172.21.15.77:0/3127266340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a35d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 >> 172.21.15.77:0/3127266340 conn(0x55e5929f4310 msgr2=0x55e592a2d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f567a7fc640 0 client.15693 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:10.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 shutdown_connections 2024-09-15T22:13:10.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 1 -- 172.21.15.77:0/3127266340 wait complete. 2024-09-15T22:13:10.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.563+0000 7f569bb13740 20 client.15693 trim_cache size 0 max 16384 2024-09-15T22:13:10.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.576+0000 7f569bb13740 1 Processor -- start 2024-09-15T22:13:10.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.576+0000 7f569bb13740 1 -- start start 2024-09-15T22:13:10.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.576+0000 7f569bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a42310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f569bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a2e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f569bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f569bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b10a0 con 0x55e592a36370 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f569bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b2be0 con 0x55e592a2b5d0 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f569bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b1770 con 0x55e592a2cb40 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48896/0 (socket says 172.21.15.77:48896) 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 -- 172.21.15.77:0/2902812941 learned_addr learned my addr 172.21.15.77:0/2902812941 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f568bfff640 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a2e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f5692ffd640 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a42310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 -- 172.21.15.77:0/2902812941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 msgr2=0x55e592a2e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a2e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 -- 172.21.15.77:0/2902812941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 msgr2=0x55e592a42310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a42310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 -- 172.21.15.77:0/2902812941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e592a28960 con 0x55e592a36370 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f56937fe640 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f568c000f80 tx=0x7f568c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99b73d8b33a534e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f5688ff9640 1 -- 172.21.15.77:0/2902812941 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c031c60 con 0x55e592a36370 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f5688ff9640 1 -- 172.21.15.77:0/2902812941 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c004780 con 0x55e592a36370 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.577+0000 7f5688ff9640 1 -- 172.21.15.77:0/2902812941 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c030800 con 0x55e592a36370 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 -- 172.21.15.77:0/2902812941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 msgr2=0x55e592a2e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f568c000f80 tx=0x7f568c0095c0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 -- 172.21.15.77:0/2902812941 shutdown_connections 2024-09-15T22:13:10.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a36370 0x55e592a2e660 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a2e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 --2- 172.21.15.77:0/2902812941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a2b5d0 0x55e592a42310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 -- 172.21.15.77:0/2902812941 >> 172.21.15.77:0/2902812941 conn(0x55e5929f4310 msgr2=0x55e592a32f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 -- 172.21.15.77:0/2902812941 shutdown_connections 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.578+0000 7f569bb13740 1 -- 172.21.15.77:0/2902812941 wait complete. 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 -- start start 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 Processor -- start 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a39f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 0x55e592a3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b2510 con 0x55e592a2b5d0 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b1e40 con 0x55e592a36370 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56941af080 con 0x55e592a2cb40 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f569bb13740 10 client.0 _ll_register_callbacks cb 0x7f568c009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f5692ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f5692ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48902/0 (socket says 172.21.15.77:48902) 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f5692ffd640 1 -- 172.21.15.77:0/96650374 learned_addr learned my addr 172.21.15.77:0/96650374 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f56937fe640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 0x55e592a3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f568bfff640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f5692ffd640 1 -- 172.21.15.77:0/96650374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 msgr2=0x55e592a39f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.579+0000 7f5692ffd640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a39f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f5692ffd640 1 -- 172.21.15.77:0/96650374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 msgr2=0x55e592a3a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f5692ffd640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 0x55e592a3a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f5692ffd640 1 -- 172.21.15.77:0/96650374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e592a3acb0 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f56937fe640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 0x55e592a3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f5692ffd640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5684011620 tx=0x7f5684005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376bc71bac0b9e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f567a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684015040 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f5684004030 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568c004780 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5684036050 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f569bb13740 10 client.15699 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e592ab0f40 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.580+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684005520 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5684011b10 con 0x55e592a2b5d0 2024-09-15T22:13:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 10 client.15699 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 10 client.15699 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 20 client.15699 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592ab1390 0x55e592ab3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 4 client.15699 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f568c004780 con 0x55e592ab1390 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f568bfff640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592ab1390 0x55e592ab3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f568bfff640 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592ab1390 0x55e592ab3780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64a9d396005e9adc server_cookie=447c48fd6547dcfa in_seq=0 out_seq=0 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f567a7fc640 10 client.15699 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f567a7fc640 1 -- 172.21.15.77:0/96650374 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f568c004780 con 0x55e592ab1390 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f567a7fc640 10 client.15699 handle_command_reply: tid=0 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 client.15699 shutdown 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 msgr2=0x55e592a3ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5684011620 tx=0x7f5684005870 comp rx=0 tx=0).stop 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 shutdown_connections 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e592ab1390 0x55e592ab3780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f567a7fc640 0 client.15699 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e592a36370 0x55e592a3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e592a2cb40 0x55e592a39f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 --2- 172.21.15.77:0/96650374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e592a2b5d0 0x55e592a3ebc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.581+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 >> 172.21.15.77:0/96650374 conn(0x55e5929f4310 msgr2=0x55e592a264e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 shutdown_connections 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569bb13740 1 -- 172.21.15.77:0/96650374 wait complete. 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569bb13740 20 client.15699 trim_cache size 0 max 16384 2024-09-15T22:13:10.593 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:10.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 msgr2=0x7f56800a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 0x7f56800a5aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f568c004ab0 tx=0x7f568c004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 msgr2=0x7f5694100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d740 tx=0x7f567c00dc00 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 shutdown_connections 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56800a35e0 0x7f56800a5aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5694108290 0x7f5694101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5694103740 0x7f5694100f30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 --2- 172.21.15.77:0/2524973187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5694102d90 0x7f56941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.582+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 >> 172.21.15.77:0/2524973187 conn(0x7f56940fc7d0 msgr2=0x7f5694106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:10.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.583+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 shutdown_connections 2024-09-15T22:13:10.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:10.583+0000 7f569973a640 1 -- 172.21.15.77:0/2524973187 wait complete. 2024-09-15T22:13:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:10 smithi196 ceph-mon[35109]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:10 smithi117 ceph-mon[35021]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:11.354 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:13:11.616 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:12.422 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-15T22:13:12.422 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:12 smithi077 ceph-mon[30980]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-15T22:13:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:12 smithi196 ceph-mon[35109]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-15T22:13:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:12 smithi117 ceph-mon[35021]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-15T22:13:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:14 smithi077 ceph-mon[30980]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:14 smithi196 ceph-mon[35109]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:14 smithi117 ceph-mon[35021]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 -- 172.21.15.77:0/4109930046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 msgr2=0x7f7f8010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 --2- 172.21.15.77:0/4109930046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8010ec40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009930 tx=0x7f7f7802a420 comp rx=0 tx=0).stop 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 -- 172.21.15.77:0/4109930046 shutdown_connections 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 --2- 172.21.15.77:0/4109930046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8010ec40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 --2- 172.21.15.77:0/4109930046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f801036e0 0x7f7f80107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 --2- 172.21.15.77:0/4109930046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f80102d30 0x7f7f80103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 -- 172.21.15.77:0/4109930046 >> 172.21.15.77:0/4109930046 conn(0x7f7f800fc7d0 msgr2=0x7f7f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 -- 172.21.15.77:0/4109930046 shutdown_connections 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.751+0000 7f7f87b43640 1 -- 172.21.15.77:0/4109930046 wait complete. 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 Processor -- start 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 -- start start 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f80102d30 0x7f7f80198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 0x7f7f80198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801120a0 con 0x7f7f801082a0 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80111f20 con 0x7f7f80102d30 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f87b43640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80112220 con 0x7f7f801036e0 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f860e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f860e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48920/0 (socket says 172.21.15.77:48920) 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f860e0640 1 -- 172.21.15.77:0/633978393 learned_addr learned my addr 172.21.15.77:0/633978393 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f850de640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 0x7f7f80198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.752+0000 7f7f858df640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f80102d30 0x7f7f80198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 -- 172.21.15.77:0/633978393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 msgr2=0x7f7f80198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 0x7f7f80198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 -- 172.21.15.77:0/633978393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f80102d30 msgr2=0x7f7f80198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f80102d30 0x7f7f80198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 -- 172.21.15.77:0/633978393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8019d520 con 0x7f7f801082a0 2024-09-15T22:13:15.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f850de640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 0x7f7f80198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:15.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f860e0640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f78005ab0 tx=0x7f7f780040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f76ffd640 1 -- 172.21.15.77:0/633978393 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7803f030 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8019d7b0 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f76ffd640 1 -- 172.21.15.77:0/633978393 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f780044c0 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f76ffd640 1 -- 172.21.15.77:0/633978393 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78035070 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.753+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a50f0 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 Processor -- start 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f76ffd640 1 -- 172.21.15.77:0/633978393 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f7f78042030 con 0x7f7f801082a0 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 -- start start 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a29b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0c797e2f0 con 0x55f0c7a19b40 2024-09-15T22:13:15.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0c7925cf0 con 0x55f0c7a27780 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0c797a2b0 con 0x55f0c7a185d0 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f850de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f850de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48934/0 (socket says 172.21.15.77:48934) 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f858df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.756+0000 7f7f850de640 1 -- 172.21.15.77:0/2871393375 learned_addr learned my addr 172.21.15.77:0/2871393375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f76ffd640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 0x7f7f6c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a29b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 -- 172.21.15.77:0/2871393375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 msgr2=0x55f0c7a18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f860e0640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 0x7f7f6c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 -- 172.21.15.77:0/2871393375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 msgr2=0x55f0c7a29b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a29b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 -- 172.21.15.77:0/2871393375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c790a3a0 con 0x55f0c7a19b40 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f858df640 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f76ffd640 1 -- 172.21.15.77:0/633978393 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7f780f0680 con 0x7f7f801082a0 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f850de640 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55f0c7a19070 tx=0x7f7f7000bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48473202d32d0cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f6bfff640 1 -- 172.21.15.77:0/2871393375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f700122d0 con 0x55f0c7a19b40 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f860e0640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 0x7f7f6c0a5a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ed8b0 tx=0x7f7f7808c0b0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f6bfff640 1 -- 172.21.15.77:0/2871393375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7000e820 con 0x55f0c7a19b40 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.757+0000 7f7f6bfff640 1 -- 172.21.15.77:0/2871393375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70005470 con 0x55f0c7a19b40 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 -- 172.21.15.77:0/2871393375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 msgr2=0x55f0c7a18090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55f0c7a19070 tx=0x7f7f7000bb90 comp rx=0 tx=0).stop 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 -- 172.21.15.77:0/2871393375 shutdown_connections 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a29b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/2871393375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a18090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 -- 172.21.15.77:0/2871393375 >> 172.21.15.77:0/2871393375 conn(0x55f0c79e1310 msgr2=0x55f0c7a14c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 -- 172.21.15.77:0/2871393375 shutdown_connections 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.758+0000 7f7f89f1c740 1 -- 172.21.15.77:0/2871393375 wait complete. 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 -- start start 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 Processor -- start 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 0x55f0c7a27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a24980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0c7a2a420 con 0x55f0c7a185d0 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0c7939060 con 0x55f0c7a27780 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0c7a2a5a0 con 0x55f0c7a19b40 2024-09-15T22:13:15.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f89f1c740 10 client.0 _ll_register_callbacks cb 0x55f0c768c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f850de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 0x55f0c7a27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f850de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 0x55f0c7a27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48936/0 (socket says 172.21.15.77:48936) 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f858df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.759+0000 7f7f850de640 1 -- 172.21.15.77:0/4195969005 learned_addr learned my addr 172.21.15.77:0/4195969005 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f850de640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a24980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 -- 172.21.15.77:0/4195969005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 msgr2=0x55f0c7a24980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a24980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 -- 172.21.15.77:0/4195969005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 msgr2=0x55f0c7a27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 0x55f0c7a27130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 -- 172.21.15.77:0/4195969005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c76ef3c0 con 0x55f0c7a19b40 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f850de640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f858df640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a243e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c005fd0 tx=0x7f7f7c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22b87ecc7275c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01f060 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7000e820 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7f7c003970 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c026050 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f89f1c740 10 client.25126 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f0c7a35ed0 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.760+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7000e820 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7f7c019370 con 0x55f0c7a19b40 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 10 client.25126 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 10 client.25126 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 20 client.25126 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7a373e0 0x55f0c7a39800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 4 client.25126 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:15.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c798b860 con 0x55f0c7a373e0 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f850de640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7a373e0 0x55f0c7a39800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f697fa640 10 client.25126 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.761+0000 7f7f850de640 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7a373e0 0x55f0c7a39800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12255a54dc05efd6 server_cookie=17985679c649c71f in_seq=0 out_seq=0 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f697fa640 1 -- 172.21.15.77:0/4195969005 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0c798b860 con 0x55f0c7a373e0 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f697fa640 10 client.25126 handle_command_reply: tid=0 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 client.25126 shutdown 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 msgr2=0x55f0c7a243e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a243e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c005fd0 tx=0x7f7f7c00f790 comp rx=0 tx=0).stop 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 shutdown_connections 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7a373e0 0x55f0c7a39800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a243e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/4195969005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a185d0 0x55f0c7a27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f697fa640 0 client.25126 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 >> 172.21.15.77:0/4195969005 conn(0x55f0c79e1310 msgr2=0x55f0c7a1a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 shutdown_connections 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 1 -- 172.21.15.77:0/4195969005 wait complete. 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.762+0000 7f7f89f1c740 20 client.25126 trim_cache size 0 max 16384 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 Processor -- start 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 -- start start 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a3f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801acc50 con 0x55f0c7a19b40 2024-09-15T22:13:15.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ae790 con 0x55f0c7a27780 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ad320 con 0x55f0c7a185d0 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f858df640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f850de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f850de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a243b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44044/0 (socket says 172.21.15.77:44044) 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f850de640 1 -- 172.21.15.77:0/1308500772 learned_addr learned my addr 172.21.15.77:0/1308500772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f858df640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48944/0 (socket says 172.21.15.77:48944) 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.788+0000 7f7f850de640 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a3f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 -- 172.21.15.77:0/1308500772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 msgr2=0x55f0c7a243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 -- 172.21.15.77:0/1308500772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 msgr2=0x55f0c7a3f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a3f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 -- 172.21.15.77:0/1308500772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c7a274a0 con 0x55f0c7a19b40 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f858df640 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0197c0 tx=0x7f7f7c0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7c3426fe97bb41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f6b7fe640 1 -- 172.21.15.77:0/1308500772 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0199b0 con 0x55f0c7a19b40 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f6b7fe640 1 -- 172.21.15.77:0/1308500772 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c003e50 con 0x55f0c7a19b40 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.789+0000 7f7f6b7fe640 1 -- 172.21.15.77:0/1308500772 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c005770 con 0x55f0c7a19b40 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1308500772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 msgr2=0x55f0c7a336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0197c0 tx=0x7f7f7c0195a0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1308500772 shutdown_connections 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a27780 0x55f0c7a3f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a19b40 0x55f0c7a336c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1308500772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a185d0 0x55f0c7a243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1308500772 >> 172.21.15.77:0/1308500772 conn(0x55f0c79e1310 msgr2=0x55f0c7a23ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1308500772 shutdown_connections 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1308500772 wait complete. 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 -- start start 2024-09-15T22:13:15.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.790+0000 7f7f89f1c740 1 Processor -- start 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a185d0 0x55f0c7a2f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 0x55f0c7a2ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ae0c0 con 0x55f0c7a27780 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801ad9f0 con 0x55f0c7a185d0 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801aac30 con 0x55f0c7a19b40 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f89f1c740 10 client.0 _ll_register_callbacks cb 0x7f7f7c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f850de640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 0x55f0c7a2ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44060/0 (socket says 172.21.15.77:44060) 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 -- 172.21.15.77:0/1950557405 learned_addr learned my addr 172.21.15.77:0/1950557405 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f850de640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a185d0 0x55f0c7a2f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 -- 172.21.15.77:0/1950557405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a185d0 msgr2=0x55f0c7a2f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a185d0 0x55f0c7a2f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 -- 172.21.15.77:0/1950557405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 msgr2=0x55f0c7a2ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 0x55f0c7a2ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f858df640 1 -- 172.21.15.77:0/1950557405 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c7a2b570 con 0x55f0c7a19b40 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.791+0000 7f7f850de640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 0x55f0c7a2ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f858df640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009540 tx=0x7f7f7c009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789077ebc7223222 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c038020 con 0x55f0c7a19b40 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7f7c0050b0 con 0x55f0c7a19b40 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c003e50 con 0x55f0c7a19b40 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c043050 con 0x55f0c7a19b40 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 10 client.25132 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:15.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f0c7aa1b20 con 0x55f0c7a19b40 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c003e50 con 0x55f0c7a19b40 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7f7c034460 con 0x55f0c7a19b40 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 10 client.25132 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 10 client.25132 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 20 client.25132 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.792+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7aa21e0 0x55f0c7aa4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f89f1c740 4 client.25132 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0c7985d70 con 0x55f0c7aa21e0 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f850de640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7aa21e0 0x55f0c7aa4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f697fa640 10 client.25132 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f850de640 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7aa21e0 0x55f0c7aa4600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63a787058886868e server_cookie=eda13853b070c885 in_seq=0 out_seq=0 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f697fa640 1 -- 172.21.15.77:0/1950557405 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0c7985d70 con 0x55f0c7aa21e0 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f697fa640 10 client.25132 handle_command_reply: tid=0 2024-09-15T22:13:15.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f89f1c740 1 client.25132 shutdown 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 msgr2=0x55f0c7a2fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.793+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009540 tx=0x7f7f7c009330 comp rx=0 tx=0).stop 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 shutdown_connections 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f0c7aa21e0 0x55f0c7aa4600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0c7a27780 0x55f0c7a2ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0c7a19b40 0x55f0c7a2fd50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f697fa640 0 client.25132 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 --2- 172.21.15.77:0/1950557405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0c7a185d0 0x55f0c7a2f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 >> 172.21.15.77:0/1950557405 conn(0x55f0c79e1310 msgr2=0x55f0c7a25c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 shutdown_connections 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 1 -- 172.21.15.77:0/1950557405 wait complete. 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f89f1c740 20 client.25132 trim_cache size 0 max 16384 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 msgr2=0x7f7f6c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 0x7f7f6c0a5a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ed8b0 tx=0x7f7f7808c0b0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 msgr2=0x7f7f8019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f78005ab0 tx=0x7f7f780040c0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 shutdown_connections 2024-09-15T22:13:15.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f6c0a3560 0x7f7f6c0a5a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f801082a0 0x7f7f8019ce40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f801036e0 0x7f7f80198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 --2- 172.21.15.77:0/633978393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f80102d30 0x7f7f80198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 >> 172.21.15.77:0/633978393 conn(0x7f7f800fc7d0 msgr2=0x7f7f80105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.794+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 shutdown_connections 2024-09-15T22:13:15.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:14.795+0000 7f7f87b43640 1 -- 172.21.15.77:0/633978393 wait complete. 2024-09-15T22:13:15.498 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:16.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 -- 172.21.15.77:0/848621372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 msgr2=0x7f3cbc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:16.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/848621372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc10da40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009ca0 tx=0x7f3cb402a950 comp rx=0 tx=0).stop 2024-09-15T22:13:16.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 -- 172.21.15.77:0/848621372 shutdown_connections 2024-09-15T22:13:16.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/848621372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc10da40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/848621372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc102570 0x7f3cbc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/848621372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc100e20 0x7f3cbc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 -- 172.21.15.77:0/848621372 >> 172.21.15.77:0/848621372 conn(0x7f3cbc0fc7d0 msgr2=0x7f3cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 -- 172.21.15.77:0/848621372 shutdown_connections 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.016+0000 7f3cc31a8640 1 -- 172.21.15.77:0/848621372 wait complete. 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 Processor -- start 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 -- start start 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 0x7f3cbc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc102570 0x7f3cbc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc110f80 con 0x7f3cbc102ef0 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc110e00 con 0x7f3cbc102570 2024-09-15T22:13:16.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc31a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc111100 con 0x7f3cbc100e20 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc1745640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc1745640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48986/0 (socket says 172.21.15.77:48986) 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc1745640 1 -- 172.21.15.77:0/834902912 learned_addr learned my addr 172.21.15.77:0/834902912 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cbbfff640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc102570 0x7f3cbc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.017+0000 7f3cc0f44640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 0x7f3cbc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 -- 172.21.15.77:0/834902912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 msgr2=0x7f3cbc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 0x7f3cbc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 -- 172.21.15.77:0/834902912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc102570 msgr2=0x7f3cbc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc102570 0x7f3cbc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 -- 172.21.15.77:0/834902912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc19d3b0 con 0x7f3cbc102ef0 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc0f44640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 0x7f3cbc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc1745640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3cb402ae70 tx=0x7f3cb402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:16.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4034040 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb4030920 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc19d640 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40047f0 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.018+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc1a4f80 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.020+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cbc1021b0 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.021+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f3cb4037020 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.021+0000 7f3cb9ffb640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 0x7f3cac0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.022+0000 7f3cc0f44640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 0x7f3cac0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.022+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f3cb40f11a0 con 0x7f3cbc102ef0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.022+0000 7f3cc0f44640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 0x7f3cac0a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005d00 tx=0x7f3cb0005c70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:16.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.026+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cb40bd710 con 0x7f3cbc102ef0 2024-09-15T22:13:16.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.272+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cbc10a0a0 con 0x7f3cbc102ef0 2024-09-15T22:13:16.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.273+0000 7f3cb9ffb640 1 -- 172.21.15.77:0/834902912 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f3cb40bd010 con 0x7f3cbc102ef0 2024-09-15T22:13:16.275 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:13:16.275 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:13:16.275 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 msgr2=0x7f3cac0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 0x7f3cac0a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005d00 tx=0x7f3cb0005c70 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 msgr2=0x7f3cbc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3cb402ae70 tx=0x7f3cb402f5d0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 shutdown_connections 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3cac0a3560 0x7f3cac0a5a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cbc102ef0 0x7f3cbc19ccd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc102570 0x7f3cbc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 --2- 172.21.15.77:0/834902912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cbc100e20 0x7f3cbc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 >> 172.21.15.77:0/834902912 conn(0x7f3cbc0fc7d0 msgr2=0x7f3cbc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 shutdown_connections 2024-09-15T22:13:16.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:16.276+0000 7f3cc31a8640 1 -- 172.21.15.77:0/834902912 wait complete. 2024-09-15T22:13:16.536 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:16 smithi077 ceph-mon[30980]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:16.536 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:16 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/834902912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:16.566 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:13:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:16 smithi196 ceph-mon[35109]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:16 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/834902912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:16 smithi117 ceph-mon[35021]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:13:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:16 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/834902912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:17.306 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-15T22:13:17.306 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-15T22:13:17.386 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 -- 172.21.15.77:0/3753842802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c102570 msgr2=0x7fdb6c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 --2- 172.21.15.77:0/3753842802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c102570 0x7fdb6c1029b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009900 tx=0x7fdb6002a480 comp rx=0 tx=0).stop 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 -- 172.21.15.77:0/3753842802 shutdown_connections 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 --2- 172.21.15.77:0/3753842802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 --2- 172.21.15.77:0/3753842802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c102570 0x7fdb6c1029b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 --2- 172.21.15.77:0/3753842802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c100e20 0x7fdb6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.921+0000 7fdb72952640 1 -- 172.21.15.77:0/3753842802 >> 172.21.15.77:0/3753842802 conn(0x7fdb6c0fc7d0 msgr2=0x7fdb6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.922+0000 7fdb72952640 1 -- 172.21.15.77:0/3753842802 shutdown_connections 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.922+0000 7fdb72952640 1 -- 172.21.15.77:0/3753842802 wait complete. 2024-09-15T22:13:17.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.922+0000 7fdb72952640 1 Processor -- start 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.922+0000 7fdb72952640 1 -- start start 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 0x7fdb6c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c19a010 0x7fdb6c1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c110f50 con 0x7fdb6c100e20 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c110dd0 con 0x7fdb6c102ef0 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb72952640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1110d0 con 0x7fdb6c19a010 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39370/0 (socket says 172.21.15.77:39370) 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3404790371 learned_addr learned my addr 172.21.15.77:0/3404790371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb70eef640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c19a010 0x7fdb6c1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 0x7fdb6c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3404790371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c19a010 msgr2=0x7fdb6c1a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c19a010 0x7fdb6c1a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 msgr2=0x7fdb6c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 0x7fdb6c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.923+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3404790371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c1a4de0 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 0x7fdb6c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004770 tx=0x7fdb60003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb697fa640 1 -- 172.21.15.77:0/3404790371 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6003f020 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb697fa640 1 -- 172.21.15.77:0/3404790371 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60031c30 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c1a5070 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a5500 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.924+0000 7fdb697fa640 1 -- 172.21.15.77:0/3404790371 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60035070 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.927+0000 7fdb697fa640 1 -- 172.21.15.77:0/3404790371 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fdb60042020 con 0x7fdb6c102ef0 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.927+0000 7fdb74d2b740 1 Processor -- start 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.927+0000 7fdb74d2b740 1 -- start start 2024-09-15T22:13:17.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb697fa640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 0x7fdb540a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da8015d0 0x55e2da801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da812b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e2da7672f0 con 0x55e2da802b40 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e2da70ecf0 con 0x55e2da8015d0 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e2da7632b0 con 0x55e2da810780 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb697fa640 1 -- 172.21.15.77:0/3404790371 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdb600f0a90 con 0x7fdb6c102ef0 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49022/0 (socket says 172.21.15.77:49022) 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 -- 172.21.15.77:0/1238347836 learned_addr learned my addr 172.21.15.77:0/1238347836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da8015d0 0x55e2da801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb70eef640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 0x7fdb540a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 -- 172.21.15.77:0/1238347836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 msgr2=0x55e2da812b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da812b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 -- 172.21.15.77:0/1238347836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da8015d0 msgr2=0x55e2da801a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da8015d0 0x55e2da801a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.928+0000 7fdb6bfff640 1 -- 172.21.15.77:0/1238347836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2da6f33a0 con 0x55e2da802b40 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.929+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005fd0 tx=0x7fdb5c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46dcc71d75e3538 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.929+0000 7fdb527fc640 1 -- 172.21.15.77:0/1238347836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c016070 con 0x55e2da802b40 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.929+0000 7fdb70eef640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 0x7fdb540a5a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e2da802070 tx=0x7fdb64009b30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.929+0000 7fdb527fc640 1 -- 172.21.15.77:0/1238347836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5c005500 con 0x55e2da802b40 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.929+0000 7fdb527fc640 1 -- 172.21.15.77:0/1238347836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0116d0 con 0x55e2da802b40 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- 172.21.15.77:0/1238347836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 msgr2=0x55e2da801090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005fd0 tx=0x7fdb5c00e6c0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- 172.21.15.77:0/1238347836 shutdown_connections 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da812b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da8015d0 0x55e2da801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/1238347836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da802b40 0x55e2da801090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- 172.21.15.77:0/1238347836 >> 172.21.15.77:0/1238347836 conn(0x55e2da7ca310 msgr2=0x55e2da7fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- 172.21.15.77:0/1238347836 shutdown_connections 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- 172.21.15.77:0/1238347836 wait complete. 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.930+0000 7fdb74d2b740 1 -- start start 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 Processor -- start 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49024/0 (socket says 172.21.15.77:49024) 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb6bfff640 1 -- 172.21.15.77:0/164648082 learned_addr learned my addr 172.21.15.77:0/164648082 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da802b40 0x55e2da80d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 0x55e2da80d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e2da813480 con 0x55e2da8015d0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e2da722060 con 0x55e2da810780 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.931+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e2da813600 con 0x55e2da802b40 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da802b40 0x55e2da80d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb70eef640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 0x55e2da80d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 -- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da802b40 msgr2=0x55e2da80d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da802b40 0x55e2da80d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 -- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 msgr2=0x55e2da80d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb74d2b740 10 client.0 _ll_register_callbacks cb 0x55e2da4758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 0x55e2da80d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2da4d83c0 con 0x55e2da8015d0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5c005500 con 0x55e2da8015d0 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb70eef640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 0x55e2da80d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:17.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00ebc0 tx=0x7fdb5c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52ba2f29302eece server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.932+0000 7fdb4b7fe640 10 client.15732 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/164648082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c028060 con 0x55e2da8015d0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/164648082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdb5c004e20 con 0x55e2da8015d0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/164648082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb5c016040 con 0x55e2da8015d0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 10 client.15732 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e2da81ede0 con 0x55e2da8015d0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/164648082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb5c00f040 con 0x55e2da8015d0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 10 client.15732 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 20 client.15732 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da822ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 4 client.15732 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.933+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2da774860 con 0x55e2da820640 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.934+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da822ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.934+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da822ae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ee24b613a00658 server_cookie=4525c7a352156ba4 in_seq=0 out_seq=0 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.934+0000 7fdb4b7fe640 10 client.15732 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/164648082 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2da774860 con 0x55e2da820640 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb4b7fe640 10 client.15732 handle_command_reply: tid=0 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 client.15732 shutdown 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 msgr2=0x55e2da8100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00ebc0 tx=0x7fdb5c00eef0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 shutdown_connections 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da822ae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da810780 0x55e2da80d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb4b7fe640 0 client.15732 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da802b40 0x55e2da80d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/164648082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da8100e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 >> 172.21.15.77:0/164648082 conn(0x55e2da7ca310 msgr2=0x55e2da803930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.935+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 shutdown_connections 2024-09-15T22:13:17.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.936+0000 7fdb74d2b740 1 -- 172.21.15.77:0/164648082 wait complete. 2024-09-15T22:13:17.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.936+0000 7fdb74d2b740 20 client.15732 trim_cache size 0 max 16384 2024-09-15T22:13:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 Processor -- start 2024-09-15T22:13:17.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 -- start start 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da8015d0 0x55e2da81c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da80cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5800fe30 con 0x55e2da810780 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58011970 con 0x55e2da802b40 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58010500 con 0x55e2da8015d0 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb70eef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb70eef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49028/0 (socket says 172.21.15.77:49028) 2024-09-15T22:13:17.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.960+0000 7fdb70eef640 1 -- 172.21.15.77:0/532482218 learned_addr learned my addr 172.21.15.77:0/532482218 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da80cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 -- 172.21.15.77:0/532482218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da8015d0 msgr2=0x55e2da81c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da8015d0 0x55e2da81c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 -- 172.21.15.77:0/532482218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 msgr2=0x55e2da80cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da80cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 -- 172.21.15.77:0/532482218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2da80dc60 con 0x55e2da810780 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da80cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb70eef640 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009e30 tx=0x7fdb64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18e6f23363d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb51ffb640 1 -- 172.21.15.77:0/532482218 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64004b20 con 0x55e2da810780 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb51ffb640 1 -- 172.21.15.77:0/532482218 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64004cc0 con 0x55e2da810780 2024-09-15T22:13:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.961+0000 7fdb51ffb640 1 -- 172.21.15.77:0/532482218 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401e820 con 0x55e2da810780 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 -- 172.21.15.77:0/532482218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 msgr2=0x55e2da8283b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009e30 tx=0x7fdb64014000 comp rx=0 tx=0).stop 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 -- 172.21.15.77:0/532482218 shutdown_connections 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da810780 0x55e2da8283b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da80cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/532482218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da8015d0 0x55e2da81c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 -- 172.21.15.77:0/532482218 >> 172.21.15.77:0/532482218 conn(0x55e2da7ca310 msgr2=0x55e2da80bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 -- 172.21.15.77:0/532482218 shutdown_connections 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.963+0000 7fdb74d2b740 1 -- 172.21.15.77:0/532482218 wait complete. 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 -- start start 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 Processor -- start 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da884bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da8061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb580112a0 con 0x55e2da8015d0 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58010bd0 con 0x55e2da802b40 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.964+0000 7fdb74d2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5800de10 con 0x55e2da810780 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb74d2b740 10 client.0 _ll_register_callbacks cb 0x7fdb60005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39396/0 (socket says 172.21.15.77:39396) 2024-09-15T22:13:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3135453508 learned_addr learned my addr 172.21.15.77:0/3135453508 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:17.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da884bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 msgr2=0x55e2da8061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da8061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3135453508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 msgr2=0x55e2da884bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da884bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6b7fe640 1 -- 172.21.15.77:0/3135453508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2da8067a0 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.965+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da884bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.966+0000 7fdb6b7fe640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb600bc4f0 tx=0x7fdb600b8bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e380e82cba3da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.966+0000 7fdb4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.966+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600c0ba0 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.966+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdb600b8e60 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.966+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb64004cc0 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.967+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb600c5460 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.967+0000 7fdb74d2b740 10 client.25187 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.967+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2da807a80 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.967+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb600761c0 con 0x55e2da802b40 2024-09-15T22:13:17.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.967+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb600bbc40 con 0x55e2da802b40 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 10 client.25187 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 10 client.25187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 20 client.25187 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da8178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 4 client.25187 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdb64004cc0 con 0x55e2da820640 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da8178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb4b7fe640 10 client.25187 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.968+0000 7fdb6bfff640 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da8178d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ad710544f8759b4 server_cookie=8a5ba5f7e46e3b27 in_seq=0 out_seq=0 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb4b7fe640 1 -- 172.21.15.77:0/3135453508 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdb64004cc0 con 0x55e2da820640 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb4b7fe640 10 client.25187 handle_command_reply: tid=0 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 client.25187 shutdown 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 msgr2=0x55e2da8850f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb600bc4f0 tx=0x7fdb600b8bb0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 shutdown_connections 2024-09-15T22:13:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55e2da820640 0x55e2da8178d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb4b7fe640 0 client.25187 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e2da810780 0x55e2da8061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2da802b40 0x55e2da8850f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 --2- 172.21.15.77:0/3135453508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2da8015d0 0x55e2da884bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.969+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 >> 172.21.15.77:0/3135453508 conn(0x55e2da7ca310 msgr2=0x55e2da80bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.970+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 shutdown_connections 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.970+0000 7fdb74d2b740 1 -- 172.21.15.77:0/3135453508 wait complete. 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.970+0000 7fdb74d2b740 20 client.25187 trim_cache size 0 max 16384 2024-09-15T22:13:17.973 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 msgr2=0x7fdb540a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 0x7fdb540a5a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e2da802070 tx=0x7fdb64009b30 comp rx=0 tx=0).stop 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 msgr2=0x7fdb6c199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb60004770 tx=0x7fdb60003c10 comp rx=0 tx=0).stop 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 shutdown_connections 2024-09-15T22:13:17.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb540a3560 0x7fdb540a5a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb6c19a010 0x7fdb6c1a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c102ef0 0x7fdb6c199ad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 --2- 172.21.15.77:0/3404790371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb6c100e20 0x7fdb6c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 >> 172.21.15.77:0/3404790371 conn(0x7fdb6c0fc7d0 msgr2=0x7fdb6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 shutdown_connections 2024-09-15T22:13:17.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:17.971+0000 7fdb72952640 1 -- 172.21.15.77:0/3404790371 wait complete. 2024-09-15T22:13:17.986 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:18 smithi077 ceph-mon[30980]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-15T22:13:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:18 smithi196 ceph-mon[35109]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-15T22:13:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:18 smithi117 ceph-mon[35021]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-15T22:13:18.813 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:13:19.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 -- 172.21.15.77:0/2564002122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c100a90 msgr2=0x7f333c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 --2- 172.21.15.77:0/2564002122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c100a90 0x7f333c100ef0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f332c009930 tx=0x7f332c02a420 comp rx=0 tx=0).stop 2024-09-15T22:13:19.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 -- 172.21.15.77:0/2564002122 shutdown_connections 2024-09-15T22:13:19.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 --2- 172.21.15.77:0/2564002122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c101430 0x7f333c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 --2- 172.21.15.77:0/2564002122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c100a90 0x7f333c100ef0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 --2- 172.21.15.77:0/2564002122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f333c106a90 0x7f333c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 -- 172.21.15.77:0/2564002122 >> 172.21.15.77:0/2564002122 conn(0x7f333c0fc7d0 msgr2=0x7f333c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 -- 172.21.15.77:0/2564002122 shutdown_connections 2024-09-15T22:13:19.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.304+0000 7f3343a61640 1 -- 172.21.15.77:0/2564002122 wait complete. 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.305+0000 7f3343a61640 1 Processor -- start 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.305+0000 7f3343a61640 1 -- start start 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f333c101430 0x7f333c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 0x7f333c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111f60 con 0x7f333c106a90 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111de0 con 0x7f333c101430 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3343a61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1120e0 con 0x7f333c100a90 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3341ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 0x7f333c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52054/0 (socket says 172.21.15.77:52054) 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 -- 172.21.15.77:0/410910683 learned_addr learned my addr 172.21.15.77:0/410910683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 -- 172.21.15.77:0/410910683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f333c101430 msgr2=0x7f333c198980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f333c101430 0x7f333c198980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 -- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 msgr2=0x7f333c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 0x7f333c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f33417fd640 1 -- 172.21.15.77:0/410910683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19d3f0 con 0x7f333c100a90 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.306+0000 7f3341ffe640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 0x7f333c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f33417fd640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f333800d950 tx=0x7f333800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f33327fc640 1 -- 172.21.15.77:0/410910683 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338017070 con 0x7f333c100a90 2024-09-15T22:13:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c19d6e0 con 0x7f333c100a90 2024-09-15T22:13:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a4fc0 con 0x7f333c100a90 2024-09-15T22:13:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f33327fc640 1 -- 172.21.15.77:0/410910683 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3338009c30 con 0x7f333c100a90 2024-09-15T22:13:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.307+0000 7f33327fc640 1 -- 172.21.15.77:0/410910683 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333800f600 con 0x7f333c100a90 2024-09-15T22:13:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.310+0000 7f33327fc640 1 -- 172.21.15.77:0/410910683 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f333800f7a0 con 0x7f333c100a90 2024-09-15T22:13:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.310+0000 7f3345e3a740 1 Processor -- start 2024-09-15T22:13:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.310+0000 7f3345e3a740 1 -- start start 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0ef40 0x5604c9b1d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b1dd50 0x5604c9b20140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604c9a742a0 con 0x5604c9b0e620 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5604c9a1bb10 con 0x5604c9b1dd50 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3345e3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5604c9a70260 con 0x5604c9b0ef40 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f33327fc640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 0x7f33280a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33674/0 (socket says 172.21.15.77:33674) 2024-09-15T22:13:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 -- 172.21.15.77:0/3489410365 learned_addr learned my addr 172.21.15.77:0/3489410365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3340ffc640 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b1dd50 0x5604c9b20140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 0x7f33280a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f33327fc640 1 -- 172.21.15.77:0/410910683 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f33380cfe00 con 0x7f333c100a90 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 -- 172.21.15.77:0/3489410365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0ef40 msgr2=0x5604c9b1d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0ef40 0x5604c9b1d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 -- 172.21.15.77:0/3489410365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b1dd50 msgr2=0x5604c9b20140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b1dd50 0x5604c9b20140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.311+0000 7f3341ffe640 1 -- 172.21.15.77:0/3489410365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c97e5b90 con 0x5604c9b0e620 2024-09-15T22:13:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.312+0000 7f3341ffe640 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3334012e20 tx=0x7f33340147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cebd7f00baea7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.312+0000 7f3341ffe640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 0x7f33280a5aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3334012fd0 tx=0x7f333401f000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.312+0000 7f33277fe640 1 -- 172.21.15.77:0/3489410365 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334020440 con 0x5604c9b0e620 2024-09-15T22:13:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.312+0000 7f33277fe640 1 -- 172.21.15.77:0/3489410365 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334020ac0 con 0x5604c9b0e620 2024-09-15T22:13:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.312+0000 7f33277fe640 1 -- 172.21.15.77:0/3489410365 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333401f9c0 con 0x5604c9b0e620 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- 172.21.15.77:0/3489410365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 msgr2=0x5604c9b0ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3334012e20 tx=0x7f33340147e0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- 172.21.15.77:0/3489410365 shutdown_connections 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b1dd50 0x5604c9b20140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0ef40 0x5604c9b1d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 --2- 172.21.15.77:0/3489410365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b0ea00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- 172.21.15.77:0/3489410365 >> 172.21.15.77:0/3489410365 conn(0x5604c9b0acf0 msgr2=0x5604c9b0b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- 172.21.15.77:0/3489410365 shutdown_connections 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- 172.21.15.77:0/3489410365 wait complete. 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 -- start start 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.313+0000 7f3345e3a740 1 Processor -- start 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b1b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5604c9b20860 con 0x5604c9b0e620 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5604c99fc6e0 con 0x5604c9b0ef40 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5604c9b209e0 con 0x5604c9b1dd50 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f33417fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f33417fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33684/0 (socket says 172.21.15.77:33684) 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f33417fd640 1 -- 172.21.15.77:0/1728412850 learned_addr learned my addr 172.21.15.77:0/1728412850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3345e3a740 10 client.0 _ll_register_callbacks cb 0x5604c9a20a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f3340ffc640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.314+0000 7f33417fd640 1 -- 172.21.15.77:0/1728412850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 msgr2=0x5604c9b1b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3341ffe640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b1b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f33417fd640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b1b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f33417fd640 1 -- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 msgr2=0x5604c9b1b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f33417fd640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b1b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f33417fd640 1 -- 172.21.15.77:0/1728412850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c9b29e40 con 0x5604c9b0e620 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3340ffc640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3341ffe640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b1b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f33417fd640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3338009610 tx=0x7f3338009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d02941cb4b249a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338020070 con 0x5604c9b0e620 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f33380862b0 con 0x5604c9b0e620 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3334020ac0 con 0x5604c9b0e620 2024-09-15T22:13:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3338084e60 con 0x5604c9b0e620 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3345e3a740 10 client.15750 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604c9b2c7d0 con 0x5604c9b0e620 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.315+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334020ac0 con 0x5604c9b0e620 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33380ba270 con 0x5604c9b0e620 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 10 client.15750 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 10 client.15750 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 20 client.15750 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b2e100 0x5604c9b30560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 4 client.15750 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604c9a819d0 con 0x5604c9b2e100 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.316+0000 7f3340ffc640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b2e100 0x5604c9b30560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3340ffc640 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b2e100 0x5604c9b30560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4389e3f916dfcf server_cookie=e797a7948c9b2e1e in_seq=0 out_seq=0 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3324ff9640 10 client.15750 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3324ff9640 1 -- 172.21.15.77:0/1728412850 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604c9a819d0 con 0x5604c9b2e100 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3324ff9640 10 client.15750 handle_command_reply: tid=0 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3345e3a740 1 client.15750 shutdown 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 msgr2=0x5604c9b1ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.317+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3338009610 tx=0x7f3338009f20 comp rx=0 tx=0).stop 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 shutdown_connections 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b2e100 0x5604c9b30560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b1b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 --2- 172.21.15.77:0/1728412850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b1ac90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 >> 172.21.15.77:0/1728412850 conn(0x5604c9b0acf0 msgr2=0x5604c9b10a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3324ff9640 0 client.15750 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 shutdown_connections 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 1 -- 172.21.15.77:0/1728412850 wait complete. 2024-09-15T22:13:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.318+0000 7f3345e3a740 20 client.15750 trim_cache size 0 max 16384 2024-09-15T22:13:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 Processor -- start 2024-09-15T22:13:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 -- start start 2024-09-15T22:13:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0e620 0x5604c9b19560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f332000fd20 con 0x5604c9b1dd50 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3320011860 con 0x5604c9b0ef40 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.350+0000 7f3345e3a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33200103f0 con 0x5604c9b0e620 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33696/0 (socket says 172.21.15.77:33696) 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 -- 172.21.15.77:0/4030083142 learned_addr learned my addr 172.21.15.77:0/4030083142 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3340ffc640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f33417fd640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0e620 0x5604c9b19560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 -- 172.21.15.77:0/4030083142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0e620 msgr2=0x5604c9b19560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0e620 0x5604c9b19560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 -- 172.21.15.77:0/4030083142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 msgr2=0x5604c9b35e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b35e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 -- 172.21.15.77:0/4030083142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c9b16610 con 0x5604c9b1dd50 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3340ffc640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.351+0000 7f3341ffe640 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3334006010 tx=0x7f3334012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f0925aa0725687 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.352+0000 7f3326ffd640 1 -- 172.21.15.77:0/4030083142 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340049a0 con 0x5604c9b1dd50 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.352+0000 7f3326ffd640 1 -- 172.21.15.77:0/4030083142 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334004b40 con 0x5604c9b1dd50 2024-09-15T22:13:19.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.352+0000 7f3326ffd640 1 -- 172.21.15.77:0/4030083142 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334029450 con 0x5604c9b1dd50 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- 172.21.15.77:0/4030083142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 msgr2=0x5604c9b363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3334006010 tx=0x7f3334012d20 comp rx=0 tx=0).stop 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- 172.21.15.77:0/4030083142 shutdown_connections 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b1dd50 0x5604c9b363b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b35e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 --2- 172.21.15.77:0/4030083142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b0e620 0x5604c9b19560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- 172.21.15.77:0/4030083142 >> 172.21.15.77:0/4030083142 conn(0x5604c9b0acf0 msgr2=0x5604c9b09e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- 172.21.15.77:0/4030083142 shutdown_connections 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- 172.21.15.77:0/4030083142 wait complete. 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.353+0000 7f3345e3a740 1 -- start start 2024-09-15T22:13:19.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.354+0000 7f3345e3a740 1 Processor -- start 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.354+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.354+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.354+0000 7f3345e3a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b21cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3345e3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3320011190 con 0x5604c9b0e620 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33710/0 (socket says 172.21.15.77:33710) 2024-09-15T22:13:19.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 -- 172.21.15.77:0/2064291327 learned_addr learned my addr 172.21.15.77:0/2064291327 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3345e3a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3320010ac0 con 0x5604c9b0ef40 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f332000dd00 con 0x5604c9b1dd50 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3345e3a740 10 client.0 _ll_register_callbacks cb 0x7f333809b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3340ffc640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 -- 172.21.15.77:0/2064291327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 msgr2=0x5604c9b21cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b21cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 -- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 msgr2=0x5604c9b217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b217b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 -- 172.21.15.77:0/2064291327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c9b19c40 con 0x5604c9b0e620 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f33417fd640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f33380cd0c0 tx=0x7f333800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d4a0a1dfaccdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338020070 con 0x5604c9b0e620 2024-09-15T22:13:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f33380978b0 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.355+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3334004b40 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33380a47d0 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 10 client.15756 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604c9b98cb0 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334004b40 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33380a10a0 con 0x5604c9b0e620 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 10 client.15756 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 10 client.15756 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 20 client.15756 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b992b0 0x5604c9b9b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 4 client.15756 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x5604c9a7bd20 con 0x5604c9b992b0 2024-09-15T22:13:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.356+0000 7f3340ffc640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b992b0 0x5604c9b9b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.357+0000 7f3340ffc640 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b992b0 0x5604c9b9b6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a034d0f65634faa1 server_cookie=fbaeb74b285072a0 in_seq=0 out_seq=0 2024-09-15T22:13:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.357+0000 7f3324ff9640 10 client.15756 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3324ff9640 1 -- 172.21.15.77:0/2064291327 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5604c9a7bd20 con 0x5604c9b992b0 2024-09-15T22:13:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3324ff9640 10 client.15756 handle_command_reply: tid=0 2024-09-15T22:13:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 client.15756 shutdown 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 msgr2=0x5604c9b26480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f33380cd0c0 tx=0x7f333800bf60 comp rx=0 tx=0).stop 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 shutdown_connections 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5604c9b992b0 0x5604c9b9b6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5604c9b1dd50 0x5604c9b21cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3324ff9640 0 client.15756 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5604c9b0ef40 0x5604c9b217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 --2- 172.21.15.77:0/2064291327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5604c9b0e620 0x5604c9b26480 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.370+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 >> 172.21.15.77:0/2064291327 conn(0x5604c9b0acf0 msgr2=0x5604c9b09e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.371+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 shutdown_connections 2024-09-15T22:13:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.371+0000 7f3345e3a740 1 -- 172.21.15.77:0/2064291327 wait complete. 2024-09-15T22:13:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.371+0000 7f3345e3a740 20 client.15756 trim_cache size 0 max 16384 2024-09-15T22:13:19.373 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:19.373 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 30, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 2, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "7d71dbcf": { 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 2, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.2, 2024-09-15T22:13:19.374 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:19.375 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 msgr2=0x7f33280a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 0x7f33280a5aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3334012fd0 tx=0x7f333401f000 comp rx=0 tx=0).stop 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 msgr2=0x7f333c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f333800d950 tx=0x7f333800de10 comp rx=0 tx=0).stop 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 shutdown_connections 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33280a35e0 0x7f33280a5aa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f333c106a90 0x7f333c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f333c101430 0x7f333c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 --2- 172.21.15.77:0/410910683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f333c100a90 0x7f333c198440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 >> 172.21.15.77:0/410910683 conn(0x7f333c0fc7d0 msgr2=0x7f333c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 shutdown_connections 2024-09-15T22:13:19.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:19.372+0000 7f3343a61640 1 -- 172.21.15.77:0/410910683 wait complete. 2024-09-15T22:13:19.386 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:20.056 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7d71dbcf', quiesce took 7.6 seconds. Will release after: 22.6 2024-09-15T22:13:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 -- 172.21.15.77:0/3727669933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 msgr2=0x7f72ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/3727669933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac1029b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002a480 comp rx=0 tx=0).stop 2024-09-15T22:13:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 -- 172.21.15.77:0/3727669933 shutdown_connections 2024-09-15T22:13:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/3727669933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac102ef0 0x7f72ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/3727669933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac1029b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/3727669933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac100e20 0x7f72ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 -- 172.21.15.77:0/3727669933 >> 172.21.15.77:0/3727669933 conn(0x7f72ac0fc7d0 msgr2=0x7f72ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 -- 172.21.15.77:0/3727669933 shutdown_connections 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.490+0000 7f72b1cc8640 1 -- 172.21.15.77:0/3727669933 wait complete. 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 Processor -- start 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 -- start start 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac100e20 0x7f72ac197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 0x7f72ac19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac111020 con 0x7f72ac102570 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac110ea0 con 0x7f72ac102ef0 2024-09-15T22:13:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.491+0000 7f72b1cc8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1111a0 con 0x7f72ac100e20 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33730/0 (socket says 172.21.15.77:33730) 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 -- 172.21.15.77:0/2539889938 learned_addr learned my addr 172.21.15.77:0/2539889938 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac100e20 0x7f72ac197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72abfff640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 0x7f72ac19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 -- 172.21.15.77:0/2539889938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac100e20 msgr2=0x7f72ac197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac100e20 0x7f72ac197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 -- 172.21.15.77:0/2539889938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 msgr2=0x7f72ac19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 0x7f72ac19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 -- 172.21.15.77:0/2539889938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac19c4f0 con 0x7f72ac102570 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72abfff640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 0x7f72ac19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72a8ff9640 1 -- 172.21.15.77:0/2539889938 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a003f070 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac19c780 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72a8ff9640 1 -- 172.21.15.77:0/2539889938 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a0031e30 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72a8ff9640 1 -- 172.21.15.77:0/2539889938 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0035030 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.492+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a40c0 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72a8ff9640 1 -- 172.21.15.77:0/2539889938 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f72a0042070 con 0x7f72ac102570 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 Processor -- start 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 -- start start 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef8302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef83b580 0x558cef83d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cef7942f0 con 0x558cef8307e0 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558cef73bcf0 con 0x558cef831d50 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72b40a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cef7902b0 con 0x558cef83b580 2024-09-15T22:13:20.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72a8ff9640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 0x7f72940a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72ab7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33734/0 (socket says 172.21.15.77:33734) 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72ab7fe640 1 -- 172.21.15.77:0/467981162 learned_addr learned my addr 172.21.15.77:0/467981162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.495+0000 7f72aaffd640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef83b580 0x558cef83d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72abfff640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 0x7f72940a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72a8ff9640 1 -- 172.21.15.77:0/2539889938 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f72a00f0d30 con 0x7f72ac102570 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72abfff640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef8302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 -- 172.21.15.77:0/467981162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef83b580 msgr2=0x558cef83d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef83b580 0x558cef83d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 -- 172.21.15.77:0/467981162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 msgr2=0x558cef8302a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef8302a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 -- 172.21.15.77:0/467981162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cef7203a0 con 0x558cef8307e0 2024-09-15T22:13:20.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72abfff640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef8302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b820 tx=0x7f729c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770c3437a841828c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f72abfff640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 0x7f72940a5a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f72a400af80 tx=0x7f72a400a3d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f7291ffb640 1 -- 172.21.15.77:0/467981162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c00d950 con 0x558cef8307e0 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f7291ffb640 1 -- 172.21.15.77:0/467981162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c019070 con 0x558cef8307e0 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.496+0000 7f7291ffb640 1 -- 172.21.15.77:0/467981162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0123e0 con 0x558cef8307e0 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- 172.21.15.77:0/467981162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 msgr2=0x558cef830c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b820 tx=0x7f729c00bce0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- 172.21.15.77:0/467981162 shutdown_connections 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef83b580 0x558cef83d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef8307e0 0x558cef830c20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 --2- 172.21.15.77:0/467981162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef8302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- 172.21.15.77:0/467981162 >> 172.21.15.77:0/467981162 conn(0x558cef7f7310 msgr2=0x558cef82ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- 172.21.15.77:0/467981162 shutdown_connections 2024-09-15T22:13:20.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- 172.21.15.77:0/467981162 wait complete. 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.497+0000 7f72b40a1740 1 -- start start 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 Processor -- start 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef8307e0 0x558cef83af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef831d50 0x558cef838200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558cef83e280 con 0x558cef83b580 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558cef74f060 con 0x558cef8307e0 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cef83e400 con 0x558cef831d50 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72b40a1740 10 client.0 _ll_register_callbacks cb 0x558cef4a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33746/0 (socket says 172.21.15.77:33746) 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 -- 172.21.15.77:0/2540484753 learned_addr learned my addr 172.21.15.77:0/2540484753 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:20.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef831d50 0x558cef838200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef8307e0 0x558cef83af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 -- 172.21.15.77:0/2540484753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef831d50 msgr2=0x558cef838200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef831d50 0x558cef838200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 -- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef8307e0 msgr2=0x558cef83af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef8307e0 0x558cef83af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.498+0000 7f72abfff640 1 -- 172.21.15.77:0/2540484753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cef5053c0 con 0x558cef83b580 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72abfff640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72a4012c70 tx=0x7f72a4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d029f22fa31aaff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4022020 con 0x558cef83b580 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f72a401e610 con 0x558cef83b580 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f729c019070 con 0x558cef83b580 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72a402d050 con 0x558cef83b580 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 10 client.15768 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:20.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cef849cb0 con 0x558cef83b580 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f729c019070 con 0x558cef83b580 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f72a4010040 con 0x558cef83b580 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 10 client.15768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 20 client.15768 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef84d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 4 client.15768 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.499+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cef7a1860 con 0x558cef84b170 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.500+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef84d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.500+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef84d610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5bc83c3bfeece63 server_cookie=70bb3e9fe72db621 in_seq=0 out_seq=0 2024-09-15T22:13:20.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.500+0000 7f728affd640 10 client.15768 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f728affd640 1 -- 172.21.15.77:0/2540484753 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cef7a1860 con 0x558cef84b170 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f728affd640 10 client.15768 handle_command_reply: tid=0 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 client.15768 shutdown 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 msgr2=0x558cef8387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f72a4012c70 tx=0x7f72a4004b50 comp rx=0 tx=0).stop 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 shutdown_connections 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef84d610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef8387f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef831d50 0x558cef838200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f728affd640 0 client.15768 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2540484753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef8307e0 0x558cef83af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 >> 172.21.15.77:0/2540484753 conn(0x558cef7f7310 msgr2=0x558cef832b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 shutdown_connections 2024-09-15T22:13:20.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 1 -- 172.21.15.77:0/2540484753 wait complete. 2024-09-15T22:13:20.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.501+0000 7f72b40a1740 20 client.15768 trim_cache size 0 max 16384 2024-09-15T22:13:20.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.518+0000 7f72b40a1740 1 Processor -- start 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.518+0000 7f72b40a1740 1 -- start start 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef847520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef82c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1acd00 con 0x558cef83b580 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1ae840 con 0x558cef831d50 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72b40a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1ad3d0 con 0x558cef8307e0 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33754/0 (socket says 172.21.15.77:33754) 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 -- 172.21.15.77:0/2169980395 learned_addr learned my addr 172.21.15.77:0/2169980395 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef82c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef847520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 -- 172.21.15.77:0/2169980395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 msgr2=0x558cef847520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef847520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 -- 172.21.15.77:0/2169980395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 msgr2=0x558cef82c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef82c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 -- 172.21.15.77:0/2169980395 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cef82d390 con 0x558cef83b580 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72aaffd640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef82c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.519+0000 7f72abfff640 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f72a400a660 tx=0x7f72a400a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc4d356530abab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.520+0000 7f72917fa640 1 -- 172.21.15.77:0/2169980395 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4022070 con 0x558cef83b580 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.520+0000 7f72917fa640 1 -- 172.21.15.77:0/2169980395 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4017040 con 0x558cef83b580 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.520+0000 7f72917fa640 1 -- 172.21.15.77:0/2169980395 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4016070 con 0x558cef83b580 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- 172.21.15.77:0/2169980395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 msgr2=0x558cef82ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f72a400a660 tx=0x7f72a400a690 comp rx=0 tx=0).stop 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- 172.21.15.77:0/2169980395 shutdown_connections 2024-09-15T22:13:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef83b580 0x558cef82ce50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef831d50 0x558cef82c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 --2- 172.21.15.77:0/2169980395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef847520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- 172.21.15.77:0/2169980395 >> 172.21.15.77:0/2169980395 conn(0x558cef7f7310 msgr2=0x558cef8381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- 172.21.15.77:0/2169980395 shutdown_connections 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- 172.21.15.77:0/2169980395 wait complete. 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 -- start start 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.521+0000 7f72b40a1740 1 Processor -- start 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef835da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef83b580 0x558cef852f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1ae170 con 0x558cef831d50 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1adaa0 con 0x558cef83b580 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1aace0 con 0x558cef8307e0 2024-09-15T22:13:20.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72b40a1740 10 client.0 _ll_register_callbacks cb 0x7f72a400abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33758/0 (socket says 172.21.15.77:33758) 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 -- 172.21.15.77:0/4267583746 learned_addr learned my addr 172.21.15.77:0/4267583746 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72abfff640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef83b580 0x558cef852f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef835da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 -- 172.21.15.77:0/4267583746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 msgr2=0x558cef835da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef835da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 -- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef83b580 msgr2=0x558cef852f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef83b580 0x558cef852f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.522+0000 7f72aaffd640 1 -- 172.21.15.77:0/4267583746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cef853770 con 0x558cef831d50 2024-09-15T22:13:20.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:20 smithi077 ceph-mon[30980]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f72aaffd640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f72a00b9460 tx=0x7f72a003b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18af7cccf331bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f728affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0049040 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f72a00b9100 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a4017040 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72a00bd190 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f72b40a1740 10 client.15774 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.523+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558cef8b7fe0 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a4017040 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f72a00a7040 con 0x558cef831d50 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 20 client.15774 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef8ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 4 client.15774 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cef79bd70 con 0x558cef84b170 2024-09-15T22:13:20.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.524+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef8ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f72ab7fe640 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef8ba6f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb841fecd08b55a3 server_cookie=a824f8a205ed7e7e in_seq=0 out_seq=0 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f728affd640 10 client.15774 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f728affd640 1 -- 172.21.15.77:0/4267583746 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cef79bd70 con 0x558cef84b170 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f728affd640 10 client.15774 handle_command_reply: tid=0 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f72b40a1740 1 client.15774 shutdown 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 msgr2=0x558cef852a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.525+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f72a00b9460 tx=0x7f72a003b620 comp rx=0 tx=0).stop 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 shutdown_connections 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558cef84b170 0x558cef8ba6f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558cef83b580 0x558cef852f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558cef831d50 0x558cef852a00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 --2- 172.21.15.77:0/4267583746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cef8307e0 0x558cef835da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 >> 172.21.15.77:0/4267583746 conn(0x558cef7f7310 msgr2=0x558cef8366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f728affd640 0 client.15774 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 shutdown_connections 2024-09-15T22:13:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 1 -- 172.21.15.77:0/4267583746 wait complete. 2024-09-15T22:13:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.526+0000 7f72b40a1740 20 client.15774 trim_cache size 0 max 16384 2024-09-15T22:13:20.537 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 msgr2=0x7f72940a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 0x7f72940a5a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f72a400af80 tx=0x7f72a400a3d0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 msgr2=0x7f72ac197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0003c80 comp rx=0 tx=0).stop 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 shutdown_connections 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f72940a3560 0x7f72940a5a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ac102ef0 0x7f72ac19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72ac102570 0x7f72ac197a80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 --2- 172.21.15.77:0/2539889938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72ac100e20 0x7f72ac197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 >> 172.21.15.77:0/2539889938 conn(0x7f72ac0fc7d0 msgr2=0x7f72ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 shutdown_connections 2024-09-15T22:13:20.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:20.527+0000 7f72b1cc8640 1 -- 172.21.15.77:0/2539889938 wait complete. 2024-09-15T22:13:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:20 smithi196 ceph-mon[35109]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:20 smithi117 ceph-mon[35021]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:21.227 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:13:21.495 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.029+0000 7fb9bb791640 1 -- 172.21.15.77:0/1088801848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 msgr2=0x7fb9b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.029+0000 7fb9bb791640 1 --2- 172.21.15.77:0/1088801848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4101200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009900 tx=0x7fb9b002a480 comp rx=0 tx=0).stop 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 -- 172.21.15.77:0/1088801848 shutdown_connections 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 --2- 172.21.15.77:0/1088801848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102ef0 0x7fb9b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 --2- 172.21.15.77:0/1088801848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102570 0x7fb9b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 --2- 172.21.15.77:0/1088801848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4101200 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 -- 172.21.15.77:0/1088801848 >> 172.21.15.77:0/1088801848 conn(0x7fb9b40fc7d0 msgr2=0x7fb9b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 -- 172.21.15.77:0/1088801848 shutdown_connections 2024-09-15T22:13:22.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.030+0000 7fb9bb791640 1 -- 172.21.15.77:0/1088801848 wait complete. 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 Processor -- start 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 -- start start 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102570 0x7fb9b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102ef0 0x7fb9b419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b4112060 con 0x7fb9b4100e20 2024-09-15T22:13:22.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b4111ee0 con 0x7fb9b4102570 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.031+0000 7fb9bb791640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41121e0 con 0x7fb9b4102ef0 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33782/0 (socket says 172.21.15.77:33782) 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 -- 172.21.15.77:0/3406151916 learned_addr learned my addr 172.21.15.77:0/3406151916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102570 0x7fb9b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b9d2e640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102ef0 0x7fb9b419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 -- 172.21.15.77:0/3406151916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102ef0 msgr2=0x7fb9b419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102ef0 0x7fb9b419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 -- 172.21.15.77:0/3406151916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102570 msgr2=0x7fb9b4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102570 0x7fb9b4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 -- 172.21.15.77:0/3406151916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b419d4b0 con 0x7fb9b4100e20 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0006010 tx=0x7fb9b0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.032+0000 7fb9aa7fc640 1 -- 172.21.15.77:0/3406151916 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b003f070 con 0x7fb9b4100e20 2024-09-15T22:13:22.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.033+0000 7fb9aa7fc640 1 -- 172.21.15.77:0/3406151916 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b0031c20 con 0x7fb9b4100e20 2024-09-15T22:13:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.033+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b419d740 con 0x7fb9b4100e20 2024-09-15T22:13:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.033+0000 7fb9aa7fc640 1 -- 172.21.15.77:0/3406151916 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0035030 con 0x7fb9b4100e20 2024-09-15T22:13:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.033+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b410e400 con 0x7fb9b4100e20 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.035+0000 7fb9bdb6a740 1 Processor -- start 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.035+0000 7fb9bdb6a740 1 -- start start 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.035+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e5706090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.035+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e57065d0 0x5587e5706a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587e566c2f0 con 0x5587e5719b80 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587e5613cf0 con 0x5587e57065d0 2024-09-15T22:13:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587e56682b0 con 0x5587e5707b40 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b9d2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e5706090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33798/0 (socket says 172.21.15.77:33798) 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 -- 172.21.15.77:0/3935882102 learned_addr learned my addr 172.21.15.77:0/3935882102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 -- 172.21.15.77:0/3935882102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 msgr2=0x5587e5706090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e5706090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9aa7fc640 1 -- 172.21.15.77:0/3406151916 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb9b0042070 con 0x7fb9b4100e20 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 -- 172.21.15.77:0/3935882102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e57065d0 msgr2=0x5587e5706a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e57065d0 0x5587e5706a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b8d2c640 1 -- 172.21.15.77:0/3935882102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e55f83a0 con 0x5587e5719b80 2024-09-15T22:13:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.036+0000 7fb9b9d2e640 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e5706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400aca0 tx=0x7fb9a400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b878c91b21f7139 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb99f7fe640 1 -- 172.21.15.77:0/3935882102 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a401d070 con 0x5587e5719b80 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb99f7fe640 1 -- 172.21.15.77:0/3935882102 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4018b20 con 0x5587e5719b80 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb99f7fe640 1 -- 172.21.15.77:0/3935882102 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4010cb0 con 0x5587e5719b80 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb9aa7fc640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 0x7fb9a00a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb9b9d2e640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 0x7fb9a00a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.037+0000 7fb9aa7fc640 1 -- 172.21.15.77:0/3406151916 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb9b00f0e30 con 0x7fb9b4100e20 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3935882102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 msgr2=0x5587e571bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400aca0 tx=0x7fb9a400f790 comp rx=0 tx=0).stop 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3935882102 shutdown_connections 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e5719b80 0x5587e571bf70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e57065d0 0x5587e5706a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3935882102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e5706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9b9d2e640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 0x7fb9a00a5b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0092f0 tx=0x7fb9ac012040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3935882102 >> 172.21.15.77:0/3935882102 conn(0x5587e56cf310 msgr2=0x5587e5702c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3935882102 shutdown_connections 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3935882102 wait complete. 2024-09-15T22:13:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 -- start start 2024-09-15T22:13:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.038+0000 7fb9bdb6a740 1 Processor -- start 2024-09-15T22:13:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e57168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5719b80 0x5587e5717080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5587e571c880 con 0x5587e57065d0 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5587e5627060 con 0x5587e5719b80 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5587e571ca00 con 0x5587e5707b40 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9bdb6a740 10 client.0 _ll_register_callbacks cb 0x5587e537a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33804/0 (socket says 172.21.15.77:33804) 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 -- 172.21.15.77:0/3299734621 learned_addr learned my addr 172.21.15.77:0/3299734621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e57168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b9d2e640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5719b80 0x5587e5717080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 -- 172.21.15.77:0/3299734621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 msgr2=0x5587e57168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e57168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 -- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5719b80 msgr2=0x5587e5717080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5719b80 0x5587e5717080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.039+0000 7fb9b952d640 1 -- 172.21.15.77:0/3299734621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e53dd3c0 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00bcb20 tx=0x7fb9b00bca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4513a83186c14043 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb99cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0049030 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb9b00a70f0 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a4018b20 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9b00a7330 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb9bdb6a740 10 client.15786 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587e571c880 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.040+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4018b20 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb9b00db380 con 0x5587e57065d0 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 10 client.15786 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:22.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 20 client.15786 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5729860 0x5587e572bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 4 client.15786 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e5679860 con 0x5587e5729860 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5729860 0x5587e572bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb99cff9640 10 client.15786 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.041+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5729860 0x5587e572bcf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872f1b42ae933eee server_cookie=945b9136bdc9829f in_seq=0 out_seq=0 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb99cff9640 1 -- 172.21.15.77:0/3299734621 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587e5679860 con 0x5587e5729860 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb99cff9640 10 client.15786 handle_command_reply: tid=0 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 client.15786 shutdown 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 msgr2=0x5587e5719640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00bcb20 tx=0x7fb9b00bca20 comp rx=0 tx=0).stop 2024-09-15T22:13:22.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 shutdown_connections 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5729860 0x5587e572bcf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5719b80 0x5587e5717080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5707b40 0x5587e57168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb99cff9640 0 client.15786 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3299734621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5719640 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 >> 172.21.15.77:0/3299734621 conn(0x5587e56cf310 msgr2=0x5587e5708960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.042+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 shutdown_connections 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.043+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3299734621 wait complete. 2024-09-15T22:13:22.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.043+0000 7fb9bdb6a740 20 client.15786 trim_cache size 0 max 16384 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 Processor -- start 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 -- start start 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5731590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5731ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41acbf0 con 0x5587e57065d0 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41ae730 con 0x5587e5707b40 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41ad2c0 con 0x5587e5719b80 2024-09-15T22:13:22.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33810/0 (socket says 172.21.15.77:33810) 2024-09-15T22:13:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.067+0000 7fb9b952d640 1 -- 172.21.15.77:0/1890507916 learned_addr learned my addr 172.21.15.77:0/1890507916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 -- 172.21.15.77:0/1890507916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 msgr2=0x5587e5731ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5731590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5731ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 -- 172.21.15.77:0/1890507916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 msgr2=0x5587e5731590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5731590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 -- 172.21.15.77:0/1890507916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e57173f0 con 0x5587e57065d0 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5731590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb9b952d640 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0041000 tx=0x7fb9b0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ec43000696c45a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.068+0000 7fb99effd640 1 -- 172.21.15.77:0/1890507916 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0049070 con 0x5587e57065d0 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.069+0000 7fb99effd640 1 -- 172.21.15.77:0/1890507916 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b0031dd0 con 0x5587e57065d0 2024-09-15T22:13:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.069+0000 7fb99effd640 1 -- 172.21.15.77:0/1890507916 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00c5370 con 0x5587e57065d0 2024-09-15T22:13:22.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/1890507916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 msgr2=0x5587e5702e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0041000 tx=0x7fb9b0005900 comp rx=0 tx=0).stop 2024-09-15T22:13:22.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/1890507916 shutdown_connections 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5731590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/1890507916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e5702e10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/1890507916 >> 172.21.15.77:0/1890507916 conn(0x5587e56cf310 msgr2=0x5587e5702700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/1890507916 shutdown_connections 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/1890507916 wait complete. 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 -- start start 2024-09-15T22:13:22.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.070+0000 7fb9bdb6a740 1 Processor -- start 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5713550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5713a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41ae060 con 0x5587e57065d0 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41ad990 con 0x5587e5707b40 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b41aabd0 con 0x5587e5719b80 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9bdb6a740 10 client.0 _ll_register_callbacks cb 0x7fb9b00b8e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9b952d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33812/0 (socket says 172.21.15.77:33812) 2024-09-15T22:13:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.071+0000 7fb9b952d640 1 -- 172.21.15.77:0/3537680770 learned_addr learned my addr 172.21.15.77:0/3537680770 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5713550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 -- 172.21.15.77:0/3537680770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 msgr2=0x5587e5713a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5713a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 -- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 msgr2=0x5587e5713550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5713550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 -- 172.21.15.77:0/3537680770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e57141d0 con 0x5587e57065d0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9b952d640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00c52d0 tx=0x7fb9b003b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec61968e446d3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb99cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0049030 con 0x5587e57065d0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb9b00c5570 con 0x5587e57065d0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b0031dd0 con 0x5587e57065d0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9b00cc050 con 0x5587e57065d0 2024-09-15T22:13:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9bdb6a740 10 client.15792 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.072+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587e57141d0 con 0x5587e57065d0 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b00bcb90 con 0x5587e57065d0 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb9b00a3070 con 0x5587e57065d0 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 20 client.15792 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5794790 0x5587e5796b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 4 client.15792 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb9b0031dd0 con 0x5587e5794790 2024-09-15T22:13:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5794790 0x5587e5796b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.073+0000 7fb99cff9640 10 client.15792 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9b8d2c640 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5794790 0x5587e5796b80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27076a9813ec3e7 server_cookie=381c78c1ca06eaef in_seq=0 out_seq=0 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb99cff9640 1 -- 172.21.15.77:0/3537680770 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb9b0031dd0 con 0x5587e5794790 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb99cff9640 10 client.15792 handle_command_reply: tid=0 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9bdb6a740 1 client.15792 shutdown 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 msgr2=0x5587e57222d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00c52d0 tx=0x7fb9b003b7b0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 shutdown_connections 2024-09-15T22:13:22.081 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:22.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.074+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5587e5794790 0x5587e5796b80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb99cff9640 0 client.15792 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5587e5719b80 0x5587e5713a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587e5707b40 0x5587e5713550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 --2- 172.21.15.77:0/3537680770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5587e57065d0 0x5587e57222d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 >> 172.21.15.77:0/3537680770 conn(0x5587e56cf310 msgr2=0x5587e57014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 shutdown_connections 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 1 -- 172.21.15.77:0/3537680770 wait complete. 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bdb6a740 20 client.15792 trim_cache size 0 max 16384 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 msgr2=0x7fb9a00a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 0x7fb9a00a5b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0092f0 tx=0x7fb9ac012040 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 msgr2=0x7fb9b4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0006010 tx=0x7fb9b0003f80 comp rx=0 tx=0).stop 2024-09-15T22:13:22.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 shutdown_connections 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb9a00a3660 0x7fb9a00a5b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9b4102ef0 0x7fb9b419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4102570 0x7fb9b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 --2- 172.21.15.77:0/3406151916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb9b4100e20 0x7fb9b4198500 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.075+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 >> 172.21.15.77:0/3406151916 conn(0x7fb9b40fc7d0 msgr2=0x7fb9b41068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.076+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 shutdown_connections 2024-09-15T22:13:22.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:22.076+0000 7fb9bb791640 1 -- 172.21.15.77:0/3406151916 wait complete. 2024-09-15T22:13:22.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:22 smithi196 ceph-mon[35109]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:22 smithi117 ceph-mon[35021]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:22.781 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:22 smithi077 ceph-mon[30980]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:23.817 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:13:23.818 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:13:24.097 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:24.518 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:24 smithi077 ceph-mon[30980]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-15T22:13:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:24 smithi196 ceph-mon[35109]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-15T22:13:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:24 smithi117 ceph-mon[35021]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-15T22:13:24.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 -- 172.21.15.77:0/808601929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 msgr2=0x7f280c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 --2- 172.21.15.77:0/808601929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c10eaa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2804009ca0 tx=0x7f280402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 -- 172.21.15.77:0/808601929 shutdown_connections 2024-09-15T22:13:24.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 --2- 172.21.15.77:0/808601929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c10eaa0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 --2- 172.21.15.77:0/808601929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 0x7f280c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 --2- 172.21.15.77:0/808601929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 -- 172.21.15.77:0/808601929 >> 172.21.15.77:0/808601929 conn(0x7f280c0fd8f0 msgr2=0x7f280c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 -- 172.21.15.77:0/808601929 shutdown_connections 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.604+0000 7f28128be640 1 -- 172.21.15.77:0/808601929 wait complete. 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.605+0000 7f28128be640 1 Processor -- start 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.605+0000 7f28128be640 1 -- start start 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 0x7f280c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f280c111f80 con 0x7f280c106f20 2024-09-15T22:13:24.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f280c111e00 con 0x7f280c1027a0 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f28128be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f280c112100 con 0x7f280c101df0 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f280bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 0x7f280c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33844/0 (socket says 172.21.15.77:33844) 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 -- 172.21.15.77:0/3914335309 learned_addr learned my addr 172.21.15.77:0/3914335309 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f280bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52180/0 (socket says 172.21.15.77:52180) 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 -- 172.21.15.77:0/3914335309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 msgr2=0x7f280c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 -- 172.21.15.77:0/3914335309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 msgr2=0x7f280c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 0x7f280c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.606+0000 7f2810e5b640 1 -- 172.21.15.77:0/3914335309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c19d350 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f2810e5b640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2804031a20 tx=0x7f2804031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f28097fa640 1 -- 172.21.15.77:0/3914335309 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280404b070 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c19d5e0 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f28097fa640 1 -- 172.21.15.77:0/3914335309 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280403cc80 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f28097fa640 1 -- 172.21.15.77:0/3914335309 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040045b0 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.607+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c074420 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.611+0000 7f28097fa640 1 -- 172.21.15.77:0/3914335309 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f280404e070 con 0x7f280c106f20 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.611+0000 7f2814c97740 1 Processor -- start 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.611+0000 7f2814c97740 1 -- start start 2024-09-15T22:13:24.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad32a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad337940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aad287280 con 0x558aad32ac90 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558aad22eb10 con 0x558aad32a370 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2814c97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558aad283240 con 0x558aad335550 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad32a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33850/0 (socket says 172.21.15.77:33850) 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280bfff640 1 -- 172.21.15.77:0/958335670 learned_addr learned my addr 172.21.15.77:0/958335670 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad337940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f28097fa640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 0x7f27f40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 -- 172.21.15.77:0/958335670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 msgr2=0x558aad337940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad337940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 -- 172.21.15.77:0/958335670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 msgr2=0x558aad335010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 -- 172.21.15.77:0/958335670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aad23b1f0 con 0x558aad32a370 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280bfff640 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f28097fa640 1 -- 172.21.15.77:0/3914335309 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f28040f4b80 con 0x7f280c106f20 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f2810e5b640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 0x7f27f40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.612+0000 7f280b7fe640 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad32a750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f280000afd0 tx=0x7f280000a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58dc520f6daf08ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.613+0000 7f27f27fc640 1 -- 172.21.15.77:0/958335670 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800005020 con 0x558aad32a370 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.613+0000 7f27f27fc640 1 -- 172.21.15.77:0/958335670 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28000047a0 con 0x558aad32a370 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.613+0000 7f27f27fc640 1 -- 172.21.15.77:0/958335670 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280000f910 con 0x558aad32a370 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- 172.21.15.77:0/958335670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 msgr2=0x558aad32a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2810e5b640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 0x7f27f40a5a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28040f1ec0 tx=0x7f2804033040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad32a750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f280000afd0 tx=0x7f280000a680 comp rx=0 tx=0).stop 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- 172.21.15.77:0/958335670 shutdown_connections 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad337940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad335010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 --2- 172.21.15.77:0/958335670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad32a750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- 172.21.15.77:0/958335670 >> 172.21.15.77:0/958335670 conn(0x558aad326900 msgr2=0x558aad326ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- 172.21.15.77:0/958335670 shutdown_connections 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- 172.21.15.77:0/958335670 wait complete. 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.614+0000 7f2814c97740 1 -- start start 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 Processor -- start 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32ac90 0x558aad332990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad332ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aad338120 con 0x558aad335550 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558aad241f10 con 0x558aad32ac90 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558aad3382a0 con 0x558aad32a370 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f2814c97740 10 client.0 _ll_register_callbacks cb 0x558aad20aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32ac90 0x558aad332990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52204/0 (socket says 172.21.15.77:52204) 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 -- 172.21.15.77:0/2507640509 learned_addr learned my addr 172.21.15.77:0/2507640509 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad332ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 -- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32ac90 msgr2=0x558aad332990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32ac90 0x558aad332990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 -- 172.21.15.77:0/2507640509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 msgr2=0x558aad332ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad332ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.615+0000 7f280b7fe640 1 -- 172.21.15.77:0/2507640509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aacff86a0 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f280b7fe640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f280000a820 tx=0x7f280000a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e0f8ce9e5d637c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f27eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280000f8b0 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f280000fa50 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28000047a0 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2800005520 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f2814c97740 10 client.25189 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.616+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558aad343d30 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2800015040 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2800010910 con 0x558aad32a370 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 10 client.25189 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:24.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 10 client.25189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 20 client.25189 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad347470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 4 client.25189 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f28000047a0 con 0x558aad345080 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.617+0000 7f280bfff640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad347470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.618+0000 7f27eb7fe640 10 client.25189 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.618+0000 7f280bfff640 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad347470 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0860528db64df3c server_cookie=cf1040512b9f6f4c in_seq=0 out_seq=0 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.618+0000 7f27eb7fe640 1 -- 172.21.15.77:0/2507640509 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f28000047a0 con 0x558aad345080 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.618+0000 7f27eb7fe640 10 client.25189 handle_command_reply: tid=0 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.618+0000 7f2814c97740 1 client.25189 shutdown 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 msgr2=0x558aad332450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f280000a820 tx=0x7f280000a510 comp rx=0 tx=0).stop 2024-09-15T22:13:24.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 shutdown_connections 2024-09-15T22:13:24.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad347470 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f27eb7fe640 0 client.25189 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:24.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad332ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32ac90 0x558aad332990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 --2- 172.21.15.77:0/2507640509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32a370 0x558aad332450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 >> 172.21.15.77:0/2507640509 conn(0x558aad326900 msgr2=0x558aad31d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 shutdown_connections 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 1 -- 172.21.15.77:0/2507640509 wait complete. 2024-09-15T22:13:24.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.619+0000 7f2814c97740 20 client.25189 trim_cache size 0 max 16384 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 Processor -- start 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 -- start start 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad332d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 0x558aad34cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1acbf0 con 0x558aad335550 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ae730 con 0x558aad32a370 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f2814c97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ad2c0 con 0x558aad32ac90 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33862/0 (socket says 172.21.15.77:33862) 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f280b7fe640 1 -- 172.21.15.77:0/2616988411 learned_addr learned my addr 172.21.15.77:0/2616988411 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.653+0000 7f280bfff640 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 0x558aad34cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 -- 172.21.15.77:0/2616988411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 msgr2=0x558aad332d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad332d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 -- 172.21.15.77:0/2616988411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 msgr2=0x558aad34d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 -- 172.21.15.77:0/2616988411 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aad3479d0 con 0x558aad32ac90 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280b7fe640 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f280bfff640 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 0x558aad34cc00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558aad331ce0 tx=0x7f27fc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c4125c4a14b85f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f27f1ffb640 1 -- 172.21.15.77:0/2616988411 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc005020 con 0x558aad32ac90 2024-09-15T22:13:24.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f27f1ffb640 1 -- 172.21.15.77:0/2616988411 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc0047a0 con 0x558aad32ac90 2024-09-15T22:13:24.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.654+0000 7f27f1ffb640 1 -- 172.21.15.77:0/2616988411 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc026730 con 0x558aad32ac90 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 -- 172.21.15.77:0/2616988411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 msgr2=0x558aad34cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 0x558aad34cc00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558aad331ce0 tx=0x7f27fc00b390 comp rx=0 tx=0).stop 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 -- 172.21.15.77:0/2616988411 shutdown_connections 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad335550 0x558aad34d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad32ac90 0x558aad34cc00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 --2- 172.21.15.77:0/2616988411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad332d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.655+0000 7f2814c97740 1 -- 172.21.15.77:0/2616988411 >> 172.21.15.77:0/2616988411 conn(0x558aad326900 msgr2=0x558aad31d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.656+0000 7f2814c97740 1 -- 172.21.15.77:0/2616988411 shutdown_connections 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.656+0000 7f2814c97740 1 -- 172.21.15.77:0/2616988411 wait complete. 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.656+0000 7f2814c97740 1 -- start start 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.656+0000 7f2814c97740 1 Processor -- start 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad33d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad33a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ae060 con 0x558aad32ac90 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1ad990 con 0x558aad32a370 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1aabd0 con 0x558aad335550 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f2814c97740 10 client.0 _ll_register_callbacks cb 0x7f27fc004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33870/0 (socket says 172.21.15.77:33870) 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 -- 172.21.15.77:0/301170002 learned_addr learned my addr 172.21.15.77:0/301170002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 -- 172.21.15.77:0/301170002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 msgr2=0x558aad33a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280b7fe640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad33a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad33a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 -- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 msgr2=0x558aad33d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad33d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 -- 172.21.15.77:0/301170002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aad33ab50 con 0x558aad32ac90 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280b7fe640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad33a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.657+0000 7f280bfff640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006010 tx=0x7f27fc004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e92c928b35ee456 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f27eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc005020 con 0x558aad32ac90 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f27fc0051c0 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27fc0047a0 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27fc039050 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f2814c97740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558aad3b05a0 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.658+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc0047a0 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f27fc02e020 con 0x558aad32ac90 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 20 client.15807 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad3b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 4 client.15807 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558aad2947f0 con 0x558aad345080 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f280b7fe640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad3b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f280b7fe640 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad3b2c30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9605336e06d48d39 server_cookie=89efa1fbff2b1fe0 in_seq=0 out_seq=0 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.659+0000 7f27eb7fe640 10 client.15807 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.660+0000 7f27eb7fe640 1 -- 172.21.15.77:0/301170002 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558aad2947f0 con 0x558aad345080 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.660+0000 7f27eb7fe640 10 client.15807 handle_command_reply: tid=0 2024-09-15T22:13:24.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.660+0000 7f2814c97740 1 client.15807 shutdown 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 msgr2=0x558aad33dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006010 tx=0x7f27fc004b80 comp rx=0 tx=0).stop 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 shutdown_connections 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558aad345080 0x558aad3b2c30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f27eb7fe640 0 client.15807 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558aad335550 0x558aad33a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aad32ac90 0x558aad33dc10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 --2- 172.21.15.77:0/301170002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558aad32a370 0x558aad33d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 >> 172.21.15.77:0/301170002 conn(0x558aad326900 msgr2=0x558aad32bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 shutdown_connections 2024-09-15T22:13:24.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 1 -- 172.21.15.77:0/301170002 wait complete. 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.661+0000 7f2814c97740 20 client.15807 trim_cache size 0 max 16384 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "807a0296-102d-4f15-9d49-294672719660", 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:13:24.668 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:24.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.662+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 msgr2=0x7f27f40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.662+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 0x7f27f40a5a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28040f1ec0 tx=0x7f2804033040 comp rx=0 tx=0).stop 2024-09-15T22:13:24.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.662+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 msgr2=0x7f280c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.662+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2804031a20 tx=0x7f2804031a50 comp rx=0 tx=0).stop 2024-09-15T22:13:24.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 shutdown_connections 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f27f40a3560 0x7f27f40a5a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f280c106f20 0x7f280c19cc70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f280c1027a0 0x7f280c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 --2- 172.21.15.77:0/3914335309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f280c101df0 0x7f280c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 >> 172.21.15.77:0/3914335309 conn(0x7f280c0fd8f0 msgr2=0x7f280c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 shutdown_connections 2024-09-15T22:13:24.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:24.663+0000 7f28128be640 1 -- 172.21.15.77:0/3914335309 wait complete. 2024-09-15T22:13:25.469 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:26.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 -- 172.21.15.77:0/2434375322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100100 msgr2=0x7f1048100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 --2- 172.21.15.77:0/2434375322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100100 0x7f1048100560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1038009900 tx=0x7f103802a410 comp rx=0 tx=0).stop 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 -- 172.21.15.77:0/2434375322 shutdown_connections 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 --2- 172.21.15.77:0/2434375322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048100aa0 0x7f104810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 --2- 172.21.15.77:0/2434375322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100100 0x7f1048100560 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 --2- 172.21.15.77:0/2434375322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048106b80 0x7f1048106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 -- 172.21.15.77:0/2434375322 >> 172.21.15.77:0/2434375322 conn(0x7f1048077230 msgr2=0x7f10480fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 -- 172.21.15.77:0/2434375322 shutdown_connections 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.322+0000 7f10505d1640 1 -- 172.21.15.77:0/2434375322 wait complete. 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.323+0000 7f10505d1640 1 Processor -- start 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 -- start start 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048100100 0x7f10481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 0x7f10481989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1048112050 con 0x7f1048100aa0 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1048111ed0 con 0x7f1048100100 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f10505d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10481121d0 con 0x7f1048106b80 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f104eb6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f104eb6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52246/0 (socket says 172.21.15.77:52246) 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 0x7f10481989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f104eb6e640 1 -- 172.21.15.77:0/2698596240 learned_addr learned my addr 172.21.15.77:0/2698596240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.324+0000 7f104e36d640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048100100 0x7f10481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 -- 172.21.15.77:0/2698596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048100100 msgr2=0x7f10481984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048100100 0x7f10481984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 -- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 msgr2=0x7f10481989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 0x7f10481989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 -- 172.21.15.77:0/2698596240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104819d450 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104db6c640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 0x7f10481989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f104eb6e640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f104000dac0 tx=0x7f104000df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f103f7fe640 1 -- 172.21.15.77:0/2698596240 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040017070 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f103f7fe640 1 -- 172.21.15.77:0/2698596240 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104000be30 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104819d740 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f103f7fe640 1 -- 172.21.15.77:0/2698596240 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000f890 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.325+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104819db40 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.329+0000 7f10529aa740 1 Processor -- start 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.329+0000 7f103f7fe640 1 -- 172.21.15.77:0/2698596240 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f104000fb70 con 0x7f1048106b80 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.329+0000 7f10529aa740 1 -- start start 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 0x55adeaf56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf65550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf65a90 0x55adeaf67e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55adeaebc2e0 con 0x55adeaf56360 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55adeae63ce0 con 0x55adeaf65a90 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f10529aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adeaeb82a0 con 0x55adeaf56c80 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 0x55adeaf56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 0x55adeaf56740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33906/0 (socket says 172.21.15.77:33906) 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f104db6c640 1 -- 172.21.15.77:0/3474048794 learned_addr learned my addr 172.21.15.77:0/3474048794 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f104e36d640 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf65550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.330+0000 7f103f7fe640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 0x7f10340a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f103f7fe640 1 -- 172.21.15.77:0/2698596240 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f10400d02a0 con 0x7f1048106b80 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104eb6e640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 0x7f10340a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 -- 172.21.15.77:0/3474048794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf65a90 msgr2=0x55adeaf67e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf65a90 0x55adeaf67e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 -- 172.21.15.77:0/3474048794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 msgr2=0x55adeaf56740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 0x55adeaf56740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 -- 172.21.15.77:0/3474048794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeae48440 con 0x55adeaf56c80 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104eb6e640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 0x7f10340a5a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10400cd4d0 tx=0x7f104000b8a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f104e36d640 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf65550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f104400bb70 tx=0x7f104400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1022b5d0ed12d39e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f103cff9640 1 -- 172.21.15.77:0/3474048794 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044010e50 con 0x55adeaf56c80 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.331+0000 7f103cff9640 1 -- 172.21.15.77:0/3474048794 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10440035f0 con 0x55adeaf56c80 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.332+0000 7f103cff9640 1 -- 172.21.15.77:0/3474048794 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044011640 con 0x55adeaf56c80 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.332+0000 7f10529aa740 1 -- 172.21.15.77:0/3474048794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 msgr2=0x55adeaf65550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.332+0000 7f10529aa740 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf65550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f104400bb70 tx=0x7f104400bf70 comp rx=0 tx=0).stop 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 -- 172.21.15.77:0/3474048794 shutdown_connections 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf65a90 0x55adeaf67e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf65550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 --2- 172.21.15.77:0/3474048794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56360 0x55adeaf56740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 -- 172.21.15.77:0/3474048794 >> 172.21.15.77:0/3474048794 conn(0x55adeaf52a30 msgr2=0x55adeaf52e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 -- 172.21.15.77:0/3474048794 shutdown_connections 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 -- 172.21.15.77:0/3474048794 wait complete. 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 -- start start 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.333+0000 7f10529aa740 1 Processor -- start 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf65450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33922/0 (socket says 172.21.15.77:33922) 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 -- 172.21.15.77:0/2672270144 learned_addr learned my addr 172.21.15.77:0/2672270144 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55adeaf68660 con 0x55adeaf65a90 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55adeae77050 con 0x55adeaf56360 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adeaf687e0 con 0x55adeaf56c80 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104e36d640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 -- 172.21.15.77:0/2672270144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 msgr2=0x55adeaf62700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf62700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 -- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 msgr2=0x55adeaf65450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf65450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.334+0000 7f104db6c640 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeac2d3c0 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f10529aa740 10 client.0 _ll_register_callbacks cb 0x55adeaeb7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f104db6c640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1038006010 tx=0x7f1038005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3d88acc6ba780a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f1031ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10440035f0 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f1031ffb640 1 -- 172.21.15.77:0/2672270144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038035040 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f1031ffb640 1 -- 172.21.15.77:0/2672270144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1038004380 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f1031ffb640 1 -- 172.21.15.77:0/2672270144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10380035f0 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f10529aa740 10 client.15819 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.335+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55adeaf73f40 con 0x55adeaf65a90 2024-09-15T22:13:26.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f1031ffb640 1 -- 172.21.15.77:0/2672270144 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1038003790 con 0x55adeaf65a90 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 20 client.15819 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeaf777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 4 client.15819 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adeaec9850 con 0x55adeaf752f0 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.336+0000 7f104e36d640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeaf777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.337+0000 7f1031ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.337+0000 7f104e36d640 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeaf777b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3074df07d1392f server_cookie=b2d623ebc64aeada in_seq=0 out_seq=0 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.337+0000 7f1031ffb640 1 -- 172.21.15.77:0/2672270144 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adeaec9850 con 0x55adeaf752f0 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.337+0000 7f1031ffb640 10 client.15819 handle_command_reply: tid=0 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 client.15819 shutdown 2024-09-15T22:13:26.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 msgr2=0x55adeaf62cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1038006010 tx=0x7f1038005de0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 shutdown_connections 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeaf777b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f1031ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf62cf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf62700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 --2- 172.21.15.77:0/2672270144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf65450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 >> 172.21.15.77:0/2672270144 conn(0x55adeaf52a30 msgr2=0x55adeaf58790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 shutdown_connections 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 1 -- 172.21.15.77:0/2672270144 wait complete. 2024-09-15T22:13:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.338+0000 7f10529aa740 20 client.15819 trim_cache size 0 max 16384 2024-09-15T22:13:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.356+0000 7f10529aa740 1 Processor -- start 2024-09-15T22:13:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.356+0000 7f10529aa740 1 -- start start 2024-09-15T22:13:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf71930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 0x55adeaf62c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104e36d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 0x55adeaf62c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104e36d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 0x55adeaf62c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33928/0 (socket says 172.21.15.77:33928) 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104e36d640 1 -- 172.21.15.77:0/1225812301 learned_addr learned my addr 172.21.15.77:0/1225812301 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf65a90 0x55adeaf7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10481acc50 con 0x55adeaf56c80 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10481ae790 con 0x55adeaf56360 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10481ad320 con 0x55adeaf65a90 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf71930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 -- 172.21.15.77:0/1225812301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf65a90 msgr2=0x55adeaf7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf65a90 0x55adeaf7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 -- 172.21.15.77:0/1225812301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 msgr2=0x55adeaf62c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 0x55adeaf62c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.357+0000 7f104db6c640 1 -- 172.21.15.77:0/1225812301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeaf5d4f0 con 0x55adeaf56360 2024-09-15T22:13:26.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.358+0000 7f104db6c640 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf71930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1038033000 tx=0x7f103803f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a4b793978e32ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.358+0000 7f1033fff640 1 -- 172.21.15.77:0/1225812301 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038035040 con 0x55adeaf56360 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.358+0000 7f1033fff640 1 -- 172.21.15.77:0/1225812301 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038058070 con 0x55adeaf56360 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.358+0000 7f1033fff640 1 -- 172.21.15.77:0/1225812301 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038005030 con 0x55adeaf56360 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 msgr2=0x55adeaf71930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf71930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1038033000 tx=0x7f103803f070 comp rx=0 tx=0).stop 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 shutdown_connections 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf65a90 0x55adeaf7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf56c80 0x55adeaf62c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 --2- 172.21.15.77:0/1225812301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf71930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 >> 172.21.15.77:0/1225812301 conn(0x55adeaf52a30 msgr2=0x55adeaf51b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 shutdown_connections 2024-09-15T22:13:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.359+0000 7f10529aa740 1 -- 172.21.15.77:0/1225812301 wait complete. 2024-09-15T22:13:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.360+0000 7f10529aa740 1 -- start start 2024-09-15T22:13:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.360+0000 7f10529aa740 1 Processor -- start 2024-09-15T22:13:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.360+0000 7f10529aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf6e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f10529aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f10529aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf69b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104db6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf69b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104db6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf6e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104db6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf6e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:57438/0 (socket says 172.21.15.77:57438) 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104e36d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104e36d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52270/0 (socket says 172.21.15.77:52270) 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f104db6c640 1 -- 172.21.15.77:0/2821063850 learned_addr learned my addr 172.21.15.77:0/2821063850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10481ae0c0 con 0x55adeaf65a90 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10481ad9f0 con 0x55adeaf56360 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.361+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10481aac30 con 0x55adeaf56c80 2024-09-15T22:13:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 -- 172.21.15.77:0/2821063850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 msgr2=0x55adeaf695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 -- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 msgr2=0x55adeaf6e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf6e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeaf6a340 con 0x55adeaf65a90 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f10529aa740 10 client.0 _ll_register_callbacks cb 0x7f1038009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1038058070 con 0x55adeaf65a90 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f104db6c640 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf69b10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f10380092a0 tx=0x7f103802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc328e62c7e6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f1031ffb640 10 client.15825 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:26.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f1031ffb640 1 -- 172.21.15.77:0/2821063850 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038035040 con 0x55adeaf65a90 2024-09-15T22:13:26.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.362+0000 7f1031ffb640 1 -- 172.21.15.77:0/2821063850 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1038005030 con 0x55adeaf65a90 2024-09-15T22:13:26.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f1031ffb640 1 -- 172.21.15.77:0/2821063850 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1038005270 con 0x55adeaf65a90 2024-09-15T22:13:26.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:26.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55adeafdfed0 con 0x55adeaf65a90 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f1031ffb640 1 -- 172.21.15.77:0/2821063850 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1038005410 con 0x55adeaf65a90 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 20 client.15825 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeafe2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 4 client.15825 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adeaec3d60 con 0x55adeaf752f0 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.363+0000 7f104e36d640 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeafe2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f1031ffb640 10 client.15825 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f104e36d640 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeafe2630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dca4e27027ae942 server_cookie=2f5af10a39a27ec3 in_seq=0 out_seq=0 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f1031ffb640 1 -- 172.21.15.77:0/2821063850 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55adeaec3d60 con 0x55adeaf752f0 2024-09-15T22:13:26.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f1031ffb640 10 client.15825 handle_command_reply: tid=0 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f10529aa740 1 client.15825 shutdown 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 msgr2=0x55adeaf69b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf69b10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f10380092a0 tx=0x7f103802a410 comp rx=0 tx=0).stop 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 shutdown_connections 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.364+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55adeaf752f0 0x55adeafe2630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55adeaf65a90 0x55adeaf69b10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adeaf56c80 0x55adeaf695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 --2- 172.21.15.77:0/2821063850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55adeaf56360 0x55adeaf6e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 >> 172.21.15.77:0/2821063850 conn(0x55adeaf52a30 msgr2=0x55adeaf63f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f1031ffb640 0 client.15825 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 shutdown_connections 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 1 -- 172.21.15.77:0/2821063850 wait complete. 2024-09-15T22:13:26.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.365+0000 7f10529aa740 20 client.15825 trim_cache size 0 max 16384 2024-09-15T22:13:26.370 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:13:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:13:26.370 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:26.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 msgr2=0x7f10340a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 0x7f10340a5a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10400cd4d0 tx=0x7f104000b8a0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 msgr2=0x7f104819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f104000dac0 tx=0x7f104000df80 comp rx=0 tx=0).stop 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 shutdown_connections 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10340a3560 0x7f10340a5a20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1048106b80 0x7f104819cd70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1048100aa0 0x7f10481989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 --2- 172.21.15.77:0/2698596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1048100100 0x7f10481984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 >> 172.21.15.77:0/2698596240 conn(0x7f1048077230 msgr2=0x7f10480fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 shutdown_connections 2024-09-15T22:13:26.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:26.366+0000 7f10505d1640 1 -- 172.21.15.77:0/2698596240 wait complete. 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:26 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:26 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:26 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:13:27.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:13:27.073 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:27.353 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 -- 172.21.15.77:0/1232579073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 msgr2=0x7f615010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 --2- 172.21.15.77:0/1232579073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615010da40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6148009ca0 tx=0x7f614802a970 comp rx=0 tx=0).stop 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 -- 172.21.15.77:0/1232579073 shutdown_connections 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 --2- 172.21.15.77:0/1232579073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615010da40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 --2- 172.21.15.77:0/1232579073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 0x7f61501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 --2- 172.21.15.77:0/1232579073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f6150101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 -- 172.21.15.77:0/1232579073 >> 172.21.15.77:0/1232579073 conn(0x7f61500fc7d0 msgr2=0x7f61500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 -- 172.21.15.77:0/1232579073 shutdown_connections 2024-09-15T22:13:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.893+0000 7f6157705640 1 -- 172.21.15.77:0/1232579073 wait complete. 2024-09-15T22:13:27.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.894+0000 7f6157705640 1 Processor -- start 2024-09-15T22:13:27.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 -- start start 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f61501a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 0x7f61501a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f615010fec0 con 0x7f6150102ef0 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f615010fd40 con 0x7f6150100e20 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6157705640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6150110040 con 0x7f6150102570 2024-09-15T22:13:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6155ca2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6155ca2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33964/0 (socket says 172.21.15.77:33964) 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6155ca2640 1 -- 172.21.15.77:0/2830822433 learned_addr learned my addr 172.21.15.77:0/2830822433 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f6154ca0640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 0x7f61501a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.895+0000 7f61554a1640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f61501a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 -- 172.21.15.77:0/2830822433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 msgr2=0x7f61501a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 0x7f61501a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 -- 172.21.15.77:0/2830822433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 msgr2=0x7f61501a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f61501a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 -- 172.21.15.77:0/2830822433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615019eff0 con 0x7f6150102ef0 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f61554a1640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f61501a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f6155ca2640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6148005e30 tx=0x7f6148030880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148034030 con 0x7f6150102ef0 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.896+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614802f8a0 con 0x7f6150102ef0 2024-09-15T22:13:27.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.897+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148039420 con 0x7f6150102ef0 2024-09-15T22:13:27.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.897+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615019f280 con 0x7f6150102ef0 2024-09-15T22:13:27.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.897+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6150195fe0 con 0x7f6150102ef0 2024-09-15T22:13:27.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.898+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61501021b0 con 0x7f6150102ef0 2024-09-15T22:13:27.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.900+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f6148037020 con 0x7f6150102ef0 2024-09-15T22:13:27.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.901+0000 7f61467fc640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 0x7f613c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.901+0000 7f61554a1640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 0x7f613c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.901+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61480f12b0 con 0x7f6150102ef0 2024-09-15T22:13:27.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.902+0000 7f61554a1640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 0x7f613c0a5aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f615019fa00 tx=0x7f614c005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:27.907+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61480bd7a0 con 0x7f6150102ef0 2024-09-15T22:13:28.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.327+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f615019fe20 con 0x7f6150102ef0 2024-09-15T22:13:28.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.328+0000 7f61467fc640 1 -- 172.21.15.77:0/2830822433 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f615019fe20 con 0x7f6150102ef0 2024-09-15T22:13:28.330 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:13:28.330 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:13:28.330 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 msgr2=0x7f613c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 0x7f613c0a5aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f615019fa00 tx=0x7f614c005f90 comp rx=0 tx=0).stop 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 msgr2=0x7f615019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6148005e30 tx=0x7f6148030880 comp rx=0 tx=0).stop 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 shutdown_connections 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f613c0a35e0 0x7f613c0a5aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6150102ef0 0x7f615019e8c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6150102570 0x7f61501a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:28.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 --2- 172.21.15.77:0/2830822433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6150100e20 0x7f61501a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:28.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.331+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 >> 172.21.15.77:0/2830822433 conn(0x7f61500fc7d0 msgr2=0x7f61501068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:28.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.332+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 shutdown_connections 2024-09-15T22:13:28.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:28.332+0000 7f6157705640 1 -- 172.21.15.77:0/2830822433 wait complete. 2024-09-15T22:13:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:28 smithi077 ceph-mon[30980]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:13:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:28 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2830822433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:28 smithi196 ceph-mon[35109]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:13:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:28 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2830822433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:28 smithi117 ceph-mon[35021]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-15T22:13:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:28 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2830822433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:29.044 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:13:29.313 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:29.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 -- 172.21.15.77:0/1209566627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 msgr2=0x7faed4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 --2- 172.21.15.77:0/1209566627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4101200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faec4009890 tx=0x7faec402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 -- 172.21.15.77:0/1209566627 shutdown_connections 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 --2- 172.21.15.77:0/1209566627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 --2- 172.21.15.77:0/1209566627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faed4102570 0x7faed41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 --2- 172.21.15.77:0/1209566627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4101200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 -- 172.21.15.77:0/1209566627 >> 172.21.15.77:0/1209566627 conn(0x7faed40fc7d0 msgr2=0x7faed40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 -- 172.21.15.77:0/1209566627 shutdown_connections 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.867+0000 7faeda1ea640 1 -- 172.21.15.77:0/1209566627 wait complete. 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 Processor -- start 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 -- start start 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faed4102570 0x7faed4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faed4110f50 con 0x7faed4102570 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faed4110dd0 con 0x7faed4100e20 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.868+0000 7faeda1ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed41110d0 con 0x7faed4102ef0 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42880/0 (socket says 172.21.15.77:42880) 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41774/0 (socket says 172.21.15.77:41774) 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 -- 172.21.15.77:0/2432385936 learned_addr learned my addr 172.21.15.77:0/2432385936 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 -- 172.21.15.77:0/2432385936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 msgr2=0x7faed4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 -- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faed4102570 msgr2=0x7faed4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faed4102570 0x7faed4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 -- 172.21.15.77:0/2432385936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed419c3e0 con 0x7faed4102ef0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.869+0000 7faed3fff640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faecc009fd0 tx=0x7faecc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.870+0000 7faed17fa640 1 -- 172.21.15.77:0/2432385936 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc004b20 con 0x7faed4102ef0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.870+0000 7faed17fa640 1 -- 172.21.15.77:0/2432385936 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faecc003e50 con 0x7faed4102ef0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.870+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed419c6d0 con 0x7faed4102ef0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.870+0000 7faed17fa640 1 -- 172.21.15.77:0/2432385936 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00b630 con 0x7faed4102ef0 2024-09-15T22:13:29.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.870+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed4069f70 con 0x7faed4102ef0 2024-09-15T22:13:29.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.873+0000 7faedc5c3740 1 Processor -- start 2024-09-15T22:13:29.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faed17fa640 1 -- 172.21.15.77:0/2432385936 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7faecc01d020 con 0x7faed4102ef0 2024-09-15T22:13:29.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 -- start start 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5591bc9212f0 con 0x5591bc9c5440 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591bc8c8cf0 con 0x5591bc9c5df0 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.874+0000 7faedc5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5591bc91d2b0 con 0x5591bc9caa40 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49320/0 (socket says 172.21.15.77:49320) 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 -- 172.21.15.77:0/1610972326 learned_addr learned my addr 172.21.15.77:0/1610972326 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faed37fe640 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 -- 172.21.15.77:0/1610972326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 msgr2=0x5591bc9ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faed17fa640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 0x7faec00a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 -- 172.21.15.77:0/1610972326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 msgr2=0x5591bc9c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.875+0000 7faecbfff640 1 -- 172.21.15.77:0/1610972326 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bc8ad3a0 con 0x5591bc9caa40 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faecbfff640 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9caef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fc0 tx=0x7faebc005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a42682b52811e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faec9ffb640 1 -- 172.21.15.77:0/1610972326 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc005020 con 0x5591bc9caa40 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faec9ffb640 1 -- 172.21.15.77:0/1610972326 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc0047a0 con 0x5591bc9caa40 2024-09-15T22:13:29.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faec9ffb640 1 -- 172.21.15.77:0/1610972326 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc00f740 con 0x5591bc9caa40 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faed17fa640 1 -- 172.21.15.77:0/2432385936 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7faecc0d0340 con 0x7faed4102ef0 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.876+0000 7faed3fff640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 0x7faec00a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 -- 172.21.15.77:0/1610972326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 msgr2=0x5591bc9caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9caef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faebc009fc0 tx=0x7faebc005cc0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faed3fff640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 0x7faec00a5aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faecc0cd570 tx=0x7faecc0d05c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 -- 172.21.15.77:0/1610972326 shutdown_connections 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9caef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 --2- 172.21.15.77:0/1610972326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 -- 172.21.15.77:0/1610972326 >> 172.21.15.77:0/1610972326 conn(0x5591bc984310 msgr2=0x5591bc9b76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 -- 172.21.15.77:0/1610972326 shutdown_connections 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.877+0000 7faedc5c3740 1 -- 172.21.15.77:0/1610972326 wait complete. 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.878+0000 7faedc5c3740 1 -- start start 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.878+0000 7faedc5c3740 1 Processor -- start 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5591bc9d18d0 con 0x5591bc9c5440 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591bc8dc060 con 0x5591bc9c5df0 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5591bc9d1a50 con 0x5591bc9caa40 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faedc5c3740 10 client.0 _ll_register_callbacks cb 0x5591bc62f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:29.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41790/0 (socket says 172.21.15.77:41790) 2024-09-15T22:13:29.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 -- 172.21.15.77:0/2411309672 learned_addr learned my addr 172.21.15.77:0/2411309672 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:29.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 -- 172.21.15.77:0/2411309672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 msgr2=0x5591bc9ce640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:29.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9ce640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.879+0000 7faed37fe640 1 -- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 msgr2=0x5591bc9d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faed37fe640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faed37fe640 1 -- 172.21.15.77:0/2411309672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bc6923c0 con 0x5591bc9c5df0 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faecbfff640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faed37fe640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faec4009890 tx=0x7faec40303d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423dd4f8c9ded61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faeb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4037040 con 0x5591bc9c5df0 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faebc0047a0 con 0x5591bc9c5df0 2024-09-15T22:13:29.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.880+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7faec40035f0 con 0x5591bc9c5df0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.881+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faec404c050 con 0x5591bc9c5df0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.881+0000 7faedc5c3740 10 client.25259 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.881+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591bc9dd310 con 0x5591bc9c5df0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.881+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc0047a0 con 0x5591bc9c5df0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faec4030a80 con 0x5591bc9c5df0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 10 client.25259 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 10 client.25259 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 20 client.25259 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9debe0 0x5591bc9e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 4 client.25259 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591bc92e860 con 0x5591bc9debe0 2024-09-15T22:13:29.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faecbfff640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9debe0 0x5591bc9e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faecbfff640 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9debe0 0x5591bc9e1060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec63ed377749bd28 server_cookie=906db10d61517618 in_seq=0 out_seq=0 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.882+0000 7faeb2ffd640 10 client.25259 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.883+0000 7faeb2ffd640 1 -- 172.21.15.77:0/2411309672 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591bc92e860 con 0x5591bc9debe0 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.883+0000 7faeb2ffd640 10 client.25259 handle_command_reply: tid=0 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.883+0000 7faedc5c3740 1 client.25259 shutdown 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.884+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 msgr2=0x5591bc9ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.884+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faec4009890 tx=0x7faec40303d0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 shutdown_connections 2024-09-15T22:13:29.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9debe0 0x5591bc9e1060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9caa40 0x5591bc9ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9c5df0 0x5591bc9ce100 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 --2- 172.21.15.77:0/2411309672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 >> 172.21.15.77:0/2411309672 conn(0x5591bc984310 msgr2=0x5591bc9c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faeb2ffd640 0 client.25259 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 shutdown_connections 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 1 -- 172.21.15.77:0/2411309672 wait complete. 2024-09-15T22:13:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.888+0000 7faedc5c3740 20 client.25259 trim_cache size 0 max 16384 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.913+0000 7faedc5c3740 1 Processor -- start 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.913+0000 7faedc5c3740 1 -- start start 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faed41acc80 con 0x5591bc9c5440 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faed41ae7c0 con 0x5591bc9caa40 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faedc5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed41ad350 con 0x5591bc9c5df0 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49348/0 (socket says 172.21.15.77:49348) 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faecbfff640 1 -- 172.21.15.77:0/621653697 learned_addr learned my addr 172.21.15.77:0/621653697 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:29.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faed37fe640 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faed37fe640 1 -- 172.21.15.77:0/621653697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 msgr2=0x5591bc9c4790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:29.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faed37fe640 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9c4790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faed37fe640 1 -- 172.21.15.77:0/621653697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 msgr2=0x5591bc9dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.914+0000 7faed37fe640 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faed37fe640 1 -- 172.21.15.77:0/621653697 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bc9d11c0 con 0x5591bc9c5df0 2024-09-15T22:13:29.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faecbfff640 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:29.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faed37fe640 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9c4250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faec402a9e0 tx=0x7faec4005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74317a5ce819128 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faec97fa640 1 -- 172.21.15.77:0/621653697 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4037070 con 0x5591bc9c5df0 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faec97fa640 1 -- 172.21.15.77:0/621653697 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec403de20 con 0x5591bc9c5df0 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.915+0000 7faec97fa640 1 -- 172.21.15.77:0/621653697 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4005490 con 0x5591bc9c5df0 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 -- 172.21.15.77:0/621653697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 msgr2=0x5591bc9c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9c4250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faec402a9e0 tx=0x7faec4005980 comp rx=0 tx=0).stop 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 -- 172.21.15.77:0/621653697 shutdown_connections 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9c4250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 --2- 172.21.15.77:0/621653697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 -- 172.21.15.77:0/621653697 >> 172.21.15.77:0/621653697 conn(0x5591bc984310 msgr2=0x5591bc9cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 -- 172.21.15.77:0/621653697 shutdown_connections 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.916+0000 7faedc5c3740 1 -- 172.21.15.77:0/621653697 wait complete. 2024-09-15T22:13:29.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 -- start start 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 Processor -- start 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9d74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faed41ae0f0 con 0x5591bc9c5440 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faed41ada20 con 0x5591bc9caa40 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faed41aac60 con 0x5591bc9c5df0 2024-09-15T22:13:29.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faedc5c3740 10 client.0 _ll_register_callbacks cb 0x7faebc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:29.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.917+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49360/0 (socket says 172.21.15.77:49360) 2024-09-15T22:13:29.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 -- 172.21.15.77:0/3182112558 learned_addr learned my addr 172.21.15.77:0/3182112558 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faed37fe640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9d74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 -- 172.21.15.77:0/3182112558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 msgr2=0x5591bc9d74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9d74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 -- 172.21.15.77:0/3182112558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 msgr2=0x5591bc9e5d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9e5d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 -- 172.21.15.77:0/3182112558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bc9c5b90 con 0x5591bc9c5440 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faecbfff640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faebc019040 tx=0x7faebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86308a55a0d41267 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.918+0000 7faeb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc004400 con 0x5591bc9c5440 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7faebc0035f0 con 0x5591bc9c5440 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec403de20 con 0x5591bc9c5440 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faebc033050 con 0x5591bc9c5440 2024-09-15T22:13:29.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faedc5c3740 10 client.15846 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:29.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591bc9c3440 con 0x5591bc9c5440 2024-09-15T22:13:29.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec403de20 con 0x5591bc9c5440 2024-09-15T22:13:29.926 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.919+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faebc027830 con 0x5591bc9c5440 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 10 client.15846 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 10 client.15846 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 20 client.15846 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9c3790 0x5591bc9c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 4 client.15846 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591bc928d70 con 0x5591bc9c3790 2024-09-15T22:13:29.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faed37fe640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9c3790 0x5591bc9c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.920+0000 7faeb2ffd640 10 client.15846 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faed37fe640 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9c3790 0x5591bc9c3ba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73e174e65dff7d3d server_cookie=b67a70bb67175111 in_seq=0 out_seq=0 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faeb2ffd640 1 -- 172.21.15.77:0/3182112558 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591bc928d70 con 0x5591bc9c3790 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faeb2ffd640 10 client.15846 handle_command_reply: tid=0 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 client.15846 shutdown 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 msgr2=0x5591bc9d6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faebc019040 tx=0x7faebc0046c0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 shutdown_connections 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5591bc9c3790 0x5591bc9c3ba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faeb2ffd640 0 client.15846 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591bc9caa40 0x5591bc9e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5591bc9c5df0 0x5591bc9d74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 --2- 172.21.15.77:0/3182112558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5591bc9c5440 0x5591bc9d6f70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.921+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 >> 172.21.15.77:0/3182112558 conn(0x5591bc984310 msgr2=0x5591bc9c4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 shutdown_connections 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faedc5c3740 1 -- 172.21.15.77:0/3182112558 wait complete. 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faedc5c3740 20 client.15846 trim_cache size 0 max 16384 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 msgr2=0x7faec00a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 0x7faec00a5aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faecc0cd570 tx=0x7faecc0d05c0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 msgr2=0x7faed419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.922+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faecc009fd0 tx=0x7faecc00c800 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 shutdown_connections 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faec00a35e0 0x7faec00a5aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed4102ef0 0x7faed419bd00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faed4102570 0x7faed4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 --2- 172.21.15.77:0/2432385936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faed4100e20 0x7faed4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 >> 172.21.15.77:0/2432385936 conn(0x7faed40fc7d0 msgr2=0x7faed4104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 shutdown_connections 2024-09-15T22:13:29.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:29.923+0000 7faeda1ea640 1 -- 172.21.15.77:0/2432385936 wait complete. 2024-09-15T22:13:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:30 smithi196 ceph-mon[35109]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:30 smithi117 ceph-mon[35021]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:30.617 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:30 smithi077 ceph-mon[30980]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T22:13:30.618 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:13:30.962 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:31.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 -- 172.21.15.77:0/982993082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 msgr2=0x7fd4f810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/982993082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f810ea20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009be0 tx=0x7fd4f002a6c0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 -- 172.21.15.77:0/982993082 shutdown_connections 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/982993082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f810ea20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/982993082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/982993082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 0x7fd4f8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.548+0000 7fd4fcb71640 1 -- 172.21.15.77:0/982993082 >> 172.21.15.77:0/982993082 conn(0x7fd4f8077230 msgr2=0x7fd4f80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.549+0000 7fd4fcb71640 1 -- 172.21.15.77:0/982993082 shutdown_connections 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.549+0000 7fd4fcb71640 1 -- 172.21.15.77:0/982993082 wait complete. 2024-09-15T22:13:31.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.550+0000 7fd4fcb71640 1 Processor -- start 2024-09-15T22:13:31.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.550+0000 7fd4fcb71640 1 -- start start 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.550+0000 7fd4fcb71640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.550+0000 7fd4fcb71640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49386/0 (socket says 172.21.15.77:49386) 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4fcb71640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 0x7fd4f819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41804/0 (socket says 172.21.15.77:41804) 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810fe10 con 0x7fd4f8100100 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4f659c640 1 -- 172.21.15.77:0/4045956895 learned_addr learned my addr 172.21.15.77:0/4045956895 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810fc90 con 0x7fd4f8100aa0 2024-09-15T22:13:31.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.551+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810ff90 con 0x7fd4f8106b80 2024-09-15T22:13:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 0x7fd4f819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 -- 172.21.15.77:0/4045956895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 msgr2=0x7fd4f819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 0x7fd4f819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 -- 172.21.15.77:0/4045956895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 msgr2=0x7fd4f8196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f819b1e0 con 0x7fd4f8100100 2024-09-15T22:13:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.552+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009be0 tx=0x7fd4e80071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.553+0000 7fd4e77fe640 1 -- 172.21.15.77:0/4045956895 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8007e00 con 0x7fd4f8100100 2024-09-15T22:13:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.553+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f819b4d0 con 0x7fd4f8100100 2024-09-15T22:13:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.553+0000 7fd4e77fe640 1 -- 172.21.15.77:0/4045956895 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e801e070 con 0x7fd4f8100100 2024-09-15T22:13:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.553+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a2db0 con 0x7fd4f8100100 2024-09-15T22:13:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.556+0000 7fd4e77fe640 1 -- 172.21.15.77:0/4045956895 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8019870 con 0x7fd4f8100100 2024-09-15T22:13:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.556+0000 7fd4fef4a740 1 Processor -- start 2024-09-15T22:13:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.557+0000 7fd4e77fe640 1 -- 172.21.15.77:0/4045956895 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fd4e8019ad0 con 0x7fd4f8100100 2024-09-15T22:13:31.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.557+0000 7fd4fef4a740 1 -- start start 2024-09-15T22:13:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4e77fe640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 0x7fd4e00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b496730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b3ef2f0 con 0x55dc8b48ab10 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b396cf0 con 0x55dc8b494340 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b3eb2b0 con 0x55dc8b4895a0 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4e77fe640 1 -- 172.21.15.77:0/4045956895 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd4e809cb10 con 0x7fd4f8100100 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.558+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 0x7fd4e00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49388/0 (socket says 172.21.15.77:49388) 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 -- 172.21.15.77:0/2133402700 learned_addr learned my addr 172.21.15.77:0/2133402700 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:31.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 -- 172.21.15.77:0/2133402700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 msgr2=0x55dc8b4899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 -- 172.21.15.77:0/2133402700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 msgr2=0x55dc8b496730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b496730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.559+0000 7fd4f659c640 1 -- 172.21.15.77:0/2133402700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8b37b3a0 con 0x55dc8b48ab10 2024-09-15T22:13:31.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.560+0000 7fd4f659c640 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800f760 tx=0x7fd4e809fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fb7dc3d80eb157 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.560+0000 7fd4e4ff9640 1 -- 172.21.15.77:0/2133402700 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a4850 con 0x55dc8b48ab10 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.560+0000 7fd4e4ff9640 1 -- 172.21.15.77:0/2133402700 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e809e7a0 con 0x55dc8b48ab10 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.560+0000 7fd4e4ff9640 1 -- 172.21.15.77:0/2133402700 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e809fb50 con 0x55dc8b48ab10 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.561+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2133402700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 msgr2=0x55dc8b489060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.561+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800f760 tx=0x7fd4e809fec0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.561+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 0x7fd4e00a5a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0004af0 tx=0x7fd4f0009280 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2133402700 shutdown_connections 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b496730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2133402700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b489060 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2133402700 >> 172.21.15.77:0/2133402700 conn(0x55dc8b452310 msgr2=0x55dc8b485c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2133402700 shutdown_connections 2024-09-15T22:13:31.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2133402700 wait complete. 2024-09-15T22:13:31.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 -- start start 2024-09-15T22:13:31.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.562+0000 7fd4fef4a740 1 Processor -- start 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b493d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b496f80 con 0x55dc8b48ab10 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b3aa060 con 0x55dc8b494340 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dc8b497100 con 0x55dc8b4895a0 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4fef4a740 10 client.0 _ll_register_callbacks cb 0x55dc8b0fd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49394/0 (socket says 172.21.15.77:49394) 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/4185876398 learned_addr learned my addr 172.21.15.77:0/4185876398 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.563+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b493d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 msgr2=0x55dc8b493d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b493d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/4185876398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 msgr2=0x55dc8b4917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/4185876398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8b1603c0 con 0x55dc8b48ab10 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b493d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec012f70 tx=0x7fd4ec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19bf7eb46f64203 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.564+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00fdc0 con 0x55dc8b48ab10 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd4ec0120d0 con 0x55dc8b48ab10 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e809e7a0 con 0x55dc8b48ab10 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec0215f0 con 0x55dc8b48ab10 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4fef4a740 10 client.15858 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:31.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dc8b4ab350 con 0x55dc8b48ab10 2024-09-15T22:13:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.565+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e809e7a0 con 0x55dc8b48ab10 2024-09-15T22:13:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd4ec010430 con 0x55dc8b48ab10 2024-09-15T22:13:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 10 client.15858 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 10 client.15858 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:31.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 20 client.15858 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b4ac810 0x55dc8b4aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 4 client.15858 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8b3fc860 con 0x55dc8b4ac810 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b4ac810 0x55dc8b4aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.566+0000 7fd4ddffb640 10 client.15858 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.567+0000 7fd4f659c640 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b4ac810 0x55dc8b4aec80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b503e934537a0d8b server_cookie=1ab17716873d4d09 in_seq=0 out_seq=0 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4ddffb640 1 -- 172.21.15.77:0/4185876398 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc8b3fc860 con 0x55dc8b4ac810 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4ddffb640 10 client.15858 handle_command_reply: tid=0 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 client.15858 shutdown 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 msgr2=0x55dc8b491040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec012f70 tx=0x7fd4ec00f790 comp rx=0 tx=0).stop 2024-09-15T22:13:31.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 shutdown_connections 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b4ac810 0x55dc8b4aec80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b48ab10 0x55dc8b491040 secure :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec012f70 tx=0x7fd4ec00f790 comp rx=0 tx=0).stop 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/4185876398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b493d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 >> 172.21.15.77:0/4185876398 conn(0x55dc8b452310 msgr2=0x55dc8b48b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.568+0000 7fd4ddffb640 0 client.15858 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.569+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 shutdown_connections 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.569+0000 7fd4fef4a740 1 -- 172.21.15.77:0/4185876398 wait complete. 2024-09-15T22:13:31.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.569+0000 7fd4fef4a740 20 client.15858 trim_cache size 0 max 16384 2024-09-15T22:13:31.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.594+0000 7fd4fef4a740 1 Processor -- start 2024-09-15T22:13:31.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.595+0000 7fd4fef4a740 1 -- start start 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.595+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.595+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 0x55dc8b480ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.595+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 0x55dc8b480ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 0x55dc8b480ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42936/0 (socket says 172.21.15.77:42936) 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41808/0 (socket says 172.21.15.77:41808) 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49398/0 (socket says 172.21.15.77:49398) 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/2389202634 learned_addr learned my addr 172.21.15.77:0/2389202634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81acd40 con 0x55dc8b492fb0 2024-09-15T22:13:31.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ae880 con 0x55dc8b4895a0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.596+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ad410 con 0x55dc8b494340 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 -- 172.21.15.77:0/2389202634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 msgr2=0x55dc8b480ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 0x55dc8b480ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 -- 172.21.15.77:0/2389202634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 msgr2=0x55dc8b48a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 -- 172.21.15.77:0/2389202634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8b491b40 con 0x55dc8b492fb0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f659c640 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4f6d9d640 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002abe0 tx=0x7fd4f003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2684b6d70fc958 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4dffff640 1 -- 172.21.15.77:0/2389202634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0036050 con 0x55dc8b492fb0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4dffff640 1 -- 172.21.15.77:0/2389202634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f0044040 con 0x55dc8b492fb0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.597+0000 7fd4dffff640 1 -- 172.21.15.77:0/2389202634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00316a0 con 0x55dc8b492fb0 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.598+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 msgr2=0x55dc8b4b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.598+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002abe0 tx=0x7fd4f003e000 comp rx=0 tx=0).stop 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 shutdown_connections 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4b4590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b494340 0x55dc8b480ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/2389202634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b4895a0 0x55dc8b48a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 >> 172.21.15.77:0/2389202634 conn(0x55dc8b452310 msgr2=0x55dc8b4928a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 shutdown_connections 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 -- 172.21.15.77:0/2389202634 wait complete. 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 -- start start 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.599+0000 7fd4fef4a740 1 Processor -- start 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4fef4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4a29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42950/0 (socket says 172.21.15.77:42950) 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49414/0 (socket says 172.21.15.77:49414) 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4f659c640 1 -- 172.21.15.77:0/1114887871 learned_addr learned my addr 172.21.15.77:0/1114887871 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.600+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ae1b0 con 0x55dc8b492fb0 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81adae0 con 0x55dc8b494340 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81aad20 con 0x55dc8b4895a0 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 msgr2=0x55dc8b4a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/1114887871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 msgr2=0x55dc8b4a3440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:31.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4f5d9b640 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8b5165a0 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4fef4a740 10 client.0 _ll_register_callbacks cb 0x7fd4ec012bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.601+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f0044040 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4f5d9b640 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec010a40 tx=0x7fd4ec0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b1990784a56da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4ddffb640 10 client.15864 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4ddffb640 1 -- 172.21.15.77:0/1114887871 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec02e020 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4ddffb640 1 -- 172.21.15.77:0/1114887871 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd4ec021200 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4ddffb640 1 -- 172.21.15.77:0/1114887871 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec010ac0 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4fef4a740 10 client.15864 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.602+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55dc8b5175d0 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4ddffb640 1 -- 172.21.15.77:0/1114887871 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd4ec031020 con 0x55dc8b492fb0 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 10 client.15864 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:31.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 20 client.15864 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b517cd0 0x55dc8b51a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 4 client.15864 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc8b3f6d70 con 0x55dc8b517cd0 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.603+0000 7fd4f659c640 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b517cd0 0x55dc8b51a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4ddffb640 10 client.15864 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4f659c640 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b517cd0 0x55dc8b51a190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d927948a068e118 server_cookie=720dcd9108305c26 in_seq=0 out_seq=0 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4ddffb640 1 -- 172.21.15.77:0/1114887871 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc8b3f6d70 con 0x55dc8b517cd0 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4ddffb640 10 client.15864 handle_command_reply: tid=0 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 client.15864 shutdown 2024-09-15T22:13:31.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 msgr2=0x55dc8b4a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec010a40 tx=0x7fd4ec0106e0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 shutdown_connections 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55dc8b517cd0 0x55dc8b51a190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc8b494340 0x55dc8b4a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc8b492fb0 0x55dc8b4a2f00 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec010a40 tx=0x7fd4ec0106e0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4ddffb640 0 client.15864 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 --2- 172.21.15.77:0/1114887871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dc8b4895a0 0x55dc8b4a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.604+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 >> 172.21.15.77:0/1114887871 conn(0x55dc8b452310 msgr2=0x55dc8b480a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.605+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 shutdown_connections 2024-09-15T22:13:31.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.606+0000 7fd4fef4a740 1 -- 172.21.15.77:0/1114887871 wait complete. 2024-09-15T22:13:31.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.606+0000 7fd4fef4a740 20 client.15864 trim_cache size 0 max 16384 2024-09-15T22:13:31.701 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.606+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 msgr2=0x7fd4e00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.606+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 0x7fd4e00a5a20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0004af0 tx=0x7fd4f0009280 comp rx=0 tx=0).stop 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 msgr2=0x7fd4f8196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009be0 tx=0x7fd4e80071c0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 shutdown_connections 2024-09-15T22:13:31.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd4e00a3560 0x7fd4e00a5a20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4f8106b80 0x7fd4f819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4f8100aa0 0x7fd4f8196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.607+0000 7fd4fcb71640 1 --2- 172.21.15.77:0/4045956895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8100100 0x7fd4f8196230 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.608+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 >> 172.21.15.77:0/4045956895 conn(0x7fd4f8077230 msgr2=0x7fd4f80fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.608+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 shutdown_connections 2024-09-15T22:13:31.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:31.608+0000 7fd4fcb71640 1 -- 172.21.15.77:0/4045956895 wait complete. 2024-09-15T22:13:32.414 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:13:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:32 smithi196 ceph-mon[35109]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:32 smithi117 ceph-mon[35021]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:32.814 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:32 smithi077 ceph-mon[30980]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-15T22:13:33.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.197+0000 7fd36cea9640 1 -- 172.21.15.77:0/3408019804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd368102d30 msgr2=0x7fd368103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.197+0000 7fd36cea9640 1 --2- 172.21.15.77:0/3408019804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd368102d30 0x7fd368103110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd358009930 tx=0x7fd35802a420 comp rx=0 tx=0).stop 2024-09-15T22:13:33.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 -- 172.21.15.77:0/3408019804 shutdown_connections 2024-09-15T22:13:33.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 --2- 172.21.15.77:0/3408019804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681082a0 0x7fd36810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 --2- 172.21.15.77:0/3408019804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3681036e0 0x7fd368107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 --2- 172.21.15.77:0/3408019804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd368102d30 0x7fd368103110 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 -- 172.21.15.77:0/3408019804 >> 172.21.15.77:0/3408019804 conn(0x7fd3680fc7d0 msgr2=0x7fd3680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 -- 172.21.15.77:0/3408019804 shutdown_connections 2024-09-15T22:13:33.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.198+0000 7fd36cea9640 1 -- 172.21.15.77:0/3408019804 wait complete. 2024-09-15T22:13:33.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.199+0000 7fd36cea9640 1 Processor -- start 2024-09-15T22:13:33.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.199+0000 7fd36cea9640 1 -- start start 2024-09-15T22:13:33.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368102d30 0x7fd368198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681036e0 0x7fd368198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681120a0 con 0x7fd3681082a0 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd368111f20 con 0x7fd368102d30 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36cea9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd368112220 con 0x7fd3681036e0 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49432/0 (socket says 172.21.15.77:49432) 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 -- 172.21.15.77:0/1153764251 learned_addr learned my addr 172.21.15.77:0/1153764251 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd36659c640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368102d30 0x7fd368198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd365d9b640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681036e0 0x7fd368198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 -- 172.21.15.77:0/1153764251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681036e0 msgr2=0x7fd368198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681036e0 0x7fd368198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 -- 172.21.15.77:0/1153764251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368102d30 msgr2=0x7fd368198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368102d30 0x7fd368198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 -- 172.21.15.77:0/1153764251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36819d5e0 con 0x7fd3681082a0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.200+0000 7fd366d9d640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd36000eaa0 tx=0x7fd36000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.201+0000 7fd3577fe640 1 -- 172.21.15.77:0/1153764251 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36000cdb0 con 0x7fd3681082a0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.201+0000 7fd3577fe640 1 -- 172.21.15.77:0/1153764251 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd360005020 con 0x7fd3681082a0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.201+0000 7fd3577fe640 1 -- 172.21.15.77:0/1153764251 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3600116b0 con 0x7fd3681082a0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.201+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36819d8d0 con 0x7fd3681082a0 2024-09-15T22:13:33.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.201+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3681008c0 con 0x7fd3681082a0 2024-09-15T22:13:33.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.204+0000 7fd3577fe640 1 -- 172.21.15.77:0/1153764251 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fd360005560 con 0x7fd3681082a0 2024-09-15T22:13:33.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd36f282740 1 Processor -- start 2024-09-15T22:13:33.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd36f282740 1 -- start start 2024-09-15T22:13:33.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd3577fe640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 0x7fd3500a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd365d9b640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 0x7fd3500a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd36f282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd36f282740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.205+0000 7fd3577fe640 1 -- 172.21.15.77:0/1153764251 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd3600d03e0 con 0x7fd3681082a0 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36f282740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36f282740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a60be72f0 con 0x557a60c8c370 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36f282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a60b8ecf0 con 0x557a60c82b40 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36f282740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a60be32b0 con 0x557a60c815d0 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41846/0 (socket says 172.21.15.77:41846) 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd36659c640 1 -- 172.21.15.77:0/2948945763 learned_addr learned my addr 172.21.15.77:0/2948945763 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:33.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 0x7fd3500a5b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3680fee90 tx=0x7fd35c00f360 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd366d9d640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 -- 172.21.15.77:0/2948945763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 msgr2=0x557a60c81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c81090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 -- 172.21.15.77:0/2948945763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 msgr2=0x557a60c8e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c8e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd365d9b640 1 -- 172.21.15.77:0/2948945763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a60b733a0 con 0x557a60c815d0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.206+0000 7fd366d9d640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.207+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c81a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b66c3de21f692e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.207+0000 7fd354ff9640 1 -- 172.21.15.77:0/2948945763 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c025400 con 0x557a60c815d0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.207+0000 7fd354ff9640 1 -- 172.21.15.77:0/2948945763 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c01d8e0 con 0x557a60c815d0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.207+0000 7fd354ff9640 1 -- 172.21.15.77:0/2948945763 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0108f0 con 0x557a60c815d0 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 -- 172.21.15.77:0/2948945763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 msgr2=0x557a60c81a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c81a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c017000 comp rx=0 tx=0).stop 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 -- 172.21.15.77:0/2948945763 shutdown_connections 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c8e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c81a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 --2- 172.21.15.77:0/2948945763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.208+0000 7fd36f282740 1 -- 172.21.15.77:0/2948945763 >> 172.21.15.77:0/2948945763 conn(0x557a60c4a310 msgr2=0x557a60c7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.209+0000 7fd36f282740 1 -- 172.21.15.77:0/2948945763 shutdown_connections 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.209+0000 7fd36f282740 1 -- 172.21.15.77:0/2948945763 wait complete. 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.209+0000 7fd36f282740 1 -- start start 2024-09-15T22:13:33.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.209+0000 7fd36f282740 1 Processor -- start 2024-09-15T22:13:33.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c88fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a60c8f070 con 0x557a60c8c370 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a60ba2060 con 0x557a60c82b40 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd366d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd36f282740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a60c8f1f0 con 0x557a60c815d0 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd366d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49442/0 (socket says 172.21.15.77:49442) 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.210+0000 7fd366d9d640 1 -- 172.21.15.77:0/162423971 learned_addr learned my addr 172.21.15.77:0/162423971 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd365d9b640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c88fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 -- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 msgr2=0x557a60c8bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd36659c640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 -- 172.21.15.77:0/162423971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 msgr2=0x557a60c88fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c88fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 -- 172.21.15.77:0/162423971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a609583c0 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd36659c640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd365d9b640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd36f282740 10 client.0 _ll_register_callbacks cb 0x557a608f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd366d9d640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd3600cd6a0 tx=0x7fd36009c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ede7bc525d8eef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c01d8e0 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.211+0000 7fd34dffb640 10 client.15873 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd34dffb640 1 -- 172.21.15.77:0/162423971 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36001f040 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd34dffb640 1 -- 172.21.15.77:0/162423971 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd3600d00b0 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd34dffb640 1 -- 172.21.15.77:0/162423971 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3600a0120 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x557a60c9a940 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd34dffb640 1 -- 172.21.15.77:0/162423971 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd36000f040 con 0x557a60c8c370 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:33.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 20 client.15873 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60c9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 4 client.15873 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.212+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a60bf4860 con 0x557a60c9bcf0 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.213+0000 7fd36659c640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60c9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.213+0000 7fd36659c640 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60c9e140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc4587f6a0afd2c6 server_cookie=7a8872863e0db450 in_seq=0 out_seq=0 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.213+0000 7fd34dffb640 10 client.15873 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd34dffb640 1 -- 172.21.15.77:0/162423971 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a60bf4860 con 0x557a60c9bcf0 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd34dffb640 10 client.15873 handle_command_reply: tid=0 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 client.15873 shutdown 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 msgr2=0x557a60c895d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd3600cd6a0 tx=0x7fd36009c980 comp rx=0 tx=0).stop 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 shutdown_connections 2024-09-15T22:13:33.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60c9e140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd34dffb640 0 client.15873 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c895d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c88fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 --2- 172.21.15.77:0/162423971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 >> 172.21.15.77:0/162423971 conn(0x557a60c4a310 msgr2=0x557a60c83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 shutdown_connections 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.214+0000 7fd36f282740 1 -- 172.21.15.77:0/162423971 wait complete. 2024-09-15T22:13:33.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.215+0000 7fd36f282740 20 client.15873 trim_cache size 0 max 16384 2024-09-15T22:13:33.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.227+0000 7fd36f282740 1 Processor -- start 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.227+0000 7fd36f282740 1 -- start start 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b11c0 con 0x557a60c8c370 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b2d00 con 0x557a60c82b40 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36f282740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b1890 con 0x557a60c815d0 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd365d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd365d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41874/0 (socket says 172.21.15.77:41874) 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd365d9b640 1 -- 172.21.15.77:0/2971255180 learned_addr learned my addr 172.21.15.77:0/2971255180 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd366d9d640 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd36659c640 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd365d9b640 1 -- 172.21.15.77:0/2971255180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 msgr2=0x557a60c8b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.228+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd365d9b640 1 -- 172.21.15.77:0/2971255180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 msgr2=0x557a60c84660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c84660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd365d9b640 1 -- 172.21.15.77:0/2971255180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a60c87530 con 0x557a60c82b40 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c00faf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc28d5ca896e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd34ffff640 1 -- 172.21.15.77:0/2971255180 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c031020 con 0x557a60c82b40 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd34ffff640 1 -- 172.21.15.77:0/2971255180 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c01da90 con 0x557a60c82b40 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.229+0000 7fd34ffff640 1 -- 172.21.15.77:0/2971255180 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004680 con 0x557a60c82b40 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- 172.21.15.77:0/2971255180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 msgr2=0x557a60c84120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005fd0 tx=0x7fd35c00faf0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- 172.21.15.77:0/2971255180 shutdown_connections 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c8c370 0x557a60c84660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c82b40 0x557a60c84120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 --2- 172.21.15.77:0/2971255180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c815d0 0x557a60c8b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- 172.21.15.77:0/2971255180 >> 172.21.15.77:0/2971255180 conn(0x557a60c4a310 msgr2=0x557a60c890c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- 172.21.15.77:0/2971255180 shutdown_connections 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- 172.21.15.77:0/2971255180 wait complete. 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.230+0000 7fd36f282740 1 -- start start 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 Processor -- start 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 0x557a60c90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c8c370 0x557a60d05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b2630 con 0x557a60c82b40 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b1f60 con 0x557a60c815d0 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681af1a0 con 0x557a60c8c370 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36f282740 10 client.0 _ll_register_callbacks cb 0x7fd360098460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:33.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd366d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c8c370 0x557a60d05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd365d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 0x557a60c90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41886/0 (socket says 172.21.15.77:41886) 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.231+0000 7fd36659c640 1 -- 172.21.15.77:0/2879229000 learned_addr learned my addr 172.21.15.77:0/2879229000 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 -- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c8c370 msgr2=0x557a60d05a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c8c370 0x557a60d05a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 -- 172.21.15.77:0/2879229000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 msgr2=0x557a60c90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 0x557a60c90a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 -- 172.21.15.77:0/2879229000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a60c91340 con 0x557a60c815d0 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 0x557a60c90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36659c640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd35802a940 tx=0x7fd358005510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a7121c4090299a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd34dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:33.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358035020 con 0x557a60c815d0 2024-09-15T22:13:33.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c01da90 con 0x557a60c815d0 2024-09-15T22:13:33.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd358031960 con 0x557a60c815d0 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd358059050 con 0x557a60c815d0 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36f282740 10 client.25289 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a60d07090 con 0x557a60c815d0 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.232+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c01da90 con 0x557a60c815d0 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd35803a040 con 0x557a60c815d0 2024-09-15T22:13:33.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 10 client.25289 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 10 client.25289 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 20 client.25289 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60d097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 4 client.25289 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a60beed70 con 0x557a60c9bcf0 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60d097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.233+0000 7fd34dffb640 10 client.25289 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd365d9b640 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60d097c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2985d1b6fafff1ca server_cookie=a7224c8e5d1983ac in_seq=0 out_seq=0 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd34dffb640 1 -- 172.21.15.77:0/2879229000 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a60beed70 con 0x557a60c9bcf0 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd34dffb640 10 client.25289 handle_command_reply: tid=0 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 client.25289 shutdown 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 msgr2=0x557a60c90520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd35802a940 tx=0x7fd358005510 comp rx=0 tx=0).stop 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 shutdown_connections 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557a60c9bcf0 0x557a60d097c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a60c8c370 0x557a60d05a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd34dffb640 0 client.25289 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a60c82b40 0x557a60c90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 --2- 172.21.15.77:0/2879229000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a60c815d0 0x557a60c90520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 >> 172.21.15.77:0/2879229000 conn(0x557a60c4a310 msgr2=0x557a60c7c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.234+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 shutdown_connections 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36f282740 1 -- 172.21.15.77:0/2879229000 wait complete. 2024-09-15T22:13:33.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36f282740 20 client.25289 trim_cache size 0 max 16384 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 msgr2=0x7fd3500a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 0x7fd3500a5b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3680fee90 tx=0x7fd35c00f360 comp rx=0 tx=0).stop 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 msgr2=0x7fd36819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd36000eaa0 tx=0x7fd36000ef60 comp rx=0 tx=0).stop 2024-09-15T22:13:33.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 shutdown_connections 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd3500a36b0 0x7fd3500a5b70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3681082a0 0x7fd36819cf00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3681036e0 0x7fd368198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 --2- 172.21.15.77:0/1153764251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368102d30 0x7fd368198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.235+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 >> 172.21.15.77:0/1153764251 conn(0x7fd3680fc7d0 msgr2=0x7fd368105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.236+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 shutdown_connections 2024-09-15T22:13:33.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:33.236+0000 7fd36cea9640 1 -- 172.21.15.77:0/1153764251 wait complete. 2024-09-15T22:13:34.058 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:13:34.334 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:34.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:34 smithi077 ceph-mon[30980]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:34.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:34 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:34 smithi117 ceph-mon[35021]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:34 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:34.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:34 smithi196 ceph-mon[35109]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:34.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:34 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.889+0000 7fa0ce236640 1 -- 172.21.15.77:0/3648557747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c81036d0 msgr2=0x7fa0c810bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.889+0000 7fa0ce236640 1 --2- 172.21.15.77:0/3648557747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c81036d0 0x7fa0c810bbc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009900 tx=0x7fa0bc02a480 comp rx=0 tx=0).stop 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 -- 172.21.15.77:0/3648557747 shutdown_connections 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 --2- 172.21.15.77:0/3648557747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c810c190 0x7fa0c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 --2- 172.21.15.77:0/3648557747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c81036d0 0x7fa0c810bbc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 --2- 172.21.15.77:0/3648557747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 -- 172.21.15.77:0/3648557747 >> 172.21.15.77:0/3648557747 conn(0x7fa0c80fc7d0 msgr2=0x7fa0c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 -- 172.21.15.77:0/3648557747 shutdown_connections 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.890+0000 7fa0ce236640 1 -- 172.21.15.77:0/3648557747 wait complete. 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.891+0000 7fa0ce236640 1 Processor -- start 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.891+0000 7fa0ce236640 1 -- start start 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.892+0000 7fa0ce236640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.892+0000 7fa0ce236640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.892+0000 7fa0ce236640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.892+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.892+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43008/0 (socket says 172.21.15.77:43008) 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49472/0 (socket says 172.21.15.77:49472) 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1411886712 learned_addr learned my addr 172.21.15.77:0/1411886712 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111fc0 con 0x7fa0c810c190 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111e40 con 0x7fa0c8102db0 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8112140 con 0x7fa0c81036d0 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c77fe640 1 -- 172.21.15.77:0/1411886712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 msgr2=0x7fa0c8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c77fe640 1 -- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 msgr2=0x7fa0c819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.893+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c77fe640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819d490 con 0x7fa0c8102db0 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c81984e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c4ff9640 1 -- 172.21.15.77:0/1411886712 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8020070 con 0x7fa0c8102db0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c4ff9640 1 -- 172.21.15.77:0/1411886712 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b800ccc0 con 0x7fa0c8102db0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.894+0000 7fa0c4ff9640 1 -- 172.21.15.77:0/1411886712 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801bb40 con 0x7fa0c8102db0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.895+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c819d780 con 0x7fa0c8102db0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.895+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c8100940 con 0x7fa0c8102db0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 Processor -- start 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 -- start start 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571753625d0 0x557175362a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571752c82f0 con 0x557175363b40 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55717526fcf0 con 0x5571753625d0 2024-09-15T22:13:34.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.898+0000 7fa0d060f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5571752c42b0 con 0x55717536d370 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49474/0 (socket says 172.21.15.77:49474) 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 -- 172.21.15.77:0/2504019861 learned_addr learned my addr 172.21.15.77:0/2504019861 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 -- 172.21.15.77:0/2504019861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 msgr2=0x55717536f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 -- 172.21.15.77:0/2504019861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571753625d0 msgr2=0x557175362a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571753625d0 0x557175362a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 -- 172.21.15.77:0/2504019861 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571752543a0 con 0x557175363b40 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.899+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00117e0 tx=0x7fa0c0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c396e52ac0ecc02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.900+0000 7fa0adffb640 1 -- 172.21.15.77:0/2504019861 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003c30 con 0x557175363b40 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.900+0000 7fa0adffb640 1 -- 172.21.15.77:0/2504019861 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c000fe80 con 0x557175363b40 2024-09-15T22:13:34.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.900+0000 7fa0adffb640 1 -- 172.21.15.77:0/2504019861 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00193e0 con 0x557175363b40 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.900+0000 7fa0c4ff9640 1 -- 172.21.15.77:0/1411886712 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fa0b80137d0 con 0x7fa0c8102db0 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 -- 172.21.15.77:0/2504019861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 msgr2=0x557175362090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00117e0 tx=0x7fa0c0011ca0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 -- 172.21.15.77:0/2504019861 shutdown_connections 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571753625d0 0x557175362a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 --2- 172.21.15.77:0/2504019861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557175363b40 0x557175362090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 -- 172.21.15.77:0/2504019861 >> 172.21.15.77:0/2504019861 conn(0x55717532b310 msgr2=0x55717535ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 -- 172.21.15.77:0/2504019861 shutdown_connections 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0d060f740 1 -- 172.21.15.77:0/2504019861 wait complete. 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.901+0000 7fa0c4ff9640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 0x7fa0b00a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.902+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 0x7fa0b00a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.902+0000 7fa0c4ff9640 1 -- 172.21.15.77:0/1411886712 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa0b80d46f0 con 0x7fa0c8102db0 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.902+0000 7fa0d060f740 1 -- start start 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.902+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 0x7fa0b00a59b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557175363070 tx=0x7fa0bc0046c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.903+0000 7fa0d060f740 1 Processor -- start 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.903+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.903+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x557175369ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49476/0 (socket says 172.21.15.77:49476) 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c77fe640 1 -- 172.21.15.77:0/1115805630 learned_addr learned my addr 172.21.15.77:0/1115805630 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x557175369ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557175370070 con 0x5571753625d0 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557175283060 con 0x557175363b40 2024-09-15T22:13:34.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5571753701f0 con 0x55717536d370 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1115805630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 msgr2=0x55717536a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.904+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1115805630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 msgr2=0x55717536cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571750393c0 con 0x557175363b40 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0d060f740 10 client.0 _ll_register_callbacks cb 0x557174fd68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571752d5860 con 0x557175363b40 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x557175369ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd08aa3c93a5148 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0a6ffd640 10 client.25298 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1115805630 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc035040 con 0x557175363b40 2024-09-15T22:13:34.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.905+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1115805630 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa0bc003b80 con 0x557175363b40 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1115805630 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc003dc0 con 0x557175363b40 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 10 client.25298 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55717537ba00 con 0x557175363b40 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1115805630 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa0bc044030 con 0x557175363b40 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 10 client.25298 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 10 client.25298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 20 client.25298 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55717537cdb0 0x55717537f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 4 client.25298 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.906+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571752cfd70 con 0x55717537cdb0 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.907+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55717537cdb0 0x55717537f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.907+0000 7fa0a6ffd640 10 client.25298 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.907+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55717537cdb0 0x55717537f1e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f682734002cab8d server_cookie=19dfe19b0b29e3ff in_seq=0 out_seq=0 2024-09-15T22:13:34.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1115805630 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571752cfd70 con 0x55717537cdb0 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0a6ffd640 10 client.25298 handle_command_reply: tid=0 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 client.25298 shutdown 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 msgr2=0x557175369ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x557175369ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc009260 comp rx=0 tx=0).stop 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 shutdown_connections 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55717537cdb0 0x55717537f1e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55717536d370 0x55717536a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x557175369ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1115805630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x55717536cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 >> 172.21.15.77:0/1115805630 conn(0x55717532b310 msgr2=0x557175364930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0a6ffd640 0 client.25298 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 shutdown_connections 2024-09-15T22:13:34.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.908+0000 7fa0d060f740 1 -- 172.21.15.77:0/1115805630 wait complete. 2024-09-15T22:13:34.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.909+0000 7fa0d060f740 20 client.25298 trim_cache size 0 max 16384 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.933+0000 7fa0d060f740 1 Processor -- start 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.933+0000 7fa0d060f740 1 -- start start 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.933+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.933+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.933+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41910/0 (socket says 172.21.15.77:41910) 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49482/0 (socket says 172.21.15.77:49482) 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0d060f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81acd30 con 0x55717536d370 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0d060f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81ae870 con 0x557175363b40 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0d060f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81ad400 con 0x5571753625d0 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43036/0 (socket says 172.21.15.77:43036) 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 -- 172.21.15.77:0/63458369 learned_addr learned my addr 172.21.15.77:0/63458369 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 -- 172.21.15.77:0/63458369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 msgr2=0x5571753698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 -- 172.21.15.77:0/63458369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 msgr2=0x557175384ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 -- 172.21.15.77:0/63458369 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55717536a950 con 0x5571753625d0 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.934+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0b809cf90 tx=0x7fa0b801b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6119a5f1b1563827 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.935+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.935+0000 7fa0ad7fa640 1 -- 172.21.15.77:0/63458369 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80a91d0 con 0x5571753625d0 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.935+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.935+0000 7fa0ad7fa640 1 -- 172.21.15.77:0/63458369 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b809c040 con 0x5571753625d0 2024-09-15T22:13:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.935+0000 7fa0ad7fa640 1 -- 172.21.15.77:0/63458369 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80d4530 con 0x5571753625d0 2024-09-15T22:13:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- 172.21.15.77:0/63458369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 msgr2=0x557175379390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa0b809cf90 tx=0x7fa0b801b5e0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- 172.21.15.77:0/63458369 shutdown_connections 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55717536d370 0x557175384ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557175363b40 0x5571753698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 --2- 172.21.15.77:0/63458369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571753625d0 0x557175379390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- 172.21.15.77:0/63458369 >> 172.21.15.77:0/63458369 conn(0x55717532b310 msgr2=0x5571753688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- 172.21.15.77:0/63458369 shutdown_connections 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- 172.21.15.77:0/63458369 wait complete. 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.936+0000 7fa0d060f740 1 -- start start 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 Processor -- start 2024-09-15T22:13:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x557175371290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55717536d370 0x557175371d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81ae1a0 con 0x5571753625d0 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81adad0 con 0x55717536d370 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0d060f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81aad10 con 0x557175363b40 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43046/0 (socket says 172.21.15.77:43046) 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1141059257 learned_addr learned my addr 172.21.15.77:0/1141059257 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x557175371290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.937+0000 7fa0c7fff640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55717536d370 0x557175371d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0d060f740 10 client.0 _ll_register_callbacks cb 0x7fa0bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55717536d370 msgr2=0x557175371d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55717536d370 0x557175371d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1141059257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 msgr2=0x557175371290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x557175371290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 -- 172.21.15.77:0/1141059257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571753e6a80 con 0x557175363b40 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x557175371290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0c6ffd640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc004770 tx=0x7fa0bc043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b24ddca0e26aeda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc005d50 con 0x557175363b40 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa0bc03d860 con 0x557175363b40 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b809c040 con 0x557175363b40 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc03da80 con 0x557175363b40 2024-09-15T22:13:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.938+0000 7fa0d060f740 10 client.25276 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.939+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571753f8c60 con 0x557175363b40 2024-09-15T22:13:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.939+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0bc060070 con 0x557175363b40 2024-09-15T22:13:34.946 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.939+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa0bc044040 con 0x557175363b40 2024-09-15T22:13:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 10 client.25276 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 10 client.25276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 20 client.25276 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5571753f9380 0x5571753fb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 4 client.25276 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa0b809c040 con 0x5571753f9380 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.940+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5571753f9380 0x5571753fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0c77fe640 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5571753f9380 0x5571753fb770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab77370a91a3059b server_cookie=c4104102fdfaad78 in_seq=0 out_seq=0 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0a6ffd640 10 client.25276 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0a6ffd640 1 -- 172.21.15.77:0/1141059257 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa0b809c040 con 0x5571753f9380 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0a6ffd640 10 client.25276 handle_command_reply: tid=0 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 client.25276 shutdown 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 msgr2=0x5571753717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc004770 tx=0x7fa0bc043000 comp rx=0 tx=0).stop 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 shutdown_connections 2024-09-15T22:13:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5571753f9380 0x5571753fb770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0a6ffd640 0 client.25276 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55717536d370 0x557175371d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557175363b40 0x5571753717d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 --2- 172.21.15.77:0/1141059257 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571753625d0 0x557175371290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 >> 172.21.15.77:0/1141059257 conn(0x55717532b310 msgr2=0x55717536c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 shutdown_connections 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.941+0000 7fa0d060f740 1 -- 172.21.15.77:0/1141059257 wait complete. 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0d060f740 20 client.25276 trim_cache size 0 max 16384 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 msgr2=0x7fa0b00a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 0x7fa0b00a59b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557175363070 tx=0x7fa0bc0046c0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 msgr2=0x7fa0c81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c81984e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b800edc0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 shutdown_connections 2024-09-15T22:13:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a34f0 0x7fa0b00a59b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c810c190 0x7fa0c819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c81036d0 0x7fa0c8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 --2- 172.21.15.77:0/1411886712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8102db0 0x7fa0c81984e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.942+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 >> 172.21.15.77:0/1411886712 conn(0x7fa0c80fc7d0 msgr2=0x7fa0c80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.943+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 shutdown_connections 2024-09-15T22:13:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:34.943+0000 7fa0ce236640 1 -- 172.21.15.77:0/1411886712 wait complete. 2024-09-15T22:13:35.664 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:13:36.021 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:35 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:35 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:35 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:36.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 -- 172.21.15.77:0/3013268223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c101360 msgr2=0x7fa54c10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 --2- 172.21.15.77:0/3013268223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c101360 0x7fa54c10df90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa540009900 tx=0x7fa54002a410 comp rx=0 tx=0).stop 2024-09-15T22:13:36.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 -- 172.21.15.77:0/3013268223 shutdown_connections 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 --2- 172.21.15.77:0/3013268223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c10e4d0 0x7fa54c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 --2- 172.21.15.77:0/3013268223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c101360 0x7fa54c10df90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 --2- 172.21.15.77:0/3013268223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c1009b0 0x7fa54c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 -- 172.21.15.77:0/3013268223 >> 172.21.15.77:0/3013268223 conn(0x7fa54c0fc7d0 msgr2=0x7fa54c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 -- 172.21.15.77:0/3013268223 shutdown_connections 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.566+0000 7fa55089f640 1 -- 172.21.15.77:0/3013268223 wait complete. 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.567+0000 7fa55089f640 1 Processor -- start 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.567+0000 7fa55089f640 1 -- start start 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.567+0000 7fa55089f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 0x7fa54c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.567+0000 7fa55089f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 0x7fa54c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 0x7fa54c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49500/0 (socket says 172.21.15.77:49500) 2024-09-15T22:13:36.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa55089f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c10e4d0 0x7fa54c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa55089f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c114220 con 0x7fa54c1009b0 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43066/0 (socket says 172.21.15.77:43066) 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 -- 172.21.15.77:0/2027046357 learned_addr learned my addr 172.21.15.77:0/2027046357 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa55089f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1140a0 con 0x7fa54c10e4d0 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1143a0 con 0x7fa54c101360 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c10e4d0 0x7fa54c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 -- 172.21.15.77:0/2027046357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c10e4d0 msgr2=0x7fa54c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c10e4d0 0x7fa54c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 -- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 msgr2=0x7fa54c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 0x7fa54c1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.568+0000 7fa549d9b640 1 -- 172.21.15.77:0/2027046357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c19ace0 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa549d9b640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa540006010 tx=0x7fa5400042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa53b7fe640 1 -- 172.21.15.77:0/2027046357 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54003f030 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c19af70 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa53b7fe640 1 -- 172.21.15.77:0/2027046357 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa540003ce0 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c1a72c0 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.569+0000 7fa53b7fe640 1 -- 172.21.15.77:0/2027046357 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540035070 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 Processor -- start 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa53b7fe640 1 -- 172.21.15.77:0/2027046357 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fa540042030 con 0x7fa54c101360 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 -- start start 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c5c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c67b80 0x55af54c69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af54bba2f0 con 0x55af54c5e350 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af54b61cf0 con 0x55af54c67b80 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.573+0000 7fa552c78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af54bb62b0 con 0x55af54c5cde0 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43074/0 (socket says 172.21.15.77:43074) 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 -- 172.21.15.77:0/2056096416 learned_addr learned my addr 172.21.15.77:0/2056096416 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa549d9b640 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c67b80 0x55af54c69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c5c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa53b7fe640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 0x7fa5340a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 -- 172.21.15.77:0/2056096416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c67b80 msgr2=0x55af54c69f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c67b80 0x55af54c69f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 -- 172.21.15.77:0/2056096416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 msgr2=0x55af54c5c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c5c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54a59c640 1 -- 172.21.15.77:0/2056096416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af54b463a0 con 0x55af54c5cde0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 0x7fa5340a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.574+0000 7fa53b7fe640 1 -- 172.21.15.77:0/2027046357 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa5400f0810 con 0x7fa54c101360 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.575+0000 7fa54a59c640 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00bd10 tx=0x7fa53c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9cb8bd0dd65292f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.575+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 0x7fa5340a5aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa54400a2c0 tx=0x7fa544004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.575+0000 7fa538ff9640 1 -- 172.21.15.77:0/2056096416 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c004030 con 0x55af54c5cde0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.575+0000 7fa538ff9640 1 -- 172.21.15.77:0/2056096416 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c0041d0 con 0x55af54c5cde0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.575+0000 7fa538ff9640 1 -- 172.21.15.77:0/2056096416 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0117e0 con 0x55af54c5cde0 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- 172.21.15.77:0/2056096416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 msgr2=0x55af54c5d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00bd10 tx=0x7fa53c0107a0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- 172.21.15.77:0/2056096416 shutdown_connections 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c67b80 0x55af54c69f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c5cde0 0x55af54c5d220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 --2- 172.21.15.77:0/2056096416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c5c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- 172.21.15.77:0/2056096416 >> 172.21.15.77:0/2056096416 conn(0x55af54c1d310 msgr2=0x55af54c59470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- 172.21.15.77:0/2056096416 shutdown_connections 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- 172.21.15.77:0/2056096416 wait complete. 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 -- start start 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.576+0000 7fa552c78740 1 Processor -- start 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c5cde0 0x55af54c67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa549d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c64e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa549d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49516/0 (socket says 172.21.15.77:49516) 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c5cde0 0x55af54c67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa549d9b640 1 -- 172.21.15.77:0/3455123188 learned_addr learned my addr 172.21.15.77:0/3455123188 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af54c6a880 con 0x55af54c5e350 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af54b75060 con 0x55af54c5cde0 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.577+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af54c6aa00 con 0x55af54c67b80 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c64e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 -- 172.21.15.77:0/3455123188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 msgr2=0x55af54c64e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c64e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 -- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c5cde0 msgr2=0x55af54c67500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c5cde0 0x55af54c67500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5492b3c0 con 0x55af54c5e350 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa552c78740 10 client.0 _ll_register_callbacks cb 0x55af548c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c0041d0 con 0x55af54c5e350 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa549d9b640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa5400b8b40 tx=0x7fa540005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9b409c2af2deea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa531ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa531ffb640 1 -- 172.21.15.77:0/3455123188 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540049050 con 0x55af54c5e350 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.578+0000 7fa531ffb640 1 -- 172.21.15.77:0/3455123188 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa5400f0470 con 0x55af54c5e350 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa531ffb640 1 -- 172.21.15.77:0/3455123188 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5400b8980 con 0x55af54c5e350 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 10 client.15906 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55af54c760d0 con 0x55af54c5e350 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa531ffb640 1 -- 172.21.15.77:0/3455123188 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa5400bc940 con 0x55af54c5e350 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 10 client.15906 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 20 client.15906 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:36.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54c775c0 0x55af54c79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 4 client.15906 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.579+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af54bc7860 con 0x55af54c775c0 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.580+0000 7fa54a59c640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54c775c0 0x55af54c79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.580+0000 7fa54a59c640 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54c775c0 0x55af54c79a10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6122e85170fa464b server_cookie=90aedf0dceb30400 in_seq=0 out_seq=0 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.580+0000 7fa531ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.581+0000 7fa531ffb640 1 -- 172.21.15.77:0/3455123188 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af54bc7860 con 0x55af54c775c0 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.581+0000 7fa531ffb640 10 client.15906 handle_command_reply: tid=0 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.581+0000 7fa552c78740 1 client.15906 shutdown 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 msgr2=0x55af54c64800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa5400b8b40 tx=0x7fa540005bd0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 shutdown_connections 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54c775c0 0x55af54c79a10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c64e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5e350 0x55af54c64800 secure :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa5400b8b40 tx=0x7fa540005bd0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa531ffb640 0 client.15906 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 --2- 172.21.15.77:0/3455123188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c5cde0 0x55af54c67500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 >> 172.21.15.77:0/3455123188 conn(0x55af54c1d310 msgr2=0x55af54c5f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 shutdown_connections 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 1 -- 172.21.15.77:0/3455123188 wait complete. 2024-09-15T22:13:36.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.582+0000 7fa552c78740 20 client.15906 trim_cache size 0 max 16384 2024-09-15T22:13:36.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.606+0000 7fa552c78740 1 Processor -- start 2024-09-15T22:13:36.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.606+0000 7fa552c78740 1 -- start start 2024-09-15T22:13:36.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1acd40 con 0x55af54c5cde0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1ae880 con 0x55af54c775c0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa552c78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1ad410 con 0x55af54c67b80 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49518/0 (socket says 172.21.15.77:49518) 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 -- 172.21.15.77:0/1294041880 learned_addr learned my addr 172.21.15.77:0/1294041880 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa549d9b640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 -- 172.21.15.77:0/1294041880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 msgr2=0x55af54c5f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c5f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 -- 172.21.15.77:0/1294041880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 msgr2=0x55af54c5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c5fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54a59c640 1 -- 172.21.15.77:0/1294041880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af54c67100 con 0x55af54c5cde0 2024-09-15T22:13:36.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa549d9b640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:36.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.607+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:36.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.608+0000 7fa54a59c640 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55af54c5a2d0 tx=0x7fa53c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865672b411b1995f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.608+0000 7fa533fff640 1 -- 172.21.15.77:0/1294041880 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c005040 con 0x55af54c5cde0 2024-09-15T22:13:36.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.608+0000 7fa533fff640 1 -- 172.21.15.77:0/1294041880 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c0114b0 con 0x55af54c5cde0 2024-09-15T22:13:36.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.608+0000 7fa533fff640 1 -- 172.21.15.77:0/1294041880 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0276a0 con 0x55af54c5cde0 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- 172.21.15.77:0/1294041880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 msgr2=0x55af54c66770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55af54c5a2d0 tx=0x7fa53c00bc10 comp rx=0 tx=0).stop 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- 172.21.15.77:0/1294041880 shutdown_connections 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c5f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 --2- 172.21.15.77:0/1294041880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c66770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- 172.21.15.77:0/1294041880 >> 172.21.15.77:0/1294041880 conn(0x55af54c1d310 msgr2=0x55af54c66060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- 172.21.15.77:0/1294041880 shutdown_connections 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- 172.21.15.77:0/1294041880 wait complete. 2024-09-15T22:13:36.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 -- start start 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.609+0000 7fa552c78740 1 Processor -- start 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c6fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1ae1b0 con 0x55af54c5cde0 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1adae0 con 0x55af54c775c0 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1aad40 con 0x55af54c67b80 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa552c78740 10 client.0 _ll_register_callbacks cb 0x7fa540005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa54ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa54ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41968/0 (socket says 172.21.15.77:41968) 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa54ad9d640 1 -- 172.21.15.77:0/868550972 learned_addr learned my addr 172.21.15.77:0/868550972 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa54a59c640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c6fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.610+0000 7fa549d9b640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 -- 172.21.15.77:0/868550972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 msgr2=0x55af54c701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 -- 172.21.15.77:0/868550972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 msgr2=0x55af54c6fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c6fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 -- 172.21.15.77:0/868550972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af54c626d0 con 0x55af54c775c0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54a59c640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c6fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa549d9b640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa54ad9d640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa54401ee00 tx=0x7fa544004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414ecc14b336c503 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402d060 con 0x55af54c775c0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c0114b0 con 0x55af54c775c0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa54401d850 con 0x55af54c775c0 2024-09-15T22:13:36.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa54401ee30 con 0x55af54c775c0 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa552c78740 10 client.25316 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.611+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af54ce2830 con 0x55af54c775c0 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c0114b0 con 0x55af54c775c0 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa544027030 con 0x55af54c775c0 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 10 client.25316 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 20 client.25316 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54ce2ee0 0x55af54ce5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 4 client.25316 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.612+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af54bc1d70 con 0x55af54ce2ee0 2024-09-15T22:13:36.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa54a59c640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54ce2ee0 0x55af54ce5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa54a59c640 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54ce2ee0 0x55af54ce5320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e99084f677dd8fd server_cookie=9fb1623e6e8ed0ea in_seq=0 out_seq=0 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa531ffb640 10 client.25316 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa531ffb640 1 -- 172.21.15.77:0/868550972 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af54bc1d70 con 0x55af54ce2ee0 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa531ffb640 10 client.25316 handle_command_reply: tid=0 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa552c78740 1 client.25316 shutdown 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 msgr2=0x55af54c6cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.613+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa54401ee00 tx=0x7fa544004ab0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 shutdown_connections 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af54ce2ee0 0x55af54ce5320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af54c775c0 0x55af54c6cc60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af54c67b80 0x55af54c701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 --2- 172.21.15.77:0/868550972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af54c5cde0 0x55af54c6fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 >> 172.21.15.77:0/868550972 conn(0x55af54c1d310 msgr2=0x55af54c66880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa531ffb640 0 client.25316 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 shutdown_connections 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 1 -- 172.21.15.77:0/868550972 wait complete. 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.614+0000 7fa552c78740 20 client.25316 trim_cache size 0 max 16384 2024-09-15T22:13:36.621 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 msgr2=0x7fa5340a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 0x7fa5340a5aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa54400a2c0 tx=0x7fa544004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 msgr2=0x7fa54c1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa540006010 tx=0x7fa5400042f0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 shutdown_connections 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa5340a35e0 0x7fa5340a5aa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa54c10e4d0 0x7fa54c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa54c101360 0x7fa54c1a0930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 --2- 172.21.15.77:0/2027046357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa54c1009b0 0x7fa54c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 >> 172.21.15.77:0/2027046357 conn(0x7fa54c0fc7d0 msgr2=0x7fa54c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:36.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 shutdown_connections 2024-09-15T22:13:36.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:36.615+0000 7fa55089f640 1 -- 172.21.15.77:0/2027046357 wait complete. 2024-09-15T22:13:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:36 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:36 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:36 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:37.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:36 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:37.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:36 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:37.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:36 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:36 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:36 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:37.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:36 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:13:38.309 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:13:38.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:13:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:37 smithi077 ceph-mon[30980]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:37 smithi196 ceph-mon[35109]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:37 smithi117 ceph-mon[35021]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:38.567 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 -- 172.21.15.77:0/2946961665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 msgr2=0x7fb184106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/2946961665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb184106e50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb180009930 tx=0x7fb18002a420 comp rx=0 tx=0).stop 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 -- 172.21.15.77:0/2946961665 shutdown_connections 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/2946961665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 0x7fb18410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/2946961665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/2946961665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb184106e50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.109+0000 7fb18b3e9640 1 -- 172.21.15.77:0/2946961665 >> 172.21.15.77:0/2946961665 conn(0x7fb1840fc790 msgr2=0x7fb1840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.110+0000 7fb18b3e9640 1 -- 172.21.15.77:0/2946961665 shutdown_connections 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.110+0000 7fb18b3e9640 1 -- 172.21.15.77:0/2946961665 wait complete. 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 Processor -- start 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 -- start start 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 0x7fb1841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb18419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb184110e30 con 0x7fb184106a70 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb184110cb0 con 0x7fb184101410 2024-09-15T22:13:39.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb18b3e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb184110fb0 con 0x7fb184100a70 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb189185640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb189185640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51970/0 (socket says 172.21.15.77:51970) 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb189185640 1 -- 172.21.15.77:0/1718549696 learned_addr learned my addr 172.21.15.77:0/1718549696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb189986640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb18419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.111+0000 7fb188984640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 0x7fb1841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 -- 172.21.15.77:0/1718549696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 msgr2=0x7fb1841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 0x7fb1841989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 -- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 msgr2=0x7fb18419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb18419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 -- 172.21.15.77:0/1718549696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18419d3d0 con 0x7fb184100a70 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb189185640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb180009230 tx=0x7fb180004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb17a7fc640 1 -- 172.21.15.77:0/1718549696 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18003f030 con 0x7fb184100a70 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb17a7fc640 1 -- 172.21.15.77:0/1718549696 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb180003ad0 con 0x7fb184100a70 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18419d660 con 0x7fb184100a70 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb17a7fc640 1 -- 172.21.15.77:0/1718549696 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180035070 con 0x7fb184100a70 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.112+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1840fce00 con 0x7fb184100a70 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.116+0000 7fb18d7c2740 1 Processor -- start 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.116+0000 7fb18d7c2740 1 -- start start 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.116+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b2b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.116+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2bc90 0x55ec67b36010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.116+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec67a88280 con 0x55ec67b36550 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec67a2fb10 con 0x55ec67b2b370 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec67a84240 con 0x55ec67b2bc90 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44956/0 (socket says 172.21.15.77:44956) 2024-09-15T22:13:39.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 -- 172.21.15.77:0/995094037 learned_addr learned my addr 172.21.15.77:0/995094037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb189986640 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb17a7fc640 1 -- 172.21.15.77:0/1718549696 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb180042030 con 0x7fb184100a70 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 -- 172.21.15.77:0/995094037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2bc90 msgr2=0x55ec67b36010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2bc90 0x55ec67b36010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 -- 172.21.15.77:0/995094037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 msgr2=0x55ec67b2b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b2b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.117+0000 7fb188984640 1 -- 172.21.15.77:0/995094037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec67a3c1f0 con 0x55ec67b36550 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb188984640 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb17400bce0 tx=0x7fb1740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0abd30f9ee24f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb16f7fe640 1 -- 172.21.15.77:0/995094037 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174005220 con 0x55ec67b36550 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb16f7fe640 1 -- 172.21.15.77:0/995094037 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb174004c80 con 0x55ec67b36550 2024-09-15T22:13:39.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb16f7fe640 1 -- 172.21.15.77:0/995094037 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1740118c0 con 0x55ec67b36550 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb17a7fc640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 0x7fb1700a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb17a7fc640 1 -- 172.21.15.77:0/1718549696 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb1800f0010 con 0x7fb184100a70 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.118+0000 7fb189986640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 0x7fb1700a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb189986640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 0x7fb1700a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005fd0 tx=0x7fb17c009600 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 -- 172.21.15.77:0/995094037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 msgr2=0x55ec67b38940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb17400bce0 tx=0x7fb1740107a0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 -- 172.21.15.77:0/995094037 shutdown_connections 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b38940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2bc90 0x55ec67b36010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/995094037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b2b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.119+0000 7fb18d7c2740 1 -- 172.21.15.77:0/995094037 >> 172.21.15.77:0/995094037 conn(0x55ec67b27900 msgr2=0x55ec67b27ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.120+0000 7fb18d7c2740 1 -- 172.21.15.77:0/995094037 shutdown_connections 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.120+0000 7fb18d7c2740 1 -- 172.21.15.77:0/995094037 wait complete. 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.120+0000 7fb18d7c2740 1 -- start start 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.120+0000 7fb18d7c2740 1 Processor -- start 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ec67b39120 con 0x55ec67b2bc90 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44972/0 (socket says 172.21.15.77:44972) 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb188984640 1 -- 172.21.15.77:0/1853851352 learned_addr learned my addr 172.21.15.77:0/1853851352 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:39.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ec67a42f10 con 0x55ec67b2b370 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.121+0000 7fb189986640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb189185640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ec67b392a0 con 0x55ec67b36550 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb18d7c2740 10 client.0 _ll_register_callbacks cb 0x55ec67a0ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 -- 172.21.15.77:0/1853851352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 msgr2=0x55ec67b34050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b34050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 -- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 msgr2=0x55ec67b335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec677f96a0 con 0x55ec67b2bc90 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb188984640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb174005550 tx=0x7fb1740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117514d5758199c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17402a060 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174004c80 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.122+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb174004e30 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb174031050 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 10 client.15924 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec67b44e90 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb174004c80 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb174011aa0 con 0x55ec67b2bc90 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 10 client.15924 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 20 client.15924 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67b462a0 0x55ec67b48690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 4 client.15924 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.123+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec67a957f0 con 0x55ec67b462a0 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.124+0000 7fb189185640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67b462a0 0x55ec67b48690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.124+0000 7fb189185640 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67b462a0 0x55ec67b48690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c6f10522da031e server_cookie=8138d2dfee57a332 in_seq=0 out_seq=0 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.124+0000 7fb16cff9640 10 client.15924 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb16cff9640 1 -- 172.21.15.77:0/1853851352 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec67a957f0 con 0x55ec67b462a0 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb16cff9640 10 client.15924 handle_command_reply: tid=0 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 client.15924 shutdown 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 msgr2=0x55ec67b33b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb174005550 tx=0x7fb1740048f0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 shutdown_connections 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67b462a0 0x55ec67b48690 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b34050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b2bc90 0x55ec67b33b10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/1853851352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 >> 172.21.15.77:0/1853851352 conn(0x55ec67b27900 msgr2=0x55ec67b2d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb16cff9640 0 client.15924 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 shutdown_connections 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 1 -- 172.21.15.77:0/1853851352 wait complete. 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.125+0000 7fb18d7c2740 20 client.15924 trim_cache size 0 max 16384 2024-09-15T22:13:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.142+0000 7fb18d7c2740 1 Processor -- start 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.142+0000 7fb18d7c2740 1 -- start start 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2b370 0x55ec67b33ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b4b810 0x55ec67b30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841accf0 con 0x55ec67b36550 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ae830 con 0x55ec67b4b810 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ad3c0 con 0x55ec67b2b370 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44986/0 (socket says 172.21.15.77:44986) 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 -- 172.21.15.77:0/3427439800 learned_addr learned my addr 172.21.15.77:0/3427439800 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb189185640 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2b370 0x55ec67b33ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb189986640 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b4b810 0x55ec67b30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 -- 172.21.15.77:0/3427439800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2b370 msgr2=0x55ec67b33ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2b370 0x55ec67b33ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 -- 172.21.15.77:0/3427439800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b4b810 msgr2=0x55ec67b30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b4b810 0x55ec67b30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 -- 172.21.15.77:0/3427439800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec67b325a0 con 0x55ec67b36550 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.143+0000 7fb188984640 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb174001eb0 tx=0x7fb1740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00d0f0c327cd2f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb16effd640 1 -- 172.21.15.77:0/3427439800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17400be90 con 0x55ec67b36550 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb16effd640 1 -- 172.21.15.77:0/3427439800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb174010ca0 con 0x55ec67b36550 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb16effd640 1 -- 172.21.15.77:0/3427439800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174004bf0 con 0x55ec67b36550 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb18d7c2740 1 -- 172.21.15.77:0/3427439800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 msgr2=0x55ec67b4b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb174001eb0 tx=0x7fb1740107a0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb18d7c2740 1 -- 172.21.15.77:0/3427439800 shutdown_connections 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.144+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b4b810 0x55ec67b30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b36550 0x55ec67b4b2d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/3427439800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b2b370 0x55ec67b33ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- 172.21.15.77:0/3427439800 >> 172.21.15.77:0/3427439800 conn(0x55ec67b27900 msgr2=0x55ec67b33590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- 172.21.15.77:0/3427439800 shutdown_connections 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- 172.21.15.77:0/3427439800 wait complete. 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- start start 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 Processor -- start 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ae160 con 0x55ec67b4b810 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ada90 con 0x55ec67b2b370 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841aacd0 con 0x55ec67b36550 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.145+0000 7fb18d7c2740 10 client.0 _ll_register_callbacks cb 0x7fb17c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44990/0 (socket says 172.21.15.77:44990) 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 -- 172.21.15.77:0/630061164 learned_addr learned my addr 172.21.15.77:0/630061164 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189185640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb188984640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 -- 172.21.15.77:0/630061164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 msgr2=0x55ec67b4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 -- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 msgr2=0x55ec67b4ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b4ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 -- 172.21.15.77:0/630061164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec67b1a830 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb189986640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb17c011e20 tx=0x7fb17c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6045a73f78bc95b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c02f060 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb17c003d80 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.146+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174010ca0 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb17c036050 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 10 client.15930 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec67bb1790 con 0x55ec67b4b810 2024-09-15T22:13:39.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17c020e00 con 0x55ec67b4b810 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb17c029030 con 0x55ec67b4b810 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 20 client.15930 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67bb1f40 0x55ec67bb4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 4 client.15930 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.147+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb174010ca0 con 0x55ec67bb1f40 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.148+0000 7fb189185640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67bb1f40 0x55ec67bb4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.148+0000 7fb189185640 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67bb1f40 0x55ec67bb4360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101ed8a5f1c7e42c server_cookie=78c4524f5ee03d9a in_seq=0 out_seq=0 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.148+0000 7fb16cff9640 10 client.15930 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.149+0000 7fb16cff9640 1 -- 172.21.15.77:0/630061164 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb174010ca0 con 0x55ec67bb1f40 2024-09-15T22:13:39.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.149+0000 7fb16cff9640 10 client.15930 handle_command_reply: tid=0 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 client.15930 shutdown 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 msgr2=0x55ec67b49380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb17c011e20 tx=0x7fb17c004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 shutdown_connections 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ec67bb1f40 0x55ec67bb4360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ec67b4b810 0x55ec67b49380 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ec67b36550 0x55ec67b4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 --2- 172.21.15.77:0/630061164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec67b2b370 0x55ec67b4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 >> 172.21.15.77:0/630061164 conn(0x55ec67b27900 msgr2=0x55ec67b1e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb16cff9640 0 client.15930 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:39.172 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:39.172 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:13:39.172 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "44d68642-e903-4cfa-ab6b-d66f4de1563d", 2024-09-15T22:13:39.172 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:13:39.172 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 shutdown_connections 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 1 -- 172.21.15.77:0/630061164 wait complete. 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.150+0000 7fb18d7c2740 20 client.15930 trim_cache size 0 max 16384 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 msgr2=0x7fb1700a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 0x7fb1700a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005fd0 tx=0x7fb17c009600 comp rx=0 tx=0).stop 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 msgr2=0x7fb184198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb180009230 tx=0x7fb180004270 comp rx=0 tx=0).stop 2024-09-15T22:13:39.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 shutdown_connections 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1700a35e0 0x7fb1700a5aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb184106a70 0x7fb18419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb184101410 0x7fb1841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 --2- 172.21.15.77:0/1718549696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb184100a70 0x7fb184198470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 >> 172.21.15.77:0/1718549696 conn(0x7fb1840fc790 msgr2=0x7fb18406ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 shutdown_connections 2024-09-15T22:13:39.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:39.151+0000 7fb18b3e9640 1 -- 172.21.15.77:0/1718549696 wait complete. 2024-09-15T22:13:39.860 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:39 smithi077 ceph-mon[30980]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:39 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:13:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:39 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:39 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:13:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:39 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:40.250 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:39 smithi196 ceph-mon[35109]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:39 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:13:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:39 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:39 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:13:40.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:39 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:39 smithi117 ceph-mon[35021]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-15T22:13:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:39 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:13:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:39 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:39 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:13:40.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:39 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:41.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 -- 172.21.15.77:0/2090308551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f81012c0 msgr2=0x7f13f810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 --2- 172.21.15.77:0/2090308551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f81012c0 0x7f13f810df80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a410 comp rx=0 tx=0).stop 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 -- 172.21.15.77:0/2090308551 shutdown_connections 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 --2- 172.21.15.77:0/2090308551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 0x7f13f81108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 --2- 172.21.15.77:0/2090308551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f81012c0 0x7f13f810df80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 --2- 172.21.15.77:0/2090308551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f8100910 0x7f13f8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 -- 172.21.15.77:0/2090308551 >> 172.21.15.77:0/2090308551 conn(0x7f13f80fc780 msgr2=0x7f13f80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 -- 172.21.15.77:0/2090308551 shutdown_connections 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 -- 172.21.15.77:0/2090308551 wait complete. 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.986+0000 7f13ff79c640 1 Processor -- start 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 -- start start 2024-09-15T22:13:41.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 0x7f13f819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 0x7f13f819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8114140 con 0x7f13f8100910 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8113fc0 con 0x7f13f81012c0 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13ff79c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81142c0 con 0x7f13f810e4c0 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fd538640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fd538640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45004/0 (socket says 172.21.15.77:45004) 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fd538640 1 -- 172.21.15.77:0/3029068102 learned_addr learned my addr 172.21.15.77:0/3029068102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 0x7f13f819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fdd39640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 0x7f13f819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 -- 172.21.15.77:0/3029068102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 msgr2=0x7f13f819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 0x7f13f819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 -- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 msgr2=0x7f13f819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fcd37640 1 -- 172.21.15.77:0/3029068102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f819f670 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.987+0000 7f13fd538640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13fcd37640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 0x7f13f819ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005940 tx=0x7f13e80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13ee7fc640 1 -- 172.21.15.77:0/3029068102 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e803f030 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f819f900 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13ee7fc640 1 -- 172.21.15.77:0/3029068102 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e8003ce0 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f81a7240 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.988+0000 7f13ee7fc640 1 -- 172.21.15.77:0/3029068102 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8035070 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.990+0000 7f13ee7fc640 1 -- 172.21.15.77:0/3029068102 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f13e8042030 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.991+0000 7f13ee7fc640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 0x7f13e40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.991+0000 7f13ee7fc640 1 -- 172.21.15.77:0/3029068102 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f13e80f07e0 con 0x7f13f81012c0 2024-09-15T22:13:41.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.991+0000 7f13fd538640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 0x7f13e40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.992+0000 7f13fd538640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 0x7f13e40a5a20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005fd0 tx=0x7f13f400b480 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.992+0000 7f1401b75740 1 Processor -- start 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.992+0000 7f1401b75740 1 -- start start 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dd4a0 0x5583fe6dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5583fe6432e0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583fe5eace0 con 0x5583fe6dddc0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f1401b75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fe63f2a0 con 0x5583fe6e8680 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fdd39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60970/0 (socket says 172.21.15.77:60970) 2024-09-15T22:13:41.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 -- 172.21.15.77:0/4191579008 learned_addr learned my addr 172.21.15.77:0/4191579008 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fdd39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52042/0 (socket says 172.21.15.77:52042) 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 -- 172.21.15.77:0/4191579008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 msgr2=0x5583fe6eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 -- 172.21.15.77:0/4191579008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dd4a0 msgr2=0x5583fe6dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dd4a0 0x5583fe6dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.993+0000 7f13fcd37640 1 -- 172.21.15.77:0/4191579008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fe5cf440 con 0x5583fe6dddc0 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.994+0000 7f13fcd37640 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b8f10 tx=0x7f13e80b81e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58f6f6f217f6ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.994+0000 7f13e37fe640 1 -- 172.21.15.77:0/4191579008 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049040 con 0x5583fe6dddc0 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.994+0000 7f13e37fe640 1 -- 172.21.15.77:0/4191579008 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80dc070 con 0x5583fe6dddc0 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.994+0000 7f13e37fe640 1 -- 172.21.15.77:0/4191579008 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80f0440 con 0x5583fe6dddc0 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 -- 172.21.15.77:0/4191579008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 msgr2=0x5583fe6e8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b8f10 tx=0x7f13e80b81e0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 -- 172.21.15.77:0/4191579008 shutdown_connections 2024-09-15T22:13:41.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e8140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 --2- 172.21.15.77:0/4191579008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dd4a0 0x5583fe6dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 -- 172.21.15.77:0/4191579008 >> 172.21.15.77:0/4191579008 conn(0x5583fe6d9a30 msgr2=0x5583fe6d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 -- 172.21.15.77:0/4191579008 shutdown_connections 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.995+0000 7f1401b75740 1 -- 172.21.15.77:0/4191579008 wait complete. 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 -- start start 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 Processor -- start 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe6e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f13fdd39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe6e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60986/0 (socket says 172.21.15.77:60986) 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f13fcd37640 1 -- 172.21.15.77:0/1147012289 learned_addr learned my addr 172.21.15.77:0/1147012289 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.996+0000 7f1401b75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5583fe6eb010 con 0x5583fe6e8680 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583fe5fe050 con 0x5583fe6dddc0 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5583fe6eb190 con 0x5583fe6dd4a0 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f1401b75740 10 client.0 _ll_register_callbacks cb 0x5583fe63e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 -- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 msgr2=0x5583fe6e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 -- 172.21.15.77:0/1147012289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 msgr2=0x5583fe6e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe6e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fe3b43c0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fdd39640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe6e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fcd37640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:41.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13fd538640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e80b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13f4011b40 tx=0x7f13f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17eef831b415aea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.997+0000 7f13e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f401cdd0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f13f4027480 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80dc070 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13f4027680 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f1401b75740 10 client.25318 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583fe6f6a40 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80dc070 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f13f401e9e0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f1401b75740 10 client.25318 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.998+0000 7f1401b75740 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f1401b75740 20 client.25318 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe6f8150 0x5583fe6fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f1401b75740 4 client.25318 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583fe650850 con 0x5583fe6f8150 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f13fcd37640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe6f8150 0x5583fe6fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f13fcd37640 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe6f8150 0x5583fe6fa5a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7872aabd7e861712 server_cookie=f99768faa0809bb6 in_seq=0 out_seq=0 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:40.999+0000 7f13e0ff9640 10 client.25318 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.000+0000 7f13e0ff9640 1 -- 172.21.15.77:0/1147012289 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583fe650850 con 0x5583fe6f8150 2024-09-15T22:13:41.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.000+0000 7f13e0ff9640 10 client.25318 handle_command_reply: tid=0 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.000+0000 7f1401b75740 1 client.25318 shutdown 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.000+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 msgr2=0x5583fe6e80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e80b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13f4011b40 tx=0x7f13f40046c0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 shutdown_connections 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe6f8150 0x5583fe6fa5a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe6e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 --2- 172.21.15.77:0/1147012289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e80b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 >> 172.21.15.77:0/1147012289 conn(0x5583fe6d9a30 msgr2=0x5583fe6df760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f13e0ff9640 0 client.25318 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 shutdown_connections 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 1 -- 172.21.15.77:0/1147012289 wait complete. 2024-09-15T22:13:41.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.001+0000 7f1401b75740 20 client.25318 trim_cache size 0 max 16384 2024-09-15T22:13:41.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 Processor -- start 2024-09-15T22:13:41.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 -- start start 2024-09-15T22:13:41.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81b1130 con 0x5583fe6e8680 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81b2c70 con 0x5583fe6dddc0 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f1401b75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81b1800 con 0x5583fe6dd4a0 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f13fdd39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f13fdd39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45036/0 (socket says 172.21.15.77:45036) 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f13fdd39640 1 -- 172.21.15.77:0/2211043436 learned_addr learned my addr 172.21.15.77:0/2211043436 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.034+0000 7f13fd538640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 -- 172.21.15.77:0/2211043436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 msgr2=0x5583fe6e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 -- 172.21.15.77:0/2211043436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 msgr2=0x5583fe7003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 -- 172.21.15.77:0/2211043436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fe6e5e60 con 0x5583fe6dddc0 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fdd39640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6ffe70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005c20 tx=0x7f13e80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59dc6e96c6509f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13e2ffd640 1 -- 172.21.15.77:0/2211043436 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80da070 con 0x5583fe6dddc0 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13e2ffd640 1 -- 172.21.15.77:0/2211043436 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80dc040 con 0x5583fe6dddc0 2024-09-15T22:13:41.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.035+0000 7f13e2ffd640 1 -- 172.21.15.77:0/2211043436 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049040 con 0x5583fe6dddc0 2024-09-15T22:13:41.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 -- 172.21.15.77:0/2211043436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 msgr2=0x5583fe6ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6ffe70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005c20 tx=0x7f13e80045e0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 -- 172.21.15.77:0/2211043436 shutdown_connections 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6e8680 0x5583fe7003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dddc0 0x5583fe6ffe70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 --2- 172.21.15.77:0/2211043436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6dd4a0 0x5583fe6e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 -- 172.21.15.77:0/2211043436 >> 172.21.15.77:0/2211043436 conn(0x5583fe6d9a30 msgr2=0x5583fe6e4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.036+0000 7f1401b75740 1 -- 172.21.15.77:0/2211043436 shutdown_connections 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.037+0000 7f1401b75740 1 -- 172.21.15.77:0/2211043436 wait complete. 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.037+0000 7f1401b75740 1 -- start start 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.037+0000 7f1401b75740 1 Processor -- start 2024-09-15T22:13:41.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.037+0000 7f1401b75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 0x5583fe6ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f1401b75740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fcd37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45038/0 (socket says 172.21.15.77:45038) 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fcd37640 1 -- 172.21.15.77:0/2818274651 learned_addr learned my addr 172.21.15.77:0/2818274651 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81b25a0 con 0x5583fe6dddc0 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81b1ed0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81af110 con 0x5583fe6e8680 2024-09-15T22:13:41.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 0x5583fe6ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fdd39640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 -- 172.21.15.77:0/2818274651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 msgr2=0x5583fe6edb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6edb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 -- 172.21.15.77:0/2818274651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 msgr2=0x5583fe6ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fd538640 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fe769920 con 0x5583fe6dd4a0 2024-09-15T22:13:41.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.038+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f1401b75740 10 client.0 _ll_register_callbacks cb 0x7f13f4011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80dc040 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f13fd538640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 0x5583fe6ed0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13f401d3b0 tx=0x7f13f401db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c24dc1f4f4bd85f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f13e0ff9640 10 client.25352 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f13e0ff9640 1 -- 172.21.15.77:0/2818274651 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4011720 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f13e0ff9640 1 -- 172.21.15.77:0/2818274651 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f13f401dc70 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f13e0ff9640 1 -- 172.21.15.77:0/2818274651 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13f4004020 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f1401b75740 10 client.25352 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.039+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5583fe76a9b0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f13e0ff9640 1 -- 172.21.15.77:0/2818274651 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f13f40041c0 con 0x5583fe6dd4a0 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 10 client.25352 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 10 client.25352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:41.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 20 client.25352 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe76b0a0 0x5583fe76d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 4 client.25352 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583fe64ad60 con 0x5583fe76b0a0 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe76b0a0 0x5583fe76d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f13e0ff9640 10 client.25352 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.040+0000 7f13fcd37640 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe76b0a0 0x5583fe76d560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7702794cadb014 server_cookie=7b048f50e4a9a6d8 in_seq=0 out_seq=0 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f13e0ff9640 1 -- 172.21.15.77:0/2818274651 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5583fe64ad60 con 0x5583fe76b0a0 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f13e0ff9640 10 client.25352 handle_command_reply: tid=0 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 client.25352 shutdown 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 msgr2=0x5583fe6ed0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 0x5583fe6ed0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13f401d3b0 tx=0x7f13f401db00 comp rx=0 tx=0).stop 2024-09-15T22:13:41.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 shutdown_connections 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5583fe76b0a0 0x5583fe76d560 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f13e0ff9640 0 client.25352 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583fe6e8680 0x5583fe6edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583fe6dddc0 0x5583fe6ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 --2- 172.21.15.77:0/2818274651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583fe6dd4a0 0x5583fe6ed0f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 >> 172.21.15.77:0/2818274651 conn(0x5583fe6d9a30 msgr2=0x5583fe6d8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 shutdown_connections 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 1 -- 172.21.15.77:0/2818274651 wait complete. 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.041+0000 7f1401b75740 20 client.25352 trim_cache size 0 max 16384 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:41.050 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:13:41.051 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:13:41.051 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:41.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 msgr2=0x7f13e40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 0x7f13e40a5a20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005fd0 tx=0x7f13f400b480 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 msgr2=0x7f13f819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 0x7f13f819ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13e8005940 tx=0x7f13e80042f0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 shutdown_connections 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f13e40a3560 0x7f13e40a5a20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13f810e4c0 0x7f13f819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f81012c0 0x7f13f819ac00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 --2- 172.21.15.77:0/3029068102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8100910 0x7f13f819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 >> 172.21.15.77:0/3029068102 conn(0x7f13f80fc780 msgr2=0x7f13f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 shutdown_connections 2024-09-15T22:13:41.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:41.043+0000 7f13ff79c640 1 -- 172.21.15.77:0/3029068102 wait complete. 2024-09-15T22:13:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:40 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:13:41.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:40 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:13:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:40 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:13:41.873 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:13:41.874 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:42.229 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:42.301 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:41 smithi077 ceph-mon[30980]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:42.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:41 smithi196 ceph-mon[35109]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:41 smithi117 ceph-mon[35021]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-15T22:13:42.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:42.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 -- 172.21.15.77:0/3326533446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c102570 msgr2=0x7f753c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 --2- 172.21.15.77:0/3326533446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c102570 0x7f753c1029b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02a480 comp rx=0 tx=0).stop 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 -- 172.21.15.77:0/3326533446 shutdown_connections 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 --2- 172.21.15.77:0/3326533446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 --2- 172.21.15.77:0/3326533446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c102570 0x7f753c1029b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 --2- 172.21.15.77:0/3326533446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.763+0000 7f7543fb8640 1 -- 172.21.15.77:0/3326533446 >> 172.21.15.77:0/3326533446 conn(0x7f753c0fc7d0 msgr2=0x7f753c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.764+0000 7f7543fb8640 1 -- 172.21.15.77:0/3326533446 shutdown_connections 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.765+0000 7f7543fb8640 1 -- 172.21.15.77:0/3326533446 wait complete. 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.766+0000 7f7543fb8640 1 Processor -- start 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.766+0000 7f7543fb8640 1 -- start start 2024-09-15T22:13:42.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.766+0000 7f7543fb8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7543fb8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7541d54640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7543fb8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 0x7f753c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7541d54640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52096/0 (socket says 172.21.15.77:52096) 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7541553640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7541553640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:32784/0 (socket says 172.21.15.77:32784) 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7541d54640 1 -- 172.21.15.77:0/4130264609 learned_addr learned my addr 172.21.15.77:0/4130264609 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7542555640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 0x7f753c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f753c110fd0 con 0x7f753c19bfd0 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f753c110e50 con 0x7f753c102ef0 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.767+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f753c111150 con 0x7f753c100e20 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 -- 172.21.15.77:0/4130264609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 msgr2=0x7f753c197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:42.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 -- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 msgr2=0x7f753c19c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 0x7f753c19c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19ca90 con 0x7f753c102ef0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7542555640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 0x7f753c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541d54640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.768+0000 7f7541553640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f752c008000 tx=0x7f752c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.769+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c03f020 con 0x7f753c102ef0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.769+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19ccc0 con 0x7f753c102ef0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.769+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c031c30 con 0x7f753c102ef0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.769+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c06a140 con 0x7f753c102ef0 2024-09-15T22:13:42.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.770+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c035070 con 0x7f753c102ef0 2024-09-15T22:13:42.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.770+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c1021b0 con 0x7f753c102ef0 2024-09-15T22:13:42.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.772+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f752c042020 con 0x7f753c102ef0 2024-09-15T22:13:42.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.773+0000 7f7532ffd640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 0x7f75280a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:42.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.773+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f752c0f09f0 con 0x7f753c102ef0 2024-09-15T22:13:42.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.774+0000 7f7541d54640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 0x7f75280a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:42.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.775+0000 7f7541d54640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 0x7f75280a5900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7538011500 tx=0x7f7538005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:42.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:42.778+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f752c049020 con 0x7f753c102ef0 2024-09-15T22:13:43.005 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:43.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.143+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753c198a60 con 0x7f753c102ef0 2024-09-15T22:13:43.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.144+0000 7f7532ffd640 1 -- 172.21.15.77:0/4130264609 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f752c0bce50 con 0x7f753c102ef0 2024-09-15T22:13:43.147 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:13:43.147 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:13:43.147 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 msgr2=0x7f75280a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 0x7f75280a5900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7538011500 tx=0x7f7538005f90 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 msgr2=0x7f753c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f752c008000 tx=0x7f752c003c10 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 shutdown_connections 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75280a3440 0x7f75280a5900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f753c19bfd0 0x7f753c19c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f753c102ef0 0x7f753c197c40 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f752c008000 tx=0x7f752c003c10 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 --2- 172.21.15.77:0/4130264609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f753c100e20 0x7f753c197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 >> 172.21.15.77:0/4130264609 conn(0x7f753c0fc7d0 msgr2=0x7f753c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:43.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 shutdown_connections 2024-09-15T22:13:43.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.146+0000 7f7543fb8640 1 -- 172.21.15.77:0/4130264609 wait complete. 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 -- 172.21.15.77:0/2705129432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102ef0 msgr2=0x7f1d9810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/2705129432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102ef0 0x7f1d9810da40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009ca0 tx=0x7f1d9002a950 comp rx=0 tx=0).stop 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 -- 172.21.15.77:0/2705129432 shutdown_connections 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/2705129432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102ef0 0x7f1d9810da40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/2705129432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102570 0x7f1d981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/2705129432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 0x7f1d98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 -- 172.21.15.77:0/2705129432 >> 172.21.15.77:0/2705129432 conn(0x7f1d980fc7d0 msgr2=0x7f1d980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.702+0000 7f1d9ee16640 1 -- 172.21.15.77:0/2705129432 shutdown_connections 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.703+0000 7f1d9ee16640 1 -- 172.21.15.77:0/2705129432 wait complete. 2024-09-15T22:13:43.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 Processor -- start 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 -- start start 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 0x7f1d981992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 0x7f1d981a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d97fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45072/0 (socket says 172.21.15.77:45072) 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d97fff640 1 -- 172.21.15.77:0/1099501528 learned_addr learned my addr 172.21.15.77:0/1099501528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98110f80 con 0x7f1d98102570 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98110e00 con 0x7f1d98100e20 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98111100 con 0x7f1d98102ef0 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.704+0000 7f1d9cbb2640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 0x7f1d981992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d9d3b3640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 0x7f1d981a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:43.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 -- 172.21.15.77:0/1099501528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 msgr2=0x7f1d981a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 0x7f1d981a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 -- 172.21.15.77:0/1099501528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 msgr2=0x7f1d981992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 0x7f1d981992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981a4870 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d9d3b3640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 0x7f1d981a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d97fff640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d950 tx=0x7f1d8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017030 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d84004020 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.705+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d981970a0 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.706+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840047a0 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.706+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981975e0 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.707+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98101200 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.708+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f1d840041d0 con 0x7f1d98102570 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.709+0000 7f1d95ffb640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 0x7f1d880a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:43.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.709+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1d840cfbf0 con 0x7f1d98102570 2024-09-15T22:13:43.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.709+0000 7f1d9cbb2640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 0x7f1d880a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:43.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.710+0000 7f1d9cbb2640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 0x7f1d880a5aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005d00 tx=0x7f1d8c00f820 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:43.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.713+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8409c050 con 0x7f1d98102570 2024-09-15T22:13:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.932+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d98197890 con 0x7f1d98102570 2024-09-15T22:13:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.933+0000 7f1d95ffb640 1 -- 172.21.15.77:0/1099501528 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11989 (secure 0 0 0) 0x7f1d8409b950 con 0x7f1d98102570 2024-09-15T22:13:43.936 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:13:43.936 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":30,"btime":"2024-09-15T22:12:23:161865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:12:23.111653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:13:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 30 2024-09-15T22:13:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 msgr2=0x7f1d880a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 0x7f1d880a5aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005d00 tx=0x7f1d8c00f820 comp rx=0 tx=0).stop 2024-09-15T22:13:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 msgr2=0x7f1d981997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d950 tx=0x7f1d8400de10 comp rx=0 tx=0).stop 2024-09-15T22:13:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 shutdown_connections 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1d880a35e0 0x7f1d880a5aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d98102ef0 0x7f1d981a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d98102570 0x7f1d981997f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 --2- 172.21.15.77:0/1099501528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d98100e20 0x7f1d981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.936+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 >> 172.21.15.77:0/1099501528 conn(0x7f1d980fc7d0 msgr2=0x7f1d98106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.937+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 shutdown_connections 2024-09-15T22:13:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:43.937+0000 7f1d9ee16640 1 -- 172.21.15.77:0/1099501528 wait complete. 2024-09-15T22:13:44.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:43 smithi077 ceph-mon[30980]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:44.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:43 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4130264609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:44.353 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:13:44.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:43 smithi196 ceph-mon[35109]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:44.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:43 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4130264609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:43 smithi117 ceph-mon[35021]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:43 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4130264609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:45.030 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:44 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1099501528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:45.057 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 7d71dbcf --release --await' 2024-09-15T22:13:45.058 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 7d71dbcf --release --await 2024-09-15T22:13:45.169 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:45.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:44 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1099501528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:44 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1099501528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 -- 172.21.15.77:0/2310999744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102d30 msgr2=0x7fcc44103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/2310999744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102d30 0x7fcc44103110 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009930 tx=0x7fcc3402a420 comp rx=0 tx=0).stop 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 -- 172.21.15.77:0/2310999744 shutdown_connections 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/2310999744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441082a0 0x7fcc4410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/2310999744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc441036e0 0x7fcc44107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/2310999744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102d30 0x7fcc44103110 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 -- 172.21.15.77:0/2310999744 >> 172.21.15.77:0/2310999744 conn(0x7fcc440fc7d0 msgr2=0x7fcc440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 -- 172.21.15.77:0/2310999744 shutdown_connections 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.686+0000 7fcc49fc2640 1 -- 172.21.15.77:0/2310999744 wait complete. 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 Processor -- start 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 -- start start 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 0x7fcc441985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 0x7fcc44198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441120a0 con 0x7fcc441082a0 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44111f20 con 0x7fcc44102d30 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc49fc2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112220 con 0x7fcc441036e0 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc43fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 0x7fcc44198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 0x7fcc44198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52122/0 (socket says 172.21.15.77:52122) 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc43fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45100/0 (socket says 172.21.15.77:45100) 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc42ffd640 1 -- 172.21.15.77:0/3439637626 learned_addr learned my addr 172.21.15.77:0/3439637626 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.687+0000 7fcc437fe640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 0x7fcc441985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 -- 172.21.15.77:0/3439637626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 msgr2=0x7fcc44198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 0x7fcc44198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 -- 172.21.15.77:0/3439637626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 msgr2=0x7fcc441985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 0x7fcc441985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 -- 172.21.15.77:0/3439637626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4419d570 con 0x7fcc441082a0 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc437fe640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 0x7fcc441985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc43fff640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00e940 tx=0x7fcc3c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc40ff9640 1 -- 172.21.15.77:0/3439637626 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c004d20 con 0x7fcc441082a0 2024-09-15T22:13:45.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc40ff9640 1 -- 172.21.15.77:0/3439637626 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc3c005020 con 0x7fcc441082a0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4419d860 con 0x7fcc441082a0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc40ff9640 1 -- 172.21.15.77:0/3439637626 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0116b0 con 0x7fcc441082a0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.688+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc441a5140 con 0x7fcc441082a0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.690+0000 7fcc40ff9640 1 -- 172.21.15.77:0/3439637626 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fcc3c0054d0 con 0x7fcc441082a0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 Processor -- start 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 -- start start 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bad5d0 0x55c813bada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c813b132f0 con 0x55c813bb8370 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c813abacf0 con 0x55c813bad5d0 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c813b0f2b0 con 0x55c813baeb40 2024-09-15T22:13:45.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc40ff9640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 0x7fcc2c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45102/0 (socket says 172.21.15.77:45102) 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.691+0000 7fcc42ffd640 1 -- 172.21.15.77:0/528709862 learned_addr learned my addr 172.21.15.77:0/528709862 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc437fe640 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bad5d0 0x55c813bada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc43fff640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 0x7fcc2c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 -- 172.21.15.77:0/528709862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 msgr2=0x55c813bad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 -- 172.21.15.77:0/528709862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bad5d0 msgr2=0x55c813bada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bad5d0 0x55c813bada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 -- 172.21.15.77:0/528709862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c813a9f3a0 con 0x55c813bb8370 2024-09-15T22:13:45.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc40ff9640 1 -- 172.21.15.77:0/3439637626 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcc3c0d0720 con 0x7fcc441082a0 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800afc0 tx=0x7fcc3800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aabd253424cfaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc43fff640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 0x7fcc2c0a5a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0cd950 tx=0x7fcc3c06e200 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc29ffb640 1 -- 172.21.15.77:0/528709862 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800fe10 con 0x55c813bb8370 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc29ffb640 1 -- 172.21.15.77:0/528709862 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38005500 con 0x55c813bb8370 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.692+0000 7fcc29ffb640 1 -- 172.21.15.77:0/528709862 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38020630 con 0x55c813bb8370 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- 172.21.15.77:0/528709862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 msgr2=0x55c813bba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800afc0 tx=0x7fcc3800a860 comp rx=0 tx=0).stop 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- 172.21.15.77:0/528709862 shutdown_connections 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bb8370 0x55c813bba760 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bad5d0 0x55c813bada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/528709862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- 172.21.15.77:0/528709862 >> 172.21.15.77:0/528709862 conn(0x55c813b76310 msgr2=0x55c813ba9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- 172.21.15.77:0/528709862 shutdown_connections 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- 172.21.15.77:0/528709862 wait complete. 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.693+0000 7fcc4c39b740 1 -- start start 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 Processor -- start 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bb5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c813bbb070 con 0x55c813bad5d0 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c813ace060 con 0x55c813baeb40 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c813bbb1f0 con 0x55c813bb8370 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc4c39b740 10 client.0 _ll_register_callbacks cb 0x55c8138218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45118/0 (socket says 172.21.15.77:45118) 2024-09-15T22:13:45.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 -- 172.21.15.77:0/2770515307 learned_addr learned my addr 172.21.15.77:0/2770515307 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bb5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc437fe640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 -- 172.21.15.77:0/2770515307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 msgr2=0x55c813bb5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bb5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 -- 172.21.15.77:0/2770515307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 msgr2=0x55c813bb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.694+0000 7fcc42ffd640 1 -- 172.21.15.77:0/2770515307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8138843c0 con 0x55c813bad5d0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcc38004e10 tx=0x7fcc3800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b56e29c7d470cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38004fd0 con 0x55c813bad5d0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcc38005170 con 0x55c813bad5d0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc38005500 con 0x55c813bad5d0 2024-09-15T22:13:45.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc38037050 con 0x55c813bad5d0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c813bc6b40 con 0x55c813bad5d0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38005500 con 0x55c813bad5d0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc3801f030 con 0x55c813bad5d0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 10 client.15969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 20 client.15969 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813bc7e90 0x55c813bca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 4 client.15969 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c813b20860 con 0x55c813bc7e90 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.695+0000 7fcc437fe640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813bc7e90 0x55c813bca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.696+0000 7fcc22ffd640 10 client.15969 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.696+0000 7fcc437fe640 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813bc7e90 0x55c813bca280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81c342baed990a0e server_cookie=64506a5825ab7494 in_seq=0 out_seq=0 2024-09-15T22:13:45.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.696+0000 7fcc22ffd640 1 -- 172.21.15.77:0/2770515307 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c813b20860 con 0x55c813bc7e90 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.696+0000 7fcc22ffd640 10 client.15969 handle_command_reply: tid=0 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.696+0000 7fcc4c39b740 1 client.15969 shutdown 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 msgr2=0x55c813bb7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcc38004e10 tx=0x7fcc3800a380 comp rx=0 tx=0).stop 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 shutdown_connections 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813bc7e90 0x55c813bca280 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bb5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/2770515307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb7d80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 >> 172.21.15.77:0/2770515307 conn(0x55c813b76310 msgr2=0x55c813baf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc22ffd640 0 client.15969 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 shutdown_connections 2024-09-15T22:13:45.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 1 -- 172.21.15.77:0/2770515307 wait complete. 2024-09-15T22:13:45.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.697+0000 7fcc4c39b740 20 client.15969 trim_cache size 0 max 16384 2024-09-15T22:13:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 Processor -- start 2024-09-15T22:13:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 -- start start 2024-09-15T22:13:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bc4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bb8370 0x55c813bb4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441acdb0 con 0x55c813bad5d0 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441ae8f0 con 0x55c813bb8370 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441ad480 con 0x55c813baeb40 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45132/0 (socket says 172.21.15.77:45132) 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc42ffd640 1 -- 172.21.15.77:0/4086165623 learned_addr learned my addr 172.21.15.77:0/4086165623 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.711+0000 7fcc437fe640 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bc4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bb8370 0x55c813bb4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 -- 172.21.15.77:0/4086165623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 msgr2=0x55c813bc4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bc4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 -- 172.21.15.77:0/4086165623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bb8370 msgr2=0x55c813bb4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bb8370 0x55c813bb4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 -- 172.21.15.77:0/4086165623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c813bb59b0 con 0x55c813bad5d0 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcc380058c0 tx=0x7fcc38031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e60eac1a026ebf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc297fa640 1 -- 172.21.15.77:0/4086165623 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38015850 con 0x55c813bad5d0 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc297fa640 1 -- 172.21.15.77:0/4086165623 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38040070 con 0x55c813bad5d0 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.712+0000 7fcc297fa640 1 -- 172.21.15.77:0/4086165623 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38031d20 con 0x55c813bad5d0 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- 172.21.15.77:0/4086165623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 msgr2=0x55c813ba9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcc380058c0 tx=0x7fcc38031650 comp rx=0 tx=0).stop 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- 172.21.15.77:0/4086165623 shutdown_connections 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813bb8370 0x55c813bb4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813baeb40 0x55c813bc4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/4086165623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813ba9da0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- 172.21.15.77:0/4086165623 >> 172.21.15.77:0/4086165623 conn(0x55c813b76310 msgr2=0x55c813bb3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- 172.21.15.77:0/4086165623 shutdown_connections 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- 172.21.15.77:0/4086165623 wait complete. 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.713+0000 7fcc4c39b740 1 -- start start 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 Processor -- start 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bc0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bc09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441ae220 con 0x55c813bad5d0 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441adb50 con 0x55c813baeb40 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441aad90 con 0x55c813bb8370 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc4c39b740 10 client.0 _ll_register_callbacks cb 0x7fcc38015f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45140/0 (socket says 172.21.15.77:45140) 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc42ffd640 1 -- 172.21.15.77:0/1712672190 learned_addr learned my addr 172.21.15.77:0/1712672190 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc437fe640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bc0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.714+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bc09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 -- 172.21.15.77:0/1712672190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 msgr2=0x55c813bc09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bc09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 -- 172.21.15.77:0/1712672190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 msgr2=0x55c813bc0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bc0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 -- 172.21.15.77:0/1712672190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c813bc0f80 con 0x55c813bad5d0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc437fe640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bc0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc42ffd640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800a9f0 tx=0x7fcc38005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de9b667bde85f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3804a060 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcc38031370 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc38040070 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc38031930 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc4c39b740 10 client.15975 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.715+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c813c31900 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38040070 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc3801f040 con 0x55c813bad5d0 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 10 client.15975 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 10 client.15975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 20 client.15975 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813c31e60 0x55c813c34250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 4 client.15975 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c813b1ad70 con 0x55c813c31e60 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc437fe640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813c31e60 0x55c813c34250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc22ffd640 10 client.15975 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.716+0000 7fcc437fe640 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813c31e60 0x55c813c34250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d2248cf372d0315 server_cookie=121cb6e86de62cf0 in_seq=0 out_seq=0 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc22ffd640 1 -- 172.21.15.77:0/1712672190 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c813b1ad70 con 0x55c813c31e60 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc22ffd640 10 client.15975 handle_command_reply: tid=0 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 client.15975 shutdown 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 msgr2=0x55c813bb2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800a9f0 tx=0x7fcc38005b10 comp rx=0 tx=0).stop 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 shutdown_connections 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c813c31e60 0x55c813c34250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c813bb8370 0x55c813bc09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc22ffd640 0 client.15975 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:13:45.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c813baeb40 0x55c813bc0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 --2- 172.21.15.77:0/1712672190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c813bad5d0 0x55c813bb2900 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 >> 172.21.15.77:0/1712672190 conn(0x55c813b76310 msgr2=0x55c813bb4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 shutdown_connections 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 1 -- 172.21.15.77:0/1712672190 wait complete. 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.717+0000 7fcc4c39b740 20 client.15975 trim_cache size 0 max 16384 2024-09-15T22:13:45.730 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 msgr2=0x7fcc2c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 0x7fcc2c0a5a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0cd950 tx=0x7fcc3c06e200 comp rx=0 tx=0).stop 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 msgr2=0x7fcc4419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00e940 tx=0x7fcc3c00ee00 comp rx=0 tx=0).stop 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 shutdown_connections 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcc2c0a3560 0x7fcc2c0a5a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc441082a0 0x7fcc4419ce90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc441036e0 0x7fcc44198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 --2- 172.21.15.77:0/3439637626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc44102d30 0x7fcc441985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:45.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 >> 172.21.15.77:0/3439637626 conn(0x7fcc440fc7d0 msgr2=0x7fcc44105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:45.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 shutdown_connections 2024-09-15T22:13:45.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:45.718+0000 7fcc49fc2640 1 -- 172.21.15.77:0/3439637626 wait complete. 2024-09-15T22:13:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:45 smithi077 ceph-mon[30980]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:45 smithi196 ceph-mon[35109]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:45 smithi117 ceph-mon[35021]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:47.810 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:13:47.881 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:48.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:47 smithi077 ceph-mon[30980]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:48.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:47 smithi196 ceph-mon[35109]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:48.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:47 smithi117 ceph-mon[35021]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:48.389 INFO:tasks.workunit.client.0.smithi077.stderr:Updating files: 95% (12346/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 -- 172.21.15.77:0/2224243371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 msgr2=0x7f97e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/2224243371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0101200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009900 tx=0x7f97dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 -- 172.21.15.77:0/2224243371 shutdown_connections 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/2224243371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102ef0 0x7f97e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/2224243371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97e0102570 0x7f97e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/2224243371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0101200 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 -- 172.21.15.77:0/2224243371 >> 172.21.15.77:0/2224243371 conn(0x7f97e00fc7d0 msgr2=0x7f97e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 -- 172.21.15.77:0/2224243371 shutdown_connections 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.439+0000 7f97e7ae3640 1 -- 172.21.15.77:0/2224243371 wait complete. 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 Processor -- start 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 -- start start 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102570 0x7f97e0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97e0102ef0 0x7f97e01a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0110fa0 con 0x7f97e0100e20 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0110e20 con 0x7f97e0102ef0 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e7ae3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0111120 con 0x7f97e0102570 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45160/0 (socket says 172.21.15.77:45160) 2024-09-15T22:13:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.440+0000 7f97e587f640 1 -- 172.21.15.77:0/3115977070 learned_addr learned my addr 172.21.15.77:0/3115977070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e507e640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102570 0x7f97e0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 -- 172.21.15.77:0/3115977070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102570 msgr2=0x7f97e0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102570 0x7f97e0199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 -- 172.21.15.77:0/3115977070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97e0102ef0 msgr2=0x7f97e01a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97e0102ef0 0x7f97e01a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 -- 172.21.15.77:0/3115977070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e01a48f0 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e587f640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006010 tx=0x7f97dc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97d2ffd640 1 -- 172.21.15.77:0/3115977070 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc03f070 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97d2ffd640 1 -- 172.21.15.77:0/3115977070 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97dc031e30 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e01a4ac0 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97d2ffd640 1 -- 172.21.15.77:0/3115977070 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc035030 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.441+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e01a4f70 con 0x7f97e0100e20 2024-09-15T22:13:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.443+0000 7f97d2ffd640 1 -- 172.21.15.77:0/3115977070 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f97dc042070 con 0x7f97e0100e20 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 Processor -- start 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 -- start start 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd8742440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd874cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfd86a7620 con 0x55bfd874d240 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bfd864ee50 con 0x55bfd8742060 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bfd86a35e0 con 0x55bfd8742980 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97d2ffd640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 0x7f97cc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.444+0000 7f97e507e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45172/0 (socket says 172.21.15.77:45172) 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 -- 172.21.15.77:0/3837168339 learned_addr learned my addr 172.21.15.77:0/3837168339 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e6080640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 0x7f97cc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e6080640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd8742440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97d2ffd640 1 -- 172.21.15.77:0/3115977070 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f97dc0f0d90 con 0x7f97e0100e20 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e587f640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd874cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 -- 172.21.15.77:0/3837168339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 msgr2=0x55bfd874cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd874cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 -- 172.21.15.77:0/3837168339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 msgr2=0x55bfd8742440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd8742440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 -- 172.21.15.77:0/3837168339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd863fd40 con 0x55bfd874d240 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e587f640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd874cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e6080640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd8742440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e6080640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 0x7f97cc0a5aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97d8013730 tx=0x7f97d8009bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97e507e640 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f97d400a8e0 tx=0x7f97d400ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea35c196b59dbb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97cbfff640 1 -- 172.21.15.77:0/3837168339 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4015070 con 0x55bfd874d240 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97cbfff640 1 -- 172.21.15.77:0/3837168339 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d400cb40 con 0x55bfd874d240 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.445+0000 7f97cbfff640 1 -- 172.21.15.77:0/3837168339 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d40185c0 con 0x55bfd874d240 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 -- 172.21.15.77:0/3837168339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 msgr2=0x55bfd874f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f97d400a8e0 tx=0x7f97d400ada0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 -- 172.21.15.77:0/3837168339 shutdown_connections 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd874f630 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd874cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/3837168339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd8742440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 -- 172.21.15.77:0/3837168339 >> 172.21.15.77:0/3837168339 conn(0x55bfd873e5f0 msgr2=0x55bfd873e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 -- 172.21.15.77:0/3837168339 shutdown_connections 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.446+0000 7f97e9ebc740 1 -- 172.21.15.77:0/3837168339 wait complete. 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 -- start start 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 Processor -- start 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd874a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 0x55bfd874aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfd874fe10 con 0x55bfd8742060 2024-09-15T22:13:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bfd862ff10 con 0x55bfd8742980 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bfd874ff90 con 0x55bfd874d240 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e9ebc740 10 client.0 _ll_register_callbacks cb 0x55bfd8653db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45184/0 (socket says 172.21.15.77:45184) 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 -- 172.21.15.77:0/131565981 learned_addr learned my addr 172.21.15.77:0/131565981 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e6080640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 0x55bfd874aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 -- 172.21.15.77:0/131565981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 msgr2=0x55bfd874aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e507e640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd874a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 0x55bfd874aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 -- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 msgr2=0x55bfd874a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd874a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e587f640 1 -- 172.21.15.77:0/131565981 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd83b6580 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.447+0000 7f97e6080640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 0x55bfd874aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e587f640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b8e70 tx=0x7f97dc0bc7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c885c8696c3d492c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc049040 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f97dc03b5f0 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d400cb40 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97dc0bd110 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfd8480960 con 0x55bfd8742060 2024-09-15T22:13:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d400cb40 con 0x55bfd8742060 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f97dc0c51f0 con 0x55bfd8742060 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 10 client.15987 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 20 client.15987 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd875cbc0 0x55bfd875f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 4 client.15987 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.448+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfd86b4b90 con 0x55bfd875cbc0 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.449+0000 7f97e507e640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd875cbc0 0x55bfd875f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.449+0000 7f97e507e640 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd875cbc0 0x55bfd875f030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd22538cbcb01cf4 server_cookie=f8ce09e2daa1d5f7 in_seq=0 out_seq=0 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.449+0000 7f97c97fa640 10 client.15987 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97c97fa640 1 -- 172.21.15.77:0/131565981 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfd86b4b90 con 0x55bfd875cbc0 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97c97fa640 10 client.15987 handle_command_reply: tid=0 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 client.15987 shutdown 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 msgr2=0x55bfd874a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0b8e70 tx=0x7f97dc0bc7e0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 shutdown_connections 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd875cbc0 0x55bfd875f030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd874d240 0x55bfd874aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd874a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/131565981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8742060 0x55bfd874a060 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97c97fa640 0 client.15987 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 >> 172.21.15.77:0/131565981 conn(0x55bfd873e5f0 msgr2=0x55bfd87442d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 shutdown_connections 2024-09-15T22:13:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 1 -- 172.21.15.77:0/131565981 wait complete. 2024-09-15T22:13:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.450+0000 7f97e9ebc740 20 client.15987 trim_cache size 0 max 16384 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.467+0000 7f97e9ebc740 1 Processor -- start 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.467+0000 7f97e9ebc740 1 -- start start 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd87648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd8764e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01aed90 con 0x55bfd874d240 2024-09-15T22:13:48.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b08d0 con 0x55bfd8742980 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01af460 con 0x55bfd8742060 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e587f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e587f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52200/0 (socket says 172.21.15.77:52200) 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e587f640 1 -- 172.21.15.77:0/1900385837 learned_addr learned my addr 172.21.15.77:0/1900385837 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd8764e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e507e640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd87648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 -- 172.21.15.77:0/1900385837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 msgr2=0x55bfd87441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 -- 172.21.15.77:0/1900385837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 msgr2=0x55bfd87648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd87648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e6080640 1 -- 172.21.15.77:0/1900385837 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd87427b0 con 0x55bfd874d240 2024-09-15T22:13:48.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.468+0000 7f97e587f640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.469+0000 7f97e6080640 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd8764e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f97d8005fd0 tx=0x7f97d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfbf62159cb1a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.469+0000 7f97cb7fe640 1 -- 172.21.15.77:0/1900385837 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d801d7f0 con 0x55bfd874d240 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.469+0000 7f97cb7fe640 1 -- 172.21.15.77:0/1900385837 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d801de70 con 0x55bfd874d240 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.469+0000 7f97cb7fe640 1 -- 172.21.15.77:0/1900385837 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d801f450 con 0x55bfd874d240 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- 172.21.15.77:0/1900385837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 msgr2=0x55bfd8764e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd8764e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f97d8005fd0 tx=0x7f97d8016000 comp rx=0 tx=0).stop 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- 172.21.15.77:0/1900385837 shutdown_connections 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd8764e10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742980 0x55bfd87648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/1900385837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742060 0x55bfd87441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- 172.21.15.77:0/1900385837 >> 172.21.15.77:0/1900385837 conn(0x55bfd873e5f0 msgr2=0x55bfd8743ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- 172.21.15.77:0/1900385837 shutdown_connections 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- 172.21.15.77:0/1900385837 wait complete. 2024-09-15T22:13:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 -- start start 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.470+0000 7f97e9ebc740 1 Processor -- start 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd87c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd87c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b0200 con 0x55bfd874d240 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01afb30 con 0x55bfd8742060 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01acd70 con 0x55bfd8742980 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e9ebc740 10 client.0 _ll_register_callbacks cb 0x7f97dc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e507e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e6080640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd87c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e507e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52202/0 (socket says 172.21.15.77:52202) 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e507e640 1 -- 172.21.15.77:0/2325270356 learned_addr learned my addr 172.21.15.77:0/2325270356 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.471+0000 7f97e587f640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd87c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 -- 172.21.15.77:0/2325270356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 msgr2=0x55bfd87c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 -- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 msgr2=0x55bfd87c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd87c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 -- 172.21.15.77:0/2325270356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd87c2070 con 0x55bfd874d240 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e587f640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd87c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e507e640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e6080640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd87c1870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f97d8011720 tx=0x7f97d80286b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ee86ab7bddfd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8037060 con 0x55bfd874d240 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f97d801d800 con 0x55bfd874d240 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d801de70 con 0x55bfd874d240 2024-09-15T22:13:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97d803e050 con 0x55bfd874d240 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e9ebc740 10 client.15993 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfd8754c50 con 0x55bfd874d240 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.472+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d801da00 con 0x55bfd874d240 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f97d801c040 con 0x55bfd874d240 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 10 client.15993 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 10 client.15993 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 20 client.15993 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd8755320 0x55bfd8755730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 4 client.15993 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7d71dbcf", "release": true, "await": true}] 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7d71dbcf", "release": true, "await": true}) -- 0x7f97d801de70 con 0x55bfd8755320 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e587f640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd8755320 0x55bfd8755730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:48.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97c97fa640 10 client.15993 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.473+0000 7f97e587f640 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd8755320 0x55bfd8755730 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ea191602844712 server_cookie=5681aa0e5b81f306 in_seq=0 out_seq=0 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.475+0000 7f97c97fa640 1 -- 172.21.15.77:0/2325270356 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f97d801de70 con 0x55bfd8755320 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.475+0000 7f97c97fa640 10 client.15993 handle_command_reply: tid=0 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.475+0000 7f97e9ebc740 1 client.15993 shutdown 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 msgr2=0x55bfd87c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd87c1870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f97d8011720 tx=0x7f97d80286b0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 shutdown_connections 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bfd8755320 0x55bfd8755730 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd874d240 0x55bfd87c1870 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfd8742980 0x55bfd87c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97c97fa640 0 client.15993 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 --2- 172.21.15.77:0/2325270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd8742060 0x55bfd87c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 >> 172.21.15.77:0/2325270356 conn(0x55bfd873e5f0 msgr2=0x55bfd873d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 shutdown_connections 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 1 -- 172.21.15.77:0/2325270356 wait complete. 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.476+0000 7f97e9ebc740 20 client.15993 trim_cache size 0 max 16384 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 30, 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 4, 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "7d71dbcf": { 2024-09-15T22:13:48.486 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 4, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 150.2, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:13:48.487 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 msgr2=0x7f97cc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 0x7f97cc0a5aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97d8013730 tx=0x7f97d8009bc0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 msgr2=0x7f97e0199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006010 tx=0x7f97dc003c80 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 shutdown_connections 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97cc0a35e0 0x7f97cc0a5aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97e0102ef0 0x7f97e01a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97e0102570 0x7f97e0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 --2- 172.21.15.77:0/3115977070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97e0100e20 0x7f97e0199330 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 >> 172.21.15.77:0/3115977070 conn(0x7f97e00fc7d0 msgr2=0x7f97e0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 shutdown_connections 2024-09-15T22:13:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:48.477+0000 7f97e7ae3640 1 -- 172.21.15.77:0/3115977070 wait complete. 2024-09-15T22:13:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T22:13:49.089 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-09-15T22:13:49.090 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-15T22:13:49.094 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T22:13:49.158 INFO:tasks.workunit.client.0.smithi077.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T22:13:49.160 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:13:49.161 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T22:13:49.392 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7d71dbcf', total seconds elapsed: 37.0 2024-09-15T22:13:49.993 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:49 smithi077 ceph-mon[30980]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 -- 172.21.15.77:0/2677169766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8106a80 msgr2=0x7f06c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 --2- 172.21.15.77:0/2677169766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8106a80 0x7f06c8106e60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009930 tx=0x7f06b802a420 comp rx=0 tx=0).stop 2024-09-15T22:13:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 -- 172.21.15.77:0/2677169766 shutdown_connections 2024-09-15T22:13:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 --2- 172.21.15.77:0/2677169766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 0x7f06c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 --2- 172.21.15.77:0/2677169766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8100a80 0x7f06c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 --2- 172.21.15.77:0/2677169766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8106a80 0x7f06c8106e60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 -- 172.21.15.77:0/2677169766 >> 172.21.15.77:0/2677169766 conn(0x7f06c80fc7a0 msgr2=0x7f06c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 -- 172.21.15.77:0/2677169766 shutdown_connections 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.991+0000 7f06cd849640 1 -- 172.21.15.77:0/2677169766 wait complete. 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 Processor -- start 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 -- start start 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 0x7f06c819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8106a80 0x7f06c819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8110e80 con 0x7f06c8100a80 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8110d00 con 0x7f06c8101420 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06cd849640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8111000 con 0x7f06c8106a80 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44642/0 (socket says 172.21.15.77:44642) 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 -- 172.21.15.77:0/3359500096 learned_addr learned my addr 172.21.15.77:0/3359500096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c67fc640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 0x7f06c819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c77fe640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8106a80 0x7f06c819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 -- 172.21.15.77:0/3359500096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8106a80 msgr2=0x7f06c819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8106a80 0x7f06c819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 -- 172.21.15.77:0/3359500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 msgr2=0x7f06c819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 0x7f06c819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.992+0000 7f06c6ffd640 1 -- 172.21.15.77:0/3359500096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c819f660 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009230 tx=0x7f06b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06cc847640 1 -- 172.21.15.77:0/3359500096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b803f030 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06cc847640 1 -- 172.21.15.77:0/3359500096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8003ad0 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c819f8f0 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06cc847640 1 -- 172.21.15.77:0/3359500096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8035070 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.993+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81a7230 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cc847640 1 -- 172.21.15.77:0/3359500096 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f06b8042030 con 0x7f06c8100a80 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 Processor -- start 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 -- start start 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757e2370 0x5595757e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55957573d2f0 con 0x5595757d75d0 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5595756e4cf0 con 0x5595757d8b40 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06cfc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595757392b0 con 0x5595757e2370 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44654/0 (socket says 172.21.15.77:44654) 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06c6ffd640 1 -- 172.21.15.77:0/309817590 learned_addr learned my addr 172.21.15.77:0/309817590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06c77fe640 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.996+0000 7f06c67fc640 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757e2370 0x5595757e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06cc847640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 0x7f06b00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c77fe640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 0x7f06b00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 -- 172.21.15.77:0/309817590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757e2370 msgr2=0x5595757e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757e2370 0x5595757e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 -- 172.21.15.77:0/309817590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 msgr2=0x5595757d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 -- 172.21.15.77:0/309817590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595756c93a0 con 0x5595757d75d0 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06cc847640 1 -- 172.21.15.77:0/3359500096 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f06b80f0820 con 0x7f06c8100a80 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ed990 tx=0x7f06b804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe20c3c6b9ab4355 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06c77fe640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 0x7f06b00a5a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c0004ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06adffb640 1 -- 172.21.15.77:0/309817590 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80bc0c0 con 0x5595757d75d0 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06adffb640 1 -- 172.21.15.77:0/309817590 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b80bcc80 con 0x5595757d75d0 2024-09-15T22:13:50.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.997+0000 7f06adffb640 1 -- 172.21.15.77:0/309817590 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b804a260 con 0x5595757d75d0 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- 172.21.15.77:0/309817590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 msgr2=0x5595757d7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06b80ed990 tx=0x7f06b804b000 comp rx=0 tx=0).stop 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- 172.21.15.77:0/309817590 shutdown_connections 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757e2370 0x5595757e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757d7a10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 --2- 172.21.15.77:0/309817590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- 172.21.15.77:0/309817590 >> 172.21.15.77:0/309817590 conn(0x5595757a0310 msgr2=0x5595757d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- 172.21.15.77:0/309817590 shutdown_connections 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- 172.21.15.77:0/309817590 wait complete. 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.998+0000 7f06cfc22740 1 -- start start 2024-09-15T22:13:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 Processor -- start 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757defe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757e2370 0x5595757df5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55957573d2f0 con 0x5595757d75d0 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5595756f8060 con 0x5595757e2370 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595757e4e30 con 0x5595757d8b40 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06cfc22740 10 client.0 _ll_register_callbacks cb 0x55957544b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44656/0 (socket says 172.21.15.77:44656) 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 -- 172.21.15.77:0/816058333 learned_addr learned my addr 172.21.15.77:0/816058333 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c77fe640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757e2370 0x5595757df5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c67fc640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757defe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 -- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 msgr2=0x5595757defe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757defe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 -- 172.21.15.77:0/816058333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757e2370 msgr2=0x5595757df5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757e2370 0x5595757df5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 -- 172.21.15.77:0/816058333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595754ae3c0 con 0x5595757d75d0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005e70 tx=0x7f06b80f0b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef42f1066ab6dac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:49.999+0000 7f06a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80d5060 con 0x5595757d75d0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f06b80d0760 con 0x5595757d75d0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b80bcc80 con 0x5595757d75d0 2024-09-15T22:13:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06b802e260 con 0x5595757d75d0 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 10 client.16005 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595757f0710 con 0x5595757d75d0 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b80bcc80 con 0x5595757d75d0 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f06b8003c80 con 0x5595757d75d0 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 10 client.16005 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 10 client.16005 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 20 client.16005 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5595757f1a60 0x5595757f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 4 client.16005 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.000+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957574a860 con 0x5595757f1a60 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.001+0000 7f06c67fc640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5595757f1a60 0x5595757f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.001+0000 7f06a6ffd640 10 client.16005 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.001+0000 7f06c67fc640 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5595757f1a60 0x5595757f3e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bbaea9b0a167615 server_cookie=8e70dff1ef3bef9b in_seq=0 out_seq=0 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06a6ffd640 1 -- 172.21.15.77:0/816058333 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55957574a860 con 0x5595757f1a60 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06a6ffd640 10 client.16005 handle_command_reply: tid=0 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 client.16005 shutdown 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 msgr2=0x5595757e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f06b8005e70 tx=0x7f06b80f0b10 comp rx=0 tx=0).stop 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 shutdown_connections 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5595757f1a60 0x5595757f3e50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757e2370 0x5595757df5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757defe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06a6ffd640 0 client.16005 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 --2- 172.21.15.77:0/816058333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757d75d0 0x5595757e1d30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 >> 172.21.15.77:0/816058333 conn(0x5595757a0310 msgr2=0x5595757d9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 shutdown_connections 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 1 -- 172.21.15.77:0/816058333 wait complete. 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.002+0000 7f06cfc22740 20 client.16005 trim_cache size 0 max 16384 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 Processor -- start 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 -- start start 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d75d0 0x5595757d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b1130 con 0x5595757e2370 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b2c70 con 0x5595757d75d0 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06cfc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b1800 con 0x5595757d8b40 2024-09-15T22:13:50.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44666/0 (socket says 172.21.15.77:44666) 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06c77fe640 1 -- 172.21.15.77:0/2368190815 learned_addr learned my addr 172.21.15.77:0/2368190815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.015+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d75d0 0x5595757d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 -- 172.21.15.77:0/2368190815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 msgr2=0x5595757edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 -- 172.21.15.77:0/2368190815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d75d0 msgr2=0x5595757d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d75d0 0x5595757d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 -- 172.21.15.77:0/2368190815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595757e2060 con 0x5595757e2370 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06c77fe640 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f06c0014310 tx=0x7f06c00097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d31fb5a042bfc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06ad7fa640 1 -- 172.21.15.77:0/2368190815 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001a040 con 0x5595757e2370 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06ad7fa640 1 -- 172.21.15.77:0/2368190815 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00119e0 con 0x5595757e2370 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.016+0000 7f06ad7fa640 1 -- 172.21.15.77:0/2368190815 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0005440 con 0x5595757e2370 2024-09-15T22:13:50.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- 172.21.15.77:0/2368190815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 msgr2=0x5595757d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f06c0014310 tx=0x7f06c00097c0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- 172.21.15.77:0/2368190815 shutdown_connections 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x5595757d2bf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d8b40 0x5595757edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 --2- 172.21.15.77:0/2368190815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d75d0 0x5595757d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- 172.21.15.77:0/2368190815 >> 172.21.15.77:0/2368190815 conn(0x5595757a0310 msgr2=0x5595757d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- 172.21.15.77:0/2368190815 shutdown_connections 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- 172.21.15.77:0/2368190815 wait complete. 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 -- start start 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.017+0000 7f06cfc22740 1 Processor -- start 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d75d0 0x5595757e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b25a0 con 0x5595757e2370 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b1ed0 con 0x5595757d8b40 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81af110 con 0x5595757d75d0 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 10 client.0 _ll_register_callbacks cb 0x7f06c0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44672/0 (socket says 172.21.15.77:44672) 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 -- 172.21.15.77:0/593097797 learned_addr learned my addr 172.21.15.77:0/593097797 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d75d0 0x5595757e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c67fc640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 -- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d75d0 msgr2=0x5595757e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d75d0 0x5595757e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 -- 172.21.15.77:0/593097797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 msgr2=0x5595757e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 -- 172.21.15.77:0/593097797 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957585bbc0 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06c77fe640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06c0011830 tx=0x7f06c0009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f517384dd5c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001a040 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f06c0011da0 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.018+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00119e0 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f06c0012e60 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 10 client.16011 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55957585ce00 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00119e0 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f06c0020040 con 0x5595757e2370 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 10 client.16011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:13:50.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 20 client.16011 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55957585d4b0 0x55957585f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 4 client.16011 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559575744d70 con 0x55957585d4b0 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.019+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55957585d4b0 0x55957585f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06a6ffd640 10 client.16011 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06c6ffd640 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55957585d4b0 0x55957585f8a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6b64f0a7b28f753 server_cookie=684656c8a2bf62a9 in_seq=0 out_seq=0 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06a6ffd640 1 -- 172.21.15.77:0/593097797 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559575744d70 con 0x55957585d4b0 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06a6ffd640 10 client.16011 handle_command_reply: tid=0 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 client.16011 shutdown 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 msgr2=0x55957585b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f06c0011830 tx=0x7f06c0009720 comp rx=0 tx=0).stop 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 shutdown_connections 2024-09-15T22:13:50.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55957585d4b0 0x55957585f8a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595757e2370 0x55957585b680 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5595757d8b40 0x5595757e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06a6ffd640 0 client.16011 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:13:50.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 --2- 172.21.15.77:0/593097797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595757d75d0 0x5595757e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.071 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 >> 172.21.15.77:0/593097797 conn(0x5595757a0310 msgr2=0x5595757d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 shutdown_connections 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 1 -- 172.21.15.77:0/593097797 wait complete. 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.020+0000 7f06cfc22740 20 client.16011 trim_cache size 0 max 16384 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 msgr2=0x7f06b00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 0x7f06b00a5a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f06c0005fd0 tx=0x7f06c0004ab0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 msgr2=0x7f06c819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009230 tx=0x7f06b8004270 comp rx=0 tx=0).stop 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 shutdown_connections 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06b00a3560 0x7f06b00a5a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06c8106a80 0x7f06c819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06c8101420 0x7f06c819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 --2- 172.21.15.77:0/3359500096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c8100a80 0x7f06c819a6b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 >> 172.21.15.77:0/3359500096 conn(0x7f06c80fc7a0 msgr2=0x7f06c806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 shutdown_connections 2024-09-15T22:13:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:50.021+0000 7f06cd849640 1 -- 172.21.15.77:0/3359500096 wait complete. 2024-09-15T22:13:50.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:49 smithi196 ceph-mon[35109]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:49 smithi117 ceph-mon[35021]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-15T22:13:50.897 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:13:51.057 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T22:13:51.138 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T22:13:51.157 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:51.181 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:13:51.184 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:13:51.184 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T22:13:51.232 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:13:51.233 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:13:51.233 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T22:13:51.249 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-15T22:13:51.250 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-15T22:13:51.251 DEBUG:teuthology.orchestra.run.smithi077:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-15T22:13:51.311 INFO:tasks.workunit.client.0.smithi077.stderr:+ dbench 1 2024-09-15T22:13:51.354 INFO:tasks.workunit.client.0.smithi077.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-15T22:13:51.354 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:13:51.354 INFO:tasks.workunit.client.0.smithi077.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-15T22:13:51.358 INFO:tasks.workunit.client.0.smithi077.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-15T22:13:51.458 INFO:tasks.workunit.client.0.smithi077.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-15T22:13:51.458 INFO:tasks.workunit.client.0.smithi077.stdout:releasing clients 2024-09-15T22:13:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 -- 172.21.15.77:0/881494879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d41068d0 msgr2=0x7fa4d4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/881494879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d41068d0 0x7fa4d4106cb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009930 tx=0x7fa4c402a420 comp rx=0 tx=0).stop 2024-09-15T22:13:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 -- 172.21.15.77:0/881494879 shutdown_connections 2024-09-15T22:13:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/881494879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d4101270 0x7fa4d410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/881494879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41008d0 0x7fa4d4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/881494879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d41068d0 0x7fa4d4106cb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 -- 172.21.15.77:0/881494879 >> 172.21.15.77:0/881494879 conn(0x7fa4d40fc610 msgr2=0x7fa4d40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 -- 172.21.15.77:0/881494879 shutdown_connections 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 -- 172.21.15.77:0/881494879 wait complete. 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.708+0000 7fa4d3fff640 1 Processor -- start 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 -- start start 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d41008d0 0x7fa4d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41068d0 0x7fa4d41a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4110cd0 con 0x7fa4d4101270 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4110b50 con 0x7fa4d41008d0 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d3fff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4110e50 con 0x7fa4d41068d0 2024-09-15T22:13:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d41008d0 0x7fa4d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44686/0 (socket says 172.21.15.77:44686) 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3706528853 learned_addr learned my addr 172.21.15.77:0/3706528853 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41068d0 0x7fa4d41a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3706528853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41068d0 msgr2=0x7fa4d41a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41068d0 0x7fa4d41a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3706528853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d41008d0 msgr2=0x7fa4d4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d41008d0 0x7fa4d4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3706528853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41a4720 con 0x7fa4d4101270 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.709+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800d770 tx=0x7fa4c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.710+0000 7fa4d8a00640 1 -- 172.21.15.77:0/3706528853 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8017070 con 0x7fa4d4101270 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.710+0000 7fa4d8a00640 1 -- 172.21.15.77:0/3706528853 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c8005500 con 0x7fa4d4101270 2024-09-15T22:13:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.710+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41a48f0 con 0x7fa4d4101270 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.710+0000 7fa4d8a00640 1 -- 172.21.15.77:0/3706528853 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8004e00 con 0x7fa4d4101270 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.710+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a4e00 con 0x7fa4d4101270 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.712+0000 7fa4d8a00640 1 -- 172.21.15.77:0/3706528853 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fa4c8004fa0 con 0x7fa4d4101270 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.712+0000 7fa4dbddb740 1 Processor -- start 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 -- start start 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d8a00640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 0x7fa4bc0a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe13f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 0x55efe1402940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55efe135b2f0 con 0x55efe13f6d20 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efe1302cf0 con 0x55efe1400550 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55efe13572b0 con 0x55efe13f57b0 2024-09-15T22:13:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 0x7fa4bc0a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44694/0 (socket says 172.21.15.77:44694) 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/2456739211 learned_addr learned my addr 172.21.15.77:0/2456739211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe13f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d8a00640 1 -- 172.21.15.77:0/3706528853 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4c80cfbe0 con 0x7fa4d4101270 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/2456739211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 msgr2=0x55efe13f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe13f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/2456739211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 msgr2=0x55efe1402940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 0x55efe1402940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 0x55efe1402940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/2456739211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe12e73a0 con 0x55efe13f6d20 2024-09-15T22:13:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.713+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 0x55efe1402940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.714+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa4c402a940 tx=0x7fa4c4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a037865abdb18b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.714+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 0x7fa4bc0a5b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55efe13f6250 tx=0x7fa4cc009a60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.714+0000 7fa4b9ffb640 1 -- 172.21.15.77:0/2456739211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4040070 con 0x55efe13f6d20 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.714+0000 7fa4b9ffb640 1 -- 172.21.15.77:0/2456739211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40319f0 con 0x55efe13f6d20 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.714+0000 7fa4b9ffb640 1 -- 172.21.15.77:0/2456739211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4035040 con 0x55efe13f6d20 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- 172.21.15.77:0/2456739211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 msgr2=0x55efe13f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa4c402a940 tx=0x7fa4c4005c00 comp rx=0 tx=0).stop 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- 172.21.15.77:0/2456739211 shutdown_connections 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe1400550 0x55efe1402940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe13f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/2456739211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f5270 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- 172.21.15.77:0/2456739211 >> 172.21.15.77:0/2456739211 conn(0x55efe13be310 msgr2=0x55efe13f1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- 172.21.15.77:0/2456739211 shutdown_connections 2024-09-15T22:13:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- 172.21.15.77:0/2456739211 wait complete. 2024-09-15T22:13:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 -- start start 2024-09-15T22:13:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.715+0000 7fa4dbddb740 1 Processor -- start 2024-09-15T22:13:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe13fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55efe1407600 con 0x55efe13f57b0 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efe1316060 con 0x55efe13f6d20 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55efe1407780 con 0x55efe1400550 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 10 client.0 _ll_register_callbacks cb 0x55efe10698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44700/0 (socket says 172.21.15.77:44700) 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/1118352578 learned_addr learned my addr 172.21.15.77:0/1118352578 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe13fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 msgr2=0x55efe13fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13fd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/1118352578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 msgr2=0x55efe13fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe13fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 -- 172.21.15.77:0/1118352578 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe10cc3c0 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe13fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40056d0 tx=0x7fa4c402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8030f9fa9e15c0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4040030 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4c4004260 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.716+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c40319f0 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4c4056050 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 10 client.16023 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efe1412e80 con 0x55efe13f57b0 2024-09-15T22:13:51.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40319f0 con 0x55efe13f57b0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa4c40038d0 con 0x55efe13f57b0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 10 client.16023 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 20 client.16023 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe14141d0 0x55efe14165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 4 client.16023 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efe1368860 con 0x55efe14141d0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.717+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe14141d0 0x55efe14165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.718+0000 7fa4b2ffd640 10 client.16023 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.718+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe14141d0 0x55efe14165c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73b2984b5590673f server_cookie=c7f4938bf43624d in_seq=0 out_seq=0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/1118352578 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efe1368860 con 0x55efe14141d0 2024-09-15T22:13:51.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4b2ffd640 10 client.16023 handle_command_reply: tid=0 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 client.16023 shutdown 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 msgr2=0x55efe13ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40056d0 tx=0x7fa4c402a420 comp rx=0 tx=0).stop 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 shutdown_connections 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe14141d0 0x55efe14165c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe13fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/1118352578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f57b0 0x55efe13ffd70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 >> 172.21.15.77:0/1118352578 conn(0x55efe13be310 msgr2=0x55efe13f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4b2ffd640 0 client.16023 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 shutdown_connections 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 1 -- 172.21.15.77:0/1118352578 wait complete. 2024-09-15T22:13:51.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.719+0000 7fa4dbddb740 20 client.16023 trim_cache size 0 max 16384 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.732+0000 7fa4dbddb740 1 Processor -- start 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.732+0000 7fa4dbddb740 1 -- start start 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f57b0 0x55efe13f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13f8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41aca90 con 0x55efe13f6d20 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41ae5d0 con 0x55efe13f57b0 2024-09-15T22:13:51.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41ad160 con 0x55efe1400550 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44704/0 (socket says 172.21.15.77:44704) 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3159555770 learned_addr learned my addr 172.21.15.77:0/3159555770 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f57b0 0x55efe13f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3159555770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 msgr2=0x55efe13f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3159555770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f57b0 msgr2=0x55efe13f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f57b0 0x55efe13f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 -- 172.21.15.77:0/3159555770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe13f2b40 con 0x55efe13f6d20 2024-09-15T22:13:51.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.733+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8097e50 tx=0x7fa4c807f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c62ac5fd2bba494 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4b97fa640 1 -- 172.21.15.77:0/3159555770 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8020040 con 0x55efe13f6d20 2024-09-15T22:13:51.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4b97fa640 1 -- 172.21.15.77:0/3159555770 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c809fcf0 con 0x55efe13f6d20 2024-09-15T22:13:51.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4b97fa640 1 -- 172.21.15.77:0/3159555770 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8009cc0 con 0x55efe13f6d20 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3159555770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 msgr2=0x55efe13f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8097e50 tx=0x7fa4c807f040 comp rx=0 tx=0).stop 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3159555770 shutdown_connections 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe1400550 0x55efe13f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe13f6d20 0x55efe13f8300 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3159555770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f57b0 0x55efe13f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3159555770 >> 172.21.15.77:0/3159555770 conn(0x55efe13be310 msgr2=0x55efe13f0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.734+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3159555770 shutdown_connections 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3159555770 wait complete. 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 -- start start 2024-09-15T22:13:51.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 Processor -- start 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe1419a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe1416de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41adf00 con 0x55efe1400550 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41ad830 con 0x55efe13f6d20 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41aaa70 con 0x55efe13f57b0 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4dbddb740 10 client.0 _ll_register_callbacks cb 0x7fa4c402aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44708/0 (socket says 172.21.15.77:44708) 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.735+0000 7fa4d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe1416de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 -- 172.21.15.77:0/3080087301 learned_addr learned my addr 172.21.15.77:0/3080087301 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe1419a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 -- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 msgr2=0x55efe1419a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe1419a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 -- 172.21.15.77:0/3080087301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 msgr2=0x55efe1416de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe1416de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 -- 172.21.15.77:0/3080087301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe1417860 con 0x55efe1400550 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe1419a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d27fc640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe1416de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4d37fe640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc011e70 tx=0x7fa4cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a426a266b8481ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc01f890 con 0x55efe1400550 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4cc01fa30 con 0x55efe1400550 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c809fcf0 con 0x55efe1400550 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4cc01fc30 con 0x55efe1400550 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.736+0000 7fa4dbddb740 10 client.16029 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:51.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55efe147e8c0 con 0x55efe1400550 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c809fcf0 con 0x55efe1400550 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa4cc028070 con 0x55efe1400550 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 10 client.16029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 20 client.16029 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe147f090 0x55efe14814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 4 client.16029 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efe1362d70 con 0x55efe147f090 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe147f090 0x55efe14814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4d2ffd640 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe147f090 0x55efe14814b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db2b2e80734aa3f2 server_cookie=9e47c311f993071a in_seq=0 out_seq=0 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.737+0000 7fa4b2ffd640 10 client.16029 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4b2ffd640 1 -- 172.21.15.77:0/3080087301 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efe1362d70 con 0x55efe147f090 2024-09-15T22:13:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4b2ffd640 10 client.16029 handle_command_reply: tid=0 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 client.16029 shutdown 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 msgr2=0x55efe1417320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc011e70 tx=0x7fa4cc004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 shutdown_connections 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55efe147f090 0x55efe14814b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55efe1400550 0x55efe1417320 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe13f6d20 0x55efe1416de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 --2- 172.21.15.77:0/3080087301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55efe13f57b0 0x55efe1419a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 >> 172.21.15.77:0/3080087301 conn(0x55efe13be310 msgr2=0x55efe13f0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4b2ffd640 0 client.16029 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 shutdown_connections 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 1 -- 172.21.15.77:0/3080087301 wait complete. 2024-09-15T22:13:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.738+0000 7fa4dbddb740 20 client.16029 trim_cache size 0 max 16384 2024-09-15T22:13:51.746 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 msgr2=0x7fa4bc0a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 0x7fa4bc0a5b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55efe13f6250 tx=0x7fa4cc009a60 comp rx=0 tx=0).stop 2024-09-15T22:13:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 msgr2=0x7fa4d41996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800d770 tx=0x7fa4c800dc30 comp rx=0 tx=0).stop 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 shutdown_connections 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4bc0a36b0 0x7fa4bc0a5b70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d41068d0 0x7fa4d41a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d4101270 0x7fa4d41996a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 --2- 172.21.15.77:0/3706528853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d41008d0 0x7fa4d4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 >> 172.21.15.77:0/3706528853 conn(0x7fa4d40fc610 msgr2=0x7fa4d40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 shutdown_connections 2024-09-15T22:13:51.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:51.739+0000 7fa4d3fff640 1 -- 172.21.15.77:0/3706528853 wait complete. 2024-09-15T22:13:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:51 smithi077 ceph-mon[30980]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:51 smithi196 ceph-mon[35109]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:51 smithi117 ceph-mon[35021]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-15T22:13:52.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 3343 35.38 MB/sec warmup 1 sec latency 17.706 ms 2024-09-15T22:13:52.476 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:13:52.766 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 -- 172.21.15.77:0/1313007746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0102ef0 msgr2=0x7fefb010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 --2- 172.21.15.77:0/1313007746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0102ef0 0x7fefb010da40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009ca0 tx=0x7fefa802a950 comp rx=0 tx=0).stop 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 -- 172.21.15.77:0/1313007746 shutdown_connections 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 --2- 172.21.15.77:0/1313007746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0102ef0 0x7fefb010da40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 --2- 172.21.15.77:0/1313007746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 0x7fefb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 --2- 172.21.15.77:0/1313007746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0100e20 0x7fefb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 -- 172.21.15.77:0/1313007746 >> 172.21.15.77:0/1313007746 conn(0x7fefb00fc7d0 msgr2=0x7fefb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 -- 172.21.15.77:0/1313007746 shutdown_connections 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.261+0000 7fefb5d62640 1 -- 172.21.15.77:0/1313007746 wait complete. 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 Processor -- start 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 -- start start 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 0x7fefb0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0102ef0 0x7fefb019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0110f80 con 0x7fefb0100e20 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0110e00 con 0x7fefb0102ef0 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefb5d62640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0111100 con 0x7fefb0102570 2024-09-15T22:13:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44736/0 (socket says 172.21.15.77:44736) 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefaf7fe640 1 -- 172.21.15.77:0/3589877756 learned_addr learned my addr 172.21.15.77:0/3589877756 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefaeffd640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 0x7fefb0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.262+0000 7fefaffff640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0102ef0 0x7fefb019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 -- 172.21.15.77:0/3589877756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 msgr2=0x7fefb0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 0x7fefb0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 -- 172.21.15.77:0/3589877756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0102ef0 msgr2=0x7fefb019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0102ef0 0x7fefb019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 -- 172.21.15.77:0/3589877756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb019c3b0 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9f0 tx=0x7fefa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefacff9640 1 -- 172.21.15.77:0/3589877756 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa000cd80 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb019c6a0 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefacff9640 1 -- 172.21.15.77:0/3589877756 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0010910 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefacff9640 1 -- 172.21.15.77:0/3589877756 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0018990 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.263+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb01a3f80 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 Processor -- start 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 -- start start 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefacff9640 1 -- 172.21.15.77:0/3589877756 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fefa0010ac0 con 0x7fefb0100e20 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461fefa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461fefaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56461fe602f0 con 0x56461ff07580 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56461fe07cf0 con 0x56461fefbb40 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefb813b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56461fe5c2b0 con 0x56461fefa5d0 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefaffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461fefa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefaffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461fefa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59006/0 (socket says 172.21.15.77:59006) 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44752/0 (socket says 172.21.15.77:44752) 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.266+0000 7fefaffff640 1 -- 172.21.15.77:0/651564009 learned_addr learned my addr 172.21.15.77:0/651564009 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 -- 172.21.15.77:0/651564009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 msgr2=0x56461fefaa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461fefaa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 -- 172.21.15.77:0/651564009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 msgr2=0x56461fefa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461fefa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 -- 172.21.15.77:0/651564009 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461fdec3a0 con 0x56461ff07580 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaeffd640 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009bd0 tx=0x7fefa4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ef536c96df512d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefacff9640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 0x7fef980a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fef95ffb640 1 -- 172.21.15.77:0/651564009 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004230 con 0x56461ff07580 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fef95ffb640 1 -- 172.21.15.77:0/651564009 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4009e40 con 0x56461ff07580 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fef95ffb640 1 -- 172.21.15.77:0/651564009 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004d70 con 0x56461ff07580 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefaffff640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 0x7fef980a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.267+0000 7fefacff9640 1 -- 172.21.15.77:0/3589877756 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fefa00d05e0 con 0x7fefb0100e20 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- 172.21.15.77:0/651564009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 msgr2=0x56461ff09970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefaffff640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 0x7fef980a5a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009ca0 tx=0x7fefa8005c50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009bd0 tx=0x7fefa4011810 comp rx=0 tx=0).stop 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- 172.21.15.77:0/651564009 shutdown_connections 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff09970 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461fefaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 --2- 172.21.15.77:0/651564009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461fefa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- 172.21.15.77:0/651564009 >> 172.21.15.77:0/651564009 conn(0x56461fec3310 msgr2=0x56461fef6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- 172.21.15.77:0/651564009 shutdown_connections 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- 172.21.15.77:0/651564009 wait complete. 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.268+0000 7fefb813b740 1 -- start start 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 Processor -- start 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefbb40 0x56461ff04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56461ff0a280 con 0x56461ff07580 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56461fe1b060 con 0x56461fefa5d0 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56461ff0a400 con 0x56461fefbb40 2024-09-15T22:13:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefb813b740 10 client.0 _ll_register_callbacks cb 0x56461fb6e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59022/0 (socket says 172.21.15.77:59022) 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefbb40 0x56461ff04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaf7fe640 1 -- 172.21.15.77:0/371742142 learned_addr learned my addr 172.21.15.77:0/371742142 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 -- 172.21.15.77:0/371742142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefbb40 msgr2=0x56461ff04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefbb40 0x56461ff04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 -- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 msgr2=0x56461ff06f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 -- 172.21.15.77:0/371742142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461fbd13c0 con 0x56461ff07580 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fefaffff640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff047e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006010 tx=0x7fefa80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c63faeffe6a326 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.269+0000 7fef8effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8056060 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56461fe6d860 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fefa8003f00 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa805d050 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 10 client.16041 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56461ff15d50 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56461fe6d860 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fefa8030c00 con 0x56461ff07580 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 20 client.16041 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff17160 0x56461ff19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 4 client.16041 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56461fe67d70 con 0x56461ff17160 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.270+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff17160 0x56461ff19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.271+0000 7fef8effd640 10 client.16041 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.271+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff17160 0x56461ff19590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f93f5bc7f84bb19d server_cookie=69ba87e90379390c in_seq=0 out_seq=0 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.271+0000 7fef8effd640 1 -- 172.21.15.77:0/371742142 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56461fe67d70 con 0x56461ff17160 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.271+0000 7fef8effd640 10 client.16041 handle_command_reply: tid=0 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.271+0000 7fefb813b740 1 client.16041 shutdown 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 msgr2=0x56461ff047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff047e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006010 tx=0x7fefa80095c0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 shutdown_connections 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff17160 0x56461ff19590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff047e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefbb40 0x56461ff04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 --2- 172.21.15.77:0/371742142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefa5d0 0x56461ff06f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 >> 172.21.15.77:0/371742142 conn(0x56461fec3310 msgr2=0x56461fefc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fef8effd640 0 client.16041 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 shutdown_connections 2024-09-15T22:13:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 1 -- 172.21.15.77:0/371742142 wait complete. 2024-09-15T22:13:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.272+0000 7fefb813b740 20 client.16041 trim_cache size 0 max 16384 2024-09-15T22:13:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 Processor -- start 2024-09-15T22:13:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 -- start start 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff13520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461ff07580 0x56461fef6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01acd00 con 0x56461fefbb40 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01ae840 con 0x56461ff07580 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefb813b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01ad3d0 con 0x56461fefa5d0 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44772/0 (socket says 172.21.15.77:44772) 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefaeffd640 1 -- 172.21.15.77:0/2483591918 learned_addr learned my addr 172.21.15.77:0/2483591918 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.286+0000 7fefaffff640 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461ff07580 0x56461fef6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 -- 172.21.15.77:0/2483591918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 msgr2=0x56461ff13520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff13520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 -- 172.21.15.77:0/2483591918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461ff07580 msgr2=0x56461fef6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461ff07580 0x56461fef6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 -- 172.21.15.77:0/2483591918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461fef64f0 con 0x56461fefbb40 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fefaeffd640 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009bd0 tx=0x7fefa4009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e80a32c072ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fef957fa640 1 -- 172.21.15.77:0/2483591918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa401e040 con 0x56461fefbb40 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fef957fa640 1 -- 172.21.15.77:0/2483591918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4029070 con 0x56461fefbb40 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.287+0000 7fef957fa640 1 -- 172.21.15.77:0/2483591918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4024480 con 0x56461fefbb40 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- 172.21.15.77:0/2483591918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 msgr2=0x56461fef6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009bd0 tx=0x7fefa4009850 comp rx=0 tx=0).stop 2024-09-15T22:13:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- 172.21.15.77:0/2483591918 shutdown_connections 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461ff07580 0x56461fef6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461fefbb40 0x56461fef6700 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 --2- 172.21.15.77:0/2483591918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff13520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- 172.21.15.77:0/2483591918 >> 172.21.15.77:0/2483591918 conn(0x56461fec3310 msgr2=0x56461ff041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- 172.21.15.77:0/2483591918 shutdown_connections 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- 172.21.15.77:0/2483591918 wait complete. 2024-09-15T22:13:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.288+0000 7fefb813b740 1 -- start start 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 Processor -- start 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff0f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461ff0fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01ae170 con 0x56461ff07580 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01adaa0 con 0x56461fefbb40 2024-09-15T22:13:53.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefb01aace0 con 0x56461fefa5d0 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefb813b740 10 client.0 _ll_register_callbacks cb 0x7fefa8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44786/0 (socket says 172.21.15.77:44786) 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461ff0fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaffff640 1 -- 172.21.15.77:0/192655051 learned_addr learned my addr 172.21.15.77:0/192655051 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff0f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 -- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 msgr2=0x56461ff0fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461ff0fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 -- 172.21.15.77:0/192655051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 msgr2=0x56461ff00840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaf7fe640 1 -- 172.21.15.77:0/192655051 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461ff00d80 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.289+0000 7fefaffff640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefaf7fe640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff0f690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9c0 tx=0x7fefa0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981f5fdf07233b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00d0420 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fefa00b0330 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa4029070 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa00b0530 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 10 client.25402 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56461ff81550 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4029070 con 0x56461fefa5d0 2024-09-15T22:13:53.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fefa009b880 con 0x56461fefa5d0 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 10 client.25402 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 10 client.25402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 20 client.25402 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff81c00 0x56461ff83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 4 client.25402 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.290+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56461fde4170 con 0x56461ff81c00 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefaeffd640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff81c00 0x56461ff83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefaeffd640 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff81c00 0x56461ff83ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7a2711f94db9157 server_cookie=fbab8dfd89439810 in_seq=0 out_seq=0 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fef8effd640 10 client.25402 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fef8effd640 1 -- 172.21.15.77:0/192655051 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56461fde4170 con 0x56461ff81c00 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fef8effd640 10 client.25402 handle_command_reply: tid=0 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 client.25402 shutdown 2024-09-15T22:13:53.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 msgr2=0x56461ff0f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff0f690 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9c0 tx=0x7fefa0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 shutdown_connections 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56461ff81c00 0x56461ff83ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56461ff07580 0x56461ff00840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56461fefbb40 0x56461ff0fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fef8effd640 0 client.25402 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 --2- 172.21.15.77:0/192655051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56461fefa5d0 0x56461ff0f690 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 >> 172.21.15.77:0/192655051 conn(0x56461fec3310 msgr2=0x56461fef66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 shutdown_connections 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.291+0000 7fefb813b740 1 -- 172.21.15.77:0/192655051 wait complete. 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb813b740 20 client.25402 trim_cache size 0 max 16384 2024-09-15T22:13:53.300 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 msgr2=0x7fef980a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 0x7fef980a5a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009ca0 tx=0x7fefa8005c50 comp rx=0 tx=0).stop 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 msgr2=0x7fefb0197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e9f0 tx=0x7fefa000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 shutdown_connections 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fef980a3560 0x7fef980a5a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb0102ef0 0x7fefb019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefb0102570 0x7fefb0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 --2- 172.21.15.77:0/3589877756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefb0100e20 0x7fefb0197400 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 >> 172.21.15.77:0/3589877756 conn(0x7fefb00fc7d0 msgr2=0x7fefb0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 shutdown_connections 2024-09-15T22:13:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:53.292+0000 7fefb5d62640 1 -- 172.21.15.77:0/3589877756 wait complete. 2024-09-15T22:13:53.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 6154 24.06 MB/sec warmup 2 sec latency 34.359 ms 2024-09-15T22:13:54.093 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:13:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:53 smithi077 ceph-mon[30980]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:53 smithi196 ceph-mon[35109]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:53 smithi117 ceph-mon[35021]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:54.363 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:54.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 9082 21.56 MB/sec warmup 3 sec latency 9.951 ms 2024-09-15T22:13:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 -- 172.21.15.77:0/2544616105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 msgr2=0x7f35041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 --2- 172.21.15.77:0/2544616105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041029b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009900 tx=0x7f34f002a480 comp rx=0 tx=0).stop 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 -- 172.21.15.77:0/2544616105 shutdown_connections 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 --2- 172.21.15.77:0/2544616105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f350410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 --2- 172.21.15.77:0/2544616105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041029b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 --2- 172.21.15.77:0/2544616105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 0x7f3504101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 -- 172.21.15.77:0/2544616105 >> 172.21.15.77:0/2544616105 conn(0x7f35040fc7d0 msgr2=0x7f35040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 -- 172.21.15.77:0/2544616105 shutdown_connections 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.872+0000 7f350adc2640 1 -- 172.21.15.77:0/2544616105 wait complete. 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 Processor -- start 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 -- start start 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 0x7f3504199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f35041a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3504110f80 con 0x7f3504102570 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3504110e00 con 0x7f3504102ef0 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350adc2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3504111100 con 0x7f3504100e20 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3503fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3503fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44804/0 (socket says 172.21.15.77:44804) 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3503fff640 1 -- 172.21.15.77:0/1271220662 learned_addr learned my addr 172.21.15.77:0/1271220662 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 0x7f3504199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f350935f640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f35041a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3503fff640 1 -- 172.21.15.77:0/1271220662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 msgr2=0x7f3504199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.873+0000 7f3503fff640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 0x7f3504199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3503fff640 1 -- 172.21.15.77:0/1271220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 msgr2=0x7f35041a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3503fff640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f35041a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3503fff640 1 -- 172.21.15.77:0/1271220662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35041a4950 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f350935f640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f35041a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3503fff640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006010 tx=0x7f34f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3501ffb640 1 -- 172.21.15.77:0/1271220662 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f003f070 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041a4b20 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3501ffb640 1 -- 172.21.15.77:0/1271220662 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f0031c30 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f3501ffb640 1 -- 172.21.15.77:0/1271220662 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0035030 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.874+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a4fd0 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f3501ffb640 1 -- 172.21.15.77:0/1271220662 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f34f0042070 con 0x7f3504102570 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 Processor -- start 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 -- start start 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560675be82f0 con 0x560675c8d370 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560675b8fcf0 con 0x560675c825d0 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f350d19b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560675be42b0 con 0x560675c83b40 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f3503fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59068/0 (socket says 172.21.15.77:59068) 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.877+0000 7f3508b5e640 1 -- 172.21.15.77:0/1894924591 learned_addr learned my addr 172.21.15.77:0/1894924591 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 -- 172.21.15.77:0/1894924591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 msgr2=0x560675c82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f350935f640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3501ffb640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 0x7f34f40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 -- 172.21.15.77:0/1894924591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 msgr2=0x560675c8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 -- 172.21.15.77:0/1894924591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560675b743a0 con 0x560675c825d0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f350935f640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3503fff640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f350935f640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 0x7f34f40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b760 tx=0x7f34f800bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f38b11b8905ee5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f3501ffb640 1 -- 172.21.15.77:0/1271220662 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f34f00f0980 con 0x7f3504102570 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f34eeffd640 1 -- 172.21.15.77:0/1894924591 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8003dd0 con 0x560675c825d0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f34eeffd640 1 -- 172.21.15.77:0/1894924591 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f8015070 con 0x560675c825d0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.878+0000 7f34eeffd640 1 -- 172.21.15.77:0/1894924591 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004e10 con 0x560675c825d0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- 172.21.15.77:0/1894924591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 msgr2=0x560675c82a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b760 tx=0x7f34f800bc20 comp rx=0 tx=0).stop 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350935f640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 0x7f34f40a5a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0099e0 tx=0x7f34fc004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- 172.21.15.77:0/1894924591 shutdown_connections 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c825d0 0x560675c82a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 --2- 172.21.15.77:0/1894924591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- 172.21.15.77:0/1894924591 >> 172.21.15.77:0/1894924591 conn(0x560675c4b310 msgr2=0x560675c7ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- 172.21.15.77:0/1894924591 shutdown_connections 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- 172.21.15.77:0/1894924591 wait complete. 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.879+0000 7f350d19b740 1 -- start start 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 Processor -- start 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 0x560675c8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560675c90070 con 0x560675c825d0 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560675ba3060 con 0x560675c8d370 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560675c901f0 con 0x560675c83b40 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350d19b740 10 client.0 _ll_register_callbacks cb 0x5606758f68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44826/0 (socket says 172.21.15.77:44826) 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f3508b5e640 1 -- 172.21.15.77:0/1683126240 learned_addr learned my addr 172.21.15.77:0/1683126240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f350935f640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 0x560675c8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.880+0000 7f3503fff640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 -- 172.21.15.77:0/1683126240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 msgr2=0x560675c89fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c89fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 -- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 msgr2=0x560675c8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 0x560675c8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 -- 172.21.15.77:0/1683126240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606759593c0 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3503fff640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c89fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350935f640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 0x560675c8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34f80039c0 tx=0x7f34f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a7685f7ad1fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8028060 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f8015070 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f34f8011640 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34f802f050 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350d19b740 10 client.16059 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560675c9ba30 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f8015070 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34f8022070 con 0x560675c825d0 2024-09-15T22:13:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350d19b740 10 client.16059 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.881+0000 7f350d19b740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f350d19b740 20 client.16059 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675c9ce90 0x560675c9f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f350d19b740 4 client.16059 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560675bf5860 con 0x560675c9ce90 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f3503fff640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675c9ce90 0x560675c9f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f34e3fff640 10 client.16059 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.882+0000 7f3503fff640 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675c9ce90 0x560675c9f280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e62c804551080d server_cookie=75551fb7f9d813c9 in_seq=0 out_seq=0 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f34e3fff640 1 -- 172.21.15.77:0/1683126240 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560675bf5860 con 0x560675c9ce90 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f34e3fff640 10 client.16059 handle_command_reply: tid=0 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 client.16059 shutdown 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 msgr2=0x560675c8cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34f80039c0 tx=0x7f34f80048f0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 shutdown_connections 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675c9ce90 0x560675c9f280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c8d370 0x560675c8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f34e3fff640 0 client.16059 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c83b40 0x560675c89fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 --2- 172.21.15.77:0/1683126240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c8cd20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 >> 172.21.15.77:0/1683126240 conn(0x560675c4b310 msgr2=0x560675c84960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 shutdown_connections 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 1 -- 172.21.15.77:0/1683126240 wait complete. 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.883+0000 7f350d19b740 20 client.16059 trim_cache size 0 max 16384 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.897+0000 7f350d19b740 1 Processor -- start 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.897+0000 7f350d19b740 1 -- start start 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c85120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c8d370 0x560675c85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35041aee00 con 0x560675c825d0 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35041b0940 con 0x560675c83b40 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350d19b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35041af4d0 con 0x560675c8d370 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44830/0 (socket says 172.21.15.77:44830) 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 -- 172.21.15.77:0/674442158 learned_addr learned my addr 172.21.15.77:0/674442158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f350935f640 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c8d370 0x560675c85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3503fff640 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c85120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 -- 172.21.15.77:0/674442158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c8d370 msgr2=0x560675c85660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c8d370 0x560675c85660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 -- 172.21.15.77:0/674442158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 msgr2=0x560675c85120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c85120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 -- 172.21.15.77:0/674442158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560675c85ba0 con 0x560675c825d0 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f3508b5e640 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34f8019990 tx=0x7f34f8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ce7f16c4c5d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f34ee7fc640 1 -- 172.21.15.77:0/674442158 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8028070 con 0x560675c825d0 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f34ee7fc640 1 -- 172.21.15.77:0/674442158 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f8015040 con 0x560675c825d0 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.898+0000 7f34ee7fc640 1 -- 172.21.15.77:0/674442158 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80116a0 con 0x560675c825d0 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- 172.21.15.77:0/674442158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 msgr2=0x560675c7dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34f8019990 tx=0x7f34f8001e50 comp rx=0 tx=0).stop 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- 172.21.15.77:0/674442158 shutdown_connections 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c8d370 0x560675c85660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c85120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 --2- 172.21.15.77:0/674442158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c825d0 0x560675c7dfb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- 172.21.15.77:0/674442158 >> 172.21.15.77:0/674442158 conn(0x560675c4b310 msgr2=0x560675c88f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- 172.21.15.77:0/674442158 shutdown_connections 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- 172.21.15.77:0/674442158 wait complete. 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 -- start start 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.899+0000 7f350d19b740 1 Processor -- start 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35041b0270 con 0x560675c8d370 2024-09-15T22:13:54.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35041afba0 con 0x560675c83b40 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35041acde0 con 0x560675c825d0 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350d19b740 10 client.0 _ll_register_callbacks cb 0x7f34f00f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3503fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3508b5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39714/0 (socket says 172.21.15.77:39714) 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3508b5e640 1 -- 172.21.15.77:0/1735738560 learned_addr learned my addr 172.21.15.77:0/1735738560 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 -- 172.21.15.77:0/1735738560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 msgr2=0x560675c91170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 -- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 msgr2=0x560675c916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 -- 172.21.15.77:0/1735738560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560675d06820 con 0x560675c8d370 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:54.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f3503fff640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f350935f640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c91bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34fc012fc0 tx=0x7f34fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c2110e0cd67570 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.900+0000 7f34e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc010670 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f34fc010810 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f8015040 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34fc010a10 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560675d07b00 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f8015040 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34fc01e430 con 0x560675c8d370 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 20 client.16065 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:54.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675d082d0 0x560675d0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 4 client.16065 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.901+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560675befd70 con 0x560675d082d0 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675d082d0 0x560675d0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f3508b5e640 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675d082d0 0x560675d0a6f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f190c2b2e77d691 server_cookie=8a1db9d922b41cf1 in_seq=0 out_seq=0 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f34e3fff640 10 client.16065 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f34e3fff640 1 -- 172.21.15.77:0/1735738560 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560675befd70 con 0x560675d082d0 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f34e3fff640 10 client.16065 handle_command_reply: tid=0 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 client.16065 shutdown 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 msgr2=0x560675c91bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c91bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34fc012fc0 tx=0x7f34fc004ab0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 shutdown_connections 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560675d082d0 0x560675d0a6f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560675c8d370 0x560675c91bf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f34e3fff640 0 client.16065 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:54.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560675c83b40 0x560675c916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 --2- 172.21.15.77:0/1735738560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560675c825d0 0x560675c91170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 >> 172.21.15.77:0/1735738560 conn(0x560675c4b310 msgr2=0x560675c8b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.932 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 shutdown_connections 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.902+0000 7f350d19b740 1 -- 172.21.15.77:0/1735738560 wait complete. 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350d19b740 20 client.16065 trim_cache size 0 max 16384 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 msgr2=0x7f34f40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 0x7f34f40a5a20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0099e0 tx=0x7f34fc004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 msgr2=0x7f35041998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006010 tx=0x7f34f0003c10 comp rx=0 tx=0).stop 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 shutdown_connections 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3560 0x7f34f40a5a20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3504102ef0 0x7f35041a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3504102570 0x7f35041998d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 --2- 172.21.15.77:0/1271220662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3504100e20 0x7f3504199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:54.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 >> 172.21.15.77:0/1271220662 conn(0x7f35040fc7d0 msgr2=0x7f35040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:54.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 shutdown_connections 2024-09-15T22:13:54.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:54.903+0000 7f350adc2640 1 -- 172.21.15.77:0/1271220662 wait complete. 2024-09-15T22:13:55.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 11628 19.79 MB/sec warmup 4 sec latency 39.197 ms 2024-09-15T22:13:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:55 smithi077 ceph-mon[30980]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:56.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:55 smithi196 ceph-mon[35109]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:55 smithi117 ceph-mon[35021]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-15T22:13:56.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 14554 19.49 MB/sec warmup 5 sec latency 140.075 ms 2024-09-15T22:13:56.622 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:13:56.623 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:13:56.881 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:57.298 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:56 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:13:57.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:56 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:13:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:56 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 -- 172.21.15.77:0/644694467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d581027a0 msgr2=0x7f6d581067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/644694467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d581027a0 0x7f6d581067f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02a410 comp rx=0 tx=0).stop 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 -- 172.21.15.77:0/644694467 shutdown_connections 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/644694467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58106f20 0x7f6d5810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/644694467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d581027a0 0x7f6d581067f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/644694467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d58101df0 0x7f6d581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 -- 172.21.15.77:0/644694467 >> 172.21.15.77:0/644694467 conn(0x7f6d580fd8f0 msgr2=0x7f6d580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 -- 172.21.15.77:0/644694467 shutdown_connections 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 -- 172.21.15.77:0/644694467 wait complete. 2024-09-15T22:13:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.381+0000 7f6d5d30f640 1 Processor -- start 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 -- start start 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 0x7f6d580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d581027a0 0x7f6d58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 0x7f6d58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58111f80 con 0x7f6d58106f20 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58111e00 con 0x7f6d58101df0 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d5d30f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112100 con 0x7f6d581027a0 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 0x7f6d580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d567fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d581027a0 0x7f6d58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 0x7f6d58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 0x7f6d580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59124/0 (socket says 172.21.15.77:59124) 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d56ffd640 1 -- 172.21.15.77:0/4077721409 learned_addr learned my addr 172.21.15.77:0/4077721409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:57.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 -- 172.21.15.77:0/4077721409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d581027a0 msgr2=0x7f6d58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d581027a0 0x7f6d58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 -- 172.21.15.77:0/4077721409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 msgr2=0x7f6d580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 0x7f6d580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 -- 172.21.15.77:0/4077721409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581a69a0 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.382+0000 7f6d577fe640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 0x7f6d58074f70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ec50 tx=0x7f6d5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.383+0000 7f6d3ffff640 1 -- 172.21.15.77:0/4077721409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003b70 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.383+0000 7f6d3ffff640 1 -- 172.21.15.77:0/4077721409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d50003d10 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.383+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581a6c90 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.383+0000 7f6d3ffff640 1 -- 172.21.15.77:0/4077721409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50010890 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.383+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581a7150 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.385+0000 7f6d3ffff640 1 -- 172.21.15.77:0/4077721409 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6d50003fc0 con 0x7f6d58106f20 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.385+0000 7f6d5f6e8740 1 Processor -- start 2024-09-15T22:13:57.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 -- start start 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88759b60 0x564b88759f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b88767130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b886bf280 con 0x564b8875a480 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b88666b10 con 0x564b88759b60 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b886bb240 con 0x564b88764d40 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44880/0 (socket says 172.21.15.77:44880) 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 -- 172.21.15.77:0/3788521994 learned_addr learned my addr 172.21.15.77:0/3788521994 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d3ffff640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 0x7f6d400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d567fc640 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88759b60 0x564b88759f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d577fe640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 0x7f6d400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 -- 172.21.15.77:0/3788521994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 msgr2=0x564b88767130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b88767130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 -- 172.21.15.77:0/3788521994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88759b60 msgr2=0x564b88759f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88759b60 0x564b88759f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 -- 172.21.15.77:0/3788521994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b886731f0 con 0x564b8875a480 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d3ffff640 1 -- 172.21.15.77:0/4077721409 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6d500d0580 con 0x7f6d58106f20 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.386+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009dc0 tx=0x7f6d48011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b649d42478e886a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d3d7fa640 1 -- 172.21.15.77:0/3788521994 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48005220 con 0x564b8875a480 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d3d7fa640 1 -- 172.21.15.77:0/3788521994 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d48004c80 con 0x564b8875a480 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d3d7fa640 1 -- 172.21.15.77:0/3788521994 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48019880 con 0x564b8875a480 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/3788521994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 msgr2=0x564b88764800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d577fe640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 0x7f6d400a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d500cda40 tx=0x7f6d5000ef50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009dc0 tx=0x7f6d48011810 comp rx=0 tx=0).stop 2024-09-15T22:13:57.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/3788521994 shutdown_connections 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b88767130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88764800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/3788521994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88759b60 0x564b88759f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.387+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/3788521994 >> 172.21.15.77:0/3788521994 conn(0x564b887560f0 msgr2=0x564b887564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/3788521994 shutdown_connections 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/3788521994 wait complete. 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- start start 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 Processor -- start 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88759b60 0x564b887647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b887622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b88767910 con 0x564b8875a480 2024-09-15T22:13:57.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b88679f10 con 0x564b88764d40 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b88767a90 con 0x564b88759b60 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d5f6e8740 10 client.0 _ll_register_callbacks cb 0x564b88642a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d56ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44882/0 (socket says 172.21.15.77:44882) 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d56ffd640 1 -- 172.21.15.77:0/4025452008 learned_addr learned my addr 172.21.15.77:0/4025452008 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.388+0000 7f6d567fc640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b887622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 -- 172.21.15.77:0/4025452008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88759b60 msgr2=0x564b887647e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88759b60 0x564b887647e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 -- 172.21.15.77:0/4025452008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 msgr2=0x564b88761b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 -- 172.21.15.77:0/4025452008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b884306a0 con 0x564b88764d40 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:57.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d567fc640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b887622c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c005940 tx=0x7f6d4c0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89aa79fa07e70dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c035040 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6d4c004240 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d48004c80 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d4c004440 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d5f6e8740 10 client.25469 fetch_fsmap learned FSMap version 31 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564b88773450 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.389+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d48004c80 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f6d4c046720 con 0x564b88764d40 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 10 client.25469 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 20 client.25469 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:57.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b88774910 0x564b88776d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 4 client.25469 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b886cc7f0 con 0x564b88774910 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b88774910 0x564b88776d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b88774910 0x564b88776d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6293db05a7d2cd73 server_cookie=d1bef193d6d3f86b in_seq=0 out_seq=0 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.390+0000 7f6d3aff5640 10 client.25469 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d3aff5640 1 -- 172.21.15.77:0/4025452008 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b886cc7f0 con 0x564b88774910 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d3aff5640 10 client.25469 handle_command_reply: tid=0 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 client.25469 shutdown 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 msgr2=0x564b887622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b887622c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c005940 tx=0x7f6d4c0317e0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 shutdown_connections 2024-09-15T22:13:57.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b88774910 0x564b88776d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b887622c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b88761b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/4025452008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88759b60 0x564b887647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 >> 172.21.15.77:0/4025452008 conn(0x564b887560f0 msgr2=0x564b88755270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d3aff5640 0 client.25469 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 shutdown_connections 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/4025452008 wait complete. 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.391+0000 7f6d5f6e8740 20 client.25469 trim_cache size 0 max 16384 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 Processor -- start 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 -- start start 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b8875a480 0x564b88763990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b8877c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b1050 con 0x564b88759b60 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b2bc0 con 0x564b88764d40 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b1720 con 0x564b8875a480 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44892/0 (socket says 172.21.15.77:44892) 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 -- 172.21.15.77:0/946269117 learned_addr learned my addr 172.21.15.77:0/946269117 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b8875a480 0x564b88763990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b8877c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 -- 172.21.15.77:0/946269117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b8875a480 msgr2=0x564b88763990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b8875a480 0x564b88763990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 -- 172.21.15.77:0/946269117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 msgr2=0x564b8877c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b8877c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.409+0000 7f6d567fc640 1 -- 172.21.15.77:0/946269117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b8875a2b0 con 0x564b88759b60 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d567fc640 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c004650 tx=0x7f6d4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835ca04cdecfcde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d3cff9640 1 -- 172.21.15.77:0/946269117 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c05a070 con 0x564b88759b60 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d3cff9640 1 -- 172.21.15.77:0/946269117 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d4c005520 con 0x564b88759b60 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d3cff9640 1 -- 172.21.15.77:0/946269117 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c035050 con 0x564b88759b60 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/946269117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 msgr2=0x564b88761410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.410+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c004650 tx=0x7f6d4c0045e0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/946269117 shutdown_connections 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b88764d40 0x564b8877c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b8875a480 0x564b88763990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/946269117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b88759b60 0x564b88761410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/946269117 >> 172.21.15.77:0/946269117 conn(0x564b887560f0 msgr2=0x564b88763280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/946269117 shutdown_connections 2024-09-15T22:13:57.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/946269117 wait complete. 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- start start 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 Processor -- start 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b8875ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b8875f4a0 0x564b887de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b24c0 con 0x564b8875a480 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b1df0 con 0x564b8875f4a0 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581af030 con 0x564b88764d40 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d5f6e8740 10 client.0 _ll_register_callbacks cb 0x7f6d4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d567fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d567fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44902/0 (socket says 172.21.15.77:44902) 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d567fc640 1 -- 172.21.15.77:0/1122669992 learned_addr learned my addr 172.21.15.77:0/1122669992 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:57.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b8875ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.411+0000 7f6d567fc640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b8875f4a0 0x564b887de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 -- 172.21.15.77:0/1122669992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 msgr2=0x564b8875ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b8875ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 -- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b8875f4a0 msgr2=0x564b887de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b8875f4a0 0x564b887de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 -- 172.21.15.77:0/1122669992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b8876d980 con 0x564b8875a480 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b8875ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d567fc640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c02a930 tx=0x7f6d4c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42584d325d3138e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c05a070 con 0x564b8875a480 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6d4c03b4f0 con 0x564b8875a480 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d4c005520 con 0x564b8875a480 2024-09-15T22:13:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d4c06a050 con 0x564b8875a480 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d5f6e8740 10 client.16083 fetch_fsmap learned FSMap version 31 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564b887dfbe0 con 0x564b8875a480 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.412+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d4c005520 con 0x564b8875a480 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f6d4c054690 con 0x564b8875a480 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 10 client.16083 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 20 client.16083 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b887e01a0 0x564b887e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 4 client.16083 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b886c6d00 con 0x564b887e01a0 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b887e01a0 0x564b887e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d3aff5640 10 client.16083 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.413+0000 7f6d56ffd640 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b887e01a0 0x564b887e2590 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d776b550073e239 server_cookie=4808c61c4feae894 in_seq=0 out_seq=0 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.414+0000 7f6d3aff5640 1 -- 172.21.15.77:0/1122669992 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b886c6d00 con 0x564b887e01a0 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.414+0000 7f6d3aff5640 10 client.16083 handle_command_reply: tid=0 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.414+0000 7f6d5f6e8740 1 client.16083 shutdown 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 msgr2=0x564b8875ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c02a930 tx=0x7f6d4c009520 comp rx=0 tx=0).stop 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 shutdown_connections 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b887e01a0 0x564b887e2590 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d3aff5640 0 client.16083 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b8875f4a0 0x564b887de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b88764d40 0x564b8875ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 --2- 172.21.15.77:0/1122669992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b8875a480 0x564b8875ea20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 >> 172.21.15.77:0/1122669992 conn(0x564b887560f0 msgr2=0x564b8875bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 shutdown_connections 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 1 -- 172.21.15.77:0/1122669992 wait complete. 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5f6e8740 20 client.16083 trim_cache size 0 max 16384 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "c5581fe9-4699-45c4-837a-168c36ab1baa", 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:13:57.433 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 msgr2=0x7f6d400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 0x7f6d400a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d500cda40 tx=0x7f6d5000ef50 comp rx=0 tx=0).stop 2024-09-15T22:13:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 msgr2=0x7f6d58074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 0x7f6d58074f70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ec50 tx=0x7f6d5000c6a0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 shutdown_connections 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d400a33b0 0x7f6d400a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d58106f20 0x7f6d58074f70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d581027a0 0x7f6d58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.415+0000 7f6d5d30f640 1 --2- 172.21.15.77:0/4077721409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d58101df0 0x7f6d580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.416+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 >> 172.21.15.77:0/4077721409 conn(0x7f6d580fd8f0 msgr2=0x7f6d580fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.416+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 shutdown_connections 2024-09-15T22:13:57.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:57.416+0000 7f6d5d30f640 1 -- 172.21.15.77:0/4077721409 wait complete. 2024-09-15T22:13:57.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 17094 18.22 MB/sec warmup 6 sec latency 204.683 ms 2024-09-15T22:13:58.232 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: pgmap v49: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 602 KiB/s wr, 81 op/s 2024-09-15T22:13:58.232 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:13:58.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:13:58.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:13:58.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: pgmap v49: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 602 KiB/s wr, 81 op/s 2024-09-15T22:13:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:13:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:13:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:13:58.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: pgmap v49: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 602 KiB/s wr, 81 op/s 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:13:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:58.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 19724 17.44 MB/sec warmup 7 sec latency 11.416 ms 2024-09-15T22:13:58.497 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:59.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 -- 172.21.15.77:0/2990614519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70101420 msgr2=0x7fba7010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 --2- 172.21.15.77:0/2990614519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70101420 0x7fba7010d950 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fba68009cd0 tx=0x7fba6802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 -- 172.21.15.77:0/2990614519 shutdown_connections 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 --2- 172.21.15.77:0/2990614519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70101420 0x7fba7010d950 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 --2- 172.21.15.77:0/2990614519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 0x7fba70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 --2- 172.21.15.77:0/2990614519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70106a80 0x7fba70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 -- 172.21.15.77:0/2990614519 >> 172.21.15.77:0/2990614519 conn(0x7fba700fc7a0 msgr2=0x7fba700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 -- 172.21.15.77:0/2990614519 shutdown_connections 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.983+0000 7fba74820640 1 -- 172.21.15.77:0/2990614519 wait complete. 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 Processor -- start 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 -- start start 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 0x7fba70199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70101420 0x7fba70199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba70110e80 con 0x7fba70106a80 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba70110d00 con 0x7fba70100a80 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba74820640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba70111000 con 0x7fba70101420 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50974/0 (socket says 172.21.15.77:50974) 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 -- 172.21.15.77:0/1982821815 learned_addr learned my addr 172.21.15.77:0/1982821815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70101420 0x7fba70199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6e59c640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 0x7fba70199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 -- 172.21.15.77:0/1982821815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70101420 msgr2=0x7fba70199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70101420 0x7fba70199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 -- 172.21.15.77:0/1982821815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 msgr2=0x7fba70199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 0x7fba70199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.984+0000 7fba6ed9d640 1 -- 172.21.15.77:0/1982821815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a47c0 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba6ed9d640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba68005cc0 tx=0x7fba68032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba5f7fe640 1 -- 172.21.15.77:0/1982821815 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68004040 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba5f7fe640 1 -- 172.21.15.77:0/1982821815 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba680046c0 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba5f7fe640 1 -- 172.21.15.77:0/1982821815 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68030dc0 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a4990 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.985+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a4e40 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.987+0000 7fba5f7fe640 1 -- 172.21.15.77:0/1982821815 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fba680041e0 con 0x7fba70106a80 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.987+0000 7fba76bf9740 1 Processor -- start 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 -- start start 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc09880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d1bb6f2e0 con 0x558d1bc14680 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d1bb16ce0 con 0x558d1bc09dc0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba76bf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d1bb6b2a0 con 0x558d1bc094a0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba5f7fe640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 0x7fba580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc09880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:33884/0 (socket says 172.21.15.77:33884) 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 -- 172.21.15.77:0/1456718513 learned_addr learned my addr 172.21.15.77:0/1456718513 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6ed9d640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 0x7fba580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba5f7fe640 1 -- 172.21.15.77:0/1982821815 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fba6802e030 con 0x7fba70106a80 2024-09-15T22:13:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 -- 172.21.15.77:0/1456718513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 msgr2=0x558d1bc09880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc09880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 -- 172.21.15.77:0/1456718513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 msgr2=0x558d1bc16a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc16a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6e59c640 1 -- 172.21.15.77:0/1456718513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1bafb440 con 0x558d1bc09dc0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.988+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc09880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.989+0000 7fba6ed9d640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 0x7fba580a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba680ee530 tx=0x7fba68003f10 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.989+0000 7fba6e59c640 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ca0 tx=0x7fba60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2443b0e71f180045 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.989+0000 7fba5cff9640 1 -- 172.21.15.77:0/1456718513 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6001e070 con 0x558d1bc09dc0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.989+0000 7fba5cff9640 1 -- 172.21.15.77:0/1456718513 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60019b10 con 0x558d1bc09dc0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.989+0000 7fba5cff9640 1 -- 172.21.15.77:0/1456718513 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6000ea00 con 0x558d1bc09dc0 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- 172.21.15.77:0/1456718513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 msgr2=0x558d1bc14140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ca0 tx=0x7fba60011810 comp rx=0 tx=0).stop 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- 172.21.15.77:0/1456718513 shutdown_connections 2024-09-15T22:13:59.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc16a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc14140 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- 172.21.15.77:0/1456718513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc09880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- 172.21.15.77:0/1456718513 >> 172.21.15.77:0/1456718513 conn(0x558d1bc05a30 msgr2=0x558d1bc05e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- 172.21.15.77:0/1456718513 shutdown_connections 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- 172.21.15.77:0/1456718513 wait complete. 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- start start 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 Processor -- start 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc13f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc11210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d1bc17250 con 0x558d1bc14680 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d1bb2a050 con 0x558d1bc09dc0 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.990+0000 7fba76bf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558d1bc173d0 con 0x558d1bc094a0 2024-09-15T22:13:59.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba76bf9740 10 client.0 _ll_register_callbacks cb 0x558d1bb6a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50988/0 (socket says 172.21.15.77:50988) 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 -- 172.21.15.77:0/3831884335 learned_addr learned my addr 172.21.15.77:0/3831884335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc11210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc13f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 -- 172.21.15.77:0/3831884335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 msgr2=0x558d1bc13f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc13f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 -- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 msgr2=0x558d1bc11210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc11210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 -- 172.21.15.77:0/3831884335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1b8e03c0 con 0x558d1bc14680 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fba6400f430 tx=0x7fba64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364fc7548637d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64005020 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fba640051c0 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba60019b10 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fba640053c0 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba76bf9740 10 client.16095 fetch_fsmap learned FSMap version 31 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.991+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d1bc22c50 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60019b10 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fba6400d550 con 0x558d1bc14680 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 10 client.16095 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 20 client.16095 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:59.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc24270 0x558d1bc266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 4 client.16095 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1bb7c850 con 0x558d1bc24270 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba6e59c640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc24270 0x558d1bc266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba55ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.992+0000 7fba6e59c640 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc24270 0x558d1bc266d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e885167b5351ee server_cookie=4a6c09d3cc436492 in_seq=0 out_seq=0 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba55ffb640 1 -- 172.21.15.77:0/3831884335 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d1bb7c850 con 0x558d1bc24270 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba55ffb640 10 client.16095 handle_command_reply: tid=0 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 client.16095 shutdown 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 msgr2=0x558d1bc11750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fba6400f430 tx=0x7fba64004730 comp rx=0 tx=0).stop 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 shutdown_connections 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc24270 0x558d1bc266d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc11750 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc11210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba55ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3831884335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc13f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 >> 172.21.15.77:0/3831884335 conn(0x558d1bc05a30 msgr2=0x558d1bc0b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 shutdown_connections 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 1 -- 172.21.15.77:0/3831884335 wait complete. 2024-09-15T22:13:59.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:58.993+0000 7fba76bf9740 20 client.16095 trim_cache size 0 max 16384 2024-09-15T22:13:59.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 Processor -- start 2024-09-15T22:13:59.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 -- start start 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc2bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba701aeda0 con 0x558d1bc14680 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b0910 con 0x558d1bc09dc0 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba76bf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba701af470 con 0x558d1bc094a0 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50992/0 (socket says 172.21.15.77:50992) 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc2bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 -- 172.21.15.77:0/537122384 learned_addr learned my addr 172.21.15.77:0/537122384 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 -- 172.21.15.77:0/537122384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 msgr2=0x558d1bc0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc0fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 -- 172.21.15.77:0/537122384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 msgr2=0x558d1bc2bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc2bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.011+0000 7fba6dd9b640 1 -- 172.21.15.77:0/537122384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1bc101c0 con 0x558d1bc14680 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba640114a0 tx=0x7fba640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec32c6a24287937 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba57fff640 1 -- 172.21.15.77:0/537122384 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64011be0 con 0x558d1bc14680 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba57fff640 1 -- 172.21.15.77:0/537122384 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64005560 con 0x558d1bc14680 2024-09-15T22:13:59.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba57fff640 1 -- 172.21.15.77:0/537122384 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64011030 con 0x558d1bc14680 2024-09-15T22:13:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 -- 172.21.15.77:0/537122384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 msgr2=0x558d1bc2c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba640114a0 tx=0x7fba640046c0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 -- 172.21.15.77:0/537122384 shutdown_connections 2024-09-15T22:13:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc14680 0x558d1bc2c4e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc09dc0 0x558d1bc2bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.012+0000 7fba76bf9740 1 --2- 172.21.15.77:0/537122384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- 172.21.15.77:0/537122384 >> 172.21.15.77:0/537122384 conn(0x558d1bc05a30 msgr2=0x558d1bc13290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- 172.21.15.77:0/537122384 shutdown_connections 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- 172.21.15.77:0/537122384 wait complete. 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- start start 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 Processor -- start 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc14680 0x558d1bc1b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba701b0210 con 0x558d1bc09dc0 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fba701afb40 con 0x558d1bc14680 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba701acd80 con 0x558d1bc094a0 2024-09-15T22:13:59.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba76bf9740 10 client.0 _ll_register_callbacks cb 0x7fba6400acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba6e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51002/0 (socket says 172.21.15.77:51002) 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48734/0 (socket says 172.21.15.77:48734) 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.013+0000 7fba6e59c640 1 -- 172.21.15.77:0/3621526147 learned_addr learned my addr 172.21.15.77:0/3621526147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc14680 0x558d1bc1b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 -- 172.21.15.77:0/3621526147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 msgr2=0x558d1bc1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 -- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc14680 msgr2=0x558d1bc1b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc14680 0x558d1bc1b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 -- 172.21.15.77:0/3621526147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1bc09bf0 con 0x558d1bc09dc0 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:59.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba6e59c640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ca0 tx=0x7fba60009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d247163166434 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6001e040 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fba60005240 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d1bb76d60 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fba6003b050 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba76bf9740 10 client.16101 fetch_fsmap learned FSMap version 31 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.014+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d1bc93700 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558d1bb76d60 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7fba6000e5d0 con 0x558d1bc09dc0 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 10 client.16101 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:59.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 20 client.16101 populate_metadata read hostname 'smithi077' 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc93d10 0x558d1bc96130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 4 client.16101 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d1baf3210 con 0x558d1bc93d10 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc93d10 0x558d1bc96130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba55ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba6dd9b640 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc93d10 0x558d1bc96130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760df6c34551205a server_cookie=bc0a1a4a06624576 in_seq=0 out_seq=0 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba55ffb640 1 -- 172.21.15.77:0/3621526147 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558d1baf3210 con 0x558d1bc93d10 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba55ffb640 10 client.16101 handle_command_reply: tid=0 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.015+0000 7fba76bf9740 1 client.16101 shutdown 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 msgr2=0x558d1bc1ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba60009ca0 tx=0x7fba60009ba0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 shutdown_connections 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558d1bc93d10 0x558d1bc96130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d1bc14680 0x558d1bc1b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba55ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1bc09dc0 0x558d1bc1ab60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 --2- 172.21.15.77:0/3621526147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558d1bc094a0 0x558d1bc1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 >> 172.21.15.77:0/3621526147 conn(0x558d1bc05a30 msgr2=0x558d1bc13290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 shutdown_connections 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 1 -- 172.21.15.77:0/3621526147 wait complete. 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba76bf9740 20 client.16101 trim_cache size 0 max 16384 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:13:59.026 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: "c5581fe9-4699-45c4-837a-168c36ab1baa": { 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "c5581fe9-4699-45c4-837a-168c36ab1baa", 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:13:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:13:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 msgr2=0x7fba580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 0x7fba580a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba680ee530 tx=0x7fba68003f10 comp rx=0 tx=0).stop 2024-09-15T22:13:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 msgr2=0x7fba701a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba68005cc0 tx=0x7fba68032f40 comp rx=0 tx=0).stop 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 shutdown_connections 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fba580a33b0 0x7fba580a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba70106a80 0x7fba701a4280 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba70101420 0x7fba70199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 --2- 172.21.15.77:0/1982821815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba70100a80 0x7fba70199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 >> 172.21.15.77:0/1982821815 conn(0x7fba700fc7a0 msgr2=0x7fba700fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 shutdown_connections 2024-09-15T22:13:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:13:59.016+0000 7fba74820640 1 -- 172.21.15.77:0/1982821815 wait complete. 2024-09-15T22:13:59.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:58 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:13:59.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:58 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:59.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:58 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:13:59.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:58 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:59.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:58 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:13:59.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:58 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:59.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 22303 17.07 MB/sec warmup 8 sec latency 18.434 ms 2024-09-15T22:13:59.787 INFO:tasks.cephfs.filesystem:scrub status for tag:c5581fe9-4699-45c4-837a-168c36ab1baa - {'path': '/', 'tag': 'c5581fe9-4699-45c4-837a-168c36ab1baa', 'options': 'recursive,force'} 2024-09-15T22:14:00.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:13:59 smithi077 ceph-mon[30980]: pgmap v50: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 602 KiB/s wr, 77 op/s 2024-09-15T22:14:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:13:59 smithi196 ceph-mon[35109]: pgmap v50: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 602 KiB/s wr, 77 op/s 2024-09-15T22:14:00.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:13:59 smithi117 ceph-mon[35021]: pgmap v50: 129 pgs: 129 active+clean; 7.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 602 KiB/s wr, 77 op/s 2024-09-15T22:14:00.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 25062 17.11 MB/sec warmup 9 sec latency 53.141 ms 2024-09-15T22:14:01.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 26651 16.02 MB/sec warmup 10 sec latency 53.673 ms 2024-09-15T22:14:02.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:01 smithi077 ceph-mon[30980]: pgmap v51: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-09-15T22:14:02.556 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:01 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:14:02.557 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:01 smithi117 ceph-mon[35021]: pgmap v51: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-09-15T22:14:02.557 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:01 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:14:02.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:01 smithi196 ceph-mon[35109]: pgmap v51: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-09-15T22:14:02.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:01 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:14:02.557 INFO:tasks.workunit.client.0.smithi077.stdout: 1 29300 15.88 MB/sec warmup 11 sec latency 12.679 ms 2024-09-15T22:14:03.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 32557 16.09 MB/sec warmup 12 sec latency 17.089 ms 2024-09-15T22:14:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:03 smithi077 ceph-mon[30980]: pgmap v52: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:04.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:03 smithi196 ceph-mon[35109]: pgmap v52: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:04.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:03 smithi117 ceph-mon[35021]: pgmap v52: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:04.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 35893 16.32 MB/sec warmup 13 sec latency 11.330 ms 2024-09-15T22:14:05.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 39057 16.41 MB/sec warmup 14 sec latency 15.316 ms 2024-09-15T22:14:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:05 smithi077 ceph-mon[30980]: pgmap v53: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:05 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:14:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:05 smithi196 ceph-mon[35109]: pgmap v53: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:05 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:14:06.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:05 smithi117 ceph-mon[35021]: pgmap v53: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-15T22:14:06.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:05 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:14:06.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 41525 15.93 MB/sec warmup 15 sec latency 36.468 ms 2024-09-15T22:14:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:06 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:14:07.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:06 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:14:07.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:06 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:14:07.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 44156 15.97 MB/sec warmup 16 sec latency 10.780 ms 2024-09-15T22:14:08.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:07 smithi077 ceph-mon[30980]: pgmap v54: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:08.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:07 smithi196 ceph-mon[35109]: pgmap v54: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:07 smithi117 ceph-mon[35021]: pgmap v54: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:08.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 46906 15.88 MB/sec warmup 17 sec latency 11.790 ms 2024-09-15T22:14:09.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 50083 16.04 MB/sec warmup 18 sec latency 8.909 ms 2024-09-15T22:14:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:09 smithi077 ceph-mon[30980]: pgmap v55: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-15T22:14:10.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:09 smithi196 ceph-mon[35109]: pgmap v55: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-15T22:14:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:09 smithi117 ceph-mon[35021]: pgmap v55: 129 pgs: 129 active+clean; 22 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-15T22:14:10.458 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53375 16.13 MB/sec warmup 19 sec latency 11.374 ms 2024-09-15T22:14:11.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 56111 15.95 MB/sec warmup 20 sec latency 17.104 ms 2024-09-15T22:14:12.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:11 smithi077 ceph-mon[30980]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-15T22:14:12.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:11 smithi196 ceph-mon[35109]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-15T22:14:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:11 smithi117 ceph-mon[35021]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-15T22:14:12.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 58781 15.83 MB/sec warmup 21 sec latency 12.691 ms 2024-09-15T22:14:13.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 61498 15.82 MB/sec warmup 22 sec latency 11.743 ms 2024-09-15T22:14:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:13 smithi077 ceph-mon[30980]: pgmap v57: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:14.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:13 smithi196 ceph-mon[35109]: pgmap v57: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:14.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:13 smithi117 ceph-mon[35021]: pgmap v57: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:14.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 64444 15.88 MB/sec warmup 23 sec latency 20.867 ms 2024-09-15T22:14:15.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 67735 16.00 MB/sec warmup 24 sec latency 18.421 ms 2024-09-15T22:14:16.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:15 smithi077 ceph-mon[30980]: pgmap v58: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:16.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:15 smithi196 ceph-mon[35109]: pgmap v58: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:15 smithi117 ceph-mon[35021]: pgmap v58: 129 pgs: 129 active+clean; 29 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-09-15T22:14:16.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 70594 15.90 MB/sec warmup 25 sec latency 16.044 ms 2024-09-15T22:14:17.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 71660 15.52 MB/sec warmup 26 sec latency 251.035 ms 2024-09-15T22:14:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:17 smithi077 ceph-mon[30980]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-15T22:14:18.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:17 smithi196 ceph-mon[35109]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-15T22:14:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:17 smithi117 ceph-mon[35021]: pgmap v59: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-09-15T22:14:18.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 74169 15.43 MB/sec warmup 27 sec latency 275.548 ms 2024-09-15T22:14:19.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 76073 15.28 MB/sec warmup 28 sec latency 21.697 ms 2024-09-15T22:14:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:19 smithi077 ceph-mon[30980]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-15T22:14:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:19 smithi196 ceph-mon[35109]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-15T22:14:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:19 smithi117 ceph-mon[35021]: pgmap v60: 129 pgs: 129 active+clean; 39 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-15T22:14:20.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 78772 15.24 MB/sec warmup 29 sec latency 10.686 ms 2024-09-15T22:14:21.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 81853 15.35 MB/sec warmup 30 sec latency 14.680 ms 2024-09-15T22:14:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:22 smithi077 ceph-mon[30980]: pgmap v61: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 410 op/s 2024-09-15T22:14:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:22 smithi196 ceph-mon[35109]: pgmap v61: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 410 op/s 2024-09-15T22:14:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:22 smithi117 ceph-mon[35021]: pgmap v61: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.5 MiB/s wr, 410 op/s 2024-09-15T22:14:22.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 84535 15.25 MB/sec warmup 31 sec latency 15.550 ms 2024-09-15T22:14:23.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 86735 15.16 MB/sec warmup 32 sec latency 11.106 ms 2024-09-15T22:14:24.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:24 smithi077 ceph-mon[30980]: pgmap v62: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:24 smithi196 ceph-mon[35109]: pgmap v62: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:24 smithi117 ceph-mon[35021]: pgmap v62: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:24.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89460 15.13 MB/sec warmup 33 sec latency 17.349 ms 2024-09-15T22:14:25.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 92589 15.23 MB/sec warmup 34 sec latency 7.414 ms 2024-09-15T22:14:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:26 smithi077 ceph-mon[30980]: pgmap v63: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:26 smithi196 ceph-mon[35109]: pgmap v63: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:26 smithi117 ceph-mon[35021]: pgmap v63: 129 pgs: 129 active+clean; 65 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.8 MiB/s wr, 323 op/s 2024-09-15T22:14:26.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 95596 15.22 MB/sec warmup 35 sec latency 13.072 ms 2024-09-15T22:14:27.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 98206 15.12 MB/sec warmup 36 sec latency 15.107 ms 2024-09-15T22:14:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:28 smithi077 ceph-mon[30980]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-15T22:14:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:28 smithi196 ceph-mon[35109]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-15T22:14:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:28 smithi117 ceph-mon[35021]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-15T22:14:28.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 100336 15.06 MB/sec warmup 37 sec latency 13.138 ms 2024-09-15T22:14:29.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103335 15.15 MB/sec warmup 38 sec latency 9.084 ms 2024-09-15T22:14:29.789 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:14:30.047 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:30 smithi077 ceph-mon[30980]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 342 op/s 2024-09-15T22:14:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:30 smithi196 ceph-mon[35109]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 342 op/s 2024-09-15T22:14:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:30 smithi117 ceph-mon[35021]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.7 MiB/s wr, 342 op/s 2024-09-15T22:14:30.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 106285 15.13 MB/sec warmup 39 sec latency 38.577 ms 2024-09-15T22:14:30.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 -- 172.21.15.77:0/3627714327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084100a80 msgr2=0x7f5084100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 --2- 172.21.15.77:0/3627714327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084100a80 0x7f5084100ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5074009930 tx=0x7f507402a420 comp rx=0 tx=0).stop 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 -- 172.21.15.77:0/3627714327 shutdown_connections 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 --2- 172.21.15.77:0/3627714327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 0x7f508410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 --2- 172.21.15.77:0/3627714327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084100a80 0x7f5084100ee0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 --2- 172.21.15.77:0/3627714327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084106a80 0x7f5084106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.544+0000 7f508b551640 1 -- 172.21.15.77:0/3627714327 >> 172.21.15.77:0/3627714327 conn(0x7f50840fc7a0 msgr2=0x7f50840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- 172.21.15.77:0/3627714327 shutdown_connections 2024-09-15T22:14:30.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- 172.21.15.77:0/3627714327 wait complete. 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 Processor -- start 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- start start 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100a80 0x7f50841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 0x7f5084199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110e40 con 0x7f5084106a80 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110cc0 con 0x7f5084100a80 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f508b551640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5084110fc0 con 0x7f5084101420 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.545+0000 7f5089aee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60156/0 (socket says 172.21.15.77:60156) 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 -- 172.21.15.77:0/3377378850 learned_addr learned my addr 172.21.15.77:0/3377378850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f50892ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100a80 0x7f50841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5088aec640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 0x7f5084199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 -- 172.21.15.77:0/3377378850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 msgr2=0x7f5084199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 0x7f5084199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 -- 172.21.15.77:0/3377378850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100a80 msgr2=0x7f50841992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100a80 0x7f50841992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 -- 172.21.15.77:0/3377378850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a4890 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f5089aee640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f507c00eb10 tx=0x7f507c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f507a7fc640 1 -- 172.21.15.77:0/3377378850 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c004b20 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f507a7fc640 1 -- 172.21.15.77:0/3377378850 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c004cc0 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f507a7fc640 1 -- 172.21.15.77:0/3377378850 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c010760 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a4a60 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.546+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a4f20 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f507a7fc640 1 -- 172.21.15.77:0/3377378850 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f507c005020 con 0x7f5084106a80 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 Processor -- start 2024-09-15T22:14:30.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 -- start start 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc6b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5626ccbd12e0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5626ccb78ce0 con 0x5626ccc6bdc0 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f508d92a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626ccbcd2a0 con 0x5626ccc76680 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc6b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc6b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60172/0 (socket says 172.21.15.77:60172) 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5088aec640 1 -- 172.21.15.77:0/1237870055 learned_addr learned my addr 172.21.15.77:0/1237870055 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5088aec640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f507a7fc640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 -- 172.21.15.77:0/1237870055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 msgr2=0x5626ccc78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5089aee640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 -- 172.21.15.77:0/1237870055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 msgr2=0x5626ccc6b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc6b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f50892ed640 1 -- 172.21.15.77:0/1237870055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ccb5d440 con 0x5626ccc6bdc0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.549+0000 7f5088aec640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.550+0000 7f507a7fc640 1 -- 172.21.15.77:0/3377378850 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f507c0d0590 con 0x7f5084106a80 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.550+0000 7f50892ed640 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc76140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c20 tx=0x7f5080011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78f2a601214cd59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.550+0000 7f506f7fe640 1 -- 172.21.15.77:0/1237870055 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080004050 con 0x5626ccc6bdc0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.550+0000 7f506f7fe640 1 -- 172.21.15.77:0/1237870055 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5080005500 con 0x5626ccc6bdc0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.550+0000 7f506f7fe640 1 -- 172.21.15.77:0/1237870055 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080004d90 con 0x5626ccc6bdc0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- 172.21.15.77:0/1237870055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 msgr2=0x5626ccc76140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f5089aee640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 0x7f50700a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f507c09c230 tx=0x7f507c06b220 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc76140 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c20 tx=0x7f5080011810 comp rx=0 tx=0).stop 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- 172.21.15.77:0/1237870055 shutdown_connections 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc76140 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- 172.21.15.77:0/1237870055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc6b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- 172.21.15.77:0/1237870055 >> 172.21.15.77:0/1237870055 conn(0x5626ccc67a30 msgr2=0x5626ccc67e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- 172.21.15.77:0/1237870055 shutdown_connections 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- 172.21.15.77:0/1237870055 wait complete. 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 -- start start 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 Processor -- start 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.551+0000 7f508d92a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc73710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5626ccc79250 con 0x5626ccc6b4a0 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5626ccb8c050 con 0x5626ccc6bdc0 2024-09-15T22:14:30.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5626ccc793d0 con 0x5626ccc76680 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 10 client.0 _ll_register_callbacks cb 0x5626ccbcc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60180/0 (socket says 172.21.15.77:60180) 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 -- 172.21.15.77:0/603435498 learned_addr learned my addr 172.21.15.77:0/603435498 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f50892ed640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc73710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 -- 172.21.15.77:0/603435498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 msgr2=0x5626ccc73710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc73710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 -- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 msgr2=0x5626ccc731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 -- 172.21.15.77:0/603435498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cc9423c0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f5088aec640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5074005940 tx=0x7f50740058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7470a0194b0fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f506cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:30.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074035040 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f5074004020 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5080005500 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f507404f050 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 10 client.16113 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.552+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626ccc84bd0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5080005500 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 10 client.16113 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 20 client.16113 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626ccc860e0 0x5626ccc88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 4 client.16113 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626ccbde850 con 0x5626ccc860e0 2024-09-15T22:14:30.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f50892ed640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626ccc860e0 0x5626ccc88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f50892ed640 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626ccc860e0 0x5626ccc88540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722e82a4ce4df010 server_cookie=a141c37344098e3 in_seq=0 out_seq=0 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.553+0000 7f506cff9640 10 client.16113 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f506cff9640 1 -- 172.21.15.77:0/603435498 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626ccbde850 con 0x5626ccc860e0 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f506cff9640 10 client.16113 handle_command_reply: tid=0 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 client.16113 shutdown 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 msgr2=0x5626ccc75f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5074005940 tx=0x7f50740058d0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 shutdown_connections 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626ccc860e0 0x5626ccc88540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc73710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 --2- 172.21.15.77:0/603435498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc75f20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f506cff9640 0 client.16113 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:30.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 >> 172.21.15.77:0/603435498 conn(0x5626ccc67a30 msgr2=0x5626ccc6d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 shutdown_connections 2024-09-15T22:14:30.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 1 -- 172.21.15.77:0/603435498 wait complete. 2024-09-15T22:14:30.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.554+0000 7f508d92a740 20 client.16113 trim_cache size 0 max 16384 2024-09-15T22:14:30.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 Processor -- start 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 -- start start 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50841aede0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b09e0 con 0x5626ccc6bdc0 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.573+0000 7f508d92a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50841af4b0 con 0x5626ccc76680 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f50892ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60182/0 (socket says 172.21.15.77:60182) 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 -- 172.21.15.77:0/2865080585 learned_addr learned my addr 172.21.15.77:0/2865080585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 -- 172.21.15.77:0/2865080585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 msgr2=0x5626ccc8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 -- 172.21.15.77:0/2865080585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 msgr2=0x5626ccc6e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc6e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 -- 172.21.15.77:0/2865080585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ccc67860 con 0x5626ccc6b4a0 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f50892ed640 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc6e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f5088aec640 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5074005ec0 tx=0x7f5074000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711a243592179439 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f506effd640 1 -- 172.21.15.77:0/2865080585 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074035070 con 0x5626ccc6b4a0 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f506effd640 1 -- 172.21.15.77:0/2865080585 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.574+0000 7f506effd640 1 -- 172.21.15.77:0/2865080585 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074003e90 con 0x5626ccc6b4a0 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- 172.21.15.77:0/2865080585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 msgr2=0x5626ccc82510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5074005ec0 tx=0x7f5074000bc0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- 172.21.15.77:0/2865080585 shutdown_connections 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc76680 0x5626ccc8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc6bdc0 0x5626ccc6e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 --2- 172.21.15.77:0/2865080585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc82510 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- 172.21.15.77:0/2865080585 >> 172.21.15.77:0/2865080585 conn(0x5626ccc67a30 msgr2=0x5626ccc6e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- 172.21.15.77:0/2865080585 shutdown_connections 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- 172.21.15.77:0/2865080585 wait complete. 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 -- start start 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.575+0000 7f508d92a740 1 Processor -- start 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc6bdc0 0x5626ccc79fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc76680 0x5626ccc7a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50841b0250 con 0x5626ccc6b4a0 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50841afb80 con 0x5626ccc76680 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50841acdc0 con 0x5626ccc6bdc0 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f508d92a740 10 client.0 _ll_register_callbacks cb 0x7f5074009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:30.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60192/0 (socket says 172.21.15.77:60192) 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 -- 172.21.15.77:0/3122385674 learned_addr learned my addr 172.21.15.77:0/3122385674 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f50892ed640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc6bdc0 0x5626ccc79fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc76680 0x5626ccc7a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 -- 172.21.15.77:0/3122385674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc6bdc0 msgr2=0x5626ccc79fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc6bdc0 0x5626ccc79fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 -- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc76680 msgr2=0x5626ccc7a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc76680 0x5626ccc7a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.576+0000 7f5088aec640 1 -- 172.21.15.77:0/3122385674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ccc7abf0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f5088aec640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5074005e90 tx=0x7f5074004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9304ffcb8e5975ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074035070 con 0x5626ccc6b4a0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f5074004970 con 0x5626ccc6b4a0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50740035f0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5074003c30 con 0x5626ccc6b4a0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 10 client.16119 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5626cccf0cb0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50740035f0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f50740535f0 con 0x5626ccc6b4a0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 10 client.16119 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 20 client.16119 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626cccf1320 0x5626cccf3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 4 client.16119 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.577+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626ccc73aa0 con 0x5626cccf1320 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f50892ed640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626cccf1320 0x5626cccf3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f506cff9640 10 client.16119 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f50892ed640 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626cccf1320 0x5626cccf3740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1edffd948b82ed65 server_cookie=914b408d01cb3056 in_seq=0 out_seq=0 2024-09-15T22:14:30.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f506cff9640 1 -- 172.21.15.77:0/3122385674 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626ccc73aa0 con 0x5626cccf1320 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f506cff9640 10 client.16119 handle_command_reply: tid=0 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 client.16119 shutdown 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 msgr2=0x5626ccc7ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5074005e90 tx=0x7f5074004d90 comp rx=0 tx=0).stop 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 shutdown_connections 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5626cccf1320 0x5626cccf3740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5626ccc76680 0x5626ccc7a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f506cff9640 0 client.16119 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626ccc6bdc0 0x5626ccc79fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 --2- 172.21.15.77:0/3122385674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5626ccc6b4a0 0x5626ccc7ec90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 >> 172.21.15.77:0/3122385674 conn(0x5626ccc67a30 msgr2=0x5626ccc716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 shutdown_connections 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.578+0000 7f508d92a740 1 -- 172.21.15.77:0/3122385674 wait complete. 2024-09-15T22:14:30.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508d92a740 20 client.16119 trim_cache size 0 max 16384 2024-09-15T22:14:30.587 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:14:30.587 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:14:30.587 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:14:30.587 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 msgr2=0x7f50700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 0x7f50700a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f507c09c230 tx=0x7f507c06b220 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 msgr2=0x7f50841a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f507c00eb10 tx=0x7f507c00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 shutdown_connections 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f50700a3430 0x7f50700a58f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5084106a80 0x7f50841a4350 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5084101420 0x7f5084199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 --2- 172.21.15.77:0/3377378850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100a80 0x7f50841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.579+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 >> 172.21.15.77:0/3377378850 conn(0x7f50840fc7a0 msgr2=0x7f50840fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.580+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 shutdown_connections 2024-09-15T22:14:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:30.580+0000 7f508b551640 1 -- 172.21.15.77:0/3377378850 wait complete. 2024-09-15T22:14:31.420 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:14:31.420 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:31.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 108366 15.00 MB/sec warmup 40 sec latency 87.540 ms 2024-09-15T22:14:31.691 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:32.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:32 smithi077 ceph-mon[30980]: pgmap v66: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.4 MiB/s wr, 433 op/s 2024-09-15T22:14:32.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- 172.21.15.77:0/942165357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 msgr2=0x7fd18010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:32.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 --2- 172.21.15.77:0/942165357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18010ea90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd178009ca0 tx=0x7fd17802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- 172.21.15.77:0/942165357 shutdown_connections 2024-09-15T22:14:32.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 --2- 172.21.15.77:0/942165357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18010ea90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 --2- 172.21.15.77:0/942165357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd1801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 --2- 172.21.15.77:0/942165357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd1801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- 172.21.15.77:0/942165357 >> 172.21.15.77:0/942165357 conn(0x7fd1800fd8c0 msgr2=0x7fd1800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- 172.21.15.77:0/942165357 shutdown_connections 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- 172.21.15.77:0/942165357 wait complete. 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 Processor -- start 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.181+0000 7fd186a4b640 1 -- start start 2024-09-15T22:14:32.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd180198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd180198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd180111f70 con 0x7fd180106f10 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd180111df0 con 0x7fd180102790 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd186a4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801120f0 con 0x7fd180101de0 2024-09-15T22:14:32.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd180198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60202/0 (socket says 172.21.15.77:60202) 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd180198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38124/0 (socket says 172.21.15.77:38124) 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 -- 172.21.15.77:0/2376550998 learned_addr learned my addr 172.21.15.77:0/2376550998 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd1777fe640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd180198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:32.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 -- 172.21.15.77:0/2376550998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 msgr2=0x7fd180198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:32.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd180198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 -- 172.21.15.77:0/2376550998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 msgr2=0x7fd180198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:32.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd180198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 -- 172.21.15.77:0/2376550998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18019d2e0 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd1777fe640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd180198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.182+0000 7fd184fe8640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd178031a20 tx=0x7fd178031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.183+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178003b70 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.183+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd178003d10 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.183+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18019d570 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.183+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17802fbd0 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.183+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801a4eb0 con 0x7fd180106f10 2024-09-15T22:14:32.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.184+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1801021c0 con 0x7fd180106f10 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.186+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd17802fd70 con 0x7fd180106f10 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.187+0000 7fd17dffb640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 0x7fd16c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.187+0000 7fd17ffff640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 0x7fd16c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.187+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd1780f1170 con 0x7fd180106f10 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.187+0000 7fd17ffff640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 0x7fd16c0a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd17000abe0 tx=0x7fd17000a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:32.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.190+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1780bd9b0 con 0x7fd180106f10 2024-09-15T22:14:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:32 smithi196 ceph-mon[35109]: pgmap v66: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.4 MiB/s wr, 433 op/s 2024-09-15T22:14:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:32 smithi117 ceph-mon[35021]: pgmap v66: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.4 MiB/s wr, 433 op/s 2024-09-15T22:14:32.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.401+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd180199690 con 0x7fd180106f10 2024-09-15T22:14:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.402+0000 7fd17dffb640 1 -- 172.21.15.77:0/2376550998 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fd180199690 con 0x7fd180106f10 2024-09-15T22:14:32.405 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:14:32.405 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:14:32.406 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 msgr2=0x7fd16c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 0x7fd16c0a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd17000abe0 tx=0x7fd17000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 msgr2=0x7fd18019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd178031a20 tx=0x7fd178031a50 comp rx=0 tx=0).stop 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 shutdown_connections 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd16c0a3430 0x7fd16c0a58f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd180106f10 0x7fd18019cc00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd180102790 0x7fd180198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 --2- 172.21.15.77:0/2376550998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd180101de0 0x7fd180198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:32.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 >> 172.21.15.77:0/2376550998 conn(0x7fd1800fd8c0 msgr2=0x7fd1800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:32.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 shutdown_connections 2024-09-15T22:14:32.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:32.405+0000 7fd186a4b640 1 -- 172.21.15.77:0/2376550998 wait complete. 2024-09-15T22:14:32.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 110742 14.98 MB/sec warmup 41 sec latency 21.031 ms 2024-09-15T22:14:33.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2376550998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:33.126 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:14:33.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:33 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2376550998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:33.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:33 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2376550998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:33.437 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:33.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 113868 15.06 MB/sec warmup 42 sec latency 16.307 ms 2024-09-15T22:14:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 -- 172.21.15.77:0/2647233960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 msgr2=0x7fcd6c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 --2- 172.21.15.77:0/2647233960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c101200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009900 tx=0x7fcd5c02a480 comp rx=0 tx=0).stop 2024-09-15T22:14:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 -- 172.21.15.77:0/2647233960 shutdown_connections 2024-09-15T22:14:33.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 --2- 172.21.15.77:0/2647233960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 --2- 172.21.15.77:0/2647233960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102570 0x7fcd6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 --2- 172.21.15.77:0/2647233960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c101200 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 -- 172.21.15.77:0/2647233960 >> 172.21.15.77:0/2647233960 conn(0x7fcd6c0fc7d0 msgr2=0x7fcd6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 -- 172.21.15.77:0/2647233960 shutdown_connections 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.946+0000 7fcd72432640 1 -- 172.21.15.77:0/2647233960 wait complete. 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 Processor -- start 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 -- start start 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102570 0x7fcd6c196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c10fe80 con 0x7fcd6c100e20 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c10fd00 con 0x7fcd6c102ef0 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.947+0000 7fcd72432640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c110000 con 0x7fcd6c102570 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60218/0 (socket says 172.21.15.77:60218) 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 -- 172.21.15.77:0/3849229730 learned_addr learned my addr 172.21.15.77:0/3849229730 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102570 0x7fcd6c196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 -- 172.21.15.77:0/3849229730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102570 msgr2=0x7fcd6c196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102570 0x7fcd6c196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 -- 172.21.15.77:0/3849229730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 msgr2=0x7fcd6c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 -- 172.21.15.77:0/3849229730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c19b270 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd6bfff640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd697fa640 1 -- 172.21.15.77:0/3849229730 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c03f070 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd697fa640 1 -- 172.21.15.77:0/3849229730 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c031c30 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c19b500 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd697fa640 1 -- 172.21.15.77:0/3849229730 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c035030 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.948+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c1a2e40 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd697fa640 1 -- 172.21.15.77:0/3849229730 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcd5c042070 con 0x7fcd6c100e20 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 Processor -- start 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 -- start start 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267fb40 0x558ea267e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea267ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd697fa640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 0x7fcd540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558ea25e42f0 con 0x558ea2689370 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ea258bcf0 con 0x558ea267e5d0 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.951+0000 7fcd7480b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ea25e02b0 con 0x558ea267fb40 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60220/0 (socket says 172.21.15.77:60220) 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 -- 172.21.15.77:0/962332134 learned_addr learned my addr 172.21.15.77:0/962332134 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd6bfff640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 0x7fcd540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea267ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267fb40 0x558ea267e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd697fa640 1 -- 172.21.15.77:0/3849229730 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcd5c0f0380 con 0x7fcd6c100e20 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 -- 172.21.15.77:0/962332134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267fb40 msgr2=0x558ea267e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267fb40 0x558ea267e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 -- 172.21.15.77:0/962332134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 msgr2=0x558ea267ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea267ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 -- 172.21.15.77:0/962332134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea25703a0 con 0x558ea2689370 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea267ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd6bfff640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 0x7fcd540a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0b8d40 tx=0x7fcd5c086d60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd709cf640 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fd0 tx=0x7fcd64009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e797b0d5ccd2f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd527fc640 1 -- 172.21.15.77:0/962332134 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64013a10 con 0x558ea2689370 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd527fc640 1 -- 172.21.15.77:0/962332134 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd64003e50 con 0x558ea2689370 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.952+0000 7fcd527fc640 1 -- 172.21.15.77:0/962332134 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64004430 con 0x558ea2689370 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 -- 172.21.15.77:0/962332134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 msgr2=0x558ea268b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fd0 tx=0x7fcd64009500 comp rx=0 tx=0).stop 2024-09-15T22:14:33.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 -- 172.21.15.77:0/962332134 shutdown_connections 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea268b760 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea267ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 --2- 172.21.15.77:0/962332134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267fb40 0x558ea267e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 -- 172.21.15.77:0/962332134 >> 172.21.15.77:0/962332134 conn(0x558ea2647310 msgr2=0x558ea267ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 -- 172.21.15.77:0/962332134 shutdown_connections 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.953+0000 7fcd7480b740 1 -- 172.21.15.77:0/962332134 wait complete. 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 -- start start 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 Processor -- start 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea2688d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea2689370 0x558ea2686610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558ea268c070 con 0x558ea267fb40 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ea259f060 con 0x558ea267e5d0 2024-09-15T22:14:33.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ea268c1f0 con 0x558ea2689370 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd7480b740 10 client.0 _ll_register_callbacks cb 0x558ea22f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60234/0 (socket says 172.21.15.77:60234) 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea2689370 0x558ea2686610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3990121997 learned_addr learned my addr 172.21.15.77:0/3990121997 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.954+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea2688d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3990121997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea2689370 msgr2=0x558ea2686610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea2689370 0x558ea2686610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3990121997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 msgr2=0x558ea2688d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea2688d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3990121997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea23553c0 con 0x558ea267fb40 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcd60012720 tx=0x7fcd60012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e54b6bd1e38137 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60004030 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcd600041d0 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd64003e50 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd60025050 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd7480b740 10 client.16137 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ea2697b70 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.955+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd64003e50 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcd60003b60 con 0x558ea267fb40 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 10 client.16137 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 10 client.16137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 20 client.16137 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2699080 0x558ea269b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 4 client.16137 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ea25f1860 con 0x558ea2699080 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2699080 0x558ea269b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd4b7fe640 10 client.16137 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.956+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2699080 0x558ea269b470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2b99ccd24e6fec6 server_cookie=e39e9051efb327ac in_seq=0 out_seq=0 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/3990121997 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ea25f1860 con 0x558ea2699080 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd4b7fe640 10 client.16137 handle_command_reply: tid=0 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 client.16137 shutdown 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 msgr2=0x558ea2686020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcd60012720 tx=0x7fcd60012be0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 shutdown_connections 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2699080 0x558ea269b470 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea2689370 0x558ea2686610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2686020 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3990121997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267e5d0 0x558ea2688d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 >> 172.21.15.77:0/3990121997 conn(0x558ea2647310 msgr2=0x558ea2680960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd4b7fe640 0 client.16137 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:14:33.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 shutdown_connections 2024-09-15T22:14:33.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 1 -- 172.21.15.77:0/3990121997 wait complete. 2024-09-15T22:14:33.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.957+0000 7fcd7480b740 20 client.16137 trim_cache size 0 max 16384 2024-09-15T22:14:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 Processor -- start 2024-09-15T22:14:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 -- start start 2024-09-15T22:14:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea267ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea2689370 0x558ea26858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1accc0 con 0x558ea267fb40 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1ae8c0 con 0x558ea2689370 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.971+0000 7fcd7480b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1ad390 con 0x558ea267e5d0 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea267ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60236/0 (socket says 172.21.15.77:60236) 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3499246781 learned_addr learned my addr 172.21.15.77:0/3499246781 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea267ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38168/0 (socket says 172.21.15.77:38168) 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd709cf640 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea2689370 0x558ea26858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3499246781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 msgr2=0x558ea267ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea267ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3499246781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea2689370 msgr2=0x558ea26858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea2689370 0x558ea26858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 -- 172.21.15.77:0/3499246781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ea267b960 con 0x558ea267fb40 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcd600126f0 tx=0x7fcd60009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c728c703efa3efef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd51ffb640 1 -- 172.21.15.77:0/3499246781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6002e070 con 0x558ea267fb40 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd51ffb640 1 -- 172.21.15.77:0/3499246781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd600056f0 con 0x558ea267fb40 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.972+0000 7fcd51ffb640 1 -- 172.21.15.77:0/3499246781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60022040 con 0x558ea267fb40 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 -- 172.21.15.77:0/3499246781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 msgr2=0x558ea2695310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcd600126f0 tx=0x7fcd60009f40 comp rx=0 tx=0).stop 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 -- 172.21.15.77:0/3499246781 shutdown_connections 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea2689370 0x558ea26858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea267fb40 0x558ea2695310 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 --2- 172.21.15.77:0/3499246781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea267ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 -- 172.21.15.77:0/3499246781 >> 172.21.15.77:0/3499246781 conn(0x558ea2647310 msgr2=0x558ea267a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 -- 172.21.15.77:0/3499246781 shutdown_connections 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.973+0000 7fcd7480b740 1 -- 172.21.15.77:0/3499246781 wait complete. 2024-09-15T22:14:33.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 -- start start 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 Processor -- start 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea2691900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 0x558ea2682a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1ae130 con 0x558ea2689370 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1ada60 con 0x558ea267fb40 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1aaca0 con 0x558ea267e5d0 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd7480b740 10 client.0 _ll_register_callbacks cb 0x7fcd60009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd709cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60250/0 (socket says 172.21.15.77:60250) 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd709cf640 1 -- 172.21.15.77:0/805341987 learned_addr learned my addr 172.21.15.77:0/805341987 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd709cf640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea2691900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.974+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 0x558ea2682a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 -- 172.21.15.77:0/805341987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 msgr2=0x558ea2691900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea2691900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 -- 172.21.15.77:0/805341987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 msgr2=0x558ea2682a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 0x558ea2682a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 -- 172.21.15.77:0/805341987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ea2695850 con 0x558ea2689370 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 0x558ea2682a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd709cf640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400cbf0 tx=0x7fcd60009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10baca3eeb7d0606 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd640050c0 con 0x558ea2689370 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcd64005260 con 0x558ea2689370 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd600056f0 con 0x558ea2689370 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd64005480 con 0x558ea2689370 2024-09-15T22:14:33.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd7480b740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ea267a4f0 con 0x558ea2689370 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.975+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd600056f0 con 0x558ea2689370 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcd6400cce0 con 0x558ea2689370 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 10 client.16143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 20 client.16143 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2703b40 0x558ea2705f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 4 client.16143 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ea25ebd70 con 0x558ea2703b40 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2703b40 0x558ea2705f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd4b7fe640 10 client.16143 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd6b7fe640 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2703b40 0x558ea2705f30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35362124949bfbee server_cookie=f4915a0b931bb6d3 in_seq=0 out_seq=0 2024-09-15T22:14:33.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd4b7fe640 1 -- 172.21.15.77:0/805341987 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ea25ebd70 con 0x558ea2703b40 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.976+0000 7fcd4b7fe640 10 client.16143 handle_command_reply: tid=0 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 client.16143 shutdown 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 msgr2=0x558ea2682fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400cbf0 tx=0x7fcd60009e50 comp rx=0 tx=0).stop 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 shutdown_connections 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558ea2703b40 0x558ea2705f30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558ea2689370 0x558ea2682fc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ea267fb40 0x558ea2682a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 --2- 172.21.15.77:0/805341987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ea267e5d0 0x558ea2691900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 >> 172.21.15.77:0/805341987 conn(0x558ea2647310 msgr2=0x558ea2685890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd4b7fe640 0 client.16143 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 shutdown_connections 2024-09-15T22:14:33.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 1 -- 172.21.15.77:0/805341987 wait complete. 2024-09-15T22:14:33.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.977+0000 7fcd7480b740 20 client.16143 trim_cache size 0 max 16384 2024-09-15T22:14:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:14:33.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 msgr2=0x7fcd540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 0x7fcd540a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0b8d40 tx=0x7fcd5c086d60 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 msgr2=0x7fcd6c1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c003c10 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 shutdown_connections 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcd540a33b0 0x7fcd540a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd6c102ef0 0x7fcd6c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd6c102570 0x7fcd6c196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 --2- 172.21.15.77:0/3849229730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd6c100e20 0x7fcd6c1962c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 >> 172.21.15.77:0/3849229730 conn(0x7fcd6c0fc7d0 msgr2=0x7fcd6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 shutdown_connections 2024-09-15T22:14:33.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:33.978+0000 7fcd72432640 1 -- 172.21.15.77:0/3849229730 wait complete. 2024-09-15T22:14:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:34 smithi077 ceph-mon[30980]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:34 smithi196 ceph-mon[35109]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:34 smithi117 ceph-mon[35021]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:34.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 116709 15.03 MB/sec warmup 43 sec latency 30.734 ms 2024-09-15T22:14:34.779 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:14:35.038 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:35.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 118673 14.94 MB/sec warmup 44 sec latency 39.080 ms 2024-09-15T22:14:35.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 -- 172.21.15.77:0/505625193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c103650 msgr2=0x7f8b0c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 --2- 172.21.15.77:0/505625193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c103650 0x7f8b0c10bb40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009900 tx=0x7f8b0002a480 comp rx=0 tx=0).stop 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 -- 172.21.15.77:0/505625193 shutdown_connections 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 --2- 172.21.15.77:0/505625193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c10c160 0x7f8b0c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 --2- 172.21.15.77:0/505625193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c103650 0x7f8b0c10bb40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 --2- 172.21.15.77:0/505625193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 -- 172.21.15.77:0/505625193 >> 172.21.15.77:0/505625193 conn(0x7f8b0c0fc790 msgr2=0x7f8b0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 -- 172.21.15.77:0/505625193 shutdown_connections 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 -- 172.21.15.77:0/505625193 wait complete. 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.541+0000 7f8b115cf640 1 Processor -- start 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 -- start start 2024-09-15T22:14:35.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c103650 0x7f8b0c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 0x7f8b0c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c111fa0 con 0x7f8b0c10c160 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c111e20 con 0x7f8b0c102d30 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b115cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c112120 con 0x7f8b0c103650 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 0x7f8b0c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 0x7f8b0c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60260/0 (socket says 172.21.15.77:60260) 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0b7fe640 1 -- 172.21.15.77:0/4107991287 learned_addr learned my addr 172.21.15.77:0/4107991287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c103650 0x7f8b0c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 -- 172.21.15.77:0/4107991287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c103650 msgr2=0x7f8b0c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c103650 0x7f8b0c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 -- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 msgr2=0x7f8b0c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 0x7f8b0c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.542+0000 7f8b0affd640 1 -- 172.21.15.77:0/4107991287 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c1a6960 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8b0affd640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c100990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e9f0 tx=0x7f8afc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8af3fff640 1 -- 172.21.15.77:0/4107991287 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00ccc0 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c1a6c50 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8af3fff640 1 -- 172.21.15.77:0/4107991287 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc010910 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8af3fff640 1 -- 172.21.15.77:0/4107991287 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc018960 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.543+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a7190 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.546+0000 7f8af3fff640 1 -- 172.21.15.77:0/4107991287 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8afc010ac0 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.546+0000 7f8b139a8740 1 Processor -- start 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.547+0000 7f8af3fff640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 0x7f8af40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.547+0000 7f8af3fff640 1 -- 172.21.15.77:0/4107991287 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8afc0d0530 con 0x7f8b0c102d30 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 -- start start 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 0x7f8af40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e93ab40 0x563e2e939090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e946760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563e2e89f2f0 con 0x563e2e9395d0 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e2e846cf0 con 0x563e2e944370 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b139a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e2e89b2b0 con 0x563e2e93ab40 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60276/0 (socket says 172.21.15.77:60276) 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.548+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/1349079752 learned_addr learned my addr 172.21.15.77:0/1349079752 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 0x7f8af40a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009ed0 tx=0x7f8b040094d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e946760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/1349079752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e93ab40 msgr2=0x563e2e939090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e93ab40 0x563e2e939090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/1349079752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 msgr2=0x563e2e946760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e946760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/1349079752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2e82b3a0 con 0x563e2e9395d0 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.549+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009900 tx=0x7f8b00032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd9caa5ffaaf7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.551+0000 7f8af17fa640 1 -- 172.21.15.77:0/1349079752 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0003f070 con 0x563e2e9395d0 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.551+0000 7f8af17fa640 1 -- 172.21.15.77:0/1349079752 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b00032e20 con 0x563e2e9395d0 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.551+0000 7f8af17fa640 1 -- 172.21.15.77:0/1349079752 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00035070 con 0x563e2e9395d0 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- 172.21.15.77:0/1349079752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 msgr2=0x563e2e939a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009900 tx=0x7f8b00032c30 comp rx=0 tx=0).stop 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- 172.21.15.77:0/1349079752 shutdown_connections 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e946760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e939a10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1349079752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e93ab40 0x563e2e939090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- 172.21.15.77:0/1349079752 >> 172.21.15.77:0/1349079752 conn(0x563e2e902310 msgr2=0x563e2e935c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- 172.21.15.77:0/1349079752 shutdown_connections 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- 172.21.15.77:0/1349079752 wait complete. 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.552+0000 7f8b139a8740 1 -- start start 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 Processor -- start 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e943e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e944370 0x563e2e941860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563e2e947010 con 0x563e2e9395d0 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e2e85a060 con 0x563e2e93ab40 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e2e947190 con 0x563e2e944370 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b139a8740 10 client.0 _ll_register_callbacks cb 0x563e2e5ad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:35.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58162/0 (socket says 172.21.15.77:58162) 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.553+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/735890448 learned_addr learned my addr 172.21.15.77:0/735890448 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e944370 msgr2=0x563e2e941860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e944370 0x563e2e941860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0affd640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e943e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/735890448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 msgr2=0x563e2e943e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e943e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/735890448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2e6103c0 con 0x563e2e93ab40 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.554+0000 7f8b0affd640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e943e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.555+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b00005350 tx=0x7f8b00031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce15de8ac33bfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.555+0000 7f8aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.555+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0006d060 con 0x563e2e93ab40 2024-09-15T22:14:35.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.555+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8b00004440 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.555+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b00032e20 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.556+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b0003f030 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.556+0000 7f8b139a8740 10 client.25544 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.556+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e2e94fa40 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b00032e20 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8b00004a60 con 0x563e2e93ab40 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 10 client.25544 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 10 client.25544 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 20 client.25544 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:35.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e950e50 0x563e2e9532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 4 client.25544 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e2e8ac860 con 0x563e2e950e50 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.557+0000 7f8b0affd640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e950e50 0x563e2e9532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.558+0000 7f8aeeff5640 10 client.25544 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.558+0000 7f8b0affd640 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e950e50 0x563e2e9532b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c635acaec2e491be server_cookie=79b9baccba48ab5d in_seq=0 out_seq=0 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.559+0000 7f8aeeff5640 1 -- 172.21.15.77:0/735890448 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e2e8ac860 con 0x563e2e950e50 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.559+0000 7f8aeeff5640 10 client.25544 handle_command_reply: tid=0 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.559+0000 7f8b139a8740 1 client.25544 shutdown 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.559+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 msgr2=0x563e2e9410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.559+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b00005350 tx=0x7f8b00031850 comp rx=0 tx=0).stop 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 shutdown_connections 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e950e50 0x563e2e9532b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e944370 0x563e2e941860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e9410d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8aeeff5640 0 client.25544 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 --2- 172.21.15.77:0/735890448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e9395d0 0x563e2e943e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 >> 172.21.15.77:0/735890448 conn(0x563e2e902310 msgr2=0x563e2e93b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 shutdown_connections 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 1 -- 172.21.15.77:0/735890448 wait complete. 2024-09-15T22:14:35.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.560+0000 7f8b139a8740 20 client.25544 trim_cache size 0 max 16384 2024-09-15T22:14:35.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 Processor -- start 2024-09-15T22:14:35.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 -- start start 2024-09-15T22:14:35.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e962530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e944370 0x563e2e945ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1acd60 con 0x563e2e944370 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ae960 con 0x563e2e93ab40 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b139a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ad430 con 0x563e2e9395d0 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.581+0000 7f8b0affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e962530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e944370 0x563e2e945ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58166/0 (socket says 172.21.15.77:58166) 2024-09-15T22:14:35.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/3232465668 learned_addr learned my addr 172.21.15.77:0/3232465668 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/3232465668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 msgr2=0x563e2e962530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e962530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/3232465668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e944370 msgr2=0x563e2e945ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e944370 0x563e2e945ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 -- 172.21.15.77:0/3232465668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2e9594a0 con 0x563e2e93ab40 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.582+0000 7f8b0a7fc640 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b00004fd0 tx=0x7f8b00032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd60de0e8fbef5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.583+0000 7f8af0ff9640 1 -- 172.21.15.77:0/3232465668 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0006d040 con 0x563e2e93ab40 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.583+0000 7f8af0ff9640 1 -- 172.21.15.77:0/3232465668 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b00004ba0 con 0x563e2e93ab40 2024-09-15T22:14:35.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.583+0000 7f8af0ff9640 1 -- 172.21.15.77:0/3232465668 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00077080 con 0x563e2e93ab40 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 -- 172.21.15.77:0/3232465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 msgr2=0x563e2e945760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b00004fd0 tx=0x7f8b00032c30 comp rx=0 tx=0).stop 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 -- 172.21.15.77:0/3232465668 shutdown_connections 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e944370 0x563e2e945ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e93ab40 0x563e2e945760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 --2- 172.21.15.77:0/3232465668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e962530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 -- 172.21.15.77:0/3232465668 >> 172.21.15.77:0/3232465668 conn(0x563e2e902310 msgr2=0x563e2e943150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 -- 172.21.15.77:0/3232465668 shutdown_connections 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.584+0000 7f8b139a8740 1 -- 172.21.15.77:0/3232465668 wait complete. 2024-09-15T22:14:35.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 -- start start 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 Processor -- start 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e93db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 0x563e2e93e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ae1d0 con 0x563e2e93ab40 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1adb00 con 0x563e2e944370 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1aad40 con 0x563e2e9395d0 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.585+0000 7f8b139a8740 10 client.0 _ll_register_callbacks cb 0x7f8b04007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 0x563e2e93e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58172/0 (socket says 172.21.15.77:58172) 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 0x563e2e93e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60298/0 (socket says 172.21.15.77:60298) 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0b7fe640 1 -- 172.21.15.77:0/1140532902 learned_addr learned my addr 172.21.15.77:0/1140532902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.586+0000 7f8b0affd640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e93db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 -- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 msgr2=0x563e2e93db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e93db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 -- 172.21.15.77:0/1140532902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 msgr2=0x563e2e93e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 0x563e2e93e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 -- 172.21.15.77:0/1140532902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2e94dad0 con 0x563e2e944370 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8b0b7fe640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b04016ef0 tx=0x7f8b04004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f7bdfeb3d7c98b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:35.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.587+0000 7f8aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:14:35.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04028060 con 0x563e2e944370 2024-09-15T22:14:35.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8b04023640 con 0x563e2e944370 2024-09-15T22:14:35.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b00004ba0 con 0x563e2e944370 2024-09-15T22:14:35.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8b04023840 con 0x563e2e944370 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8b139a8740 10 client.25550 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e2e9beaa0 con 0x563e2e944370 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.588+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b040223e0 con 0x563e2e944370 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8b0402c3a0 con 0x563e2e944370 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 10 client.25550 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 10 client.25550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 20 client.25550 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e9bf180 0x563e2e9c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 4 client.25550 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8b00004ba0 con 0x563e2e9bf180 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.589+0000 7f8b0affd640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e9bf180 0x563e2e9c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:35.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8aeeff5640 10 client.25550 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8b0affd640 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e9bf180 0x563e2e9c15c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed9d2c0d1cfd28a0 server_cookie=ae413be3c0837a31 in_seq=0 out_seq=0 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8aeeff5640 1 -- 172.21.15.77:0/1140532902 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8b00004ba0 con 0x563e2e9bf180 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8aeeff5640 10 client.25550 handle_command_reply: tid=0 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8b139a8740 1 client.25550 shutdown 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 msgr2=0x563e2e94d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.590+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b04016ef0 tx=0x7f8b04004a40 comp rx=0 tx=0).stop 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 shutdown_connections 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e2e9bf180 0x563e2e9c15c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e2e944370 0x563e2e94d430 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.596 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:14:35.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e2e93ab40 0x563e2e93e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 --2- 172.21.15.77:0/1140532902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e2e9395d0 0x563e2e93db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 >> 172.21.15.77:0/1140532902 conn(0x563e2e902310 msgr2=0x563e2e942930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8aeeff5640 0 client.25550 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 shutdown_connections 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 1 -- 172.21.15.77:0/1140532902 wait complete. 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.591+0000 7f8b139a8740 20 client.25550 trim_cache size 0 max 16384 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.592+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 msgr2=0x7f8af40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.592+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 0x7f8af40a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009ed0 tx=0x7f8b040094d0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.592+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 msgr2=0x7f8b0c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.592+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c100990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e9f0 tx=0x7f8afc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.592+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 shutdown_connections 2024-09-15T22:14:35.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8af40a33b0 0x7f8af40a5870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b0c10c160 0x7f8b0c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b0c103650 0x7f8b0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 --2- 172.21.15.77:0/4107991287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b0c102d30 0x7f8b0c100990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 >> 172.21.15.77:0/4107991287 conn(0x7f8b0c0fc790 msgr2=0x7f8b0c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 shutdown_connections 2024-09-15T22:14:35.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:35.593+0000 7f8b115cf640 1 -- 172.21.15.77:0/4107991287 wait complete. 2024-09-15T22:14:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:36 smithi077 ceph-mon[30980]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:36 smithi196 ceph-mon[35109]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:36 smithi117 ceph-mon[35021]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-09-15T22:14:36.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121177 14.92 MB/sec warmup 45 sec latency 32.877 ms 2024-09-15T22:14:36.515 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:14:36.829 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:37.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:37 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:37.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:37 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:37.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:37 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:37.326 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:37 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:14:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:37 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:37 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:37 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:37.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:37 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:14:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:37 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:37 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:37 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:37 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:14:37.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 124139 14.97 MB/sec warmup 46 sec latency 36.423 ms 2024-09-15T22:14:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 -- 172.21.15.77:0/1136566523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90106f20 msgr2=0x7f7d9010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 --2- 172.21.15.77:0/1136566523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90106f20 0x7f7d9010eaa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009ca0 tx=0x7f7d8802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 -- 172.21.15.77:0/1136566523 shutdown_connections 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 --2- 172.21.15.77:0/1136566523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90106f20 0x7f7d9010eaa0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 --2- 172.21.15.77:0/1136566523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d901027a0 0x7f7d901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 --2- 172.21.15.77:0/1136566523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d90101df0 0x7f7d901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.563+0000 7f7d97645640 1 -- 172.21.15.77:0/1136566523 >> 172.21.15.77:0/1136566523 conn(0x7f7d900fd8f0 msgr2=0x7f7d900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.564+0000 7f7d97645640 1 -- 172.21.15.77:0/1136566523 shutdown_connections 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.564+0000 7f7d97645640 1 -- 172.21.15.77:0/1136566523 wait complete. 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.564+0000 7f7d97645640 1 Processor -- start 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.564+0000 7f7d97645640 1 -- start start 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 0x7f7d90198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d90106f20 0x7f7d9019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90111f80 con 0x7f7d90101df0 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90111e00 con 0x7f7d901027a0 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112100 con 0x7f7d90106f20 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60314/0 (socket says 172.21.15.77:60314) 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 -- 172.21.15.77:0/2783768022 learned_addr learned my addr 172.21.15.77:0/2783768022 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d94be0640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 0x7f7d90198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d95be2640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d90106f20 0x7f7d9019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 -- 172.21.15.77:0/2783768022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d90106f20 msgr2=0x7f7d9019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d90106f20 0x7f7d9019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 -- 172.21.15.77:0/2783768022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 msgr2=0x7f7d90198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 0x7f7d90198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 -- 172.21.15.77:0/2783768022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9019d390 con 0x7f7d90101df0 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d94be0640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 0x7f7d90198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d950 tx=0x7f7d8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d867fc640 1 -- 172.21.15.77:0/2783768022 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c017030 con 0x7f7d90101df0 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d867fc640 1 -- 172.21.15.77:0/2783768022 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c004020 con 0x7f7d90101df0 2024-09-15T22:14:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9019d680 con 0x7f7d90101df0 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.565+0000 7f7d867fc640 1 -- 172.21.15.77:0/2783768022 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0047a0 con 0x7f7d90101df0 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.566+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a4f60 con 0x7f7d90101df0 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.568+0000 7f7d867fc640 1 -- 172.21.15.77:0/2783768022 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7d8c0041d0 con 0x7f7d90101df0 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 Processor -- start 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 -- start start 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb50a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb517760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560efb4702f0 con 0x560efb50a5d0 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560efb417cf0 con 0x560efb50bb40 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560efb46c2b0 con 0x560efb515370 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d867fc640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 0x7f7d7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d95be2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb50a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60320/0 (socket says 172.21.15.77:60320) 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d953e1640 1 -- 172.21.15.77:0/981516937 learned_addr learned my addr 172.21.15.77:0/981516937 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d94be0640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb517760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.569+0000 7f7d95be2640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 0x7f7d7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d867fc640 1 -- 172.21.15.77:0/2783768022 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7d8c0cfa00 con 0x7f7d90101df0 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 -- 172.21.15.77:0/981516937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 msgr2=0x560efb517760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb517760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 -- 172.21.15.77:0/981516937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 msgr2=0x560efb50aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 -- 172.21.15.77:0/981516937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efb3fc3a0 con 0x560efb50bb40 2024-09-15T22:14:37.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d94be0640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb517760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d953e1640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 0x7f7d7c0a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009bf0 tx=0x7f7d88036000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d95be2640 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb50a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006010 tx=0x7f7d88031650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1599d40ef5fb5125 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d7b7fe640 1 -- 172.21.15.77:0/981516937 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88055070 con 0x560efb50bb40 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d7b7fe640 1 -- 172.21.15.77:0/981516937 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d88036070 con 0x560efb50bb40 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.570+0000 7f7d7b7fe640 1 -- 172.21.15.77:0/981516937 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88049400 con 0x560efb50bb40 2024-09-15T22:14:37.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- 172.21.15.77:0/981516937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 msgr2=0x560efb50a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb50a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006010 tx=0x7f7d88031650 comp rx=0 tx=0).stop 2024-09-15T22:14:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- 172.21.15.77:0/981516937 shutdown_connections 2024-09-15T22:14:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb517760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb50aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/981516937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb50a090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- 172.21.15.77:0/981516937 >> 172.21.15.77:0/981516937 conn(0x560efb4d3310 msgr2=0x560efb506c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- 172.21.15.77:0/981516937 shutdown_connections 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- 172.21.15.77:0/981516937 wait complete. 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.571+0000 7f7d99a1e740 1 -- start start 2024-09-15T22:14:37.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 Processor -- start 2024-09-15T22:14:37.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 0x560efb514d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb512600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560efb517e90 con 0x560efb50bb40 2024-09-15T22:14:37.821 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:14:37.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560efb42b060 con 0x560efb50a5d0 2024-09-15T22:14:37.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d94be0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560efb518010 con 0x560efb515370 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 0x560efb514d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d94be0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60324/0 (socket says 172.21.15.77:60324) 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d94be0640 1 -- 172.21.15.77:0/284605748 learned_addr learned my addr 172.21.15.77:0/284605748 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d95be2640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb512600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.572+0000 7f7d99a1e740 10 client.0 _ll_register_callbacks cb 0x560efb17e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:37.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 -- 172.21.15.77:0/284605748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 msgr2=0x560efb514d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 0x560efb514d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 -- 172.21.15.77:0/284605748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 msgr2=0x560efb512010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 -- 172.21.15.77:0/284605748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efb1e13c0 con 0x560efb515370 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d94be0640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d953e1640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 0x560efb514d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d95be2640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb512600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006010 tx=0x7f7d880095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddaad9a680fa1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88055030 con 0x560efb515370 2024-09-15T22:14:37.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d88036070 con 0x560efb515370 2024-09-15T22:14:37.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7d8802fe70 con 0x560efb515370 2024-09-15T22:14:37.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d88066050 con 0x560efb515370 2024-09-15T22:14:37.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d99a1e740 10 client.25507 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:37.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560efb517e90 con 0x560efb515370 2024-09-15T22:14:37.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.573+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d88036070 con 0x560efb515370 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7d8805b950 con 0x560efb515370 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 10 client.25507 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 10 client.25507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 20 client.25507 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb526f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 4 client.25507 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:37.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560efb47d860 con 0x560efb524b10 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.574+0000 7f7d953e1640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb526f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d78ff9640 10 client.25507 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d953e1640 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb526f70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79301196a0ea74b server_cookie=9021343827541f92 in_seq=0 out_seq=0 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d78ff9640 1 -- 172.21.15.77:0/284605748 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560efb47d860 con 0x560efb524b10 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d78ff9640 10 client.25507 handle_command_reply: tid=0 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 client.25507 shutdown 2024-09-15T22:14:37.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 msgr2=0x560efb512600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb512600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d88006010 tx=0x7f7d880095c0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 shutdown_connections 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb526f70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb512600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb512010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d78ff9640 0 client.25507 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/284605748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50a5d0 0x560efb514d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.575+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 >> 172.21.15.77:0/284605748 conn(0x560efb4d3310 msgr2=0x560efb50c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.576+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 shutdown_connections 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.576+0000 7f7d99a1e740 1 -- 172.21.15.77:0/284605748 wait complete. 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.576+0000 7f7d99a1e740 20 client.25507 trim_cache size 0 max 16384 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.588+0000 7f7d99a1e740 1 Processor -- start 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.588+0000 7f7d99a1e740 1 -- start start 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb52c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb52cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901aee70 con 0x560efb50a5d0 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901b0a70 con 0x560efb50bb40 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901af540 con 0x560efb515370 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60326/0 (socket says 172.21.15.77:60326) 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 -- 172.21.15.77:0/2920674854 learned_addr learned my addr 172.21.15.77:0/2920674854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d94be0640 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb52c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 -- 172.21.15.77:0/2920674854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 msgr2=0x560efb52cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb52cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 -- 172.21.15.77:0/2920674854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 msgr2=0x560efb52c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb52c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 -- 172.21.15.77:0/2920674854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efb50dae0 con 0x560efb50a5d0 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.589+0000 7f7d953e1640 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005fd0 tx=0x7f7d8c0d1e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4b541d59d3205b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.590+0000 7f7d7affd640 1 -- 172.21.15.77:0/2920674854 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c079040 con 0x560efb50a5d0 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.590+0000 7f7d7affd640 1 -- 172.21.15.77:0/2920674854 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c09fcf0 con 0x560efb50a5d0 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.590+0000 7f7d7affd640 1 -- 172.21.15.77:0/2920674854 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0cd030 con 0x560efb50a5d0 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 -- 172.21.15.77:0/2920674854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 msgr2=0x560efb521130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005fd0 tx=0x7f7d8c0d1e50 comp rx=0 tx=0).stop 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 -- 172.21.15.77:0/2920674854 shutdown_connections 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb515370 0x560efb52cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb50bb40 0x560efb52c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/2920674854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50a5d0 0x560efb521130 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 -- 172.21.15.77:0/2920674854 >> 172.21.15.77:0/2920674854 conn(0x560efb4d3310 msgr2=0x560efb511fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 -- 172.21.15.77:0/2920674854 shutdown_connections 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.591+0000 7f7d99a1e740 1 -- 172.21.15.77:0/2920674854 wait complete. 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 -- start start 2024-09-15T22:14:37.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 Processor -- start 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb50a5d0 0x560efb518a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 0x560efb51d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901b02e0 con 0x560efb50bb40 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901afc10 con 0x560efb515370 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901ace50 con 0x560efb50a5d0 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d99a1e740 10 client.0 _ll_register_callbacks cb 0x7f7d88053070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:37.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d94be0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d94be0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60328/0 (socket says 172.21.15.77:60328) 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d94be0640 1 -- 172.21.15.77:0/1754033867 learned_addr learned my addr 172.21.15.77:0/1754033867 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d95be2640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 0x560efb51d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.592+0000 7f7d953e1640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb50a5d0 0x560efb518a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 -- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb50a5d0 msgr2=0x560efb518a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb50a5d0 0x560efb518a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 -- 172.21.15.77:0/1754033867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 msgr2=0x560efb51d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 0x560efb51d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 -- 172.21.15.77:0/1754033867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efb51d6c0 con 0x560efb50bb40 2024-09-15T22:14:37.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d95be2640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 0x560efb51d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d94be0640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x560efb5118a0 tx=0x7f7d80012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c9e61d65205861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004050 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7d800041f0 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c09fcf0 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d8002b050 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d99a1e740 10 client.16176 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560efb58f350 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c09fcf0 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7d8001d530 con 0x560efb50bb40 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.593+0000 7f7d99a1e740 10 client.16176 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:37.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 20 client.16176 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb591bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 4 client.16176 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560efb477d70 con 0x560efb524b10 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d953e1640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb591bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d953e1640 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb591bc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0f34001e014ed66 server_cookie=cead79b0036da15d in_seq=0 out_seq=0 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d78ff9640 10 client.16176 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d78ff9640 1 -- 172.21.15.77:0/1754033867 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560efb477d70 con 0x560efb524b10 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d78ff9640 10 client.16176 handle_command_reply: tid=0 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.594+0000 7f7d99a1e740 1 client.16176 shutdown 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 msgr2=0x560efb518f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x560efb5118a0 tx=0x7f7d80012e70 comp rx=0 tx=0).stop 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 shutdown_connections 2024-09-15T22:14:37.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560efb524b10 0x560efb591bc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560efb515370 0x560efb51d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560efb50bb40 0x560efb518f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 --2- 172.21.15.77:0/1754033867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560efb50a5d0 0x560efb518a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 >> 172.21.15.77:0/1754033867 conn(0x560efb4d3310 msgr2=0x560efb5108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d78ff9640 0 client.16176 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 shutdown_connections 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 1 -- 172.21.15.77:0/1754033867 wait complete. 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d99a1e740 20 client.16176 trim_cache size 0 max 16384 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 msgr2=0x7f7d7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 0x7f7d7c0a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009bf0 tx=0x7f7d88036000 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 msgr2=0x7f7d901983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.595+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d950 tx=0x7f7d8c00de10 comp rx=0 tx=0).stop 2024-09-15T22:14:37.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 shutdown_connections 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7d7c0a33b0 0x7f7d7c0a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d90106f20 0x7f7d9019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d901027a0 0x7f7d90198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 --2- 172.21.15.77:0/2783768022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d90101df0 0x7f7d901983e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 >> 172.21.15.77:0/2783768022 conn(0x7f7d900fd8f0 msgr2=0x7f7d900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 shutdown_connections 2024-09-15T22:14:37.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:37.596+0000 7f7d97645640 1 -- 172.21.15.77:0/2783768022 wait complete. 2024-09-15T22:14:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:38 smithi077 ceph-mon[30980]: pgmap v69: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2024-09-15T22:14:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:38 smithi196 ceph-mon[35109]: pgmap v69: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2024-09-15T22:14:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:38 smithi117 ceph-mon[35021]: pgmap v69: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 294 op/s 2024-09-15T22:14:38.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 126564 14.84 MB/sec warmup 47 sec latency 20.859 ms 2024-09-15T22:14:38.640 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:14:38.907 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:39.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 -- 172.21.15.77:0/3217225237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178100a70 msgr2=0x7f9178100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 --2- 172.21.15.77:0/3217225237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178100a70 0x7f9178100ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9160009930 tx=0x7f916002a420 comp rx=0 tx=0).stop 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 -- 172.21.15.77:0/3217225237 shutdown_connections 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 --2- 172.21.15.77:0/3217225237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178101410 0x7f917810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 --2- 172.21.15.77:0/3217225237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178100a70 0x7f9178100ed0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 --2- 172.21.15.77:0/3217225237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178106a70 0x7f9178106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 -- 172.21.15.77:0/3217225237 >> 172.21.15.77:0/3217225237 conn(0x7f91780fc790 msgr2=0x7f91780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 -- 172.21.15.77:0/3217225237 shutdown_connections 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.394+0000 7f917ca58640 1 -- 172.21.15.77:0/3217225237 wait complete. 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.395+0000 7f917ca58640 1 Processor -- start 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.395+0000 7f917ca58640 1 -- start start 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178101410 0x7f91781997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178106a70 0x7f91781a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9178110ec0 con 0x7f9178106a70 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9178110d40 con 0x7f9178100a70 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917ca58640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9178111040 con 0x7f9178101410 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35858/0 (socket says 172.21.15.77:35858) 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 -- 172.21.15.77:0/3099457369 learned_addr learned my addr 172.21.15.77:0/3099457369 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178106a70 0x7f91781a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f916ffff640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178101410 0x7f91781997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 -- 172.21.15.77:0/3099457369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178101410 msgr2=0x7f91781997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178101410 0x7f91781997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 -- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178106a70 msgr2=0x7f91781a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178106a70 0x7f91781a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 -- 172.21.15.77:0/3099457369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9178197080 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.396+0000 7f917659c640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f916800e9f0 tx=0x7f916800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.397+0000 7f916f7fe640 1 -- 172.21.15.77:0/3099457369 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916800cd80 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.397+0000 7f916f7fe640 1 -- 172.21.15.77:0/3099457369 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9168010910 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.397+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9178197370 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.397+0000 7f916f7fe640 1 -- 172.21.15.77:0/3099457369 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168018990 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.397+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9178197880 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.399+0000 7f916f7fe640 1 -- 172.21.15.77:0/3099457369 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9168018b30 con 0x7f9178100a70 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 Processor -- start 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 -- start start 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f546a8b2f0 con 0x55f546b26b40 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f546a32cf0 con 0x55f546b255d0 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917ee31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f546a872b0 con 0x55f546b30370 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f916f7fe640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 0x7f91640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f917659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38196/0 (socket says 172.21.15.77:38196) 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f9176d9d640 1 -- 172.21.15.77:0/2018436666 learned_addr learned my addr 172.21.15.77:0/2018436666 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.400+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 0x7f91640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f916f7fe640 1 -- 172.21.15.77:0/3099457369 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f91680d0410 con 0x7f9178100a70 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 -- 172.21.15.77:0/2018436666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 msgr2=0x55f546b32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 -- 172.21.15.77:0/2018436666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 msgr2=0x55f546b25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 -- 172.21.15.77:0/2018436666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f546a173a0 con 0x55f546b26b40 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 0x7f91640a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91700099e0 tx=0x7f9170009c80 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f9176d9d640 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f917000fc20 tx=0x7f917001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6e250fc8cfa025 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f916cff9640 1 -- 172.21.15.77:0/2018436666 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170022070 con 0x55f546b26b40 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f916cff9640 1 -- 172.21.15.77:0/2018436666 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917000ee50 con 0x55f546b26b40 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.401+0000 7f916cff9640 1 -- 172.21.15.77:0/2018436666 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170004410 con 0x55f546b26b40 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 -- 172.21.15.77:0/2018436666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 msgr2=0x55f546b25090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f917000fc20 tx=0x7f917001d000 comp rx=0 tx=0).stop 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 -- 172.21.15.77:0/2018436666 shutdown_connections 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 --2- 172.21.15.77:0/2018436666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b25090 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 -- 172.21.15.77:0/2018436666 >> 172.21.15.77:0/2018436666 conn(0x55f546aee310 msgr2=0x55f546b21c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 -- 172.21.15.77:0/2018436666 shutdown_connections 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.402+0000 7f917ee31740 1 -- 172.21.15.77:0/2018436666 wait complete. 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 -- start start 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 Processor -- start 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b2d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f546b33010 con 0x55f546b255d0 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f546a46060 con 0x55f546b26b40 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f546b33190 con 0x55f546b30370 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917ee31740 10 client.0 _ll_register_callbacks cb 0x55f5467998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38212/0 (socket says 172.21.15.77:38212) 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 -- 172.21.15.77:0/3886014985 learned_addr learned my addr 172.21.15.77:0/3886014985 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f916ffff640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b2d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 -- 172.21.15.77:0/3886014985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 msgr2=0x55f546b2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 -- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 msgr2=0x55f546b2d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b2d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f917659c640 1 -- 172.21.15.77:0/3886014985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5467fc3c0 con 0x55f546b255d0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.403+0000 7f916ffff640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b2d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917659c640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91680987b0 tx=0x7f916809c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64c3555f155d92a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168026040 con 0x55f546b255d0 2024-09-15T22:14:39.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f91680989c0 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917000ee50 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91680d0160 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 10 client.16188 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f546b3eb50 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917000ee50 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f916809b930 con 0x55f546b255d0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 10 client.16188 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 10 client.16188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 20 client.16188 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546b42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 4 client.16188 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.404+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f546a98860 con 0x55f546b3ff60 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.405+0000 7f916ffff640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546b42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.405+0000 7f916ffff640 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546b42350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1f736f19296cdc2 server_cookie=209cc7e0a79a4d25 in_seq=0 out_seq=0 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.405+0000 7f915dffb640 10 client.16188 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f915dffb640 1 -- 172.21.15.77:0/3886014985 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f546a98860 con 0x55f546b3ff60 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f915dffb640 10 client.16188 handle_command_reply: tid=0 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 client.16188 shutdown 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 msgr2=0x55f546b2fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91680987b0 tx=0x7f916809c350 comp rx=0 tx=0).stop 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 shutdown_connections 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546b42350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b2d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f915dffb640 0 client.16188 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 --2- 172.21.15.77:0/3886014985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b255d0 0x55f546b2fd70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 >> 172.21.15.77:0/3886014985 conn(0x55f546aee310 msgr2=0x55f546b27960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 shutdown_connections 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 1 -- 172.21.15.77:0/3886014985 wait complete. 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.406+0000 7f917ee31740 20 client.16188 trim_cache size 0 max 16384 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 Processor -- start 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 -- start start 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b255d0 0x55f546b28120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b28660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b10d0 con 0x55f546b30370 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b2cd0 con 0x55f546b26b40 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917ee31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b17a0 con 0x55f546b255d0 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f917659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b255d0 0x55f546b28120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38228/0 (socket says 172.21.15.77:38228) 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 -- 172.21.15.77:0/3711685727 learned_addr learned my addr 172.21.15.77:0/3711685727 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f916ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b28660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 -- 172.21.15.77:0/3711685727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b255d0 msgr2=0x55f546b28120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b255d0 0x55f546b28120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 -- 172.21.15.77:0/3711685727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 msgr2=0x55f546b28660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b28660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.420+0000 7f9176d9d640 1 -- 172.21.15.77:0/3711685727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f546b300e0 con 0x55f546b30370 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.421+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9170003f50 tx=0x7f9170005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9235681c97b90f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.421+0000 7f915ffff640 1 -- 172.21.15.77:0/3711685727 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170022050 con 0x55f546b30370 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.421+0000 7f915ffff640 1 -- 172.21.15.77:0/3711685727 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917000ee20 con 0x55f546b30370 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.421+0000 7f915ffff640 1 -- 172.21.15.77:0/3711685727 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170005570 con 0x55f546b30370 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- 172.21.15.77:0/3711685727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 msgr2=0x55f546b47ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9170003f50 tx=0x7f9170005f70 comp rx=0 tx=0).stop 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- 172.21.15.77:0/3711685727 shutdown_connections 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b30370 0x55f546b47ba0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b26b40 0x55f546b28660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- 172.21.15.77:0/3711685727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b255d0 0x55f546b28120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- 172.21.15.77:0/3711685727 >> 172.21.15.77:0/3711685727 conn(0x55f546aee310 msgr2=0x55f546b20d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- 172.21.15.77:0/3711685727 shutdown_connections 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- 172.21.15.77:0/3711685727 wait complete. 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- start start 2024-09-15T22:14:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 Processor -- start 2024-09-15T22:14:39.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b2540 con 0x55f546b26b40 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.422+0000 7f917ee31740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91781b1e70 con 0x55f546b255d0 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917ee31740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91781af0b0 con 0x55f546b30370 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917ee31740 10 client.0 _ll_register_callbacks cb 0x7f916800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f916ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35882/0 (socket says 172.21.15.77:35882) 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 -- 172.21.15.77:0/3321847943 learned_addr learned my addr 172.21.15.77:0/3321847943 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f9176d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56684/0 (socket says 172.21.15.77:56684) 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f916ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38240/0 (socket says 172.21.15.77:38240) 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 -- 172.21.15.77:0/3321847943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 msgr2=0x55f546b2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 -- 172.21.15.77:0/3321847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 msgr2=0x55f546b29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 -- 172.21.15.77:0/3321847943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f546b2abb0 con 0x55f546b255d0 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f9176d9d640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f916ffff640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917659c640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91680a08c0 tx=0x7f916800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb10371dff1c7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f915dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916809bcb0 con 0x55f546b255d0 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f916809b9a0 con 0x55f546b255d0 2024-09-15T22:14:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917000ee20 con 0x55f546b255d0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91680cdc50 con 0x55f546b255d0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917ee31740 10 client.25583 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.423+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f546baac00 con 0x55f546b255d0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917000ee20 con 0x55f546b255d0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f916809b3a0 con 0x55f546b255d0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 10 client.25583 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 20 client.25583 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546bad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 4 client.25583 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f546a92d70 con 0x55f546b3ff60 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f916ffff640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546bad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f916ffff640 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546bad140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a588bfddd3b88583 server_cookie=c607d2383ed8c7ce in_seq=0 out_seq=0 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.424+0000 7f915dffb640 10 client.25583 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f915dffb640 1 -- 172.21.15.77:0/3321847943 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f546a92d70 con 0x55f546b3ff60 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f915dffb640 10 client.25583 handle_command_reply: tid=0 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 client.25583 shutdown 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 msgr2=0x55f546b38c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91680a08c0 tx=0x7f916800c840 comp rx=0 tx=0).stop 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 shutdown_connections 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f546b3ff60 0x55f546bad140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f546b30370 0x55f546b2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f915dffb640 0 client.25583 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f546b26b40 0x55f546b29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 --2- 172.21.15.77:0/3321847943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f546b255d0 0x55f546b38c50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 >> 172.21.15.77:0/3321847943 conn(0x55f546aee310 msgr2=0x55f546b2e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 shutdown_connections 2024-09-15T22:14:39.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 1 -- 172.21.15.77:0/3321847943 wait complete. 2024-09-15T22:14:39.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ee31740 20 client.25583 trim_cache size 0 max 16384 2024-09-15T22:14:39.443 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 msgr2=0x7f91640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 0x7f91640a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91700099e0 tx=0x7f9170009c80 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 msgr2=0x7f9178199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.425+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f916800e9f0 tx=0x7f916800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 shutdown_connections 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91640a33b0 0x7f91640a5870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9178106a70 0x7f91781a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9178101410 0x7f91781997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 --2- 172.21.15.77:0/3099457369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9178100a70 0x7f9178199290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 >> 172.21.15.77:0/3099457369 conn(0x7f91780fc790 msgr2=0x7f91780fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 shutdown_connections 2024-09-15T22:14:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:39.426+0000 7f917ca58640 1 -- 172.21.15.77:0/3099457369 wait complete. 2024-09-15T22:14:39.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 128554 14.79 MB/sec warmup 48 sec latency 18.678 ms 2024-09-15T22:14:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:40 smithi077 ceph-mon[30980]: pgmap v70: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-15T22:14:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:40 smithi196 ceph-mon[35109]: pgmap v70: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-15T22:14:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:40 smithi117 ceph-mon[35021]: pgmap v70: 129 pgs: 129 active+clean; 78 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-15T22:14:40.444 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:14:40.459 INFO:tasks.workunit.client.0.smithi077.stdout: 1 131645 14.87 MB/sec warmup 49 sec latency 15.010 ms 2024-09-15T22:14:40.716 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:41.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- 172.21.15.77:0/817365615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8102ef0 msgr2=0x7f69c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/817365615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8102ef0 0x7f69c810da40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009ca0 tx=0x7f69c002a950 comp rx=0 tx=0).stop 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- 172.21.15.77:0/817365615 shutdown_connections 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/817365615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8102ef0 0x7f69c810da40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/817365615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102570 0x7f69c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/817365615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8100e20 0x7f69c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- 172.21.15.77:0/817365615 >> 172.21.15.77:0/817365615 conn(0x7f69c80fc7d0 msgr2=0x7f69c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- 172.21.15.77:0/817365615 shutdown_connections 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- 172.21.15.77:0/817365615 wait complete. 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 Processor -- start 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.382+0000 7f69ce9ec640 1 -- start start 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8102570 0x7f69c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102ef0 0x7f69c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8110f80 con 0x7f69c8100e20 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8110e00 con 0x7f69c8102ef0 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ce9ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69c8111100 con 0x7f69c8102570 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38248/0 (socket says 172.21.15.77:38248) 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 -- 172.21.15.77:0/204418697 learned_addr learned my addr 172.21.15.77:0/204418697 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c77fe640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8102570 0x7f69c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69ccf89640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102ef0 0x7f69c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 -- 172.21.15.77:0/204418697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8102570 msgr2=0x7f69c81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8102570 0x7f69c81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 -- 172.21.15.77:0/204418697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102ef0 msgr2=0x7f69c819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102ef0 0x7f69c819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 -- 172.21.15.77:0/204418697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c819d440 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.383+0000 7f69c7fff640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f69b800d770 tx=0x7f69b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.384+0000 7f69c57fa640 1 -- 172.21.15.77:0/204418697 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8017070 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.384+0000 7f69c57fa640 1 -- 172.21.15.77:0/204418697 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b8005500 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.384+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c819d730 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.384+0000 7f69c57fa640 1 -- 172.21.15.77:0/204418697 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8004e00 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.384+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c81a5010 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69c57fa640 1 -- 172.21.15.77:0/204418697 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f69b8004fa0 con 0x7f69c8100e20 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 Processor -- start 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 -- start start 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f24448b40 0x559f24447090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 0x559f24447a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f24454760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f243ad2f0 con 0x559f244475d0 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f24354cf0 con 0x559f24448b40 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f243a92b0 con 0x559f24452370 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 0x559f24447a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 0x559f24447a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38262/0 (socket says 172.21.15.77:38262) 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69c7fff640 1 -- 172.21.15.77:0/557816574 learned_addr learned my addr 172.21.15.77:0/557816574 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69ccf89640 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f24448b40 0x559f24447090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.387+0000 7f69c77fe640 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f24454760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c57fa640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 0x7f69b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69ccf89640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 0x7f69b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 -- 172.21.15.77:0/557816574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f24448b40 msgr2=0x559f24447090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f24448b40 0x559f24447090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 -- 172.21.15.77:0/557816574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 msgr2=0x559f24447a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 0x559f24447a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 -- 172.21.15.77:0/557816574 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f243393a0 con 0x559f24452370 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c57fa640 1 -- 172.21.15.77:0/204418697 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f69b80cfa40 con 0x7f69c8100e20 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69c77fe640 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f24454760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009bf0 tx=0x7f69bc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc0655ccd7dad1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69ccf89640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 0x7f69b00a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69c0004ad0 tx=0x7f69c002a950 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69ae7fc640 1 -- 172.21.15.77:0/557816574 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004050 con 0x559f24452370 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69ae7fc640 1 -- 172.21.15.77:0/557816574 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc005500 con 0x559f24452370 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.388+0000 7f69ae7fc640 1 -- 172.21.15.77:0/557816574 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004e00 con 0x559f24452370 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 -- 172.21.15.77:0/557816574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 msgr2=0x559f24454760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f24454760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009bf0 tx=0x7f69bc011810 comp rx=0 tx=0).stop 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 -- 172.21.15.77:0/557816574 shutdown_connections 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f24454760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f244475d0 0x559f24447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/557816574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f24448b40 0x559f24447090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 -- 172.21.15.77:0/557816574 >> 172.21.15.77:0/557816574 conn(0x559f24410310 msgr2=0x559f24443c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 -- 172.21.15.77:0/557816574 shutdown_connections 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.389+0000 7f69d0dc5740 1 -- 172.21.15.77:0/557816574 wait complete. 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 -- start start 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 Processor -- start 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f2444f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f24455070 con 0x559f24448b40 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f24368060 con 0x559f244475d0 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f244551f0 con 0x559f24452370 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69d0dc5740 10 client.0 _ll_register_callbacks cb 0x559f240bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:41.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35904/0 (socket says 172.21.15.77:35904) 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38276/0 (socket says 172.21.15.77:38276) 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c7fff640 1 -- 172.21.15.77:0/2888307502 learned_addr learned my addr 172.21.15.77:0/2888307502 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69ccf89640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f2444f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c7fff640 1 -- 172.21.15.77:0/2888307502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 msgr2=0x559f2444f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.390+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f2444f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69c7fff640 1 -- 172.21.15.77:0/2888307502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 msgr2=0x559f2444ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69c7fff640 1 -- 172.21.15.77:0/2888307502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2411e3c0 con 0x559f244475d0 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69b800d740 tx=0x7f69b80057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b9c5b379e0626f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8020040 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f69b8009cc0 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc005500 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69b809b3f0 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69d0dc5740 10 client.25595 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f24460a90 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc005500 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.391+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f69b80cd0d0 con 0x559f244475d0 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 10 client.25595 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 10 client.25595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 20 client.25595 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f24461f50 0x559f244643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 4 client.25595 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f243ba860 con 0x559f24461f50 2024-09-15T22:14:41.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f24461f50 0x559f244643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f24461f50 0x559f244643f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6140b8613b109b0a server_cookie=fd3f3eab915df7a0 in_seq=0 out_seq=0 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.392+0000 7f69a77fe640 10 client.25595 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69a77fe640 1 -- 172.21.15.77:0/2888307502 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f243ba860 con 0x559f24461f50 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69a77fe640 10 client.25595 handle_command_reply: tid=0 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 client.25595 shutdown 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 msgr2=0x559f24451c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69b800d740 tx=0x7f69b80057b0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 shutdown_connections 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f24461f50 0x559f244643f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24452370 0x559f2444f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24448b40 0x559f2444ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2888307502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24451c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69a77fe640 0 client.25595 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:41.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 >> 172.21.15.77:0/2888307502 conn(0x559f24410310 msgr2=0x559f24449960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 shutdown_connections 2024-09-15T22:14:41.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2888307502 wait complete. 2024-09-15T22:14:41.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.393+0000 7f69d0dc5740 20 client.25595 trim_cache size 0 max 16384 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.408+0000 7f69d0dc5740 1 Processor -- start 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.408+0000 7f69d0dc5740 1 -- start start 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f2444a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24453760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81aedd0 con 0x559f24452370 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81b09d0 con 0x559f244475d0 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81af4a0 con 0x559f24448b40 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35914/0 (socket says 172.21.15.77:35914) 2024-09-15T22:14:41.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 -- 172.21.15.77:0/2294730316 learned_addr learned my addr 172.21.15.77:0/2294730316 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f2444a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69ccf89640 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24453760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 -- 172.21.15.77:0/2294730316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 msgr2=0x559f2444a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f2444a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 -- 172.21.15.77:0/2294730316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 msgr2=0x559f24453760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24453760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.409+0000 7f69c7fff640 1 -- 172.21.15.77:0/2294730316 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f24464a70 con 0x559f244475d0 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.410+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69b80cd450 tx=0x7f69b8099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59eacbb4241da282 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.410+0000 7f69adffb640 1 -- 172.21.15.77:0/2294730316 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8014030 con 0x559f244475d0 2024-09-15T22:14:41.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.410+0000 7f69adffb640 1 -- 172.21.15.77:0/2294730316 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b80a4650 con 0x559f244475d0 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.410+0000 7f69adffb640 1 -- 172.21.15.77:0/2294730316 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8020040 con 0x559f244475d0 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2294730316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 msgr2=0x559f2445e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f69b80cd450 tx=0x7f69b8099fa0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2294730316 shutdown_connections 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24453760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f2444a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2294730316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f2445e310 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2294730316 >> 172.21.15.77:0/2294730316 conn(0x559f24410310 msgr2=0x559f2444ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2294730316 shutdown_connections 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2294730316 wait complete. 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 -- start start 2024-09-15T22:14:41.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.411+0000 7f69d0dc5740 1 Processor -- start 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f24469970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24459480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81b0240 con 0x559f24452370 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81afb70 con 0x559f244475d0 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69c81acdb0 con 0x559f24448b40 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69d0dc5740 10 client.0 _ll_register_callbacks cb 0x7f69b80d3280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69ccf89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24459480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f24469970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35928/0 (socket says 172.21.15.77:35928) 2024-09-15T22:14:41.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 -- 172.21.15.77:0/2297304002 learned_addr learned my addr 172.21.15.77:0/2297304002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 -- 172.21.15.77:0/2297304002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 msgr2=0x559f24469970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f24469970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 -- 172.21.15.77:0/2297304002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 msgr2=0x559f24459480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24459480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 -- 172.21.15.77:0/2297304002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f24459b20 con 0x559f244475d0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69ccf89640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24459480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69c7fff640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f69b8097ef0 tx=0x7f69b809fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0dfbccfbe4e18cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.412+0000 7f69a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8014030 con 0x559f244475d0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f69b8060e00 con 0x559f244475d0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b80a4650 con 0x559f244475d0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69b8020070 con 0x559f244475d0 2024-09-15T22:14:41.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 10 client.25601 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f24459b20 con 0x559f244475d0 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b8066020 con 0x559f244475d0 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f69b80b4be0 con 0x559f244475d0 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 10 client.25601 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 20 client.25601 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f244ccc90 0x559f244cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 4 client.25601 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f243b4d70 con 0x559f244ccc90 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.413+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f244ccc90 0x559f244cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69a77fe640 10 client.25601 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:41.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69c77fe640 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f244ccc90 0x559f244cf080 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4d47304fc67e4e server_cookie=88f8bbdd600afe92 in_seq=0 out_seq=0 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69a77fe640 1 -- 172.21.15.77:0/2297304002 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f243b4d70 con 0x559f244ccc90 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69a77fe640 10 client.25601 handle_command_reply: tid=0 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 client.25601 shutdown 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 msgr2=0x559f24469430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f69b8097ef0 tx=0x7f69b809fea0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 shutdown_connections 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559f244ccc90 0x559f244cf080 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f24452370 0x559f24459480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f24448b40 0x559f24469970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69a77fe640 0 client.25601 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 --2- 172.21.15.77:0/2297304002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f244475d0 0x559f24469430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 >> 172.21.15.77:0/2297304002 conn(0x559f24410310 msgr2=0x559f2444ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 shutdown_connections 2024-09-15T22:14:41.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 1 -- 172.21.15.77:0/2297304002 wait complete. 2024-09-15T22:14:41.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.414+0000 7f69d0dc5740 20 client.25601 trim_cache size 0 max 16384 2024-09-15T22:14:41.426 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 msgr2=0x7f69b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 0x7f69b00a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69c0004ad0 tx=0x7f69c002a950 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 msgr2=0x7f69c8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f69b800d770 tx=0x7f69b800dc30 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 shutdown_connections 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69b00a33b0 0x7f69b00a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69c8102ef0 0x7f69c819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69c8102570 0x7f69c81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 --2- 172.21.15.77:0/204418697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69c8100e20 0x7f69c8198490 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:41.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 >> 172.21.15.77:0/204418697 conn(0x7f69c80fc7d0 msgr2=0x7f69c806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 shutdown_connections 2024-09-15T22:14:41.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:41.415+0000 7f69ce9ec640 1 -- 172.21.15.77:0/204418697 wait complete. 2024-09-15T22:14:41.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 134287 14.82 MB/sec warmup 50 sec latency 32.849 ms 2024-09-15T22:14:42.110 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:42 smithi077 ceph-mon[30980]: pgmap v71: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:42 smithi196 ceph-mon[35109]: pgmap v71: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:42 smithi117 ceph-mon[35021]: pgmap v71: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-15T22:14:42.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 136512 14.76 MB/sec warmup 51 sec latency 10.776 ms 2024-09-15T22:14:43.140 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:14:43.141 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:14:43.406 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:43.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 139190 14.76 MB/sec warmup 52 sec latency 18.209 ms 2024-09-15T22:14:43.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 -- 172.21.15.77:0/404954647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 msgr2=0x7f3384103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:43.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 --2- 172.21.15.77:0/404954647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384103110 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3378009930 tx=0x7f337802a420 comp rx=0 tx=0).stop 2024-09-15T22:14:43.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 -- 172.21.15.77:0/404954647 shutdown_connections 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 --2- 172.21.15.77:0/404954647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841082a0 0x7f338410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 --2- 172.21.15.77:0/404954647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841036e0 0x7f3384107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 --2- 172.21.15.77:0/404954647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384103110 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 -- 172.21.15.77:0/404954647 >> 172.21.15.77:0/404954647 conn(0x7f33840fc7d0 msgr2=0x7f33840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 -- 172.21.15.77:0/404954647 shutdown_connections 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 -- 172.21.15.77:0/404954647 wait complete. 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.969+0000 7f338ad2b640 1 Processor -- start 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 -- start start 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841036e0 0x7f3384100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841082a0 0x7f3384101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33841120a0 con 0x7f3384102d30 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3384111f20 con 0x7f33841036e0 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f338ad2b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3384112220 con 0x7f33841082a0 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38314/0 (socket says 172.21.15.77:38314) 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 -- 172.21.15.77:0/2071801450 learned_addr learned my addr 172.21.15.77:0/2071801450 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f33892c8640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841082a0 0x7f3384101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3383fff640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841036e0 0x7f3384100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 -- 172.21.15.77:0/2071801450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841082a0 msgr2=0x7f3384101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841082a0 0x7f3384101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 -- 172.21.15.77:0/2071801450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841036e0 msgr2=0x7f3384100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841036e0 0x7f3384100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 -- 172.21.15.77:0/2071801450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33841a6a80 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3378009230 tx=0x7f3378004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.970+0000 7f3381ffb640 1 -- 172.21.15.77:0/2071801450 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337803f030 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.971+0000 7f3381ffb640 1 -- 172.21.15.77:0/2071801450 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378004440 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.971+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33841a6d10 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.971+0000 7f3381ffb640 1 -- 172.21.15.77:0/2071801450 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035070 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.971+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841a71a0 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.973+0000 7f3381ffb640 1 -- 172.21.15.77:0/2071801450 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3378042030 con 0x7f3384102d30 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 Processor -- start 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 -- start start 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908bef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c90824280 con 0x563c908c9d40 2024-09-15T22:14:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c907cbb10 con 0x563c908beb60 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f338d104740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c90820240 con 0x563c908bf480 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3381ffb640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 0x7f33740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3388ac7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38322/0 (socket says 172.21.15.77:38322) 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3383fff640 1 -- 172.21.15.77:0/3775606926 learned_addr learned my addr 172.21.15.77:0/3775606926 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f33892c8640 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f33892c8640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 0x7f33740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3383fff640 1 -- 172.21.15.77:0/3775606926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 msgr2=0x563c908c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.974+0000 7f3383fff640 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f3383fff640 1 -- 172.21.15.77:0/3775606926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 msgr2=0x563c908bef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f3383fff640 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908bef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f3383fff640 1 -- 172.21.15.77:0/3775606926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c907d81f0 con 0x563c908c9d40 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f3381ffb640 1 -- 172.21.15.77:0/2071801450 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f33780f0340 con 0x7f3384102d30 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f3383fff640 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3370009c20 tx=0x7f3370011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d06733910ffa26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f336effd640 1 -- 172.21.15.77:0/3775606926 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004230 con 0x563c908c9d40 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f336effd640 1 -- 172.21.15.77:0/3775606926 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33700043d0 con 0x563c908c9d40 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.975+0000 7f336effd640 1 -- 172.21.15.77:0/3775606926 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004d50 con 0x563c908c9d40 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- 172.21.15.77:0/3775606926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 msgr2=0x563c908cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3370009c20 tx=0x7f3370011810 comp rx=0 tx=0).stop 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f33892c8640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 0x7f33740a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f337c005fd0 tx=0x7f337c009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- 172.21.15.77:0/3775606926 shutdown_connections 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908cc130 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 --2- 172.21.15.77:0/3775606926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908bef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- 172.21.15.77:0/3775606926 >> 172.21.15.77:0/3775606926 conn(0x563c908bb0f0 msgr2=0x563c908bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- 172.21.15.77:0/3775606926 shutdown_connections 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- 172.21.15.77:0/3775606926 wait complete. 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 -- start start 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.976+0000 7f338d104740 1 Processor -- start 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908c9d40 0x563c908c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c908cc910 con 0x563c908bf480 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c907def10 con 0x563c908c9d40 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c908cca90 con 0x563c908beb60 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f338d104740 10 client.0 _ll_register_callbacks cb 0x563c907a7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3388ac7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38328/0 (socket says 172.21.15.77:38328) 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3388ac7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56770/0 (socket says 172.21.15.77:56770) 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 -- 172.21.15.77:0/1626045418 learned_addr learned my addr 172.21.15.77:0/1626045418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f33892c8640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908c9d40 0x563c908c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 -- 172.21.15.77:0/1626045418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 msgr2=0x563c908c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 -- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908c9d40 msgr2=0x563c908c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908c9d40 0x563c908c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 -- 172.21.15.77:0/1626045418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c905956a0 con 0x563c908bf480 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3388ac7640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3383fff640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f33700048f0 tx=0x7f3370019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a983952c55615dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3363fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.977+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370028020 con 0x563c908bf480 2024-09-15T22:14:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f3370024620 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33700043d0 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3370033050 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 10 client.16224 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563c908d8440 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33700043d0 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3370018070 con 0x563c908bf480 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 10 client.16224 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 20 client.16224 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c908d9850 0x563c908dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 4 client.16224 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c908317f0 con 0x563c908d9850 2024-09-15T22:14:43.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.978+0000 7f3388ac7640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c908d9850 0x563c908dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.979+0000 7f3388ac7640 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c908d9850 0x563c908dbc40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88753ac56ef8651b server_cookie=dfa796a1e889e6bb in_seq=0 out_seq=0 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.979+0000 7f3363fff640 10 client.16224 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f3363fff640 1 -- 172.21.15.77:0/1626045418 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c908317f0 con 0x563c908d9850 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f3363fff640 10 client.16224 handle_command_reply: tid=0 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 client.16224 shutdown 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 msgr2=0x563c908c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f33700048f0 tx=0x7f3370019b00 comp rx=0 tx=0).stop 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 shutdown_connections 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c908d9850 0x563c908dbc40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908c9d40 0x563c908c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908c6b40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f3363fff640 0 client.16224 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 --2- 172.21.15.77:0/1626045418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908beb60 0x563c908c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.980+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 >> 172.21.15.77:0/1626045418 conn(0x563c908bb0f0 msgr2=0x563c908c0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:43.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.981+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 shutdown_connections 2024-09-15T22:14:43.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.981+0000 7f338d104740 1 -- 172.21.15.77:0/1626045418 wait complete. 2024-09-15T22:14:43.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:43.981+0000 7f338d104740 20 client.16224 trim_cache size 0 max 16384 2024-09-15T22:14:44.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 Processor -- start 2024-09-15T22:14:44.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 -- start start 2024-09-15T22:14:44.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b10e0 con 0x563c908c9d40 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b2ce0 con 0x563c908beb60 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f338d104740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b17b0 con 0x563c908bf480 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f3383fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f3383fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56778/0 (socket says 172.21.15.77:56778) 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f3383fff640 1 -- 172.21.15.77:0/1417315460 learned_addr learned my addr 172.21.15.77:0/1417315460 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f33892c8640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.000+0000 7f3388ac7640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 -- 172.21.15.77:0/1417315460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 msgr2=0x563c908c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 -- 172.21.15.77:0/1417315460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 msgr2=0x563c908e1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908e1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 -- 172.21.15.77:0/1417315460 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c908c7640 con 0x563c908bf480 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f33892c8640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908e1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f3383fff640 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33700059f0 tx=0x7f33700197f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60154a07ab0ae778 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f336e7fc640 1 -- 172.21.15.77:0/1417315460 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370028070 con 0x563c908bf480 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f336e7fc640 1 -- 172.21.15.77:0/1417315460 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337001f040 con 0x563c908bf480 2024-09-15T22:14:44.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.001+0000 7f336e7fc640 1 -- 172.21.15.77:0/1417315460 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004710 con 0x563c908bf480 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- 172.21.15.77:0/1417315460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 msgr2=0x563c908c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33700059f0 tx=0x7f33700197f0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- 172.21.15.77:0/1417315460 shutdown_connections 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908c9d40 0x563c908e1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908bf480 0x563c908c5950 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 --2- 172.21.15.77:0/1417315460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- 172.21.15.77:0/1417315460 >> 172.21.15.77:0/1417315460 conn(0x563c908bb0f0 msgr2=0x563c908bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- 172.21.15.77:0/1417315460 shutdown_connections 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- 172.21.15.77:0/1417315460 wait complete. 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 -- start start 2024-09-15T22:14:44.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.002+0000 7f338d104740 1 Processor -- start 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908c9d40 0x563c908caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b2550 con 0x563c908bf480 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f33841b1e80 con 0x563c908beb60 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f33841af0c0 con 0x563c908c9d40 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f338d104740 10 client.0 _ll_register_callbacks cb 0x7f33780bc1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38342/0 (socket says 172.21.15.77:38342) 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 -- 172.21.15.77:0/2682740744 learned_addr learned my addr 172.21.15.77:0/2682740744 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f33892c8640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908c9d40 0x563c908caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 -- 172.21.15.77:0/2682740744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908c9d40 msgr2=0x563c908caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:44.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908c9d40 0x563c908caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 -- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 msgr2=0x563c908d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 -- 172.21.15.77:0/2682740744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c908dbfb0 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3383fff640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f33700198a0 tx=0x7f3370019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af4293b3badbb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.003+0000 7f3363fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370028070 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f33700048d0 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337001f040 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3370004dd0 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 10 client.16230 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563c908d0a30 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337001f040 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3370018040 con 0x563c908bf480 2024-09-15T22:14:44.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 10 client.16230 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 20 client.16230 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c90944900 0x563c90946d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 4 client.16230 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.004+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c9082bd00 con 0x563c90944900 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.005+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c90944900 0x563c90946d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.005+0000 7f3388ac7640 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c90944900 0x563c90946d20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3c0a9e6ecadbf0 server_cookie=28a68829f7d18723 in_seq=0 out_seq=0 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.005+0000 7f3363fff640 10 client.16230 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f3363fff640 1 -- 172.21.15.77:0/2682740744 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c9082bd00 con 0x563c90944900 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f3363fff640 10 client.16230 handle_command_reply: tid=0 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 client.16230 shutdown 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 msgr2=0x563c908caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f33700198a0 tx=0x7f3370019700 comp rx=0 tx=0).stop 2024-09-15T22:14:44.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 shutdown_connections 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c90944900 0x563c90946d20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c908c9d40 0x563c908caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c908bf480 0x563c908caa30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 --2- 172.21.15.77:0/2682740744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c908beb60 0x563c908d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 >> 172.21.15.77:0/2682740744 conn(0x563c908bb0f0 msgr2=0x563c908c5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f3363fff640 0 client.16230 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 shutdown_connections 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 1 -- 172.21.15.77:0/2682740744 wait complete. 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.006+0000 7f338d104740 20 client.16230 trim_cache size 0 max 16384 2024-09-15T22:14:44.012 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:14:44.013 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:14:44.013 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "5bbef4cc-0b86-4628-8b46-cb66886f17b0", 2024-09-15T22:14:44.013 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:14:44.013 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:14:44.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 msgr2=0x7f33740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 0x7f33740a5870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f337c005fd0 tx=0x7f337c009500 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 msgr2=0x7f3384100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3378009230 tx=0x7f3378004040 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 shutdown_connections 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f33740a33b0 0x7f33740a5870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33841082a0 0x7f3384101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33841036e0 0x7f3384100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 --2- 172.21.15.77:0/2071801450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3384102d30 0x7f3384100990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 >> 172.21.15.77:0/2071801450 conn(0x7f33840fc7d0 msgr2=0x7f3384105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 shutdown_connections 2024-09-15T22:14:44.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:44.007+0000 7f338ad2b640 1 -- 172.21.15.77:0/2071801450 wait complete. 2024-09-15T22:14:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:44 smithi077 ceph-mon[30980]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:44 smithi196 ceph-mon[35109]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:44 smithi117 ceph-mon[35021]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:44.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 142237 14.82 MB/sec warmup 53 sec latency 21.881 ms 2024-09-15T22:14:44.892 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:14:45.165 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:45.287 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:14:45.287 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.287 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:14:45.288 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:14:45.288 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.288 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:14:45.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:14:45.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:14:45.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:45.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 144931 14.79 MB/sec warmup 54 sec latency 25.444 ms 2024-09-15T22:14:45.479 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:14:45.480 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-15T22:14:45.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.666+0000 7f165c68a640 1 -- 172.21.15.77:0/3726116421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541082a0 msgr2=0x7f165410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.666+0000 7f165c68a640 1 --2- 172.21.15.77:0/3726116421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541082a0 0x7f165410ec40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f164c009cd0 tx=0x7f164c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.666+0000 7f165c68a640 1 -- 172.21.15.77:0/3726116421 shutdown_connections 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.666+0000 7f165c68a640 1 --2- 172.21.15.77:0/3726116421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541082a0 0x7f165410ec40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 --2- 172.21.15.77:0/3726116421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541036e0 0x7f1654107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 --2- 172.21.15.77:0/3726116421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1654102d30 0x7f1654103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 -- 172.21.15.77:0/3726116421 >> 172.21.15.77:0/3726116421 conn(0x7f16540fc7d0 msgr2=0x7f16540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 -- 172.21.15.77:0/3726116421 shutdown_connections 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 -- 172.21.15.77:0/3726116421 wait complete. 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 Processor -- start 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.667+0000 7f165c68a640 1 -- start start 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16541036e0 0x7f1654198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541082a0 0x7f165419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16541120a0 con 0x7f1654102d30 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1654111f20 con 0x7f16541082a0 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165c68a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1654112220 con 0x7f16541036e0 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38352/0 (socket says 172.21.15.77:38352) 2024-09-15T22:14:45.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 -- 172.21.15.77:0/3776661836 learned_addr learned my addr 172.21.15.77:0/3776661836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f1659c25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16541036e0 0x7f1654198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165ac27640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541082a0 0x7f165419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 -- 172.21.15.77:0/3776661836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16541036e0 msgr2=0x7f1654198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16541036e0 0x7f1654198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 -- 172.21.15.77:0/3776661836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541082a0 msgr2=0x7f165419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541082a0 0x7f165419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 -- 172.21.15.77:0/3776661836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165419d4e0 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.668+0000 7f165a426640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1650009b60 tx=0x7f1650007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.669+0000 7f164b7fe640 1 -- 172.21.15.77:0/3776661836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650009d70 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.669+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165419d7d0 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.669+0000 7f164b7fe640 1 -- 172.21.15.77:0/3776661836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1650007c40 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.669+0000 7f164b7fe640 1 -- 172.21.15.77:0/3776661836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650005280 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.669+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541008c0 con 0x7f1654102d30 2024-09-15T22:14:45.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.671+0000 7f164b7fe640 1 -- 172.21.15.77:0/3776661836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1650005420 con 0x7f1654102d30 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.671+0000 7f165ea63740 1 Processor -- start 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 -- start start 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d2a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f1659c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ac27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f1659c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38360/0 (socket says 172.21.15.77:38360) 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ac27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56820/0 (socket says 172.21.15.77:56820) 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f1659c25640 1 -- 172.21.15.77:0/463788757 learned_addr learned my addr 172.21.15.77:0/463788757 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:45.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592b3c852e0 con 0x5592b3d2a680 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592b3c2cce0 con 0x5592b3d1fdc0 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592b3c812a0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f165a426640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d2a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.672+0000 7f164b7fe640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 0x7f16400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 0x7f16400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 -- 172.21.15.77:0/463788757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 msgr2=0x5592b3d2a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f164b7fe640 1 -- 172.21.15.77:0/3776661836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f16500d3540 con 0x7f1654102d30 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d2a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 -- 172.21.15.77:0/463788757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 msgr2=0x5592b3d2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 -- 172.21.15.77:0/463788757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b3c11440 con 0x5592b3d1f4a0 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f1659c25640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:45.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165a426640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d2a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.673+0000 7f165ac27640 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f164c009ca0 tx=0x7f164c031a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce3c2557e4823779 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f1648ff9640 1 -- 172.21.15.77:0/463788757 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c003e00 con 0x5592b3d1f4a0 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ac27640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 0x7f16400a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5592b3d20460 tx=0x7f164c04c000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f1648ff9640 1 -- 172.21.15.77:0/463788757 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164c036070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f1648ff9640 1 -- 172.21.15.77:0/463788757 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c030c80 con 0x5592b3d1f4a0 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 msgr2=0x5592b3d1f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f164c009ca0 tx=0x7f164c031a70 comp rx=0 tx=0).stop 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 shutdown_connections 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d2a680 0x5592b3d2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d2a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 --2- 172.21.15.77:0/463788757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d1f880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 >> 172.21.15.77:0/463788757 conn(0x5592b3d1ba30 msgr2=0x5592b3d1be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 shutdown_connections 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.674+0000 7f165ea63740 1 -- 172.21.15.77:0/463788757 wait complete. 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 -- start start 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 Processor -- start 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d2a680 0x5592b3d278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592b3d2d250 con 0x5592b3d1f4a0 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592b3c40050 con 0x5592b3d1fdc0 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592b3d2d3d0 con 0x5592b3d2a680 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ea63740 10 client.0 _ll_register_callbacks cb 0x5592b3c80160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38364/0 (socket says 172.21.15.77:38364) 2024-09-15T22:14:45.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165a426640 1 -- 172.21.15.77:0/594738348 learned_addr learned my addr 172.21.15.77:0/594738348 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f165ac27640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d2a680 0x5592b3d278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.675+0000 7f1659c25640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 -- 172.21.15.77:0/594738348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d2a680 msgr2=0x5592b3d278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d2a680 0x5592b3d278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 -- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 msgr2=0x5592b3d27310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d27310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 -- 172.21.15.77:0/594738348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b39f63c0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165a426640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f165000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4492f0f283c70a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f163dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650020040 con 0x5592b3d1f4a0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f165000f030 con 0x5592b3d1f4a0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164c036070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1650007df0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165ea63740 10 client.16242 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5592b3d38cf0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.676+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164c036070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f16500d3180 con 0x5592b3d1f4a0 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 10 client.16242 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 20 client.16242 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3d3a270 0x5592b3d3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 4 client.16242 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b3c92850 con 0x5592b3d3a270 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.677+0000 7f1659c25640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3d3a270 0x5592b3d3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f163dffb640 10 client.16242 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f1659c25640 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3d3a270 0x5592b3d3c6d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a05c1083771e4c6 server_cookie=2e1a8b8d691671fe in_seq=0 out_seq=0 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f163dffb640 1 -- 172.21.15.77:0/594738348 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592b3c92850 con 0x5592b3d3a270 2024-09-15T22:14:45.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f163dffb640 10 client.16242 handle_command_reply: tid=0 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f165ea63740 1 client.16242 shutdown 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 msgr2=0x5592b3d2a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.678+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1650007fd0 tx=0x7f165000f3c0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 shutdown_connections 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3d3a270 0x5592b3d3c6d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d2a680 0x5592b3d278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d1fdc0 0x5592b3d27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 --2- 172.21.15.77:0/594738348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d2a060 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 >> 172.21.15.77:0/594738348 conn(0x5592b3d1ba30 msgr2=0x5592b3d1ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f163dffb640 0 client.16242 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 shutdown_connections 2024-09-15T22:14:45.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 1 -- 172.21.15.77:0/594738348 wait complete. 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.679+0000 7f165ea63740 20 client.16242 trim_cache size 0 max 16384 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 Processor -- start 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 -- start start 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d41f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16541acc00 con 0x5592b3d1fdc0 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ae800 con 0x5592b3d2a680 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ea63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ad2d0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f1659c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f1659c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38370/0 (socket says 172.21.15.77:38370) 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f1659c25640 1 -- 172.21.15.77:0/1135266943 learned_addr learned my addr 172.21.15.77:0/1135266943 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165a426640 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.707+0000 7f165ac27640 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d41f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 -- 172.21.15.77:0/1135266943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 msgr2=0x5592b3d364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 -- 172.21.15.77:0/1135266943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 msgr2=0x5592b3d41f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d41f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 -- 172.21.15.77:0/1135266943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b3d262c0 con 0x5592b3d1fdc0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f1659c25640 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5592b3d26ba0 tx=0x7f164400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ce4410ee77cdc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f163ffff640 1 -- 172.21.15.77:0/1135266943 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644003d80 con 0x5592b3d1fdc0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f163ffff640 1 -- 172.21.15.77:0/1135266943 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1644011920 con 0x5592b3d1fdc0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.708+0000 7f163ffff640 1 -- 172.21.15.77:0/1135266943 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164401d7f0 con 0x5592b3d1fdc0 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- 172.21.15.77:0/1135266943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 msgr2=0x5592b3d21600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5592b3d26ba0 tx=0x7f164400b390 comp rx=0 tx=0).stop 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- 172.21.15.77:0/1135266943 shutdown_connections 2024-09-15T22:14:45.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d41f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1fdc0 0x5592b3d21600 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 --2- 172.21.15.77:0/1135266943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1f4a0 0x5592b3d364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- 172.21.15.77:0/1135266943 >> 172.21.15.77:0/1135266943 conn(0x5592b3d1ba30 msgr2=0x5592b3d20ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- 172.21.15.77:0/1135266943 shutdown_connections 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- 172.21.15.77:0/1135266943 wait complete. 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.709+0000 7f165ea63740 1 -- start start 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 Processor -- start 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1fdc0 0x5592b3d23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d24230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ae070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16541ad9a0 con 0x5592b3d2a680 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16541aabe0 con 0x5592b3d1fdc0 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ea63740 10 client.0 _ll_register_callbacks cb 0x7f165009b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165a426640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38382/0 (socket says 172.21.15.77:38382) 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165a426640 1 -- 172.21.15.77:0/3097610993 learned_addr learned my addr 172.21.15.77:0/3097610993 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f1659c25640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1fdc0 0x5592b3d23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.710+0000 7f165ac27640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d24230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 -- 172.21.15.77:0/3097610993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1fdc0 msgr2=0x5592b3d23cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1fdc0 0x5592b3d23cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 -- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 msgr2=0x5592b3d24230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d24230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 -- 172.21.15.77:0/3097610993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b3d24770 con 0x5592b3d1f4a0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165a426640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f16500d0f50 tx=0x7f1650005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765c75a606efb5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f163dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165007f070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1644011920 con 0x5592b3d1f4a0 2024-09-15T22:14:45.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f165009e9e0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1650020070 con 0x5592b3d1f4a0 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165ea63740 10 client.16248 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5592b3da4bc0 con 0x5592b3d1f4a0 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.711+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1644011920 con 0x5592b3d1f4a0 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f165009e450 con 0x5592b3d1f4a0 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 10 client.16248 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 20 client.16248 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:45.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3da53d0 0x5592b3da77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 4 client.16248 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592b3c8cd60 con 0x5592b3da53d0 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "5bbef4cc-0b86-4628-8b46-cb66886f17b0": { 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "5bbef4cc-0b86-4628-8b46-cb66886f17b0", 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:45.758 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f1659c25640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3da53d0 0x5592b3da77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f1659c25640 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3da53d0 0x5592b3da77f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee49a978932f6e1 server_cookie=3ff79aa931b83669 in_seq=0 out_seq=0 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.712+0000 7f163dffb640 10 client.16248 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f163dffb640 1 -- 172.21.15.77:0/3097610993 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5592b3c8cd60 con 0x5592b3da53d0 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f163dffb640 10 client.16248 handle_command_reply: tid=0 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 client.16248 shutdown 2024-09-15T22:14:45.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 msgr2=0x5592b3d32a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f16500d0f50 tx=0x7f1650005bf0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 shutdown_connections 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b3da53d0 0x5592b3da77f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b3d2a680 0x5592b3d24230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f163dffb640 0 client.16248 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b3d1fdc0 0x5592b3d23cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 --2- 172.21.15.77:0/3097610993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b3d1f4a0 0x5592b3d32a20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 >> 172.21.15.77:0/3097610993 conn(0x5592b3d1ba30 msgr2=0x5592b3d21710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 shutdown_connections 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 1 -- 172.21.15.77:0/3097610993 wait complete. 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.713+0000 7f165ea63740 20 client.16248 trim_cache size 0 max 16384 2024-09-15T22:14:45.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 msgr2=0x7f16400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 0x7f16400a5870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5592b3d20460 tx=0x7f164c04c000 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 msgr2=0x7f1654198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1650009b60 tx=0x7f1650007590 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 shutdown_connections 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16400a33b0 0x7f16400a5870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16541082a0 0x7f165419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16541036e0 0x7f1654198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 --2- 172.21.15.77:0/3776661836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1654102d30 0x7f1654198530 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 >> 172.21.15.77:0/3776661836 conn(0x7f16540fc7d0 msgr2=0x7f1654105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 shutdown_connections 2024-09-15T22:14:45.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:45.714+0000 7f165c68a640 1 -- 172.21.15.77:0/3776661836 wait complete. 2024-09-15T22:14:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:46 smithi077 ceph-mon[30980]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:46 smithi196 ceph-mon[35109]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:46 smithi117 ceph-mon[35021]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-15T22:14:46.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 146989 14.73 MB/sec warmup 55 sec latency 63.711 ms 2024-09-15T22:14:47.230 INFO:tasks.cephfs.filesystem:scrub status for tag:5bbef4cc-0b86-4628-8b46-cb66886f17b0 - {'path': '/', 'tag': '5bbef4cc-0b86-4628-8b46-cb66886f17b0', 'options': 'recursive,force'} 2024-09-15T22:14:47.299 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:47.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 149583 14.72 MB/sec warmup 56 sec latency 28.879 ms 2024-09-15T22:14:47.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.900+0000 7f4651b8f640 1 -- 172.21.15.77:0/2710592539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 msgr2=0x7f464c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.900+0000 7f4651b8f640 1 --2- 172.21.15.77:0/2710592539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 0x7f464c110250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f463c009900 tx=0x7f463c02a440 comp rx=0 tx=0).stop 2024-09-15T22:14:47.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.902+0000 7f4651b8f640 1 -- 172.21.15.77:0/2710592539 shutdown_connections 2024-09-15T22:14:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.902+0000 7f4651b8f640 1 --2- 172.21.15.77:0/2710592539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 0x7f464c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.902+0000 7f4651b8f640 1 --2- 172.21.15.77:0/2710592539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.902+0000 7f4651b8f640 1 --2- 172.21.15.77:0/2710592539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 0x7f464c110250 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.902+0000 7f4651b8f640 1 -- 172.21.15.77:0/2710592539 >> 172.21.15.77:0/2710592539 conn(0x7f464c06e140 msgr2=0x7f464c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.906+0000 7f4651b8f640 1 -- 172.21.15.77:0/2710592539 shutdown_connections 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.909+0000 7f4651b8f640 1 -- 172.21.15.77:0/2710592539 wait complete. 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 Processor -- start 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 -- start start 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 0x7f464c118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 0x7f464c111d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f464c120ed0 con 0x7f464c109830 2024-09-15T22:14:47.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f464c120d50 con 0x7f464c109450 2024-09-15T22:14:47.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.910+0000 7f4651b8f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f464c121050 con 0x7f464c11b140 2024-09-15T22:14:47.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.912+0000 7f464bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 0x7f464c111d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.912+0000 7f464affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.912+0000 7f464affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38396/0 (socket says 172.21.15.77:38396) 2024-09-15T22:14:47.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.912+0000 7f464affd640 1 -- 172.21.15.77:0/3084792267 learned_addr learned my addr 172.21.15.77:0/3084792267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 -- 172.21.15.77:0/3084792267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 msgr2=0x7f464c111d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 0x7f464c111d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 -- 172.21.15.77:0/3084792267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 msgr2=0x7f464c118750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 0x7f464c118750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 -- 172.21.15.77:0/3084792267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c112360 con 0x7f464c109830 2024-09-15T22:14:47.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.913+0000 7f464affd640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f464000b450 tx=0x7f464000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.918+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640015020 con 0x7f464c109830 2024-09-15T22:14:47.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.918+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4640011d70 con 0x7f464c109830 2024-09-15T22:14:47.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.918+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464000f340 con 0x7f464c109830 2024-09-15T22:14:47.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.919+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c116280 con 0x7f464c109830 2024-09-15T22:14:47.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.919+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c116770 con 0x7f464c109830 2024-09-15T22:14:47.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.921+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4638005290 con 0x7f464c109830 2024-09-15T22:14:47.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.923+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4640018020 con 0x7f464c109830 2024-09-15T22:14:47.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.924+0000 7f4648ff9640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 0x7f46340a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.924+0000 7f464b7fe640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 0x7f46340a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.925+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f46400cf9b0 con 0x7f464c109830 2024-09-15T22:14:47.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.925+0000 7f464b7fe640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 0x7f46340a5a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f463c005c20 tx=0x7f463c000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:47.934+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f464001f020 con 0x7f464c109830 2024-09-15T22:14:48.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.198+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4638005850 con 0x7f464c109830 2024-09-15T22:14:48.201 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:48 smithi077 ceph-mon[30980]: pgmap v74: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 278 op/s 2024-09-15T22:14:48.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.199+0000 7f4648ff9640 1 -- 172.21.15.77:0/3084792267 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f464000d040 con 0x7f464c109830 2024-09-15T22:14:48.201 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:14:48.202 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:14:48.202 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 msgr2=0x7f46340a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 0x7f46340a5a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f463c005c20 tx=0x7f463c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 msgr2=0x7f464c1117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f464000b450 tx=0x7f464000b910 comp rx=0 tx=0).stop 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 shutdown_connections 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46340a35c0 0x7f46340a5a80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:48.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f464c11b140 0x7f464c111d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:48.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f464c109830 0x7f464c1117e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:48.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 --2- 172.21.15.77:0/3084792267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464c109450 0x7f464c118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:48.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 >> 172.21.15.77:0/3084792267 conn(0x7f464c06e140 msgr2=0x7f464c11bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:48.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 shutdown_connections 2024-09-15T22:14:48.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:48.202+0000 7f4651b8f640 1 -- 172.21.15.77:0/3084792267 wait complete. 2024-09-15T22:14:48.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:48 smithi196 ceph-mon[35109]: pgmap v74: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 278 op/s 2024-09-15T22:14:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:48 smithi117 ceph-mon[35021]: pgmap v74: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 278 op/s 2024-09-15T22:14:48.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 150073 14.49 MB/sec warmup 57 sec latency 242.121 ms 2024-09-15T22:14:49.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-15T22:14:49.019 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-15T22:14:49.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:49 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3084792267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:49.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:49 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3084792267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:49.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:49 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3084792267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:49.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:49.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 152352 14.46 MB/sec warmup 58 sec latency 29.375 ms 2024-09-15T22:14:49.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 -- 172.21.15.77:0/2842726662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c101360 msgr2=0x7ffa8c10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 --2- 172.21.15.77:0/2842726662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c101360 0x7ffa8c10df90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009900 tx=0x7ffa7402a410 comp rx=0 tx=0).stop 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 -- 172.21.15.77:0/2842726662 shutdown_connections 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 --2- 172.21.15.77:0/2842726662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 --2- 172.21.15.77:0/2842726662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c101360 0x7ffa8c10df90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 --2- 172.21.15.77:0/2842726662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 -- 172.21.15.77:0/2842726662 >> 172.21.15.77:0/2842726662 conn(0x7ffa8c0fc7d0 msgr2=0x7ffa8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 -- 172.21.15.77:0/2842726662 shutdown_connections 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.913+0000 7ffa908dd640 1 -- 172.21.15.77:0/2842726662 wait complete. 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.914+0000 7ffa908dd640 1 Processor -- start 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.914+0000 7ffa908dd640 1 -- start start 2024-09-15T22:14:49.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c101360 0x7ffa8c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1142f0 con 0x7ffa8c10e4d0 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c114170 con 0x7ffa8c1009b0 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa908dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c114470 con 0x7ffa8c101360 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34196/0 (socket says 172.21.15.77:34196) 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/565295364 learned_addr learned my addr 172.21.15.77:0/565295364 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c101360 0x7ffa8c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 -- 172.21.15.77:0/565295364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c101360 msgr2=0x7ffa8c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c101360 0x7ffa8c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 -- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 msgr2=0x7ffa8c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 -- 172.21.15.77:0/565295364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c19ad30 con 0x7ffa8c1009b0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c1a0440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009fd0 tx=0x7ffa7c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.915+0000 7ffa83fff640 1 -- 172.21.15.77:0/565295364 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c015020 con 0x7ffa8c1009b0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.916+0000 7ffa83fff640 1 -- 172.21.15.77:0/565295364 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c011d10 con 0x7ffa8c1009b0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.916+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c19b020 con 0x7ffa8c1009b0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.916+0000 7ffa83fff640 1 -- 172.21.15.77:0/565295364 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00f2e0 con 0x7ffa8c1009b0 2024-09-15T22:14:49.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.916+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a7310 con 0x7ffa8c1009b0 2024-09-15T22:14:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.918+0000 7ffa83fff640 1 -- 172.21.15.77:0/565295364 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa7c018020 con 0x7ffa8c1009b0 2024-09-15T22:14:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.918+0000 7ffa83fff640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 0x7ffa780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.918+0000 7ffa92cb6740 1 Processor -- start 2024-09-15T22:14:49.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 0x7ffa780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa83fff640 1 -- 172.21.15.77:0/565295364 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ffa7c0cf6a0 con 0x7ffa8c1009b0 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 -- start start 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230778850 0x563230778c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230785e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5632306dd160 con 0x563230783a30 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563230684720 con 0x563230778850 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5632306d9120 con 0x563230779170 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 0x7ffa780a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74005b50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa81d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa81d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51892/0 (socket says 172.21.15.77:51892) 2024-09-15T22:14:49.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230785e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa81d9b640 1 -- 172.21.15.77:0/933474150 learned_addr learned my addr 172.21.15.77:0/933474150 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.919+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230778850 0x563230778c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 -- 172.21.15.77:0/933474150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230778850 msgr2=0x563230778c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230778850 0x563230778c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 -- 172.21.15.77:0/933474150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 msgr2=0x563230785e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230785e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 -- 172.21.15.77:0/933474150 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323044eba0 con 0x563230779170 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa74033000 tx=0x7ffa7403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae7e51f514c85d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa80d99640 1 -- 172.21.15.77:0/933474150 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7404d070 con 0x563230779170 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa80d99640 1 -- 172.21.15.77:0/933474150 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74004ba0 con 0x563230779170 2024-09-15T22:14:49.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.920+0000 7ffa80d99640 1 -- 172.21.15.77:0/933474150 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035040 con 0x563230779170 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- 172.21.15.77:0/933474150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 msgr2=0x5632307834f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa74033000 tx=0x7ffa7403d000 comp rx=0 tx=0).stop 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- 172.21.15.77:0/933474150 shutdown_connections 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230785e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230779170 0x5632307834f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/933474150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230778850 0x563230778c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- 172.21.15.77:0/933474150 >> 172.21.15.77:0/933474150 conn(0x563230774de0 msgr2=0x5632307751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- 172.21.15.77:0/933474150 shutdown_connections 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- 172.21.15.77:0/933474150 wait complete. 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.921+0000 7ffa92cb6740 1 -- start start 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 Processor -- start 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x5632307808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230780e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563230786600 con 0x563230783a30 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563230665900 con 0x563230779170 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563230786780 con 0x563230778850 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa92cb6740 10 client.0 _ll_register_callbacks cb 0x563230663780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x5632307808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34220/0 (socket says 172.21.15.77:34220) 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/4068556903 learned_addr learned my addr 172.21.15.77:0/4068556903 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230780e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 -- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 msgr2=0x563230780e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230780e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 -- 172.21.15.77:0/4068556903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 msgr2=0x563230781340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8a59c640 1 -- 172.21.15.77:0/4068556903 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632306a5c50 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.922+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x5632307808c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ccbc0 tx=0x7ffa7c09b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91901e10530d8320 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c09b750 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa74004ba0 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ffa7c0ccd30 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c09fb00 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 10 client.25588 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563230792290 con 0x563230778850 2024-09-15T22:14:49.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74004ba0 con 0x563230778850 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ffa7c0cf370 con 0x563230778850 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 10 client.25588 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 10 client.25588 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 20 client.25588 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307935e0 0x5632307959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 4 client.25588 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.923+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632306ea890 con 0x5632307935e0 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.924+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307935e0 0x5632307959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.924+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307935e0 0x5632307959f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6faaa6029a76a47 server_cookie=503a6e6d17678ec in_seq=0 out_seq=0 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.924+0000 7ffa71ffb640 10 client.25588 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa71ffb640 1 -- 172.21.15.77:0/4068556903 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632306ea890 con 0x5632307935e0 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa71ffb640 10 client.25588 handle_command_reply: tid=0 2024-09-15T22:14:49.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 client.25588 shutdown 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 msgr2=0x5632307808c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x5632307808c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ccbc0 tx=0x7ffa7c09b350 comp rx=0 tx=0).stop 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 shutdown_connections 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307935e0 0x5632307959f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x563230781340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230780e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa71ffb640 0 client.25588 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/4068556903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x5632307808c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 >> 172.21.15.77:0/4068556903 conn(0x563230774de0 msgr2=0x56323077aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 shutdown_connections 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 1 -- 172.21.15.77:0/4068556903 wait complete. 2024-09-15T22:14:49.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.925+0000 7ffa92cb6740 20 client.25588 trim_cache size 0 max 16384 2024-09-15T22:14:49.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 Processor -- start 2024-09-15T22:14:49.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 -- start start 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230773dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x56323079b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b10f0 con 0x563230783a30 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b2cf0 con 0x563230779170 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b17c0 con 0x563230778850 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34232/0 (socket says 172.21.15.77:34232) 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/3157781073 learned_addr learned my addr 172.21.15.77:0/3157781073 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230773dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa81d9b640 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x56323079b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/3157781073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 msgr2=0x563230773dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230773dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/3157781073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 msgr2=0x56323079b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x56323079b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.942+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/3157781073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632307703f0 con 0x563230783a30 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400fd00 tx=0x7ffa84009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eda41125a6cc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa73fff640 1 -- 172.21.15.77:0/3157781073 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400fdf0 con 0x563230783a30 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa73fff640 1 -- 172.21.15.77:0/3157781073 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa840120d0 con 0x563230783a30 2024-09-15T22:14:49.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa73fff640 1 -- 172.21.15.77:0/3157781073 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840059d0 con 0x563230783a30 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 -- 172.21.15.77:0/3157781073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 msgr2=0x56323079b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400fd00 tx=0x7ffa84009500 comp rx=0 tx=0).stop 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 -- 172.21.15.77:0/3157781073 shutdown_connections 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x56323079b800 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x56323079b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/3157781073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230773dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 -- 172.21.15.77:0/3157781073 >> 172.21.15.77:0/3157781073 conn(0x563230774de0 msgr2=0x5632307736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 -- 172.21.15.77:0/3157781073 shutdown_connections 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.943+0000 7ffa92cb6740 1 -- 172.21.15.77:0/3157781073 wait complete. 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 -- start start 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 Processor -- start 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230787fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230788520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x5632307f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b2560 con 0x563230783a30 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b1e90 con 0x563230779170 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1af0d0 con 0x563230778850 2024-09-15T22:14:49.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa92cb6740 10 client.0 _ll_register_callbacks cb 0x7ffa7c010f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x5632307f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230787fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230787fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51920/0 (socket says 172.21.15.77:51920) 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8a59c640 1 -- 172.21.15.77:0/1088920743 learned_addr learned my addr 172.21.15.77:0/1088920743 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa81d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230788520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/1088920743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 msgr2=0x563230787fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.944+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230787fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 msgr2=0x563230788520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230788520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa8ad9d640 1 -- 172.21.15.77:0/1088920743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632307f7890 con 0x563230783a30 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa8ad9d640 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x5632307f72d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400fcd0 tx=0x7ffa84005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c28858e203683f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:14:49.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400f970 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ffa84004a40 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa840120d0 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa84036050 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 10 client.16269 fetch_fsmap learned FSMap version 32 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632307fec20 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa840120d0 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ffa84010bb0 con 0x563230783a30 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 10 client.16269 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 10 client.16269 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 20 client.16269 populate_metadata read hostname 'smithi077' 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307ff380 0x5632308017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 4 client.16269 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-15T22:14:49.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.945+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x5632306e4be0 con 0x5632307ff380 2024-09-15T22:14:49.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.946+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307ff380 0x5632308017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.946+0000 7ffa8a59c640 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307ff380 0x5632308017a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361aecc8eaf407e6 server_cookie=27269cf9137d766f in_seq=0 out_seq=0 2024-09-15T22:14:49.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:49.946+0000 7ffa71ffb640 10 client.16269 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa71ffb640 1 -- 172.21.15.77:0/1088920743 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5632306e4be0 con 0x5632307ff380 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa71ffb640 10 client.16269 handle_command_reply: tid=0 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 client.16269 shutdown 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 msgr2=0x5632307f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x5632307f72d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400fcd0 tx=0x7ffa84005bd0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 shutdown_connections 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5632307ff380 0x5632308017a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563230783a30 0x5632307f72d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa71ffb640 0 client.16269 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.034+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563230779170 0x563230788520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa92cb6740 1 --2- 172.21.15.77:0/1088920743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563230778850 0x563230787fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 >> 172.21.15.77:0/1088920743 conn(0x563230774de0 msgr2=0x5632307810a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 shutdown_connections 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa92cb6740 1 -- 172.21.15.77:0/1088920743 wait complete. 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa92cb6740 20 client.16269 trim_cache size 0 max 16384 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:14:50.038 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 6, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "94f4c326": { 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 6, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 139.6, 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:14:50.039 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:14:50.040 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 msgr2=0x7ffa780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 0x7ffa780a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74005b50 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 msgr2=0x7ffa8c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c1a0440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009fd0 tx=0x7ffa7c00b870 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 shutdown_connections 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa780a3430 0x7ffa780a58f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa8c10e4d0 0x7ffa8c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa8c101360 0x7ffa8c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 --2- 172.21.15.77:0/565295364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa8c1009b0 0x7ffa8c1a0440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:50.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.035+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 >> 172.21.15.77:0/565295364 conn(0x7ffa8c0fc7d0 msgr2=0x7ffa8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:50.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.036+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 shutdown_connections 2024-09-15T22:14:50.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:14:50.036+0000 7ffa908dd640 1 -- 172.21.15.77:0/565295364 wait complete. 2024-09-15T22:14:50.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:50 smithi077 ceph-mon[30980]: pgmap v75: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-15T22:14:50.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:50 smithi196 ceph-mon[35109]: pgmap v75: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-15T22:14:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:50 smithi117 ceph-mon[35021]: pgmap v75: 129 pgs: 129 active+clean; 91 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-15T22:14:50.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 14.29 MB/sec warmup 59 sec latency 429.642 ms 2024-09-15T22:14:50.947 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '94f4c326', quiesce took 5.5 seconds. Will release after: 14.05 2024-09-15T22:14:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:14:51.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:14:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:14:51.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 14.05 MB/sec warmup 60 sec latency 1429.738 ms 2024-09-15T22:14:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:52 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:14:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:52 smithi077 ceph-mon[30980]: pgmap v76: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 436 op/s 2024-09-15T22:14:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:52 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:14:52.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:52 smithi196 ceph-mon[35109]: pgmap v76: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 436 op/s 2024-09-15T22:14:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:52 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:14:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:52 smithi117 ceph-mon[35021]: pgmap v76: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 436 op/s 2024-09-15T22:14:52.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 13.82 MB/sec warmup 61 sec latency 2429.876 ms 2024-09-15T22:14:53.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 13.60 MB/sec warmup 62 sec latency 3429.967 ms 2024-09-15T22:14:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:54 smithi077 ceph-mon[30980]: pgmap v77: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:54 smithi196 ceph-mon[35109]: pgmap v77: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:54 smithi117 ceph-mon[35021]: pgmap v77: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:54.460 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 13.39 MB/sec warmup 63 sec latency 4430.051 ms 2024-09-15T22:14:55.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 13.18 MB/sec warmup 64 sec latency 5430.131 ms 2024-09-15T22:14:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:56 smithi077 ceph-mon[30980]: pgmap v78: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:56.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:56 smithi196 ceph-mon[35109]: pgmap v78: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:56 smithi117 ceph-mon[35021]: pgmap v78: 129 pgs: 129 active+clean; 86 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 345 op/s 2024-09-15T22:14:56.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.97 MB/sec warmup 65 sec latency 6430.255 ms 2024-09-15T22:14:57.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.78 MB/sec warmup 66 sec latency 7430.356 ms 2024-09-15T22:14:58.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:14:58 smithi077 ceph-mon[30980]: pgmap v79: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-15T22:14:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:14:58 smithi196 ceph-mon[35109]: pgmap v79: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-15T22:14:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:14:58 smithi117 ceph-mon[35021]: pgmap v79: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-09-15T22:14:58.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.59 MB/sec warmup 67 sec latency 8430.457 ms 2024-09-15T22:14:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.40 MB/sec warmup 68 sec latency 9430.570 ms 2024-09-15T22:15:00.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:00 smithi077 ceph-mon[30980]: pgmap v80: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-15T22:15:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:00 smithi196 ceph-mon[35109]: pgmap v80: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-15T22:15:00.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:00 smithi117 ceph-mon[35021]: pgmap v80: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-09-15T22:15:00.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.22 MB/sec warmup 69 sec latency 10430.686 ms 2024-09-15T22:15:01.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 12.05 MB/sec warmup 70 sec latency 11430.804 ms 2024-09-15T22:15:02.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:02 smithi077 ceph-mon[30980]: pgmap v81: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 276 op/s 2024-09-15T22:15:02.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:02 smithi196 ceph-mon[35109]: pgmap v81: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 276 op/s 2024-09-15T22:15:02.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:02 smithi117 ceph-mon[35021]: pgmap v81: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 276 op/s 2024-09-15T22:15:02.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.88 MB/sec warmup 71 sec latency 12430.889 ms 2024-09-15T22:15:03.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.71 MB/sec warmup 72 sec latency 13430.993 ms 2024-09-15T22:15:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:04 smithi077 ceph-mon[30980]: pgmap v82: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:04.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:04 smithi196 ceph-mon[35109]: pgmap v82: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:04.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:04 smithi117 ceph-mon[35021]: pgmap v82: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:04.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.55 MB/sec warmup 73 sec latency 14431.053 ms 2024-09-15T22:15:04.998 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:15:05.258 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:05.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.40 MB/sec warmup 74 sec latency 15431.105 ms 2024-09-15T22:15:05.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- 172.21.15.77:0/77248508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 msgr2=0x7f16e01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/77248508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e01031d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009ca0 tx=0x7f16d802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- 172.21.15.77:0/77248508 shutdown_connections 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/77248508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 0x7f16e010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/77248508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/77248508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e01031d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- 172.21.15.77:0/77248508 >> 172.21.15.77:0/77248508 conn(0x7f16e00fc7d0 msgr2=0x7f16e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- 172.21.15.77:0/77248508 shutdown_connections 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- 172.21.15.77:0/77248508 wait complete. 2024-09-15T22:15:05.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 Processor -- start 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.733+0000 7f16e7b5b640 1 -- start start 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e0100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 0x7f16e01014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0112110 con 0x7f16e0102df0 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111f90 con 0x7f16e01037a0 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0112290 con 0x7f16e010c2f0 2024-09-15T22:15:05.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50860/0 (socket says 172.21.15.77:50860) 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 -- 172.21.15.77:0/2439192202 learned_addr learned my addr 172.21.15.77:0/2439192202 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e60f8640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 0x7f16e01014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e50f6640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e0100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 -- 172.21.15.77:0/2439192202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 msgr2=0x7f16e01014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 0x7f16e01014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 -- 172.21.15.77:0/2439192202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 msgr2=0x7f16e0100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e0100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 -- 172.21.15.77:0/2439192202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e01a6ad0 con 0x7f16e0102df0 2024-09-15T22:15:05.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e50f6640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e58f7640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f16d802ade0 tx=0x7f16d80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d802fa20 con 0x7f16e0102df0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d802fbc0 con 0x7f16e0102df0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.734+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e01a6d60 con 0x7f16e0102df0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.735+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d80467b0 con 0x7f16e0102df0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.735+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a7260 con 0x7f16e0102df0 2024-09-15T22:15:05.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.735+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e01031d0 con 0x7f16e0102df0 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.736+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f16d8046950 con 0x7f16e0102df0 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.737+0000 7f16d6ffd640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.737+0000 7f16e50f6640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.737+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f16d80f1440 con 0x7f16e0102df0 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.738+0000 7f16e50f6640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 0x7f16cc0a58f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16e01024e0 tx=0x7f16d0005e50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:05.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.741+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16d80453e0 con 0x7f16e0102df0 2024-09-15T22:15:05.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.943+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16e01072f0 con 0x7f16e0102df0 2024-09-15T22:15:05.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.944+0000 7f16d6ffd640 1 -- 172.21.15.77:0/2439192202 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f16d80bdc80 con 0x7f16e0102df0 2024-09-15T22:15:05.946 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:15:05.946 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:15:05.947 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:15:05.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 msgr2=0x7f16cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:05.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 0x7f16cc0a58f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f16e01024e0 tx=0x7f16d0005e50 comp rx=0 tx=0).stop 2024-09-15T22:15:05.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 msgr2=0x7f16e0100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f16d802ade0 tx=0x7f16d80319f0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 shutdown_connections 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f16cc0a3430 0x7f16cc0a58f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e010c2f0 0x7f16e01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16e01037a0 0x7f16e0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 --2- 172.21.15.77:0/2439192202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16e0102df0 0x7f16e0100a50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 >> 172.21.15.77:0/2439192202 conn(0x7f16e00fc7d0 msgr2=0x7f16e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 shutdown_connections 2024-09-15T22:15:05.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:05.947+0000 7f16e7b5b640 1 -- 172.21.15.77:0/2439192202 wait complete. 2024-09-15T22:15:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:06 smithi077 ceph-mon[30980]: pgmap v83: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:06 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2439192202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:06 smithi196 ceph-mon[35109]: pgmap v83: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:06.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:06 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2439192202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:06 smithi117 ceph-mon[35021]: pgmap v83: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-09-15T22:15:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:06 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2439192202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:06.461 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.24 MB/sec warmup 75 sec latency 16431.196 ms 2024-09-15T22:15:06.680 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 94f4c326 --release --await' 2024-09-15T22:15:06.680 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 94f4c326 --release --await 2024-09-15T22:15:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:07.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 -- 172.21.15.77:0/887466885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45810c180 msgr2=0x7fc45810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 --2- 172.21.15.77:0/887466885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45810c180 0x7fc45810e610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc450009ca0 tx=0x7fc45002a950 comp rx=0 tx=0).stop 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 -- 172.21.15.77:0/887466885 shutdown_connections 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 --2- 172.21.15.77:0/887466885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45810c180 0x7fc45810e610 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 --2- 172.21.15.77:0/887466885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458103700 0x7fc45810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 --2- 172.21.15.77:0/887466885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc458102de0 0x7fc4581031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 -- 172.21.15.77:0/887466885 >> 172.21.15.77:0/887466885 conn(0x7fc4580fc7a0 msgr2=0x7fc4580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 -- 172.21.15.77:0/887466885 shutdown_connections 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.416+0000 7fc460393640 1 -- 172.21.15.77:0/887466885 wait complete. 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 Processor -- start 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 -- start start 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 0x7fc458198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 0x7fc45819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc458112030 con 0x7fc458103700 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc458111eb0 con 0x7fc45810c180 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc460393640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581121b0 con 0x7fc458102de0 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50868/0 (socket says 172.21.15.77:50868) 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 -- 172.21.15.77:0/4293620811 learned_addr learned my addr 172.21.15.77:0/4293620811 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45e12f640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 0x7fc458198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45e930640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 0x7fc45819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 -- 172.21.15.77:0/4293620811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 msgr2=0x7fc458198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 0x7fc458198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.417+0000 7fc45d92e640 1 -- 172.21.15.77:0/4293620811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 msgr2=0x7fc45819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc45d92e640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 0x7fc45819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc45d92e640 1 -- 172.21.15.77:0/4293620811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45819d4b0 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc45e12f640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 0x7fc458198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc45e930640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 0x7fc45819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc45d92e640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b4a0 tx=0x7fc44800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc44f7fe640 1 -- 172.21.15.77:0/4293620811 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448014070 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45819d7a0 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc44f7fe640 1 -- 172.21.15.77:0/4293620811 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448010af0 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc44f7fe640 1 -- 172.21.15.77:0/4293620811 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448017d10 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.418+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4581a5080 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.420+0000 7fc44f7fe640 1 -- 172.21.15.77:0/4293620811 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc448017070 con 0x7fc458103700 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.420+0000 7fc46276c740 1 Processor -- start 2024-09-15T22:15:07.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 -- start start 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc44f7fe640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 0x7fc4440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e88630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560516de0620 con 0x560516e7b980 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560516d87e50 con 0x560516e7b060 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc46276c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560516ddc5e0 con 0x560516e86240 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45e930640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 0x7fc4440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc44f7fe640 1 -- 172.21.15.77:0/4293620811 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc4480cf440 con 0x7fc458103700 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50876/0 (socket says 172.21.15.77:50876) 2024-09-15T22:15:07.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 -- 172.21.15.77:0/3076722533 learned_addr learned my addr 172.21.15.77:0/3076722533 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45e12f640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45e930640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e88630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 -- 172.21.15.77:0/3076722533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 msgr2=0x560516e88630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e88630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 -- 172.21.15.77:0/3076722533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 msgr2=0x560516e7b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45d92e640 1 -- 172.21.15.77:0/3076722533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560516d78d40 con 0x560516e7b980 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.421+0000 7fc45e12f640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc45e930640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 0x7fc4440a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc450005e30 tx=0x7fc45002fb00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc448097ec0 tx=0x7fc448009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167b6d3ea89529ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc44cdea640 1 -- 172.21.15.77:0/3076722533 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44801e050 con 0x560516e7b980 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc44cdea640 1 -- 172.21.15.77:0/3076722533 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44806b8d0 con 0x560516e7b980 2024-09-15T22:15:07.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc44cdea640 1 -- 172.21.15.77:0/3076722533 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480ccad0 con 0x560516e7b980 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc46276c740 1 -- 172.21.15.77:0/3076722533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 msgr2=0x560516e85d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.422+0000 7fc46276c740 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc448097ec0 tx=0x7fc448009390 comp rx=0 tx=0).stop 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- 172.21.15.77:0/3076722533 shutdown_connections 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e88630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e85d00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- 172.21.15.77:0/3076722533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- 172.21.15.77:0/3076722533 >> 172.21.15.77:0/3076722533 conn(0x560516e775f0 msgr2=0x560516e779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- 172.21.15.77:0/3076722533 shutdown_connections 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- 172.21.15.77:0/3076722533 wait complete. 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- start start 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 Processor -- start 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b980 0x560516e83670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e83bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560516e88e10 con 0x560516e7b060 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560516d68f10 con 0x560516e7b980 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560516e88f90 con 0x560516e86240 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc46276c740 10 client.0 _ll_register_callbacks cb 0x560516d8cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc45e12f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc45e12f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50890/0 (socket says 172.21.15.77:50890) 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.423+0000 7fc45e12f640 1 -- 172.21.15.77:0/1992958786 learned_addr learned my addr 172.21.15.77:0/1992958786 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e930640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e83bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45d92e640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b980 0x560516e83670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 -- 172.21.15.77:0/1992958786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 msgr2=0x560516e83bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e83bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 -- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b980 msgr2=0x560516e83670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b980 0x560516e83670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 -- 172.21.15.77:0/1992958786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560516aef580 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e930640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e83bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc45e12f640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d4a0 tx=0x7fc45400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf993f34cda7360b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45401c060 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc4540146f0 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44806b8d0 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc454023050 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc46276c740 10 client.16287 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560516bb9960 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44806b8d0 con 0x560516e7b060 2024-09-15T22:15:07.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc454015b70 con 0x560516e7b060 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc46276c740 10 client.16287 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.424+0000 7fc46276c740 10 client.16287 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc46276c740 20 client.16287 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516e95e10 0x560516e98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc46276c740 4 client.16287 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560516dedb90 con 0x560516e95e10 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc45d92e640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516e95e10 0x560516e98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc45d92e640 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516e95e10 0x560516e98270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f567998df389dae8 server_cookie=c488253b7a6b8fc8 in_seq=0 out_seq=0 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.425+0000 7fc441ffb640 10 client.16287 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc441ffb640 1 -- 172.21.15.77:0/1992958786 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560516dedb90 con 0x560516e95e10 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc441ffb640 10 client.16287 handle_command_reply: tid=0 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 client.16287 shutdown 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 msgr2=0x560516e83130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d4a0 tx=0x7fc45400d960 comp rx=0 tx=0).stop 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 shutdown_connections 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516e95e10 0x560516e98270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e83bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b980 0x560516e83670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 --2- 172.21.15.77:0/1992958786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b060 0x560516e83130 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 >> 172.21.15.77:0/1992958786 conn(0x560516e775f0 msgr2=0x560516e7d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc441ffb640 0 client.16287 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 shutdown_connections 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 1 -- 172.21.15.77:0/1992958786 wait complete. 2024-09-15T22:15:07.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.426+0000 7fc46276c740 20 client.16287 trim_cache size 0 max 16384 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 Processor -- start 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 -- start start 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e76640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581aef80 con 0x560516e7b980 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581b0b80 con 0x560516e7b060 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc46276c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581af650 con 0x560516e86240 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50894/0 (socket says 172.21.15.77:50894) 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 -- 172.21.15.77:0/3900913453 learned_addr learned my addr 172.21.15.77:0/3900913453 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45e930640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e76640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45e12f640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 -- 172.21.15.77:0/3900913453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 msgr2=0x560516e76640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e76640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 -- 172.21.15.77:0/3900913453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 msgr2=0x560516e7dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45d92e640 1 -- 172.21.15.77:0/3900913453 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560516e85070 con 0x560516e7b980 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.444+0000 7fc45e12f640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:07.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc45d92e640 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc44800bfd0 tx=0x7fc4480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bac9891d201703 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc443fff640 1 -- 172.21.15.77:0/3900913453 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44801e040 con 0x560516e7b980 2024-09-15T22:15:07.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc443fff640 1 -- 172.21.15.77:0/3900913453 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4480cf170 con 0x560516e7b980 2024-09-15T22:15:07.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc443fff640 1 -- 172.21.15.77:0/3900913453 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480a3cd0 con 0x560516e7b980 2024-09-15T22:15:07.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 -- 172.21.15.77:0/3900913453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 msgr2=0x560516e7e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc44800bfd0 tx=0x7fc4480047d0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 -- 172.21.15.77:0/3900913453 shutdown_connections 2024-09-15T22:15:07.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e86240 0x560516e76640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e7e4c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 --2- 172.21.15.77:0/3900913453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e7b060 0x560516e7dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.445+0000 7fc46276c740 1 -- 172.21.15.77:0/3900913453 >> 172.21.15.77:0/3900913453 conn(0x560516e775f0 msgr2=0x560516e75f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- 172.21.15.77:0/3900913453 shutdown_connections 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- 172.21.15.77:0/3900913453 wait complete. 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- start start 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 Processor -- start 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e7b060 0x560516e802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e86240 0x560516e80d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581b03f0 con 0x560516e7b980 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581afd20 con 0x560516e86240 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581acf60 con 0x560516e7b060 2024-09-15T22:15:07.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc46276c740 10 client.0 _ll_register_callbacks cb 0x7fc454007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc45d92e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50910/0 (socket says 172.21.15.77:50910) 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.446+0000 7fc45d92e640 1 -- 172.21.15.77:0/423797446 learned_addr learned my addr 172.21.15.77:0/423797446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45e12f640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e7b060 0x560516e802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 -- 172.21.15.77:0/423797446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e7b060 msgr2=0x560516e802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e7b060 0x560516e802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 -- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e86240 msgr2=0x560516e80d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e86240 0x560516e80d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 -- 172.21.15.77:0/423797446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560516effc10 con 0x560516e7b980 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc45d92e640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc448004700 tx=0x7fc4480a4260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68cb476a5088c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448097b60 con 0x560516e7b980 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc4480cca10 con 0x560516e7b980 2024-09-15T22:15:07.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4480cf170 con 0x560516e7b980 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4480ccc10 con 0x560516e7b980 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 10 client.16293 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560516f00f50 con 0x560516e7b980 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4480cf170 con 0x560516e7b980 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc4480b4170 con 0x560516e7b980 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 10 client.16293 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 10 client.16293 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 20 client.16293 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516f01240 0x560516f03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 4 client.16293 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "94f4c326", "release": true, "await": true}] 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.447+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "94f4c326", "release": true, "await": true}) -- 0x560516de80a0 con 0x560516f01240 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.448+0000 7fc45e12f640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516f01240 0x560516f03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:07.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.448+0000 7fc441ffb640 10 client.16293 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:07.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.448+0000 7fc45e12f640 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516f01240 0x560516f03630 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e5c28b9ef9ef9c server_cookie=b60a89a2037c76bf in_seq=0 out_seq=0 2024-09-15T22:15:07.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc441ffb640 1 -- 172.21.15.77:0/423797446 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560516de80a0 con 0x560516f01240 2024-09-15T22:15:07.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc441ffb640 10 client.16293 handle_command_reply: tid=0 2024-09-15T22:15:07.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 client.16293 shutdown 2024-09-15T22:15:07.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 msgr2=0x560516e80810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc448004700 tx=0x7fc4480a4260 comp rx=0 tx=0).stop 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 shutdown_connections 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560516f01240 0x560516f03630 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560516e86240 0x560516e80d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560516e7b980 0x560516e80810 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc441ffb640 0 client.16293 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 --2- 172.21.15.77:0/423797446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560516e7b060 0x560516e802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.458+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 >> 172.21.15.77:0/423797446 conn(0x560516e775f0 msgr2=0x560516e838f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 shutdown_connections 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc46276c740 1 -- 172.21.15.77:0/423797446 wait complete. 2024-09-15T22:15:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc46276c740 20 client.16293 trim_cache size 0 max 16384 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 8, 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "94f4c326": { 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 8, 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 139.6, 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:07.463 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:07.464 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:07.464 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:07.464 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 msgr2=0x7fc4440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 0x7fc4440a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc450005e30 tx=0x7fc45002fb00 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 msgr2=0x7fc458198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b4a0 tx=0x7fc44800b960 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 shutdown_connections 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4440a33b0 0x7fc4440a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc45810c180 0x7fc45819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc458103700 0x7fc458198a40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 --2- 172.21.15.77:0/4293620811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc458102de0 0x7fc458198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 >> 172.21.15.77:0/4293620811 conn(0x7fc4580fc7a0 msgr2=0x7fc4580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 shutdown_connections 2024-09-15T22:15:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:07.459+0000 7fc460393640 1 -- 172.21.15.77:0/4293620811 wait complete. 2024-09-15T22:15:07.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153105 11.10 MB/sec warmup 76 sec latency 17431.236 ms 2024-09-15T22:15:08.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:08 smithi077 ceph-mon[30980]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-09-15T22:15:08.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:08 smithi196 ceph-mon[35109]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-09-15T22:15:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:08 smithi117 ceph-mon[35021]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-09-15T22:15:08.369 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '94f4c326', total seconds elapsed: 22.9 2024-09-15T22:15:08.369 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-15T22:15:08.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:15:08.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153812 10.98 MB/sec warmup 77 sec latency 17432.405 ms 2024-09-15T22:15:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:09.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 156562 11.06 MB/sec warmup 78 sec latency 41.766 ms 2024-09-15T22:15:09.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 -- 172.21.15.77:0/216648952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 msgr2=0x7fc6ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 --2- 172.21.15.77:0/216648952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec1067e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009900 tx=0x7fc6e002a410 comp rx=0 tx=0).stop 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 -- 172.21.15.77:0/216648952 shutdown_connections 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 --2- 172.21.15.77:0/216648952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 --2- 172.21.15.77:0/216648952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec1067e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 --2- 172.21.15.77:0/216648952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 -- 172.21.15.77:0/216648952 >> 172.21.15.77:0/216648952 conn(0x7fc6ec0fd8c0 msgr2=0x7fc6ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 -- 172.21.15.77:0/216648952 shutdown_connections 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.552+0000 7fc6f408d640 1 -- 172.21.15.77:0/216648952 wait complete. 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 Processor -- start 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 -- start start 2024-09-15T22:15:09.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec111f70 con 0x7fc6ec102790 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec111df0 con 0x7fc6ec101de0 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f408d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec1120f0 con 0x7fc6ec106f10 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1628640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1628640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54804/0 (socket says 172.21.15.77:54804) 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1628640 1 -- 172.21.15.77:0/3485630246 learned_addr learned my addr 172.21.15.77:0/3485630246 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f262a640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 -- 172.21.15.77:0/3485630246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 msgr2=0x7fc6ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:09.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 -- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 msgr2=0x7fc6ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1e29640 1 -- 172.21.15.77:0/3485630246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec1a6910 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f262a640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.553+0000 7fc6f1628640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6f1e29640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec0744f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400e9f0 tx=0x7fc6e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400ccc0 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec1a6c00 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4010910 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4018960 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.554+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec1a7140 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.555+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec1021c0 con 0x7fc6ec101de0 2024-09-15T22:15:09.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.556+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc6e4010ac0 con 0x7fc6ec101de0 2024-09-15T22:15:09.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.556+0000 7fc6eaffd640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 0x7fc6d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:09.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.557+0000 7fc6f1628640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 0x7fc6d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:09.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.557+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc6e40d0590 con 0x7fc6ec101de0 2024-09-15T22:15:09.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.557+0000 7fc6f1628640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 0x7fc6d40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0031550 tx=0x7fc6e0005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:09.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.560+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6e409cc30 con 0x7fc6ec101de0 2024-09-15T22:15:09.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.798+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6ec075a20 con 0x7fc6ec101de0 2024-09-15T22:15:09.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.799+0000 7fc6eaffd640 1 -- 172.21.15.77:0/3485630246 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fc6ec075a20 con 0x7fc6ec101de0 2024-09-15T22:15:09.801 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:15:09.801 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:15:09.801 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:15:09.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 msgr2=0x7fc6d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 0x7fc6d40a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0031550 tx=0x7fc6e0005a20 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 msgr2=0x7fc6ec0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec0744f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400e9f0 tx=0x7fc6e400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 shutdown_connections 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6d40a33b0 0x7fc6d40a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6ec106f10 0x7fc6ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6ec102790 0x7fc6ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 --2- 172.21.15.77:0/3485630246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6ec101de0 0x7fc6ec0744f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 >> 172.21.15.77:0/3485630246 conn(0x7fc6ec0fd8c0 msgr2=0x7fc6ec0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 shutdown_connections 2024-09-15T22:15:09.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:09.801+0000 7fc6f408d640 1 -- 172.21.15.77:0/3485630246 wait complete. 2024-09-15T22:15:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:10 smithi077 ceph-mon[30980]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-15T22:15:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:10 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3485630246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:10.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:10 smithi196 ceph-mon[35109]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-15T22:15:10.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:10 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3485630246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:10 smithi117 ceph-mon[35021]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-15T22:15:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:10 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3485630246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:10.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 159648 11.14 MB/sec warmup 79 sec latency 11.016 ms 2024-09-15T22:15:10.588 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-15T22:15:10.588 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-15T22:15:10.860 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:11.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 161986 11.11 MB/sec warmup 80 sec latency 20.916 ms 2024-09-15T22:15:11.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1516301374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf4106f20 msgr2=0x7fdcf410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1516301374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf4106f20 0x7fdcf410eaa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009ca0 tx=0x7fdcec02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1516301374 shutdown_connections 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1516301374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf4106f20 0x7fdcf410eaa0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1516301374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41027a0 0x7fdcf41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1516301374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1516301374 >> 172.21.15.77:0/1516301374 conn(0x7fdcf40fd8f0 msgr2=0x7fdcf40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1516301374 shutdown_connections 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.433+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1516301374 wait complete. 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 Processor -- start 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 -- start start 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf41027a0 0x7fdcf4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 0x7fdcf4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4111f80 con 0x7fdcf41027a0 2024-09-15T22:15:11.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4111e00 con 0x7fdcf4106f20 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.434+0000 7fdcfa6fb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4112100 con 0x7fdcf4101df0 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 0x7fdcf4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:46716/0 (socket says 172.21.15.77:46716) 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf3fff640 1 -- 172.21.15.77:0/1433589375 learned_addr learned my addr 172.21.15.77:0/1433589375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf41027a0 0x7fdcf4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 -- 172.21.15.77:0/1433589375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 msgr2=0x7fdcf40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 -- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf41027a0 msgr2=0x7fdcf4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf41027a0 0x7fdcf4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 -- 172.21.15.77:0/1433589375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf41a6890 con 0x7fdcf4106f20 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:11.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf8c98640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 0x7fdcf4074f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcec02ade0 tx=0x7fdcec0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.435+0000 7fdcf17fa640 1 -- 172.21.15.77:0/1433589375 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec003b70 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.436+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf41a6b20 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.436+0000 7fdcf17fa640 1 -- 172.21.15.77:0/1433589375 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcec003d10 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.436+0000 7fdcf17fa640 1 -- 172.21.15.77:0/1433589375 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec02fc40 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.436+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf41a70b0 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.438+0000 7fdcf17fa640 1 -- 172.21.15.77:0/1433589375 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdcec004050 con 0x7fdcf4106f20 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.438+0000 7fdcfcad4740 1 Processor -- start 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.438+0000 7fdcfcad4740 1 -- start start 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdcbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb4cd23160 con 0x55bb4cdbe850 2024-09-15T22:15:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb4ccca720 con 0x55bb4cdbf170 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf17fa640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 0x7fdcdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4cd1f120 con 0x55bb4cdc9a30 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54818/0 (socket says 172.21.15.77:54818) 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1779020971 learned_addr learned my addr 172.21.15.77:0/1779020971 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf8c98640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 0x7fdcdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdcbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf17fa640 1 -- 172.21.15.77:0/1433589375 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdcec0f0f00 con 0x7fdcf4106f20 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1779020971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 msgr2=0x55bb4cdcbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdcbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1779020971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 msgr2=0x55bb4cdc94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1779020971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4ca94ba0 con 0x55bb4cdbe850 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.439+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21f93fe45df1908 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcf8c98640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 0x7fdcdc0a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0bcc50 tx=0x7fdcec0bcd20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcda7fc640 1 -- 172.21.15.77:0/1779020971 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8013cf0 con 0x55bb4cdbe850 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcda7fc640 1 -- 172.21.15.77:0/1779020971 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce8005020 con 0x55bb4cdbe850 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcda7fc640 1 -- 172.21.15.77:0/1779020971 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004a00 con 0x55bb4cdbe850 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1779020971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 msgr2=0x55bb4cdbec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.440+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce8005850 comp rx=0 tx=0).stop 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1779020971 shutdown_connections 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdcbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1779020971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdbec30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1779020971 >> 172.21.15.77:0/1779020971 conn(0x55bb4cdbade0 msgr2=0x55bb4cdbb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1779020971 shutdown_connections 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1779020971 wait complete. 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- start start 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 Processor -- start 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdc7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bb4cdcc600 con 0x55bb4cdbe850 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb4ccab900 con 0x55bb4cdbf170 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bb4cdcc780 con 0x55bb4cdc9a30 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.441+0000 7fdcfcad4740 10 client.0 _ll_register_callbacks cb 0x55bb4cca9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54822/0 (socket says 172.21.15.77:54822) 2024-09-15T22:15:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1492369311 learned_addr learned my addr 172.21.15.77:0/1492369311 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdc7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1492369311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 msgr2=0x55bb4cdc7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdc7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 msgr2=0x55bb4cdc6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1492369311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4ccebc50 con 0x55bb4cdbe850 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdce8004650 tx=0x7fdce80056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2750d71fbe1397 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8011d30 con 0x55bb4cdbe850 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdce80046c0 con 0x55bb4cdbe850 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce8005020 con 0x55bb4cdbe850 2024-09-15T22:15:11.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdce80048c0 con 0x55bb4cdbe850 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcfcad4740 10 client.16311 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.442+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb4cdd8270 con 0x55bb4cdbe850 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce8005020 con 0x55bb4cdbe850 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdce801fab0 con 0x55bb4cdbe850 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 10 client.16311 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 10 client.16311 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 20 client.16311 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4cdd95c0 0x55bb4cddb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 4 client.16311 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4cd30890 con 0x55bb4cdd95c0 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.443+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4cdd95c0 0x55bb4cddb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4cdd95c0 0x55bb4cddb9b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3e4545f37541fe server_cookie=12a155d6718fe844 in_seq=0 out_seq=0 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcd37fe640 10 client.16311 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:11.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1492369311 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb4cd30890 con 0x55bb4cdd95c0 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcd37fe640 10 client.16311 handle_command_reply: tid=0 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 client.16311 shutdown 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 msgr2=0x55bb4cdc68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdce8004650 tx=0x7fdce80056f0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 shutdown_connections 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4cdd95c0 0x55bb4cddb9b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdc7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbf170 0x55bb4cdc6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1492369311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc68c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcd37fe640 0 client.16311 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.444+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 >> 172.21.15.77:0/1492369311 conn(0x55bb4cdbade0 msgr2=0x55bb4cdc0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.445+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 shutdown_connections 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.445+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1492369311 wait complete. 2024-09-15T22:15:11.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.445+0000 7fdcfcad4740 20 client.16311 trim_cache size 0 max 16384 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.462+0000 7fdcfcad4740 1 Processor -- start 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.462+0000 7fdcfcad4740 1 -- start start 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 0x55bb4cdd4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdd54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41b10b0 con 0x55bb4cdbf170 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41b2cb0 con 0x55bb4cdbe850 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41b1780 con 0x55bb4cdc9a30 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 0x55bb4cdd4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51528/0 (socket says 172.21.15.77:51528) 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 0x55bb4cdd4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54824/0 (socket says 172.21.15.77:54824) 2024-09-15T22:15:11.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1584960856 learned_addr learned my addr 172.21.15.77:0/1584960856 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdd54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1584960856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 msgr2=0x55bb4cdd54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdd54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1584960856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 msgr2=0x55bb4cdd4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 0x55bb4cdd4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1584960856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4cdbac10 con 0x55bb4cdbe850 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce8013e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2028075fc9d562 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcd9ffb640 1 -- 172.21.15.77:0/1584960856 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8009a00 con 0x55bb4cdbe850 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcd9ffb640 1 -- 172.21.15.77:0/1584960856 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce8004680 con 0x55bb4cdbe850 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.463+0000 7fdcd9ffb640 1 -- 172.21.15.77:0/1584960856 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce802c8e0 con 0x55bb4cdbe850 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1584960856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 msgr2=0x55bb4cdc12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce8013e50 comp rx=0 tx=0).stop 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1584960856 shutdown_connections 2024-09-15T22:15:11.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdd54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbf170 0x55bb4cdd4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1584960856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdbe850 0x55bb4cdc12b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1584960856 >> 172.21.15.77:0/1584960856 conn(0x55bb4cdbade0 msgr2=0x55bb4cdd4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1584960856 shutdown_connections 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.464+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1584960856 wait complete. 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 -- start start 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 Processor -- start 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdbf170 0x55bb4cdcde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41b2520 con 0x55bb4cdbe850 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41b1e50 con 0x55bb4cdc9a30 2024-09-15T22:15:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf41af090 con 0x55bb4cdbf170 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcfcad4740 10 client.0 _ll_register_callbacks cb 0x7fdce4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54838/0 (socket says 172.21.15.77:54838) 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcf3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdbf170 0x55bb4cdcde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.465+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1617452521 learned_addr learned my addr 172.21.15.77:0/1617452521 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1617452521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdbf170 msgr2=0x55bb4cdcde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdbf170 0x55bb4cdcde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdc9a30 msgr2=0x55bb4cdce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 -- 172.21.15.77:0/1617452521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4cdbba70 con 0x55bb4cdbe850 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcf37fe640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdce8001eb0 tx=0x7fdce8013de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e21965ce5e09e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004c60 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce8004680 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fdce8004e00 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdce8004040 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcfcad4740 10 client.16317 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb4cdb63f0 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.466+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce8004680 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdce80035f0 con 0x55bb4cdbe850 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 10 client.16317 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 10 client.16317 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 20 client.16317 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4ce44900 0x55bb4ce46cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 4 client.16317 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-15T22:15:11.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x55bb4cd2abe0 con 0x55bb4ce44900 2024-09-15T22:15:11.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4ce44900 0x55bb4ce46cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:11.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcf3fff640 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4ce44900 0x55bb4ce46cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b900272c8e3032c server_cookie=3b32f36faf4d6e66 in_seq=0 out_seq=0 2024-09-15T22:15:11.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.467+0000 7fdcd37fe640 10 client.16317 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcd37fe640 1 -- 172.21.15.77:0/1617452521 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bb4cd2abe0 con 0x55bb4ce44900 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcd37fe640 10 client.16317 handle_command_reply: tid=0 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 client.16317 shutdown 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 msgr2=0x55bb4cdcd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdce8001eb0 tx=0x7fdce8013de0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 shutdown_connections 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bb4ce44900 0x55bb4ce46cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb4cdc9a30 0x55bb4cdce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bb4cdbf170 0x55bb4cdcde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcd37fe640 0 client.16317 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 --2- 172.21.15.77:0/1617452521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bb4cdbe850 0x55bb4cdcd910 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 >> 172.21.15.77:0/1617452521 conn(0x55bb4cdbade0 msgr2=0x55bb4cdd50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 shutdown_connections 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 1 -- 172.21.15.77:0/1617452521 wait complete. 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.546+0000 7fdcfcad4740 20 client.16317 trim_cache size 0 max 16384 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-15T22:15:11.550 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 10, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "70bc1cae": { 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 10, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 139.6, 2024-09-15T22:15:11.551 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:11.552 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 msgr2=0x7fdcdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 0x7fdcdc0a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0bcc50 tx=0x7fdcec0bcd20 comp rx=0 tx=0).stop 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 msgr2=0x7fdcf4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 0x7fdcf4074f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcec02ade0 tx=0x7fdcec0319f0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 shutdown_connections 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcdc0a33b0 0x7fdcdc0a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf4106f20 0x7fdcf4074f70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcf41027a0 0x7fdcf4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 --2- 172.21.15.77:0/1433589375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcf4101df0 0x7fdcf40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:11.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 >> 172.21.15.77:0/1433589375 conn(0x7fdcf40fd8f0 msgr2=0x7fdcf40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:11.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 shutdown_connections 2024-09-15T22:15:11.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:11.547+0000 7fdcfa6fb640 1 -- 172.21.15.77:0/1433589375 wait complete. 2024-09-15T22:15:12.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:12 smithi196 ceph-mon[35109]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 521 KiB/s wr, 94 op/s 2024-09-15T22:15:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:12 smithi117 ceph-mon[35021]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 521 KiB/s wr, 94 op/s 2024-09-15T22:15:12.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.97 MB/sec warmup 81 sec latency 957.195 ms 2024-09-15T22:15:12.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:12 smithi077 ceph-mon[30980]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 521 KiB/s wr, 94 op/s 2024-09-15T22:15:12.595 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70bc1cae', quiesce took 4.2 seconds. Will release after: 15.350000000000001 2024-09-15T22:15:13.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.84 MB/sec warmup 82 sec latency 1957.250 ms 2024-09-15T22:15:14.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:14 smithi196 ceph-mon[35109]: pgmap v87: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:14.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:14 smithi117 ceph-mon[35021]: pgmap v87: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:14.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.70 MB/sec warmup 83 sec latency 2957.361 ms 2024-09-15T22:15:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:14 smithi077 ceph-mon[30980]: pgmap v87: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:15.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.58 MB/sec warmup 84 sec latency 3957.484 ms 2024-09-15T22:15:16.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:16 smithi196 ceph-mon[35109]: pgmap v88: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:16 smithi117 ceph-mon[35021]: pgmap v88: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:16.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.45 MB/sec warmup 85 sec latency 4957.534 ms 2024-09-15T22:15:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:16 smithi077 ceph-mon[30980]: pgmap v88: 129 pgs: 129 active+clean; 90 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 498 KiB/s wr, 74 op/s 2024-09-15T22:15:17.231 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:15:17.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.33 MB/sec warmup 86 sec latency 5957.582 ms 2024-09-15T22:15:17.487 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:18.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 -- 172.21.15.77:0/3414155582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614101420 msgr2=0x7f061410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3414155582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614101420 0x7f061410d950 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f060c009cd0 tx=0x7f060c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 -- 172.21.15.77:0/3414155582 shutdown_connections 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3414155582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614101420 0x7f061410d950 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3414155582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0614100a80 0x7f0614100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3414155582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0614106a80 0x7f0614106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 -- 172.21.15.77:0/3414155582 >> 172.21.15.77:0/3414155582 conn(0x7f06140fc7a0 msgr2=0x7f06140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 -- 172.21.15.77:0/3414155582 shutdown_connections 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.037+0000 7f061c2a0640 1 -- 172.21.15.77:0/3414155582 wait complete. 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 Processor -- start 2024-09-15T22:15:18.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 -- start start 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0614101420 0x7f0614197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0614106a80 0x7f061419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0614110ea0 con 0x7f0614100a80 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0614110d20 con 0x7f0614106a80 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061c2a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111020 con 0x7f0614101420 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061a03c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061a03c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54864/0 (socket says 172.21.15.77:54864) 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.038+0000 7f061a03c640 1 -- 172.21.15.77:0/3159798916 learned_addr learned my addr 172.21.15.77:0/3159798916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 -- 172.21.15.77:0/3159798916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0614101420 msgr2=0x7f0614197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0614101420 0x7f0614197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 -- 172.21.15.77:0/3159798916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0614106a80 msgr2=0x7f061419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0614106a80 0x7f061419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 -- 172.21.15.77:0/3159798916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061419c290 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061a03c640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f061000b4a0 tx=0x7f061000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f06077fe640 1 -- 172.21.15.77:0/3159798916 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610014070 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061419c580 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f06077fe640 1 -- 172.21.15.77:0/3159798916 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0610010af0 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f06077fe640 1 -- 172.21.15.77:0/3159798916 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610017d10 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.039+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141a3e60 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.041+0000 7f06077fe640 1 -- 172.21.15.77:0/3159798916 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0610017070 con 0x7f0614100a80 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 Processor -- start 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 -- start start 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb229a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb229fd0 0x559dfb234350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559dfb18d2e0 con 0x559dfb234890 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559dfb134ce0 con 0x559dfb2296b0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061e679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559dfb1892a0 con 0x559dfb229fd0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f06077fe640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061983b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061a83d640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061983b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54870/0 (socket says 172.21.15.77:54870) 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061983b640 1 -- 172.21.15.77:0/1729696098 learned_addr learned my addr 172.21.15.77:0/1729696098 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f06077fe640 1 -- 172.21.15.77:0/3159798916 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f06100cf440 con 0x7f0614100a80 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061a03c640 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb229fd0 0x559dfb234350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.042+0000 7f061a83d640 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb229a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 -- 172.21.15.77:0/1729696098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb229fd0 msgr2=0x559dfb234350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb229fd0 0x559dfb234350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 -- 172.21.15.77:0/1729696098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 msgr2=0x559dfb229a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb229a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 -- 172.21.15.77:0/1729696098 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfb119440 con 0x559dfb234890 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061983b640 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f060800aa40 tx=0x7f060800af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7c6a09d23a21b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f061a83d640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f060c009430 tx=0x7f060c031670 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f0604dea640 1 -- 172.21.15.77:0/1729696098 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060800ce40 con 0x559dfb234890 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f0604dea640 1 -- 172.21.15.77:0/1729696098 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06080035f0 con 0x559dfb234890 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.043+0000 7f0604dea640 1 -- 172.21.15.77:0/1729696098 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060800f690 con 0x559dfb234890 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- 172.21.15.77:0/1729696098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 msgr2=0x559dfb236c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f060800aa40 tx=0x7f060800af00 comp rx=0 tx=0).stop 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- 172.21.15.77:0/1729696098 shutdown_connections 2024-09-15T22:15:18.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb236c80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb229fd0 0x559dfb234350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- 172.21.15.77:0/1729696098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb229a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- 172.21.15.77:0/1729696098 >> 172.21.15.77:0/1729696098 conn(0x559dfb225c40 msgr2=0x559dfb226020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- 172.21.15.77:0/1729696098 shutdown_connections 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- 172.21.15.77:0/1729696098 wait complete. 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- start start 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 Processor -- start 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb231390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559dfb237460 con 0x559dfb234890 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559dfb148050 con 0x559dfb229fd0 2024-09-15T22:15:18.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559dfb2375e0 con 0x559dfb2296b0 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.044+0000 7f061e679740 10 client.0 _ll_register_callbacks cb 0x559dfb188160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54874/0 (socket says 172.21.15.77:54874) 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 -- 172.21.15.77:0/2399003870 learned_addr learned my addr 172.21.15.77:0/2399003870 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a03c640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061983b640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb231390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 -- 172.21.15.77:0/2399003870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 msgr2=0x559dfb2340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2340e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 -- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 msgr2=0x559dfb231390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb231390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 -- 172.21.15.77:0/2399003870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfaefe3c0 con 0x559dfb234890 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a03c640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:18.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061a83d640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3c6e0ac9cb36b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f05fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c04f070 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f060c049830 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06080035f0 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f060c049a30 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061e679740 10 client.16329 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dfb242d30 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.045+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06080035f0 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f060c030070 con 0x559dfb234890 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 10 client.16329 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 20 client.16329 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:18.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb244200 0x559dfb2466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 4 client.16329 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dfb19a850 con 0x559dfb244200 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061a03c640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb244200 0x559dfb2466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f05fdffb640 10 client.16329 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.046+0000 7f061a03c640 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb244200 0x559dfb2466c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed67123ade233bb5 server_cookie=bc4270216a985c7b in_seq=0 out_seq=0 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f05fdffb640 1 -- 172.21.15.77:0/2399003870 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559dfb19a850 con 0x559dfb244200 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f05fdffb640 10 client.16329 handle_command_reply: tid=0 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 client.16329 shutdown 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 msgr2=0x559dfb2318d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c042000 comp rx=0 tx=0).stop 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 shutdown_connections 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb244200 0x559dfb2466c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb2318d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb231390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 --2- 172.21.15.77:0/2399003870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 >> 172.21.15.77:0/2399003870 conn(0x559dfb225c40 msgr2=0x559dfb222b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f05fdffb640 0 client.16329 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 shutdown_connections 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.047+0000 7f061e679740 1 -- 172.21.15.77:0/2399003870 wait complete. 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.048+0000 7f061e679740 20 client.16329 trim_cache size 0 max 16384 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 Processor -- start 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 -- start start 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb22cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06141accb0 con 0x559dfb234890 2024-09-15T22:15:18.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ae8b0 con 0x559dfb229fd0 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061e679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ad380 con 0x559dfb2296b0 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.064+0000 7f061a83d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54888/0 (socket says 172.21.15.77:54888) 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 -- 172.21.15.77:0/1727439661 learned_addr learned my addr 172.21.15.77:0/1727439661 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a03c640 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061983b640 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb22cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 -- 172.21.15.77:0/1727439661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 msgr2=0x559dfb2406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 -- 172.21.15.77:0/1727439661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 msgr2=0x559dfb22cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb22cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 -- 172.21.15.77:0/1727439661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfb225a70 con 0x559dfb234890 2024-09-15T22:15:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f061a83d640 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f060c009cd0 tx=0x7f060c03bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7d03fb0b2f54a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f05fffff640 1 -- 172.21.15.77:0/1727439661 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c04f040 con 0x559dfb234890 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f05fffff640 1 -- 172.21.15.77:0/1727439661 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c036050 con 0x559dfb234890 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.065+0000 7f05fffff640 1 -- 172.21.15.77:0/1727439661 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c049cc0 con 0x559dfb234890 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- 172.21.15.77:0/1727439661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 msgr2=0x559dfb230370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f060c009cd0 tx=0x7f060c03bf90 comp rx=0 tx=0).stop 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- 172.21.15.77:0/1727439661 shutdown_connections 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb234890 0x559dfb230370 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb229fd0 0x559dfb22cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 --2- 172.21.15.77:0/1727439661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb2296b0 0x559dfb2406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- 172.21.15.77:0/1727439661 >> 172.21.15.77:0/1727439661 conn(0x559dfb225c40 msgr2=0x559dfb22fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- 172.21.15.77:0/1727439661 shutdown_connections 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- 172.21.15.77:0/1727439661 wait complete. 2024-09-15T22:15:18.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 -- start start 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.066+0000 7f061e679740 1 Processor -- start 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb23cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb234890 0x559dfb22e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ae120 con 0x559dfb229fd0 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ada50 con 0x559dfb2296b0 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06141aac90 con 0x559dfb234890 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 10 client.0 _ll_register_callbacks cb 0x7f060c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54904/0 (socket says 172.21.15.77:54904) 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 -- 172.21.15.77:0/190593565 learned_addr learned my addr 172.21.15.77:0/190593565 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061a03c640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb23cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 -- 172.21.15.77:0/190593565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb234890 msgr2=0x559dfb22e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb234890 0x559dfb22e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 -- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 msgr2=0x559dfb23cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb23cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 -- 172.21.15.77:0/190593565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfb22e770 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061983b640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f060800aa40 tx=0x7f060800f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fd1773082af640 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f05fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060800fa90 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f060800fc30 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060c036050 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f060800fe30 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 10 client.16335 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.067+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559dfb23c490 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c036050 con 0x559dfb229fd0 2024-09-15T22:15:18.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0608018630 con 0x559dfb229fd0 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 10 client.16335 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 20 client.16335 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb2af320 0x559dfb2b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 4 client.16335 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559dfb194d60 con 0x559dfb2af320 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061a03c640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb2af320 0x559dfb2b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f05fdffb640 10 client.16335 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.068+0000 7f061a03c640 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb2af320 0x559dfb2b1740 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8f71f892cbda8d server_cookie=a961ac6f717ba1aa in_seq=0 out_seq=0 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f05fdffb640 1 -- 172.21.15.77:0/190593565 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559dfb194d60 con 0x559dfb2af320 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f05fdffb640 10 client.16335 handle_command_reply: tid=0 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 client.16335 shutdown 2024-09-15T22:15:18.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 msgr2=0x559dfb22dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f060800aa40 tx=0x7f060800f450 comp rx=0 tx=0).stop 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 shutdown_connections 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:15:18.093 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559dfb2af320 0x559dfb2b1740 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559dfb234890 0x559dfb22e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f05fdffb640 0 client.16335 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559dfb229fd0 0x559dfb22dcf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 --2- 172.21.15.77:0/190593565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559dfb2296b0 0x559dfb23cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 >> 172.21.15.77:0/190593565 conn(0x559dfb225c40 msgr2=0x559dfb230480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 shutdown_connections 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 1 -- 172.21.15.77:0/190593565 wait complete. 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061e679740 20 client.16335 trim_cache size 0 max 16384 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 msgr2=0x7f06000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f060c009430 tx=0x7f060c031670 comp rx=0 tx=0).stop 2024-09-15T22:15:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 msgr2=0x7f06141972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.069+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f061000b4a0 tx=0x7f061000b960 comp rx=0 tx=0).stop 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 shutdown_connections 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0614106a80 0x7f061419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0614101420 0x7f0614197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 --2- 172.21.15.77:0/3159798916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614100a80 0x7f06141972e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 >> 172.21.15.77:0/3159798916 conn(0x7f06140fc7a0 msgr2=0x7f06140fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 shutdown_connections 2024-09-15T22:15:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:18.070+0000 7f061c2a0640 1 -- 172.21.15.77:0/3159798916 wait complete. 2024-09-15T22:15:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:18 smithi077 ceph-mon[30980]: pgmap v89: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 686 KiB/s wr, 100 op/s 2024-09-15T22:15:18.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:18 smithi196 ceph-mon[35109]: pgmap v89: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 686 KiB/s wr, 100 op/s 2024-09-15T22:15:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:18 smithi117 ceph-mon[35021]: pgmap v89: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 686 KiB/s wr, 100 op/s 2024-09-15T22:15:18.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.21 MB/sec warmup 87 sec latency 6957.658 ms 2024-09-15T22:15:18.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:15:18.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:15:19.174 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:19.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 10.10 MB/sec warmup 88 sec latency 7957.713 ms 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 -- 172.21.15.77:0/3651921709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0100a80 msgr2=0x7f6db0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/3651921709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0100a80 0x7f6db0100ee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009930 tx=0x7f6d9802a420 comp rx=0 tx=0).stop 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 -- 172.21.15.77:0/3651921709 shutdown_connections 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/3651921709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0101420 0x7f6db010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/3651921709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0100a80 0x7f6db0100ee0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/3651921709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 -- 172.21.15.77:0/3651921709 >> 172.21.15.77:0/3651921709 conn(0x7f6db00fc7a0 msgr2=0x7f6db00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 -- 172.21.15.77:0/3651921709 shutdown_connections 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.656+0000 7f6db4d2a640 1 -- 172.21.15.77:0/3651921709 wait complete. 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 Processor -- start 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 -- start start 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0100a80 0x7f6db00fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db00fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0110ea0 con 0x7f6db0101420 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0110d20 con 0x7f6db0106a80 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6db4d2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0111020 con 0x7f6db0100a80 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56626/0 (socket says 172.21.15.77:56626) 2024-09-15T22:15:19.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6da7fff640 1 -- 172.21.15.77:0/2252217996 learned_addr learned my addr 172.21.15.77:0/2252217996 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.657+0000 7f6dae59c640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0100a80 0x7f6db00fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 -- 172.21.15.77:0/2252217996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0100a80 msgr2=0x7f6db00fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0100a80 0x7f6db00fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 -- 172.21.15.77:0/2252217996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 msgr2=0x7f6db00fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6daed9d640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db00fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db00fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 -- 172.21.15.77:0/2252217996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db01a6930 con 0x7f6db0101420 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6daed9d640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db00fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da7fff640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009230 tx=0x7f6d98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9803f030 con 0x7f6db0101420 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d98003ad0 con 0x7f6db0101420 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db01a6bc0 con 0x7f6db0101420 2024-09-15T22:15:19.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98035070 con 0x7f6db0101420 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.658+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db01a70f0 con 0x7f6db0101420 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.659+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0100ee0 con 0x7f6db0101420 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.660+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6d98042030 con 0x7f6db0101420 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.661+0000 7f6da77fe640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 0x7f6d9c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.661+0000 7f6dae59c640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 0x7f6d9c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.661+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6d980f0550 con 0x7f6db0101420 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.661+0000 7f6dae59c640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 0x7f6d9c0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005fd0 tx=0x7f6da0005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:19.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.665+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d980bcbf0 con 0x7f6db0101420 2024-09-15T22:15:19.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.867+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6db002d7c0 con 0x7f6db0101420 2024-09-15T22:15:19.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.867+0000 7f6da77fe640 1 -- 172.21.15.77:0/2252217996 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f6d980bc4f0 con 0x7f6db0101420 2024-09-15T22:15:19.870 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:15:19.870 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:15:19.871 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:15:19.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.869+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 msgr2=0x7f6d9c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:19.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.869+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 0x7f6d9c0a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6da0005fd0 tx=0x7f6da0005850 comp rx=0 tx=0).stop 2024-09-15T22:15:19.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.869+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 msgr2=0x7f6db00fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.869+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009230 tx=0x7f6d98004270 comp rx=0 tx=0).stop 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 shutdown_connections 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d9c0a3430 0x7f6d9c0a58f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0106a80 0x7f6db00fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db0101420 0x7f6db00fd340 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 --2- 172.21.15.77:0/2252217996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db0100a80 0x7f6db00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 >> 172.21.15.77:0/2252217996 conn(0x7f6db00fc7a0 msgr2=0x7f6db006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 shutdown_connections 2024-09-15T22:15:19.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:19.870+0000 7f6db4d2a640 1 -- 172.21.15.77:0/2252217996 wait complete. 2024-09-15T22:15:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:20 smithi077 ceph-mon[30980]: pgmap v90: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 685 KiB/s wr, 95 op/s 2024-09-15T22:15:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:20 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2252217996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:20 smithi196 ceph-mon[35109]: pgmap v90: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 685 KiB/s wr, 95 op/s 2024-09-15T22:15:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:20 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2252217996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:20 smithi117 ceph-mon[35021]: pgmap v90: 129 pgs: 129 active+clean; 92 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 685 KiB/s wr, 95 op/s 2024-09-15T22:15:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:20 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2252217996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:20.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.98 MB/sec warmup 89 sec latency 8957.749 ms 2024-09-15T22:15:20.771 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:15:21.015 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:21.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.87 MB/sec warmup 90 sec latency 9957.791 ms 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 -- 172.21.15.77:0/384124933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda7810c170 msgr2=0x7fda7810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 --2- 172.21.15.77:0/384124933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda7810c170 0x7fda7810e610 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fda70009ca0 tx=0x7fda7002a950 comp rx=0 tx=0).stop 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 -- 172.21.15.77:0/384124933 shutdown_connections 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 --2- 172.21.15.77:0/384124933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda7810c170 0x7fda7810e610 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 --2- 172.21.15.77:0/384124933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda78103660 0x7fda7810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 --2- 172.21.15.77:0/384124933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78102d40 0x7fda78103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 -- 172.21.15.77:0/384124933 >> 172.21.15.77:0/384124933 conn(0x7fda780fc7a0 msgr2=0x7fda780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 -- 172.21.15.77:0/384124933 shutdown_connections 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.510+0000 7fda7e7be640 1 -- 172.21.15.77:0/384124933 wait complete. 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.511+0000 7fda7e7be640 1 Processor -- start 2024-09-15T22:15:21.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.511+0000 7fda7e7be640 1 -- start start 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78103660 0x7fda78198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 0x7fda7819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda78112030 con 0x7fda78102d40 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda78111eb0 con 0x7fda7810c170 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda781121b0 con 0x7fda78103660 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56646/0 (socket says 172.21.15.77:56646) 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 -- 172.21.15.77:0/4213712214 learned_addr learned my addr 172.21.15.77:0/4213712214 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 0x7fda7819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda6ffff640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78103660 0x7fda78198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 -- 172.21.15.77:0/4213712214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78103660 msgr2=0x7fda78198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78103660 0x7fda78198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 -- 172.21.15.77:0/4213712214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 msgr2=0x7fda7819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 0x7fda7819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 -- 172.21.15.77:0/4213712214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7819d4e0 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 0x7fda7819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda77fff640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b4a0 tx=0x7fda6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda75ffb640 1 -- 172.21.15.77:0/4213712214 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68014070 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda75ffb640 1 -- 172.21.15.77:0/4213712214 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda68010af0 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7819d7d0 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda75ffb640 1 -- 172.21.15.77:0/4213712214 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68017d10 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.512+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda781a50b0 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.514+0000 7fda75ffb640 1 -- 172.21.15.77:0/4213712214 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fda68017070 con 0x7fda78102d40 2024-09-15T22:15:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 Processor -- start 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 -- start start 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49eb40 0x55827f49d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f4ac780 0x55827f4aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55827f4032f0 con 0x55827f49d5d0 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55827f3aacf0 con 0x55827f4ac780 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda80b97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55827f3ff2b0 con 0x55827f49eb40 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda75ffb640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 0x7fda640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56648/0 (socket says 172.21.15.77:56648) 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda77fff640 1 -- 172.21.15.77:0/4205498005 learned_addr learned my addr 172.21.15.77:0/4205498005 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 0x7fda640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49eb40 0x55827f49d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.515+0000 7fda6ffff640 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f4ac780 0x55827f4aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda75ffb640 1 -- 172.21.15.77:0/4213712214 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fda680cf4a0 con 0x7fda78102d40 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 -- 172.21.15.77:0/4205498005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49eb40 msgr2=0x55827f49d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49eb40 0x55827f49d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 -- 172.21.15.77:0/4205498005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f4ac780 msgr2=0x55827f4aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f4ac780 0x55827f4aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 -- 172.21.15.77:0/4205498005 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827f38f3a0 con 0x55827f49d5d0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda77fff640 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda6809b480 tx=0x7fda680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3693d5376d0ddfb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 0x7fda640a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda70005fc0 tx=0x7fda70005c30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda6e7fc640 1 -- 172.21.15.77:0/4205498005 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6801e070 con 0x55827f49d5d0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda6e7fc640 1 -- 172.21.15.77:0/4205498005 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6809bbf0 con 0x55827f49d5d0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.516+0000 7fda6e7fc640 1 -- 172.21.15.77:0/4205498005 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda680ccad0 con 0x55827f49d5d0 2024-09-15T22:15:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- 172.21.15.77:0/4205498005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 msgr2=0x55827f49da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda6809b480 tx=0x7fda680046c0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- 172.21.15.77:0/4205498005 shutdown_connections 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f4ac780 0x55827f4aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49d5d0 0x55827f49da10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 --2- 172.21.15.77:0/4205498005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49eb40 0x55827f49d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- 172.21.15.77:0/4205498005 >> 172.21.15.77:0/4205498005 conn(0x55827f466310 msgr2=0x55827f499c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- 172.21.15.77:0/4205498005 shutdown_connections 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- 172.21.15.77:0/4205498005 wait complete. 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 -- start start 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.517+0000 7fda80b97740 1 Processor -- start 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f4a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55827f4af480 con 0x55827f49eb40 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55827f3be060 con 0x55827f49d5d0 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55827f4af600 con 0x55827f4ac780 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 10 client.0 _ll_register_callbacks cb 0x55827f1118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56660/0 (socket says 172.21.15.77:56660) 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda6ffff640 1 -- 172.21.15.77:0/3703366968 learned_addr learned my addr 172.21.15.77:0/3703366968 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f4a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 -- 172.21.15.77:0/3703366968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 msgr2=0x55827f4a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f4a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 -- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 msgr2=0x55827f4a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 -- 172.21.15.77:0/3703366968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827f1743c0 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda6ffff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda77fff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4ac1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda680a4320 tx=0x7fda68017590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d2a1d74db465ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda577fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6801e040 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6809bbf0 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.518+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fda68097b60 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda68010ca0 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 10 client.25712 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55827f4baf50 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6809bbf0 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fda680972d0 con 0x55827f49d5d0 2024-09-15T22:15:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 10 client.25712 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 10 client.25712 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 20 client.25712 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f4bc2a0 0x55827f4be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 4 client.25712 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55827f410860 con 0x55827f4bc2a0 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda6ffff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f4bc2a0 0x55827f4be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda577fe640 10 client.25712 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.519+0000 7fda6ffff640 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f4bc2a0 0x55827f4be690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1188262bbd19fd server_cookie=b2abba0285d5d0af in_seq=0 out_seq=0 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda577fe640 1 -- 172.21.15.77:0/3703366968 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55827f410860 con 0x55827f4bc2a0 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda577fe640 10 client.25712 handle_command_reply: tid=0 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 client.25712 shutdown 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 msgr2=0x55827f4ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4ac1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda680a4320 tx=0x7fda68017590 comp rx=0 tx=0).stop 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 shutdown_connections 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f4bc2a0 0x55827f4be690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f4a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda577fe640 0 client.25712 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f4a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 --2- 172.21.15.77:0/3703366968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4ac1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 >> 172.21.15.77:0/3703366968 conn(0x55827f466310 msgr2=0x55827f49f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 shutdown_connections 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 1 -- 172.21.15.77:0/3703366968 wait complete. 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.520+0000 7fda80b97740 20 client.25712 trim_cache size 0 max 16384 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 Processor -- start 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 -- start start 2024-09-15T22:15:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f498bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda781ace00 con 0x55827f49eb40 2024-09-15T22:15:21.584 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:15:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda781aea00 con 0x55827f49d5d0 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda80b97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda781ad4d0 con 0x55827f4ac780 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56672/0 (socket says 172.21.15.77:56672) 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 -- 172.21.15.77:0/2136358483 learned_addr learned my addr 172.21.15.77:0/2136358483 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda77fff640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f498bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 -- 172.21.15.77:0/2136358483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 msgr2=0x55827f498bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f498bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 -- 172.21.15.77:0/2136358483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 msgr2=0x55827f4b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 -- 172.21.15.77:0/2136358483 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55827f4a9d90 con 0x55827f49eb40 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f498bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6ffff640 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fda60006a70 tx=0x7fda60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a228662b8c1eb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6dffb640 1 -- 172.21.15.77:0/2136358483 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60014d10 con 0x55827f49eb40 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6dffb640 1 -- 172.21.15.77:0/2136358483 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6001b7b0 con 0x55827f49eb40 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.534+0000 7fda6dffb640 1 -- 172.21.15.77:0/2136358483 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60013720 con 0x55827f49eb40 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- 172.21.15.77:0/2136358483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 msgr2=0x55827f49e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fda60006a70 tx=0x7fda60008040 comp rx=0 tx=0).stop 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- 172.21.15.77:0/2136358483 shutdown_connections 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f4ac780 0x55827f498bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f49eb40 0x55827f49e410 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 --2- 172.21.15.77:0/2136358483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49d5d0 0x55827f4b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- 172.21.15.77:0/2136358483 >> 172.21.15.77:0/2136358483 conn(0x55827f466310 msgr2=0x55827f4984e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- 172.21.15.77:0/2136358483 shutdown_connections 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- 172.21.15.77:0/2136358483 wait complete. 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 -- start start 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.535+0000 7fda80b97740 1 Processor -- start 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49d5d0 0x55827f4c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 0x55827f4bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda781ae270 con 0x55827f4ac780 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda781adba0 con 0x55827f49eb40 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda781aade0 con 0x55827f49d5d0 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 10 client.0 _ll_register_callbacks cb 0x7fda7806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56688/0 (socket says 172.21.15.77:56688) 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 0x55827f4bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 -- 172.21.15.77:0/773301936 learned_addr learned my addr 172.21.15.77:0/773301936 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda77fff640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49d5d0 0x55827f4c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 -- 172.21.15.77:0/773301936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49d5d0 msgr2=0x55827f4c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49d5d0 0x55827f4c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 -- 172.21.15.77:0/773301936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 msgr2=0x55827f4bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 0x55827f4bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 -- 172.21.15.77:0/773301936 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827f4bfa20 con 0x55827f4ac780 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda6ffff640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 0x55827f4bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda7cd5b640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fda70035000 tx=0x7fda700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12f0fe5acc0c1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda577fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70040030 con 0x55827f4ac780 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fda70004510 con 0x55827f4ac780 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.536+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6001b7b0 con 0x55827f4ac780 2024-09-15T22:15:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda70043040 con 0x55827f4ac780 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 10 client.16359 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55827f526920 con 0x55827f4ac780 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6001b7b0 con 0x55827f4ac780 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fda70052070 con 0x55827f4ac780 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 10 client.16359 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 10 client.16359 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 20 client.16359 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f527040 0x55827f529460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 4 client.16359 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55827f40ad70 con 0x55827f527040 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda77fff640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f527040 0x55827f529460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda77fff640 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f527040 0x55827f529460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff57442d7720d5b4 server_cookie=8633bbcdc410c71a in_seq=0 out_seq=0 2024-09-15T22:15:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 10 client.16359 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 1 -- 172.21.15.77:0/773301936 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55827f40ad70 con 0x55827f527040 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.537+0000 7fda577fe640 10 client.16359 handle_command_reply: tid=0 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 client.16359 shutdown 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 msgr2=0x55827f4bf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fda70035000 tx=0x7fda700095c0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 shutdown_connections 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55827f527040 0x55827f529460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55827f4ac780 0x55827f4bf460 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55827f49eb40 0x55827f4bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda577fe640 0 client.16359 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 --2- 172.21.15.77:0/773301936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55827f49d5d0 0x55827f4c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 >> 172.21.15.77:0/773301936 conn(0x55827f466310 msgr2=0x55827f49f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 shutdown_connections 2024-09-15T22:15:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 1 -- 172.21.15.77:0/773301936 wait complete. 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda80b97740 20 client.16359 trim_cache size 0 max 16384 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 msgr2=0x7fda640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 0x7fda640a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fda70005fc0 tx=0x7fda70005c30 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 msgr2=0x7fda78198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b4a0 tx=0x7fda6800b960 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 shutdown_connections 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda640a33b0 0x7fda640a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda7810c170 0x7fda7819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda78103660 0x7fda78198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 --2- 172.21.15.77:0/4213712214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda78102d40 0x7fda78198530 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.538+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 >> 172.21.15.77:0/4213712214 conn(0x7fda780fc7a0 msgr2=0x7fda780fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.539+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 shutdown_connections 2024-09-15T22:15:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:21.539+0000 7fda7e7be640 1 -- 172.21.15.77:0/4213712214 wait complete. 2024-09-15T22:15:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:22 smithi077 ceph-mon[30980]: pgmap v91: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 779 KiB/s wr, 103 op/s 2024-09-15T22:15:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:22 smithi196 ceph-mon[35109]: pgmap v91: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 779 KiB/s wr, 103 op/s 2024-09-15T22:15:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:22 smithi117 ceph-mon[35021]: pgmap v91: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 779 KiB/s wr, 103 op/s 2024-09-15T22:15:22.426 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:15:22.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.76 MB/sec warmup 91 sec latency 10957.820 ms 2024-09-15T22:15:22.671 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 -- 172.21.15.77:0/2607855276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208101420 msgr2=0x7f920810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 --2- 172.21.15.77:0/2607855276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208101420 0x7f920810d950 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9200009cd0 tx=0x7f920002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 -- 172.21.15.77:0/2607855276 shutdown_connections 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 --2- 172.21.15.77:0/2607855276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208101420 0x7f920810d950 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 --2- 172.21.15.77:0/2607855276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208100a80 0x7f9208100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 --2- 172.21.15.77:0/2607855276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 0x7f9208106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 -- 172.21.15.77:0/2607855276 >> 172.21.15.77:0/2607855276 conn(0x7f92080fc7a0 msgr2=0x7f92080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 -- 172.21.15.77:0/2607855276 shutdown_connections 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.139+0000 7f920cf72640 1 -- 172.21.15.77:0/2607855276 wait complete. 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 Processor -- start 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 -- start start 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 0x7f9208199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 0x7f92081a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9208110e80 con 0x7f9208100a80 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9208110d00 con 0x7f9208101420 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920cf72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9208111000 con 0x7f9208106a80 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56710/0 (socket says 172.21.15.77:56710) 2024-09-15T22:15:23.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f920659c640 1 -- 172.21.15.77:0/3706379554 learned_addr learned my addr 172.21.15.77:0/3706379554 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f9206d9d640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 0x7f92081a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.140+0000 7f9205d9b640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 0x7f9208199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 -- 172.21.15.77:0/3706379554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 msgr2=0x7f92081a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 0x7f92081a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 -- 172.21.15.77:0/3706379554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 msgr2=0x7f9208199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 0x7f9208199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 -- 172.21.15.77:0/3706379554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92081a47c0 con 0x7f9208100a80 2024-09-15T22:15:23.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f9205d9b640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 0x7f9208199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920659c640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f91f800d770 tx=0x7f91f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f91ff7fe640 1 -- 172.21.15.77:0/3706379554 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8017070 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92081a4990 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f91ff7fe640 1 -- 172.21.15.77:0/3706379554 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8005500 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f91ff7fe640 1 -- 172.21.15.77:0/3706379554 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8004e00 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.141+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a4ea0 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 Processor -- start 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 -- start start 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f91ff7fe640 1 -- 172.21.15.77:0/3706379554 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f91f8004fa0 con 0x7f9208100a80 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 0x5644c0f99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0fa4370 0x5644c0fa6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5644c0eff2f0 con 0x5644c0f995d0 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5644c0ea6cf0 con 0x5644c0f9ab40 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920f34b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644c0efb2b0 con 0x5644c0fa4370 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.144+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56720/0 (socket says 172.21.15.77:56720) 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 -- 172.21.15.77:0/2326171039 learned_addr learned my addr 172.21.15.77:0/2326171039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f9206d9d640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 0x5644c0f99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f9205d9b640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0fa4370 0x5644c0fa6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 -- 172.21.15.77:0/2326171039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0fa4370 msgr2=0x5644c0fa6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0fa4370 0x5644c0fa6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 -- 172.21.15.77:0/2326171039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 msgr2=0x5644c0f99090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 0x5644c0f99090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f91ff7fe640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 0x7f91f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 -- 172.21.15.77:0/2326171039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c0e8b3a0 con 0x5644c0f995d0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f9206d9d640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 0x5644c0f99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f9205d9b640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 0x7f91f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f91ff7fe640 1 -- 172.21.15.77:0/3706379554 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f91f80cfa80 con 0x7f9208100a80 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f920659c640 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cce00 tx=0x7f91f80d85c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6d3f9c2924483c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f91fcff9640 1 -- 172.21.15.77:0/2326171039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8020070 con 0x5644c0f995d0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f91fcff9640 1 -- 172.21.15.77:0/2326171039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f80046c0 con 0x5644c0f995d0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.145+0000 7f91fcff9640 1 -- 172.21.15.77:0/2326171039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80d7040 con 0x5644c0f995d0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 -- 172.21.15.77:0/2326171039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 msgr2=0x5644c0f99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f9205d9b640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 0x7f91f40a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005fd0 tx=0x7f91f0005f40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cce00 tx=0x7f91f80d85c0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 -- 172.21.15.77:0/2326171039 shutdown_connections 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0fa4370 0x5644c0fa6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f99a10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 --2- 172.21.15.77:0/2326171039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0f9ab40 0x5644c0f99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 -- 172.21.15.77:0/2326171039 >> 172.21.15.77:0/2326171039 conn(0x5644c0f62310 msgr2=0x5644c0f95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 -- 172.21.15.77:0/2326171039 shutdown_connections 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.146+0000 7f920f34b740 1 -- 172.21.15.77:0/2326171039 wait complete. 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 -- start start 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 Processor -- start 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f995d0 0x5644c0fa3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5644c0fa7010 con 0x5644c0f9ab40 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5644c0eba060 con 0x5644c0fa4370 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644c0fa7190 con 0x5644c0f995d0 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f920f34b740 10 client.0 _ll_register_callbacks cb 0x5644c0c0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56722/0 (socket says 172.21.15.77:56722) 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 -- 172.21.15.77:0/1821703895 learned_addr learned my addr 172.21.15.77:0/1821703895 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 -- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f995d0 msgr2=0x5644c0fa3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f995d0 0x5644c0fa3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 -- 172.21.15.77:0/1821703895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 msgr2=0x5644c0fa1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.147+0000 7f9205d9b640 1 -- 172.21.15.77:0/1821703895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c0c703c0 con 0x5644c0f9ab40 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f9205d9b640 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f91f00117c0 tx=0x7f91f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7188e3f3de397d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0022070 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f91f0010ae0 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644c0f0c860 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91f002d050 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 10 client.16371 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644c0fb2ab0 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644c0f0c860 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f91f001d3b0 con 0x5644c0f9ab40 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 10 client.16371 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 10 client.16371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 20 client.16371 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c0fb3f10 0x5644c0fb63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 4 client.16371 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.148+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c0f06d70 con 0x5644c0fb3f10 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.149+0000 7f920659c640 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c0fb3f10 0x5644c0fb63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.149+0000 7f91edffb640 10 client.16371 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.149+0000 7f920659c640 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c0fb3f10 0x5644c0fb63b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b33806bd79a7b04d server_cookie=3118beac5aba1906 in_seq=0 out_seq=0 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f91edffb640 1 -- 172.21.15.77:0/1821703895 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644c0f06d70 con 0x5644c0fb3f10 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f91edffb640 10 client.16371 handle_command_reply: tid=0 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 client.16371 shutdown 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 msgr2=0x5644c0fa1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f91f00117c0 tx=0x7f91f0004730 comp rx=0 tx=0).stop 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 shutdown_connections 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c0fb3f10 0x5644c0fb63b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f9ab40 0x5644c0fa1020 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 --2- 172.21.15.77:0/1821703895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f995d0 0x5644c0fa3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 >> 172.21.15.77:0/1821703895 conn(0x5644c0f62310 msgr2=0x5644c0f9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f91edffb640 0 client.16371 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 shutdown_connections 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 1 -- 172.21.15.77:0/1821703895 wait complete. 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.150+0000 7f920f34b740 20 client.16371 trim_cache size 0 max 16384 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 Processor -- start 2024-09-15T22:15:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 -- start start 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fb02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92081acc80 con 0x5644c0f995d0 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ae880 con 0x5644c0fa4370 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920f34b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ad350 con 0x5644c0f9ab40 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56730/0 (socket says 172.21.15.77:56730) 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 -- 172.21.15.77:0/3800652333 learned_addr learned my addr 172.21.15.77:0/3800652333 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f9206d9d640 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 -- 172.21.15.77:0/3800652333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 msgr2=0x5644c0fb02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fb02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 -- 172.21.15.77:0/3800652333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 msgr2=0x5644c0fa08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 -- 172.21.15.77:0/3800652333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c0f9ffd0 con 0x5644c0f995d0 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f9206d9d640 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.163+0000 7f920659c640 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cd490 tx=0x7f91f8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69006e7892a94c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f91effff640 1 -- 172.21.15.77:0/3800652333 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b3070 con 0x5644c0f995d0 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f91effff640 1 -- 172.21.15.77:0/3800652333 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8015070 con 0x5644c0f995d0 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f91effff640 1 -- 172.21.15.77:0/3800652333 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8020040 con 0x5644c0f995d0 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 -- 172.21.15.77:0/3800652333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 msgr2=0x5644c0f95e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cd490 tx=0x7f91f8005bb0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 -- 172.21.15.77:0/3800652333 shutdown_connections 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c0fa08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fb02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 --2- 172.21.15.77:0/3800652333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0f95e10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 -- 172.21.15.77:0/3800652333 >> 172.21.15.77:0/3800652333 conn(0x5644c0f62310 msgr2=0x5644c0f95700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 -- 172.21.15.77:0/3800652333 shutdown_connections 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.164+0000 7f920f34b740 1 -- 172.21.15.77:0/3800652333 wait complete. 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 -- start start 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 Processor -- start 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fa80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c1016c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ae0f0 con 0x5644c0f995d0 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ada20 con 0x5644c0fa4370 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92081aac60 con 0x5644c0f9ab40 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920f34b740 10 client.0 _ll_register_callbacks cb 0x7f91f0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56734/0 (socket says 172.21.15.77:56734) 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 -- 172.21.15.77:0/2922625928 learned_addr learned my addr 172.21.15.77:0/2922625928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 -- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 msgr2=0x5644c0fa80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fa80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 -- 172.21.15.77:0/2922625928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 msgr2=0x5644c1016c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f9206d9d640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c1016c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c1016c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f920659c640 1 -- 172.21.15.77:0/2922625928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c10172b0 con 0x5644c0f995d0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.165+0000 7f9206d9d640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c1016c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f920659c640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f91f8099cd0 tx=0x7f91f80057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84ac5baa8cef559 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f91edffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b8020 con 0x5644c0f995d0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f91f8009cc0 con 0x5644c0f995d0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f8015070 con 0x5644c0f995d0 2024-09-15T22:15:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91f8020050 con 0x5644c0f995d0 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f920f34b740 10 client.16377 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.166+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644c101e5f0 con 0x5644c0f995d0 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f809c120 con 0x5644c0f995d0 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f91f80ba2c0 con 0x5644c0f995d0 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 10 client.16377 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 10 client.16377 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 20 client.16377 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c101e870 0x5644c1020c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 4 client.16377 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f91f8015070 con 0x5644c101e870 2024-09-15T22:15:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.167+0000 7f9205d9b640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c101e870 0x5644c1020c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:23.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f91edffb640 10 client.16377 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f9205d9b640 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c101e870 0x5644c1020c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f131efeac6386479 server_cookie=4135b5bed5884e0d in_seq=0 out_seq=0 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f91edffb640 1 -- 172.21.15.77:0/2922625928 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f91f8015070 con 0x5644c101e870 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f91edffb640 10 client.16377 handle_command_reply: tid=0 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 client.16377 shutdown 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 msgr2=0x5644c0fa7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f91f8099cd0 tx=0x7f91f80057a0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 shutdown_connections 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5644c101e870 0x5644c1020c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5644c0fa4370 0x5644c1016c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644c0f9ab40 0x5644c0fa80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 --2- 172.21.15.77:0/2922625928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5644c0f995d0 0x5644c0fa7bb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f91edffb640 0 client.16377 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:15:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 >> 172.21.15.77:0/2922625928 conn(0x5644c0f62310 msgr2=0x5644c0fa2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 shutdown_connections 2024-09-15T22:15:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.168+0000 7f920f34b740 1 -- 172.21.15.77:0/2922625928 wait complete. 2024-09-15T22:15:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920f34b740 20 client.16377 trim_cache size 0 max 16384 2024-09-15T22:15:23.187 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 msgr2=0x7f91f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 0x7f91f40a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005fd0 tx=0x7f91f0005f40 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 msgr2=0x7f9208199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f91f800d770 tx=0x7f91f800dc30 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 shutdown_connections 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91f40a33b0 0x7f91f40a5870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9208106a80 0x7f92081a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9208101420 0x7f9208199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 --2- 172.21.15.77:0/3706379554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9208100a80 0x7f9208199200 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:23.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 >> 172.21.15.77:0/3706379554 conn(0x7f92080fc7a0 msgr2=0x7f92080fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:23.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 shutdown_connections 2024-09-15T22:15:23.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:23.169+0000 7f920cf72640 1 -- 172.21.15.77:0/3706379554 wait complete. 2024-09-15T22:15:23.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.66 MB/sec warmup 92 sec latency 11957.884 ms 2024-09-15T22:15:23.887 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:15:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:24.296 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:24 smithi077 ceph-mon[30980]: pgmap v92: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:24.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:24 smithi196 ceph-mon[35109]: pgmap v92: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:24 smithi117 ceph-mon[35021]: pgmap v92: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:24.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.55 MB/sec warmup 93 sec latency 12957.909 ms 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- 172.21.15.77:0/503081059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844101de0 msgr2=0x7fa8441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 --2- 172.21.15.77:0/503081059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844101de0 0x7fa8441021c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa838009900 tx=0x7fa83802a410 comp rx=0 tx=0).stop 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- 172.21.15.77:0/503081059 shutdown_connections 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 --2- 172.21.15.77:0/503081059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844106f10 0x7fa84410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 --2- 172.21.15.77:0/503081059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa8441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 --2- 172.21.15.77:0/503081059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844101de0 0x7fa8441021c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- 172.21.15.77:0/503081059 >> 172.21.15.77:0/503081059 conn(0x7fa8440fd8c0 msgr2=0x7fa8440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- 172.21.15.77:0/503081059 shutdown_connections 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- 172.21.15.77:0/503081059 wait complete. 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 Processor -- start 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.656+0000 7fa84ac35640 1 -- start start 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844101de0 0x7fa8441985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa844198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112010 con 0x7fa844106f10 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111e90 con 0x7fa844102790 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112190 con 0x7fa844101de0 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56760/0 (socket says 172.21.15.77:56760) 2024-09-15T22:15:24.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 -- 172.21.15.77:0/154284740 learned_addr learned my addr 172.21.15.77:0/154284740 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8489d1640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844101de0 0x7fa8441985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa843fff640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa844198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 -- 172.21.15.77:0/154284740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844101de0 msgr2=0x7fa8441985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844101de0 0x7fa8441985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 -- 172.21.15.77:0/154284740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 msgr2=0x7fa844198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa844198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 -- 172.21.15.77:0/154284740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419d560 con 0x7fa844106f10 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa843fff640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa844198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa8491d2640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00b970 tx=0x7fa83c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa841ffb640 1 -- 172.21.15.77:0/154284740 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c019070 con 0x7fa844106f10 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419d850 con 0x7fa844106f10 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.657+0000 7fa841ffb640 1 -- 172.21.15.77:0/154284740 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa83c00cb40 con 0x7fa844106f10 2024-09-15T22:15:24.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.658+0000 7fa841ffb640 1 -- 172.21.15.77:0/154284740 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007650 con 0x7fa844106f10 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.658+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a5130 con 0x7fa844106f10 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.660+0000 7fa841ffb640 1 -- 172.21.15.77:0/154284740 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa83c01c070 con 0x7fa844106f10 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 Processor -- start 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 -- start start 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561be1f302f0 con 0x561be1fca5d0 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561be1ed7cf0 con 0x561be1fd5370 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa84d00e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561be1f2c2b0 con 0x561be1fcbb40 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa841ffb640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 0x7fa8340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa8489d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa8489d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56776/0 (socket says 172.21.15.77:56776) 2024-09-15T22:15:24.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa8489d1640 1 -- 172.21.15.77:0/4188983907 learned_addr learned my addr 172.21.15.77:0/4188983907 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:24.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa843fff640 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa8491d2640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 0x7fa8340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa843fff640 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.661+0000 7fa841ffb640 1 -- 172.21.15.77:0/154284740 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa83c0cf610 con 0x7fa844106f10 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 -- 172.21.15.77:0/4188983907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 msgr2=0x561be1fca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 -- 172.21.15.77:0/4188983907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 msgr2=0x561be1fd7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 -- 172.21.15.77:0/4188983907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be1ebc3a0 con 0x561be1fca5d0 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8491d2640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 0x7fa8340a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0cd020 tx=0x7fa83c06b2e0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa838005ab0 tx=0x7fa838031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc5ff4122de856 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa82effd640 1 -- 172.21.15.77:0/4188983907 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83803f070 con 0x561be1fca5d0 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa82effd640 1 -- 172.21.15.77:0/4188983907 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa838031a20 con 0x561be1fca5d0 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.662+0000 7fa82effd640 1 -- 172.21.15.77:0/4188983907 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838035070 con 0x561be1fca5d0 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- 172.21.15.77:0/4188983907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 msgr2=0x561be1fcaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa838005ab0 tx=0x7fa838031630 comp rx=0 tx=0).stop 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- 172.21.15.77:0/4188983907 shutdown_connections 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fcaa10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4188983907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- 172.21.15.77:0/4188983907 >> 172.21.15.77:0/4188983907 conn(0x561be1f93310 msgr2=0x561be1fc6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- 172.21.15.77:0/4188983907 shutdown_connections 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- 172.21.15.77:0/4188983907 wait complete. 2024-09-15T22:15:24.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 -- start start 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.663+0000 7fa84d00e740 1 Processor -- start 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fd1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561be1fd8010 con 0x561be1fca5d0 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561be1eeb060 con 0x561be1fd5370 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561be1fd8190 con 0x561be1fcbb40 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 10 client.0 _ll_register_callbacks cb 0x561be1c3e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56778/0 (socket says 172.21.15.77:56778) 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 -- 172.21.15.77:0/873780095 learned_addr learned my addr 172.21.15.77:0/873780095 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa8489d1640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fd1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 -- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 msgr2=0x561be1fd1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fd1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 -- 172.21.15.77:0/873780095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 msgr2=0x561be1fd2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 -- 172.21.15.77:0/873780095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be1ca13c0 con 0x561be1fca5d0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa843fff640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa830006aa0 tx=0x7fa8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d41679eecefb737 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa823fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83001f060 con 0x561be1fca5d0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa830014a30 con 0x561be1fca5d0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.664+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa838031a20 con 0x561be1fca5d0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa830026050 con 0x561be1fca5d0 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 10 client.16389 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:24.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561be1fe3b10 con 0x561be1fca5d0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa838031a20 con 0x561be1fca5d0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa830015040 con 0x561be1fca5d0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 10 client.16389 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 10 client.16389 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 20 client.16389 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be1fe4fd0 0x561be1fe73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 4 client.16389 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be1f3d860 con 0x561be1fe4fd0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.665+0000 7fa8489d1640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be1fe4fd0 0x561be1fe73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.666+0000 7fa823fff640 10 client.16389 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.666+0000 7fa8489d1640 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be1fe4fd0 0x561be1fe73c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14ea310a62cae3a4 server_cookie=3b5f74d0025ef453 in_seq=0 out_seq=0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa823fff640 1 -- 172.21.15.77:0/873780095 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561be1f3d860 con 0x561be1fe4fd0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa823fff640 10 client.16389 handle_command_reply: tid=0 2024-09-15T22:15:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 client.16389 shutdown 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 msgr2=0x561be1fd4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa830006aa0 tx=0x7fa8300046c0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 shutdown_connections 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be1fe4fd0 0x561be1fe73c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fcbb40 0x561be1fd1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa823fff640 0 client.16389 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 --2- 172.21.15.77:0/873780095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fd4d20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 >> 172.21.15.77:0/873780095 conn(0x561be1f93310 msgr2=0x561be1fcc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 shutdown_connections 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 1 -- 172.21.15.77:0/873780095 wait complete. 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.667+0000 7fa84d00e740 20 client.16389 trim_cache size 0 max 16384 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 Processor -- start 2024-09-15T22:15:24.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 -- start start 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fcbb40 0x561be1fd6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fd5370 0x561be1fd6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441aeeb0 con 0x561be1fca5d0 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441b0ab0 con 0x561be1fcbb40 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa84d00e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441af580 con 0x561be1fd5370 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56792/0 (socket says 172.21.15.77:56792) 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 -- 172.21.15.77:0/1125254896 learned_addr learned my addr 172.21.15.77:0/1125254896 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa8489d1640 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fcbb40 0x561be1fd6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 -- 172.21.15.77:0/1125254896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fd5370 msgr2=0x561be1fd6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fd5370 0x561be1fd6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 -- 172.21.15.77:0/1125254896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fcbb40 msgr2=0x561be1fd6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fcbb40 0x561be1fd6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.680+0000 7fa843fff640 1 -- 172.21.15.77:0/1125254896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be1fd7ba0 con 0x561be1fca5d0 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa843fff640 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b010 tx=0x7fa83000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb895ce1e0404311 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa82e7fc640 1 -- 172.21.15.77:0/1125254896 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83000ab20 con 0x561be1fca5d0 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa82e7fc640 1 -- 172.21.15.77:0/1125254896 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830032070 con 0x561be1fca5d0 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa82e7fc640 1 -- 172.21.15.77:0/1125254896 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83002d9e0 con 0x561be1fca5d0 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 -- 172.21.15.77:0/1125254896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 msgr2=0x561be1fc5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b010 tx=0x7fa83000a710 comp rx=0 tx=0).stop 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 -- 172.21.15.77:0/1125254896 shutdown_connections 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fd5370 0x561be1fd6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fcbb40 0x561be1fd6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.681+0000 7fa84d00e740 1 --2- 172.21.15.77:0/1125254896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fca5d0 0x561be1fc5bf0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- 172.21.15.77:0/1125254896 >> 172.21.15.77:0/1125254896 conn(0x561be1f93310 msgr2=0x561be1fc54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- 172.21.15.77:0/1125254896 shutdown_connections 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- 172.21.15.77:0/1125254896 wait complete. 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- start start 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 Processor -- start 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 0x561be1fdd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441b0320 con 0x561be1fcbb40 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441afc50 con 0x561be1fd5370 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441ace90 con 0x561be1fca5d0 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.682+0000 7fa84d00e740 10 client.0 _ll_register_callbacks cb 0x7fa83000a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56794/0 (socket says 172.21.15.77:56794) 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 -- 172.21.15.77:0/4292316050 learned_addr learned my addr 172.21.15.77:0/4292316050 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:24.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa843fff640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 -- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 msgr2=0x561be1fdd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa843fff640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 0x561be1fdd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 0x561be1fdd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 -- 172.21.15.77:0/4292316050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 msgr2=0x561be1fd83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 -- 172.21.15.77:0/4292316050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be1fd8920 con 0x561be1fcbb40 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa843fff640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 0x561be1fdd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa8489d1640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8380096c0 tx=0x7fa838005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3900806fbc09bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa823fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838052070 con 0x561be1fcbb40 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa838004220 con 0x561be1fcbb40 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa830032070 con 0x561be1fcbb40 2024-09-15T22:15:24.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa83805b050 con 0x561be1fcbb40 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa84d00e740 10 client.16395 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.683+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561be2050510 con 0x561be1fcbb40 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830032070 con 0x561be1fcbb40 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa838055020 con 0x561be1fcbb40 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 10 client.16395 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 10 client.16395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 20 client.16395 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be2050910 0x561be2052d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 4 client.16395 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561be1f37d70 con 0x561be2050910 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa843fff640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be2050910 0x561be2052d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa843fff640 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be2050910 0x561be2052d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f6d61322e5c68fe server_cookie=4aa3b6b759d7603 in_seq=0 out_seq=0 2024-09-15T22:15:24.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.684+0000 7fa823fff640 10 client.16395 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa823fff640 1 -- 172.21.15.77:0/4292316050 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561be1f37d70 con 0x561be2050910 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa823fff640 10 client.16395 handle_command_reply: tid=0 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 client.16395 shutdown 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 msgr2=0x561be1fdd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8380096c0 tx=0x7fa838005630 comp rx=0 tx=0).stop 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 shutdown_connections 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561be2050910 0x561be2052d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561be1fd5370 0x561be1fd83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa823fff640 0 client.16395 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561be1fcbb40 0x561be1fdd970 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 --2- 172.21.15.77:0/4292316050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561be1fca5d0 0x561be1fdd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 >> 172.21.15.77:0/4292316050 conn(0x561be1f93310 msgr2=0x561be1fc5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 shutdown_connections 2024-09-15T22:15:24.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 1 -- 172.21.15.77:0/4292316050 wait complete. 2024-09-15T22:15:24.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.685+0000 7fa84d00e740 20 client.16395 trim_cache size 0 max 16384 2024-09-15T22:15:24.699 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 msgr2=0x7fa8340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 0x7fa8340a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0cd020 tx=0x7fa83c06b2e0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 msgr2=0x7fa84419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00b970 tx=0x7fa83c00be30 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 shutdown_connections 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8340a33b0 0x7fa8340a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa844106f10 0x7fa84419ce80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102790 0x7fa844198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 --2- 172.21.15.77:0/154284740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844101de0 0x7fa8441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 >> 172.21.15.77:0/154284740 conn(0x7fa8440fd8c0 msgr2=0x7fa8440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 shutdown_connections 2024-09-15T22:15:24.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:24.686+0000 7fa84ac35640 1 -- 172.21.15.77:0/154284740 wait complete. 2024-09-15T22:15:25.376 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:15:25.462 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.45 MB/sec warmup 94 sec latency 13957.958 ms 2024-09-15T22:15:25.622 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 -- 172.21.15.77:0/946074952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 msgr2=0x7f17b810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 --2- 172.21.15.77:0/946074952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b810eb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009cd0 tx=0x7f17b002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 -- 172.21.15.77:0/946074952 shutdown_connections 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 --2- 172.21.15.77:0/946074952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b810eb00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 --2- 172.21.15.77:0/946074952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 0x7f17b8107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 --2- 172.21.15.77:0/946074952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b8100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 -- 172.21.15.77:0/946074952 >> 172.21.15.77:0/946074952 conn(0x7f17b8077210 msgr2=0x7f17b8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 -- 172.21.15.77:0/946074952 shutdown_connections 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.127+0000 7f17bd02c640 1 -- 172.21.15.77:0/946074952 wait complete. 2024-09-15T22:15:26.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 Processor -- start 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 -- start start 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 0x7f17b8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81120a0 con 0x7f17b8108160 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8111f20 con 0x7f17b8101290 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17bd02c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8112220 con 0x7f17b81008e0 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17b759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17b759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56810/0 (socket says 172.21.15.77:56810) 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17b759e640 1 -- 172.21.15.77:0/2943249562 learned_addr learned my addr 172.21.15.77:0/2943249562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.128+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 -- 172.21.15.77:0/2943249562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 msgr2=0x7f17b81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b659c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 0x7f17b8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 -- 172.21.15.77:0/2943249562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 msgr2=0x7f17b8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 0x7f17b8198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 -- 172.21.15.77:0/2943249562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b819d490 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17b759e640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b0033040 tx=0x7f17b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f179ffff640 1 -- 172.21.15.77:0/2943249562 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0004040 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b819d720 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f179ffff640 1 -- 172.21.15.77:0/2943249562 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b00046c0 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a5060 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.129+0000 7f179ffff640 1 -- 172.21.15.77:0/2943249562 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0046480 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.131+0000 7f179ffff640 1 -- 172.21.15.77:0/2943249562 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17b00305a0 con 0x7f17b8108160 2024-09-15T22:15:26.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.131+0000 7f17bf405740 1 Processor -- start 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.131+0000 7f17bf405740 1 -- start start 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd04322f0 con 0x55ffd04cdb40 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffd03d9cf0 con 0x55ffd04db780 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17bf405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffd042e2b0 con 0x55ffd04cc5d0 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56820/0 (socket says 172.21.15.77:56820) 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 -- 172.21.15.77:0/4121561637 learned_addr learned my addr 172.21.15.77:0/4121561637 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f179ffff640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 0x7f17a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b759e640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 0x7f17a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 -- 172.21.15.77:0/4121561637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 msgr2=0x55ffd04cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 -- 172.21.15.77:0/4121561637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 msgr2=0x55ffd04ddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04ddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 -- 172.21.15.77:0/4121561637 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd03be3a0 con 0x55ffd04cdb40 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f179ffff640 1 -- 172.21.15.77:0/2943249562 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f17b002e030 con 0x7f17b8108160 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b659c640 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009fc0 tx=0x7f17ac0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46687829cfec00d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.132+0000 7f17b759e640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 0x7f17a00a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17b00eeb60 tx=0x7f17b008f3c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f179d7fa640 1 -- 172.21.15.77:0/4121561637 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac005020 con 0x55ffd04cdb40 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f179d7fa640 1 -- 172.21.15.77:0/4121561637 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0056a0 con 0x55ffd04cdb40 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f179d7fa640 1 -- 172.21.15.77:0/4121561637 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0106e0 con 0x55ffd04cdb40 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 -- 172.21.15.77:0/4121561637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 msgr2=0x55ffd04cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009fc0 tx=0x7f17ac0094c0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 -- 172.21.15.77:0/4121561637 shutdown_connections 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 --2- 172.21.15.77:0/4121561637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04cc090 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 -- 172.21.15.77:0/4121561637 >> 172.21.15.77:0/4121561637 conn(0x55ffd0495310 msgr2=0x55ffd04c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 -- 172.21.15.77:0/4121561637 shutdown_connections 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.133+0000 7f17bf405740 1 -- 172.21.15.77:0/4121561637 wait complete. 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 -- start start 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 Processor -- start 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 0x55ffd04d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd04de420 con 0x55ffd04db780 2024-09-15T22:15:26.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffd03ed060 con 0x55ffd04cdb40 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffd04de5a0 con 0x55ffd04cc5d0 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17bf405740 10 client.0 _ll_register_callbacks cb 0x55ffd01408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56830/0 (socket says 172.21.15.77:56830) 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 -- 172.21.15.77:0/3090437609 learned_addr learned my addr 172.21.15.77:0/3090437609 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 0x55ffd04d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 -- 172.21.15.77:0/3090437609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 msgr2=0x55ffd04db1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 -- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 msgr2=0x55ffd04d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 0x55ffd04d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b659c640 1 -- 172.21.15.77:0/3090437609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd01a33c0 con 0x55ffd04db780 2024-09-15T22:15:26.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.134+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 0x55ffd04d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17b659c640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17ac004fb0 tx=0x7f17ac004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85617d5f71b546 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac010690 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f17ac0046c0 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac0056a0 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17ac0048c0 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 10 client.16407 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffd04e9f00 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0056a0 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f17ac00c030 con 0x55ffd04db780 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 10 client.16407 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 10 client.16407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 20 client.16407 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:26.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd04eb300 0x55ffd04ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 4 client.16407 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.135+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd043f860 con 0x55ffd04eb300 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.136+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd04eb300 0x55ffd04ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.136+0000 7f179aff5640 10 client.16407 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.136+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd04eb300 0x55ffd04ed6f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28c204039023a09e server_cookie=9d1f39f7e5b31666 in_seq=0 out_seq=0 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f179aff5640 1 -- 172.21.15.77:0/3090437609 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffd043f860 con 0x55ffd04eb300 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f179aff5640 10 client.16407 handle_command_reply: tid=0 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 client.16407 shutdown 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 msgr2=0x55ffd04d8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17ac004fb0 tx=0x7f17ac004b70 comp rx=0 tx=0).stop 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 shutdown_connections 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd04eb300 0x55ffd04ed6f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04db780 0x55ffd04d8bf0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cdb40 0x55ffd04d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 --2- 172.21.15.77:0/3090437609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cc5d0 0x55ffd04db1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f179aff5640 0 client.16407 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 >> 172.21.15.77:0/3090437609 conn(0x55ffd0495310 msgr2=0x55ffd04ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 shutdown_connections 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 1 -- 172.21.15.77:0/3090437609 wait complete. 2024-09-15T22:15:26.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.137+0000 7f17bf405740 20 client.16407 trim_cache size 0 max 16384 2024-09-15T22:15:26.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 Processor -- start 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 -- start start 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cdb40 0x55ffd04f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81acca0 con 0x55ffd04cc5d0 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ae8a0 con 0x55ffd04db780 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17bf405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ad370 con 0x55ffd04cdb40 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56844/0 (socket says 172.21.15.77:56844) 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 -- 172.21.15.77:0/234258877 learned_addr learned my addr 172.21.15.77:0/234258877 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 -- 172.21.15.77:0/234258877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cdb40 msgr2=0x55ffd04f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cdb40 0x55ffd04f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 -- 172.21.15.77:0/234258877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 msgr2=0x55ffd04c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 -- 172.21.15.77:0/234258877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd04c7310 con 0x55ffd04cc5d0 2024-09-15T22:15:26.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.150+0000 7f17b659c640 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f17ac018040 tx=0x7f17ac0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb4f658da8a681 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f179cff9640 1 -- 172.21.15.77:0/234258877 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004620 con 0x55ffd04cc5d0 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f179cff9640 1 -- 172.21.15.77:0/234258877 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac027070 con 0x55ffd04cc5d0 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f179cff9640 1 -- 172.21.15.77:0/234258877 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0108f0 con 0x55ffd04cc5d0 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- 172.21.15.77:0/234258877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 msgr2=0x55ffd04f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f17ac018040 tx=0x7f17ac0092c0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- 172.21.15.77:0/234258877 shutdown_connections 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04db780 0x55ffd04c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04cdb40 0x55ffd04f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 --2- 172.21.15.77:0/234258877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cc5d0 0x55ffd04f2c90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- 172.21.15.77:0/234258877 >> 172.21.15.77:0/234258877 conn(0x55ffd0495310 msgr2=0x55ffd04da4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- 172.21.15.77:0/234258877 shutdown_connections 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- 172.21.15.77:0/234258877 wait complete. 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.151+0000 7f17bf405740 1 -- start start 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 Processor -- start 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cc5d0 0x55ffd04dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 0x55ffd0554c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ae110 con 0x55ffd04cdb40 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81ada40 con 0x55ffd04cc5d0 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81aac80 con 0x55ffd04db780 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17bf405740 10 client.0 _ll_register_callbacks cb 0x7f17ac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56860/0 (socket says 172.21.15.77:56860) 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 -- 172.21.15.77:0/1452941144 learned_addr learned my addr 172.21.15.77:0/1452941144 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b659c640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 0x55ffd0554c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 -- 172.21.15.77:0/1452941144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 msgr2=0x55ffd0554c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 0x55ffd0554c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 -- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cc5d0 msgr2=0x55ffd04dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cc5d0 0x55ffd04dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 -- 172.21.15.77:0/1452941144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd05552a0 con 0x55ffd04cdb40 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b659c640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 0x55ffd0554c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f17b6d9d640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x55ffd04d9aa0 tx=0x7f17a8017810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5fcb37eeeb075 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f179aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.152+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800f040 con 0x55ffd04cdb40 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f17a8015860 con 0x55ffd04cdb40 2024-09-15T22:15:26.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac027070 con 0x55ffd04cdb40 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17a802c050 con 0x55ffd04cdb40 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 10 client.16413 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ffd05565f0 con 0x55ffd04cdb40 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac027070 con 0x55ffd04cdb40 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f17a801ec80 con 0x55ffd04cdb40 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 10 client.16413 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 10 client.16413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 20 client.16413 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd0556da0 0x55ffd05591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 4 client.16413 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffd0439d70 con 0x55ffd0556da0 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.153+0000 7f17b659c640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd0556da0 0x55ffd05591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:26.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17b659c640 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd0556da0 0x55ffd05591c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaea0d4a2c78787d server_cookie=23ee827d6fb85749 in_seq=0 out_seq=0 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f179aff5640 10 client.16413 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f179aff5640 1 -- 172.21.15.77:0/1452941144 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffd0439d70 con 0x55ffd0556da0 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f179aff5640 10 client.16413 handle_command_reply: tid=0 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 client.16413 shutdown 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 msgr2=0x55ffd04e02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x55ffd04d9aa0 tx=0x7f17a8017810 comp rx=0 tx=0).stop 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 shutdown_connections 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffd0556da0 0x55ffd05591c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd04db780 0x55ffd0554c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd04cdb40 0x55ffd04e02d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 --2- 172.21.15.77:0/1452941144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd04cc5d0 0x55ffd04dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f179aff5640 0 client.16413 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:15:26.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 >> 172.21.15.77:0/1452941144 conn(0x55ffd0495310 msgr2=0x55ffd04ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 shutdown_connections 2024-09-15T22:15:26.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 1 -- 172.21.15.77:0/1452941144 wait complete. 2024-09-15T22:15:26.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.154+0000 7f17bf405740 20 client.16413 trim_cache size 0 max 16384 2024-09-15T22:15:26.162 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 msgr2=0x7f17a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 0x7f17a00a5870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17b00eeb60 tx=0x7f17b008f3c0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 msgr2=0x7f17b819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b0033040 tx=0x7f17b0003970 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 shutdown_connections 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17a00a33b0 0x7f17a00a5870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17b8108160 0x7f17b819cdb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b8101290 0x7f17b8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 --2- 172.21.15.77:0/2943249562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b81008e0 0x7f17b81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:26.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 >> 172.21.15.77:0/2943249562 conn(0x7f17b8077210 msgr2=0x7f17b80ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:26.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 shutdown_connections 2024-09-15T22:15:26.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:26.155+0000 7f17bd02c640 1 -- 172.21.15.77:0/2943249562 wait complete. 2024-09-15T22:15:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:26 smithi077 ceph-mon[30980]: pgmap v93: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:26 smithi196 ceph-mon[35109]: pgmap v93: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:26 smithi117 ceph-mon[35021]: pgmap v93: 129 pgs: 129 active+clean; 93 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 281 KiB/s wr, 33 op/s 2024-09-15T22:15:26.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.35 MB/sec warmup 95 sec latency 14958.019 ms 2024-09-15T22:15:26.897 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:15:27.139 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:27.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.26 MB/sec warmup 96 sec latency 15958.045 ms 2024-09-15T22:15:27.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 -- 172.21.15.77:0/2786085265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401036c0 msgr2=0x7fae4010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2786085265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401036c0 0x7fae4010bbb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fae28009900 tx=0x7fae2802a480 comp rx=0 tx=0).stop 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 -- 172.21.15.77:0/2786085265 shutdown_connections 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2786085265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae4010c180 0x7fae4010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2786085265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401036c0 0x7fae4010bbb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2786085265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae40102da0 0x7fae40103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 -- 172.21.15.77:0/2786085265 >> 172.21.15.77:0/2786085265 conn(0x7fae400fc7a0 msgr2=0x7fae400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 -- 172.21.15.77:0/2786085265 shutdown_connections 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.636+0000 7fae45ac9640 1 -- 172.21.15.77:0/2786085265 wait complete. 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 Processor -- start 2024-09-15T22:15:27.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 -- start start 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae40102da0 0x7fae401a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae401036c0 0x7fae401a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae40112020 con 0x7fae4010c180 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae40111ea0 con 0x7fae40102da0 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae45ac9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae401121a0 con 0x7fae401036c0 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56884/0 (socket says 172.21.15.77:56884) 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 -- 172.21.15.77:0/2642477961 learned_addr learned my addr 172.21.15.77:0/2642477961 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae40102da0 0x7fae401a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 -- 172.21.15.77:0/2642477961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae401036c0 msgr2=0x7fae401a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae401036c0 0x7fae401a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 -- 172.21.15.77:0/2642477961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae40102da0 msgr2=0x7fae401a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae40102da0 0x7fae401a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.637+0000 7fae3ffff640 1 -- 172.21.15.77:0/2642477961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4019aee0 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b970 tx=0x7fae3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae3cff9640 1 -- 172.21.15.77:0/2642477961 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38019070 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4019b1d0 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae3cff9640 1 -- 172.21.15.77:0/2642477961 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3800cb40 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae3cff9640 1 -- 172.21.15.77:0/2642477961 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38007650 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.638+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a7520 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.640+0000 7fae3cff9640 1 -- 172.21.15.77:0/2642477961 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fae3801c070 con 0x7fae4010c180 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.640+0000 7fae47ea2740 1 Processor -- start 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.640+0000 7fae47ea2740 1 -- start start 2024-09-15T22:15:27.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493de370 0x55b2493e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b2493392f0 con 0x55b2493d4b40 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2492e0cf0 con 0x55b2493d35d0 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae47ea2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2493352b0 con 0x55b2493de370 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56898/0 (socket says 172.21.15.77:56898) 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 -- 172.21.15.77:0/959814336 learned_addr learned my addr 172.21.15.77:0/959814336 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493de370 0x55b2493e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 -- 172.21.15.77:0/959814336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493de370 msgr2=0x55b2493e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493de370 0x55b2493e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 -- 172.21.15.77:0/959814336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 msgr2=0x55b2493d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 -- 172.21.15.77:0/959814336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2492c53a0 con 0x55b2493d4b40 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3cff9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 0x7fae2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 0x7fae2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3effd640 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fae28031550 tx=0x7fae280056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cafe3be4c63330 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.641+0000 7fae3cff9640 1 -- 172.21.15.77:0/2642477961 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fae380cf5c0 con 0x7fae4010c180 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae35ffb640 1 -- 172.21.15.77:0/959814336 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28043020 con 0x55b2493d4b40 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae35ffb640 1 -- 172.21.15.77:0/959814336 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28031930 con 0x55b2493d4b40 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae35ffb640 1 -- 172.21.15.77:0/959814336 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28035040 con 0x55b2493d4b40 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 -- 172.21.15.77:0/959814336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 msgr2=0x55b2493d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 0x7fae2c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b2493d0ac0 tx=0x7fae30008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fae28031550 tx=0x7fae280056d0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 -- 172.21.15.77:0/959814336 shutdown_connections 2024-09-15T22:15:27.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493de370 0x55b2493e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 --2- 172.21.15.77:0/959814336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493d4b40 0x55b2493d3090 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 -- 172.21.15.77:0/959814336 >> 172.21.15.77:0/959814336 conn(0x55b24939c310 msgr2=0x55b2493cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 -- 172.21.15.77:0/959814336 shutdown_connections 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.642+0000 7fae47ea2740 1 -- 172.21.15.77:0/959814336 wait complete. 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 -- start start 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 Processor -- start 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493dafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b2493e1010 con 0x55b2493de370 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2492f4060 con 0x55b2493d4b40 2024-09-15T22:15:27.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2493e1190 con 0x55b2493d35d0 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae47ea2740 10 client.0 _ll_register_callbacks cb 0x55b2490478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56912/0 (socket says 172.21.15.77:56912) 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.643+0000 7fae3ffff640 1 -- 172.21.15.77:0/1766660225 learned_addr learned my addr 172.21.15.77:0/1766660225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3effd640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493dafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 -- 172.21.15.77:0/1766660225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 msgr2=0x55b2493ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493ddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 -- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 msgr2=0x55b2493dafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493dafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 -- 172.21.15.77:0/1766660225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2490aa3c0 con 0x55b2493de370 2024-09-15T22:15:27.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3effd640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493dafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae3ffff640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fae38098040 tx=0x7fae3809bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88a1ca0a495c616 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380a42e0 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fae380b45f0 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae28031930 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae38097ce0 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae47ea2740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2493eca90 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28031930 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fae380a3e50 con 0x55b2493de370 2024-09-15T22:15:27.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.644+0000 7fae47ea2740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae47ea2740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae47ea2740 20 client.16425 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b2493f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae47ea2740 4 client.16425 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b249346860 con 0x55b2493edff0 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b2493f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae1effd640 10 client.16425 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.645+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b2493f0470 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bf218a0d6b0fe3 server_cookie=4a35979b6013b82b in_seq=0 out_seq=0 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae1effd640 1 -- 172.21.15.77:0/1766660225 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b249346860 con 0x55b2493edff0 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae1effd640 10 client.16425 handle_command_reply: tid=0 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 client.16425 shutdown 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 msgr2=0x55b2493db570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fae38098040 tx=0x7fae3809bd70 comp rx=0 tx=0).stop 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 shutdown_connections 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b2493f0470 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493db570 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493dafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae1effd640 0 client.16425 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 --2- 172.21.15.77:0/1766660225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 >> 172.21.15.77:0/1766660225 conn(0x55b24939c310 msgr2=0x55b2493d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 shutdown_connections 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 1 -- 172.21.15.77:0/1766660225 wait complete. 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.646+0000 7fae47ea2740 20 client.16425 trim_cache size 0 max 16384 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 Processor -- start 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 -- start start 2024-09-15T22:15:27.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493ea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae401b1120 con 0x55b2493de370 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae401b2d20 con 0x55b2493d4b40 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae47ea2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae401b17f0 con 0x55b2493d35d0 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56916/0 (socket says 172.21.15.77:56916) 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 -- 172.21.15.77:0/2022500927 learned_addr learned my addr 172.21.15.77:0/2022500927 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 -- 172.21.15.77:0/2022500927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 msgr2=0x55b2493cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 -- 172.21.15.77:0/2022500927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 msgr2=0x55b2493ea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493ea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.659+0000 7fae3ffff640 1 -- 172.21.15.77:0/2022500927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2493ce310 con 0x55b2493de370 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fae3809b370 tx=0x7fae38014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e6280a4f055d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae357fa640 1 -- 172.21.15.77:0/2022500927 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38097e80 con 0x55b2493de370 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae357fa640 1 -- 172.21.15.77:0/2022500927 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae380a42e0 con 0x55b2493de370 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae357fa640 1 -- 172.21.15.77:0/2022500927 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380cf240 con 0x55b2493de370 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 -- 172.21.15.77:0/2022500927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 msgr2=0x55b2493d6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fae3809b370 tx=0x7fae38014d00 comp rx=0 tx=0).stop 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 -- 172.21.15.77:0/2022500927 shutdown_connections 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493d6120 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d4b40 0x55b2493ea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2022500927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d35d0 0x55b2493cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 -- 172.21.15.77:0/2022500927 >> 172.21.15.77:0/2022500927 conn(0x55b24939c310 msgr2=0x55b2493cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 -- 172.21.15.77:0/2022500927 shutdown_connections 2024-09-15T22:15:27.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.660+0000 7fae47ea2740 1 -- 172.21.15.77:0/2022500927 wait complete. 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 -- start start 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 Processor -- start 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d4b40 0x55b2493e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae401b2590 con 0x55b2493de370 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae401b1ec0 con 0x55b2493d35d0 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae401af100 con 0x55b2493d4b40 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae47ea2740 10 client.0 _ll_register_callbacks cb 0x7fae380098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56918/0 (socket says 172.21.15.77:56918) 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae3ffff640 1 -- 172.21.15.77:0/2832042197 learned_addr learned my addr 172.21.15.77:0/2832042197 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:27.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.661+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 -- 172.21.15.77:0/2832042197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d4b40 msgr2=0x55b2493e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3effd640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d4b40 0x55b2493e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d4b40 0x55b2493e6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 -- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 msgr2=0x55b2493e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493e67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 -- 172.21.15.77:0/2832042197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2493e25d0 con 0x55b2493de370 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae3ffff640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b940 tx=0x7fae38099b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb125add17f4a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae1effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38097e80 con 0x55b2493de370 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fae380b4480 con 0x55b2493de370 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae380a42e0 con 0x55b2493de370 2024-09-15T22:15:27.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fae380afb90 con 0x55b2493de370 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae47ea2740 10 client.16431 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2494589b0 con 0x55b2493de370 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.662+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae380a42e0 con 0x55b2493de370 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fae38050af0 con 0x55b2493de370 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 10 client.16431 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 20 client.16431 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b24945aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 4 client.16431 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b249340d70 con 0x55b2493edff0 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b24945aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae1effd640 10 client.16431 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:27.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae3f7fe640 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b24945aee0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8beea97193ddf57 server_cookie=92f485081ac3ee98 in_seq=0 out_seq=0 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae1effd640 1 -- 172.21.15.77:0/2832042197 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b249340d70 con 0x55b2493edff0 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae1effd640 10 client.16431 handle_command_reply: tid=0 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.663+0000 7fae47ea2740 1 client.16431 shutdown 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 msgr2=0x55b2493e1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b940 tx=0x7fae38099b80 comp rx=0 tx=0).stop 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 shutdown_connections 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b2493edff0 0x55b24945aee0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b2493de370 0x55b2493e1da0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2493d4b40 0x55b2493e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae1effd640 0 client.16431 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 --2- 172.21.15.77:0/2832042197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2493d35d0 0x55b2493e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 >> 172.21.15.77:0/2832042197 conn(0x55b24939c310 msgr2=0x55b2493dc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 shutdown_connections 2024-09-15T22:15:27.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 1 -- 172.21.15.77:0/2832042197 wait complete. 2024-09-15T22:15:27.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae47ea2740 20 client.16431 trim_cache size 0 max 16384 2024-09-15T22:15:27.683 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:15:27.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 msgr2=0x7fae2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 0x7fae2c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b2493d0ac0 tx=0x7fae30008040 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 msgr2=0x7fae4019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b970 tx=0x7fae3800be30 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.664+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 shutdown_connections 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae2c0a33b0 0x7fae2c0a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae4010c180 0x7fae4019a670 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae401036c0 0x7fae401a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 --2- 172.21.15.77:0/2642477961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae40102da0 0x7fae401a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 >> 172.21.15.77:0/2642477961 conn(0x7fae400fc7a0 msgr2=0x7fae40101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 shutdown_connections 2024-09-15T22:15:27.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:27.665+0000 7fae45ac9640 1 -- 172.21.15.77:0/2642477961 wait complete. 2024-09-15T22:15:27.946 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:15:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:28 smithi077 ceph-mon[30980]: pgmap v94: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 527 KiB/s wr, 45 op/s 2024-09-15T22:15:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:28 smithi117 ceph-mon[35021]: pgmap v94: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 527 KiB/s wr, 45 op/s 2024-09-15T22:15:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:28.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.16 MB/sec warmup 97 sec latency 16958.079 ms 2024-09-15T22:15:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:28 smithi196 ceph-mon[35109]: pgmap v94: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 527 KiB/s wr, 45 op/s 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 -- 172.21.15.77:0/1349581480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004100a80 msgr2=0x7f1004100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 --2- 172.21.15.77:0/1349581480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004100a80 0x7f1004100ee0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1000009930 tx=0x7f100002a420 comp rx=0 tx=0).stop 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 -- 172.21.15.77:0/1349581480 shutdown_connections 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 --2- 172.21.15.77:0/1349581480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004101420 0x7f100410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 --2- 172.21.15.77:0/1349581480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004100a80 0x7f1004100ee0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 --2- 172.21.15.77:0/1349581480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f1004106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:28.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 -- 172.21.15.77:0/1349581480 >> 172.21.15.77:0/1349581480 conn(0x7f10040fc7a0 msgr2=0x7f10040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 -- 172.21.15.77:0/1349581480 shutdown_connections 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.903+0000 7f100c09d640 1 -- 172.21.15.77:0/1349581480 wait complete. 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 Processor -- start 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 -- start start 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 0x7f10041984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f100419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1004110e40 con 0x7f1004101420 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1004110cc0 con 0x7f1004100a80 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100c09d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1004110fc0 con 0x7f1004106a80 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:28.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56932/0 (socket says 172.21.15.77:56932) 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 -- 172.21.15.77:0/2723534264 learned_addr learned my addr 172.21.15.77:0/2723534264 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009e39640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 0x7f10041984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100a63a640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f100419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 -- 172.21.15.77:0/2723534264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 msgr2=0x7f100419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f100419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 -- 172.21.15.77:0/2723534264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 msgr2=0x7f10041984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 0x7f10041984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009638640 1 -- 172.21.15.77:0/2723534264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100419d460 con 0x7f1004101420 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f1009e39640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 0x7f10041984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.904+0000 7f100a63a640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f100419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f1009638640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1000009230 tx=0x7f1000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100003f030 con 0x7f1004101420 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1000003ad0 con 0x7f1004101420 2024-09-15T22:15:28.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100419d6f0 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000035070 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.905+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10041a5030 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.906+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10041021c0 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.907+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1000042030 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.908+0000 7f0ff6ffd640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 0x7f0ff00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.908+0000 7f1009e39640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 0x7f0ff00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.908+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f10000f0600 con 0x7f1004101420 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.908+0000 7f1009e39640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 0x7f0ff00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005fd0 tx=0x7f0ff8005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:28.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:28.912+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10000bcca0 con 0x7f1004101420 2024-09-15T22:15:29.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.113+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f100410a020 con 0x7f1004101420 2024-09-15T22:15:29.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.114+0000 7f0ff6ffd640 1 -- 172.21.15.77:0/2723534264 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f10000bc5a0 con 0x7f1004101420 2024-09-15T22:15:29.115 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:15:29.115 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:15:29.116 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:15:29.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 msgr2=0x7f0ff00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:29.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 0x7f0ff00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8005fd0 tx=0x7f0ff8005850 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 msgr2=0x7f10041989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1000009230 tx=0x7f1000004270 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 shutdown_connections 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ff00a33b0 0x7f0ff00a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1004106a80 0x7f100419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1004101420 0x7f10041989f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 --2- 172.21.15.77:0/2723534264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1004100a80 0x7f10041984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 >> 172.21.15.77:0/2723534264 conn(0x7f10040fc7a0 msgr2=0x7f100406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 shutdown_connections 2024-09-15T22:15:29.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:29.116+0000 7f100c09d640 1 -- 172.21.15.77:0/2723534264 wait complete. 2024-09-15T22:15:29.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:15:29.366 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:15:29.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 9.07 MB/sec warmup 98 sec latency 17958.139 ms 2024-09-15T22:15:29.842 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 70bc1cae --release --await' 2024-09-15T22:15:29.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 70bc1cae --release --await 2024-09-15T22:15:29.944 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:30 smithi077 ceph-mon[30980]: pgmap v95: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 340 KiB/s wr, 19 op/s 2024-09-15T22:15:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:30 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2723534264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:30 smithi117 ceph-mon[35021]: pgmap v95: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 340 KiB/s wr, 19 op/s 2024-09-15T22:15:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:30 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2723534264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 -- 172.21.15.77:0/2836381716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102dc0 msgr2=0x7f1f001031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 --2- 172.21.15.77:0/2836381716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102dc0 0x7f1f001031a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002a480 comp rx=0 tx=0).stop 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 -- 172.21.15.77:0/2836381716 shutdown_connections 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 --2- 172.21.15.77:0/2836381716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f0010c1a0 0x7f1f0010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 --2- 172.21.15.77:0/2836381716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f0010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 --2- 172.21.15.77:0/2836381716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102dc0 0x7f1f001031a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 -- 172.21.15.77:0/2836381716 >> 172.21.15.77:0/2836381716 conn(0x7f1f000fc7a0 msgr2=0x7f1f000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 -- 172.21.15.77:0/2836381716 shutdown_connections 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.443+0000 7f1f06337640 1 -- 172.21.15.77:0/2836381716 wait complete. 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 Processor -- start 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 -- start start 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102dc0 0x7f1f00100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f00100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00111fb0 con 0x7f1f0010c1a0 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00111e30 con 0x7f1f001036e0 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f06337640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00112130 con 0x7f1f00102dc0 2024-09-15T22:15:30.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.444+0000 7f1f048d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1efffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102dc0 0x7f1f00100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43852/0 (socket says 172.21.15.77:43852) 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 -- 172.21.15.77:0/4043936154 learned_addr learned my addr 172.21.15.77:0/4043936154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f00100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 -- 172.21.15.77:0/4043936154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102dc0 msgr2=0x7f1f00100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102dc0 0x7f1f00100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 -- 172.21.15.77:0/4043936154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 msgr2=0x7f1f00100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f00100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 -- 172.21.15.77:0/4043936154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f001a68b0 con 0x7f1f0010c1a0 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f00100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f048d4640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800b970 tx=0x7f1ef800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1efd7fa640 1 -- 172.21.15.77:0/4043936154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8019070 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1efd7fa640 1 -- 172.21.15.77:0/4043936154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef800cb40 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f001a6ba0 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1efd7fa640 1 -- 172.21.15.77:0/4043936154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8007650 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.445+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f001a70e0 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1efd7fa640 1 -- 172.21.15.77:0/4043936154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1ef801c070 con 0x7f1f0010c1a0 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 Processor -- start 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 -- start start 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf77f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1efd7fa640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf85130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f4abedd280 con 0x55f4abf78480 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4abe84b10 con 0x55f4abf77b60 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.448+0000 7f1f08710740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f4abed9240 con 0x55f4abf82d40 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1efffff640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43866/0 (socket says 172.21.15.77:43866) 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 -- 172.21.15.77:0/667510961 learned_addr learned my addr 172.21.15.77:0/667510961 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1efd7fa640 1 -- 172.21.15.77:0/4043936154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1ef80cf410 con 0x7f1f0010c1a0 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf77f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1f048d4640 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf85130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 -- 172.21.15.77:0/667510961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 msgr2=0x55f4abf85130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf85130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 -- 172.21.15.77:0/667510961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 msgr2=0x55f4abf77f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf77f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 -- 172.21.15.77:0/667510961 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4abe911f0 con 0x55f4abf78480 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1efffff640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000c00 tx=0x7f1ef00058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40139d0 tx=0x7f1ef400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13142182d0f80e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1ee67fc640 1 -- 172.21.15.77:0/667510961 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4004030 con 0x55f4abf78480 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1ee67fc640 1 -- 172.21.15.77:0/667510961 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef400ee40 con 0x55f4abf78480 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.449+0000 7f1ee67fc640 1 -- 172.21.15.77:0/667510961 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4003a60 con 0x55f4abf78480 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- 172.21.15.77:0/667510961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 msgr2=0x55f4abf82800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40139d0 tx=0x7f1ef400b500 comp rx=0 tx=0).stop 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- 172.21.15.77:0/667510961 shutdown_connections 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf85130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf82800 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 --2- 172.21.15.77:0/667510961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf77f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- 172.21.15.77:0/667510961 >> 172.21.15.77:0/667510961 conn(0x55f4abf740f0 msgr2=0x55f4abf744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- 172.21.15.77:0/667510961 shutdown_connections 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- 172.21.15.77:0/667510961 wait complete. 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.450+0000 7f1f08710740 1 -- start start 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 Processor -- start 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf80650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f4abf85910 con 0x55f4abf78480 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f4abe97f10 con 0x55f4abf77b60 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f4abf85a90 con 0x55f4abf82d40 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f08710740 10 client.0 _ll_register_callbacks cb 0x55f4abe60a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43882/0 (socket says 172.21.15.77:43882) 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 -- 172.21.15.77:0/4245778432 learned_addr learned my addr 172.21.15.77:0/4245778432 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1f048d4640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf80650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1efffff640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 -- 172.21.15.77:0/4245778432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 msgr2=0x55f4abf80650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf80650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 -- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 msgr2=0x55f4abf7fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf7fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.451+0000 7f1eff7fe640 1 -- 172.21.15.77:0/4245778432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4abc4e6a0 con 0x55f4abf78480 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4004470 tx=0x7f1ef40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bc09fe1578376b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400eb60 con 0x55f4abf78480 2024-09-15T22:15:30.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1ef40037e0 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef400ee40 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ef40039e0 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 10 client.16449 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4abf914c0 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef400ee40 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1ef4025070 con 0x55f4abf78480 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 10 client.16449 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 20 client.16449 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4abf94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 4 client.16449 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.452+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4abeea7f0 con 0x55f4abf92810 2024-09-15T22:15:30.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.453+0000 7f1efffff640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4abf94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.453+0000 7f1edf7fe640 10 client.16449 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.453+0000 7f1efffff640 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4abf94c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f399ca231ce82320 server_cookie=34eb448f3017f990 in_seq=0 out_seq=0 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1edf7fe640 1 -- 172.21.15.77:0/4245778432 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4abeea7f0 con 0x55f4abf92810 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1edf7fe640 10 client.16449 handle_command_reply: tid=0 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 client.16449 shutdown 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 msgr2=0x55f4abf80110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4004470 tx=0x7f1ef40056d0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 shutdown_connections 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4abf94c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1edf7fe640 0 client.16449 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf82d40 0x55f4abf80650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf80110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 --2- 172.21.15.77:0/4245778432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf77b60 0x55f4abf7fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 >> 172.21.15.77:0/4245778432 conn(0x55f4abf740f0 msgr2=0x55f4abf79e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 shutdown_connections 2024-09-15T22:15:30.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 1 -- 172.21.15.77:0/4245778432 wait complete. 2024-09-15T22:15:30.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.454+0000 7f1f08710740 20 client.16449 trim_cache size 0 max 16384 2024-09-15T22:15:30.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 8.97 MB/sec warmup 99 sec latency 18958.188 ms 2024-09-15T22:15:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 Processor -- start 2024-09-15T22:15:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 -- start start 2024-09-15T22:15:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf78480 0x55f4abf9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abf9a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b10a0 con 0x55f4abf77b60 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b2ca0 con 0x55f4abf82d40 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f08710740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b1770 con 0x55f4abf78480 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43898/0 (socket says 172.21.15.77:43898) 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 -- 172.21.15.77:0/3845811902 learned_addr learned my addr 172.21.15.77:0/3845811902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf78480 0x55f4abf9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1f048d4640 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abf9a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 -- 172.21.15.77:0/3845811902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf78480 msgr2=0x55f4abf9a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf78480 0x55f4abf9a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.472+0000 7f1efffff640 1 -- 172.21.15.77:0/3845811902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 msgr2=0x55f4abf9a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1efffff640 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abf9a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1efffff640 1 -- 172.21.15.77:0/3845811902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4abf73f20 con 0x55f4abf77b60 2024-09-15T22:15:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1efffff640 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66961c6df7f23345 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1ee5ffb640 1 -- 172.21.15.77:0/3845811902 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0056070 con 0x55f4abf77b60 2024-09-15T22:15:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1ee5ffb640 1 -- 172.21.15.77:0/3845811902 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0003ad0 con 0x55f4abf77b60 2024-09-15T22:15:30.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.473+0000 7f1ee5ffb640 1 -- 172.21.15.77:0/3845811902 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035040 con 0x55f4abf77b60 2024-09-15T22:15:30.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- 172.21.15.77:0/3845811902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 msgr2=0x55f4abf81a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef0040000 comp rx=0 tx=0).stop 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- 172.21.15.77:0/3845811902 shutdown_connections 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abf9a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf78480 0x55f4abf9a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 --2- 172.21.15.77:0/3845811902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf77b60 0x55f4abf81a30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- 172.21.15.77:0/3845811902 >> 172.21.15.77:0/3845811902 conn(0x55f4abf740f0 msgr2=0x55f4abf81320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- 172.21.15.77:0/3845811902 shutdown_connections 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- 172.21.15.77:0/3845811902 wait complete. 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 -- start start 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.474+0000 7f1f08710740 1 Processor -- start 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf77b60 0x55f4abf86d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abffc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b2510 con 0x55f4abf78480 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b1e40 con 0x55f4abf82d40 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001af080 con 0x55f4abf77b60 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1f08710740 10 client.0 _ll_register_callbacks cb 0x7f1ef4013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43906/0 (socket says 172.21.15.77:43906) 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 -- 172.21.15.77:0/574153323 learned_addr learned my addr 172.21.15.77:0/574153323 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1efffff640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf77b60 0x55f4abf86d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 -- 172.21.15.77:0/574153323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf77b60 msgr2=0x55f4abf86d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf77b60 0x55f4abf86d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 -- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 msgr2=0x55f4abffc390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abffc390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 -- 172.21.15.77:0/574153323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4abf87b40 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e720 tx=0x7f1ef40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e70368f385b94db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1edf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.475+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400b3c0 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f1ef4027440 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0003ad0 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ef4040050 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 10 client.16455 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4abffd9e0 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0003ad0 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1ef4025070 con 0x55f4abf78480 2024-09-15T22:15:30.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 10 client.16455 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 20 client.16455 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4ac000220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 4 client.16455 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.476+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f4abee4d00 con 0x55f4abf92810 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.477+0000 7f1efffff640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4ac000220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.477+0000 7f1efffff640 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4ac000220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c89759776e8b614a server_cookie=2b56273b1cfb37ce in_seq=0 out_seq=0 2024-09-15T22:15:30.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.477+0000 7f1edf7fe640 10 client.16455 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1edf7fe640 1 -- 172.21.15.77:0/574153323 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4abee4d00 con 0x55f4abf92810 2024-09-15T22:15:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1edf7fe640 10 client.16455 handle_command_reply: tid=0 2024-09-15T22:15:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1f08710740 1 client.16455 shutdown 2024-09-15T22:15:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 msgr2=0x55f4abf872b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e720 tx=0x7f1ef40055a0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 shutdown_connections 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.478+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f4abf92810 0x55f4ac000220 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4abf82d40 0x55f4abffc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f4abf78480 0x55f4abf872b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1edf7fe640 0 client.16455 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 --2- 172.21.15.77:0/574153323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f4abf77b60 0x55f4abf86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 >> 172.21.15.77:0/574153323 conn(0x55f4abf740f0 msgr2=0x55f4abf73270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 shutdown_connections 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 1 -- 172.21.15.77:0/574153323 wait complete. 2024-09-15T22:15:30.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f08710740 20 client.16455 trim_cache size 0 max 16384 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "fa2d8dd2-5411-4a8c-89a7-f2124bc32faf", 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 msgr2=0x7f1ee80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000c00 tx=0x7f1ef00058d0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 msgr2=0x7f1f001014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.479+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800b970 tx=0x7f1ef800be30 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 shutdown_connections 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c1a0 0x7f1f001014a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f001036e0 0x7f1f00100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 --2- 172.21.15.77:0/4043936154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102dc0 0x7f1f00100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 >> 172.21.15.77:0/4043936154 conn(0x7f1f000fc7a0 msgr2=0x7f1f000fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 shutdown_connections 2024-09-15T22:15:30.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:30.480+0000 7f1f06337640 1 -- 172.21.15.77:0/4043936154 wait complete. 2024-09-15T22:15:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:30 smithi196 ceph-mon[35109]: pgmap v95: 129 pgs: 129 active+clean; 96 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 340 KiB/s wr, 19 op/s 2024-09-15T22:15:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:30 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2723534264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:30.728 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:31.336 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:15:31.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 162018 8.89 MB/sec warmup 100 sec latency 19958.240 ms 2024-09-15T22:15:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- 172.21.15.77:0/3017306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7341027a0 msgr2=0x7fc7341067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 --2- 172.21.15.77:0/3017306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7341027a0 0x7fc7341067f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc72402a410 comp rx=0 tx=0).stop 2024-09-15T22:15:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- 172.21.15.77:0/3017306 shutdown_connections 2024-09-15T22:15:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 --2- 172.21.15.77:0/3017306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 --2- 172.21.15.77:0/3017306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7341027a0 0x7fc7341067f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 --2- 172.21.15.77:0/3017306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc734101df0 0x7fc7341021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- 172.21.15.77:0/3017306 >> 172.21.15.77:0/3017306 conn(0x7fc7340fd8b0 msgr2=0x7fc7340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- 172.21.15.77:0/3017306 shutdown_connections 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- 172.21.15.77:0/3017306 wait complete. 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 Processor -- start 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.897+0000 7fc73c588640 1 -- start start 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7341027a0 0x7fc734198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc734111f40 con 0x7fc734101df0 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc734111dc0 con 0x7fc734106f20 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73c588640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341120c0 con 0x7fc7341027a0 2024-09-15T22:15:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43924/0 (socket says 172.21.15.77:43924) 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 -- 172.21.15.77:0/2063904368 learned_addr learned my addr 172.21.15.77:0/2063904368 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc739b23640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7341027a0 0x7fc734198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73ab25640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 -- 172.21.15.77:0/2063904368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7341027a0 msgr2=0x7fc734198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7341027a0 0x7fc734198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 -- 172.21.15.77:0/2063904368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 msgr2=0x7fc73419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 -- 172.21.15.77:0/2063904368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419d390 con 0x7fc734101df0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73ab25640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc73a324640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c920 tx=0x7fc73000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.898+0000 7fc72b7fe640 1 -- 172.21.15.77:0/2063904368 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730007c70 con 0x7fc734101df0 2024-09-15T22:15:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.899+0000 7fc72b7fe640 1 -- 172.21.15.77:0/2063904368 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc730007e10 con 0x7fc734101df0 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.899+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73419d680 con 0x7fc734101df0 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.899+0000 7fc72b7fe640 1 -- 172.21.15.77:0/2063904368 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730018890 con 0x7fc734101df0 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.899+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a4f60 con 0x7fc734101df0 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 Processor -- start 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 -- start start 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825302f20 0x563825303300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825312110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563825268620 con 0x563825312650 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56382520fe50 con 0x563825303840 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.901+0000 7fc73e961740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638252645e0 con 0x563825302f20 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc72b7fe640 1 -- 172.21.15.77:0/2063904368 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc730018a70 con 0x7fc734101df0 2024-09-15T22:15:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43932/0 (socket says 172.21.15.77:43932) 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 -- 172.21.15.77:0/1780305678 learned_addr learned my addr 172.21.15.77:0/1780305678 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc73ab25640 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825302f20 0x563825303300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc73a324640 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825312110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 -- 172.21.15.77:0/1780305678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825302f20 msgr2=0x563825303300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825302f20 0x563825303300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 -- 172.21.15.77:0/1780305678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 msgr2=0x563825312110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825312110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 -- 172.21.15.77:0/1780305678 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563825200d40 con 0x563825312650 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc739b23640 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc724031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e292cf2c1e772143 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc728ff9640 1 -- 172.21.15.77:0/1780305678 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724040070 con 0x563825312650 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc72b7fe640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 0x7fc7200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc728ff9640 1 -- 172.21.15.77:0/1780305678 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724005020 con 0x563825312650 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.902+0000 7fc728ff9640 1 -- 172.21.15.77:0/1780305678 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724035040 con 0x563825312650 2024-09-15T22:15:31.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73ab25640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 0x7fc7200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc72b7fe640 1 -- 172.21.15.77:0/2063904368 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc7300d01b0 con 0x7fc734101df0 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 -- 172.21.15.77:0/1780305678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 msgr2=0x563825314a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73ab25640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 0x7fc7200a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005fd0 tx=0x7fc72c0098d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc724031630 comp rx=0 tx=0).stop 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 -- 172.21.15.77:0/1780305678 shutdown_connections 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825314a40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825312110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 --2- 172.21.15.77:0/1780305678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825302f20 0x563825303300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 -- 172.21.15.77:0/1780305678 >> 172.21.15.77:0/1780305678 conn(0x5638252ff5f0 msgr2=0x5638252ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 -- 172.21.15.77:0/1780305678 shutdown_connections 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.903+0000 7fc73e961740 1 -- 172.21.15.77:0/1780305678 wait complete. 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 -- start start 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 Processor -- start 2024-09-15T22:15:31.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 0x56382530f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 0x56382530fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638253151c0 con 0x563825312650 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5638251f0f10 con 0x563825302f20 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563825315340 con 0x563825303840 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73e961740 10 client.0 _ll_register_callbacks cb 0x563825214db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43948/0 (socket says 172.21.15.77:43948) 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 -- 172.21.15.77:0/1830434684 learned_addr learned my addr 172.21.15.77:0/1830434684 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73a324640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 0x56382530f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc739b23640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 0x56382530fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 -- 172.21.15.77:0/1830434684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 msgr2=0x56382530fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 0x56382530fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 -- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 msgr2=0x56382530f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 0x56382530f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 -- 172.21.15.77:0/1830434684 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563824f77580 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73a324640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 0x56382530f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc739b23640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 0x56382530fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc73ab25640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc72c01d890 tx=0x7fc72c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45af378bce13bee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.904+0000 7fc71dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01e820 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc724005020 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc72c01e9c0 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc72c01ebc0 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 10 client.16467 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563825041960 con 0x563825312650 2024-09-15T22:15:31.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724005020 con 0x563825312650 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc72c00f040 con 0x563825312650 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 10 client.16467 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 10 client.16467 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 20 client.16467 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563825322320 0x5638253247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 4 client.16467 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.905+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563825275b90 con 0x563825322320 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.906+0000 7fc73a324640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563825322320 0x5638253247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.906+0000 7fc73a324640 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563825322320 0x5638253247c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1adaab23a8c9419b server_cookie=8b065ef5f271af42 in_seq=0 out_seq=0 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.906+0000 7fc71dffb640 10 client.16467 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc71dffb640 1 -- 172.21.15.77:0/1830434684 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563825275b90 con 0x563825322320 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc71dffb640 10 client.16467 handle_command_reply: tid=0 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 client.16467 shutdown 2024-09-15T22:15:31.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 msgr2=0x563825310000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc72c01d890 tx=0x7fc72c004a40 comp rx=0 tx=0).stop 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 shutdown_connections 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563825322320 0x5638253247c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825312650 0x563825310000 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc71dffb640 0 client.16467 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825303840 0x56382530fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 --2- 172.21.15.77:0/1830434684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825302f20 0x56382530f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 >> 172.21.15.77:0/1830434684 conn(0x5638252ff5f0 msgr2=0x563825305350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 shutdown_connections 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 1 -- 172.21.15.77:0/1830434684 wait complete. 2024-09-15T22:15:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.907+0000 7fc73e961740 20 client.16467 trim_cache size 0 max 16384 2024-09-15T22:15:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.926+0000 7fc73e961740 1 Processor -- start 2024-09-15T22:15:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.926+0000 7fc73e961740 1 -- start start 2024-09-15T22:15:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.926+0000 7fc73e961740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.926+0000 7fc73e961740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x5638252fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73e961740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x56382532a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73e961740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341acbb0 con 0x563825302f20 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73e961740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ae7b0 con 0x563825303840 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73e961740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ad280 con 0x563825312650 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43956/0 (socket says 172.21.15.77:43956) 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 -- 172.21.15.77:0/3446132753 learned_addr learned my addr 172.21.15.77:0/3446132753 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73ab25640 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x56382532a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc739b23640 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x5638252fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 -- 172.21.15.77:0/3446132753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 msgr2=0x56382532a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x56382532a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 -- 172.21.15.77:0/3446132753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 msgr2=0x5638252fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x5638252fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 -- 172.21.15.77:0/3446132753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638253114c0 con 0x563825302f20 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc73a324640 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc730007590 tx=0x7fc7300d26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1db8efc63a5a02e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc71ffff640 1 -- 172.21.15.77:0/3446132753 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7300a0500 con 0x563825302f20 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc71ffff640 1 -- 172.21.15.77:0/3446132753 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7300a4dc0 con 0x563825302f20 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.927+0000 7fc71ffff640 1 -- 172.21.15.77:0/3446132753 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7300987c0 con 0x563825302f20 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 -- 172.21.15.77:0/3446132753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 msgr2=0x5638252fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc730007590 tx=0x7fc7300d26c0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 -- 172.21.15.77:0/3446132753 shutdown_connections 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x56382532a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x5638252fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 --2- 172.21.15.77:0/3446132753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x5638252fa2b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 -- 172.21.15.77:0/3446132753 >> 172.21.15.77:0/3446132753 conn(0x5638252ff5f0 msgr2=0x5638252fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 -- 172.21.15.77:0/3446132753 shutdown_connections 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.928+0000 7fc73e961740 1 -- 172.21.15.77:0/3446132753 wait complete. 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 -- start start 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 Processor -- start 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x56382531ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x563825317740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ae020 con 0x563825302f20 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ad950 con 0x563825303840 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341aab90 con 0x563825312650 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73e961740 10 client.0 _ll_register_callbacks cb 0x7fc72c01d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc739b23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc73a324640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x56382531ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc739b23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49070/0 (socket says 172.21.15.77:49070) 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.929+0000 7fc739b23640 1 -- 172.21.15.77:0/3963746544 learned_addr learned my addr 172.21.15.77:0/3963746544 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73ab25640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x563825317740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 -- 172.21.15.77:0/3963746544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 msgr2=0x563825317740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x563825317740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 -- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 msgr2=0x563825317150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 -- 172.21.15.77:0/3963746544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563825317f40 con 0x563825302f20 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73ab25640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x563825317740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc739b23640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73a324640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x56382531ae10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc7300cdbc0 tx=0x7fc73000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc6fbadba51ea2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc71dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7300b4030 con 0x563825302f20 2024-09-15T22:15:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc73009a1f0 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7300a4dc0 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc73000fb30 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73e961740 10 client.16473 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.930+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56382538e1d0 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7300a4dc0 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc7300bc0b0 con 0x563825302f20 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 10 client.16473 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 10 client.16473 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 20 client.16473 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56382538e9b0 0x563825390dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 4 client.16473 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70bc1cae", "release": true, "await": true}] 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70bc1cae", "release": true, "await": true}) -- 0x5638252700a0 con 0x56382538e9b0 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.931+0000 7fc739b23640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56382538e9b0 0x563825390dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:31.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.932+0000 7fc71dffb640 10 client.16473 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.932+0000 7fc739b23640 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56382538e9b0 0x563825390dd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8909e025aa905373 server_cookie=bd9ce98b9ae0f628 in_seq=0 out_seq=0 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc71dffb640 1 -- 172.21.15.77:0/3963746544 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5638252700a0 con 0x56382538e9b0 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc71dffb640 10 client.16473 handle_command_reply: tid=0 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 client.16473 shutdown 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 msgr2=0x56382531ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x56382531ae10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc7300cdbc0 tx=0x7fc73000cf10 comp rx=0 tx=0).stop 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 shutdown_connections 2024-09-15T22:15:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56382538e9b0 0x563825390dd0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563825312650 0x563825317740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563825303840 0x563825317150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc71dffb640 0 client.16473 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 --2- 172.21.15.77:0/3963746544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563825302f20 0x56382531ae10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.943+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 >> 172.21.15.77:0/3963746544 conn(0x5638252ff5f0 msgr2=0x56382530fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 shutdown_connections 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:31.946 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 12, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "70bc1cae": { 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 12, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:15:31.947 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 139.6, 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:31.948 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:31.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73e961740 1 -- 172.21.15.77:0/3963746544 wait complete. 2024-09-15T22:15:31.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73e961740 20 client.16473 trim_cache size 0 max 16384 2024-09-15T22:15:31.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 msgr2=0x7fc7200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 0x7fc7200a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc72c005fd0 tx=0x7fc72c0098d0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 msgr2=0x7fc7341983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.944+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c920 tx=0x7fc73000cde0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 shutdown_connections 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc7200a33b0 0x7fc7200a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc734106f20 0x7fc73419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7341027a0 0x7fc734198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 --2- 172.21.15.77:0/2063904368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc734101df0 0x7fc7341983e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.945+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 >> 172.21.15.77:0/2063904368 conn(0x7fc7340fd8b0 msgr2=0x7fc7340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.946+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 shutdown_connections 2024-09-15T22:15:31.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:31.946+0000 7fc73c588640 1 -- 172.21.15.77:0/2063904368 wait complete. 2024-09-15T22:15:32.169 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:32.291 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: pgmap v96: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 347 KiB/s wr, 25 op/s 2024-09-15T22:15:32.292 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:15:32.292 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.292 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:15:32.292 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:15:32.292 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.293 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:32 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: pgmap v96: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 347 KiB/s wr, 25 op/s 2024-09-15T22:15:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:15:32.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:15:32.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:15:32.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:32 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 163398 8.91 MB/sec warmup 101 sec latency 20440.113 ms 2024-09-15T22:15:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: pgmap v96: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 347 KiB/s wr, 25 op/s 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:32 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 -- 172.21.15.77:0/2861357565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 msgr2=0x7f2b50100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 --2- 172.21.15.77:0/2861357565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b50100ed0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009930 tx=0x7f2b4402a420 comp rx=0 tx=0).stop 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 -- 172.21.15.77:0/2861357565 shutdown_connections 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 --2- 172.21.15.77:0/2861357565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50101410 0x7f2b5010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 --2- 172.21.15.77:0/2861357565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b50100ed0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 --2- 172.21.15.77:0/2861357565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50106a70 0x7f2b50106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 -- 172.21.15.77:0/2861357565 >> 172.21.15.77:0/2861357565 conn(0x7f2b500fc790 msgr2=0x7f2b500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 -- 172.21.15.77:0/2861357565 shutdown_connections 2024-09-15T22:15:32.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.680+0000 7f2b557db640 1 -- 172.21.15.77:0/2861357565 wait complete. 2024-09-15T22:15:32.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.681+0000 7f2b557db640 1 Processor -- start 2024-09-15T22:15:32.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.681+0000 7f2b557db640 1 -- start start 2024-09-15T22:15:32.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.681+0000 7f2b557db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b557db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50101410 0x7f2b500fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b557db640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50106a70 0x7f2b500fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b557db640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50110e70 con 0x7f2b50100a70 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b557db640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50110cf0 con 0x7f2b50106a70 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b557db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50110ff0 con 0x7f2b50101410 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43968/0 (socket says 172.21.15.77:43968) 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 -- 172.21.15.77:0/4202320552 learned_addr learned my addr 172.21.15.77:0/4202320552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50101410 0x7f2b500fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50106a70 0x7f2b500fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 -- 172.21.15.77:0/4202320552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50101410 msgr2=0x7f2b500fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50101410 0x7f2b500fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 -- 172.21.15.77:0/4202320552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50106a70 msgr2=0x7f2b500fd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50106a70 0x7f2b500fd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 -- 172.21.15.77:0/4202320552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b501a6950 con 0x7f2b50100a70 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e9f0 tx=0x7f2b4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.682+0000 7f2b37fff640 1 -- 172.21.15.77:0/4202320552 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000cd80 con 0x7f2b50100a70 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.683+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b501a6c40 con 0x7f2b50100a70 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.683+0000 7f2b37fff640 1 -- 172.21.15.77:0/4202320552 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b40005500 con 0x7f2b50100a70 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.683+0000 7f2b37fff640 1 -- 172.21.15.77:0/4202320552 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40010690 con 0x7f2b50100a70 2024-09-15T22:15:32.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.683+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b501a7150 con 0x7f2b50100a70 2024-09-15T22:15:32.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.685+0000 7f2b37fff640 1 -- 172.21.15.77:0/4202320552 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2b400046c0 con 0x7f2b50100a70 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b37fff640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 0x7f2b380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 Processor -- start 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 0x7f2b380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b37fff640 1 -- 172.21.15.77:0/4202320552 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2b400d0410 con 0x7f2b50100a70 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 -- start start 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b381880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b38ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55843b2e72e0 con 0x55843b381dc0 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55843b28ece0 con 0x55843b38c680 2024-09-15T22:15:32.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55843b2e32a0 con 0x55843b3814a0 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 0x7f2b380a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009900 tx=0x7f2b440058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43976/0 (socket says 172.21.15.77:43976) 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4e7fc640 1 -- 172.21.15.77:0/1339278434 learned_addr learned my addr 172.21.15.77:0/1339278434 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b38ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.686+0000 7f2b4effd640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b381880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 -- 172.21.15.77:0/1339278434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 msgr2=0x55843b381880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b381880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 -- 172.21.15.77:0/1339278434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 msgr2=0x55843b38c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 -- 172.21.15.77:0/1339278434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843b273440 con 0x55843b38c680 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:32.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b38ea70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800add0 tx=0x7f2b4800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471150aee74c2f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b357fa640 1 -- 172.21.15.77:0/1339278434 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48003b70 con 0x55843b38c680 2024-09-15T22:15:32.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b357fa640 1 -- 172.21.15.77:0/1339278434 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48003d10 con 0x55843b38c680 2024-09-15T22:15:32.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.687+0000 7f2b357fa640 1 -- 172.21.15.77:0/1339278434 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48010b80 con 0x55843b38c680 2024-09-15T22:15:32.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1339278434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 msgr2=0x55843b38ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b38ea70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800add0 tx=0x7f2b4800f790 comp rx=0 tx=0).stop 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1339278434 shutdown_connections 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b38ea70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b38c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1339278434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b381880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1339278434 >> 172.21.15.77:0/1339278434 conn(0x55843b37da30 msgr2=0x55843b37de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.688+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1339278434 shutdown_connections 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1339278434 wait complete. 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 -- start start 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 Processor -- start 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 0x55843b38bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b38c680 0x55843b3896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55843b38f250 con 0x55843b381dc0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55843b2a2050 con 0x55843b3814a0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55843b38f3d0 con 0x55843b38c680 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.689+0000 7f2b57bb4740 10 client.0 _ll_register_callbacks cb 0x55843b2e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43982/0 (socket says 172.21.15.77:43982) 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 0x55843b38bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4e7fc640 1 -- 172.21.15.77:0/2882336003 learned_addr learned my addr 172.21.15.77:0/2882336003 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b38c680 0x55843b3896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 -- 172.21.15.77:0/2882336003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b38c680 msgr2=0x55843b3896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b38c680 0x55843b3896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 -- 172.21.15.77:0/2882336003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 msgr2=0x55843b389170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 -- 172.21.15.77:0/2882336003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843b0583c0 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b4effd640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 0x55843b38bec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cd820 tx=0x7f2b4009bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab69ce223113b8d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400b4070 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2b40098a20 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b48003d10 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.690+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b400a0760 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 10 client.25790 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55843b39abc0 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48003d10 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2b4009b800 con 0x55843b3814a0 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 10 client.25790 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:32.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 20 client.25790 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b39c050 0x55843b39e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 4 client.25790 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843b2f4850 con 0x55843b39c050 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.691+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b39c050 0x55843b39e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b39c050 0x55843b39e510 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ad4d5932afefeb server_cookie=ac248099f05eb848 in_seq=0 out_seq=0 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b32ff5640 10 client.25790 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b32ff5640 1 -- 172.21.15.77:0/2882336003 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55843b2f4850 con 0x55843b39c050 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b32ff5640 10 client.25790 handle_command_reply: tid=0 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 client.25790 shutdown 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 msgr2=0x55843b38bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 0x55843b38bec0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cd820 tx=0x7f2b4009bdd0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 shutdown_connections 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b39c050 0x55843b39e510 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b38c680 0x55843b3896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b381dc0 0x55843b389170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/2882336003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b3814a0 0x55843b38bec0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b32ff5640 0 client.25790 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.692+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 >> 172.21.15.77:0/2882336003 conn(0x55843b37da30 msgr2=0x55843b383760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.693+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 shutdown_connections 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.693+0000 7f2b57bb4740 1 -- 172.21.15.77:0/2882336003 wait complete. 2024-09-15T22:15:32.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.693+0000 7f2b57bb4740 20 client.25790 trim_cache size 0 max 16384 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.714+0000 7f2b57bb4740 1 Processor -- start 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.714+0000 7f2b57bb4740 1 -- start start 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 0x55843b3a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b3a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501b1080 con 0x55843b3814a0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501b2c80 con 0x55843b38c680 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501b1750 con 0x55843b381dc0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43986/0 (socket says 172.21.15.77:43986) 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 -- 172.21.15.77:0/1380154989 learned_addr learned my addr 172.21.15.77:0/1380154989 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b3a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 0x55843b3a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 -- 172.21.15.77:0/1380154989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 msgr2=0x55843b3a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 0x55843b3a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 -- 172.21.15.77:0/1380154989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 msgr2=0x55843b3a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b3a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 -- 172.21.15.77:0/1380154989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843b37d860 con 0x55843b3814a0 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b3a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:32.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 0x55843b3a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:32.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.715+0000 7f2b4effd640 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c6a0 tx=0x7f2b40098650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a1bcb6ee53e36b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b34ff9640 1 -- 172.21.15.77:0/1380154989 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400b4070 con 0x55843b3814a0 2024-09-15T22:15:32.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b34ff9640 1 -- 172.21.15.77:0/1380154989 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b400a5020 con 0x55843b3814a0 2024-09-15T22:15:32.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b34ff9640 1 -- 172.21.15.77:0/1380154989 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400a4b90 con 0x55843b3814a0 2024-09-15T22:15:32.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1380154989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 msgr2=0x55843b388a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c6a0 tx=0x7f2b40098650 comp rx=0 tx=0).stop 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1380154989 shutdown_connections 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b38c680 0x55843b3a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.716+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b381dc0 0x55843b3a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/1380154989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b3814a0 0x55843b388a70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1380154989 >> 172.21.15.77:0/1380154989 conn(0x55843b37da30 msgr2=0x55843b37cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1380154989 shutdown_connections 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- 172.21.15.77:0/1380154989 wait complete. 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- start start 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 Processor -- start 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b381dc0 0x55843b386880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b38c680 0x55843b386dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501b24f0 con 0x55843b38c680 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501b1e20 con 0x55843b381dc0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501af060 con 0x55843b3814a0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b57bb4740 10 client.0 _ll_register_callbacks cb 0x7f2b4000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b4effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55022/0 (socket says 172.21.15.77:55022) 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b4effd640 1 -- 172.21.15.77:0/4027845192 learned_addr learned my addr 172.21.15.77:0/4027845192 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b4f7fe640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b38c680 0x55843b386dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.717+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b381dc0 0x55843b386880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 -- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b381dc0 msgr2=0x55843b386880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b381dc0 0x55843b386880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 -- 172.21.15.77:0/4027845192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b38c680 msgr2=0x55843b386dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:32.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b38c680 0x55843b386dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 -- 172.21.15.77:0/4027845192 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843b405fb0 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b4effd640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b40098810 tx=0x7f2b40004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e34831e1a6359f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400b4050 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2b40098100 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b400a5020 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b40098300 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b57bb4740 10 client.25723 fetch_fsmap learned FSMap version 32 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.718+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55843b407240 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b400a5020 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2b400a1b20 con 0x55843b3814a0 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 10 client.25723 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:32.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 20 client.25723 populate_metadata read hostname 'smithi077' 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b4076d0 0x55843b409af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 4 client.25723 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55843b37d300 con 0x55843b4076d0 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.719+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b4076d0 0x55843b409af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b4e7fc640 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b4076d0 0x55843b409af0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b984024e313a4f server_cookie=645f93076bdec439 in_seq=0 out_seq=0 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b32ff5640 10 client.25723 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b32ff5640 1 -- 172.21.15.77:0/4027845192 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55843b37d300 con 0x55843b4076d0 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b32ff5640 10 client.25723 handle_command_reply: tid=0 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 client.25723 shutdown 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 msgr2=0x55843b386340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b40098810 tx=0x7f2b40004a40 comp rx=0 tx=0).stop 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 shutdown_connections 2024-09-15T22:15:32.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55843b4076d0 0x55843b409af0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55843b38c680 0x55843b386dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55843b381dc0 0x55843b386880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b32ff5640 0 client.25723 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 --2- 172.21.15.77:0/4027845192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55843b3814a0 0x55843b386340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 >> 172.21.15.77:0/4027845192 conn(0x55843b37da30 msgr2=0x55843b382ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 shutdown_connections 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 1 -- 172.21.15.77:0/4027845192 wait complete. 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.720+0000 7f2b57bb4740 20 client.25723 trim_cache size 0 max 16384 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:15:32.727 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: "fa2d8dd2-5411-4a8c-89a7-f2124bc32faf": { 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "fa2d8dd2-5411-4a8c-89a7-f2124bc32faf", 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:15:32.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 msgr2=0x7f2b380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 0x7f2b380a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009900 tx=0x7f2b440058d0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 msgr2=0x7f2b500fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e9f0 tx=0x7f2b4000edf0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 shutdown_connections 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b380a33b0 0x7f2b380a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b50106a70 0x7f2b500fd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b50101410 0x7f2b500fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 --2- 172.21.15.77:0/4202320552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b50100a70 0x7f2b500fce40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 >> 172.21.15.77:0/4202320552 conn(0x7f2b500fc790 msgr2=0x7f2b5006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 shutdown_connections 2024-09-15T22:15:32.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:15:32.721+0000 7f2b557db640 1 -- 172.21.15.77:0/4202320552 wait complete. 2024-09-15T22:15:33.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 165786 8.91 MB/sec warmup 102 sec latency 18.370 ms 2024-09-15T22:15:33.610 INFO:tasks.cephfs.filesystem:scrub status for tag:fa2d8dd2-5411-4a8c-89a7-f2124bc32faf - {'path': '/', 'tag': 'fa2d8dd2-5411-4a8c-89a7-f2124bc32faf', 'options': 'recursive,force'} 2024-09-15T22:15:34.087 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70bc1cae', total seconds elapsed: 25.7 2024-09-15T22:15:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:34 smithi077 ceph-mon[30980]: pgmap v97: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:34 smithi117 ceph-mon[35021]: pgmap v97: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:34.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 168007 8.95 MB/sec warmup 103 sec latency 16.184 ms 2024-09-15T22:15:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:34 smithi196 ceph-mon[35109]: pgmap v97: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:35.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 170812 9.03 MB/sec warmup 104 sec latency 13.641 ms 2024-09-15T22:15:36.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 173929 9.11 MB/sec warmup 105 sec latency 15.022 ms 2024-09-15T22:15:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:36 smithi077 ceph-mon[30980]: pgmap v98: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:36 smithi196 ceph-mon[35109]: pgmap v98: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:36 smithi117 ceph-mon[35021]: pgmap v98: 129 pgs: 129 active+clean; 95 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 254 KiB/s wr, 17 op/s 2024-09-15T22:15:37.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 176252 9.10 MB/sec warmup 106 sec latency 15.437 ms 2024-09-15T22:15:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:37 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:37 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:37 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:38.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 178368 9.14 MB/sec warmup 107 sec latency 10.836 ms 2024-09-15T22:15:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:38 smithi077 ceph-mon[30980]: pgmap v99: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 KiB/s wr, 80 op/s 2024-09-15T22:15:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:15:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:38 smithi196 ceph-mon[35109]: pgmap v99: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 KiB/s wr, 80 op/s 2024-09-15T22:15:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:15:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:38 smithi117 ceph-mon[35021]: pgmap v99: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 KiB/s wr, 80 op/s 2024-09-15T22:15:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:15:39.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 181306 9.22 MB/sec warmup 108 sec latency 33.464 ms 2024-09-15T22:15:40.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 184274 9.27 MB/sec warmup 109 sec latency 31.102 ms 2024-09-15T22:15:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:40 smithi077 ceph-mon[30980]: pgmap v100: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 553 KiB/s wr, 69 op/s 2024-09-15T22:15:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:40 smithi196 ceph-mon[35109]: pgmap v100: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 553 KiB/s wr, 69 op/s 2024-09-15T22:15:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:40 smithi117 ceph-mon[35021]: pgmap v100: 129 pgs: 129 active+clean; 100 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 553 KiB/s wr, 69 op/s 2024-09-15T22:15:41.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 186863 9.29 MB/sec warmup 110 sec latency 11.092 ms 2024-09-15T22:15:41.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:41 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:15:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:41 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:15:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:41 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:15:42.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 189074 9.34 MB/sec warmup 111 sec latency 19.540 ms 2024-09-15T22:15:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:42 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:15:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:42 smithi077 ceph-mon[30980]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:15:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:42 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:15:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:42 smithi196 ceph-mon[35109]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:15:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:42 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:15:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:42 smithi117 ceph-mon[35021]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:15:43.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 192121 9.41 MB/sec warmup 112 sec latency 8.522 ms 2024-09-15T22:15:44.383 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:44 smithi077 ceph-mon[30980]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:44.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:44 smithi196 ceph-mon[35109]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:44.413 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:44 smithi117 ceph-mon[35021]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:44.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 195332 9.49 MB/sec warmup 113 sec latency 11.779 ms 2024-09-15T22:15:45.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 197594 9.48 MB/sec warmup 114 sec latency 21.185 ms 2024-09-15T22:15:46.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 200042 9.54 MB/sec warmup 115 sec latency 13.159 ms 2024-09-15T22:15:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:46 smithi077 ceph-mon[30980]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:46 smithi196 ceph-mon[35109]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:46 smithi117 ceph-mon[35021]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-09-15T22:15:47.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 202850 9.58 MB/sec warmup 116 sec latency 11.347 ms 2024-09-15T22:15:48.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 205889 9.65 MB/sec warmup 117 sec latency 12.500 ms 2024-09-15T22:15:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:48 smithi077 ceph-mon[30980]: pgmap v104: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-09-15T22:15:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:48 smithi196 ceph-mon[35109]: pgmap v104: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-09-15T22:15:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:48 smithi117 ceph-mon[35021]: pgmap v104: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-09-15T22:15:49.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 208335 9.64 MB/sec warmup 118 sec latency 12.359 ms 2024-09-15T22:15:50.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 210526 9.67 MB/sec warmup 119 sec latency 12.465 ms 2024-09-15T22:15:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:50 smithi077 ceph-mon[30980]: pgmap v105: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-15T22:15:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:50 smithi196 ceph-mon[35109]: pgmap v105: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-15T22:15:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:50 smithi117 ceph-mon[35021]: pgmap v105: 129 pgs: 129 active+clean; 116 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-09-15T22:15:52.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 216413 17.49 MB/sec execute 1 sec latency 15.312 ms 2024-09-15T22:15:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:52 smithi077 ceph-mon[30980]: pgmap v106: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-09-15T22:15:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:52 smithi196 ceph-mon[35109]: pgmap v106: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-09-15T22:15:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:52 smithi117 ceph-mon[35021]: pgmap v106: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-09-15T22:15:53.463 INFO:tasks.workunit.client.0.smithi077.stdout: 1 218921 13.47 MB/sec execute 2 sec latency 11.875 ms 2024-09-15T22:15:54.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 221149 13.45 MB/sec execute 3 sec latency 20.805 ms 2024-09-15T22:15:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:54 smithi077 ceph-mon[30980]: pgmap v107: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:54 smithi196 ceph-mon[35109]: pgmap v107: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:54 smithi117 ceph-mon[35021]: pgmap v107: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:55.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 224087 14.36 MB/sec execute 4 sec latency 15.105 ms 2024-09-15T22:15:56.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 227189 15.04 MB/sec execute 5 sec latency 13.071 ms 2024-09-15T22:15:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:56 smithi077 ceph-mon[30980]: pgmap v108: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:56 smithi196 ceph-mon[35109]: pgmap v108: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:56 smithi117 ceph-mon[35021]: pgmap v108: 129 pgs: 129 active+clean; 124 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-15T22:15:57.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 229809 14.59 MB/sec execute 6 sec latency 11.512 ms 2024-09-15T22:15:58.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 232057 14.39 MB/sec execute 7 sec latency 11.121 ms 2024-09-15T22:15:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:15:58 smithi077 ceph-mon[30980]: pgmap v109: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-15T22:15:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:15:58 smithi196 ceph-mon[35109]: pgmap v109: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-15T22:15:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:15:58 smithi117 ceph-mon[35021]: pgmap v109: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-15T22:15:59.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 234752 14.35 MB/sec execute 8 sec latency 13.288 ms 2024-09-15T22:16:00.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 237389 14.27 MB/sec execute 9 sec latency 48.479 ms 2024-09-15T22:16:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:00 smithi077 ceph-mon[30980]: pgmap v110: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:16:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:00 smithi196 ceph-mon[35109]: pgmap v110: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:16:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:00 smithi117 ceph-mon[35021]: pgmap v110: 129 pgs: 129 active+clean; 132 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:16:01.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 238142 13.39 MB/sec execute 10 sec latency 81.473 ms 2024-09-15T22:16:02.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 241069 13.49 MB/sec execute 11 sec latency 12.417 ms 2024-09-15T22:16:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:02 smithi077 ceph-mon[30980]: pgmap v111: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-09-15T22:16:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:02 smithi196 ceph-mon[35109]: pgmap v111: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-09-15T22:16:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:02 smithi117 ceph-mon[35021]: pgmap v111: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 304 op/s 2024-09-15T22:16:03.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 243641 13.34 MB/sec execute 12 sec latency 18.433 ms 2024-09-15T22:16:03.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:03.887 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:04.330 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:04 smithi077 ceph-mon[30980]: pgmap v112: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 -- 172.21.15.77:0/3321838930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4894100b20 msgr2=0x7f4894100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 --2- 172.21.15.77:0/3321838930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4894100b20 0x7f4894100f00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4884009900 tx=0x7f488402a410 comp rx=0 tx=0).stop 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 -- 172.21.15.77:0/3321838930 shutdown_connections 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 --2- 172.21.15.77:0/3321838930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f489410e500 0x7f4894110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 --2- 172.21.15.77:0/3321838930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894101440 0x7f489410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 --2- 172.21.15.77:0/3321838930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4894100b20 0x7f4894100f00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 -- 172.21.15.77:0/3321838930 >> 172.21.15.77:0/3321838930 conn(0x7f48940fc7d0 msgr2=0x7f48940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 -- 172.21.15.77:0/3321838930 shutdown_connections 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.416+0000 7f489a8b2640 1 -- 172.21.15.77:0/3321838930 wait complete. 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 Processor -- start 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 -- start start 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 0x7f48941a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101440 0x7f48941a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4894114280 con 0x7f489410e500 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4894114100 con 0x7f4894100b20 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.417+0000 7f489a8b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4894114400 con 0x7f4894101440 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4893fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 0x7f48941a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58942/0 (socket says 172.21.15.77:58942) 2024-09-15T22:16:04.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 -- 172.21.15.77:0/4113269918 learned_addr learned my addr 172.21.15.77:0/4113269918 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f48937fe640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101440 0x7f48941a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 -- 172.21.15.77:0/4113269918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101440 msgr2=0x7f48941a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101440 0x7f48941a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 -- 172.21.15.77:0/4113269918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 msgr2=0x7f48941a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 0x7f48941a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 -- 172.21.15.77:0/4113269918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489419ace0 con 0x7f489410e500 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4893fff640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 0x7f48941a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f4898e4f640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eb40 tx=0x7f488c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f48917fa640 1 -- 172.21.15.77:0/4113269918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c004b20 con 0x7f489410e500 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489419afd0 con 0x7f489410e500 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f48917fa640 1 -- 172.21.15.77:0/4113269918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c004cc0 con 0x7f489410e500 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f48917fa640 1 -- 172.21.15.77:0/4113269918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c010850 con 0x7f489410e500 2024-09-15T22:16:04.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.418+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941a72c0 con 0x7f489410e500 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.420+0000 7f48917fa640 1 -- 172.21.15.77:0/4113269918 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f488c005020 con 0x7f489410e500 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 Processor -- start 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 -- start start 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f48917fa640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 0x7f487c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f38b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f38b680 0x55c57f38da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c57f2e62e0 con 0x55c57f3804a0 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c57f28dce0 con 0x55c57f380dc0 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f489cc8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c57f2e22a0 con 0x55c57f38b680 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f4893fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f48917fa640 1 -- 172.21.15.77:0/4113269918 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f488c0d07c0 con 0x7f489410e500 2024-09-15T22:16:04.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f4893fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58946/0 (socket says 172.21.15.77:58946) 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f4893fff640 1 -- 172.21.15.77:0/3197076488 learned_addr learned my addr 172.21.15.77:0/3197076488 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f48937fe640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 0x7f487c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.421+0000 7f4898e4f640 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f38b680 0x55c57f38da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f48937fe640 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f38b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 -- 172.21.15.77:0/3197076488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f38b680 msgr2=0x55c57f38da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f38b680 0x55c57f38da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 -- 172.21.15.77:0/3197076488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 msgr2=0x55c57f38b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f38b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 -- 172.21.15.77:0/3197076488 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57f272440 con 0x55c57f3804a0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f48937fe640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 0x7f487c0a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55c57f37d890 tx=0x7f4888009920 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f4893fff640 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4884009900 tx=0x7f4884005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6758e4645fffa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f487a7fc640 1 -- 172.21.15.77:0/3197076488 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884035040 con 0x55c57f3804a0 2024-09-15T22:16:04.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f487a7fc640 1 -- 172.21.15.77:0/3197076488 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884031830 con 0x55c57f3804a0 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.422+0000 7f487a7fc640 1 -- 172.21.15.77:0/3197076488 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48840047a0 con 0x55c57f3804a0 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- 172.21.15.77:0/3197076488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 msgr2=0x55c57f380880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4884009900 tx=0x7f4884005d50 comp rx=0 tx=0).stop 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- 172.21.15.77:0/3197076488 shutdown_connections 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f38b680 0x55c57f38da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f38b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3197076488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f380880 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- 172.21.15.77:0/3197076488 >> 172.21.15.77:0/3197076488 conn(0x55c57f37ca30 msgr2=0x55c57f37ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- 172.21.15.77:0/3197076488 shutdown_connections 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- 172.21.15.77:0/3197076488 wait complete. 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.423+0000 7f489cc8b740 1 -- start start 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 Processor -- start 2024-09-15T22:16:04.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f38af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f388200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c57f38e250 con 0x55c57f38b680 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c57f2a1050 con 0x55c57f380dc0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c57f38e3d0 con 0x55c57f3804a0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f489cc8b740 10 client.0 _ll_register_callbacks cb 0x55c57f2e1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f48937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f388200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58954/0 (socket says 172.21.15.77:58954) 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 -- 172.21.15.77:0/3286915534 learned_addr learned my addr 172.21.15.77:0/3286915534 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4893fff640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f38af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 -- 172.21.15.77:0/3286915534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 msgr2=0x55c57f38af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f38af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 -- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 msgr2=0x55c57f388200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f388200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f4898e4f640 1 -- 172.21.15.77:0/3286915534 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57f0573c0 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.424+0000 7f48937fe640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f388200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f4898e4f640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c7d0 tx=0x7f488c09c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e25d85154a405b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c09ce00 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f488c0d03d0 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4884031830 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f488c01a0c0 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 10 client.16503 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c57f399c00 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884031830 con 0x55c57f38b680 2024-09-15T22:16:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f488c05c3f0 con 0x55c57f38b680 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 10 client.16503 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 20 client.16503 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f39b100 0x55c57f39d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 4 client.16503 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.425+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57f2f3850 con 0x55c57f39b100 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.426+0000 7f4893fff640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f39b100 0x55c57f39d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.426+0000 7f48737fe640 10 client.16503 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.426+0000 7f4893fff640 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f39b100 0x55c57f39d580 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f18997453d162a server_cookie=911baba31ad66681 in_seq=0 out_seq=0 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f48737fe640 1 -- 172.21.15.77:0/3286915534 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c57f2f3850 con 0x55c57f39b100 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f48737fe640 10 client.16503 handle_command_reply: tid=0 2024-09-15T22:16:04.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 client.16503 shutdown 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 msgr2=0x55c57f388740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c7d0 tx=0x7f488c09c5c0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 shutdown_connections 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f39b100 0x55c57f39d580 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f38b680 0x55c57f388740 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f388200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 --2- 172.21.15.77:0/3286915534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f38af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f48737fe640 0 client.16503 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 >> 172.21.15.77:0/3286915534 conn(0x55c57f37ca30 msgr2=0x55c57f37bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 shutdown_connections 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 1 -- 172.21.15.77:0/3286915534 wait complete. 2024-09-15T22:16:04.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.427+0000 7f489cc8b740 20 client.16503 trim_cache size 0 max 16384 2024-09-15T22:16:04.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 Processor -- start 2024-09-15T22:16:04.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 -- start start 2024-09-15T22:16:04.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 0x55c57f3974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f3881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48941b1190 con 0x55c57f380dc0 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48941b2d90 con 0x55c57f37ca30 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f489cc8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48941b1860 con 0x55c57f3804a0 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58962/0 (socket says 172.21.15.77:58962) 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4898e4f640 1 -- 172.21.15.77:0/1709150805 learned_addr learned my addr 172.21.15.77:0/1709150805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 0x55c57f3974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f48937fe640 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f3881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 -- 172.21.15.77:0/1709150805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 msgr2=0x55c57f3881c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f3881c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 -- 172.21.15.77:0/1709150805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 msgr2=0x55c57f3a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 -- 172.21.15.77:0/1709150805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57f3802d0 con 0x55c57f37ca30 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4898e4f640 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.446+0000 7f4893fff640 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 0x55c57f3974b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4884005ec0 tx=0x7f48840053f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1175ecb9e439ef54 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f4879ffb640 1 -- 172.21.15.77:0/1709150805 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884035040 con 0x55c57f37ca30 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f4879ffb640 1 -- 172.21.15.77:0/1709150805 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48840035f0 con 0x55c57f37ca30 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f4879ffb640 1 -- 172.21.15.77:0/1709150805 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884046690 con 0x55c57f37ca30 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 -- 172.21.15.77:0/1709150805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 msgr2=0x55c57f3974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 0x55c57f3974b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4884005ec0 tx=0x7f48840053f0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 -- 172.21.15.77:0/1709150805 shutdown_connections 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f380dc0 0x55c57f3a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f3804a0 0x55c57f3881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.447+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1709150805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f37ca30 0x55c57f3974b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 -- 172.21.15.77:0/1709150805 >> 172.21.15.77:0/1709150805 conn(0x55c57f3888a0 msgr2=0x55c57f3872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 -- 172.21.15.77:0/1709150805 shutdown_connections 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 -- 172.21.15.77:0/1709150805 wait complete. 2024-09-15T22:16:04.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 -- start start 2024-09-15T22:16:04.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 Processor -- start 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.448+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f37ca30 0x55c57f393b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f390ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48941b2600 con 0x55c57f3804a0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48941b1f30 con 0x55c57f380dc0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48941af170 con 0x55c57f37ca30 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f48937fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f390ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4893fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f37ca30 0x55c57f393b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37402/0 (socket says 172.21.15.77:37402) 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 -- 172.21.15.77:0/1914433379 learned_addr learned my addr 172.21.15.77:0/1914433379 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f489cc8b740 10 client.0 _ll_register_callbacks cb 0x7f488c004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:04.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 -- 172.21.15.77:0/1914433379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f37ca30 msgr2=0x55c57f393b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f37ca30 0x55c57f393b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 -- 172.21.15.77:0/1914433379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 msgr2=0x55c57f390ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f390ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 -- 172.21.15.77:0/1914433379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57f3919d0 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f4898e4f640 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f488c0058c0 tx=0x7f488c0ce080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba83138efce8a0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.449+0000 7f48737fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c09ce00 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48840035f0 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f488c09bb50 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f488c085050 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 10 client.25814 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c57f405c10 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48840035f0 con 0x55c57f380dc0 2024-09-15T22:16:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f488c09b5c0 con 0x55c57f380dc0 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 10 client.25814 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 20 client.25814 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f38b680 0x55c57f408470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 4 client.25814 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c57f2edd60 con 0x55c57f38b680 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.450+0000 7f4893fff640 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f38b680 0x55c57f408470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f48737fe640 10 client.25814 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f4893fff640 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f38b680 0x55c57f408470 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0268e988648fad server_cookie=6972d85fbfed1e83 in_seq=0 out_seq=0 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f48737fe640 1 -- 172.21.15.77:0/1914433379 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c57f2edd60 con 0x55c57f38b680 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f48737fe640 10 client.25814 handle_command_reply: tid=0 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 client.25814 shutdown 2024-09-15T22:16:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 msgr2=0x55c57f391410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f488c0058c0 tx=0x7f488c0ce080 comp rx=0 tx=0).stop 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 shutdown_connections 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c57f38b680 0x55c57f408470 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c57f380dc0 0x55c57f391410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c57f3804a0 0x55c57f390ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 --2- 172.21.15.77:0/1914433379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c57f37ca30 0x55c57f393b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.451+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 >> 172.21.15.77:0/1914433379 conn(0x55c57f3888a0 msgr2=0x55c57f389a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f48737fe640 0 client.25814 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 shutdown_connections 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489cc8b740 1 -- 172.21.15.77:0/1914433379 wait complete. 2024-09-15T22:16:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489cc8b740 20 client.25814 trim_cache size 0 max 16384 2024-09-15T22:16:04.461 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:16:04.461 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:16:04.461 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:16:04.461 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 msgr2=0x7f487c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 0x7f487c0a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55c57f37d890 tx=0x7f4888009920 comp rx=0 tx=0).stop 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 msgr2=0x7f489419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.452+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eb40 tx=0x7f488c00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 shutdown_connections 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f487c0a33b0 0x7f487c0a5870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489410e500 0x7f489419a5b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4894101440 0x7f48941a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 --2- 172.21.15.77:0/4113269918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4894100b20 0x7f48941a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 >> 172.21.15.77:0/4113269918 conn(0x7f48940fc7d0 msgr2=0x7f48940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 shutdown_connections 2024-09-15T22:16:04.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:04.453+0000 7f489a8b2640 1 -- 172.21.15.77:0/4113269918 wait complete. 2024-09-15T22:16:04.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 245734 13.30 MB/sec execute 13 sec latency 19.711 ms 2024-09-15T22:16:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:04 smithi196 ceph-mon[35109]: pgmap v112: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:04 smithi117 ceph-mon[35021]: pgmap v112: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:05.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:16:05.359 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:05.464 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248602 13.64 MB/sec execute 14 sec latency 69.018 ms 2024-09-15T22:16:05.624 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:06.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 -- 172.21.15.77:0/1539118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 msgr2=0x7f02f410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1539118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f410b9a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009900 tx=0x7f02e402a480 comp rx=0 tx=0).stop 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 -- 172.21.15.77:0/1539118991 shutdown_connections 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1539118991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 0x7f02f410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1539118991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f410b9a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1539118991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.144+0000 7f02fa80f640 1 -- 172.21.15.77:0/1539118991 >> 172.21.15.77:0/1539118991 conn(0x7f02f40fc610 msgr2=0x7f02f40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- 172.21.15.77:0/1539118991 shutdown_connections 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- 172.21.15.77:0/1539118991 wait complete. 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 Processor -- start 2024-09-15T22:16:06.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- start start 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 0x7f02f419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111dc0 con 0x7f02f41034b0 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111c40 con 0x7f02f410bfc0 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.145+0000 7f02fa80f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111f40 con 0x7f02f4102b90 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58988/0 (socket says 172.21.15.77:58988) 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 -- 172.21.15.77:0/1721878581 learned_addr learned my addr 172.21.15.77:0/1721878581 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f980d640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02fa00e640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 0x7f02f419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 -- 172.21.15.77:0/1721878581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 msgr2=0x7f02f41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:06.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 -- 172.21.15.77:0/1721878581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 msgr2=0x7f02f419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 0x7f02f419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 -- 172.21.15.77:0/1721878581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419d270 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f980d640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02f900c640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006010 tx=0x7f02e4031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e403f030 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e40043d0 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419d500 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4035070 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.146+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a4e40 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.147+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f406a850 con 0x7f02f41034b0 2024-09-15T22:16:06.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.149+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f02e4042030 con 0x7f02f41034b0 2024-09-15T22:16:06.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.149+0000 7f02eaffd640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 0x7f02e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:06.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.150+0000 7f02f980d640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 0x7f02e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:06.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.150+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f02e40f05f0 con 0x7f02f41034b0 2024-09-15T22:16:06.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.150+0000 7f02f980d640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 0x7f02e00a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009fc0 tx=0x7f02f0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:06.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.153+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02e40bc700 con 0x7f02f41034b0 2024-09-15T22:16:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:06 smithi077 ceph-mon[30980]: pgmap v113: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:06.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.383+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02f4102f70 con 0x7f02f41034b0 2024-09-15T22:16:06.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.384+0000 7f02eaffd640 1 -- 172.21.15.77:0/1721878581 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f02f4102f70 con 0x7f02f41034b0 2024-09-15T22:16:06.387 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:16:06.387 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"btime":"2024-09-15T22:14:00:967741+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:14:00.403774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:16:06.387 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-09-15T22:16:06.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 msgr2=0x7f02e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 0x7f02e00a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009fc0 tx=0x7f02f0005f90 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 msgr2=0x7f02f4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006010 tx=0x7f02e4031b20 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 shutdown_connections 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02e00a33b0 0x7f02e00a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02f410bfc0 0x7f02f419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41034b0 0x7f02f4198800 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 --2- 172.21.15.77:0/1721878581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02f4102b90 0x7f02f41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 >> 172.21.15.77:0/1721878581 conn(0x7f02f40fc610 msgr2=0x7f02f40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 shutdown_connections 2024-09-15T22:16:06.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:06.386+0000 7f02fa80f640 1 -- 172.21.15.77:0/1721878581 wait complete. 2024-09-15T22:16:06.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 251103 13.48 MB/sec execute 15 sec latency 36.011 ms 2024-09-15T22:16:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:06 smithi196 ceph-mon[35109]: pgmap v113: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:06 smithi117 ceph-mon[35021]: pgmap v113: 129 pgs: 129 active+clean; 140 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 209 op/s 2024-09-15T22:16:07.403 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:16:07.427 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:07 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1721878581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:07.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 253457 13.47 MB/sec execute 16 sec latency 12.009 ms 2024-09-15T22:16:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:07 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1721878581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:07 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1721878581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:07.665 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:08.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 -- 172.21.15.77:0/1516942227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102570 msgr2=0x7fea5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1516942227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102570 0x7fea5c1029b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009900 tx=0x7fea4c02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 -- 172.21.15.77:0/1516942227 shutdown_connections 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1516942227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102ef0 0x7fea5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1516942227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102570 0x7fea5c1029b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1516942227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 -- 172.21.15.77:0/1516942227 >> 172.21.15.77:0/1516942227 conn(0x7fea5c0fc7d0 msgr2=0x7fea5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 -- 172.21.15.77:0/1516942227 shutdown_connections 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.195+0000 7fea63b2a640 1 -- 172.21.15.77:0/1516942227 wait complete. 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.196+0000 7fea63b2a640 1 Processor -- start 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.196+0000 7fea63b2a640 1 -- start start 2024-09-15T22:16:08.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.196+0000 7fea63b2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.196+0000 7fea63b2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 0x7fea5c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea63b2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 0x7fea5c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea63b2a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c110f80 con 0x7fea5c102ef0 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea63b2a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c110e00 con 0x7fea5c102570 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea63b2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c111100 con 0x7fea5c100e20 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea620c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 0x7fea5c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea620c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 0x7fea5c1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59004/0 (socket says 172.21.15.77:59004) 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea620c7640 1 -- 172.21.15.77:0/1443915588 learned_addr learned my addr 172.21.15.77:0/1443915588 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea610c5640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 0x7fea5c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 -- 172.21.15.77:0/1443915588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 msgr2=0x7fea5c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 0x7fea5c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 -- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 msgr2=0x7fea5c1a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 0x7fea5c1a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 -- 172.21.15.77:0/1443915588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1a48b0 con 0x7fea5c100e20 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea610c5640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 0x7fea5c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea618c6640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c1992f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d6d0 tx=0x7fea5800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.197+0000 7fea52ffd640 1 -- 172.21.15.77:0/1443915588 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58017070 con 0x7fea5c100e20 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.198+0000 7fea52ffd640 1 -- 172.21.15.77:0/1443915588 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58009820 con 0x7fea5c100e20 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.198+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1a4a80 con 0x7fea5c100e20 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.198+0000 7fea52ffd640 1 -- 172.21.15.77:0/1443915588 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58004cd0 con 0x7fea5c100e20 2024-09-15T22:16:08.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.198+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1a4f90 con 0x7fea5c100e20 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.200+0000 7fea52ffd640 1 -- 172.21.15.77:0/1443915588 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fea580099d0 con 0x7fea5c100e20 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 Processor -- start 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 -- start start 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a22b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631a09772f0 con 0x5631a0a115d0 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631a091ecf0 con 0x5631a0a12b40 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea65f03740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631a09732b0 con 0x5631a0a20780 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea618c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea610c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a22b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea620c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea610c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a22b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55062/0 (socket says 172.21.15.77:55062) 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea618c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a11a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59006/0 (socket says 172.21.15.77:59006) 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.201+0000 7fea610c5640 1 -- 172.21.15.77:0/1768500335 learned_addr learned my addr 172.21.15.77:0/1768500335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 -- 172.21.15.77:0/1768500335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 msgr2=0x5631a0a22b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a22b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 -- 172.21.15.77:0/1768500335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 msgr2=0x5631a0a11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea52ffd640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 0x7fea480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 -- 172.21.15.77:0/1768500335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a09033a0 con 0x5631a0a12b40 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 0x7fea480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea620c7640 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a11090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea54009f10 tx=0x7fea540117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b8b97ece52fa3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea52ffd640 1 -- 172.21.15.77:0/1443915588 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fea580cfc00 con 0x7fea5c100e20 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea47fff640 1 -- 172.21.15.77:0/1768500335 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540128d0 con 0x5631a0a12b40 2024-09-15T22:16:08.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea47fff640 1 -- 172.21.15.77:0/1768500335 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5401caa0 con 0x5631a0a12b40 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.202+0000 7fea47fff640 1 -- 172.21.15.77:0/1768500335 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5401b910 con 0x5631a0a12b40 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- 172.21.15.77:0/1768500335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 msgr2=0x5631a0a11090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea620c7640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 0x7fea480a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea54020000 tx=0x7fea54021000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a11090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea54009f10 tx=0x7fea540117b0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- 172.21.15.77:0/1768500335 shutdown_connections 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a22b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 --2- 172.21.15.77:0/1768500335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a11090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- 172.21.15.77:0/1768500335 >> 172.21.15.77:0/1768500335 conn(0x5631a09da310 msgr2=0x5631a0a0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- 172.21.15.77:0/1768500335 shutdown_connections 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- 172.21.15.77:0/1768500335 wait complete. 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.203+0000 7fea65f03740 1 -- start start 2024-09-15T22:16:08.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 Processor -- start 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a1d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 0x5631a0a1d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631a0a23480 con 0x5631a0a20780 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5631a0932060 con 0x5631a0a12b40 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5631a0a23600 con 0x5631a0a115d0 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea65f03740 10 client.0 _ll_register_callbacks cb 0x5631a06858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea618c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea610c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a1d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea618c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55072/0 (socket says 172.21.15.77:55072) 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea610c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a1d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37468/0 (socket says 172.21.15.77:37468) 2024-09-15T22:16:08.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea618c6640 1 -- 172.21.15.77:0/1399456236 learned_addr learned my addr 172.21.15.77:0/1399456236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.204+0000 7fea620c7640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 0x5631a0a1d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 -- 172.21.15.77:0/1399456236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 msgr2=0x5631a0a200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 -- 172.21.15.77:0/1399456236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 msgr2=0x5631a0a1d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a1d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 -- 172.21.15.77:0/1399456236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a06e83c0 con 0x5631a0a20780 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea618c6640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea620c7640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 0x5631a0a1d8e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea54011cb0 tx=0x7fea54009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad246d4bb088a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea457fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:08.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54036060 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5401caa0 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fea5402e070 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea5403d050 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea65f03740 10 client.16527 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631a0a2ee90 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.205+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5401caa0 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fea5400f040 con 0x5631a0a20780 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 10 client.16527 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 10 client.16527 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 20 client.16527 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a30290 0x5631a0a32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 4 client.16527 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a0984860 con 0x5631a0a30290 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea618c6640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a30290 0x5631a0a32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea457fa640 10 client.16527 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.206+0000 7fea618c6640 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a30290 0x5631a0a32680 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c8df0fd035ad7ba server_cookie=cbd9002f77d5bdd1 in_seq=0 out_seq=0 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea457fa640 1 -- 172.21.15.77:0/1399456236 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631a0984860 con 0x5631a0a30290 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea457fa640 10 client.16527 handle_command_reply: tid=0 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 client.16527 shutdown 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 msgr2=0x5631a0a1d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 0x5631a0a1d8e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea54011cb0 tx=0x7fea54009930 comp rx=0 tx=0).stop 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 shutdown_connections 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a30290 0x5631a0a32680 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a20780 0x5631a0a1d8e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a1d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 --2- 172.21.15.77:0/1399456236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a115d0 0x5631a0a200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea457fa640 0 client.16527 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 >> 172.21.15.77:0/1399456236 conn(0x5631a09da310 msgr2=0x5631a0a13930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 shutdown_connections 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 1 -- 172.21.15.77:0/1399456236 wait complete. 2024-09-15T22:16:08.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.207+0000 7fea65f03740 20 client.16527 trim_cache size 0 max 16384 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.221+0000 7fea65f03740 1 Processor -- start 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.221+0000 7fea65f03740 1 -- start start 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a2c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a10de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a1c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1acd10 con 0x5631a0a115d0 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1ae910 con 0x5631a0a12b40 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea65f03740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1ad3e0 con 0x5631a0a20780 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea618c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a2c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea618c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a2c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59030/0 (socket says 172.21.15.77:59030) 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea618c6640 1 -- 172.21.15.77:0/596483490 learned_addr learned my addr 172.21.15.77:0/596483490 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a1c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea610c5640 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a10de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 -- 172.21.15.77:0/596483490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 msgr2=0x5631a0a10de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a10de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 -- 172.21.15.77:0/596483490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 msgr2=0x5631a0a2c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a2c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 -- 172.21.15.77:0/596483490 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a0a1fcf0 con 0x5631a0a20780 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea620c7640 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a1c360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea5401bfe0 tx=0x7fea5401c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cb7cef5bc3c055 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea477fe640 1 -- 172.21.15.77:0/596483490 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54046070 con 0x5631a0a20780 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea477fe640 1 -- 172.21.15.77:0/596483490 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5401caa0 con 0x5631a0a20780 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.222+0000 7fea477fe640 1 -- 172.21.15.77:0/596483490 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54036030 con 0x5631a0a20780 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 -- 172.21.15.77:0/596483490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 msgr2=0x5631a0a1c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a1c360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea5401bfe0 tx=0x7fea5401c400 comp rx=0 tx=0).stop 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 -- 172.21.15.77:0/596483490 shutdown_connections 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a1c360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a10de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 --2- 172.21.15.77:0/596483490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a2c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 -- 172.21.15.77:0/596483490 >> 172.21.15.77:0/596483490 conn(0x5631a09da310 msgr2=0x5631a0a1bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 -- 172.21.15.77:0/596483490 shutdown_connections 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.223+0000 7fea65f03740 1 -- 172.21.15.77:0/596483490 wait complete. 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 -- start start 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 Processor -- start 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a92f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1ae180 con 0x5631a0a115d0 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1adab0 con 0x5631a0a12b40 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1aacf0 con 0x5631a0a20780 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea620c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea618c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a92f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea620c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55086/0 (socket says 172.21.15.77:55086) 2024-09-15T22:16:08.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea620c7640 1 -- 172.21.15.77:0/4058448953 learned_addr learned my addr 172.21.15.77:0/4058448953 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea618c6640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a92f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59034/0 (socket says 172.21.15.77:59034) 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.224+0000 7fea65f03740 10 client.0 _ll_register_callbacks cb 0x7fea5401f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea610c5640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 -- 172.21.15.77:0/4058448953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 msgr2=0x5631a0a93480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a93480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 -- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 msgr2=0x5631a0a92f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a92f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 -- 172.21.15.77:0/4058448953 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a0a245a0 con 0x5631a0a20780 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea620c7640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea5401b4f0 tx=0x7fea54009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ac9ebbad238410 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea457fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54036070 con 0x5631a0a20780 2024-09-15T22:16:08.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5401caa0 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.225+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fea54022050 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea54046070 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 10 client.25765 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631a0a9a710 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5401caa0 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fea54028720 con 0x5631a0a20780 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 10 client.25765 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 10 client.25765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 20 client.25765 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a9ac30 0x5631a0a9d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 4 client.25765 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631a097ed70 con 0x5631a0a9ac30 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea618c6640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a9ac30 0x5631a0a9d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.226+0000 7fea457fa640 10 client.25765 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea618c6640 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a9ac30 0x5631a0a9d020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99a916d27190b982 server_cookie=86a33627efe67536 in_seq=0 out_seq=0 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea457fa640 1 -- 172.21.15.77:0/4058448953 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631a097ed70 con 0x5631a0a9ac30 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea457fa640 10 client.25765 handle_command_reply: tid=0 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 client.25765 shutdown 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 msgr2=0x5631a0a23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea5401b4f0 tx=0x7fea54009930 comp rx=0 tx=0).stop 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 shutdown_connections 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5631a0a9ac30 0x5631a0a9d020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5631a0a20780 0x5631a0a23eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5631a0a12b40 0x5631a0a93480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 --2- 172.21.15.77:0/4058448953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631a0a115d0 0x5631a0a92f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea457fa640 0 client.25765 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 >> 172.21.15.77:0/4058448953 conn(0x5631a09da310 msgr2=0x5631a0a1c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 shutdown_connections 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 1 -- 172.21.15.77:0/4058448953 wait complete. 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.227+0000 7fea65f03740 20 client.25765 trim_cache size 0 max 16384 2024-09-15T22:16:08.239 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:08.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 msgr2=0x7fea480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 0x7fea480a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea54020000 tx=0x7fea54021000 comp rx=0 tx=0).stop 2024-09-15T22:16:08.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 msgr2=0x7fea5c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c1992f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d6d0 tx=0x7fea5800db90 comp rx=0 tx=0).stop 2024-09-15T22:16:08.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 shutdown_connections 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea480a33b0 0x7fea480a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea5c102ef0 0x7fea5c1a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c102570 0x7fea5c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 --2- 172.21.15.77:0/1443915588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea5c100e20 0x7fea5c1992f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 >> 172.21.15.77:0/1443915588 conn(0x7fea5c0fc7d0 msgr2=0x7fea5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 shutdown_connections 2024-09-15T22:16:08.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:08.228+0000 7fea63b2a640 1 -- 172.21.15.77:0/1443915588 wait complete. 2024-09-15T22:16:08.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 255961 13.50 MB/sec execute 17 sec latency 34.853 ms 2024-09-15T22:16:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:08 smithi077 ceph-mon[30980]: pgmap v114: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 314 op/s 2024-09-15T22:16:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:08 smithi196 ceph-mon[35109]: pgmap v114: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 314 op/s 2024-09-15T22:16:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:08 smithi117 ceph-mon[35021]: pgmap v114: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 314 op/s 2024-09-15T22:16:09.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 258950 13.72 MB/sec execute 18 sec latency 37.994 ms 2024-09-15T22:16:09.627 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:16:09.946 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:10 smithi077 ceph-mon[30980]: pgmap v115: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-09-15T22:16:10.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260457 13.35 MB/sec execute 19 sec latency 81.347 ms 2024-09-15T22:16:10.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.461+0000 7fb80029a640 1 -- 172.21.15.77:0/468057958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 msgr2=0x7fb7f810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.461+0000 7fb80029a640 1 --2- 172.21.15.77:0/468057958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f810eac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009ca0 tx=0x7fb7f002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 -- 172.21.15.77:0/468057958 shutdown_connections 2024-09-15T22:16:10.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 --2- 172.21.15.77:0/468057958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f810eac0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 --2- 172.21.15.77:0/468057958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 0x7fb7f8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 --2- 172.21.15.77:0/468057958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 -- 172.21.15.77:0/468057958 >> 172.21.15.77:0/468057958 conn(0x7fb7f80fd8d0 msgr2=0x7fb7f80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 -- 172.21.15.77:0/468057958 shutdown_connections 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 -- 172.21.15.77:0/468057958 wait complete. 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 Processor -- start 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 -- start start 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.462+0000 7fb80029a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 0x7fb7f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb80029a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb80029a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111fa0 con 0x7fb7f8106f40 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb80029a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111e20 con 0x7fb7f8101e10 2024-09-15T22:16:10.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb80029a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8112120 con 0x7fb7f81027c0 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59764/0 (socket says 172.21.15.77:59764) 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 -- 172.21.15.77:0/1346273389 learned_addr learned my addr 172.21.15.77:0/1346273389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe837640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fd835640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 0x7fb7f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 -- 172.21.15.77:0/1346273389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 msgr2=0x7fb7f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 0x7fb7f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 -- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 msgr2=0x7fb7f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 -- 172.21.15.77:0/1346273389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f81a68a0 con 0x7fb7f8101e10 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7fe036640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.463+0000 7fb7eb7fe640 1 -- 172.21.15.77:0/1346273389 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00cd80 con 0x7fb7f8101e10 2024-09-15T22:16:10.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.464+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81a6b90 con 0x7fb7f8101e10 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.464+0000 7fb7eb7fe640 1 -- 172.21.15.77:0/1346273389 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec010910 con 0x7fb7f8101e10 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.464+0000 7fb7eb7fe640 1 -- 172.21.15.77:0/1346273389 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec018980 con 0x7fb7f8101e10 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.464+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a7120 con 0x7fb7f8101e10 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.466+0000 7fb7eb7fe640 1 -- 172.21.15.77:0/1346273389 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb7ec010ac0 con 0x7fb7f8101e10 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.466+0000 7fb802673740 1 Processor -- start 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.466+0000 7fb802673740 1 -- start start 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 0x557096261a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x55709626e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5570961c72f0 con 0x557096262b40 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55709616ecf0 con 0x5570962615d0 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb802673740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5570961c32b0 con 0x55709626c370 2024-09-15T22:16:10.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7eb7fe640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 0x7fb7e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe036640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 0x557096261a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42942/0 (socket says 172.21.15.77:42942) 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 -- 172.21.15.77:0/406768228 learned_addr learned my addr 172.21.15.77:0/406768228 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 0x7fb7e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fd835640 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x55709626e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7eb7fe640 1 -- 172.21.15.77:0/1346273389 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb7ec0d0340 con 0x7fb7f8101e10 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 -- 172.21.15.77:0/406768228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 msgr2=0x55709626e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x55709626e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 -- 172.21.15.77:0/406768228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 msgr2=0x557096261a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 0x557096261a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe837640 1 -- 172.21.15.77:0/406768228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570961533a0 con 0x557096262b40 2024-09-15T22:16:10.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.467+0000 7fb7fe036640 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 0x557096261a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.468+0000 7fb7fe837640 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0004af0 tx=0x7fb7f0003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3663ec24daec2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.468+0000 7fb7fe837640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 0x7fb7e40a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009ca0 tx=0x7fb7f0036000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.468+0000 7fb7e8dea640 1 -- 172.21.15.77:0/406768228 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0031d50 con 0x557096262b40 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.468+0000 7fb7e8dea640 1 -- 172.21.15.77:0/406768228 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f0036070 con 0x557096262b40 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.468+0000 7fb7e8dea640 1 -- 172.21.15.77:0/406768228 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f002fcb0 con 0x557096262b40 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- 172.21.15.77:0/406768228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 msgr2=0x557096261090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0004af0 tx=0x7fb7f0003ac0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- 172.21.15.77:0/406768228 shutdown_connections 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x55709626e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570962615d0 0x557096261a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- 172.21.15.77:0/406768228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096261090 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- 172.21.15.77:0/406768228 >> 172.21.15.77:0/406768228 conn(0x55709622a310 msgr2=0x55709625dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- 172.21.15.77:0/406768228 shutdown_connections 2024-09-15T22:16:10.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- 172.21.15.77:0/406768228 wait complete. 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- start start 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 Processor -- start 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x55709626bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096269650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55709626ee90 con 0x557096262b40 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557096182060 con 0x55709626c370 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55709626f010 con 0x5570962615d0 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb802673740 10 client.0 _ll_register_callbacks cb 0x557095ed58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.469+0000 7fb7fd835640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42944/0 (socket says 172.21.15.77:42944) 2024-09-15T22:16:10.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 -- 172.21.15.77:0/474674245 learned_addr learned my addr 172.21.15.77:0/474674245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fe036640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x55709626bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fe837640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096269650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 -- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 msgr2=0x55709626bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x55709626bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 -- 172.21.15.77:0/474674245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 msgr2=0x557096269650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096269650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 -- 172.21.15.77:0/474674245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557095f383c0 con 0x557096262b40 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7fd835640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4012700 tx=0x7fb7f4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d3314819466786 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4004050 con 0x557096262b40 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb7f40041f0 con 0x557096262b40 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f0036070 con 0x557096262b40 2024-09-15T22:16:10.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb7f4025050 con 0x557096262b40 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb802673740 10 client.16545 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55709627aa10 con 0x557096262b40 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.470+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f0036070 con 0x557096262b40 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb7f4003a40 con 0x557096262b40 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 10 client.16545 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 10 client.16545 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 20 client.16545 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55709627bf20 0x55709627e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 4 client.16545 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570961d4860 con 0x55709627bf20 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb7fe036640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55709627bf20 0x55709627e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb7e1ffb640 10 client.16545 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.471+0000 7fb7fe036640 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55709627bf20 0x55709627e340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=871fef6866c003db server_cookie=6a85c152bc98cedb in_seq=0 out_seq=0 2024-09-15T22:16:10.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/474674245 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570961d4860 con 0x55709627bf20 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb7e1ffb640 10 client.16545 handle_command_reply: tid=0 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 client.16545 shutdown 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 msgr2=0x557096269040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4012700 tx=0x7fb7f4012bc0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 shutdown_connections 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55709627bf20 0x55709627e340 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096269650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096269040 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 --2- 172.21.15.77:0/474674245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x55709626bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 >> 172.21.15.77:0/474674245 conn(0x55709622a310 msgr2=0x557096263930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb7e1ffb640 0 client.16545 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 shutdown_connections 2024-09-15T22:16:10.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.472+0000 7fb802673740 1 -- 172.21.15.77:0/474674245 wait complete. 2024-09-15T22:16:10.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.473+0000 7fb802673740 20 client.16545 trim_cache size 0 max 16384 2024-09-15T22:16:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 Processor -- start 2024-09-15T22:16:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 -- start start 2024-09-15T22:16:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x557096269010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096283c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b1060 con 0x557096262b40 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b2c60 con 0x55709626c370 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb802673740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b1730 con 0x5570962615d0 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42954/0 (socket says 172.21.15.77:42954) 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 -- 172.21.15.77:0/2164749326 learned_addr learned my addr 172.21.15.77:0/2164749326 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fe036640 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x557096269010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fe837640 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096283c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 -- 172.21.15.77:0/2164749326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 msgr2=0x557096269010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x557096269010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 -- 172.21.15.77:0/2164749326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 msgr2=0x557096283c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096283c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.486+0000 7fb7fd835640 1 -- 172.21.15.77:0/2164749326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570962699c0 con 0x557096262b40 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4003e60 tx=0x7fb7f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2af48b35913211 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb7e3fff640 1 -- 172.21.15.77:0/2164749326 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f402d020 con 0x557096262b40 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb7e3fff640 1 -- 172.21.15.77:0/2164749326 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40055a0 con 0x557096262b40 2024-09-15T22:16:10.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb7e3fff640 1 -- 172.21.15.77:0/2164749326 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4003650 con 0x557096262b40 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 -- 172.21.15.77:0/2164749326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 msgr2=0x557096278130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4003e60 tx=0x7fb7f400f790 comp rx=0 tx=0).stop 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 -- 172.21.15.77:0/2164749326 shutdown_connections 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55709626c370 0x557096283c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557096262b40 0x557096278130 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 --2- 172.21.15.77:0/2164749326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5570962615d0 0x557096269010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.487+0000 7fb802673740 1 -- 172.21.15.77:0/2164749326 >> 172.21.15.77:0/2164749326 conn(0x55709622a310 msgr2=0x557096268900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- 172.21.15.77:0/2164749326 shutdown_connections 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- 172.21.15.77:0/2164749326 wait complete. 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- start start 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 Processor -- start 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 0x5570962749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x557096265670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b24d0 con 0x5570962615d0 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b1e00 con 0x557096262b40 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81af040 con 0x55709626c370 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb7fe036640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb7fd835640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 0x5570962749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb7fe036640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42960/0 (socket says 172.21.15.77:42960) 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb7fe036640 1 -- 172.21.15.77:0/2386641918 learned_addr learned my addr 172.21.15.77:0/2386641918 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb7fe837640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x557096265670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.488+0000 7fb802673740 10 client.0 _ll_register_callbacks cb 0x7fb7f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 -- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 msgr2=0x557096265670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x557096265670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 -- 172.21.15.77:0/2386641918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 msgr2=0x5570962749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 0x5570962749c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 -- 172.21.15.77:0/2386641918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557096265d10 con 0x5570962615d0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 0x5570962749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7fe036640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec098d60 tx=0x7fb7ec00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d13a1809386f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:10.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec026040 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb7ec0cd9d0 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f40055a0 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec0cdbd0 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb802673740 10 client.16551 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5570962e65c0 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.489+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40055a0 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb7ec0bc240 con 0x5570962615d0 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 10 client.16551 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 10 client.16551 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 20 client.16551 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5570962e6c40 0x5570962e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 4 client.16551 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:10.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570961ced70 con 0x5570962e6c40 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5570962e6c40 0x5570962e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7e1ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7fd835640 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5570962e6c40 0x5570962e9030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5adf74089d241b1 server_cookie=4d86291d8fd35125 in_seq=0 out_seq=0 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7e1ffb640 1 -- 172.21.15.77:0/2386641918 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570961ced70 con 0x5570962e6c40 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb7e1ffb640 10 client.16551 handle_command_reply: tid=0 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.490+0000 7fb802673740 1 client.16551 shutdown 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 msgr2=0x557096274480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec098d60 tx=0x7fb7ec00cf20 comp rx=0 tx=0).stop 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 shutdown_connections 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5570962e6c40 0x5570962e9030 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb7e1ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55709626c370 0x557096265670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557096262b40 0x5570962749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 --2- 172.21.15.77:0/2386641918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5570962615d0 0x557096274480 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 >> 172.21.15.77:0/2386641918 conn(0x55709622a310 msgr2=0x55709626b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 shutdown_connections 2024-09-15T22:16:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 1 -- 172.21.15.77:0/2386641918 wait complete. 2024-09-15T22:16:10.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.491+0000 7fb802673740 20 client.16551 trim_cache size 0 max 16384 2024-09-15T22:16:10.499 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 msgr2=0x7fb7e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 0x7fb7e40a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009ca0 tx=0x7fb7f0036000 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 msgr2=0x7fb7f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9f0 tx=0x7fb7ec00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 shutdown_connections 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7e40a33b0 0x7fb7e40a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7f8106f40 0x7fb7f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f81027c0 0x7fb7f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 --2- 172.21.15.77:0/1346273389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f8101e10 0x7fb7f80744f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:10.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 >> 172.21.15.77:0/1346273389 conn(0x7fb7f80fd8d0 msgr2=0x7fb7f80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 shutdown_connections 2024-09-15T22:16:10.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:10.492+0000 7fb80029a640 1 -- 172.21.15.77:0/1346273389 wait complete. 2024-09-15T22:16:10.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:10 smithi196 ceph-mon[35109]: pgmap v115: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-09-15T22:16:10.625 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:10 smithi117 ceph-mon[35021]: pgmap v115: 129 pgs: 129 active+clean; 157 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-09-15T22:16:11.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:16:11.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 263057 13.39 MB/sec execute 20 sec latency 10.902 ms 2024-09-15T22:16:11.699 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.204+0000 7f93fcc23640 1 -- 172.21.15.77:0/1815254459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8102570 msgr2=0x7f93f81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.204+0000 7f93fcc23640 1 --2- 172.21.15.77:0/1815254459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8102570 0x7f93f81029b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 -- 172.21.15.77:0/1815254459 shutdown_connections 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 --2- 172.21.15.77:0/1815254459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102ef0 0x7f93f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 --2- 172.21.15.77:0/1815254459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8102570 0x7f93f81029b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 --2- 172.21.15.77:0/1815254459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8100e20 0x7f93f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 -- 172.21.15.77:0/1815254459 >> 172.21.15.77:0/1815254459 conn(0x7f93f80fc7d0 msgr2=0x7f93f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 -- 172.21.15.77:0/1815254459 shutdown_connections 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 -- 172.21.15.77:0/1815254459 wait complete. 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.205+0000 7f93fcc23640 1 Processor -- start 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 -- start start 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102570 0x7f93f819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8102ef0 0x7f93f819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8110f50 con 0x7f93f8100e20 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8110dd0 con 0x7f93f8102570 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93fcc23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81110d0 con 0x7f93f8102ef0 2024-09-15T22:16:12.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42980/0 (socket says 172.21.15.77:42980) 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 -- 172.21.15.77:0/4000355039 learned_addr learned my addr 172.21.15.77:0/4000355039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102570 0x7f93f819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8102ef0 0x7f93f819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 -- 172.21.15.77:0/4000355039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8102ef0 msgr2=0x7f93f819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8102ef0 0x7f93f819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 -- 172.21.15.77:0/4000355039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102570 msgr2=0x7f93f819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102570 0x7f93f819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 -- 172.21.15.77:0/4000355039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819f690 con 0x7f93f8100e20 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93f659c640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f93e800e9f0 tx=0x7f93e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.206+0000 7f93e77fe640 1 -- 172.21.15.77:0/4000355039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e800cd80 con 0x7f93f8100e20 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.207+0000 7f93e77fe640 1 -- 172.21.15.77:0/4000355039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e8010910 con 0x7f93f8100e20 2024-09-15T22:16:12.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.207+0000 7f93e77fe640 1 -- 172.21.15.77:0/4000355039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8018990 con 0x7f93f8100e20 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.207+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819f980 con 0x7f93f8100e20 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.207+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a7260 con 0x7f93f8100e20 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.209+0000 7f93e77fe640 1 -- 172.21.15.77:0/4000355039 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f93e8010ac0 con 0x7f93f8100e20 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.209+0000 7f93feffc740 1 Processor -- start 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.209+0000 7f93feffc740 1 -- start start 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 0x5649785ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5649785542f0 con 0x5649785f9370 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5649784fbcf0 con 0x5649785efb40 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93feffc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649785502b0 con 0x5649785ee5d0 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93e77fe640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 0x7f93e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 0x5649785ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 0x5649785ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59810/0 (socket says 172.21.15.77:59810) 2024-09-15T22:16:12.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f6d9d640 1 -- 172.21.15.77:0/561182545 learned_addr learned my addr 172.21.15.77:0/561182545 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:12.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43272/0 (socket says 172.21.15.77:43272) 2024-09-15T22:16:12.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 0x7f93e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93e77fe640 1 -- 172.21.15.77:0/4000355039 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f93e80d0370 con 0x7f93f8100e20 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 -- 172.21.15.77:0/561182545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 msgr2=0x5649785ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 0x5649785ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 -- 172.21.15.77:0/561182545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 msgr2=0x5649785fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 -- 172.21.15.77:0/561182545 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649784e03a0 con 0x5649785ee5d0 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f659c640 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93e80cd780 tx=0x7f93e80c70e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9003a8a7805ed62f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.210+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 0x7f93e00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93f0005f40 tx=0x7f93f0009c50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93e4ff9640 1 -- 172.21.15.77:0/561182545 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e80cda00 con 0x5649785ee5d0 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93e4ff9640 1 -- 172.21.15.77:0/561182545 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e8098a30 con 0x5649785ee5d0 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93e4ff9640 1 -- 172.21.15.77:0/561182545 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e809ba70 con 0x5649785ee5d0 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 -- 172.21.15.77:0/561182545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 msgr2=0x5649785eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93e80cd780 tx=0x7f93e80c70e0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 -- 172.21.15.77:0/561182545 shutdown_connections 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x5649785eea10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 --2- 172.21.15.77:0/561182545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785efb40 0x5649785ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.211+0000 7f93feffc740 1 -- 172.21.15.77:0/561182545 >> 172.21.15.77:0/561182545 conn(0x5649785b7310 msgr2=0x5649785eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- 172.21.15.77:0/561182545 shutdown_connections 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- 172.21.15.77:0/561182545 wait complete. 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- start start 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 Processor -- start 2024-09-15T22:16:12.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649785f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785efb40 0x5649785f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5649785fc010 con 0x5649785f9370 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56497850f060 con 0x5649785ee5d0 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649785fc190 con 0x5649785efb40 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93feffc740 10 client.0 _ll_register_callbacks cb 0x5649782628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93f6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93f6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42998/0 (socket says 172.21.15.77:42998) 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785efb40 0x5649785f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93f6d9d640 1 -- 172.21.15.77:0/1937211814 learned_addr learned my addr 172.21.15.77:0/1937211814 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:12.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.212+0000 7f93f659c640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649785f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 -- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785efb40 msgr2=0x5649785f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785efb40 0x5649785f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 -- 172.21.15.77:0/1937211814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 msgr2=0x5649785f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649785f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 -- 172.21.15.77:0/1937211814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649782c53c0 con 0x5649785f9370 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f93f001f8e0 tx=0x7f93f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded68d73b46ab499 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0026070 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e8098a30 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f93f000ed70 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f93f0033050 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93feffc740 10 client.16563 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564978607b40 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.213+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e8098a30 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f93f0004410 con 0x5649785f9370 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 10 client.16563 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 10 client.16563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:12.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 20 client.16563 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x56497860b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 4 client.16563 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564978561860 con 0x564978609000 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93f659c640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x56497860b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93f659c640 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x56497860b450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e77b4320fcff0a1e server_cookie=91f34cc6309fe47c in_seq=0 out_seq=0 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.214+0000 7f93ddffb640 10 client.16563 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.215+0000 7f93ddffb640 1 -- 172.21.15.77:0/1937211814 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564978561860 con 0x564978609000 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.215+0000 7f93ddffb640 10 client.16563 handle_command_reply: tid=0 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.215+0000 7f93feffc740 1 client.16563 shutdown 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 msgr2=0x5649785f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f93f001f8e0 tx=0x7f93f0009b80 comp rx=0 tx=0).stop 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 shutdown_connections 2024-09-15T22:16:12.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x56497860b450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93ddffb640 0 client.16563 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785f9370 0x5649785f67e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785efb40 0x5649785f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 --2- 172.21.15.77:0/1937211814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649785f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 >> 172.21.15.77:0/1937211814 conn(0x5649785b7310 msgr2=0x5649785f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 shutdown_connections 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 1 -- 172.21.15.77:0/1937211814 wait complete. 2024-09-15T22:16:12.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.216+0000 7f93feffc740 20 client.16563 trim_cache size 0 max 16384 2024-09-15T22:16:12.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 Processor -- start 2024-09-15T22:16:12.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 -- start start 2024-09-15T22:16:12.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649786052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785f9370 0x5649785f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b10f0 con 0x5649785efb40 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b2cf0 con 0x5649785ee5d0 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.229+0000 7f93feffc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b17c0 con 0x5649785f9370 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43010/0 (socket says 172.21.15.77:43010) 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 -- 172.21.15.77:0/1263273421 learned_addr learned my addr 172.21.15.77:0/1263273421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785f9370 0x5649785f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f659c640 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649786052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 -- 172.21.15.77:0/1263273421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785f9370 msgr2=0x5649785f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785f9370 0x5649785f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 -- 172.21.15.77:0/1263273421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 msgr2=0x5649786052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649786052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 -- 172.21.15.77:0/1263273421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649785fbad0 con 0x5649785efb40 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f93ec02a9a0 tx=0x7f93ec004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a135a61d34c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93dffff640 1 -- 172.21.15.77:0/1263273421 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec035040 con 0x5649785efb40 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93dffff640 1 -- 172.21.15.77:0/1263273421 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec004330 con 0x5649785efb40 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.230+0000 7f93dffff640 1 -- 172.21.15.77:0/1263273421 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec031a00 con 0x5649785efb40 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- 172.21.15.77:0/1263273421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 msgr2=0x5649785f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f93ec02a9a0 tx=0x7f93ec004180 comp rx=0 tx=0).stop 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- 172.21.15.77:0/1263273421 shutdown_connections 2024-09-15T22:16:12.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785f9370 0x5649785f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f1120 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 --2- 172.21.15.77:0/1263273421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785ee5d0 0x5649786052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- 172.21.15.77:0/1263273421 >> 172.21.15.77:0/1263273421 conn(0x5649785b7310 msgr2=0x5649785f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- 172.21.15.77:0/1263273421 shutdown_connections 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- 172.21.15.77:0/1263273421 wait complete. 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.231+0000 7f93feffc740 1 -- start start 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 Processor -- start 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x564978601c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785f9370 0x5649785f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b2560 con 0x5649785efb40 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b1e90 con 0x5649785f9370 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81af0d0 con 0x5649785ee5d0 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93feffc740 10 client.0 _ll_register_callbacks cb 0x7f93f001f7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x564978601c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x564978601c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43312/0 (socket says 172.21.15.77:43312) 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43018/0 (socket says 172.21.15.77:43018) 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f659c640 1 -- 172.21.15.77:0/2160131478 learned_addr learned my addr 172.21.15.77:0/2160131478 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f6d9d640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785f9370 0x5649785f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 -- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 msgr2=0x564978601c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x564978601c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 -- 172.21.15.77:0/2160131478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785f9370 msgr2=0x5649785f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785f9370 0x5649785f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 -- 172.21.15.77:0/2160131478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564978672cf0 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93f5d9b640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x5649785ef090 tx=0x7f93ec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8647def697e0ec09 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.232+0000 7f93ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec035030 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f93ec032d50 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec004330 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f93ec056050 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 10 client.16569 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564978673e40 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec004330 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f93ec004be0 con 0x5649785efb40 2024-09-15T22:16:12.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 10 client.16569 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 20 client.16569 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x5649786764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 4 client.16569 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56497855bd70 con 0x564978609000 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.233+0000 7f93f659c640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x5649786764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93ddffb640 10 client.16569 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93f659c640 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x5649786764f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5da3c7b36089fb09 server_cookie=9125e14ed740ee44 in_seq=0 out_seq=0 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93ddffb640 1 -- 172.21.15.77:0/2160131478 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56497855bd70 con 0x564978609000 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93ddffb640 10 client.16569 handle_command_reply: tid=0 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 client.16569 shutdown 2024-09-15T22:16:12.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 msgr2=0x5649785f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x5649785ef090 tx=0x7f93ec031400 comp rx=0 tx=0).stop 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 shutdown_connections 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x564978609000 0x5649786764f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649785f9370 0x5649785f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649785efb40 0x5649785f2f40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 --2- 172.21.15.77:0/2160131478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649785ee5d0 0x564978601c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93ddffb640 0 client.16569 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 >> 172.21.15.77:0/2160131478 conn(0x5649785b7310 msgr2=0x5649785e94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 shutdown_connections 2024-09-15T22:16:12.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 1 -- 172.21.15.77:0/2160131478 wait complete. 2024-09-15T22:16:12.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.234+0000 7f93feffc740 20 client.16569 trim_cache size 0 max 16384 2024-09-15T22:16:12.249 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 msgr2=0x7f93e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 0x7f93e00a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93f0005f40 tx=0x7f93f0009c50 comp rx=0 tx=0).stop 2024-09-15T22:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 msgr2=0x7f93f819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f93e800e9f0 tx=0x7f93e800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 shutdown_connections 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93e00a33b0 0x7f93e00a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93f8102ef0 0x7f93f819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93f8102570 0x7f93f819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 --2- 172.21.15.77:0/4000355039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93f8100e20 0x7f93f819a6e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 >> 172.21.15.77:0/4000355039 conn(0x7f93f80fc7d0 msgr2=0x7f93f806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 shutdown_connections 2024-09-15T22:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:12.235+0000 7f93fcc23640 1 -- 172.21.15.77:0/4000355039 wait complete. 2024-09-15T22:16:12.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 266144 13.59 MB/sec execute 21 sec latency 13.867 ms 2024-09-15T22:16:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:12 smithi077 ceph-mon[30980]: pgmap v116: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-09-15T22:16:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:12 smithi196 ceph-mon[35109]: pgmap v116: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-09-15T22:16:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:12 smithi117 ceph-mon[35021]: pgmap v116: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 333 op/s 2024-09-15T22:16:13.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 268827 13.54 MB/sec execute 22 sec latency 10.730 ms 2024-09-15T22:16:13.618 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:16:13.919 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:14.309 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:14 smithi077 ceph-mon[30980]: pgmap v117: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:14.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/719767673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 msgr2=0x7fe5e81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/719767673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e81067d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009900 tx=0x7fe5d002a410 comp rx=0 tx=0).stop 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/719767673 shutdown_connections 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/719767673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8106f00 0x7fe5e810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/719767673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e81067d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/719767673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8101dd0 0x7fe5e81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/719767673 >> 172.21.15.77:0/719767673 conn(0x7fe5e80fd8b0 msgr2=0x7fe5e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/719767673 shutdown_connections 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.399+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/719767673 wait complete. 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 Processor -- start 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 -- start start 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 0x7fe5e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 0x7fe5e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8111f60 con 0x7fe5e8102780 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8111de0 con 0x7fe5e8106f00 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5ed6b3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81120e0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 0x7fe5e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43040/0 (socket says 172.21.15.77:43040) 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5dffff640 1 -- 172.21.15.77:0/4142715840 learned_addr learned my addr 172.21.15.77:0/4142715840 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 0x7fe5e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/4142715840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 msgr2=0x7fe5e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 0x7fe5e8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 msgr2=0x7fe5e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/4142715840 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e81a68e0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5dffff640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.400+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 0x7fe5e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 0x7fe5e80744f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9f0 tx=0x7fe5d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5e4ff9640 1 -- 172.21.15.77:0/4142715840 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800ccc0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e81a6bd0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5e4ff9640 1 -- 172.21.15.77:0/4142715840 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8010910 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5e4ff9640 1 -- 172.21.15.77:0/4142715840 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8018960 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.401+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e81a7160 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.403+0000 7fe5e4ff9640 1 -- 172.21.15.77:0/4142715840 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe5d8010ac0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 Processor -- start 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 -- start start 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d266090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d266a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56111d1cc2f0 con 0x56111d271370 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56111d173cf0 con 0x56111d2665d0 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56111d1c82b0 con 0x56111d267b40 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d266a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43056/0 (socket says 172.21.15.77:43056) 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5dffff640 1 -- 172.21.15.77:0/403496397 learned_addr learned my addr 172.21.15.77:0/403496397 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5e77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d266090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.404+0000 7fe5e4ff9640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 0x7fe5d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 -- 172.21.15.77:0/403496397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 msgr2=0x56111d266090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d266090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 -- 172.21.15.77:0/403496397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 msgr2=0x56111d266a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 0x7fe5d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5e4ff9640 1 -- 172.21.15.77:0/4142715840 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe5d80d03d0 con 0x7fe5e8101dd0 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d266a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 -- 172.21.15.77:0/403496397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d1583a0 con 0x56111d271370 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d266a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dffff640 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000c00 tx=0x7fe5d0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa4d7ae0e05c5df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 0x7fe5d40a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0005c40 tx=0x7fe5e0004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dd7fa640 1 -- 172.21.15.77:0/403496397 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0040070 con 0x56111d271370 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dd7fa640 1 -- 172.21.15.77:0/403496397 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0031970 con 0x56111d271370 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.405+0000 7fe5dd7fa640 1 -- 172.21.15.77:0/403496397 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0035040 con 0x56111d271370 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 -- 172.21.15.77:0/403496397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 msgr2=0x56111d273760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000c00 tx=0x7fe5d0005dd0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 -- 172.21.15.77:0/403496397 shutdown_connections 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d271370 0x56111d273760 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d266a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/403496397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d266090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 -- 172.21.15.77:0/403496397 >> 172.21.15.77:0/403496397 conn(0x56111d22f310 msgr2=0x56111d262c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 -- 172.21.15.77:0/403496397 shutdown_connections 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.406+0000 7fe5efa8c740 1 -- 172.21.15.77:0/403496397 wait complete. 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 -- start start 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 Processor -- start 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d270d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d271370 0x56111d26e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56111d274010 con 0x56111d267b40 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56111d187060 con 0x56111d2665d0 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56111d274190 con 0x56111d271370 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5efa8c740 10 client.0 _ll_register_callbacks cb 0x56111ceda8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43064/0 (socket says 172.21.15.77:43064) 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 -- 172.21.15.77:0/1558300478 learned_addr learned my addr 172.21.15.77:0/1558300478 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:14.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d270d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d271370 0x56111d26e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 -- 172.21.15.77:0/1558300478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d271370 msgr2=0x56111d26e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d271370 0x56111d26e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 -- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 msgr2=0x56111d270d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d270d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5dffff640 1 -- 172.21.15.77:0/1558300478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111cf3d3c0 con 0x56111d267b40 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.407+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d270d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5dffff640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005ab0 tx=0x7fe5d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526d4b5441255f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d004b020 con 0x56111d267b40 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe5d00035f0 con 0x56111d267b40 2024-09-15T22:16:14.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d0031970 con 0x56111d267b40 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5d0056050 con 0x56111d267b40 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 10 client.16581 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56111d27fa60 con 0x56111d267b40 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0031970 con 0x56111d267b40 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe5d0004b90 con 0x56111d267b40 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 10 client.16581 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 20 client.16581 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d280db0 0x56111d2831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 4 client.16581 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.408+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56111d1d9860 con 0x56111d280db0 2024-09-15T22:16:14.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.409+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d280db0 0x56111d2831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.409+0000 7fe5ce7fc640 10 client.16581 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.409+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d280db0 0x56111d2831a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f79cd076d207c9b server_cookie=5811b62e0559943b in_seq=0 out_seq=0 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.409+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/1558300478 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56111d1d9860 con 0x56111d280db0 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.409+0000 7fe5ce7fc640 10 client.16581 handle_command_reply: tid=0 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 client.16581 shutdown 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 msgr2=0x56111d26dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005ab0 tx=0x7fe5d0004270 comp rx=0 tx=0).stop 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 shutdown_connections 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d280db0 0x56111d2831a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d271370 0x56111d26e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d267b40 0x56111d26dfd0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/1558300478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d2665d0 0x56111d270d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 >> 172.21.15.77:0/1558300478 conn(0x56111d22f310 msgr2=0x56111d268960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5ce7fc640 0 client.16581 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 shutdown_connections 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 1 -- 172.21.15.77:0/1558300478 wait complete. 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.410+0000 7fe5efa8c740 20 client.16581 trim_cache size 0 max 16384 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 Processor -- start 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 -- start start 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d268fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d269500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b0fe0 con 0x56111d2665d0 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b2be0 con 0x56111d271370 2024-09-15T22:16:14.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b16b0 con 0x56111d267b40 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43072/0 (socket says 172.21.15.77:43072) 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/3197706929 learned_addr learned my addr 172.21.15.77:0/3197706929 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.423+0000 7fe5dffff640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d268fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d269500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/3197706929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 msgr2=0x56111d268fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d268fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/3197706929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 msgr2=0x56111d269500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d269500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/3197706929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d263960 con 0x56111d2665d0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d269500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80cdde0 tx=0x7fe5d80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7094de0c38aa0f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5dcff9640 1 -- 172.21.15.77:0/3197706929 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d809c420 con 0x56111d2665d0 2024-09-15T22:16:14.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5dcff9640 1 -- 172.21.15.77:0/3197706929 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80a0720 con 0x56111d2665d0 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.424+0000 7fe5dcff9640 1 -- 172.21.15.77:0/3197706929 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d809b870 con 0x56111d2665d0 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- 172.21.15.77:0/3197706929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 msgr2=0x56111d27d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80cdde0 tx=0x7fe5d80bc040 comp rx=0 tx=0).stop 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- 172.21.15.77:0/3197706929 shutdown_connections 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d269500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d268fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/3197706929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d27d2b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- 172.21.15.77:0/3197706929 >> 172.21.15.77:0/3197706929 conn(0x56111d22f310 msgr2=0x56111d26d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- 172.21.15.77:0/3197706929 shutdown_connections 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- 172.21.15.77:0/3197706929 wait complete. 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 -- start start 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.425+0000 7fe5efa8c740 1 Processor -- start 2024-09-15T22:16:14.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d279cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d26abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b2450 con 0x56111d2665d0 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b1d80 con 0x56111d271370 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81aefc0 con 0x56111d267b40 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5efa8c740 10 client.0 _ll_register_callbacks cb 0x7fe5d80d3df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d279cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43086/0 (socket says 172.21.15.77:43086) 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/2277785866 learned_addr learned my addr 172.21.15.77:0/2277785866 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.426+0000 7fe5e77fe640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d26abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/2277785866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 msgr2=0x56111d279cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d279cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 msgr2=0x56111d26abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d26abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 -- 172.21.15.77:0/2277785866 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d26b400 con 0x56111d2665d0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5dffff640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d279cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5e6ffd640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80a4ab0 tx=0x7fe5d809a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc636c481529d903 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80d0170 con 0x56111d2665d0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe5d809b870 con 0x56111d2665d0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d80a0720 con 0x56111d2665d0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5d809c420 con 0x56111d2665d0 2024-09-15T22:16:14.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5efa8c740 10 client.16587 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56111d2eb460 con 0x56111d2665d0 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80a0720 con 0x56111d2665d0 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.427+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe5d8098170 con 0x56111d2665d0 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 10 client.16587 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 10 client.16587 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 20 client.16587 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d2eba80 0x56111d2ede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 4 client.16587 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56111d1d3d70 con 0x56111d2eba80 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5dffff640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d2eba80 0x56111d2ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:14.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5ce7fc640 10 client.16587 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5dffff640 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d2eba80 0x56111d2ede70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d8921acd5dff504 server_cookie=c4757938e701962 in_seq=0 out_seq=0 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5ce7fc640 1 -- 172.21.15.77:0/2277785866 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56111d1d3d70 con 0x56111d2eba80 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5ce7fc640 10 client.16587 handle_command_reply: tid=0 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.428+0000 7fe5efa8c740 1 client.16587 shutdown 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 msgr2=0x56111d2797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80a4ab0 tx=0x7fe5d809a890 comp rx=0 tx=0).stop 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 shutdown_connections 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56111d2eba80 0x56111d2ede70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56111d271370 0x56111d26abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56111d267b40 0x56111d279cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 --2- 172.21.15.77:0/2277785866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56111d2665d0 0x56111d2797b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5ce7fc640 0 client.16587 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 >> 172.21.15.77:0/2277785866 conn(0x56111d22f310 msgr2=0x56111d2614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 shutdown_connections 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 1 -- 172.21.15.77:0/2277785866 wait complete. 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5efa8c740 20 client.16587 trim_cache size 0 max 16384 2024-09-15T22:16:14.449 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:14.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.429+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 msgr2=0x7fe5d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 0x7fe5d40a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0005c40 tx=0x7fe5e0004a40 comp rx=0 tx=0).stop 2024-09-15T22:16:14.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 msgr2=0x7fe5e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:14.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 0x7fe5e80744f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9f0 tx=0x7fe5d800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 shutdown_connections 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5d40a33b0 0x7fe5d40a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e8106f00 0x7fe5e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5e8102780 0x7fe5e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 --2- 172.21.15.77:0/4142715840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5e8101dd0 0x7fe5e80744f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 >> 172.21.15.77:0/4142715840 conn(0x7fe5e80fd8b0 msgr2=0x7fe5e80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 shutdown_connections 2024-09-15T22:16:14.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:14.430+0000 7fe5ed6b3640 1 -- 172.21.15.77:0/4142715840 wait complete. 2024-09-15T22:16:14.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 271037 13.52 MB/sec execute 23 sec latency 29.587 ms 2024-09-15T22:16:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:14 smithi196 ceph-mon[35109]: pgmap v117: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:14 smithi117 ceph-mon[35021]: pgmap v117: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:15.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 273523 13.54 MB/sec execute 24 sec latency 56.609 ms 2024-09-15T22:16:15.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:16:15.878 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:16.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:16 smithi077 ceph-mon[30980]: pgmap v118: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.368+0000 7f639e84c640 1 -- 172.21.15.77:0/2085914408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 msgr2=0x7f6398101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.368+0000 7f639e84c640 1 --2- 172.21.15.77:0/2085914408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398101200 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f638800fb10 tx=0x7f638802a200 comp rx=0 tx=0).stop 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 -- 172.21.15.77:0/2085914408 shutdown_connections 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 --2- 172.21.15.77:0/2085914408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102ef0 0x7f639810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 --2- 172.21.15.77:0/2085914408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102570 0x7f63981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 --2- 172.21.15.77:0/2085914408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398101200 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 -- 172.21.15.77:0/2085914408 >> 172.21.15.77:0/2085914408 conn(0x7f63980fc7d0 msgr2=0x7f63980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 -- 172.21.15.77:0/2085914408 shutdown_connections 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.369+0000 7f639e84c640 1 -- 172.21.15.77:0/2085914408 wait complete. 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.370+0000 7f639e84c640 1 Processor -- start 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.370+0000 7f639e84c640 1 -- start start 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102570 0x7f6398073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102ef0 0x7f639806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f639810fec0 con 0x7f6398100e20 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f639810fd40 con 0x7f6398102570 2024-09-15T22:16:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f639e84c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6398110040 con 0x7f6398102ef0 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f6397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f6397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43098/0 (socket says 172.21.15.77:43098) 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.371+0000 7f6397fff640 1 -- 172.21.15.77:0/2230945045 learned_addr learned my addr 172.21.15.77:0/2230945045 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.372+0000 7f638ffff640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102570 0x7f6398073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.372+0000 7f639cde9640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102ef0 0x7f639806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 -- 172.21.15.77:0/2230945045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102ef0 msgr2=0x7f639806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102ef0 0x7f639806cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 -- 172.21.15.77:0/2230945045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102570 msgr2=0x7f6398073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102570 0x7f6398073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 -- 172.21.15.77:0/2230945045 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639806d4c0 con 0x7f6398100e20 2024-09-15T22:16:16.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6397fff640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6388009230 tx=0x7f6388005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.373+0000 7f6395ffb640 1 -- 172.21.15.77:0/2230945045 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388004030 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.374+0000 7f6395ffb640 1 -- 172.21.15.77:0/2230945045 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6388004db0 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.374+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639806d750 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.374+0000 7f6395ffb640 1 -- 172.21.15.77:0/2230945045 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638800ec20 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.374+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6398071740 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.376+0000 7f6395ffb640 1 -- 172.21.15.77:0/2230945045 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6388030030 con 0x7f6398100e20 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.376+0000 7f63a0c25740 1 Processor -- start 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.376+0000 7f63a0c25740 1 -- start start 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.376+0000 7f6395ffb640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 0x7f63840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.376+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6c0370 0x55dddd6c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f63a0c25740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dddd61b2f0 con 0x55dddd6b6b40 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f63a0c25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dddd5c2cf0 con 0x55dddd6b55d0 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f63a0c25740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dddd6172b0 con 0x55dddd6c0370 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6395ffb640 1 -- 172.21.15.77:0/2230945045 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f63880efe00 con 0x7f6398100e20 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f639cde9640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 0x7f63840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43100/0 (socket says 172.21.15.77:43100) 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 -- 172.21.15.77:0/2125103042 learned_addr learned my addr 172.21.15.77:0/2125103042 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f639cde9640 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6c0370 0x55dddd6c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f638ffff640 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 -- 172.21.15.77:0/2125103042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6c0370 msgr2=0x55dddd6c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6c0370 0x55dddd6c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 -- 172.21.15.77:0/2125103042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 msgr2=0x55dddd6b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 -- 172.21.15.77:0/2125103042 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddd5a73a0 con 0x55dddd6b6b40 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f6397fff640 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f63880c45f0 tx=0x7f638802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427756ff4804ed7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f639cde9640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 0x7f63840a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f6390009df0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f638e7fc640 1 -- 172.21.15.77:0/2125103042 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638800f730 con 0x55dddd6b6b40 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f638e7fc640 1 -- 172.21.15.77:0/2125103042 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638802e020 con 0x55dddd6b6b40 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.377+0000 7f638e7fc640 1 -- 172.21.15.77:0/2125103042 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388004360 con 0x55dddd6b6b40 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 -- 172.21.15.77:0/2125103042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 msgr2=0x55dddd6b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f63880c45f0 tx=0x7f638802a200 comp rx=0 tx=0).stop 2024-09-15T22:16:16.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 -- 172.21.15.77:0/2125103042 shutdown_connections 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6c0370 0x55dddd6c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 --2- 172.21.15.77:0/2125103042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b6b40 0x55dddd6b5090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 -- 172.21.15.77:0/2125103042 >> 172.21.15.77:0/2125103042 conn(0x55dddd67e310 msgr2=0x55dddd6b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 -- 172.21.15.77:0/2125103042 shutdown_connections 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.378+0000 7f63a0c25740 1 -- 172.21.15.77:0/2125103042 wait complete. 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 -- start start 2024-09-15T22:16:16.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 Processor -- start 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b55d0 0x55dddd6bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6c0370 0x55dddd6bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dddd6c2e30 con 0x55dddd6b55d0 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dddd5d6060 con 0x55dddd6c0370 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dddd6c2fb0 con 0x55dddd6b6b40 2024-09-15T22:16:16.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.379+0000 7f63a0c25740 10 client.0 _ll_register_callbacks cb 0x55dddd3298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:16.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43364/0 (socket says 172.21.15.77:43364) 2024-09-15T22:16:16.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 -- 172.21.15.77:0/364171831 learned_addr learned my addr 172.21.15.77:0/364171831 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:16.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f6397fff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b55d0 0x55dddd6bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f639cde9640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6c0370 0x55dddd6bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 -- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6c0370 msgr2=0x55dddd6bd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6c0370 0x55dddd6bd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 -- 172.21.15.77:0/364171831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b55d0 msgr2=0x55dddd6bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b55d0 0x55dddd6bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 -- 172.21.15.77:0/364171831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddd38c3c0 con 0x55dddd6b6b40 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f638ffff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f638000bac0 tx=0x7f638000bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eafdf8e7e3cc6e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f63777fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380024020 con 0x55dddd6b6b40 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638802e020 con 0x55dddd6b6b40 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6380004050 con 0x55dddd6b6b40 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.380+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f638002f050 con 0x55dddd6b6b40 2024-09-15T22:16:16.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 10 client.25831 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dddd6ce8a0 con 0x55dddd6b6b40 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638802e020 con 0x55dddd6b6b40 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6380020450 con 0x55dddd6b6b40 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 10 client.25831 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 10 client.25831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 20 client.25831 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd6cfd00 0x55dddd6d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 4 client.25831 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:16.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dddd628860 con 0x55dddd6cfd00 2024-09-15T22:16:16.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f6397fff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd6cfd00 0x55dddd6d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f6397fff640 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd6cfd00 0x55dddd6d2180 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e67cfefe98ef80 server_cookie=8976b5edfb6bb958 in_seq=0 out_seq=0 2024-09-15T22:16:16.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.381+0000 7f63777fe640 10 client.25831 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:16.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63777fe640 1 -- 172.21.15.77:0/364171831 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dddd628860 con 0x55dddd6cfd00 2024-09-15T22:16:16.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63777fe640 10 client.25831 handle_command_reply: tid=0 2024-09-15T22:16:16.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 client.25831 shutdown 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 msgr2=0x55dddd6bd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f638000bac0 tx=0x7f638000bec0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 shutdown_connections 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd6cfd00 0x55dddd6d2180 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6c0370 0x55dddd6bd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6bd030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 --2- 172.21.15.77:0/364171831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6b55d0 0x55dddd6bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 >> 172.21.15.77:0/364171831 conn(0x55dddd67e310 msgr2=0x55dddd6b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63777fe640 0 client.25831 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:16.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 shutdown_connections 2024-09-15T22:16:16.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.382+0000 7f63a0c25740 1 -- 172.21.15.77:0/364171831 wait complete. 2024-09-15T22:16:16.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.383+0000 7f63a0c25740 20 client.25831 trim_cache size 0 max 16384 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.395+0000 7f63a0c25740 1 Processor -- start 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.395+0000 7f63a0c25740 1 -- start start 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd6d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b5510 con 0x55dddd6c0370 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b7110 con 0x55dddd6b55d0 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f63a0c25740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b5be0 con 0x55dddd6b6b40 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f639cde9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd6d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59928/0 (socket says 172.21.15.77:59928) 2024-09-15T22:16:16.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 -- 172.21.15.77:0/4024901912 learned_addr learned my addr 172.21.15.77:0/4024901912 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f638ffff640 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 -- 172.21.15.77:0/4024901912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 msgr2=0x55dddd6cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 -- 172.21.15.77:0/4024901912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 msgr2=0x55dddd6d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd6d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 -- 172.21.15.77:0/4024901912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddd6b8120 con 0x55dddd6b55d0 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f6397fff640 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f6388005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4cc5fa038064113 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f638dffb640 1 -- 172.21.15.77:0/4024901912 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880c0260 con 0x55dddd6b55d0 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f638dffb640 1 -- 172.21.15.77:0/4024901912 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638802e040 con 0x55dddd6b55d0 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.396+0000 7f638dffb640 1 -- 172.21.15.77:0/4024901912 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880ed490 con 0x55dddd6b55d0 2024-09-15T22:16:16.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- 172.21.15.77:0/4024901912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 msgr2=0x55dddd6b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f6388005750 comp rx=0 tx=0).stop 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- 172.21.15.77:0/4024901912 shutdown_connections 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd6d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b6b40 0x55dddd6cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4024901912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b55d0 0x55dddd6b1e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- 172.21.15.77:0/4024901912 >> 172.21.15.77:0/4024901912 conn(0x55dddd67e310 msgr2=0x55dddd6b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- 172.21.15.77:0/4024901912 shutdown_connections 2024-09-15T22:16:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- 172.21.15.77:0/4024901912 wait complete. 2024-09-15T22:16:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.397+0000 7f63a0c25740 1 -- start start 2024-09-15T22:16:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 Processor -- start 2024-09-15T22:16:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b55d0 0x55dddd6c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b6980 con 0x55dddd6c0370 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b62b0 con 0x55dddd6b6b40 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63981b34f0 con 0x55dddd6b55d0 2024-09-15T22:16:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f63a0c25740 10 client.0 _ll_register_callbacks cb 0x7f638000b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f639cde9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f6397fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b55d0 0x55dddd6c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59938/0 (socket says 172.21.15.77:59938) 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f639cde9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43136/0 (socket says 172.21.15.77:43136) 2024-09-15T22:16:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 -- 172.21.15.77:0/4193823851 learned_addr learned my addr 172.21.15.77:0/4193823851 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 -- 172.21.15.77:0/4193823851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b55d0 msgr2=0x55dddd6c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b55d0 0x55dddd6c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 -- 172.21.15.77:0/4193823851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 msgr2=0x55dddd738a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f638ffff640 1 -- 172.21.15.77:0/4193823851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddd6cc4b0 con 0x55dddd6b6b40 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.398+0000 7f639cde9640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f638ffff640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f638000ba90 tx=0x7f6380020680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33d58cd74303d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:16:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380024040 con 0x55dddd6b6b40 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638802e040 con 0x55dddd6b6b40 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f638000b270 con 0x55dddd6b6b40 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6380041050 con 0x55dddd6b6b40 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 10 client.25910 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dddd73a1a0 con 0x55dddd6b6b40 2024-09-15T22:16:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638802e040 con 0x55dddd6b6b40 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f63800039d0 con 0x55dddd6b6b40 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 10 client.25910 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 20 client.25910 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd73aa60 0x55dddd73ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 4 client.25910 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.399+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dddd622d70 con 0x55dddd73aa60 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f6397fff640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd73aa60 0x55dddd73ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63777fe640 10 client.25910 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f6397fff640 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd73aa60 0x55dddd73ce80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13179dde1a9c9168 server_cookie=4f5f4581b21cb3f2 in_seq=0 out_seq=0 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63777fe640 1 -- 172.21.15.77:0/4193823851 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dddd622d70 con 0x55dddd73aa60 2024-09-15T22:16:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63777fe640 10 client.25910 handle_command_reply: tid=0 2024-09-15T22:16:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 client.25910 shutdown 2024-09-15T22:16:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 msgr2=0x55dddd6c3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f638000ba90 tx=0x7f6380020680 comp rx=0 tx=0).stop 2024-09-15T22:16:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 shutdown_connections 2024-09-15T22:16:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dddd73aa60 0x55dddd73ce80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dddd6c0370 0x55dddd738a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dddd6b6b40 0x55dddd6c3b80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63777fe640 0 client.25910 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:16.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 --2- 172.21.15.77:0/4193823851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dddd6b55d0 0x55dddd6c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 >> 172.21.15.77:0/4193823851 conn(0x55dddd67e310 msgr2=0x55dddd6bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 shutdown_connections 2024-09-15T22:16:16.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.400+0000 7f63a0c25740 1 -- 172.21.15.77:0/4193823851 wait complete. 2024-09-15T22:16:16.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f63a0c25740 20 client.25910 trim_cache size 0 max 16384 2024-09-15T22:16:16.422 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:16.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 msgr2=0x7f63840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 0x7f63840a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f6390009df0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 msgr2=0x7f6398073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:16.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6388009230 tx=0x7f6388005660 comp rx=0 tx=0).stop 2024-09-15T22:16:16.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 shutdown_connections 2024-09-15T22:16:16.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63840a3430 0x7f63840a58f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6398102ef0 0x7f639806cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6398102570 0x7f6398073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 --2- 172.21.15.77:0/2230945045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398100e20 0x7f6398073990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 >> 172.21.15.77:0/2230945045 conn(0x7f63980fc7d0 msgr2=0x7f63980fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 shutdown_connections 2024-09-15T22:16:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:16.401+0000 7f639e84c640 1 -- 172.21.15.77:0/2230945045 wait complete. 2024-09-15T22:16:16.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 275913 13.44 MB/sec execute 25 sec latency 40.066 ms 2024-09-15T22:16:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:16 smithi196 ceph-mon[35109]: pgmap v118: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:16 smithi117 ceph-mon[35021]: pgmap v118: 129 pgs: 129 active+clean; 166 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 219 op/s 2024-09-15T22:16:17.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 278270 13.44 MB/sec execute 26 sec latency 15.091 ms 2024-09-15T22:16:18.073 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:16:18.073 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:16:18.331 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:18.453 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:18 smithi077 ceph-mon[30980]: pgmap v119: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-09-15T22:16:18.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 280953 13.47 MB/sec execute 27 sec latency 15.330 ms 2024-09-15T22:16:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:18 smithi196 ceph-mon[35109]: pgmap v119: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-09-15T22:16:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:18 smithi117 ceph-mon[35021]: pgmap v119: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.0 MiB/s wr, 341 op/s 2024-09-15T22:16:19.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.263+0000 7fb187c72640 1 -- 172.21.15.77:0/3802714559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb180103ff0 msgr2=0x7fb1801043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.263+0000 7fb187c72640 1 --2- 172.21.15.77:0/3802714559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb180103ff0 0x7fb1801043d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c02a410 comp rx=0 tx=0).stop 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 -- 172.21.15.77:0/3802714559 shutdown_connections 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 --2- 172.21.15.77:0/3802714559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180109120 0x7fb180110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 --2- 172.21.15.77:0/3802714559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1801049a0 0x7fb1801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 --2- 172.21.15.77:0/3802714559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb180103ff0 0x7fb1801043d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 -- 172.21.15.77:0/3802714559 >> 172.21.15.77:0/3802714559 conn(0x7fb1800ffa90 msgr2=0x7fb180101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 -- 172.21.15.77:0/3802714559 shutdown_connections 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 -- 172.21.15.77:0/3802714559 wait complete. 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 Processor -- start 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.264+0000 7fb187c72640 1 -- start start 2024-09-15T22:16:19.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180103ff0 0x7fb18019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb180109120 0x7fb18019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb180114190 con 0x7fb1801049a0 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb180114010 con 0x7fb180103ff0 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb187c72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb180114310 con 0x7fb180109120 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59092/0 (socket says 172.21.15.77:59092) 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 -- 172.21.15.77:0/552504922 learned_addr learned my addr 172.21.15.77:0/552504922 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb185a0e640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180103ff0 0x7fb18019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18620f640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb180109120 0x7fb18019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 -- 172.21.15.77:0/552504922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb180109120 msgr2=0x7fb18019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb180109120 0x7fb18019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 -- 172.21.15.77:0/552504922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180103ff0 msgr2=0x7fb18019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180103ff0 0x7fb18019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 -- 172.21.15.77:0/552504922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18019f5d0 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.265+0000 7fb18520d640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e9f0 tx=0x7fb17000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.266+0000 7fb176ffd640 1 -- 172.21.15.77:0/552504922 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17000cd80 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.266+0000 7fb176ffd640 1 -- 172.21.15.77:0/552504922 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb170005500 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.266+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18019f8c0 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.266+0000 7fb176ffd640 1 -- 172.21.15.77:0/552504922 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170010690 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.266+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1801a71a0 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.268+0000 7fb18a04b740 1 Processor -- start 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.268+0000 7fb176ffd640 1 -- 172.21.15.77:0/552504922 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb1700046c0 con 0x7fb1801049a0 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 -- start start 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 0x55ef277d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef2772a280 con 0x55ef277cfd40 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef276d1b10 con 0x55ef277c4b60 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18a04b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef27726240 con 0x55ef277c5480 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 0x55ef277d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb185a0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37532/0 (socket says 172.21.15.77:37532) 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 -- 172.21.15.77:0/2271331316 learned_addr learned my addr 172.21.15.77:0/2271331316 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 0x55ef277d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59094/0 (socket says 172.21.15.77:59094) 2024-09-15T22:16:19.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 -- 172.21.15.77:0/2271331316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 msgr2=0x55ef277cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 -- 172.21.15.77:0/2271331316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 msgr2=0x55ef277d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 0x55ef277d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.269+0000 7fb18620f640 1 -- 172.21.15.77:0/2271331316 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef276de1f0 con 0x55ef277c4b60 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb176ffd640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 0x7fb16c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb18620f640 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb178009f50 tx=0x7fb1780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e578fcdfbe63349f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb185a0e640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 0x7fb16c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb16bfff640 1 -- 172.21.15.77:0/2271331316 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178004d20 con 0x55ef277c4b60 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb176ffd640 1 -- 172.21.15.77:0/552504922 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb1700d0400 con 0x7fb1801049a0 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb16bfff640 1 -- 172.21.15.77:0/2271331316 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb178004050 con 0x55ef277c4b60 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.270+0000 7fb16bfff640 1 -- 172.21.15.77:0/2271331316 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178005280 con 0x55ef277c4b60 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- 172.21.15.77:0/2271331316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 msgr2=0x55ef277c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb185a0e640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 0x7fb16c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005ab0 tx=0x7fb17c005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb178009f50 tx=0x7fb1780118d0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- 172.21.15.77:0/2271331316 shutdown_connections 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277cfd40 0x55ef277d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2271331316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277c4b60 0x55ef277c4f40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- 172.21.15.77:0/2271331316 >> 172.21.15.77:0/2271331316 conn(0x55ef277c10f0 msgr2=0x55ef277c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- 172.21.15.77:0/2271331316 shutdown_connections 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- 172.21.15.77:0/2271331316 wait complete. 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 -- start start 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.271+0000 7fb18a04b740 1 Processor -- start 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef277d2910 con 0x55ef277c4b60 2024-09-15T22:16:19.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef276e4f10 con 0x55ef277cfd40 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef277d2a90 con 0x55ef277c5480 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 10 client.0 _ll_register_callbacks cb 0x55ef276ada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59110/0 (socket says 172.21.15.77:59110) 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 -- 172.21.15.77:0/3325618035 learned_addr learned my addr 172.21.15.77:0/3325618035 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18620f640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18520d640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 -- 172.21.15.77:0/3325618035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 msgr2=0x55ef277cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 -- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 msgr2=0x55ef277cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 -- 172.21.15.77:0/3325618035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2749b6a0 con 0x55ef277c4b60 2024-09-15T22:16:19.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18620f640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b0aa68187b6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb1697fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c04e070 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb17c004970 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.272+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb178004050 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb17c05a050 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 10 client.16617 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef277de4f0 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb178004050 con 0x55ef277c4b60 2024-09-15T22:16:19.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb17c0035f0 con 0x55ef277c4b60 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 10 client.16617 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 20 client.16617 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef277df980 0x55ef277e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 4 client.16617 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef277377f0 con 0x55ef277df980 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.273+0000 7fb18520d640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef277df980 0x55ef277e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.274+0000 7fb1697fa640 10 client.16617 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.274+0000 7fb18520d640 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef277df980 0x55ef277e1de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b16768b732a639 server_cookie=c7701ff806c882c0 in_seq=0 out_seq=0 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb1697fa640 1 -- 172.21.15.77:0/3325618035 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef277377f0 con 0x55ef277df980 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb1697fa640 10 client.16617 handle_command_reply: tid=0 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 client.16617 shutdown 2024-09-15T22:16:19.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 msgr2=0x55ef277ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c009260 comp rx=0 tx=0).stop 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 shutdown_connections 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef277df980 0x55ef277e1de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3325618035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277ccc00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb1697fa640 0 client.16617 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 >> 172.21.15.77:0/3325618035 conn(0x55ef277c10f0 msgr2=0x55ef277c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 shutdown_connections 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 1 -- 172.21.15.77:0/3325618035 wait complete. 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.275+0000 7fb18a04b740 20 client.16617 trim_cache size 0 max 16384 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 Processor -- start 2024-09-15T22:16:19.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 -- start start 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c4b60 0x55ef277c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801b10c0 con 0x55ef277c5480 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801b2cc0 con 0x55ef277cfd40 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18a04b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801b1790 con 0x55ef277c4b60 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.293+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59114/0 (socket says 172.21.15.77:59114) 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 -- 172.21.15.77:0/2144119113 learned_addr learned my addr 172.21.15.77:0/2144119113 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb185a0e640 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c4b60 0x55ef277c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18620f640 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 -- 172.21.15.77:0/2144119113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c4b60 msgr2=0x55ef277c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c4b60 0x55ef277c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 -- 172.21.15.77:0/2144119113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 msgr2=0x55ef277e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 -- 172.21.15.77:0/2144119113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef277c52b0 con 0x55ef277c5480 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb18520d640 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb170098d70 tx=0x7fb1700d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6955e60b67cba1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb16b7fe640 1 -- 172.21.15.77:0/2144119113 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17009c170 con 0x55ef277c5480 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb16b7fe640 1 -- 172.21.15.77:0/2144119113 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17009caa0 con 0x55ef277c5480 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.294+0000 7fb16b7fe640 1 -- 172.21.15.77:0/2144119113 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17009bd00 con 0x55ef277c5480 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- 172.21.15.77:0/2144119113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 msgr2=0x55ef277e76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb170098d70 tx=0x7fb1700d2040 comp rx=0 tx=0).stop 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- 172.21.15.77:0/2144119113 shutdown_connections 2024-09-15T22:16:19.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c5480 0x55ef277e76b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 --2- 172.21.15.77:0/2144119113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c4b60 0x55ef277c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- 172.21.15.77:0/2144119113 >> 172.21.15.77:0/2144119113 conn(0x55ef277c10f0 msgr2=0x55ef277ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- 172.21.15.77:0/2144119113 shutdown_connections 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- 172.21.15.77:0/2144119113 wait complete. 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 -- start start 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.295+0000 7fb18a04b740 1 Processor -- start 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801b2530 con 0x55ef277c4b60 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801b1e60 con 0x55ef277cfd40 2024-09-15T22:16:19.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1801af0a0 con 0x55ef277c5480 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 10 client.0 _ll_register_callbacks cb 0x7fb17c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59122/0 (socket says 172.21.15.77:59122) 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 -- 172.21.15.77:0/3584712694 learned_addr learned my addr 172.21.15.77:0/3584712694 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18620f640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18520d640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 -- 172.21.15.77:0/3584712694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 msgr2=0x55ef277ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 -- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 msgr2=0x55ef277ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 -- 172.21.15.77:0/3584712694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef27849880 con 0x55ef277c4b60 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18620f640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb185a0e640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0039c0 tx=0x7fb17c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c428d82d9ba185 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:19.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb1697fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c04e040 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb17009caa0 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.296+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb17c0046c0 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb17c06f050 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 10 client.16623 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef2784ab60 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb17009caa0 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb17c0327a0 con 0x55ef277c4b60 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 10 client.16623 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 20 client.16623 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2784b310 0x55ef2784d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:19.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 4 client.16623 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef27731d00 con 0x55ef2784b310 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.297+0000 7fb18520d640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2784b310 0x55ef2784d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "3a27e433-1ff2-468c-99f5-ccbf4dfa8095", 2024-09-15T22:16:19.325 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb18520d640 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2784b310 0x55ef2784d730 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7979ea2f8015c514 server_cookie=7816610eaa247ec3 in_seq=0 out_seq=0 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb1697fa640 10 client.16623 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb1697fa640 1 -- 172.21.15.77:0/3584712694 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef27731d00 con 0x55ef2784b310 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb1697fa640 10 client.16623 handle_command_reply: tid=0 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb18a04b740 1 client.16623 shutdown 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 msgr2=0x55ef277c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.298+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0039c0 tx=0x7fb17c003820 comp rx=0 tx=0).stop 2024-09-15T22:16:19.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 shutdown_connections 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2784b310 0x55ef2784d730 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef277cfd40 0x55ef277ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef277c5480 0x55ef277ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 --2- 172.21.15.77:0/3584712694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef277c4b60 0x55ef277c9dd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb1697fa640 0 client.16623 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 >> 172.21.15.77:0/3584712694 conn(0x55ef277c10f0 msgr2=0x55ef277dab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 shutdown_connections 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 1 -- 172.21.15.77:0/3584712694 wait complete. 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb18a04b740 20 client.16623 trim_cache size 0 max 16384 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 msgr2=0x7fb16c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 0x7fb16c0a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005ab0 tx=0x7fb17c005a20 comp rx=0 tx=0).stop 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 msgr2=0x7fb18019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:19.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e9f0 tx=0x7fb17000edf0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 shutdown_connections 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb16c0a3430 0x7fb16c0a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb180109120 0x7fb18019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801049a0 0x7fb18019ab60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 --2- 172.21.15.77:0/552504922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb180103ff0 0x7fb18019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.299+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 >> 172.21.15.77:0/552504922 conn(0x7fb1800ffa90 msgr2=0x7fb180100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.300+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 shutdown_connections 2024-09-15T22:16:19.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:19.300+0000 7fb187c72640 1 -- 172.21.15.77:0/552504922 wait complete. 2024-09-15T22:16:19.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 283949 13.61 MB/sec execute 28 sec latency 55.802 ms 2024-09-15T22:16:20.187 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:20.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 286194 13.43 MB/sec execute 29 sec latency 54.053 ms 2024-09-15T22:16:20.485 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: pgmap v120: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:20 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: pgmap v120: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:20 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: pgmap v120: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:20 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:21.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 -- 172.21.15.77:0/2862434692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c101de0 msgr2=0x7f750c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 --2- 172.21.15.77:0/2862434692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c101de0 0x7f750c1021c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 -- 172.21.15.77:0/2862434692 shutdown_connections 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 --2- 172.21.15.77:0/2862434692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c106f10 0x7f750c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 --2- 172.21.15.77:0/2862434692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c102790 0x7f750c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 --2- 172.21.15.77:0/2862434692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c101de0 0x7f750c1021c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 -- 172.21.15.77:0/2862434692 >> 172.21.15.77:0/2862434692 conn(0x7f750c0fd8c0 msgr2=0x7f750c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 -- 172.21.15.77:0/2862434692 shutdown_connections 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.014+0000 7f751344d640 1 -- 172.21.15.77:0/2862434692 wait complete. 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 Processor -- start 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 -- start start 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c101de0 0x7f750c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c106f10 0x7f750c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f750c111f70 con 0x7f750c102790 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f750c111df0 con 0x7f750c106f10 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f751344d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1120f0 con 0x7f750c101de0 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f75111e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c101de0 0x7f750c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59148/0 (socket says 172.21.15.77:59148) 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.015+0000 7f75109e8640 1 -- 172.21.15.77:0/2576314781 learned_addr learned my addr 172.21.15.77:0/2576314781 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75119ea640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c106f10 0x7f750c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 -- 172.21.15.77:0/2576314781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c101de0 msgr2=0x7f750c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c101de0 0x7f750c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 -- 172.21.15.77:0/2576314781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c106f10 msgr2=0x7f750c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c106f10 0x7f750c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 -- 172.21.15.77:0/2576314781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c19f620 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75109e8640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00ea60 tx=0x7f74fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75027fc640 1 -- 172.21.15.77:0/2576314781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00cde0 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75027fc640 1 -- 172.21.15.77:0/2576314781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc0035f0 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c19f910 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f75027fc640 1 -- 172.21.15.77:0/2576314781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc010670 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.016+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c1a71f0 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75027fc640 1 -- 172.21.15.77:0/2576314781 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f74fc0037a0 con 0x7f750c102790 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 Processor -- start 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 -- start start 2024-09-15T22:16:21.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4d342e0 con 0x55d9a4dce4a0 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4cdbce0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f7515826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4d302a0 con 0x55d9a4dd9680 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75119ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75111e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75119ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59164/0 (socket says 172.21.15.77:59164) 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75119ea640 1 -- 172.21.15.77:0/2612806225 learned_addr learned my addr 172.21.15.77:0/2612806225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75109e8640 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.019+0000 7f75027fc640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 0x7f74f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 -- 172.21.15.77:0/2612806225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 msgr2=0x55d9a4ddba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 -- 172.21.15.77:0/2612806225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 msgr2=0x55d9a4dce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 -- 172.21.15.77:0/2612806225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a4cc0440 con 0x55d9a4dcedc0 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75119ea640 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75027fc640 1 -- 172.21.15.77:0/2576314781 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f74fc0d0370 con 0x7f750c102790 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75119ea640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 0x7f74f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75111e9640 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd9140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7508005940 tx=0x7f7508005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e99a0a215a4c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f74f77fe640 1 -- 172.21.15.77:0/2612806225 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508040070 con 0x55d9a4dcedc0 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f75119ea640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 0x7f74f80a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75040107d0 tx=0x7f7504004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f74f77fe640 1 -- 172.21.15.77:0/2612806225 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7508031870 con 0x55d9a4dcedc0 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.020+0000 7f74f77fe640 1 -- 172.21.15.77:0/2612806225 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508035040 con 0x55d9a4dcedc0 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- 172.21.15.77:0/2612806225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 msgr2=0x55d9a4dd9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd9140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7508005940 tx=0x7f7508005be0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- 172.21.15.77:0/2612806225 shutdown_connections 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd9140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 --2- 172.21.15.77:0/2612806225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- 172.21.15.77:0/2612806225 >> 172.21.15.77:0/2612806225 conn(0x55d9a4dcaa30 msgr2=0x55d9a4dcae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- 172.21.15.77:0/2612806225 shutdown_connections 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- 172.21.15.77:0/2612806225 wait complete. 2024-09-15T22:16:21.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 -- start start 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.021+0000 7f7515826740 1 Processor -- start 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4ddc250 con 0x55d9a4dd9680 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4cef050 con 0x55d9a4dce4a0 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4ddc3d0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f7515826740 10 client.0 _ll_register_callbacks cb 0x55d9a4d2f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75111e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75111e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37596/0 (socket says 172.21.15.77:37596) 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59172/0 (socket says 172.21.15.77:59172) 2024-09-15T22:16:21.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75111e9640 1 -- 172.21.15.77:0/2995868692 learned_addr learned my addr 172.21.15.77:0/2995868692 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:21.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75109e8640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 -- 172.21.15.77:0/2995868692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dcedc0 msgr2=0x55d9a4dd6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 -- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 msgr2=0x55d9a4dd8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 -- 172.21.15.77:0/2995868692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a4aa53c0 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75111e9640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f75119ea640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f750400f790 tx=0x7f7504004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8d7c267393d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.022+0000 7f74f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504004d20 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7504005020 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7508031870 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7504005220 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 10 client.16635 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9a4de7c20 con 0x55d9a4dd9680 2024-09-15T22:16:21.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7508031870 con 0x55d9a4dd9680 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7504026070 con 0x55d9a4dd9680 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 10 client.16635 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 20 client.16635 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4deb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 4 client.16635 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.023+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a4d41850 con 0x55d9a4de91b0 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.024+0000 7f75111e9640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4deb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.024+0000 7f75111e9640 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4deb630 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb94aee36e7c5993 server_cookie=e26fa4c49599e8b4 in_seq=0 out_seq=0 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.024+0000 7f74f4ff9640 10 client.16635 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:21.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.024+0000 7f74f4ff9640 1 -- 172.21.15.77:0/2995868692 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9a4d41850 con 0x55d9a4de91b0 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.024+0000 7f74f4ff9640 10 client.16635 handle_command_reply: tid=0 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 client.16635 shutdown 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 msgr2=0x55d9a4dd6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f750400f790 tx=0x7f7504004ab0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 shutdown_connections 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4deb630 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f74f4ff9640 0 client.16635 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dd9680 0x55d9a4dd6750 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 --2- 172.21.15.77:0/2995868692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dd8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 >> 172.21.15.77:0/2995868692 conn(0x55d9a4dcaa30 msgr2=0x55d9a4dc9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 shutdown_connections 2024-09-15T22:16:21.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 1 -- 172.21.15.77:0/2995868692 wait complete. 2024-09-15T22:16:21.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.025+0000 7f7515826740 20 client.16635 trim_cache size 0 max 16384 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 Processor -- start 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 -- start start 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dcae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dd9680 0x55d9a4df0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1b1190 con 0x55d9a4dcedc0 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1b2d90 con 0x55d9a4dd9680 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f7515826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1b1860 con 0x55d9a4dce4a0 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59182/0 (socket says 172.21.15.77:59182) 2024-09-15T22:16:21.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f75109e8640 1 -- 172.21.15.77:0/2111911677 learned_addr learned my addr 172.21.15.77:0/2111911677 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f75111e9640 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dcae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.042+0000 7f75119ea640 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dd9680 0x55d9a4df0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 -- 172.21.15.77:0/2111911677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dce4a0 msgr2=0x55d9a4dcae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dcae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 -- 172.21.15.77:0/2111911677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dd9680 msgr2=0x55d9a4df0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dd9680 0x55d9a4df0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 -- 172.21.15.77:0/2111911677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a4dd92d0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f75109e8640 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f74fc09c240 tx=0x7f74fc098680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1d5dd43d82f585 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f74f6ffd640 1 -- 172.21.15.77:0/2111911677 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc09c080 con 0x55d9a4dcedc0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f74f6ffd640 1 -- 172.21.15.77:0/2111911677 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc09c9b0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.043+0000 7f74f6ffd640 1 -- 172.21.15.77:0/2111911677 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0d0170 con 0x55d9a4dcedc0 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- 172.21.15.77:0/2111911677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 msgr2=0x55d9a4dd8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f74fc09c240 tx=0x7f74fc098680 comp rx=0 tx=0).stop 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- 172.21.15.77:0/2111911677 shutdown_connections 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dd9680 0x55d9a4df0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4dd8180 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 --2- 172.21.15.77:0/2111911677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dce4a0 0x55d9a4dcae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- 172.21.15.77:0/2111911677 >> 172.21.15.77:0/2111911677 conn(0x55d9a4dcaa30 msgr2=0x55d9a4dd7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- 172.21.15.77:0/2111911677 shutdown_connections 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- 172.21.15.77:0/2111911677 wait complete. 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 -- start start 2024-09-15T22:16:21.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.044+0000 7f7515826740 1 Processor -- start 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4de1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1b2600 con 0x55d9a4dcedc0 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1b1f30 con 0x55d9a4dce4a0 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1af170 con 0x55d9a4dd9680 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f7515826740 10 client.0 _ll_register_callbacks cb 0x7f7504009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59194/0 (socket says 172.21.15.77:59194) 2024-09-15T22:16:21.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75119ea640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 -- 172.21.15.77:0/3472884822 learned_addr learned my addr 172.21.15.77:0/3472884822 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75111e9640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4de1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 -- 172.21.15.77:0/3472884822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 msgr2=0x55d9a4ddf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 -- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 msgr2=0x55d9a4de1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4de1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75109e8640 1 -- 172.21.15.77:0/3472884822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a4ddf830 con 0x55d9a4dcedc0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.045+0000 7f75119ea640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f75109e8640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f74fc098c80 tx=0x7f74fc09a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945db30655b4771c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc09c2b0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f74fc0cd9a0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc09c9b0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74fc0a0660 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f7515826740 10 client.16641 fetch_fsmap learned FSMap version 32 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9a4e53c20 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc09c9b0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.046+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f74fc0a02c0 con 0x55d9a4dcedc0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 10 client.16641 fetch_fsmap finished waiting for FSMap version 32 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 20 client.16641 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4e563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 4 client.16641 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9a4d3bd60 con 0x55d9a4de91b0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f75111e9640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4e563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:21.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f74f4ff9640 10 client.16641 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.047+0000 7f75111e9640 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4e563a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d210fd3d6a97568 server_cookie=76cb577b648d3e4c in_seq=0 out_seq=0 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f74f4ff9640 1 -- 172.21.15.77:0/3472884822 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9a4d3bd60 con 0x55d9a4de91b0 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f74f4ff9640 10 client.16641 handle_command_reply: tid=0 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 client.16641 shutdown 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 msgr2=0x55d9a4ddedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f74fc098c80 tx=0x7f74fc09a7c0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 shutdown_connections 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d9a4de91b0 0x55d9a4e563a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9a4dd9680 0x55d9a4ddf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9a4dcedc0 0x55d9a4ddedb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f74f4ff9640 0 client.16641 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 --2- 172.21.15.77:0/3472884822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9a4dce4a0 0x55d9a4de1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 >> 172.21.15.77:0/3472884822 conn(0x55d9a4dcaa30 msgr2=0x55d9a4dc9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 shutdown_connections 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 1 -- 172.21.15.77:0/3472884822 wait complete. 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.048+0000 7f7515826740 20 client.16641 trim_cache size 0 max 16384 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "3a27e433-1ff2-468c-99f5-ccbf4dfa8095": { 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:16:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "3a27e433-1ff2-468c-99f5-ccbf4dfa8095", 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 msgr2=0x7f74f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 0x7f74f80a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f75040107d0 tx=0x7f7504004a40 comp rx=0 tx=0).stop 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 msgr2=0x7f750c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:21.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00ea60 tx=0x7f74fc00ef20 comp rx=0 tx=0).stop 2024-09-15T22:16:21.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 shutdown_connections 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f74f80a33b0 0x7f74f80a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f750c106f10 0x7f750c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f750c102790 0x7f750c19ac50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 --2- 172.21.15.77:0/2576314781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f750c101de0 0x7f750c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 >> 172.21.15.77:0/2576314781 conn(0x7f750c0fd8c0 msgr2=0x7f750c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 shutdown_connections 2024-09-15T22:16:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:21.049+0000 7f751344d640 1 -- 172.21.15.77:0/2576314781 wait complete. 2024-09-15T22:16:21.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 288043 13.39 MB/sec execute 30 sec latency 39.581 ms 2024-09-15T22:16:21.891 INFO:tasks.cephfs.filesystem:scrub status for tag:3a27e433-1ff2-468c-99f5-ccbf4dfa8095 - {'path': '/', 'tag': '3a27e433-1ff2-468c-99f5-ccbf4dfa8095', 'options': 'recursive,force'} 2024-09-15T22:16:22.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 291151 13.56 MB/sec execute 31 sec latency 9.938 ms 2024-09-15T22:16:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:22 smithi077 ceph-mon[30980]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-09-15T22:16:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:22 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:22 smithi196 ceph-mon[35109]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-09-15T22:16:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:22 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:22 smithi117 ceph-mon[35021]: pgmap v121: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-09-15T22:16:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:22 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:23.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 294041 13.56 MB/sec execute 32 sec latency 11.381 ms 2024-09-15T22:16:24.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 296201 13.49 MB/sec execute 33 sec latency 13.875 ms 2024-09-15T22:16:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:24 smithi077 ceph-mon[30980]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:24 smithi196 ceph-mon[35109]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:24 smithi117 ceph-mon[35021]: pgmap v122: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:25.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 298783 13.51 MB/sec execute 34 sec latency 11.620 ms 2024-09-15T22:16:26.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 301371 13.53 MB/sec execute 35 sec latency 44.859 ms 2024-09-15T22:16:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:26 smithi077 ceph-mon[30980]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:26 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:16:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:26 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:16:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:26 smithi196 ceph-mon[35109]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:26 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:16:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:26 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:16:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:26 smithi117 ceph-mon[35021]: pgmap v123: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.5 MiB/s wr, 242 op/s 2024-09-15T22:16:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:26 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:16:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:26 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:16:27.465 INFO:tasks.workunit.client.0.smithi077.stdout: 1 303613 13.44 MB/sec execute 36 sec latency 14.131 ms 2024-09-15T22:16:28.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 305973 13.46 MB/sec execute 37 sec latency 11.455 ms 2024-09-15T22:16:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:28 smithi077 ceph-mon[30980]: pgmap v124: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-15T22:16:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:28 smithi196 ceph-mon[35109]: pgmap v124: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-15T22:16:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:28 smithi117 ceph-mon[35021]: pgmap v124: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-09-15T22:16:29.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 308938 13.58 MB/sec execute 38 sec latency 9.609 ms 2024-09-15T22:16:30.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 311982 13.61 MB/sec execute 39 sec latency 10.537 ms 2024-09-15T22:16:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:30 smithi077 ceph-mon[30980]: pgmap v125: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-09-15T22:16:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:30 smithi196 ceph-mon[35109]: pgmap v125: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-09-15T22:16:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:30 smithi117 ceph-mon[35021]: pgmap v125: 129 pgs: 129 active+clean; 180 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2024-09-15T22:16:31.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 313439 13.44 MB/sec execute 40 sec latency 43.911 ms 2024-09-15T22:16:32.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 316324 13.53 MB/sec execute 41 sec latency 11.670 ms 2024-09-15T22:16:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:32 smithi077 ceph-mon[30980]: pgmap v126: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.7 MiB/s wr, 354 op/s 2024-09-15T22:16:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:32 smithi196 ceph-mon[35109]: pgmap v126: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.7 MiB/s wr, 354 op/s 2024-09-15T22:16:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:32 smithi117 ceph-mon[35021]: pgmap v126: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.7 MiB/s wr, 354 op/s 2024-09-15T22:16:33.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 319387 13.63 MB/sec execute 42 sec latency 15.453 ms 2024-09-15T22:16:34.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 322031 13.60 MB/sec execute 43 sec latency 20.229 ms 2024-09-15T22:16:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:34 smithi077 ceph-mon[30980]: pgmap v127: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:34 smithi196 ceph-mon[35109]: pgmap v127: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:34 smithi117 ceph-mon[35021]: pgmap v127: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:35.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 324057 13.52 MB/sec execute 44 sec latency 19.716 ms 2024-09-15T22:16:36.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 326731 13.60 MB/sec execute 45 sec latency 60.696 ms 2024-09-15T22:16:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:36 smithi077 ceph-mon[30980]: pgmap v128: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:36.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:36 smithi196 ceph-mon[35109]: pgmap v128: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:36 smithi117 ceph-mon[35021]: pgmap v128: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:16:37.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 329532 13.59 MB/sec execute 46 sec latency 19.976 ms 2024-09-15T22:16:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:37 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:37 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:37 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:16:38.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 331746 13.54 MB/sec execute 47 sec latency 12.240 ms 2024-09-15T22:16:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:38 smithi077 ceph-mon[30980]: pgmap v129: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-09-15T22:16:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:16:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:38 smithi196 ceph-mon[35109]: pgmap v129: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-09-15T22:16:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:16:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:38 smithi117 ceph-mon[35021]: pgmap v129: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-09-15T22:16:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:38.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:16:39.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 334218 13.55 MB/sec execute 48 sec latency 16.322 ms 2024-09-15T22:16:40.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 337284 13.65 MB/sec execute 49 sec latency 9.057 ms 2024-09-15T22:16:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:40 smithi077 ceph-mon[30980]: pgmap v130: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-09-15T22:16:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:40 smithi196 ceph-mon[35109]: pgmap v130: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-09-15T22:16:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:40 smithi117 ceph-mon[35021]: pgmap v130: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 241 op/s 2024-09-15T22:16:41.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 339739 13.60 MB/sec execute 50 sec latency 78.701 ms 2024-09-15T22:16:42.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 341508 13.52 MB/sec execute 51 sec latency 16.958 ms 2024-09-15T22:16:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:42 smithi077 ceph-mon[30980]: pgmap v131: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.3 MiB/s wr, 362 op/s 2024-09-15T22:16:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:42 smithi196 ceph-mon[35109]: pgmap v131: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.3 MiB/s wr, 362 op/s 2024-09-15T22:16:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:42 smithi117 ceph-mon[35021]: pgmap v131: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.3 MiB/s wr, 362 op/s 2024-09-15T22:16:43.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 344466 13.61 MB/sec execute 52 sec latency 20.352 ms 2024-09-15T22:16:44.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 347571 13.68 MB/sec execute 53 sec latency 10.962 ms 2024-09-15T22:16:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:44 smithi077 ceph-mon[30980]: pgmap v132: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:44 smithi196 ceph-mon[35109]: pgmap v132: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:44 smithi117 ceph-mon[35021]: pgmap v132: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:45.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 350006 13.59 MB/sec execute 54 sec latency 11.676 ms 2024-09-15T22:16:46.466 INFO:tasks.workunit.client.0.smithi077.stdout: 1 351942 13.57 MB/sec execute 55 sec latency 45.909 ms 2024-09-15T22:16:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:46 smithi077 ceph-mon[30980]: pgmap v133: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:46 smithi196 ceph-mon[35109]: pgmap v133: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:46 smithi117 ceph-mon[35021]: pgmap v133: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:16:46.703 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-15T22:16:46.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:47.012 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:47.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 354821 13.63 MB/sec execute 56 sec latency 36.860 ms 2024-09-15T22:16:47.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.524+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/555442593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c100e20 msgr2=0x7fd59c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/555442593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c100e20 0x7fd59c101200 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd598009900 tx=0x7fd59802a480 comp rx=0 tx=0).stop 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/555442593 shutdown_connections 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/555442593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102ef0 0x7fd59c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/555442593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c102570 0x7fd59c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/555442593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c100e20 0x7fd59c101200 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/555442593 >> 172.21.15.77:0/555442593 conn(0x7fd59c0fc7d0 msgr2=0x7fd59c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/555442593 shutdown_connections 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/555442593 wait complete. 2024-09-15T22:16:47.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 Processor -- start 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.525+0000 7fd5a3a0f640 1 -- start start 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c100e20 0x7fd59c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102570 0x7fd59c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c110f80 con 0x7fd59c102ef0 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c110e00 con 0x7fd59c102570 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c111100 con 0x7fd59c100e20 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48398/0 (socket says 172.21.15.77:48398) 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a17ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c100e20 0x7fd59c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 -- 172.21.15.77:0/1383326962 learned_addr learned my addr 172.21.15.77:0/1383326962 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a0faa640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102570 0x7fd59c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 -- 172.21.15.77:0/1383326962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c100e20 msgr2=0x7fd59c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c100e20 0x7fd59c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 -- 172.21.15.77:0/1383326962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102570 msgr2=0x7fd59c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102570 0x7fd59c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 -- 172.21.15.77:0/1383326962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c19d570 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a1fac640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd59400efd0 tx=0x7fd59400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594003b70 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c19d860 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd594003d10 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.526+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940109f0 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.527+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c1a50b0 con 0x7fd59c102ef0 2024-09-15T22:16:47.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.527+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c101200 con 0x7fd59c102ef0 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.529+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd594010b90 con 0x7fd59c102ef0 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.529+0000 7fd5927fc640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 0x7fd5880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.529+0000 7fd5a17ab640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 0x7fd5880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.529+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fd5940d0890 con 0x7fd59c102ef0 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.530+0000 7fd5a17ab640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 0x7fd5880a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd598009230 tx=0x7fd5980058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.533+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd59409cf30 con 0x7fd59c102ef0 2024-09-15T22:16:47.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.750+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd59c1a5340 con 0x7fd59c102ef0 2024-09-15T22:16:47.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.751+0000 7fd5927fc640 1 -- 172.21.15.77:0/1383326962 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fd59409c830 con 0x7fd59c102ef0 2024-09-15T22:16:47.754 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:16:47.754 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:16:47.754 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 msgr2=0x7fd5880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 0x7fd5880a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd598009230 tx=0x7fd5980058d0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 msgr2=0x7fd59c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd59400efd0 tx=0x7fd59400c6a0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 shutdown_connections 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5880a33b0 0x7fd5880a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd59c102ef0 0x7fd59c19ce40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd59c102570 0x7fd59c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 --2- 172.21.15.77:0/1383326962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd59c100e20 0x7fd59c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 >> 172.21.15.77:0/1383326962 conn(0x7fd59c0fc7d0 msgr2=0x7fd59c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 shutdown_connections 2024-09-15T22:16:47.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:47.753+0000 7fd5a3a0f640 1 -- 172.21.15.77:0/1383326962 wait complete. 2024-09-15T22:16:48.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 357304 13.56 MB/sec execute 57 sec latency 10.528 ms 2024-09-15T22:16:48.491 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:48 smithi077 ceph-mon[30980]: pgmap v134: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 359 op/s 2024-09-15T22:16:48.491 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:48 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1383326962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:48.520 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-15T22:16:48.520 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-15T22:16:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:48 smithi196 ceph-mon[35109]: pgmap v134: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 359 op/s 2024-09-15T22:16:48.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:48 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1383326962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:48 smithi117 ceph-mon[35021]: pgmap v134: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.1 MiB/s wr, 359 op/s 2024-09-15T22:16:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:48 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1383326962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:48.797 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:49.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 -- 172.21.15.77:0/3036121889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 msgr2=0x7fc66010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 --2- 172.21.15.77:0/3036121889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc66010d950 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc658009cd0 tx=0x7fc65802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 -- 172.21.15.77:0/3036121889 shutdown_connections 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 --2- 172.21.15.77:0/3036121889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc66010d950 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 --2- 172.21.15.77:0/3036121889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc660100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 --2- 172.21.15.77:0/3036121889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc660106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 -- 172.21.15.77:0/3036121889 >> 172.21.15.77:0/3036121889 conn(0x7fc6600fc7a0 msgr2=0x7fc6600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 -- 172.21.15.77:0/3036121889 shutdown_connections 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.323+0000 7fc66498d640 1 -- 172.21.15.77:0/3036121889 wait complete. 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 Processor -- start 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 -- start start 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc6601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc66019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc660110ed0 con 0x7fc660101420 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc660110d50 con 0x7fc660106a80 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.324+0000 7fc66498d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc660111050 con 0x7fc660100a80 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc657fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc6601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33966/0 (socket says 172.21.15.77:33966) 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc66019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65e59c640 1 -- 172.21.15.77:0/1299624220 learned_addr learned my addr 172.21.15.77:0/1299624220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1299624220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 msgr2=0x7fc6601984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 msgr2=0x7fc6601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc6601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1299624220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66019d460 con 0x7fc660106a80 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65e59c640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:49.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc66019cd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc658009ca0 tx=0x7fc658031e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc6577fe640 1 -- 172.21.15.77:0/1299624220 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658004040 con 0x7fc660106a80 2024-09-15T22:16:49.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc6577fe640 1 -- 172.21.15.77:0/1299624220 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6580046c0 con 0x7fc660106a80 2024-09-15T22:16:49.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66019d6f0 con 0x7fc660106a80 2024-09-15T22:16:49.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc6577fe640 1 -- 172.21.15.77:0/1299624220 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658030e40 con 0x7fc660106a80 2024-09-15T22:16:49.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.325+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6601a5030 con 0x7fc660106a80 2024-09-15T22:16:49.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.328+0000 7fc6577fe640 1 -- 172.21.15.77:0/1299624220 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc6580305a0 con 0x7fc660106a80 2024-09-15T22:16:49.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.328+0000 7fc666d66740 1 Processor -- start 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 -- start start 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc6577fe640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 0x7fc64c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ab862a0 con 0x55ef2ac20760 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ab2db10 con 0x55ef2ac21080 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc666d66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ab82260 con 0x55ef2ac2b940 2024-09-15T22:16:49.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc6577fe640 1 -- 172.21.15.77:0/1299624220 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc65802e030 con 0x7fc660106a80 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.329+0000 7fc657fff640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 0x7fc64c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33972/0 (socket says 172.21.15.77:33972) 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1665196630 learned_addr learned my addr 172.21.15.77:0/1665196630 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65e59c640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc657fff640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1665196630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 msgr2=0x55ef2ac2b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac2b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1665196630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 msgr2=0x55ef2ac20b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac20b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 -- 172.21.15.77:0/1665196630 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2a8f7b90 con 0x55ef2ac2b940 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65e59c640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:49.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc657fff640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 0x7fc64c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6601051b0 tx=0x7fc648011920 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6580c5c20 tx=0x7fc6580bd340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ceba514f199e393 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc654ff9640 1 -- 172.21.15.77:0/1665196630 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6580bdc70 con 0x55ef2ac2b940 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.330+0000 7fc654ff9640 1 -- 172.21.15.77:0/1665196630 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc658036070 con 0x55ef2ac2b940 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc654ff9640 1 -- 172.21.15.77:0/1665196630 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6580b9be0 con 0x55ef2ac2b940 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 -- 172.21.15.77:0/1665196630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 msgr2=0x55ef2ac2dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6580c5c20 tx=0x7fc6580bd340 comp rx=0 tx=0).stop 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 -- 172.21.15.77:0/1665196630 shutdown_connections 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac2dd30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 --2- 172.21.15.77:0/1665196630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac20b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 -- 172.21.15.77:0/1665196630 >> 172.21.15.77:0/1665196630 conn(0x55ef2ac1ccf0 msgr2=0x55ef2ac1d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 -- 172.21.15.77:0/1665196630 shutdown_connections 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.331+0000 7fc666d66740 1 -- 172.21.15.77:0/1665196630 wait complete. 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 -- start start 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 Processor -- start 2024-09-15T22:16:49.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ac2e270 con 0x55ef2ac20760 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ab0e6e0 con 0x55ef2ac21080 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef2ac2e3f0 con 0x55ef2ac2b940 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc666d66740 10 client.0 _ll_register_callbacks cb 0x55ef2ab32a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59140/0 (socket says 172.21.15.77:59140) 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc65e59c640 1 -- 172.21.15.77:0/2838212200 learned_addr learned my addr 172.21.15.77:0/2838212200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.332+0000 7fc657fff640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 -- 172.21.15.77:0/2838212200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 msgr2=0x55ef2ac252e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac252e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 -- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 msgr2=0x55ef2ac24da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac24da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 -- 172.21.15.77:0/2838212200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2ac291a0 con 0x55ef2ac20760 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e6b0 tx=0x7fc65000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9cf98c9716d008 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc657fff640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:49.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:49.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650027060 con 0x55ef2ac20760 2024-09-15T22:16:49.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc658036070 con 0x55ef2ac20760 2024-09-15T22:16:49.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc65000db20 con 0x55ef2ac20760 2024-09-15T22:16:49.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc65000dd20 con 0x55ef2ac20760 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.333+0000 7fc666d66740 10 client.16659 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ef2ac3a210 con 0x55ef2ac20760 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc658036070 con 0x55ef2ac20760 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc650004cc0 con 0x55ef2ac20760 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 10 client.16659 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 10 client.16659 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 20 client.16659 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2ac3b720 0x55ef2ac3db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 4 client.16659 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2ab939d0 con 0x55ef2ac3b720 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.334+0000 7fc657fff640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2ac3b720 0x55ef2ac3db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.335+0000 7fc657fff640 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2ac3b720 0x55ef2ac3db10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db265669f76cdf4 server_cookie=a535aa9937d251e1 in_seq=0 out_seq=0 2024-09-15T22:16:49.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.335+0000 7fc645ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc645ffb640 1 -- 172.21.15.77:0/2838212200 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef2ab939d0 con 0x55ef2ac3b720 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc645ffb640 10 client.16659 handle_command_reply: tid=0 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 client.16659 shutdown 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 msgr2=0x55ef2ac24860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e6b0 tx=0x7fc65000eb70 comp rx=0 tx=0).stop 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 shutdown_connections 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2ac3b720 0x55ef2ac3db10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac21080 0x55ef2ac24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 --2- 172.21.15.77:0/2838212200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac20760 0x55ef2ac24860 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 >> 172.21.15.77:0/2838212200 conn(0x55ef2ac1ccf0 msgr2=0x55ef2ac22a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc645ffb640 0 client.16659 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:49.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 shutdown_connections 2024-09-15T22:16:49.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 1 -- 172.21.15.77:0/2838212200 wait complete. 2024-09-15T22:16:49.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.336+0000 7fc666d66740 20 client.16659 trim_cache size 0 max 16384 2024-09-15T22:16:49.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 Processor -- start 2024-09-15T22:16:49.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 -- start start 2024-09-15T22:16:49.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 0x55ef2ac25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601acc20 con 0x55ef2ac21080 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ae760 con 0x55ef2ac20760 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc666d66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ad2f0 con 0x55ef2ac2b940 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc657fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc657fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59142/0 (socket says 172.21.15.77:59142) 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.355+0000 7fc657fff640 1 -- 172.21.15.77:0/3375969113 learned_addr learned my addr 172.21.15.77:0/3375969113 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc65e59c640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 0x55ef2ac25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 -- 172.21.15.77:0/3375969113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 msgr2=0x55ef2ac43960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac43960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 -- 172.21.15.77:0/3375969113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 msgr2=0x55ef2ac25130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 0x55ef2ac25130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 -- 172.21.15.77:0/3375969113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2ac18260 con 0x55ef2ac21080 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc65e59c640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 0x55ef2ac25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc657fff640 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc648000c00 tx=0x7fc64801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b9d2cd6a9784cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc647fff640 1 -- 172.21.15.77:0/3375969113 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64801a040 con 0x55ef2ac21080 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc647fff640 1 -- 172.21.15.77:0/3375969113 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc648018040 con 0x55ef2ac21080 2024-09-15T22:16:49.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.356+0000 7fc647fff640 1 -- 172.21.15.77:0/3375969113 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64802a8a0 con 0x55ef2ac21080 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 -- 172.21.15.77:0/3375969113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 msgr2=0x55ef2ac43420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc648000c00 tx=0x7fc64801f000 comp rx=0 tx=0).stop 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 -- 172.21.15.77:0/3375969113 shutdown_connections 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac2b940 0x55ef2ac43960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac43420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 --2- 172.21.15.77:0/3375969113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac20760 0x55ef2ac25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 -- 172.21.15.77:0/3375969113 >> 172.21.15.77:0/3375969113 conn(0x55ef2ac1ccf0 msgr2=0x55ef2ac288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 -- 172.21.15.77:0/3375969113 shutdown_connections 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.357+0000 7fc666d66740 1 -- 172.21.15.77:0/3375969113 wait complete. 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 -- start start 2024-09-15T22:16:49.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 Processor -- start 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac34530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac2b940 0x55ef2ac34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ae090 con 0x55ef2ac21080 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601ad9c0 con 0x55ef2ac2b940 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601aac00 con 0x55ef2ac20760 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc666d66740 10 client.0 _ll_register_callbacks cb 0x7fc650009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc657fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac34530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc65e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33994/0 (socket says 172.21.15.77:33994) 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc65e59c640 1 -- 172.21.15.77:0/2752175696 learned_addr learned my addr 172.21.15.77:0/2752175696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.358+0000 7fc65ed9d640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac2b940 0x55ef2ac34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 -- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac2b940 msgr2=0x55ef2ac34a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac2b940 0x55ef2ac34a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 -- 172.21.15.77:0/2752175696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 msgr2=0x55ef2ac34530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac34530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 -- 172.21.15.77:0/2752175696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2aca53f0 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc657fff640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc65e59c640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6500214f0 tx=0x7fc65000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc41d3595a3156ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000d710 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc648018040 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc6500050b0 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.359+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc6500052b0 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 10 client.25897 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ef2aca6530 con 0x55ef2ac20760 2024-09-15T22:16:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc648018040 con 0x55ef2ac20760 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc650003c40 con 0x55ef2ac20760 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 10 client.25897 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 10 client.25897 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 20 client.25897 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2aca6da0 0x55ef2aca91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 4 client.25897 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x55ef2ab8dd20 con 0x55ef2aca6da0 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.360+0000 7fc657fff640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2aca6da0 0x55ef2aca91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.361+0000 7fc645ffb640 10 client.25897 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.361+0000 7fc657fff640 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2aca6da0 0x55ef2aca91c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d431d67ad7721c server_cookie=b6488cbe5100f2bc in_seq=0 out_seq=0 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.421+0000 7fc645ffb640 1 -- 172.21.15.77:0/2752175696 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ef2ab8dd20 con 0x55ef2aca6da0 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.421+0000 7fc645ffb640 10 client.25897 handle_command_reply: tid=0 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.421+0000 7fc666d66740 1 client.25897 shutdown 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 msgr2=0x55ef2ac33ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6500214f0 tx=0x7fc65000db50 comp rx=0 tx=0).stop 2024-09-15T22:16:49.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 shutdown_connections 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef2aca6da0 0x55ef2aca91c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef2ac2b940 0x55ef2ac34a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef2ac21080 0x55ef2ac34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 --2- 172.21.15.77:0/2752175696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef2ac20760 0x55ef2ac33ff0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc645ffb640 0 client.25897 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 >> 172.21.15.77:0/2752175696 conn(0x55ef2ac1ccf0 msgr2=0x55ef2ac1b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 shutdown_connections 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 1 -- 172.21.15.77:0/2752175696 wait complete. 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc666d66740 20 client.25897 trim_cache size 0 max 16384 2024-09-15T22:16:49.426 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 14, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "e0d6ad4a": { 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 14, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:16:49.427 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 133.8, 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:49.428 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 msgr2=0x7fc64c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 0x7fc64c0a58f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6601051b0 tx=0x7fc648011920 comp rx=0 tx=0).stop 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 msgr2=0x7fc66019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc66019cd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc658009ca0 tx=0x7fc658031e60 comp rx=0 tx=0).stop 2024-09-15T22:16:49.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 shutdown_connections 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc64c0a3430 0x7fc64c0a58f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660106a80 0x7fc66019cd80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc660101420 0x7fc6601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 --2- 172.21.15.77:0/1299624220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc660100a80 0x7fc6601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.422+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 >> 172.21.15.77:0/1299624220 conn(0x7fc6600fc7a0 msgr2=0x7fc66006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.423+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 shutdown_connections 2024-09-15T22:16:49.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:49.423+0000 7fc66498d640 1 -- 172.21.15.77:0/1299624220 wait complete. 2024-09-15T22:16:49.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 13.53 MB/sec execute 58 sec latency 42.932 ms 2024-09-15T22:16:50.362 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e0d6ad4a', quiesce took 3.7 seconds. Will release after: 10.100000000000001 2024-09-15T22:16:50.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 13.30 MB/sec execute 59 sec latency 1039.400 ms 2024-09-15T22:16:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:50 smithi077 ceph-mon[30980]: pgmap v135: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-09-15T22:16:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:50 smithi196 ceph-mon[35109]: pgmap v135: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-09-15T22:16:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:50 smithi117 ceph-mon[35021]: pgmap v135: 129 pgs: 129 active+clean; 210 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 240 op/s 2024-09-15T22:16:51.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 13.08 MB/sec execute 60 sec latency 2039.494 ms 2024-09-15T22:16:51.892 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:52.177 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:52.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 12.87 MB/sec execute 61 sec latency 3039.546 ms 2024-09-15T22:16:52.572 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:52 smithi077 ceph-mon[30980]: pgmap v136: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-09-15T22:16:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:52 smithi196 ceph-mon[35109]: pgmap v136: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-09-15T22:16:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:52 smithi117 ceph-mon[35021]: pgmap v136: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 340 op/s 2024-09-15T22:16:52.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 -- 172.21.15.77:0/725406772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10102790 msgr2=0x7f2e101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 --2- 172.21.15.77:0/725406772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10102790 0x7f2e101067e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009900 tx=0x7f2e0402a410 comp rx=0 tx=0).stop 2024-09-15T22:16:52.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 -- 172.21.15.77:0/725406772 shutdown_connections 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 --2- 172.21.15.77:0/725406772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10106f10 0x7f2e1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 --2- 172.21.15.77:0/725406772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10102790 0x7f2e101067e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 --2- 172.21.15.77:0/725406772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 -- 172.21.15.77:0/725406772 >> 172.21.15.77:0/725406772 conn(0x7f2e100fd8c0 msgr2=0x7f2e100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 -- 172.21.15.77:0/725406772 shutdown_connections 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.651+0000 7f2e14939640 1 -- 172.21.15.77:0/725406772 wait complete. 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 Processor -- start 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 -- start start 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e10198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 0x7f2e10198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10111fb0 con 0x7f2e10106f10 2024-09-15T22:16:52.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10111e30 con 0x7f2e10101de0 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e14939640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10112130 con 0x7f2e10102790 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59170/0 (socket says 172.21.15.77:59170) 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 -- 172.21.15.77:0/1831770991 learned_addr learned my addr 172.21.15.77:0/1831770991 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 0x7f2e10198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e10198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 -- 172.21.15.77:0/1831770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 msgr2=0x7f2e10198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 0x7f2e10198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 -- 172.21.15.77:0/1831770991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 msgr2=0x7f2e10198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.652+0000 7f2e0ed9d640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e10198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e0ed9d640 1 -- 172.21.15.77:0/1831770991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1019d500 con 0x7f2e10106f10 2024-09-15T22:16:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 0x7f2e10198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e10198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e0ed9d640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800cad0 tx=0x7f2e0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2dff7fe640 1 -- 172.21.15.77:0/1831770991 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800ae20 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1019d7f0 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2dff7fe640 1 -- 172.21.15.77:0/1831770991 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e08005020 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2dff7fe640 1 -- 172.21.15.77:0/1831770991 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800f6b0 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.653+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101a50d0 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.655+0000 7f2dff7fe640 1 -- 172.21.15.77:0/1831770991 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2e080051d0 con 0x7f2e10106f10 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.655+0000 7f2e16d12740 1 Processor -- start 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 -- start start 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 0x55c3094e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c3094f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c30944f2e0 con 0x55c3094e94a0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c3093f6ce0 con 0x55c3094f4680 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e16d12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c30944b2a0 con 0x55c3094e9dc0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 0x55c3094e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 0x55c3094e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59184/0 (socket says 172.21.15.77:59184) 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0dd9b640 1 -- 172.21.15.77:0/3356873523 learned_addr learned my addr 172.21.15.77:0/3356873523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2dff7fe640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 0x7f2df80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c3094f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0ed9d640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 0x7f2df80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3356873523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 msgr2=0x55c3094f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c3094f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3356873523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 msgr2=0x55c3094e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 0x55c3094e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3356873523 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3093db440 con 0x55c3094e9dc0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c3094f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.656+0000 7f2dff7fe640 1 -- 172.21.15.77:0/1831770991 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2e080d0400 con 0x7f2e10106f10 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.657+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f4140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000a940 tx=0x7f2e0000ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc3aa3ab0ad2e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.657+0000 7f2e0ed9d640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 0x7f2df80a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cd960 tx=0x7f2e08005960 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.657+0000 7f2dfcff9640 1 -- 172.21.15.77:0/3356873523 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00015070 con 0x55c3094e9dc0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.657+0000 7f2dfcff9640 1 -- 172.21.15.77:0/3356873523 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0000cc00 con 0x55c3094e9dc0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.657+0000 7f2dfcff9640 1 -- 172.21.15.77:0/3356873523 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000185c0 con 0x55c3094e9dc0 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- 172.21.15.77:0/3356873523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 msgr2=0x55c3094f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f4140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000a940 tx=0x7f2e0000ae00 comp rx=0 tx=0).stop 2024-09-15T22:16:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- 172.21.15.77:0/3356873523 shutdown_connections 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c3094f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f4140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3356873523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e94a0 0x55c3094e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- 172.21.15.77:0/3356873523 >> 172.21.15.77:0/3356873523 conn(0x55c3094e5a30 msgr2=0x55c3094e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- 172.21.15.77:0/3356873523 shutdown_connections 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- 172.21.15.77:0/3356873523 wait complete. 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 -- start start 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 Processor -- start 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.658+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c3094f7250 con 0x55c3094f4680 2024-09-15T22:16:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c30940a050 con 0x55c3094e94a0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3094f73d0 con 0x55c3094e9dc0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 10 client.0 _ll_register_callbacks cb 0x55c30944a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59192/0 (socket says 172.21.15.77:59192) 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 -- 172.21.15.77:0/1390657407 learned_addr learned my addr 172.21.15.77:0/1390657407 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 -- 172.21.15.77:0/1390657407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 msgr2=0x55c3094f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 -- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 msgr2=0x55c3094f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 -- 172.21.15.77:0/1390657407 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3091c03c0 con 0x55c3094f4680 2024-09-15T22:16:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009230 tx=0x7f2e04005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8309ae6a550efa98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2df5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04035070 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2e040054f0 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0000cc00 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2e04004020 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 10 client.16677 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.659+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c309502ad0 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0000cc00 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2e040035f0 con 0x55c3094f4680 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 10 client.16677 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 20 client.16677 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:52.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c309503ee0 0x55c3095062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 4 client.16677 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c30945c850 con 0x55c309503ee0 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c309503ee0 0x55c3095062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c309503ee0 0x55c3095062f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1878e80771af436c server_cookie=e2aa77efc0a9f45e in_seq=0 out_seq=0 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.660+0000 7f2df5ffb640 10 client.16677 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2df5ffb640 1 -- 172.21.15.77:0/1390657407 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c30945c850 con 0x55c309503ee0 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2df5ffb640 10 client.16677 handle_command_reply: tid=0 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 client.16677 shutdown 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 msgr2=0x55c3094f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009230 tx=0x7f2e04005a40 comp rx=0 tx=0).stop 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 shutdown_connections 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c309503ee0 0x55c3095062f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094f4680 0x55c3094f16f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e9dc0 0x55c3094f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 --2- 172.21.15.77:0/1390657407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2df5ffb640 0 client.16677 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 >> 172.21.15.77:0/1390657407 conn(0x55c3094e5a30 msgr2=0x55c3094e4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 shutdown_connections 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 1 -- 172.21.15.77:0/1390657407 wait complete. 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.661+0000 7f2e16d12740 20 client.16677 trim_cache size 0 max 16384 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.678+0000 7f2e16d12740 1 Processor -- start 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.678+0000 7f2e16d12740 1 -- start start 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e94a0 0x55c3095004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c30950c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101aefe0 con 0x55c3094e9dc0 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b0b20 con 0x55c3094f4680 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e16d12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101af6b0 con 0x55c3094e94a0 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59198/0 (socket says 172.21.15.77:59198) 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3318962885 learned_addr learned my addr 172.21.15.77:0/3318962885 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c30950c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e94a0 0x55c3095004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3318962885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e94a0 msgr2=0x55c3095004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e94a0 0x55c3095004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3318962885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 msgr2=0x55c30950c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c30950c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.679+0000 7f2e0e59c640 1 -- 172.21.15.77:0/3318962885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3094f1f70 con 0x55c3094e9dc0 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55c3094e6890 tx=0x7f2e0000a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e997d879b9ae815d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2df7fff640 1 -- 172.21.15.77:0/3318962885 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00015040 con 0x55c3094e9dc0 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2df7fff640 1 -- 172.21.15.77:0/3318962885 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0000cbc0 con 0x55c3094e9dc0 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2df7fff640 1 -- 172.21.15.77:0/3318962885 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000279c0 con 0x55c3094e9dc0 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 -- 172.21.15.77:0/3318962885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 msgr2=0x55c30950bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55c3094e6890 tx=0x7f2e0000a840 comp rx=0 tx=0).stop 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 -- 172.21.15.77:0/3318962885 shutdown_connections 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094f4680 0x55c30950c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c30950bbc0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 --2- 172.21.15.77:0/3318962885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094e94a0 0x55c3095004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.680+0000 7f2e16d12740 1 -- 172.21.15.77:0/3318962885 >> 172.21.15.77:0/3318962885 conn(0x55c3094e5a30 msgr2=0x55c3094efa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- 172.21.15.77:0/3318962885 shutdown_connections 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- 172.21.15.77:0/3318962885 wait complete. 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- start start 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 Processor -- start 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 0x55c3094f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b0450 con 0x55c3094e9dc0 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101afd80 con 0x55c3094e94a0 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101acfc0 con 0x55c3094f4680 2024-09-15T22:16:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e16d12740 10 client.0 _ll_register_callbacks cb 0x7f2e04032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e0e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e0e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59210/0 (socket says 172.21.15.77:59210) 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e0e59c640 1 -- 172.21.15.77:0/2113331911 learned_addr learned my addr 172.21.15.77:0/2113331911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.681+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 0x55c3094f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 -- 172.21.15.77:0/2113331911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 msgr2=0x55c3094f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 0x55c3094f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 -- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 msgr2=0x55c3094f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 -- 172.21.15.77:0/2113331911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30956dcf0 con 0x55c3094e9dc0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 0x55c3094f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e0e59c640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e000275a0 tx=0x7f2e0000c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771d02f246e6b406 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00033020 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2e000115f0 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0000cbc0 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2e0003e050 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 10 client.16683 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c30956efa0 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0000cbc0 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2e00024070 con 0x55c3094e9dc0 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 10 client.16683 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.682+0000 7f2e16d12740 20 client.16683 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c30956f370 0x55c309571790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 4 client.16683 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3094f1a80 con 0x55c30956f370 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c30956f370 0x55c309571790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e0dd9b640 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c30956f370 0x55c309571790 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a564dd7a6fdaba server_cookie=f8f8a7553584f0bc in_seq=0 out_seq=0 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2df5ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2df5ffb640 1 -- 172.21.15.77:0/2113331911 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3094f1a80 con 0x55c30956f370 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2df5ffb640 10 client.16683 handle_command_reply: tid=0 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 client.16683 shutdown 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 msgr2=0x55c3094f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2e000275a0 tx=0x7f2e0000c660 comp rx=0 tx=0).stop 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 shutdown_connections 2024-09-15T22:16:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c30956f370 0x55c309571790 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3094f4680 0x55c3094f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3094e9dc0 0x55c3094f89f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 --2- 172.21.15.77:0/2113331911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3094e94a0 0x55c3094f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.683+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 >> 172.21.15.77:0/2113331911 conn(0x55c3094e5a30 msgr2=0x55c3094f3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2df5ffb640 0 client.16683 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 shutdown_connections 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e16d12740 1 -- 172.21.15.77:0/2113331911 wait complete. 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e16d12740 20 client.16683 trim_cache size 0 max 16384 2024-09-15T22:16:52.703 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:16:52.704 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:16:52.704 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:16:52.704 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 msgr2=0x7f2df80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 0x7f2df80a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2e080cd960 tx=0x7f2e08005960 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 msgr2=0x7f2e1019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800cad0 tx=0x7f2e0800cf90 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 shutdown_connections 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2df80a33b0 0x7f2df80a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e10106f10 0x7f2e1019ce20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e10102790 0x7f2e10198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 --2- 172.21.15.77:0/1831770991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e10101de0 0x7f2e10198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 >> 172.21.15.77:0/1831770991 conn(0x7f2e100fd8c0 msgr2=0x7f2e100fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:52.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 shutdown_connections 2024-09-15T22:16:52.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:52.684+0000 7f2e14939640 1 -- 172.21.15.77:0/1831770991 wait complete. 2024-09-15T22:16:53.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 12.66 MB/sec execute 62 sec latency 4039.600 ms 2024-09-15T22:16:53.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:16:53.580 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- 172.21.15.77:0/3907452907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 msgr2=0x7f6f1010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 --2- 172.21.15.77:0/3907452907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f1010d950 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009930 tx=0x7f6f0002a420 comp rx=0 tx=0).stop 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- 172.21.15.77:0/3907452907 shutdown_connections 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 --2- 172.21.15.77:0/3907452907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f1010d950 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 --2- 172.21.15.77:0/3907452907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 0x7f6f10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 --2- 172.21.15.77:0/3907452907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 0x7f6f10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- 172.21.15.77:0/3907452907 >> 172.21.15.77:0/3907452907 conn(0x7f6f100fc7a0 msgr2=0x7f6f100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- 172.21.15.77:0/3907452907 shutdown_connections 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- 172.21.15.77:0/3907452907 wait complete. 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 Processor -- start 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.326+0000 7f6f15624640 1 -- start start 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 0x7f6f101991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 0x7f6f101a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10110e80 con 0x7f6f10101420 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10110d00 con 0x7f6f10100a80 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f15624640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10111000 con 0x7f6f10106a80 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59226/0 (socket says 172.21.15.77:59226) 2024-09-15T22:16:54.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 -- 172.21.15.77:0/1430157649 learned_addr learned my addr 172.21.15.77:0/1430157649 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0effd640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 0x7f6f101991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0f7fe640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 0x7f6f101a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 -- 172.21.15.77:0/1430157649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 msgr2=0x7f6f101a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 0x7f6f101a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 -- 172.21.15.77:0/1430157649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 msgr2=0x7f6f101991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 0x7f6f101991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 -- 172.21.15.77:0/1430157649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f101a47a0 con 0x7f6f10101420 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.327+0000 7f6f0e7fc640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c920 tx=0x7f6ef800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.328+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8007c70 con 0x7f6f10101420 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.328+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f101a4970 con 0x7f6f10101420 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.328+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef8007e10 con 0x7f6f10101420 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.328+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8018890 con 0x7f6f10101420 2024-09-15T22:16:54.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.328+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f101a4eb0 con 0x7f6f10101420 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.329+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f10100ee0 con 0x7f6f10101420 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.330+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6ef8018b30 con 0x7f6f10101420 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.330+0000 7f6f07fff640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 0x7f6efc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.330+0000 7f6f0effd640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 0x7f6efc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.331+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6ef80d01b0 con 0x7f6f10101420 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.331+0000 7f6f0effd640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 0x7f6efc0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6f08005fd0 tx=0x7f6f08009680 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:54.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.334+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ef8014030 con 0x7f6f10101420 2024-09-15T22:16:54.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 12.46 MB/sec execute 63 sec latency 5039.646 ms 2024-09-15T22:16:54.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.540+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f101a5140 con 0x7f6f10101420 2024-09-15T22:16:54.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:54 smithi077 ceph-mon[30980]: pgmap v137: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:54.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.541+0000 7f6f07fff640 1 -- 172.21.15.77:0/1430157649 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6ef8018e20 con 0x7f6f10101420 2024-09-15T22:16:54.544 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:16:54.544 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:16:54.544 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 msgr2=0x7f6efc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 0x7f6efc0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6f08005fd0 tx=0x7f6f08009680 comp rx=0 tx=0).stop 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 msgr2=0x7f6f10199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c920 tx=0x7f6ef800cde0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 shutdown_connections 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6efc0a33b0 0x7f6efc0a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f10106a80 0x7f6f101a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f10101420 0x7f6f10199720 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 --2- 172.21.15.77:0/1430157649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f10100a80 0x7f6f101991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.543+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 >> 172.21.15.77:0/1430157649 conn(0x7f6f100fc7a0 msgr2=0x7f6f100fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:54.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.544+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 shutdown_connections 2024-09-15T22:16:54.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:54.544+0000 7f6f15624640 1 -- 172.21.15.77:0/1430157649 wait complete. 2024-09-15T22:16:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:54 smithi196 ceph-mon[35109]: pgmap v137: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:54 smithi117 ceph-mon[35021]: pgmap v137: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:55.330 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:16:55.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 12.26 MB/sec execute 64 sec latency 6039.698 ms 2024-09-15T22:16:55.580 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:55.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:55 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1430157649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:55.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:55 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1430157649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:55.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:55 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1430157649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:56.269 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:56 smithi077 ceph-mon[30980]: pgmap v138: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:56.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 -- 172.21.15.77:0/347918437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 msgr2=0x7fccb810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/347918437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb810eef0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009860 tx=0x7fcca802a360 comp rx=0 tx=0).stop 2024-09-15T22:16:56.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 -- 172.21.15.77:0/347918437 shutdown_connections 2024-09-15T22:16:56.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/347918437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb8109b90 0x7fccb81158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/347918437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb81091f0 0x7fccb8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/347918437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb810eef0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.282+0000 7fccbfcb2640 1 -- 172.21.15.77:0/347918437 >> 172.21.15.77:0/347918437 conn(0x7fccb806e2d0 msgr2=0x7fccb806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.283+0000 7fccbfcb2640 1 -- 172.21.15.77:0/347918437 shutdown_connections 2024-09-15T22:16:56.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.283+0000 7fccbfcb2640 1 -- 172.21.15.77:0/347918437 wait complete. 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.283+0000 7fccbfcb2640 1 Processor -- start 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 -- start start 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb81091f0 0x7fccb8114280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb81147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8117be0 con 0x7fccb810eb10 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8117a60 con 0x7fccb8109b90 2024-09-15T22:16:56.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbfcb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8117d60 con 0x7fccb81091f0 2024-09-15T22:16:56.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb81091f0 0x7fccb8114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55482/0 (socket says 172.21.15.77:55482) 2024-09-15T22:16:56.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.284+0000 7fccbd24d640 1 -- 172.21.15.77:0/4147589904 learned_addr learned my addr 172.21.15.77:0/4147589904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbe24f640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb81147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 -- 172.21.15.77:0/4147589904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb81091f0 msgr2=0x7fccb8114280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb81091f0 0x7fccb8114280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 -- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 msgr2=0x7fccb81147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb81147c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 -- 172.21.15.77:0/4147589904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb81bbd50 con 0x7fccb8109b90 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e9c0 tx=0x7fccb400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccaeffd640 1 -- 172.21.15.77:0/4147589904 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb400ccc0 con 0x7fccb8109b90 2024-09-15T22:16:56.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb81bbfe0 con 0x7fccb8109b90 2024-09-15T22:16:56.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccaeffd640 1 -- 172.21.15.77:0/4147589904 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb4005500 con 0x7fccb8109b90 2024-09-15T22:16:56.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.285+0000 7fccaeffd640 1 -- 172.21.15.77:0/4147589904 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4010430 con 0x7fccb8109b90 2024-09-15T22:16:56.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.286+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81bc490 con 0x7fccb8109b90 2024-09-15T22:16:56.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.288+0000 7fccaeffd640 1 -- 172.21.15.77:0/4147589904 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fccb40046c0 con 0x7fccb8109b90 2024-09-15T22:16:56.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 Processor -- start 2024-09-15T22:16:56.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 -- start start 2024-09-15T22:16:56.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccaeffd640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 0x7fcca40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 0x56397b961090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b96e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56397b8c72f0 con 0x56397b9615d0 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56397b86ecf0 con 0x56397b962b40 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.289+0000 7fccc208b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56397b8c32b0 con 0x56397b96c370 2024-09-15T22:16:56.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbe24f640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 0x7fcca40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccaeffd640 1 -- 172.21.15.77:0/4147589904 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fccb40d0490 con 0x7fccb8109b90 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 0x56397b961090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59252/0 (socket says 172.21.15.77:59252) 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 -- 172.21.15.77:0/3694046291 learned_addr learned my addr 172.21.15.77:0/3694046291 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbe24f640 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b96e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 -- 172.21.15.77:0/3694046291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 msgr2=0x56397b96e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b96e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 -- 172.21.15.77:0/3694046291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 msgr2=0x56397b961090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 0x56397b961090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 -- 172.21.15.77:0/3694046291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397b8533a0 con 0x56397b9615d0 2024-09-15T22:16:56.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbda4e640 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 0x56397b961090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:56.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c6a0 tx=0x7fccb4005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c65f2bc212c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fccbe24f640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 0x7fcca40a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fccb00092f0 tx=0x7fccb0009f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fcca3fff640 1 -- 172.21.15.77:0/3694046291 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4005020 con 0x56397b9615d0 2024-09-15T22:16:56.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fcca3fff640 1 -- 172.21.15.77:0/3694046291 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb401f730 con 0x56397b9615d0 2024-09-15T22:16:56.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.290+0000 7fcca3fff640 1 -- 172.21.15.77:0/3694046291 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb40d0110 con 0x56397b9615d0 2024-09-15T22:16:56.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 -- 172.21.15.77:0/3694046291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 msgr2=0x56397b961a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c6a0 tx=0x7fccb4005320 comp rx=0 tx=0).stop 2024-09-15T22:16:56.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 -- 172.21.15.77:0/3694046291 shutdown_connections 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b96e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b961a10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 --2- 172.21.15.77:0/3694046291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b962b40 0x56397b961090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 -- 172.21.15.77:0/3694046291 >> 172.21.15.77:0/3694046291 conn(0x56397b92a310 msgr2=0x56397b95dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 -- 172.21.15.77:0/3694046291 shutdown_connections 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.291+0000 7fccc208b740 1 -- 172.21.15.77:0/3694046291 wait complete. 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 -- start start 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 Processor -- start 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b969060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56397b96f010 con 0x56397b9615d0 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56397b882060 con 0x56397b96c370 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56397b96f190 con 0x56397b962b40 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccc208b740 10 client.0 _ll_register_callbacks cb 0x56397b5d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccbe24f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b969060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59262/0 (socket says 172.21.15.77:59262) 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.292+0000 7fccbe24f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55504/0 (socket says 172.21.15.77:55504) 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 -- 172.21.15.77:0/4063041603 learned_addr learned my addr 172.21.15.77:0/4063041603 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 -- 172.21.15.77:0/4063041603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 msgr2=0x56397b969060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b969060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 -- 172.21.15.77:0/4063041603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 msgr2=0x56397b9697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 -- 172.21.15.77:0/4063041603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397b6383c0 con 0x56397b9615d0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbe24f640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b969060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccbda4e640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009860 tx=0x7fcca8030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4477e8e92aaae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fcca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8037040 con 0x56397b9615d0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fcca8005040 con 0x56397b9615d0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb401f730 con 0x56397b9615d0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcca80309b0 con 0x56397b9615d0 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccc208b740 10 client.16698 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:56.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.293+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56397b97ab00 con 0x56397b9615d0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca80046c0 con 0x56397b9615d0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcca8032440 con 0x56397b9615d0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 10 client.16698 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 10 client.16698 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 20 client.16698 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b97e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 4 client.16698 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b97e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fccb401f730 con 0x56397b97bfc0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fccbd24d640 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b97e3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b1287334e8cd9ff server_cookie=379240a6675c9fee in_seq=0 out_seq=0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.294+0000 7fcca17fa640 10 client.16698 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fcca17fa640 1 -- 172.21.15.77:0/4063041603 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fccb401f730 con 0x56397b97bfc0 2024-09-15T22:16:56.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fcca17fa640 10 client.16698 handle_command_reply: tid=0 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 client.16698 shutdown 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 msgr2=0x56397b96bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009860 tx=0x7fcca8030340 comp rx=0 tx=0).stop 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 shutdown_connections 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b97e3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b9697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b969060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fcca17fa640 0 client.16698 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 --2- 172.21.15.77:0/4063041603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b96bdb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 >> 172.21.15.77:0/4063041603 conn(0x56397b92a310 msgr2=0x56397b963990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.295+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 shutdown_connections 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.296+0000 7fccc208b740 1 -- 172.21.15.77:0/4063041603 wait complete. 2024-09-15T22:16:56.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.296+0000 7fccc208b740 20 client.16698 trim_cache size 0 max 16384 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 Processor -- start 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 -- start start 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b95d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b964180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c65d0 con 0x56397b9615d0 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c8110 con 0x56397b96c370 2024-09-15T22:16:56.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccc208b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c6ca0 con 0x56397b962b40 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59272/0 (socket says 172.21.15.77:59272) 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccbda4e640 1 -- 172.21.15.77:0/557966335 learned_addr learned my addr 172.21.15.77:0/557966335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccbd24d640 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b95d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.312+0000 7fccbe24f640 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b964180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 -- 172.21.15.77:0/557966335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 msgr2=0x56397b95d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b95d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 -- 172.21.15.77:0/557966335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 msgr2=0x56397b964180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b964180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 -- 172.21.15.77:0/557966335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397b9675b0 con 0x56397b9615d0 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fccbda4e640 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcca8005e00 tx=0x7fcca8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c3f8aa9d1ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fcca37fe640 1 -- 172.21.15.77:0/557966335 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8037040 con 0x56397b9615d0 2024-09-15T22:16:56.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fcca37fe640 1 -- 172.21.15.77:0/557966335 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca8032b20 con 0x56397b9615d0 2024-09-15T22:16:56.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.313+0000 7fcca37fe640 1 -- 172.21.15.77:0/557966335 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8005290 con 0x56397b9615d0 2024-09-15T22:16:56.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- 172.21.15.77:0/557966335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 msgr2=0x56397b95cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcca8005e00 tx=0x7fcca8001e70 comp rx=0 tx=0).stop 2024-09-15T22:16:56.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- 172.21.15.77:0/557966335 shutdown_connections 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b96c370 0x56397b964180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b962b40 0x56397b95d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 --2- 172.21.15.77:0/557966335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b9615d0 0x56397b95cbf0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- 172.21.15.77:0/557966335 >> 172.21.15.77:0/557966335 conn(0x56397b92a310 msgr2=0x56397b95c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- 172.21.15.77:0/557966335 shutdown_connections 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- 172.21.15.77:0/557966335 wait complete. 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 -- start start 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.314+0000 7fccc208b740 1 Processor -- start 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b970ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbd24d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59276/0 (socket says 172.21.15.77:59276) 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbd24d640 1 -- 172.21.15.77:0/3024089966 learned_addr learned my addr 172.21.15.77:0/3024089966 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbda4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55532/0 (socket says 172.21.15.77:55532) 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c7a40 con 0x56397b962b40 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c7370 con 0x56397b9615d0 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81c45b0 con 0x56397b96c370 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccc208b740 10 client.0 _ll_register_callbacks cb 0x7fccb4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.315+0000 7fccbe24f640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b970ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 -- 172.21.15.77:0/3024089966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 msgr2=0x56397b970ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b970ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 -- 172.21.15.77:0/3024089966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 msgr2=0x56397b970450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397b9e5e30 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbda4e640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccbd24d640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fccb4000c00 tx=0x7fccb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47b687152221239 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb409cb30 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fccb40d0110 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca8032b20 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fccb40a07e0 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccc208b740 10 client.16704 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56397b9e7090 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca8032b20 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.316+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fccb40b5130 con 0x56397b962b40 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 10 client.16704 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:56.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 10 client.16704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 20 client.16704 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b9e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 4 client.16704 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56397b8d4860 con 0x56397b97bfc0 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccbda4e640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b9e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccbda4e640 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b9e9450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c59b153a7ab46d98 server_cookie=76263b12b7498dee in_seq=0 out_seq=0 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fcca17fa640 10 client.16704 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fcca17fa640 1 -- 172.21.15.77:0/3024089966 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56397b8d4860 con 0x56397b97bfc0 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fcca17fa640 10 client.16704 handle_command_reply: tid=0 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 1 client.16704 shutdown 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 msgr2=0x56397b970990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.317+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fccb4000c00 tx=0x7fccb4009f90 comp rx=0 tx=0).stop 2024-09-15T22:16:56.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 shutdown_connections 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56397b97bfc0 0x56397b9e9450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56397b96c370 0x56397b970ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56397b962b40 0x56397b970990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fcca17fa640 0 client.16704 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 --2- 172.21.15.77:0/3024089966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56397b9615d0 0x56397b970450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 >> 172.21.15.77:0/3024089966 conn(0x56397b92a310 msgr2=0x56397b95cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 shutdown_connections 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 1 -- 172.21.15.77:0/3024089966 wait complete. 2024-09-15T22:16:56.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccc208b740 20 client.16704 trim_cache size 0 max 16384 2024-09-15T22:16:56.327 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:56.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 msgr2=0x7fcca40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 0x7fcca40a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fccb00092f0 tx=0x7fccb0009f90 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 msgr2=0x7fccb8123870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e9c0 tx=0x7fccb400ee80 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.318+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 shutdown_connections 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcca40a33b0 0x7fcca40a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb810eb10 0x7fccb81147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccb8109b90 0x7fccb8123870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 --2- 172.21.15.77:0/4147589904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb81091f0 0x7fccb8114280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 >> 172.21.15.77:0/4147589904 conn(0x7fccb806e2d0 msgr2=0x7fccb8073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 shutdown_connections 2024-09-15T22:16:56.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:56.319+0000 7fccbfcb2640 1 -- 172.21.15.77:0/4147589904 wait complete. 2024-09-15T22:16:56.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 12.07 MB/sec execute 65 sec latency 7039.777 ms 2024-09-15T22:16:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:56 smithi196 ceph-mon[35109]: pgmap v138: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:56 smithi117 ceph-mon[35021]: pgmap v138: 129 pgs: 129 active+clean; 215 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-15T22:16:57.169 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:16:57.413 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:57.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.89 MB/sec execute 66 sec latency 8039.879 ms 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- 172.21.15.77:0/148816852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974106f10 msgr2=0x7f097410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 --2- 172.21.15.77:0/148816852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974106f10 0x7f097410ea90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f096c009ca0 tx=0x7f096c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- 172.21.15.77:0/148816852 shutdown_connections 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 --2- 172.21.15.77:0/148816852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974106f10 0x7f097410ea90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 --2- 172.21.15.77:0/148816852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974102790 0x7f09741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 --2- 172.21.15.77:0/148816852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 0x7f09741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- 172.21.15.77:0/148816852 >> 172.21.15.77:0/148816852 conn(0x7f09740fd880 msgr2=0x7f09740ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- 172.21.15.77:0/148816852 shutdown_connections 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- 172.21.15.77:0/148816852 wait complete. 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 Processor -- start 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.931+0000 7f097b660640 1 -- start start 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 0x7f0974198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974106f10 0x7f097419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0974111f70 con 0x7f0974102790 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0974111df0 con 0x7f0974101de0 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f097b660640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09741120f0 con 0x7f0974106f10 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59302/0 (socket says 172.21.15.77:59302) 2024-09-15T22:16:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 -- 172.21.15.77:0/2296585044 learned_addr learned my addr 172.21.15.77:0/2296585044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0979bfd640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974106f10 0x7f097419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f09793fc640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 0x7f0974198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 -- 172.21.15.77:0/2296585044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974106f10 msgr2=0x7f097419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974106f10 0x7f097419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 -- 172.21.15.77:0/2296585044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 msgr2=0x7f0974198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 0x7f0974198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 -- 172.21.15.77:0/2296585044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097419d2c0 con 0x7f0974102790 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f097000d770 tx=0x7f097000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.932+0000 7f09667fc640 1 -- 172.21.15.77:0/2296585044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970017070 con 0x7f0974102790 2024-09-15T22:16:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.933+0000 7f09667fc640 1 -- 172.21.15.77:0/2296585044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0970009820 con 0x7f0974102790 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.933+0000 7f09667fc640 1 -- 172.21.15.77:0/2296585044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970004cd0 con 0x7f0974102790 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.933+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097419d5b0 con 0x7f0974102790 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.933+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741a4e90 con 0x7f0974102790 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f09667fc640 1 -- 172.21.15.77:0/2296585044 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f09700099d0 con 0x7f0974102790 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 Processor -- start 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 -- start start 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f497f5d0 0x5636f497fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 0x5636f498c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636f48e52f0 con 0x5636f4980b40 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636f488ccf0 con 0x5636f498a370 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f097da39740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636f48e12b0 con 0x5636f497f5d0 2024-09-15T22:16:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.935+0000 7f0979bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59304/0 (socket says 172.21.15.77:59304) 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 -- 172.21.15.77:0/937198375 learned_addr learned my addr 172.21.15.77:0/937198375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0978bfb640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 0x5636f498c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f09793fc640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f497f5d0 0x5636f497fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 -- 172.21.15.77:0/937198375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f497f5d0 msgr2=0x5636f497fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f497f5d0 0x5636f497fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f09667fc640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 0x7f09600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 -- 172.21.15.77:0/937198375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 msgr2=0x5636f498c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 0x5636f498c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 -- 172.21.15.77:0/937198375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f48713a0 con 0x5636f4980b40 2024-09-15T22:16:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0978bfb640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 0x5636f498c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f09667fc640 1 -- 172.21.15.77:0/2296585044 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f09700cf980 con 0x7f0974102790 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f096c035000 tx=0x7f096c02f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e2c04db1f4eea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f0979bfd640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 0x7f09600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f095f7fe640 1 -- 172.21.15.77:0/937198375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c04a040 con 0x5636f4980b40 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f095f7fe640 1 -- 172.21.15.77:0/937198375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c036040 con 0x5636f4980b40 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.936+0000 7f095f7fe640 1 -- 172.21.15.77:0/937198375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c004850 con 0x5636f4980b40 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- 172.21.15.77:0/937198375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 msgr2=0x5636f497f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f0979bfd640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 0x7f09600a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f096c045000 tx=0x7f096c02fbe0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f096c035000 tx=0x7f096c02f500 comp rx=0 tx=0).stop 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- 172.21.15.77:0/937198375 shutdown_connections 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f498a370 0x5636f498c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f497f5d0 0x5636f497fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 --2- 172.21.15.77:0/937198375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f497f090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- 172.21.15.77:0/937198375 >> 172.21.15.77:0/937198375 conn(0x5636f4948310 msgr2=0x5636f497bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- 172.21.15.77:0/937198375 shutdown_connections 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- 172.21.15.77:0/937198375 wait complete. 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.937+0000 7f097da39740 1 -- start start 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 Processor -- start 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4989ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 0x5636f4987490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636f498ce90 con 0x5636f4980b40 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636f48a0060 con 0x5636f497f5d0 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636f498d010 con 0x5636f498a370 2024-09-15T22:16:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f097da39740 10 client.0 _ll_register_callbacks cb 0x5636f45f38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59316/0 (socket says 172.21.15.77:59316) 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 -- 172.21.15.77:0/3369197075 learned_addr learned my addr 172.21.15.77:0/3369197075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0979bfd640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 0x5636f4987490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f09793fc640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4989ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 -- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 msgr2=0x5636f4987490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 0x5636f4987490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 -- 172.21.15.77:0/3369197075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 msgr2=0x5636f4989ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4989ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 -- 172.21.15.77:0/3369197075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f46563c0 con 0x5636f4980b40 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0979bfd640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 0x5636f4987490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f0978bfb640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0970009610 tx=0x7f097009b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d94377bc55fef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.938+0000 7f095cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970020070 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f0970097f90 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c036040 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f097009b920 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 10 client.16716 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636f4998880 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096c036040 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f097009af00 con 0x5636f4980b40 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 10 client.16716 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 10 client.16716 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 20 client.16716 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f4999c90 0x5636f499c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 4 client.16716 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.939+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f48f2860 con 0x5636f4999c90 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.940+0000 7f09793fc640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f4999c90 0x5636f499c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.940+0000 7f095cff9640 10 client.16716 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.940+0000 7f09793fc640 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f4999c90 0x5636f499c0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12cb31fbe950bbc6 server_cookie=a7263098dbec99d in_seq=0 out_seq=0 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f095cff9640 1 -- 172.21.15.77:0/3369197075 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636f48f2860 con 0x5636f4999c90 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f095cff9640 10 client.16716 handle_command_reply: tid=0 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 client.16716 shutdown 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 msgr2=0x5636f4986f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0970009610 tx=0x7f097009b520 comp rx=0 tx=0).stop 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 shutdown_connections 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f4999c90 0x5636f499c0a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f498a370 0x5636f4987490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4986f50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 --2- 172.21.15.77:0/3369197075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4989ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f095cff9640 0 client.16716 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 >> 172.21.15.77:0/3369197075 conn(0x5636f4948310 msgr2=0x5636f4981960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 shutdown_connections 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 1 -- 172.21.15.77:0/3369197075 wait complete. 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.941+0000 7f097da39740 20 client.16716 trim_cache size 0 max 16384 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 Processor -- start 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 -- start start 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f4948310 0x5636f4986810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4980b40 0x5636f4982120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09741aed10 con 0x5636f497f5d0 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09741b0850 con 0x5636f4948310 2024-09-15T22:16:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f097da39740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09741af3e0 con 0x5636f4980b40 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59332/0 (socket says 172.21.15.77:59332) 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 -- 172.21.15.77:0/2800862806 learned_addr learned my addr 172.21.15.77:0/2800862806 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f09793fc640 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f4948310 0x5636f4986810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0979bfd640 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4980b40 0x5636f4982120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 -- 172.21.15.77:0/2800862806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4980b40 msgr2=0x5636f4982120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4980b40 0x5636f4982120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 -- 172.21.15.77:0/2800862806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f4948310 msgr2=0x5636f4986810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f4948310 0x5636f4986810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.954+0000 7f0978bfb640 1 -- 172.21.15.77:0/2800862806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f49854a0 con 0x5636f497f5d0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f0978bfb640 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f097009a830 tx=0x7f097009b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61c8fac41ca9bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f095effd640 1 -- 172.21.15.77:0/2800862806 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970020070 con 0x5636f497f5d0 2024-09-15T22:16:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f095effd640 1 -- 172.21.15.77:0/2800862806 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09700a4590 con 0x5636f497f5d0 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f095effd640 1 -- 172.21.15.77:0/2800862806 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097009afd0 con 0x5636f497f5d0 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 -- 172.21.15.77:0/2800862806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 msgr2=0x5636f4988ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f097009a830 tx=0x7f097009b590 comp rx=0 tx=0).stop 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 -- 172.21.15.77:0/2800862806 shutdown_connections 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4980b40 0x5636f4982120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f497f5d0 0x5636f4988ec0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 --2- 172.21.15.77:0/2800862806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f4948310 0x5636f4986810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 -- 172.21.15.77:0/2800862806 >> 172.21.15.77:0/2800862806 conn(0x5636f49875f0 msgr2=0x5636f49887b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 -- 172.21.15.77:0/2800862806 shutdown_connections 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.955+0000 7f097da39740 1 -- 172.21.15.77:0/2800862806 wait complete. 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 -- start start 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 Processor -- start 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4948310 0x5636f4992b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4983dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09741b0180 con 0x5636f4980b40 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09741afab0 con 0x5636f497f5d0 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09741accf0 con 0x5636f4948310 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f097da39740 10 client.0 _ll_register_callbacks cb 0x7f0970005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f0979bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f0979bfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59348/0 (socket says 172.21.15.77:59348) 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f0979bfd640 1 -- 172.21.15.77:0/1326361100 learned_addr learned my addr 172.21.15.77:0/1326361100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f0978bfb640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4983dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.956+0000 7f09793fc640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4948310 0x5636f4992b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 -- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4948310 msgr2=0x5636f4992b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4948310 0x5636f4992b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 -- 172.21.15.77:0/1326361100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 msgr2=0x5636f4983dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4983dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 -- 172.21.15.77:0/1326361100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f4a03940 con 0x5636f4980b40 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f0979bfd640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f096c009bf0 tx=0x7f096c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e15fa01515cd66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c05f070 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f096c030b60 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09700a4590 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f096c067050 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f097da39740 10 client.16722 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636f4a04a90 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09700a4590 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.957+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f096c062030 con 0x5636f4980b40 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 10 client.16722 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 10 client.16722 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 20 client.16722 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f498a370 0x5636f4a06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 4 client.16722 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636f48ecd70 con 0x5636f498a370 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f09793fc640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f498a370 0x5636f4a06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f095cff9640 10 client.16722 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f09793fc640 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f498a370 0x5636f4a06f40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab8f864c0938733 server_cookie=c98b73601e5cd8f9 in_seq=0 out_seq=0 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f095cff9640 1 -- 172.21.15.77:0/1326361100 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636f48ecd70 con 0x5636f498a370 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f095cff9640 10 client.16722 handle_command_reply: tid=0 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 client.16722 shutdown 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 msgr2=0x5636f4984550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f096c009bf0 tx=0x7f096c043000 comp rx=0 tx=0).stop 2024-09-15T22:16:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 shutdown_connections 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5636f498a370 0x5636f4a06f40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636f4980b40 0x5636f4984550 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f095cff9640 0 client.16722 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636f497f5d0 0x5636f4983dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 --2- 172.21.15.77:0/1326361100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636f4948310 0x5636f4992b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.958+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 >> 172.21.15.77:0/1326361100 conn(0x5636f49875f0 msgr2=0x5636f49887b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 shutdown_connections 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097da39740 1 -- 172.21.15.77:0/1326361100 wait complete. 2024-09-15T22:16:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097da39740 20 client.16722 trim_cache size 0 max 16384 2024-09-15T22:16:57.979 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 msgr2=0x7f09600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 0x7f09600a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f096c045000 tx=0x7f096c02fbe0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 msgr2=0x7f0974198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f097000d770 tx=0x7f097000db70 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 shutdown_connections 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09600a33b0 0x7f09600a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0974106f10 0x7f097419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0974102790 0x7f0974198850 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 --2- 172.21.15.77:0/2296585044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974101de0 0x7f0974198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 >> 172.21.15.77:0/2296585044 conn(0x7f09740fd880 msgr2=0x7f09740ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 shutdown_connections 2024-09-15T22:16:57.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:57.959+0000 7f097b660640 1 -- 172.21.15.77:0/2296585044 wait complete. 2024-09-15T22:16:58.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.71 MB/sec execute 67 sec latency 9039.912 ms 2024-09-15T22:16:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:16:58 smithi077 ceph-mon[30980]: pgmap v139: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 259 op/s 2024-09-15T22:16:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:16:58 smithi196 ceph-mon[35109]: pgmap v139: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 259 op/s 2024-09-15T22:16:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:16:58 smithi117 ceph-mon[35021]: pgmap v139: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 259 op/s 2024-09-15T22:16:58.795 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:16:59.040 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:59.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.54 MB/sec execute 68 sec latency 10039.937 ms 2024-09-15T22:16:59.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 -- 172.21.15.77:0/1860577352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8106f00 msgr2=0x7fe8f810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 --2- 172.21.15.77:0/1860577352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8106f00 0x7fe8f810ea80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009ca0 tx=0x7fe8f002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 -- 172.21.15.77:0/1860577352 shutdown_connections 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 --2- 172.21.15.77:0/1860577352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8106f00 0x7fe8f810ea80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 --2- 172.21.15.77:0/1860577352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8102780 0x7fe8f81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 --2- 172.21.15.77:0/1860577352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 -- 172.21.15.77:0/1860577352 >> 172.21.15.77:0/1860577352 conn(0x7fe8f80fd890 msgr2=0x7fe8f80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 -- 172.21.15.77:0/1860577352 shutdown_connections 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 -- 172.21.15.77:0/1860577352 wait complete. 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.541+0000 7fe8fd709640 1 Processor -- start 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 -- start start 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 0x7fe8f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8112000 con 0x7fe8f8102780 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8111e80 con 0x7fe8f8101dd0 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8fd709640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8112180 con 0x7fe8f8106f00 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34706/0 (socket says 172.21.15.77:34706) 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 -- 172.21.15.77:0/33352808 learned_addr learned my addr 172.21.15.77:0/33352808 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 0x7fe8f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 -- 172.21.15.77:0/33352808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 msgr2=0x7fe8f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 0x7fe8f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 -- 172.21.15.77:0/33352808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 msgr2=0x7fe8f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f67fc640 1 -- 172.21.15.77:0/33352808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f81a6ac0 con 0x7fe8f8102780 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 0x7fe8f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.542+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b4a0 tx=0x7fe8ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8dffff640 1 -- 172.21.15.77:0/33352808 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec014070 con 0x7fe8f8102780 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f81a6db0 con 0x7fe8f8102780 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8dffff640 1 -- 172.21.15.77:0/33352808 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec010af0 con 0x7fe8f8102780 2024-09-15T22:16:59.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8dffff640 1 -- 172.21.15.77:0/33352808 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017d10 con 0x7fe8f8102780 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.543+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a7270 con 0x7fe8f8102780 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.545+0000 7fe8dffff640 1 -- 172.21.15.77:0/33352808 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe8ec017070 con 0x7fe8f8102780 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.545+0000 7fe8ffae2740 1 Processor -- start 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.545+0000 7fe8ffae2740 1 -- start start 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d11b40 0x55ffd1d10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d105d0 0x55ffd1d10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8dffff640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 0x7fe8e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1c762f0 con 0x55ffd1d1b370 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1c1dcf0 con 0x55ffd1d105d0 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1c722b0 con 0x55ffd1d11b40 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 0x7fe8e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34712/0 (socket says 172.21.15.77:34712) 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 -- 172.21.15.77:0/555680915 learned_addr learned my addr 172.21.15.77:0/555680915 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8dffff640 1 -- 172.21.15.77:0/33352808 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe8ec0cf3d0 con 0x7fe8f8102780 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d105d0 0x55ffd1d10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d11b40 0x55ffd1d10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 -- 172.21.15.77:0/555680915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d11b40 msgr2=0x55ffd1d10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d11b40 0x55ffd1d10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 -- 172.21.15.77:0/555680915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d105d0 msgr2=0x55ffd1d10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d105d0 0x55ffd1d10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 -- 172.21.15.77:0/555680915 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd1c023a0 con 0x55ffd1d1b370 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 0x7fe8e00a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0031f70 tx=0x7fe8f0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9c7215899c65e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.546+0000 7fe8dd7fa640 1 -- 172.21.15.77:0/555680915 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0003970 con 0x55ffd1d1b370 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8dd7fa640 1 -- 172.21.15.77:0/555680915 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f0036040 con 0x55ffd1d1b370 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8dd7fa640 1 -- 172.21.15.77:0/555680915 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004320 con 0x55ffd1d1b370 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 -- 172.21.15.77:0/555680915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 msgr2=0x55ffd1d1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0031f70 tx=0x7fe8f0009360 comp rx=0 tx=0).stop 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 -- 172.21.15.77:0/555680915 shutdown_connections 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d1d760 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d105d0 0x55ffd1d10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/555680915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d11b40 0x55ffd1d10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 -- 172.21.15.77:0/555680915 >> 172.21.15.77:0/555680915 conn(0x55ffd1cd9310 msgr2=0x55ffd1d0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 -- 172.21.15.77:0/555680915 shutdown_connections 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.547+0000 7fe8ffae2740 1 -- 172.21.15.77:0/555680915 wait complete. 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 -- start start 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 Processor -- start 2024-09-15T22:16:59.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d17ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 0x55ffd1d185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1d1de90 con 0x55ffd1d105d0 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1c31060 con 0x55ffd1d11b40 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffd1d1e010 con 0x55ffd1d1b370 2024-09-15T22:16:59.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8ffae2740 10 client.0 _ll_register_callbacks cb 0x55ffd19848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:59.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34716/0 (socket says 172.21.15.77:34716) 2024-09-15T22:16:59.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2466742334 learned_addr learned my addr 172.21.15.77:0/2466742334 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:59.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d17ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 0x55ffd1d185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 msgr2=0x55ffd1d185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 0x55ffd1d185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2466742334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 msgr2=0x55ffd1d17ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d17ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2466742334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd19e73c0 con 0x55ffd1d105d0 2024-09-15T22:16:59.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.548+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 0x55ffd1d185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ced0 tx=0x7fe8e80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64602966392391a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8023060 con 0x55ffd1d105d0 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe8e80117e0 con 0x55ffd1d105d0 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f0036040 con 0x55ffd1d105d0 2024-09-15T22:16:59.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8e802a050 con 0x55ffd1d105d0 2024-09-15T22:16:59.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 10 client.16734 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:59.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffd1d29960 con 0x55ffd1d105d0 2024-09-15T22:16:59.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f0036040 con 0x55ffd1d105d0 2024-09-15T22:16:59.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e801d9f0 con 0x55ffd1d105d0 2024-09-15T22:16:59.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 10 client.16734 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:59.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:59.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 20 client.16734 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:59.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d2ae20 0x55ffd1d2d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 4 client.16734 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:59.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.549+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd1c83860 con 0x55ffd1d2ae20 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.550+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d2ae20 0x55ffd1d2d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.550+0000 7fe8daff5640 10 client.16734 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.550+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d2ae20 0x55ffd1d2d210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f383e5c5e2b88928 server_cookie=6c259e1302c521ab in_seq=0 out_seq=0 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8daff5640 1 -- 172.21.15.77:0/2466742334 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffd1c83860 con 0x55ffd1d2ae20 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8daff5640 10 client.16734 handle_command_reply: tid=0 2024-09-15T22:16:59.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 client.16734 shutdown 2024-09-15T22:16:59.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 msgr2=0x55ffd1d1ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ced0 tx=0x7fe8e80047d0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 shutdown_connections 2024-09-15T22:16:59.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d2ae20 0x55ffd1d2d210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d1b370 0x55ffd1d185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d17ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2466742334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d105d0 0x55ffd1d1ad40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 >> 172.21.15.77:0/2466742334 conn(0x55ffd1cd9310 msgr2=0x55ffd1d12960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8daff5640 0 client.16734 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:59.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 shutdown_connections 2024-09-15T22:16:59.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2466742334 wait complete. 2024-09-15T22:16:59.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.551+0000 7fe8ffae2740 20 client.16734 trim_cache size 0 max 16384 2024-09-15T22:16:59.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 Processor -- start 2024-09-15T22:16:59.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 -- start start 2024-09-15T22:16:59.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 0x55ffd1d27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d32a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b1210 con 0x55ffd1d11b40 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b2d50 con 0x55ffd1d1b370 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b18e0 con 0x55ffd1d178b0 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34732/0 (socket says 172.21.15.77:34732) 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.564+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2790911789 learned_addr learned my addr 172.21.15.77:0/2790911789 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d32a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2790911789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 msgr2=0x55ffd1d32a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 0x55ffd1d27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d32a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2790911789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 msgr2=0x55ffd1d27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 0x55ffd1d27130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 -- 172.21.15.77:0/2790911789 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd1d16fa0 con 0x55ffd1d11b40 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d32a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 0x55ffd1d27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e8013ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70308ec447198da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8dcff9640 1 -- 172.21.15.77:0/2790911789 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8034020 con 0x55ffd1d11b40 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8dcff9640 1 -- 172.21.15.77:0/2790911789 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e801da80 con 0x55ffd1d11b40 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.565+0000 7fe8dcff9640 1 -- 172.21.15.77:0/2790911789 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8011dc0 con 0x55ffd1d11b40 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2790911789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 msgr2=0x55ffd1d0ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000c00 tx=0x7fe8e8013ef0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2790911789 shutdown_connections 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d32a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d1b370 0x55ffd1d27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/2790911789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d11b40 0x55ffd1d0ce10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2790911789 >> 172.21.15.77:0/2790911789 conn(0x55ffd1cd9310 msgr2=0x55ffd1d0c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2790911789 shutdown_connections 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- 172.21.15.77:0/2790911789 wait complete. 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- start start 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 Processor -- start 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d23960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d14d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b2680 con 0x55ffd1d1b370 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b1fb0 con 0x55ffd1d11b40 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81af1f0 con 0x55ffd1d178b0 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8ffae2740 10 client.0 _ll_register_callbacks cb 0x7fe8e800ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:59.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34740/0 (socket says 172.21.15.77:34740) 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.566+0000 7fe8f77fe640 1 -- 172.21.15.77:0/24974513 learned_addr learned my addr 172.21.15.77:0/24974513 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f67fc640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d14d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d23960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 -- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 msgr2=0x55ffd1d14d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d14d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 -- 172.21.15.77:0/24974513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 msgr2=0x55ffd1d23960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d23960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 -- 172.21.15.77:0/24974513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd1d15a90 con 0x55ffd1d1b370 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8f77fe640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0031680 tx=0x7fe8f00316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484d8d4395b5b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8daff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00098f0 con 0x55ffd1d1b370 2024-09-15T22:16:59.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe8f0003970 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e801da80 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8f0057050 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8ffae2740 10 client.16740 fetch_fsmap learned FSMap version 34 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffd1d9e1d0 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.567+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e801da80 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe8f00045a0 con 0x55ffd1d1b370 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 10 client.16740 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 10 client.16740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 20 client.16740 populate_metadata read hostname 'smithi077' 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d9e690 0x55ffd1da0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 4 client.16740 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffd1c7dd70 con 0x55ffd1d9e690 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d9e690 0x55ffd1da0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:59.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8f6ffd640 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d9e690 0x55ffd1da0ab0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da95214f7f2ac02e server_cookie=952f58975a0786dd in_seq=0 out_seq=0 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8daff5640 10 client.16740 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8daff5640 1 -- 172.21.15.77:0/24974513 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffd1c7dd70 con 0x55ffd1d9e690 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.568+0000 7fe8daff5640 10 client.16740 handle_command_reply: tid=0 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 client.16740 shutdown 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 msgr2=0x55ffd1d15260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0031680 tx=0x7fe8f00316b0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 shutdown_connections 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ffd1d9e690 0x55ffd1da0ab0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffd1d1b370 0x55ffd1d15260 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffd1d178b0 0x55ffd1d14d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8daff5640 0 client.16740 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 --2- 172.21.15.77:0/24974513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffd1d11b40 0x55ffd1d23960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 >> 172.21.15.77:0/24974513 conn(0x55ffd1cd9310 msgr2=0x55ffd1d12110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 shutdown_connections 2024-09-15T22:16:59.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 1 -- 172.21.15.77:0/24974513 wait complete. 2024-09-15T22:16:59.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8ffae2740 20 client.16740 trim_cache size 0 max 16384 2024-09-15T22:16:59.584 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 msgr2=0x7fe8e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 0x7fe8e00a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8008040 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 msgr2=0x7fe8f8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.569+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b4a0 tx=0x7fe8ec00b960 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 shutdown_connections 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe8e00a33b0 0x7fe8e00a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8f8106f00 0x7fe8f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8f8102780 0x7fe8f8074a30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 --2- 172.21.15.77:0/33352808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8f8101dd0 0x7fe8f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 >> 172.21.15.77:0/33352808 conn(0x7fe8f80fd890 msgr2=0x7fe8f80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:59.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 shutdown_connections 2024-09-15T22:16:59.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:16:59.570+0000 7fe8fd709640 1 -- 172.21.15.77:0/33352808 wait complete. 2024-09-15T22:17:00.420 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:17:00.442 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:00 smithi077 ceph-mon[30980]: pgmap v140: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-15T22:17:00.463 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:00.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.37 MB/sec execute 69 sec latency 11039.964 ms 2024-09-15T22:17:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:00 smithi196 ceph-mon[35109]: pgmap v140: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-15T22:17:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:00 smithi117 ceph-mon[35021]: pgmap v140: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-09-15T22:17:00.676 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:01.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 -- 172.21.15.77:0/3862209960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 msgr2=0x7fa67c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 --2- 172.21.15.77:0/3862209960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c106e60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009930 tx=0x7fa66c02a420 comp rx=0 tx=0).stop 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 -- 172.21.15.77:0/3862209960 shutdown_connections 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 --2- 172.21.15.77:0/3862209960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c101420 0x7fa67c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 --2- 172.21.15.77:0/3862209960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c100a80 0x7fa67c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 --2- 172.21.15.77:0/3862209960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c106e60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 -- 172.21.15.77:0/3862209960 >> 172.21.15.77:0/3862209960 conn(0x7fa67c0fc7a0 msgr2=0x7fa67c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 -- 172.21.15.77:0/3862209960 shutdown_connections 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.160+0000 7fa68126b640 1 -- 172.21.15.77:0/3862209960 wait complete. 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 Processor -- start 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 -- start start 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 0x7fa67c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 0x7fa67c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c110e80 con 0x7fa67c106a80 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c110d00 con 0x7fa67c101420 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.161+0000 7fa68126b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111000 con 0x7fa67c100a80 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34764/0 (socket says 172.21.15.77:34764) 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 -- 172.21.15.77:0/1930604667 learned_addr learned my addr 172.21.15.77:0/1930604667 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 0x7fa67c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67affd640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 0x7fa67c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 -- 172.21.15.77:0/1930604667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 msgr2=0x7fa67c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 0x7fa67c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 -- 172.21.15.77:0/1930604667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 msgr2=0x7fa67c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 0x7fa67c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 -- 172.21.15.77:0/1930604667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1a4790 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 0x7fa67c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67affd640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 0x7fa67c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa67400cce0 tx=0x7fa67400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa663fff640 1 -- 172.21.15.77:0/1930604667 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67400f450 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa663fff640 1 -- 172.21.15.77:0/1930604667 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa67400fad0 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1a4960 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa663fff640 1 -- 172.21.15.77:0/1930604667 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674018c10 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.162+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1a4e20 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa663fff640 1 -- 172.21.15.77:0/1930604667 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa67400f5f0 con 0x7fa67c106a80 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 Processor -- start 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 -- start start 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe9b40 0x557a4abe8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a4ab4e2f0 con 0x557a4abe85d0 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a4aaf5cf0 con 0x557a4abe9b40 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.165+0000 7fa683644740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a4ab4a2b0 con 0x557a4abf7780 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34770/0 (socket says 172.21.15.77:34770) 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 -- 172.21.15.77:0/4031157223 learned_addr learned my addr 172.21.15.77:0/4031157223 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 -- 172.21.15.77:0/4031157223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 msgr2=0x557a4abf9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 -- 172.21.15.77:0/4031157223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe9b40 msgr2=0x557a4abe8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe9b40 0x557a4abe8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 -- 172.21.15.77:0/4031157223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4aada3a0 con 0x557a4abe85d0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa663fff640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 0x7fa6640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 0x7fa6640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa67affd640 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02a940 tx=0x7fa66c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01f6fa97f3dcbf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa663fff640 1 -- 172.21.15.77:0/1930604667 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa6740d0750 con 0x7fa67c106a80 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa6617fa640 1 -- 172.21.15.77:0/4031157223 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c03f070 con 0x557a4abe85d0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa6617fa640 1 -- 172.21.15.77:0/4031157223 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c0035f0 con 0x557a4abe85d0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.166+0000 7fa6617fa640 1 -- 172.21.15.77:0/4031157223 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c035070 con 0x557a4abe85d0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- 172.21.15.77:0/4031157223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 msgr2=0x557a4abe8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 0x7fa6640a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557a4abe9070 tx=0x7fa67000b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02a940 tx=0x7fa66c0316e0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- 172.21.15.77:0/4031157223 shutdown_connections 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe85d0 0x557a4abe8a10 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 --2- 172.21.15.77:0/4031157223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe9b40 0x557a4abe8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- 172.21.15.77:0/4031157223 >> 172.21.15.77:0/4031157223 conn(0x557a4abb1310 msgr2=0x557a4abe4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- 172.21.15.77:0/4031157223 shutdown_connections 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- 172.21.15.77:0/4031157223 wait complete. 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 -- start start 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.167+0000 7fa683644740 1 Processor -- start 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe85d0 0x557a4abf7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abf7780 0x557a4abf49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a4abfa420 con 0x557a4abe9b40 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a4ab09060 con 0x557a4abf7780 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a4abfa5a0 con 0x557a4abe85d0 2024-09-15T22:17:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa683644740 10 client.0 _ll_register_callbacks cb 0x557a4a85c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34774/0 (socket says 172.21.15.77:34774) 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 -- 172.21.15.77:0/459701644 learned_addr learned my addr 172.21.15.77:0/459701644 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67affd640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe85d0 0x557a4abf7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abf7780 0x557a4abf49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 -- 172.21.15.77:0/459701644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe85d0 msgr2=0x557a4abf7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe85d0 0x557a4abf7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 -- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abf7780 msgr2=0x557a4abf49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abf7780 0x557a4abf49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 -- 172.21.15.77:0/459701644 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4a8bf3c0 con 0x557a4abe9b40 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa670010d70 tx=0x7fa670004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d190c213a769cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.168+0000 7fa65eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670010e60 con 0x557a4abe9b40 2024-09-15T22:17:01.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa670005020 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c0035f0 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa67002d050 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 10 client.16752 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a4ac05e90 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c0035f0 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa67001d420 con 0x557a4abe9b40 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 10 client.16752 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 10 client.16752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 20 client.16752 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac073f0 0x557a4ac09850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 4 client.16752 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4ab5b860 con 0x557a4ac073f0 2024-09-15T22:17:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.169+0000 7fa67affd640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac073f0 0x557a4ac09850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.170+0000 7fa65eff5640 10 client.16752 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.170+0000 7fa67affd640 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac073f0 0x557a4ac09850 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5538644bfda13ffb server_cookie=fb63ccaf3c9acd3b in_seq=0 out_seq=0 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa65eff5640 1 -- 172.21.15.77:0/459701644 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a4ab5b860 con 0x557a4ac073f0 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa65eff5640 10 client.16752 handle_command_reply: tid=0 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 client.16752 shutdown 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 msgr2=0x557a4abf4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa670010d70 tx=0x7fa670004730 comp rx=0 tx=0).stop 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 shutdown_connections 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac073f0 0x557a4ac09850 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abf7780 0x557a4abf49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abf4400 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 --2- 172.21.15.77:0/459701644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe85d0 0x557a4abf7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa65eff5640 0 client.16752 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:01.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 >> 172.21.15.77:0/459701644 conn(0x557a4abb1310 msgr2=0x557a4abea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 shutdown_connections 2024-09-15T22:17:01.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 1 -- 172.21.15.77:0/459701644 wait complete. 2024-09-15T22:17:01.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.171+0000 7fa683644740 20 client.16752 trim_cache size 0 max 16384 2024-09-15T22:17:01.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 Processor -- start 2024-09-15T22:17:01.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 -- start start 2024-09-15T22:17:01.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abe4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 0x557a4ac036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1acb20 con 0x557a4abf7780 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1ae660 con 0x557a4abe85d0 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa683644740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1ad1f0 con 0x557a4abe9b40 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34778/0 (socket says 172.21.15.77:34778) 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 -- 172.21.15.77:0/2108267187 learned_addr learned my addr 172.21.15.77:0/2108267187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67affd640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abe4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 0x557a4ac036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 -- 172.21.15.77:0/2108267187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 msgr2=0x557a4ac036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 0x557a4ac036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 -- 172.21.15.77:0/2108267187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 msgr2=0x557a4abe4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abe4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67b7fe640 1 -- 172.21.15.77:0/2108267187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4abf4d60 con 0x557a4abf7780 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.184+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 0x557a4ac036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa67409c3f0 tx=0x7fa67409c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747a82b35c4d621c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa660ff9640 1 -- 172.21.15.77:0/2108267187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6740b4070 con 0x557a4abf7780 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa660ff9640 1 -- 172.21.15.77:0/2108267187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6740a0aa0 con 0x557a4abf7780 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa660ff9640 1 -- 172.21.15.77:0/2108267187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674098d60 con 0x557a4abf7780 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 -- 172.21.15.77:0/2108267187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 msgr2=0x557a4ac0f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa67409c3f0 tx=0x7fa67409c2f0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 -- 172.21.15.77:0/2108267187 shutdown_connections 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abf7780 0x557a4ac0f120 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abe9b40 0x557a4ac036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 --2- 172.21.15.77:0/2108267187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abe4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 -- 172.21.15.77:0/2108267187 >> 172.21.15.77:0/2108267187 conn(0x557a4abb1310 msgr2=0x557a4abe4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 -- 172.21.15.77:0/2108267187 shutdown_connections 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.185+0000 7fa683644740 1 -- 172.21.15.77:0/2108267187 wait complete. 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 -- start start 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 Processor -- start 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1adf90 con 0x557a4abe9b40 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1ad8c0 con 0x557a4abe85d0 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1aab00 con 0x557a4abf7780 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa683644740 10 client.0 _ll_register_callbacks cb 0x7fa670010a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:01.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34790/0 (socket says 172.21.15.77:34790) 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 -- 172.21.15.77:0/3377389280 learned_addr learned my addr 172.21.15.77:0/3377389280 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67affd640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 -- 172.21.15.77:0/3377389280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 msgr2=0x557a4abf0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 -- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 msgr2=0x557a4abff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67a7fc640 1 -- 172.21.15.77:0/3377389280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4abf0ff0 con 0x557a4abe9b40 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67b7fe640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.186+0000 7fa67affd640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa67a7fc640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa67001d8c0 tx=0x7fa67000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb417331df1313b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67001d620 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa670004040 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6740a0aa0 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa67001dcc0 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 10 client.16758 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a4ac71900 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6740a0aa0 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa67003b470 con 0x557a4abe9b40 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 10 client.16758 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 10 client.16758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 20 client.16758 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac720b0 0x557a4ac744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 4 client.16758 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:01.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.187+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4ab55d70 con 0x557a4ac720b0 2024-09-15T22:17:01.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa67affd640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac720b0 0x557a4ac744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:01.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa65eff5640 10 client.16758 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:01.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa67affd640 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac720b0 0x557a4ac744d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1458030beb005bfd server_cookie=5878b71a5898de82 in_seq=0 out_seq=0 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa65eff5640 1 -- 172.21.15.77:0/3377389280 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4ab55d70 con 0x557a4ac720b0 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa65eff5640 10 client.16758 handle_command_reply: tid=0 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 client.16758 shutdown 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 msgr2=0x557a4abffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa67001d8c0 tx=0x7fa67000dd50 comp rx=0 tx=0).stop 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 shutdown_connections 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557a4ac720b0 0x557a4ac744d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa65eff5640 0 client.16758 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a4abf7780 0x557a4abf0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a4abe9b40 0x557a4abffda0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 --2- 172.21.15.77:0/3377389280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a4abe85d0 0x557a4abff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.188+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 >> 172.21.15.77:0/3377389280 conn(0x557a4abb1310 msgr2=0x557a4abf2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 shutdown_connections 2024-09-15T22:17:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa683644740 1 -- 172.21.15.77:0/3377389280 wait complete. 2024-09-15T22:17:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa683644740 20 client.16758 trim_cache size 0 max 16384 2024-09-15T22:17:01.197 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 msgr2=0x7fa6640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 0x7fa6640a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557a4abe9070 tx=0x7fa67000b390 comp rx=0 tx=0).stop 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 msgr2=0x7fa67c1a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa67400cce0 tx=0x7fa67400a620 comp rx=0 tx=0).stop 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 shutdown_connections 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6640a33b0 0x7fa6640a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa67c106a80 0x7fa67c1a4250 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c101420 0x7fa67c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 --2- 172.21.15.77:0/1930604667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67c100a80 0x7fa67c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:01.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 >> 172.21.15.77:0/1930604667 conn(0x7fa67c0fc7a0 msgr2=0x7fa67c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:01.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 shutdown_connections 2024-09-15T22:17:01.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:01.189+0000 7fa68126b640 1 -- 172.21.15.77:0/1930604667 wait complete. 2024-09-15T22:17:01.199 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:01.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.21 MB/sec execute 70 sec latency 12039.990 ms 2024-09-15T22:17:02.425 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:17:02.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 11.05 MB/sec execute 71 sec latency 13040.056 ms 2024-09-15T22:17:02.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:02 smithi077 ceph-mon[30980]: pgmap v141: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2024-09-15T22:17:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:02 smithi196 ceph-mon[35109]: pgmap v141: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2024-09-15T22:17:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:02 smithi117 ceph-mon[35021]: pgmap v141: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 191 op/s 2024-09-15T22:17:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 -- 172.21.15.77:0/4245539183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c106f10 msgr2=0x7f786c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 --2- 172.21.15.77:0/4245539183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c106f10 0x7f786c10ea90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7864009ca0 tx=0x7f786402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 -- 172.21.15.77:0/4245539183 shutdown_connections 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 --2- 172.21.15.77:0/4245539183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c106f10 0x7f786c10ea90 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 --2- 172.21.15.77:0/4245539183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c102790 0x7f786c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 --2- 172.21.15.77:0/4245539183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c101de0 0x7f786c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 -- 172.21.15.77:0/4245539183 >> 172.21.15.77:0/4245539183 conn(0x7f786c0fd8c0 msgr2=0x7f786c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 -- 172.21.15.77:0/4245539183 shutdown_connections 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 -- 172.21.15.77:0/4245539183 wait complete. 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.938+0000 7f7871e25640 1 Processor -- start 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 -- start start 2024-09-15T22:17:02.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c101de0 0x7f786c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c106f10 0x7f786c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f786c111f70 con 0x7f786c102790 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f786c111df0 con 0x7f786c101de0 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f7871e25640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f786c1120f0 con 0x7f786c106f10 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34804/0 (socket says 172.21.15.77:34804) 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 -- 172.21.15.77:0/1266779232 learned_addr learned my addr 172.21.15.77:0/1266779232 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786bfff640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c106f10 0x7f786c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786b7fe640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c101de0 0x7f786c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:02.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 -- 172.21.15.77:0/1266779232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c106f10 msgr2=0x7f786c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c106f10 0x7f786c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 -- 172.21.15.77:0/1266779232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c101de0 msgr2=0x7f786c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c101de0 0x7f786c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 -- 172.21.15.77:0/1266779232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c19d3a0 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.939+0000 7f786affd640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f786000c890 tx=0x7f786000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.940+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860004220 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.940+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786c19d690 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.940+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78600043c0 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.940+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860004a00 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.940+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786c1a4f70 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.941+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786c1021c0 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.942+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f78600079b0 con 0x7f786c102790 2024-09-15T22:17:02.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.942+0000 7f7868ff9640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 0x7f78540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:02.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.942+0000 7f786b7fe640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 0x7f78540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:02.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.943+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f78600cfc40 con 0x7f786c102790 2024-09-15T22:17:02.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.943+0000 7f786b7fe640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 0x7f78540a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f785c005fd0 tx=0x7f785c00b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:02.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:02.947+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7860014030 con 0x7f786c102790 2024-09-15T22:17:02.992 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:03.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.162+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786c02d7c0 con 0x7f786c102790 2024-09-15T22:17:03.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.163+0000 7f7868ff9640 1 -- 172.21.15.77:0/1266779232 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f786009bc90 con 0x7f786c102790 2024-09-15T22:17:03.165 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:17:03.165 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:17:03.165 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 msgr2=0x7f78540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 0x7f78540a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f785c005fd0 tx=0x7f785c00b390 comp rx=0 tx=0).stop 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 msgr2=0x7f786c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f786000c890 tx=0x7f786000cd50 comp rx=0 tx=0).stop 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 shutdown_connections 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f78540a33b0 0x7f78540a5870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f786c106f10 0x7f786c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:03.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f786c102790 0x7f786c198930 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:03.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 --2- 172.21.15.77:0/1266779232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f786c101de0 0x7f786c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:03.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 >> 172.21.15.77:0/1266779232 conn(0x7f786c0fd8c0 msgr2=0x7f786c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:03.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 shutdown_connections 2024-09-15T22:17:03.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:03.165+0000 7f7871e25640 1 -- 172.21.15.77:0/1266779232 wait complete. 2024-09-15T22:17:03.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 10.90 MB/sec execute 72 sec latency 14040.092 ms 2024-09-15T22:17:03.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1266779232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1266779232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1266779232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:04.006 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id e0d6ad4a --release --await' 2024-09-15T22:17:04.007 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id e0d6ad4a --release --await 2024-09-15T22:17:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.205+0000 7f20c2699640 1 -- 172.21.15.77:0/2203048343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc101410 msgr2=0x7f20bc10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.205+0000 7f20c2699640 1 --2- 172.21.15.77:0/2203048343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc101410 0x7f20bc10d940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009cd0 tx=0x7f20b402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 -- 172.21.15.77:0/2203048343 shutdown_connections 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 --2- 172.21.15.77:0/2203048343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc101410 0x7f20bc10d940 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 --2- 172.21.15.77:0/2203048343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 0x7f20bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 --2- 172.21.15.77:0/2203048343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc106a70 0x7f20bc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 -- 172.21.15.77:0/2203048343 >> 172.21.15.77:0/2203048343 conn(0x7f20bc0fc790 msgr2=0x7f20bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 -- 172.21.15.77:0/2203048343 shutdown_connections 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 -- 172.21.15.77:0/2203048343 wait complete. 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 Processor -- start 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.206+0000 7f20c2699640 1 -- start start 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 0x7f20bc0fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 0x7f20bc0fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc110e70 con 0x7f20bc106a70 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc110cf0 con 0x7f20bc101410 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c2699640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc110ff0 con 0x7f20bc100a70 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34812/0 (socket says 172.21.15.77:34812) 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 -- 172.21.15.77:0/3628537075 learned_addr learned my addr 172.21.15.77:0/3628537075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 0x7f20bc0fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 0x7f20bc0fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 -- 172.21.15.77:0/3628537075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 msgr2=0x7f20bc0fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 0x7f20bc0fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 -- 172.21.15.77:0/3628537075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 msgr2=0x7f20bc0fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 0x7f20bc0fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20c0c36640 1 -- 172.21.15.77:0/3628537075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20bc1a68f0 con 0x7f20bc106a70 2024-09-15T22:17:04.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.207+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 0x7f20bc0fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20c0c36640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005cc0 tx=0x7f20b4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20b97fa640 1 -- 172.21.15.77:0/3628537075 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4004040 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20b97fa640 1 -- 172.21.15.77:0/3628537075 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b40046c0 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20bc1a6b80 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20b97fa640 1 -- 172.21.15.77:0/3628537075 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4046480 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.208+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20bc1a7110 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.210+0000 7f20b97fa640 1 -- 172.21.15.77:0/3628537075 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f20b40305a0 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20b97fa640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 0x7f20a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 0x7f20a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20b97fa640 1 -- 172.21.15.77:0/3628537075 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f20b402e030 con 0x7f20bc106a70 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20c4a72740 1 Processor -- start 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 0x7f20a40a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005fd0 tx=0x7f20ac005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.211+0000 7f20c4a72740 1 -- start start 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3b5d0 0x55c815f3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c815ea12f0 con 0x55c815f46370 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c815e48cf0 con 0x55c815f3cb40 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c4a72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c815e9d2b0 con 0x55c815f3b5d0 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34828/0 (socket says 172.21.15.77:34828) 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 -- 172.21.15.77:0/2342207080 learned_addr learned my addr 172.21.15.77:0/2342207080 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20bbfff640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3b5d0 0x55c815f3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 -- 172.21.15.77:0/2342207080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3b5d0 msgr2=0x55c815f3ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3b5d0 0x55c815f3ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 -- 172.21.15.77:0/2342207080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 msgr2=0x55c815f3b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 -- 172.21.15.77:0/2342207080 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c815e2d3a0 con 0x55c815f46370 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20bbfff640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20c0c36640 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f20b4032f10 tx=0x7f20b4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402025bb7c9ae16d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.212+0000 7f20a27fc640 1 -- 172.21.15.77:0/2342207080 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40d5070 con 0x55c815f46370 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.213+0000 7f20a27fc640 1 -- 172.21.15.77:0/2342207080 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4036020 con 0x55c815f46370 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.213+0000 7f20a27fc640 1 -- 172.21.15.77:0/2342207080 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40eec90 con 0x55c815f46370 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.213+0000 7f20c4a72740 1 -- 172.21.15.77:0/2342207080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 msgr2=0x55c815f48760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.213+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f20b4032f10 tx=0x7f20b4003f70 comp rx=0 tx=0).stop 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- 172.21.15.77:0/2342207080 shutdown_connections 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f48760 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3b5d0 0x55c815f3ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2342207080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- 172.21.15.77:0/2342207080 >> 172.21.15.77:0/2342207080 conn(0x55c815f04310 msgr2=0x55c815f37c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- 172.21.15.77:0/2342207080 shutdown_connections 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- 172.21.15.77:0/2342207080 wait complete. 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- start start 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 Processor -- start 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f45d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3cb40 0x55c815f43010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c815f48e30 con 0x55c815f46370 2024-09-15T22:17:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c815e5c060 con 0x55c815f3b5d0 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.214+0000 7f20c4a72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c815f48fb0 con 0x55c815f3cb40 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c4a72740 10 client.0 _ll_register_callbacks cb 0x55c815baf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f45d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34844/0 (socket says 172.21.15.77:34844) 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 -- 172.21.15.77:0/1712094298 learned_addr learned my addr 172.21.15.77:0/1712094298 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3cb40 0x55c815f43010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 -- 172.21.15.77:0/1712094298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3cb40 msgr2=0x55c815f43010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3cb40 0x55c815f43010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 -- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 msgr2=0x55c815f45d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f45d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 -- 172.21.15.77:0/1712094298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c815c123c0 con 0x55c815f46370 2024-09-15T22:17:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c0c36640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006010 tx=0x7f20b4030910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b8e50164a45c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f209b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40d5040 con 0x55c815f46370 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f20b40f1240 con 0x55c815f46370 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b4036020 con 0x55c815f46370 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20b40bd6f0 con 0x55c815f46370 2024-09-15T22:17:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.215+0000 7f20c4a72740 10 client.16776 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c815f48e30 con 0x55c815f46370 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b4036020 con 0x55c815f46370 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20b40dd040 con 0x55c815f46370 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 10 client.16776 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 20 client.16776 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815f57ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 4 client.16776 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c815eae860 con 0x55c815f55af0 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.216+0000 7f20bbfff640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815f57ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.217+0000 7f20bbfff640 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815f57ee0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3340377ead22db2 server_cookie=d963c15562c812b1 in_seq=0 out_seq=0 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.217+0000 7f209b7fe640 10 client.16776 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.217+0000 7f209b7fe640 1 -- 172.21.15.77:0/1712094298 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c815eae860 con 0x55c815f55af0 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.217+0000 7f209b7fe640 10 client.16776 handle_command_reply: tid=0 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.217+0000 7f20c4a72740 1 client.16776 shutdown 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 msgr2=0x55c815f43600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006010 tx=0x7f20b4030910 comp rx=0 tx=0).stop 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 shutdown_connections 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815f57ee0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f209b7fe640 0 client.16776 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f46370 0x55c815f43600 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f3cb40 0x55c815f43010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 --2- 172.21.15.77:0/1712094298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f45d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 >> 172.21.15.77:0/1712094298 conn(0x55c815f04310 msgr2=0x55c815f3d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 shutdown_connections 2024-09-15T22:17:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 1 -- 172.21.15.77:0/1712094298 wait complete. 2024-09-15T22:17:04.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.218+0000 7f20c4a72740 20 client.16776 trim_cache size 0 max 16384 2024-09-15T22:17:04.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 Processor -- start 2024-09-15T22:17:04.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 -- start start 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1b0f20 con 0x55c815f3cb40 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1b2a60 con 0x55c815f3b5d0 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c4a72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1b15f0 con 0x55c815f46370 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20bb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34854/0 (socket says 172.21.15.77:34854) 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20bb7fe640 1 -- 172.21.15.77:0/2437259301 learned_addr learned my addr 172.21.15.77:0/2437259301 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20c0c36640 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.234+0000 7f20bbfff640 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 -- 172.21.15.77:0/2437259301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 msgr2=0x55c815f5dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f5dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 -- 172.21.15.77:0/2437259301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 msgr2=0x55c815f52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f52090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 -- 172.21.15.77:0/2437259301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c815f58250 con 0x55c815f3cb40 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0fe810 tx=0x7f20b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9fdd107256c0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20a1ffb640 1 -- 172.21.15.77:0/2437259301 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0004050 con 0x55c815f3cb40 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20a1ffb640 1 -- 172.21.15.77:0/2437259301 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b00041f0 con 0x55c815f3cb40 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.235+0000 7f20a1ffb640 1 -- 172.21.15.77:0/2437259301 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b001b780 con 0x55c815f3cb40 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- 172.21.15.77:0/2437259301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 msgr2=0x55c815f5d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0fe810 tx=0x7f20b0011810 comp rx=0 tx=0).stop 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- 172.21.15.77:0/2437259301 shutdown_connections 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f5dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3cb40 0x55c815f5d7d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 --2- 172.21.15.77:0/2437259301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3b5d0 0x55c815f52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- 172.21.15.77:0/2437259301 >> 172.21.15.77:0/2437259301 conn(0x55c815f04310 msgr2=0x55c815f45090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- 172.21.15.77:0/2437259301 shutdown_connections 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- 172.21.15.77:0/2437259301 wait complete. 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.236+0000 7f20c4a72740 1 -- start start 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 Processor -- start 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f40600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1b2390 con 0x55c815f3b5d0 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1b1cc0 con 0x55c815f3cb40 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20bc1aef00 con 0x55c815f46370 2024-09-15T22:17:04.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20c4a72740 10 client.0 _ll_register_callbacks cb 0x7f20b40b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34868/0 (socket says 172.21.15.77:34868) 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 -- 172.21.15.77:0/3844951348 learned_addr learned my addr 172.21.15.77:0/3844951348 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 -- 172.21.15.77:0/3844951348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 msgr2=0x55c815f40600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f40600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 -- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 msgr2=0x55c815f3fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.237+0000 7f20bbfff640 1 -- 172.21.15.77:0/3844951348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c815fbf7a0 con 0x55c815f3b5d0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f20bbfff640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x55c815f58770 tx=0x7f20ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef9bafefee35bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f209b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:04.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac01f7b0 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f20ac01f950 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b00041f0 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20ac01fb50 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f20c4a72740 10 client.16782 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.238+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c815fc09a0 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b00041f0 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20ac028290 con 0x55c815f3b5d0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 10 client.16782 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 20 client.16782 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815fc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 4 client.16782 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c815ea8d70 con 0x55c815f55af0 2024-09-15T22:17:04.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.239+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815fc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f209b7fe640 10 client.16782 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f20bb7fe640 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815fc3170 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8da02d6fe7ba1f3 server_cookie=d76bf6e2048e70c9 in_seq=0 out_seq=0 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f209b7fe640 1 -- 172.21.15.77:0/3844951348 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c815ea8d70 con 0x55c815f55af0 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f209b7fe640 10 client.16782 handle_command_reply: tid=0 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f20c4a72740 1 client.16782 shutdown 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 msgr2=0x55c815f4eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.240+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x55c815f58770 tx=0x7f20ac004730 comp rx=0 tx=0).stop 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 shutdown_connections 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c815f55af0 0x55c815fc3170 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c815f46370 0x55c815f40600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c815f3cb40 0x55c815f3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 --2- 172.21.15.77:0/3844951348 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c815f3b5d0 0x55c815f4eae0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 >> 172.21.15.77:0/3844951348 conn(0x55c815f04310 msgr2=0x55c815f364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f209b7fe640 0 client.16782 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:04.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 shutdown_connections 2024-09-15T22:17:04.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 1 -- 172.21.15.77:0/3844951348 wait complete. 2024-09-15T22:17:04.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.241+0000 7f20c4a72740 20 client.16782 trim_cache size 0 max 16384 2024-09-15T22:17:04.253 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 msgr2=0x7f20a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 0x7f20a40a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005fd0 tx=0x7f20ac005850 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 msgr2=0x7f20bc0fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005cc0 tx=0x7f20b4003970 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 shutdown_connections 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f20a40a3430 0x7f20a40a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20bc106a70 0x7f20bc0fd800 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20bc101410 0x7f20bc0fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 --2- 172.21.15.77:0/3628537075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20bc100a70 0x7f20bc0fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 >> 172.21.15.77:0/3628537075 conn(0x7f20bc0fc790 msgr2=0x7f20bc06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 shutdown_connections 2024-09-15T22:17:04.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:04.242+0000 7f20c2699640 1 -- 172.21.15.77:0/3628537075 wait complete. 2024-09-15T22:17:04.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 10.75 MB/sec execute 73 sec latency 15040.163 ms 2024-09-15T22:17:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:04 smithi077 ceph-mon[30980]: pgmap v142: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:04 smithi196 ceph-mon[35109]: pgmap v142: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:04 smithi117 ceph-mon[35021]: pgmap v142: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:05.167 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:05.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359216 10.61 MB/sec execute 74 sec latency 16040.202 ms 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 -- 172.21.15.77:0/4189356080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074eb0 msgr2=0x7f31d4112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 --2- 172.21.15.77:0/4189356080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074eb0 0x7f31d4112650 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009900 tx=0x7f31c402a480 comp rx=0 tx=0).stop 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 -- 172.21.15.77:0/4189356080 shutdown_connections 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 --2- 172.21.15.77:0/4189356080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 0x7f31d4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 --2- 172.21.15.77:0/4189356080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074eb0 0x7f31d4112650 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 --2- 172.21.15.77:0/4189356080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31d4074590 0x7f31d4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.661+0000 7f31dc04c640 1 -- 172.21.15.77:0/4189356080 >> 172.21.15.77:0/4189356080 conn(0x7f31d40fd8c0 msgr2=0x7f31d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- 172.21.15.77:0/4189356080 shutdown_connections 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- 172.21.15.77:0/4189356080 wait complete. 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 Processor -- start 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- start start 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31d4074eb0 0x7f31d41a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 0x7f31d419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41185c0 con 0x7f31d4074590 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31d4118440 con 0x7f31d4112b90 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31dc04c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31d4118740 con 0x7f31d4074eb0 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34890/0 (socket says 172.21.15.77:34890) 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.662+0000 7f31d9de8640 1 -- 172.21.15.77:0/3324527876 learned_addr learned my addr 172.21.15.77:0/3324527876 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31da5e9640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 0x7f31d419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 -- 172.21.15.77:0/3324527876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31d4074eb0 msgr2=0x7f31d41a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31d4074eb0 0x7f31d41a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 -- 172.21.15.77:0/3324527876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 msgr2=0x7f31d419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 0x7f31d419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 -- 172.21.15.77:0/3324527876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d419f080 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b4a0 tx=0x7f31d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31caffd640 1 -- 172.21.15.77:0/3324527876 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0014070 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d419f370 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31caffd640 1 -- 172.21.15.77:0/3324527876 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d0010af0 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31caffd640 1 -- 172.21.15.77:0/3324527876 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0017d10 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.663+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d41ab660 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.665+0000 7f31caffd640 1 -- 172.21.15.77:0/3324527876 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f31d0017070 con 0x7f31d4074590 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 Processor -- start 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 -- start start 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31caffd640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 0x7f31c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589060 0x555b9a589440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a596630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b9a4ee620 con 0x555b9a589980 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b9a495e50 con 0x555b9a589060 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31de425740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b9a4ea5e0 con 0x555b9a594240 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31da5e9640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 0x7f31c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31d95e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31caffd640 1 -- 172.21.15.77:0/3324527876 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f31d00cf430 con 0x7f31d4074590 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31d95e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34896/0 (socket says 172.21.15.77:34896) 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31d95e7640 1 -- 172.21.15.77:0/358452001 learned_addr learned my addr 172.21.15.77:0/358452001 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31d9de8640 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589060 0x555b9a589440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.666+0000 7f31da5e9640 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a596630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 -- 172.21.15.77:0/358452001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 msgr2=0x555b9a596630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a596630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 -- 172.21.15.77:0/358452001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589060 msgr2=0x555b9a589440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589060 0x555b9a589440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 -- 172.21.15.77:0/358452001 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b9a486d40 con 0x555b9a589980 2024-09-15T22:17:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31da5e9640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 0x7f31c00a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31d4076600 tx=0x7f31cc014ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31d95e7640 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f31c40057c0 tx=0x7f31c4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7973e3f169284a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31bffff640 1 -- 172.21.15.77:0/358452001 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c403e070 con 0x555b9a589980 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31bffff640 1 -- 172.21.15.77:0/358452001 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4031870 con 0x555b9a589980 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.667+0000 7f31bffff640 1 -- 172.21.15.77:0/358452001 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4035040 con 0x555b9a589980 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- 172.21.15.77:0/358452001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 msgr2=0x555b9a593d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f31c40057c0 tx=0x7f31c4005d50 comp rx=0 tx=0).stop 2024-09-15T22:17:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- 172.21.15.77:0/358452001 shutdown_connections 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a596630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a593d00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 --2- 172.21.15.77:0/358452001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589060 0x555b9a589440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- 172.21.15.77:0/358452001 >> 172.21.15.77:0/358452001 conn(0x555b9a5855f0 msgr2=0x555b9a5859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- 172.21.15.77:0/358452001 shutdown_connections 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- 172.21.15.77:0/358452001 wait complete. 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 -- start start 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.668+0000 7f31de425740 1 Processor -- start 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a591610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a591b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b9a596e10 con 0x555b9a589060 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b9a476f10 con 0x555b9a589980 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b9a596f90 con 0x555b9a594240 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31de425740 10 client.0 _ll_register_callbacks cb 0x555b9a49adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34898/0 (socket says 172.21.15.77:34898) 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 -- 172.21.15.77:0/2568065603 learned_addr learned my addr 172.21.15.77:0/2568065603 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a591610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31da5e9640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a591b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 -- 172.21.15.77:0/2568065603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 msgr2=0x555b9a591b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a591b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 -- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 msgr2=0x555b9a591610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a591610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 -- 172.21.15.77:0/2568065603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b9a1fd580 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31da5e9640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a591b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a591610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.669+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f31d00978e0 tx=0x7f31d0099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9534f2c93d7a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d001e040 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f31d0010ca0 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4031870 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31d009b480 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 10 client.16794 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b9a2c7960 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4031870 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31d00ccaf0 con 0x555b9a589060 2024-09-15T22:17:05.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 10 client.16794 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 10 client.16794 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 20 client.16794 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a5a3c30 0x555b9a5a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 4 client.16794 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.670+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b9a4fbb90 con 0x555b9a5a3c30 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.671+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a5a3c30 0x555b9a5a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.671+0000 7f31bd7fa640 10 client.16794 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.671+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a5a3c30 0x555b9a5a6020 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22740f7759ae682 server_cookie=e34e899e65235cf7 in_seq=0 out_seq=0 2024-09-15T22:17:05.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31bd7fa640 1 -- 172.21.15.77:0/2568065603 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b9a4fbb90 con 0x555b9a5a3c30 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31bd7fa640 10 client.16794 handle_command_reply: tid=0 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 client.16794 shutdown 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 msgr2=0x555b9a5910d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f31d00978e0 tx=0x7f31d0099990 comp rx=0 tx=0).stop 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 shutdown_connections 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a5a3c30 0x555b9a5a6020 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a591b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a591610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31bd7fa640 0 client.16794 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 --2- 172.21.15.77:0/2568065603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a5910d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 >> 172.21.15.77:0/2568065603 conn(0x555b9a5855f0 msgr2=0x555b9a58b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 shutdown_connections 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 1 -- 172.21.15.77:0/2568065603 wait complete. 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.672+0000 7f31de425740 20 client.16794 trim_cache size 0 max 16384 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 Processor -- start 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 -- start start 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a584640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a5ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b5530 con 0x555b9a589060 2024-09-15T22:17:05.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b7070 con 0x555b9a589980 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31de425740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b5c00 con 0x555b9a594240 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34900/0 (socket says 172.21.15.77:34900) 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 -- 172.21.15.77:0/2337866 learned_addr learned my addr 172.21.15.77:0/2337866 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 -- 172.21.15.77:0/2337866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 msgr2=0x555b9a5ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a5ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a584640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 -- 172.21.15.77:0/2337866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 msgr2=0x555b9a584640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a584640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 -- 172.21.15.77:0/2337866 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b9a585090 con 0x555b9a589060 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d95e7640 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a584640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31d9de8640 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f31d009b180 tx=0x7f31d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765e0e03585d992c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31bf7fe640 1 -- 172.21.15.77:0/2337866 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d00b9070 con 0x555b9a589060 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31bf7fe640 1 -- 172.21.15.77:0/2337866 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d009f830 con 0x555b9a589060 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.689+0000 7f31bf7fe640 1 -- 172.21.15.77:0/2337866 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d00ccc30 con 0x555b9a589060 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- 172.21.15.77:0/2337866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 msgr2=0x555b9a58bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f31d009b180 tx=0x7f31d0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- 172.21.15.77:0/2337866 shutdown_connections 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a594240 0x555b9a5ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a589980 0x555b9a584640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 --2- 172.21.15.77:0/2337866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589060 0x555b9a58bac0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- 172.21.15.77:0/2337866 >> 172.21.15.77:0/2337866 conn(0x555b9a5855f0 msgr2=0x555b9a583f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- 172.21.15.77:0/2337866 shutdown_connections 2024-09-15T22:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- 172.21.15.77:0/2337866 wait complete. 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 -- start start 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.690+0000 7f31de425740 1 Processor -- start 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a589060 0x555b9a59c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a594240 0x555b9a5991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b69a0 con 0x555b9a589980 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b62d0 con 0x555b9a594240 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31d41b3510 con 0x555b9a589060 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31de425740 10 client.0 _ll_register_callbacks cb 0x7f31d009afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34906/0 (socket says 172.21.15.77:34906) 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 -- 172.21.15.77:0/3035906884 learned_addr learned my addr 172.21.15.77:0/3035906884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a589060 0x555b9a59c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31da5e9640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a594240 0x555b9a5991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 -- 172.21.15.77:0/3035906884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a589060 msgr2=0x555b9a59c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a589060 0x555b9a59c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 -- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a594240 msgr2=0x555b9a5991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a594240 0x555b9a5991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 -- 172.21.15.77:0/3035906884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b9a5997f0 con 0x555b9a589980 2024-09-15T22:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31d95e7640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f31c402a9a0 tx=0x7f31c402ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1717cd95db42693 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.691+0000 7f31bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4035040 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f31c4031880 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31d009f830 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31c405d050 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 10 client.16800 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b9a60f1c0 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d009f830 con 0x555b9a589980 2024-09-15T22:17:05.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31c4003c30 con 0x555b9a589980 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 10 client.16800 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 10 client.16800 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 20 client.16800 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a60f5c0 0x555b9a6119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 4 client.16800 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e0d6ad4a", "release": true, "await": true}] 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e0d6ad4a", "release": true, "await": true}) -- 0x555b9a4f60a0 con 0x555b9a60f5c0 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.692+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a60f5c0 0x555b9a6119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.693+0000 7f31d9de8640 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a60f5c0 0x555b9a6119b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77afa7657b98117 server_cookie=d1ae1a08b3a94786 in_seq=0 out_seq=0 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.693+0000 7f31bd7fa640 10 client.16800 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:05.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31bd7fa640 1 -- 172.21.15.77:0/3035906884 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555b9a4f60a0 con 0x555b9a60f5c0 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31bd7fa640 10 client.16800 handle_command_reply: tid=0 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 client.16800 shutdown 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 16, 2024-09-15T22:17:05.739 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "e0d6ad4a": { 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 16, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 133.8, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:17:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 msgr2=0x555b9a59c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f31c402a9a0 tx=0x7f31c402ab80 comp rx=0 tx=0).stop 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 shutdown_connections 2024-09-15T22:17:05.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b9a60f5c0 0x555b9a6119b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9a594240 0x555b9a5991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9a589980 0x555b9a59c900 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 --2- 172.21.15.77:0/3035906884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9a589060 0x555b9a59c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 >> 172.21.15.77:0/3035906884 conn(0x555b9a5855f0 msgr2=0x555b9a584750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31bd7fa640 0 client.16800 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 shutdown_connections 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 1 -- 172.21.15.77:0/3035906884 wait complete. 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.708+0000 7f31de425740 20 client.16800 trim_cache size 0 max 16384 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 msgr2=0x7f31c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 0x7f31c00a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31d4076600 tx=0x7f31cc014ab0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 msgr2=0x7f31d41a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b4a0 tx=0x7f31d000b960 comp rx=0 tx=0).stop 2024-09-15T22:17:05.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 shutdown_connections 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31c00a33b0 0x7f31c00a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4112b90 0x7f31d419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31d4074eb0 0x7f31d41a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 --2- 172.21.15.77:0/3324527876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31d4074590 0x7f31d41a4790 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 >> 172.21.15.77:0/3324527876 conn(0x7f31d40fd8c0 msgr2=0x7f31d40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 shutdown_connections 2024-09-15T22:17:05.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:05.709+0000 7f31dc04c640 1 -- 172.21.15.77:0/3324527876 wait complete. 2024-09-15T22:17:06.116 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:17:06.116 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:17:06.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 359433 10.48 MB/sec execute 75 sec latency 16280.762 ms 2024-09-15T22:17:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:06 smithi077 ceph-mon[30980]: pgmap v143: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:06 smithi196 ceph-mon[35109]: pgmap v143: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:06 smithi117 ceph-mon[35021]: pgmap v143: 129 pgs: 129 active+clean; 212 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 1.8 MiB/s wr, 91 op/s 2024-09-15T22:17:06.631 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e0d6ad4a', total seconds elapsed: 19.9 2024-09-15T22:17:06.661 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 -- 172.21.15.77:0/328565424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 msgr2=0x7f28ec106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 --2- 172.21.15.77:0/328565424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec106e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009930 tx=0x7f28dc02a420 comp rx=0 tx=0).stop 2024-09-15T22:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 -- 172.21.15.77:0/328565424 shutdown_connections 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 --2- 172.21.15.77:0/328565424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec101430 0x7f28ec10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 --2- 172.21.15.77:0/328565424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec100a90 0x7f28ec100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 --2- 172.21.15.77:0/328565424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec106e70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 -- 172.21.15.77:0/328565424 >> 172.21.15.77:0/328565424 conn(0x7f28ec0fc7d0 msgr2=0x7f28ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 -- 172.21.15.77:0/328565424 shutdown_connections 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 -- 172.21.15.77:0/328565424 wait complete. 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.173+0000 7f28f1d78640 1 Processor -- start 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 -- start start 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 0x7f28ec197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 0x7f28ec1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec110e90 con 0x7f28ec106a90 2024-09-15T22:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec110d10 con 0x7f28ec101430 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28f1d78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec111010 con 0x7f28ec100a90 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 0x7f28ec197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 0x7f28ec1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 0x7f28ec197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49182/0 (socket says 172.21.15.77:49182) 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28eb7fe640 1 -- 172.21.15.77:0/1138243163 learned_addr learned my addr 172.21.15.77:0/1138243163 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28ebfff640 1 -- 172.21.15.77:0/1138243163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 msgr2=0x7f28ec197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28ebfff640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 0x7f28ec197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28ebfff640 1 -- 172.21.15.77:0/1138243163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 msgr2=0x7f28ec1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.174+0000 7f28ebfff640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 0x7f28ec1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28ebfff640 1 -- 172.21.15.77:0/1138243163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec19c340 con 0x7f28ec106a90 2024-09-15T22:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28eaffd640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 0x7f28ec1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28ebfff640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec19bc60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fd0 tx=0x7f28e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28e8ff9640 1 -- 172.21.15.77:0/1138243163 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400ce20 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28e8ff9640 1 -- 172.21.15.77:0/1138243163 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e4005020 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec19c630 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28e8ff9640 1 -- 172.21.15.77:0/1138243163 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010720 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.175+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a3f10 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.177+0000 7f28e8ff9640 1 -- 172.21.15.77:0/1138243163 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f28e4005540 con 0x7f28ec106a90 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.177+0000 7f28f4151740 1 Processor -- start 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 -- start start 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e83e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e93340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5651e2de9280 con 0x5651e2e90f50 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651e2d90b10 con 0x5651e2e83a80 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28f4151740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651e2de5240 con 0x5651e2e843a0 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28e8ff9640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e83e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e93340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e93340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34928/0 (socket says 172.21.15.77:34928) 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 -- 172.21.15.77:0/2022792094 learned_addr learned my addr 172.21.15.77:0/2022792094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28e8ff9640 1 -- 172.21.15.77:0/1138243163 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f28e40d05f0 con 0x7f28ec106a90 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28ebfff640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 -- 172.21.15.77:0/2022792094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 msgr2=0x5651e2e90a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e90a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 -- 172.21.15.77:0/2022792094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 msgr2=0x5651e2e93340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e93340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.178+0000 7f28eaffd640 1 -- 172.21.15.77:0/2022792094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e2d9d1f0 con 0x5651e2e83a80 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e83e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28e0012b60 tx=0x7f28e000b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d989d54fea0308c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28ebfff640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 0x7f28d40a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098f60 tx=0x7f28e4005f70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28d1ffb640 1 -- 172.21.15.77:0/2022792094 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0012c50 con 0x5651e2e83a80 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28d1ffb640 1 -- 172.21.15.77:0/2022792094 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0012df0 con 0x5651e2e83a80 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28d1ffb640 1 -- 172.21.15.77:0/2022792094 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e00054b0 con 0x5651e2e83a80 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.179+0000 7f28f4151740 1 -- 172.21.15.77:0/2022792094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 msgr2=0x5651e2e83e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e83e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f28e0012b60 tx=0x7f28e000b5b0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- 172.21.15.77:0/2022792094 shutdown_connections 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e93340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e90a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- 172.21.15.77:0/2022792094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e83e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- 172.21.15.77:0/2022792094 >> 172.21.15.77:0/2022792094 conn(0x5651e2e800f0 msgr2=0x5651e2e804d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- 172.21.15.77:0/2022792094 shutdown_connections 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- 172.21.15.77:0/2022792094 wait complete. 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- start start 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 Processor -- start 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e8e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e8e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5651e2e93ac0 con 0x5651e2e90f50 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651e2da3f10 con 0x5651e2e83a80 2024-09-15T22:17:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.180+0000 7f28f4151740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651e2e93c40 con 0x5651e2e843a0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28f4151740 10 client.0 _ll_register_callbacks cb 0x5651e2d6ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e8e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:42716/0 (socket says 172.21.15.77:42716) 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 -- 172.21.15.77:0/2734778728 learned_addr learned my addr 172.21.15.77:0/2734778728 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e8e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 -- 172.21.15.77:0/2734778728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 msgr2=0x5651e2e8e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e8e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 -- 172.21.15.77:0/2734778728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 msgr2=0x5651e2e8e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e8e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 -- 172.21.15.77:0/2734778728 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e2b5a6a0 con 0x5651e2e83a80 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28eaffd640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b620 tx=0x7f28e00128c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb2578da25d6760 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28caffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0029040 con 0x5651e2e83a80 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f28e00046c0 con 0x5651e2e83a80 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.181+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e0012df0 con 0x5651e2e83a80 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28e0035050 con 0x5651e2e83a80 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 10 client.26084 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651e2e9f800 con 0x5651e2e83a80 2024-09-15T22:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0012df0 con 0x5651e2e83a80 2024-09-15T22:17:07.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28e001f630 con 0x5651e2e83a80 2024-09-15T22:17:07.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 10 client.26084 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 20 client.26084 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2ea0b50 0x5651e2ea2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 4 client.26084 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e2df67f0 con 0x5651e2ea0b50 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.182+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2ea0b50 0x5651e2ea2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28caffd640 10 client.26084 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2ea0b50 0x5651e2ea2f40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9cbdfb9a874e42 server_cookie=442b1d0944534c1f in_seq=0 out_seq=0 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28caffd640 1 -- 172.21.15.77:0/2734778728 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651e2df67f0 con 0x5651e2ea0b50 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28caffd640 10 client.26084 handle_command_reply: tid=0 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 client.26084 shutdown 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 msgr2=0x5651e2e8df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b620 tx=0x7f28e00128c0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 shutdown_connections 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2ea0b50 0x5651e2ea2f40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.183+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2e8e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2e8e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28caffd640 0 client.26084 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 1 --2- 172.21.15.77:0/2734778728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8df20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 >> 172.21.15.77:0/2734778728 conn(0x5651e2e800f0 msgr2=0x5651e2e85d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 shutdown_connections 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 1 -- 172.21.15.77:0/2734778728 wait complete. 2024-09-15T22:17:07.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.184+0000 7f28f4151740 20 client.26084 trim_cache size 0 max 16384 2024-09-15T22:17:07.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 Processor -- start 2024-09-15T22:17:07.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 -- start start 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2ea8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1accf0 con 0x5651e2e90f50 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ae830 con 0x5651e2e83a80 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28f4151740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ad3c0 con 0x5651e2e843a0 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2ea8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34952/0 (socket says 172.21.15.77:34952) 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 -- 172.21.15.77:0/4037573884 learned_addr learned my addr 172.21.15.77:0/4037573884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:07.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 -- 172.21.15.77:0/4037573884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 msgr2=0x5651e2ea8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2ea8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 -- 172.21.15.77:0/4037573884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 msgr2=0x5651e2e8d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.204+0000 7f28eaffd640 1 -- 172.21.15.77:0/4037573884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e2e8ee10 con 0x5651e2e90f50 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.205+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2ea8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.205+0000 7f28eaffd640 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b450 tx=0x7f28e000dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aa75905f71b1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.205+0000 7f28d17fa640 1 -- 172.21.15.77:0/4037573884 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0000e60 con 0x5651e2e90f50 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.205+0000 7f28d17fa640 1 -- 172.21.15.77:0/4037573884 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0005850 con 0x5651e2e90f50 2024-09-15T22:17:07.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.205+0000 7f28d17fa640 1 -- 172.21.15.77:0/4037573884 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e00122d0 con 0x5651e2e90f50 2024-09-15T22:17:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- 172.21.15.77:0/4037573884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 msgr2=0x5651e2ea8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b450 tx=0x7f28e000dfd0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- 172.21.15.77:0/4037573884 shutdown_connections 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e90f50 0x5651e2ea8bc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e843a0 0x5651e2ea8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 --2- 172.21.15.77:0/4037573884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e8d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- 172.21.15.77:0/4037573884 >> 172.21.15.77:0/4037573884 conn(0x5651e2e800f0 msgr2=0x5651e2e8e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- 172.21.15.77:0/4037573884 shutdown_connections 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- 172.21.15.77:0/4037573884 wait complete. 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 -- start start 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.206+0000 7f28f4151740 1 Processor -- start 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e98ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ae160 con 0x5651e2e843a0 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ada90 con 0x5651e2e83a80 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1aacd0 con 0x5651e2e90f50 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28f4151740 10 client.0 _ll_register_callbacks cb 0x7f28e0012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49232/0 (socket says 172.21.15.77:49232) 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 -- 172.21.15.77:0/417499110 learned_addr learned my addr 172.21.15.77:0/417499110 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34960/0 (socket says 172.21.15.77:34960) 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e98ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 -- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 msgr2=0x5651e2e98ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e98ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 -- 172.21.15.77:0/417499110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 msgr2=0x5651e2e99410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.207+0000 7f28eaffd640 1 -- 172.21.15.77:0/417499110 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e2e99950 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28eaffd640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28e0015040 tx=0x7f28e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6cdc773648f2eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28caffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e001f670 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e0005850 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f28e0012030 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28e0012270 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28f4151740 10 client.26011 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.208+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651e2f12500 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0005850 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28e0004b40 con 0x5651e2e90f50 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 10 client.26011 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 20 client.26011 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2f12b00 0x5651e2f14ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 4 client.26011 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651e2df0d00 con 0x5651e2f12b00 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2f12b00 0x5651e2f14ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28eb7fe640 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2f12b00 0x5651e2f14ef0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6003dacbbbf51a server_cookie=d66e29c5db3bd1ad in_seq=0 out_seq=0 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.209+0000 7f28caffd640 10 client.26011 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.210+0000 7f28caffd640 1 -- 172.21.15.77:0/417499110 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651e2df0d00 con 0x5651e2f12b00 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.210+0000 7f28caffd640 10 client.26011 handle_command_reply: tid=0 2024-09-15T22:17:07.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.210+0000 7f28f4151740 1 client.26011 shutdown 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.210+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 msgr2=0x5651e2e867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.210+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28e0015040 tx=0x7f28e0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 shutdown_connections 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5651e2f12b00 0x5651e2f14ef0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651e2e90f50 0x5651e2e867a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5651e2e843a0 0x5651e2e99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28caffd640 0 client.26011 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 --2- 172.21.15.77:0/417499110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651e2e83a80 0x5651e2e98ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 >> 172.21.15.77:0/417499110 conn(0x5651e2e800f0 msgr2=0x5651e2e854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 shutdown_connections 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 1 -- 172.21.15.77:0/417499110 wait complete. 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f4151740 20 client.26011 trim_cache size 0 max 16384 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "ab1a4ae4-d495-46c2-ab73-aab2d785453a", 2024-09-15T22:17:07.217 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:17:07.218 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:07.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 msgr2=0x7f28d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 0x7f28d40a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098f60 tx=0x7f28e4005f70 comp rx=0 tx=0).stop 2024-09-15T22:17:07.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 msgr2=0x7f28ec19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.211+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec19bc60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fd0 tx=0x7f28e400ef10 comp rx=0 tx=0).stop 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 shutdown_connections 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28ec106a90 0x7f28ec19bc60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28ec101430 0x7f28ec1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 --2- 172.21.15.77:0/1138243163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28ec100a90 0x7f28ec197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 >> 172.21.15.77:0/1138243163 conn(0x7f28ec0fc7d0 msgr2=0x7f28ec0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 shutdown_connections 2024-09-15T22:17:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:07.212+0000 7f28f1d78640 1 -- 172.21.15.77:0/1138243163 wait complete. 2024-09-15T22:17:07.467 INFO:tasks.workunit.client.0.smithi077.stdout: 1 361881 10.55 MB/sec execute 76 sec latency 65.804 ms 2024-09-15T22:17:07.937 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:17:08.251 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:08.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 364382 10.53 MB/sec execute 77 sec latency 10.773 ms 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: pgmap v144: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.7 MiB/s wr, 125 op/s 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:08 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: pgmap v144: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.7 MiB/s wr, 125 op/s 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:08 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: pgmap v144: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.7 MiB/s wr, 125 op/s 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:08 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 -- 172.21.15.77:0/2258412541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c38102370 msgr2=0x7f7c3810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/2258412541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c38102370 0x7f7c3810e8a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009cd0 tx=0x7f7c3002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 -- 172.21.15.77:0/2258412541 shutdown_connections 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/2258412541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c38102370 0x7f7c3810e8a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/2258412541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c38101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/2258412541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c381079d0 0x7f7c38107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 -- 172.21.15.77:0/2258412541 >> 172.21.15.77:0/2258412541 conn(0x7f7c380fd710 msgr2=0x7f7c380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 -- 172.21.15.77:0/2258412541 shutdown_connections 2024-09-15T22:17:08.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.740+0000 7f7c3df9e640 1 -- 172.21.15.77:0/2258412541 wait complete. 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 Processor -- start 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 -- start start 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 0x7f7c38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 0x7f7c38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111dd0 con 0x7f7c381079d0 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111c50 con 0x7f7c38102370 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.741+0000 7f7c3df9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38111f50 con 0x7f7c381019d0 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c3d79d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 0x7f7c38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49254/0 (socket says 172.21.15.77:49254) 2024-09-15T22:17:08.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/291103160 learned_addr learned my addr 172.21.15.77:0/291103160 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 0x7f7c38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 -- 172.21.15.77:0/291103160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 msgr2=0x7f7c380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 -- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 msgr2=0x7f7c38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 0x7f7c38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 -- 172.21.15.77:0/291103160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c381a6730 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 0x7f7c38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.742+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 0x7f7c38074a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800e9f0 tx=0x7f7c2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.743+0000 7f7c2dffb640 1 -- 172.21.15.77:0/291103160 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800ccc0 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.743+0000 7f7c2dffb640 1 -- 172.21.15.77:0/291103160 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28005020 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.743+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c381a6a20 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.743+0000 7f7c2dffb640 1 -- 172.21.15.77:0/291103160 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280049e0 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.743+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a6f10 con 0x7f7c38102370 2024-09-15T22:17:08.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.745+0000 7f7c2dffb640 1 -- 172.21.15.77:0/291103160 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c28005560 con 0x7f7c38102370 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.745+0000 7f7c415d9740 1 Processor -- start 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.745+0000 7f7c415d9740 1 -- start start 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fa043f2e0 con 0x560fa04d9f90 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fa03e6ce0 con 0x560fa04d9670 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c415d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fa043b2a0 con 0x560fa04e4850 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c2dffb640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 0x7f7c200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:42760/0 (socket says 172.21.15.77:42760) 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34988/0 (socket says 172.21.15.77:34988) 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 -- 172.21.15.77:0/1768017669 learned_addr learned my addr 172.21.15.77:0/1768017669 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c2dffb640 1 -- 172.21.15.77:0/291103160 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7c280cffd0 con 0x7f7c38102370 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 0x7f7c200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 -- 172.21.15.77:0/1768017669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 msgr2=0x560fa04e6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04e6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 -- 172.21.15.77:0/1768017669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 msgr2=0x560fa04e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 -- 172.21.15.77:0/1768017669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa03cb440 con 0x560fa04d9670 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.746+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c30035000 tx=0x7f7c30031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163de7459b3dc30e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 0x7f7c200a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c30047000 tx=0x7f7c30049000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c1effd640 1 -- 172.21.15.77:0/1768017669 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30003e00 con 0x560fa04d9670 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c1effd640 1 -- 172.21.15.77:0/1768017669 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3004e070 con 0x560fa04d9670 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c1effd640 1 -- 172.21.15.77:0/1768017669 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30030da0 con 0x560fa04d9670 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 -- 172.21.15.77:0/1768017669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 msgr2=0x560fa04d9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c30035000 tx=0x7f7c30031a50 comp rx=0 tx=0).stop 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 -- 172.21.15.77:0/1768017669 shutdown_connections 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04e6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9f90 0x560fa04e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 --2- 172.21.15.77:0/1768017669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04d9a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 -- 172.21.15.77:0/1768017669 >> 172.21.15.77:0/1768017669 conn(0x560fa04d5c00 msgr2=0x560fa04d5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 -- 172.21.15.77:0/1768017669 shutdown_connections 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.747+0000 7f7c415d9740 1 -- 172.21.15.77:0/1768017669 wait complete. 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 -- start start 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 Processor -- start 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fa04e7420 con 0x560fa04e4850 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fa03fa050 con 0x560fa04d9670 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fa04e75a0 con 0x560fa04d9f90 2024-09-15T22:17:08.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c415d9740 10 client.0 _ll_register_callbacks cb 0x560fa043a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49274/0 (socket says 172.21.15.77:49274) 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c3d79d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c2ffff640 1 -- 172.21.15.77:0/304984246 learned_addr learned my addr 172.21.15.77:0/304984246 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.748+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 -- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 msgr2=0x560fa04e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 -- 172.21.15.77:0/304984246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 msgr2=0x560fa04e1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04e1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 -- 172.21.15.77:0/304984246 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa01b03c0 con 0x560fa04d9f90 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:08.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c6a0 tx=0x7f7c2806ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=241f728988b42099 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2801c040 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7c280a4950 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3004e070 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c2809c670 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c415d9740 10 client.26023 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fa04f2e90 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.749+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3004e070 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c28004bc0 con 0x560fa04d9f90 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 10 client.26023 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:08.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 20 client.26023 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa04f42a0 0x560fa04f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 4 client.26023 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa044c850 con 0x560fa04f42a0 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa04f42a0 0x560fa04f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa04f42a0 0x560fa04f6690 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d804fd78ff4b77b server_cookie=69992378636c9ee5 in_seq=0 out_seq=0 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.750+0000 7f7c17fff640 10 client.26023 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c17fff640 1 -- 172.21.15.77:0/304984246 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fa044c850 con 0x560fa04f42a0 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c17fff640 10 client.26023 handle_command_reply: tid=0 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 client.26023 shutdown 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 msgr2=0x560fa04e14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c6a0 tx=0x7f7c2806ac60 comp rx=0 tx=0).stop 2024-09-15T22:17:08.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 shutdown_connections 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa04f42a0 0x560fa04f6690 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9f90 0x560fa04e14b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c17fff640 0 client.26023 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 --2- 172.21.15.77:0/304984246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9670 0x560fa04e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.751+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 >> 172.21.15.77:0/304984246 conn(0x560fa04d5c00 msgr2=0x560fa04db930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.752+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 shutdown_connections 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.752+0000 7f7c415d9740 1 -- 172.21.15.77:0/304984246 wait complete. 2024-09-15T22:17:08.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.752+0000 7f7c415d9740 20 client.26023 trim_cache size 0 max 16384 2024-09-15T22:17:08.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 Processor -- start 2024-09-15T22:17:08.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 -- start start 2024-09-15T22:17:08.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b0f00 con 0x560fa04d9670 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b2a40 con 0x560fa04d9f90 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c415d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b15d0 con 0x560fa04e4850 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35008/0 (socket says 172.21.15.77:35008) 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/3370070452 learned_addr learned my addr 172.21.15.77:0/3370070452 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/3370070452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 msgr2=0x560fa04fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/3370070452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 msgr2=0x560fa04fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.771+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.772+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/3370070452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa04e4570 con 0x560fa04d9670 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.772+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425c866d0789ed2d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.772+0000 7f7c1e7fc640 1 -- 172.21.15.77:0/3370070452 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010630 con 0x560fa04d9670 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.772+0000 7f7c1e7fc640 1 -- 172.21.15.77:0/3370070452 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34010cb0 con 0x560fa04d9670 2024-09-15T22:17:08.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.772+0000 7f7c1e7fc640 1 -- 172.21.15.77:0/3370070452 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3401edb0 con 0x560fa04d9670 2024-09-15T22:17:08.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- 172.21.15.77:0/3370070452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 msgr2=0x560fa04f0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c34005f90 comp rx=0 tx=0).stop 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- 172.21.15.77:0/3370070452 shutdown_connections 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04e4850 0x560fa04fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 --2- 172.21.15.77:0/3370070452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04d9670 0x560fa04f0680 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- 172.21.15.77:0/3370070452 >> 172.21.15.77:0/3370070452 conn(0x560fa04d5c00 msgr2=0x560fa04e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- 172.21.15.77:0/3370070452 shutdown_connections 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- 172.21.15.77:0/3370070452 wait complete. 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 -- start start 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.773+0000 7f7c415d9740 1 Processor -- start 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04ddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04de440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b2370 con 0x560fa04e4850 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b1ca0 con 0x560fa04d9f90 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381aeee0 con 0x560fa04d9670 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3cf9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49288/0 (socket says 172.21.15.77:49288) 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3cf9c640 1 -- 172.21.15.77:0/4290927298 learned_addr learned my addr 172.21.15.77:0/4290927298 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04de440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c415d9740 10 client.0 _ll_register_callbacks cb 0x7f7c2800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c2ffff640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04ddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 -- 172.21.15.77:0/4290927298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 msgr2=0x560fa04ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 -- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 msgr2=0x560fa04ddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04ddf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3d79d640 1 -- 172.21.15.77:0/4290927298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa04de980 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.774+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c3d79d640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04de440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7c30006010 tx=0x7f7c300308e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cbe7745c628d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004410 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f7c300045b0 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c34010cb0 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c300047b0 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 10 client.16836 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fa055ec30 con 0x560fa04e4850 2024-09-15T22:17:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34010cb0 con 0x560fa04e4850 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c30030070 con 0x560fa04e4850 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 10 client.16836 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 20 client.16836 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa055ef00 0x560fa05612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 4 client.16836 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.775+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fa0446d60 con 0x560fa055ef00 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa055ef00 0x560fa05612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c3cf9c640 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa055ef00 0x560fa05612f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe557dc2c450fc2 server_cookie=335d1967fcea480c in_seq=0 out_seq=0 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c17fff640 10 client.16836 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c17fff640 1 -- 172.21.15.77:0/4290927298 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560fa0446d60 con 0x560fa055ef00 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c17fff640 10 client.16836 handle_command_reply: tid=0 2024-09-15T22:17:08.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 client.16836 shutdown 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 msgr2=0x560fa04de440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04de440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7c30006010 tx=0x7f7c300308e0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 shutdown_connections 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560fa055ef00 0x560fa05612f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa04e4850 0x560fa04de440 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa04d9f90 0x560fa04ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 --2- 172.21.15.77:0/4290927298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa04d9670 0x560fa04ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 >> 172.21.15.77:0/4290927298 conn(0x560fa04d5c00 msgr2=0x560fa04dc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.776+0000 7f7c17fff640 0 client.16836 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 shutdown_connections 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c415d9740 1 -- 172.21.15.77:0/4290927298 wait complete. 2024-09-15T22:17:08.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c415d9740 20 client.16836 trim_cache size 0 max 16384 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "ab1a4ae4-d495-46c2-ab73-aab2d785453a": { 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "ab1a4ae4-d495-46c2-ab73-aab2d785453a", 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:08.784 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 msgr2=0x7f7c200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 0x7f7c200a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c30047000 tx=0x7f7c30049000 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.777+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 msgr2=0x7f7c38074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 0x7f7c38074a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800e9f0 tx=0x7f7c2800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 shutdown_connections 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c200a33b0 0x7f7c200a5870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c381079d0 0x7f7c38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c38102370 0x7f7c38074a30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 --2- 172.21.15.77:0/291103160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c381019d0 0x7f7c380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 >> 172.21.15.77:0/291103160 conn(0x7f7c380fd710 msgr2=0x7f7c380fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 shutdown_connections 2024-09-15T22:17:08.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:08.778+0000 7f7c3df9e640 1 -- 172.21.15.77:0/291103160 wait complete. 2024-09-15T22:17:09.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 366400 10.57 MB/sec execute 78 sec latency 25.957 ms 2024-09-15T22:17:09.689 INFO:tasks.cephfs.filesystem:scrub status for tag:ab1a4ae4-d495-46c2-ab73-aab2d785453a - {'path': '/', 'tag': 'ab1a4ae4-d495-46c2-ab73-aab2d785453a', 'options': 'recursive,force'} 2024-09-15T22:17:10.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 369318 10.65 MB/sec execute 79 sec latency 12.305 ms 2024-09-15T22:17:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:10 smithi077 ceph-mon[30980]: pgmap v145: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-15T22:17:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:10 smithi196 ceph-mon[35109]: pgmap v145: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-15T22:17:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:10 smithi117 ceph-mon[35021]: pgmap v145: 129 pgs: 129 active+clean; 211 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.8 MiB/s wr, 86 op/s 2024-09-15T22:17:11.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 372233 10.70 MB/sec execute 80 sec latency 15.869 ms 2024-09-15T22:17:12.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.62 MB/sec execute 81 sec latency 271.847 ms 2024-09-15T22:17:12.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:12 smithi077 ceph-mon[30980]: pgmap v146: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-09-15T22:17:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:12 smithi196 ceph-mon[35109]: pgmap v146: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-09-15T22:17:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:12 smithi117 ceph-mon[35021]: pgmap v146: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-09-15T22:17:13.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.49 MB/sec execute 82 sec latency 1271.950 ms 2024-09-15T22:17:14.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.37 MB/sec execute 83 sec latency 2272.066 ms 2024-09-15T22:17:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:14 smithi077 ceph-mon[30980]: pgmap v147: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:14 smithi196 ceph-mon[35109]: pgmap v147: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:14 smithi117 ceph-mon[35021]: pgmap v147: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:15.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.24 MB/sec execute 84 sec latency 3272.152 ms 2024-09-15T22:17:16.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.12 MB/sec execute 85 sec latency 4272.210 ms 2024-09-15T22:17:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:16 smithi077 ceph-mon[30980]: pgmap v148: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:16 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:17:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:16 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:17:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:16 smithi196 ceph-mon[35109]: pgmap v148: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:16 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:17:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:16 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:17:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:16 smithi117 ceph-mon[35021]: pgmap v148: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:17:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:16 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:17:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:16 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:17:17.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 10.01 MB/sec execute 86 sec latency 5272.258 ms 2024-09-15T22:17:18.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.89 MB/sec execute 87 sec latency 6272.374 ms 2024-09-15T22:17:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:18 smithi077 ceph-mon[30980]: pgmap v149: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 221 op/s 2024-09-15T22:17:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:18 smithi196 ceph-mon[35109]: pgmap v149: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 221 op/s 2024-09-15T22:17:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:18 smithi117 ceph-mon[35021]: pgmap v149: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 221 op/s 2024-09-15T22:17:19.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.78 MB/sec execute 88 sec latency 7272.460 ms 2024-09-15T22:17:20.468 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.67 MB/sec execute 89 sec latency 8272.527 ms 2024-09-15T22:17:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:20 smithi077 ceph-mon[30980]: pgmap v150: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-15T22:17:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:20 smithi196 ceph-mon[35109]: pgmap v150: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-15T22:17:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:20 smithi117 ceph-mon[35021]: pgmap v150: 129 pgs: 129 active+clean; 220 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-15T22:17:21.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.56 MB/sec execute 90 sec latency 9272.614 ms 2024-09-15T22:17:22.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.46 MB/sec execute 91 sec latency 10272.682 ms 2024-09-15T22:17:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:22 smithi077 ceph-mon[30980]: pgmap v151: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2024-09-15T22:17:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:22 smithi196 ceph-mon[35109]: pgmap v151: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2024-09-15T22:17:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:22 smithi117 ceph-mon[35021]: pgmap v151: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 209 op/s 2024-09-15T22:17:23.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.35 MB/sec execute 92 sec latency 11272.812 ms 2024-09-15T22:17:24.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.25 MB/sec execute 93 sec latency 12272.879 ms 2024-09-15T22:17:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:24 smithi077 ceph-mon[30980]: pgmap v152: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:24 smithi196 ceph-mon[35109]: pgmap v152: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:24 smithi117 ceph-mon[35021]: pgmap v152: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:25.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.15 MB/sec execute 94 sec latency 13272.990 ms 2024-09-15T22:17:26.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 373024 9.06 MB/sec execute 95 sec latency 14273.084 ms 2024-09-15T22:17:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:26 smithi077 ceph-mon[30980]: pgmap v153: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:26 smithi196 ceph-mon[35109]: pgmap v153: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:26 smithi117 ceph-mon[35021]: pgmap v153: 129 pgs: 129 active+clean; 229 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-15T22:17:27.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 374126 9.02 MB/sec execute 96 sec latency 14749.198 ms 2024-09-15T22:17:28.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 376639 9.07 MB/sec execute 97 sec latency 20.401 ms 2024-09-15T22:17:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:28 smithi077 ceph-mon[30980]: pgmap v154: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 115 op/s 2024-09-15T22:17:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:28 smithi196 ceph-mon[35109]: pgmap v154: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 115 op/s 2024-09-15T22:17:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:28 smithi117 ceph-mon[35021]: pgmap v154: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 115 op/s 2024-09-15T22:17:29.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 379364 9.12 MB/sec execute 98 sec latency 15.552 ms 2024-09-15T22:17:30.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 381172 9.13 MB/sec execute 99 sec latency 14.533 ms 2024-09-15T22:17:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:30 smithi077 ceph-mon[30980]: pgmap v155: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-09-15T22:17:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:30 smithi196 ceph-mon[35109]: pgmap v155: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-09-15T22:17:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:30 smithi117 ceph-mon[35021]: pgmap v155: 129 pgs: 129 active+clean; 224 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.3 MiB/s wr, 49 op/s 2024-09-15T22:17:31.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 383668 9.18 MB/sec execute 100 sec latency 15.713 ms 2024-09-15T22:17:32.261 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:17:32.261 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-15T22:17:32.262 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:32.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 386138 9.21 MB/sec execute 101 sec latency 26.070 ms 2024-09-15T22:17:32.527 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:32 smithi077 ceph-mon[30980]: pgmap v156: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:17:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:32 smithi196 ceph-mon[35109]: pgmap v156: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:17:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:32 smithi117 ceph-mon[35021]: pgmap v156: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:17:33.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 -- 172.21.15.77:0/2651017168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1036e0 msgr2=0x7f601c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2651017168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1036e0 0x7f601c107b90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6008009930 tx=0x7f600802a420 comp rx=0 tx=0).stop 2024-09-15T22:17:33.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 -- 172.21.15.77:0/2651017168 shutdown_connections 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2651017168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1082a0 0x7f601c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2651017168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1036e0 0x7f601c107b90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2651017168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 -- 172.21.15.77:0/2651017168 >> 172.21.15.77:0/2651017168 conn(0x7f601c0fc7d0 msgr2=0x7f601c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 -- 172.21.15.77:0/2651017168 shutdown_connections 2024-09-15T22:17:33.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.033+0000 7f6022e1a640 1 -- 172.21.15.77:0/2651017168 wait complete. 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 Processor -- start 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 -- start start 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1036e0 0x7f601c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1120a0 con 0x7f601c1082a0 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f601c111f20 con 0x7f601c102d30 2024-09-15T22:17:33.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6022e1a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f601c112220 con 0x7f601c1036e0 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f60213b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f6020bb6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f60213b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37958/0 (socket says 172.21.15.77:37958) 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f60213b7640 1 -- 172.21.15.77:0/2371656389 learned_addr learned my addr 172.21.15.77:0/2371656389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.034+0000 7f601bfff640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1036e0 0x7f601c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 -- 172.21.15.77:0/2371656389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1036e0 msgr2=0x7f601c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1036e0 0x7f601c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 -- 172.21.15.77:0/2371656389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 msgr2=0x7f601c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 -- 172.21.15.77:0/2371656389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c1a69e0 con 0x7f601c1082a0 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6020bb6640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f60213b7640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f601400eb10 tx=0x7f601400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601400ce20 con 0x7f601c1082a0 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c1a6cd0 con 0x7f601c1082a0 2024-09-15T22:17:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6014005020 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014010690 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.035+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c1a7230 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.036+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c103110 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.037+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6014005540 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.038+0000 7f6019ffb640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 0x7f600c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.038+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f60140d0690 con 0x7f601c1082a0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.038+0000 7f6020bb6640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 0x7f600c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.038+0000 7f6020bb6640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 0x7f600c0a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f601000afb0 tx=0x7f601000a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:33.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.042+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f601409cd30 con 0x7f601c1082a0 2024-09-15T22:17:33.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.266+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f601c10a2b0 con 0x7f601c1082a0 2024-09-15T22:17:33.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.267+0000 7f6019ffb640 1 -- 172.21.15.77:0/2371656389 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f601409c630 con 0x7f601c1082a0 2024-09-15T22:17:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:17:33.271 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:17:33.271 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:17:33.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 msgr2=0x7f600c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 0x7f600c0a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f601000afb0 tx=0x7f601000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 msgr2=0x7f601c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f601400eb10 tx=0x7f601400ef10 comp rx=0 tx=0).stop 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 shutdown_connections 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f600c0a33b0 0x7f600c0a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f601c1082a0 0x7f601c101410 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f601c1036e0 0x7f601c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 --2- 172.21.15.77:0/2371656389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f601c102d30 0x7f601c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.269+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 >> 172.21.15.77:0/2371656389 conn(0x7f601c0fc7d0 msgr2=0x7f601c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:33.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.270+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 shutdown_connections 2024-09-15T22:17:33.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:33.270+0000 7f6022e1a640 1 -- 172.21.15.77:0/2371656389 wait complete. 2024-09-15T22:17:33.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 387923 9.21 MB/sec execute 102 sec latency 39.682 ms 2024-09-15T22:17:33.561 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2371656389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:33.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:33 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2371656389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:33.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:33 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2371656389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:34.106 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-15T22:17:34.106 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-15T22:17:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:34 smithi077 ceph-mon[30980]: pgmap v157: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:34.375 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:34.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 390526 9.28 MB/sec execute 103 sec latency 18.275 ms 2024-09-15T22:17:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:34 smithi196 ceph-mon[35109]: pgmap v157: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:34 smithi117 ceph-mon[35021]: pgmap v157: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 -- 172.21.15.77:0/134297514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 msgr2=0x7effc810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 --2- 172.21.15.77:0/134297514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc810d940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7effc0009be0 tx=0x7effc002a6c0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 -- 172.21.15.77:0/134297514 shutdown_connections 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 --2- 172.21.15.77:0/134297514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc810d940 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 --2- 172.21.15.77:0/134297514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8100a70 0x7effc8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 --2- 172.21.15.77:0/134297514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8106a70 0x7effc8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 -- 172.21.15.77:0/134297514 >> 172.21.15.77:0/134297514 conn(0x7effc80fc790 msgr2=0x7effc80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 -- 172.21.15.77:0/134297514 shutdown_connections 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.921+0000 7effd0083640 1 -- 172.21.15.77:0/134297514 wait complete. 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 Processor -- start 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 -- start start 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8100a70 0x7effc81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8106a70 0x7effc819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.922+0000 7effd0083640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7effc8110e20 con 0x7effc8106a70 2024-09-15T22:17:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effd0083640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7effc8110ca0 con 0x7effc8101410 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effd0083640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effc8110fa0 con 0x7effc8100a70 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effcd61e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effcd61e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35840/0 (socket says 172.21.15.77:35840) 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effcd61e640 1 -- 172.21.15.77:0/1674841331 learned_addr learned my addr 172.21.15.77:0/1674841331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.923+0000 7effce620640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8106a70 0x7effc819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcde1f640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8100a70 0x7effc81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 -- 172.21.15.77:0/1674841331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8100a70 msgr2=0x7effc81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8100a70 0x7effc81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 -- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8106a70 msgr2=0x7effc819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8106a70 0x7effc819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 -- 172.21.15.77:0/1674841331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc819d370 con 0x7effc8101410 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.924+0000 7effcd61e640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb800e9c0 tx=0x7effb800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.925+0000 7effbeffd640 1 -- 172.21.15.77:0/1674841331 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb800cde0 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.925+0000 7effbeffd640 1 -- 172.21.15.77:0/1674841331 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb80035f0 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.925+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc819d660 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.925+0000 7effbeffd640 1 -- 172.21.15.77:0/1674841331 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8010690 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.925+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81a4f40 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.928+0000 7effbeffd640 1 -- 172.21.15.77:0/1674841331 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7effb80037a0 con 0x7effc8101410 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 Processor -- start 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 -- start start 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 0x55af9de5b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 0x55af9de5e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af9ddb1160 con 0x55af9de50c60 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af9dd58720 con 0x55af9de5be40 2024-09-15T22:17:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.929+0000 7effd245c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af9ddad120 con 0x55af9de51580 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effce620640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effce620640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37988/0 (socket says 172.21.15.77:37988) 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effce620640 1 -- 172.21.15.77:0/178826029 learned_addr learned my addr 172.21.15.77:0/178826029 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effcde1f640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 0x55af9de5b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effcd61e640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 0x55af9de5e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effcd61e640 1 -- 172.21.15.77:0/178826029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 msgr2=0x55af9de5b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effcd61e640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 0x55af9de5b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.930+0000 7effcd61e640 1 -- 172.21.15.77:0/178826029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 msgr2=0x55af9de51040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.931+0000 7effcd61e640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.931+0000 7effcd61e640 1 -- 172.21.15.77:0/178826029 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9db22ba0 con 0x55af9de5be40 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.931+0000 7effcde1f640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 0x55af9de5b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.931+0000 7effce620640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.931+0000 7effcd61e640 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 0x55af9de5e230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effb80cd7f0 tx=0x7effb80d8880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b6acad232e3c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effbeffd640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 0x7effb40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effb3fff640 1 -- 172.21.15.77:0/178826029 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8098620 con 0x55af9de5be40 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effce620640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 0x7effb40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effb3fff640 1 -- 172.21.15.77:0/178826029 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb809ca80 con 0x55af9de5be40 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effb3fff640 1 -- 172.21.15.77:0/178826029 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb809bb00 con 0x55af9de5be40 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.932+0000 7effbeffd640 1 -- 172.21.15.77:0/1674841331 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7effb80d0440 con 0x7effc8101410 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 -- 172.21.15.77:0/178826029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 msgr2=0x55af9de5e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 0x55af9de5e230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effb80cd7f0 tx=0x7effb80d8880 comp rx=0 tx=0).stop 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effce620640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 0x7effb40a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc0005cc0 tx=0x7effc0004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 -- 172.21.15.77:0/178826029 shutdown_connections 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de5be40 0x55af9de5e230 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de51580 0x55af9de5b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 --2- 172.21.15.77:0/178826029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de50c60 0x55af9de51040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.933+0000 7effd245c740 1 -- 172.21.15.77:0/178826029 >> 172.21.15.77:0/178826029 conn(0x55af9de4d1f0 msgr2=0x55af9de4d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.934+0000 7effd245c740 1 -- 172.21.15.77:0/178826029 shutdown_connections 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.934+0000 7effd245c740 1 -- 172.21.15.77:0/178826029 wait complete. 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.934+0000 7effd245c740 1 -- start start 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.934+0000 7effd245c740 1 Processor -- start 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de5be40 0x55af9de559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af9de489b0 con 0x55af9de51580 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af9dd39900 con 0x55af9de50c60 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af9de48b30 con 0x55af9de5be40 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effd245c740 10 client.0 _ll_register_callbacks cb 0x55af9dd37780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37994/0 (socket says 172.21.15.77:37994) 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 -- 172.21.15.77:0/3178413295 learned_addr learned my addr 172.21.15.77:0/3178413295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcde1f640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effce620640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de5be40 0x55af9de559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 -- 172.21.15.77:0/3178413295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de5be40 msgr2=0x55af9de559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de5be40 0x55af9de559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 -- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 msgr2=0x55af9de54f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de54f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcd61e640 1 -- 172.21.15.77:0/3178413295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9dd79c50 con 0x55af9de51580 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.935+0000 7effcde1f640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.936+0000 7effcd61e640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7effb80d81b0 tx=0x7effb8003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751ae1a29b3d20d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.936+0000 7effb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.936+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb809bce0 con 0x55af9de51580 2024-09-15T22:17:34.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.936+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7effb809a420 con 0x55af9de51580 2024-09-15T22:17:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.936+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb809ca80 con 0x55af9de51580 2024-09-15T22:17:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.937+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7effb809bec0 con 0x55af9de51580 2024-09-15T22:17:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.937+0000 7effd245c740 10 client.16854 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:34.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.937+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af9de772a0 con 0x55af9de51580 2024-09-15T22:17:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8098270 con 0x55af9de51580 2024-09-15T22:17:34.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7effb80b51d0 con 0x55af9de51580 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 10 client.16854 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 10 client.16854 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 20 client.16854 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de785f0 0x55af9de7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 4 client.16854 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7effb809ca80 con 0x55af9de785f0 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.938+0000 7effcde1f640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de785f0 0x55af9de7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.939+0000 7effb17fa640 10 client.16854 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.939+0000 7effcde1f640 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de785f0 0x55af9de7a9e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647f2db42128015c server_cookie=7db9d8627176472 in_seq=0 out_seq=0 2024-09-15T22:17:34.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.940+0000 7effb17fa640 1 -- 172.21.15.77:0/3178413295 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7effb809ca80 con 0x55af9de785f0 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.940+0000 7effb17fa640 10 client.16854 handle_command_reply: tid=0 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.940+0000 7effd245c740 1 client.16854 shutdown 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 msgr2=0x55af9de55480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7effb80d81b0 tx=0x7effb8003e80 comp rx=0 tx=0).stop 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 shutdown_connections 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de785f0 0x55af9de7a9e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de5be40 0x55af9de559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effb17fa640 0 client.16854 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de55480 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 --2- 172.21.15.77:0/3178413295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 >> 172.21.15.77:0/3178413295 conn(0x55af9de4d1f0 msgr2=0x55af9de52f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 shutdown_connections 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 1 -- 172.21.15.77:0/3178413295 wait complete. 2024-09-15T22:17:34.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.941+0000 7effd245c740 20 client.16854 trim_cache size 0 max 16384 2024-09-15T22:17:34.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 Processor -- start 2024-09-15T22:17:34.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 -- start start 2024-09-15T22:17:34.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de50c60 0x55af9de4b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 0x55af9de4ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7effc8073870 con 0x55af9de4d1f0 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7effc81a6100 con 0x55af9de51580 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effd245c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effc8073f40 con 0x55af9de50c60 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38002/0 (socket says 172.21.15.77:38002) 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 -- 172.21.15.77:0/3228467561 learned_addr learned my addr 172.21.15.77:0/3228467561 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcd61e640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de50c60 0x55af9de4b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effce620640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 0x55af9de4ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 -- 172.21.15.77:0/3228467561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de50c60 msgr2=0x55af9de4b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de50c60 0x55af9de4b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 -- 172.21.15.77:0/3228467561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 msgr2=0x55af9de4ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 0x55af9de4ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effcde1f640 1 -- 172.21.15.77:0/3228467561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9de4c810 con 0x55af9de4d1f0 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.969+0000 7effce620640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 0x55af9de4ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.970+0000 7effcde1f640 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7effc4005d70 tx=0x7effc4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae91534a53a3f34a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.970+0000 7effb37fe640 1 -- 172.21.15.77:0/3228467561 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc402c070 con 0x55af9de4d1f0 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.970+0000 7effb37fe640 1 -- 172.21.15.77:0/3228467561 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc401fc60 con 0x55af9de4d1f0 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.970+0000 7effb37fe640 1 -- 172.21.15.77:0/3228467561 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4027da0 con 0x55af9de4d1f0 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- 172.21.15.77:0/3228467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 msgr2=0x55af9de55810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7effc4005d70 tx=0x7effc4005f90 comp rx=0 tx=0).stop 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- 172.21.15.77:0/3228467561 shutdown_connections 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de51580 0x55af9de4ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de50c60 0x55af9de4b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 --2- 172.21.15.77:0/3228467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de4d1f0 0x55af9de55810 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- 172.21.15.77:0/3228467561 >> 172.21.15.77:0/3228467561 conn(0x55af9de543f0 msgr2=0x55af9de58f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- 172.21.15.77:0/3228467561 shutdown_connections 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- 172.21.15.77:0/3228467561 wait complete. 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 -- start start 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.971+0000 7effd245c740 1 Processor -- start 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7effc81a5a30 con 0x55af9de51580 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7effc81a5360 con 0x55af9de50c60 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effc8071850 con 0x55af9de4d1f0 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effd245c740 10 client.0 _ll_register_callbacks cb 0x7effc0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcd61e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effce620640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:40530/0 (socket says 172.21.15.77:40530) 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 -- 172.21.15.77:0/3442838919 learned_addr learned my addr 172.21.15.77:0/3442838919 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effce620640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de73190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38010/0 (socket says 172.21.15.77:38010) 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 -- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 msgr2=0x55af9de72c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de72c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 -- 172.21.15.77:0/3442838919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 msgr2=0x55af9de73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de73190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 -- 172.21.15.77:0/3442838919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9dee1f30 con 0x55af9de4d1f0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effcde1f640 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7effc4014010 tx=0x7effc4005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d28a11b702fd2c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.972+0000 7effb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc402c030 con 0x55af9de4d1f0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc401fc60 con 0x55af9de4d1f0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7effc4027460 con 0x55af9de4d1f0 2024-09-15T22:17:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7effc403f050 con 0x55af9de4d1f0 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 10 client.26053 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af9dee3100 con 0x55af9de4d1f0 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc401fc60 con 0x55af9de4d1f0 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7effc4005440 con 0x55af9de4d1f0 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 10 client.26053 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 10 client.26053 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 20 client.26053 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.973+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de5be40 0x55af9dee5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.974+0000 7effd245c740 4 client.26053 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.974+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55af9ddbe890 con 0x55af9de5be40 2024-09-15T22:17:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.974+0000 7effcd61e640 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de5be40 0x55af9dee5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.974+0000 7effcd61e640 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de5be40 0x55af9dee5850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6406e88ff3029879 server_cookie=e00f5ccba0f0ad4a in_seq=0 out_seq=0 2024-09-15T22:17:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:34.974+0000 7effb17fa640 10 client.26053 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:35.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effb17fa640 1 -- 172.21.15.77:0/3442838919 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55af9ddbe890 con 0x55af9de5be40 2024-09-15T22:17:35.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effb17fa640 10 client.26053 handle_command_reply: tid=0 2024-09-15T22:17:35.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 client.26053 shutdown 2024-09-15T22:17:35.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 msgr2=0x55af9de72710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7effc4014010 tx=0x7effc4005b20 comp rx=0 tx=0).stop 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 shutdown_connections 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55af9de5be40 0x55af9dee5850 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af9de51580 0x55af9de73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af9de50c60 0x55af9de72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 --2- 172.21.15.77:0/3442838919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af9de4d1f0 0x55af9de72710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effb17fa640 0 client.26053 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 >> 172.21.15.77:0/3442838919 conn(0x55af9de543f0 msgr2=0x55af9de58760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 shutdown_connections 2024-09-15T22:17:35.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 1 -- 172.21.15.77:0/3442838919 wait complete. 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.038+0000 7effd245c740 20 client.26053 trim_cache size 0 max 16384 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 18, 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "fbaeebd6": { 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 18, 2024-09-15T22:17:35.043 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 145.3, 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:17:35.044 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:35.045 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 msgr2=0x7effb40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 0x7effb40a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc0005cc0 tx=0x7effc0004a40 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 msgr2=0x7effc8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb800e9c0 tx=0x7effb800ee80 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 shutdown_connections 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effb40a3580 0x7effb40a5a40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc8106a70 0x7effc819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc8101410 0x7effc8198900 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 --2- 172.21.15.77:0/1674841331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8100a70 0x7effc81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 >> 172.21.15.77:0/1674841331 conn(0x7effc80fc790 msgr2=0x7effc8104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 shutdown_connections 2024-09-15T22:17:35.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:35.039+0000 7effd0083640 1 -- 172.21.15.77:0/1674841331 wait complete. 2024-09-15T22:17:35.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 9.20 MB/sec execute 104 sec latency 424.888 ms 2024-09-15T22:17:35.971 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fbaeebd6', quiesce took 3.7 seconds. Will release after: 21.650000000000002 2024-09-15T22:17:36.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 9.11 MB/sec execute 105 sec latency 1424.945 ms 2024-09-15T22:17:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:36 smithi077 ceph-mon[30980]: pgmap v158: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:36 smithi196 ceph-mon[35109]: pgmap v158: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:36 smithi117 ceph-mon[35021]: pgmap v158: 129 pgs: 129 active+clean; 225 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-09-15T22:17:37.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 9.03 MB/sec execute 106 sec latency 2425.017 ms 2024-09-15T22:17:38.469 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.94 MB/sec execute 107 sec latency 3425.058 ms 2024-09-15T22:17:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:38 smithi077 ceph-mon[30980]: pgmap v159: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-15T22:17:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:38 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:38 smithi196 ceph-mon[35109]: pgmap v159: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-15T22:17:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:38 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:38 smithi117 ceph-mon[35021]: pgmap v159: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-09-15T22:17:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:38 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:39.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.86 MB/sec execute 108 sec latency 4425.150 ms 2024-09-15T22:17:39.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:39 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:39.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:39 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:39.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:39 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:17:39.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:39 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:39 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:39.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:39 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:17:39.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:39 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:39.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:39 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:39.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:39 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:17:39.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:17:39.947 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:40 smithi077 ceph-mon[30980]: pgmap v160: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 -- 172.21.15.77:0/2960108497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 msgr2=0x7ff8a0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2960108497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a0100ee0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff890009930 tx=0x7ff89002a420 comp rx=0 tx=0).stop 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 -- 172.21.15.77:0/2960108497 shutdown_connections 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2960108497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0101420 0x7ff8a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2960108497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a0100ee0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2960108497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0106a80 0x7ff8a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 -- 172.21.15.77:0/2960108497 >> 172.21.15.77:0/2960108497 conn(0x7ff8a00fc7a0 msgr2=0x7ff8a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 -- 172.21.15.77:0/2960108497 shutdown_connections 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.431+0000 7ff8a4833640 1 -- 172.21.15.77:0/2960108497 wait complete. 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 Processor -- start 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 -- start start 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0101420 0x7ff8a01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0106a80 0x7ff8a019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0112020 con 0x7ff8a0100a80 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0111ea0 con 0x7ff8a0101420 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff8a4833640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01121a0 con 0x7ff8a0106a80 2024-09-15T22:17:40.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47160/0 (socket says 172.21.15.77:47160) 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 -- 172.21.15.77:0/2307060194 learned_addr learned my addr 172.21.15.77:0/2307060194 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0106a80 0x7ff8a019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0101420 0x7ff8a01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 -- 172.21.15.77:0/2307060194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0106a80 msgr2=0x7ff8a019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0106a80 0x7ff8a019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 -- 172.21.15.77:0/2307060194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0101420 msgr2=0x7ff8a01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0101420 0x7ff8a01a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.432+0000 7ff89e59c640 1 -- 172.21.15.77:0/2307060194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a019ad40 con 0x7ff8a0100a80 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff89e59c640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b4a0 tx=0x7ff88800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff8977fe640 1 -- 172.21.15.77:0/2307060194 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888014070 con 0x7ff8a0100a80 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a019b030 con 0x7ff8a0100a80 2024-09-15T22:17:40.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff8977fe640 1 -- 172.21.15.77:0/2307060194 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff888010af0 con 0x7ff8a0100a80 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff8977fe640 1 -- 172.21.15.77:0/2307060194 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017d10 con 0x7ff8a0100a80 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.433+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a01a7320 con 0x7ff8a0100a80 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.435+0000 7ff8977fe640 1 -- 172.21.15.77:0/2307060194 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff888017070 con 0x7ff8a0100a80 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 Processor -- start 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 -- start start 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb58880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8977fe640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 0x7ff88c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb58dc0 0x5588bbb63140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5588bbabe2e0 con 0x5588bbb63680 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588bba65ce0 con 0x5588bbb584a0 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588bbaba2a0 con 0x5588bbb58dc0 2024-09-15T22:17:40.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89e59c640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 0x7ff88c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff8977fe640 1 -- 172.21.15.77:0/2307060194 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff8880cf3d0 con 0x7ff8a0100a80 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47174/0 (socket says 172.21.15.77:47174) 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89ed9d640 1 -- 172.21.15.77:0/3921622346 learned_addr learned my addr 172.21.15.77:0/3921622346 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb58880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.436+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb58dc0 0x5588bbb63140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 -- 172.21.15.77:0/3921622346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb58dc0 msgr2=0x5588bbb63140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb58dc0 0x5588bbb63140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 -- 172.21.15.77:0/3921622346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 msgr2=0x5588bbb58880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89e59c640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 0x7ff88c0a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ccde0 tx=0x7ff888013500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb58880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 -- 172.21.15.77:0/3921622346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bba4a440 con 0x5588bbb63680 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff89800ace0 tx=0x7ff8980097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712370c3d721817 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff894ff9640 1 -- 172.21.15.77:0/3921622346 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898013040 con 0x5588bbb63680 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff894ff9640 1 -- 172.21.15.77:0/3921622346 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89801a430 con 0x5588bbb63680 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.437+0000 7ff894ff9640 1 -- 172.21.15.77:0/3921622346 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898019640 con 0x5588bbb63680 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/3921622346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 msgr2=0x5588bbb65a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff89800ace0 tx=0x7ff8980097c0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/3921622346 shutdown_connections 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb65a70 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb58dc0 0x5588bbb63140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/3921622346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb58880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/3921622346 >> 172.21.15.77:0/3921622346 conn(0x5588bbb54a30 msgr2=0x5588bbb54e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/3921622346 shutdown_connections 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/3921622346 wait complete. 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 -- start start 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.438+0000 7ff8a6c0c740 1 Processor -- start 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb584a0 0x5588bbb63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5588bbb66250 con 0x5588bbb63680 2024-09-15T22:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588bba79050 con 0x5588bbb58dc0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588bbb663d0 con 0x5588bbb584a0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 10 client.0 _ll_register_callbacks cb 0x5588bbab9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47180/0 (socket says 172.21.15.77:47180) 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 -- 172.21.15.77:0/689731118 learned_addr learned my addr 172.21.15.77:0/689731118 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb584a0 0x5588bbb63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 -- 172.21.15.77:0/689731118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb584a0 msgr2=0x5588bbb63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb584a0 0x5588bbb63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 -- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 msgr2=0x5588bbb60300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb60300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 -- 172.21.15.77:0/689731118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bb82f3c0 con 0x5588bbb63680 2024-09-15T22:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff89801c040 tx=0x7ff898014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997dbb2fb52663c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89802b060 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff898012070 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.439+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89801a430 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff898014c90 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 10 client.16872 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5588bbb71c90 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89801a430 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff898024940 con 0x5588bbb63680 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 10 client.16872 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 20 client.16872 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbb753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 4 client.16872 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588bbacb850 con 0x5588bbb72fe0 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.440+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbb753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbb753d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5e65707f34a518 server_cookie=e21623472e774191 in_seq=0 out_seq=0 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff885ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff885ffb640 1 -- 172.21.15.77:0/689731118 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588bbacb850 con 0x5588bbb72fe0 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff885ffb640 10 client.16872 handle_command_reply: tid=0 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 client.16872 shutdown 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 msgr2=0x5588bbb608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff89801c040 tx=0x7ff898014040 comp rx=0 tx=0).stop 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 shutdown_connections 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbb753d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb63680 0x5588bbb608f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb60300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff885ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/689731118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb584a0 0x5588bbb63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 >> 172.21.15.77:0/689731118 conn(0x5588bbb54a30 msgr2=0x5588bbb5a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 shutdown_connections 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.441+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/689731118 wait complete. 2024-09-15T22:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.442+0000 7ff8a6c0c740 20 client.16872 trim_cache size 0 max 16384 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 Processor -- start 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 -- start start 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbb53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b1290 con 0x5588bbb584a0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b2dd0 con 0x5588bbb58dc0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b1960 con 0x5588bbb63680 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47194/0 (socket says 172.21.15.77:47194) 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 -- 172.21.15.77:0/4007857058 learned_addr learned my addr 172.21.15.77:0/4007857058 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbb53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 -- 172.21.15.77:0/4007857058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 msgr2=0x5588bbb53a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbb53a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 -- 172.21.15.77:0/4007857058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 msgr2=0x5588bbb5fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb5fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 -- 172.21.15.77:0/4007857058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588bbb58bf0 con 0x5588bbb584a0 2024-09-15T22:17:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.459+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff898004b90 tx=0x7ff898024f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da093ac83fc30799 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff887fff640 1 -- 172.21.15.77:0/4007857058 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89803c070 con 0x5588bbb584a0 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff887fff640 1 -- 172.21.15.77:0/4007857058 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898009680 con 0x5588bbb584a0 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff887fff640 1 -- 172.21.15.77:0/4007857058 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89802b030 con 0x5588bbb584a0 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/4007857058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 msgr2=0x5588bbb6f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff898004b90 tx=0x7ff898024f50 comp rx=0 tx=0).stop 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/4007857058 shutdown_connections 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbb53a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb58dc0 0x5588bbb5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/4007857058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb584a0 0x5588bbb6f4e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.460+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/4007857058 >> 172.21.15.77:0/4007857058 conn(0x5588bbb54a30 msgr2=0x5588bbb53350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/4007857058 shutdown_connections 2024-09-15T22:17:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/4007857058 wait complete. 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- start start 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 Processor -- start 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbbd5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b2700 con 0x5588bbb58dc0 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b2030 con 0x5588bbb584a0 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01af270 con 0x5588bbb63680 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff8a6c0c740 10 client.0 _ll_register_callbacks cb 0x7ff89800a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff89dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff89dd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47204/0 (socket says 172.21.15.77:47204) 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff89dd9b640 1 -- 172.21.15.77:0/617719688 learned_addr learned my addr 172.21.15.77:0/617719688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.461+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbbd5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 -- 172.21.15.77:0/617719688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 msgr2=0x5588bbbd5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbbd5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 -- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 msgr2=0x5588bbb66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 -- 172.21.15.77:0/617719688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588bbbd6040 con 0x5588bbb58dc0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff89dd9b640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff8900092a0 tx=0x7ff890004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20b0b167de17201 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890035040 con 0x5588bbb58dc0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7ff890005040 con 0x5588bbb58dc0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff898009680 con 0x5588bbb58dc0 2024-09-15T22:17:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff890054050 con 0x5588bbb58dc0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff8a6c0c740 10 client.16878 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5588bbbdd410 con 0x5588bbb58dc0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.462+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898009680 con 0x5588bbb58dc0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff890031630 con 0x5588bbb58dc0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 10 client.16878 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 20 client.16878 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbbdfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 4 client.16878 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588bbac5d60 con 0x5588bbb72fe0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbbdfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff89ed9d640 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbbdfc50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a3123cbd7c4bc2 server_cookie=1affd945ba23fd95 in_seq=0 out_seq=0 2024-09-15T22:17:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff885ffb640 10 client.16878 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff885ffb640 1 -- 172.21.15.77:0/617719688 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5588bbac5d60 con 0x5588bbb72fe0 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff885ffb640 10 client.16878 handle_command_reply: tid=0 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.463+0000 7ff8a6c0c740 1 client.16878 shutdown 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 msgr2=0x5588bbb672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff8900092a0 tx=0x7ff890004040 comp rx=0 tx=0).stop 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 shutdown_connections 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5588bbb72fe0 0x5588bbbdfc50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588bbb63680 0x5588bbbd5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5588bbb58dc0 0x5588bbb672f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 --2- 172.21.15.77:0/617719688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588bbb584a0 0x5588bbb66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 >> 172.21.15.77:0/617719688 conn(0x5588bbb54a30 msgr2=0x5588bbb62380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff885ffb640 0 client.16878 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 shutdown_connections 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 1 -- 172.21.15.77:0/617719688 wait complete. 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a6c0c740 20 client.16878 trim_cache size 0 max 16384 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:17:40.482 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 msgr2=0x7ff88c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 0x7ff88c0a5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ccde0 tx=0x7ff888013500 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 msgr2=0x7ff8a01a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b4a0 tx=0x7ff88800b960 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 shutdown_connections 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff88c0a33b0 0x7ff88c0a5870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8a0106a80 0x7ff8a019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8a0101420 0x7ff8a01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 --2- 172.21.15.77:0/2307060194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8a0100a80 0x7ff8a01a0450 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 >> 172.21.15.77:0/2307060194 conn(0x7ff8a00fc7a0 msgr2=0x7ff8a010c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 shutdown_connections 2024-09-15T22:17:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:40.464+0000 7ff8a4833640 1 -- 172.21.15.77:0/2307060194 wait complete. 2024-09-15T22:17:40.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.78 MB/sec execute 109 sec latency 5425.193 ms 2024-09-15T22:17:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:40 smithi196 ceph-mon[35109]: pgmap v160: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-15T22:17:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:40 smithi117 ceph-mon[35021]: pgmap v160: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-09-15T22:17:41.354 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:17:41.354 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:41.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.70 MB/sec execute 110 sec latency 6425.246 ms 2024-09-15T22:17:41.598 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:42.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 -- 172.21.15.77:0/2948193465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 msgr2=0x7f46241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 --2- 172.21.15.77:0/2948193465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241029b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f461402a480 comp rx=0 tx=0).stop 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 -- 172.21.15.77:0/2948193465 shutdown_connections 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 --2- 172.21.15.77:0/2948193465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624102ef0 0x7f462410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 --2- 172.21.15.77:0/2948193465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241029b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 --2- 172.21.15.77:0/2948193465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624100e20 0x7f4624101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.081+0000 7f462c3ed640 1 -- 172.21.15.77:0/2948193465 >> 172.21.15.77:0/2948193465 conn(0x7f46240fc7d0 msgr2=0x7f46240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- 172.21.15.77:0/2948193465 shutdown_connections 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- 172.21.15.77:0/2948193465 wait complete. 2024-09-15T22:17:42.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 Processor -- start 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- start start 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 0x7f4624199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 0x7f46241a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110fa0 con 0x7f4624102570 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110e20 con 0x7f4624100e20 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f462c3ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4624111120 con 0x7f4624102ef0 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f4629988640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f4629988640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47224/0 (socket says 172.21.15.77:47224) 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.082+0000 7f4629988640 1 -- 172.21.15.77:0/4230057109 learned_addr learned my addr 172.21.15.77:0/4230057109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462a98a640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 0x7f46241a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:42.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462a189640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 0x7f4624199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 -- 172.21.15.77:0/4230057109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 msgr2=0x7f46241a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 0x7f46241a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 -- 172.21.15.77:0/4230057109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 msgr2=0x7f4624199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 0x7f4624199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 -- 172.21.15.77:0/4230057109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46241a4930 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462a98a640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 0x7f46241a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462a189640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 0x7f4624199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f4629988640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4614006010 tx=0x7f4614003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461403f070 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46241a4b00 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4614031e30 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614035030 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.083+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a4fb0 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.084+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4624101200 con 0x7f4624102570 2024-09-15T22:17:42.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.085+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4614042070 con 0x7f4624102570 2024-09-15T22:17:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.086+0000 7f461b7fe640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 0x7f46100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.086+0000 7f462a189640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 0x7f46100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.086+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f46140f0a60 con 0x7f4624102570 2024-09-15T22:17:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.086+0000 7f462a189640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 0x7f46100a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4620006fd0 tx=0x7f4620008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:42.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.090+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46140bd100 con 0x7f4624102570 2024-09-15T22:17:42.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.291+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46241a5180 con 0x7f4624102570 2024-09-15T22:17:42.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.292+0000 7f461b7fe640 1 -- 172.21.15.77:0/4230057109 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f46140bca00 con 0x7f4624102570 2024-09-15T22:17:42.294 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:17:42.295 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:17:42.295 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:17:42.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 msgr2=0x7f46100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:42.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 0x7f46100a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4620006fd0 tx=0x7f4620008040 comp rx=0 tx=0).stop 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 msgr2=0x7f46241998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4614006010 tx=0x7f4614003c80 comp rx=0 tx=0).stop 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 shutdown_connections 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46100a33b0 0x7f46100a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4624102ef0 0x7f46241a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4624102570 0x7f46241998b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 --2- 172.21.15.77:0/4230057109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4624100e20 0x7f4624199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:42.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 >> 172.21.15.77:0/4230057109 conn(0x7f46240fc7d0 msgr2=0x7f46241068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:42.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 shutdown_connections 2024-09-15T22:17:42.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:42.294+0000 7f462c3ed640 1 -- 172.21.15.77:0/4230057109 wait complete. 2024-09-15T22:17:42.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.62 MB/sec execute 111 sec latency 7425.316 ms 2024-09-15T22:17:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:42 smithi077 ceph-mon[30980]: pgmap v161: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-15T22:17:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:42 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4230057109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:42 smithi196 ceph-mon[35109]: pgmap v161: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-15T22:17:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:42 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4230057109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:42 smithi117 ceph-mon[35021]: pgmap v161: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-15T22:17:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:42 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4230057109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:43.149 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:17:43.394 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:43.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.54 MB/sec execute 112 sec latency 8425.428 ms 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 -- 172.21.15.77:0/4223297043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868102790 msgr2=0x7f58681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 --2- 172.21.15.77:0/4223297043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868102790 0x7f58681067e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f585c009900 tx=0x7f585c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 -- 172.21.15.77:0/4223297043 shutdown_connections 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 --2- 172.21.15.77:0/4223297043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 --2- 172.21.15.77:0/4223297043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868102790 0x7f58681067e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 --2- 172.21.15.77:0/4223297043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868101de0 0x7f58681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 -- 172.21.15.77:0/4223297043 >> 172.21.15.77:0/4223297043 conn(0x7f58680fd8c0 msgr2=0x7f58680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 -- 172.21.15.77:0/4223297043 shutdown_connections 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.870+0000 7f586e91e640 1 -- 172.21.15.77:0/4223297043 wait complete. 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 Processor -- start 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 -- start start 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868102790 0x7f5868198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111f30 con 0x7f5868101de0 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111db0 con 0x7f5868102790 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.871+0000 7f586e91e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58681120b0 con 0x7f5868106f10 2024-09-15T22:17:43.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47228/0 (socket says 172.21.15.77:47228) 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 -- 172.21.15.77:0/2616487206 learned_addr learned my addr 172.21.15.77:0/2616487206 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f586cebb640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f58677fe640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868102790 0x7f5868198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 -- 172.21.15.77:0/2616487206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 msgr2=0x7f586819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 -- 172.21.15.77:0/2616487206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868102790 msgr2=0x7f5868198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868102790 0x7f5868198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 -- 172.21.15.77:0/2616487206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586819d2c0 con 0x7f5868101de0 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f586cebb640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f5867fff640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f585800c920 tx=0x7f585800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f58657fa640 1 -- 172.21.15.77:0/2616487206 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858007c70 con 0x7f5868101de0 2024-09-15T22:17:43.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586819d5b0 con 0x7f5868101de0 2024-09-15T22:17:43.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f58657fa640 1 -- 172.21.15.77:0/2616487206 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5858007e10 con 0x7f5868101de0 2024-09-15T22:17:43.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f58657fa640 1 -- 172.21.15.77:0/2616487206 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858018890 con 0x7f5868101de0 2024-09-15T22:17:43.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.872+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681a4e90 con 0x7f5868101de0 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.874+0000 7f58657fa640 1 -- 172.21.15.77:0/2616487206 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5858018a70 con 0x7f5868101de0 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 Processor -- start 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 -- start start 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561ec16292f0 con 0x561ec16ce370 2024-09-15T22:17:43.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ec15d0cf0 con 0x561ec16c35d0 2024-09-15T22:17:43.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5870cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561ec16252b0 con 0x561ec16c4b40 2024-09-15T22:17:43.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47240/0 (socket says 172.21.15.77:47240) 2024-09-15T22:17:43.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 -- 172.21.15.77:0/2568425465 learned_addr learned my addr 172.21.15.77:0/2568425465 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:43.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58657fa640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 0x7f58500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5867fff640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f586cebb640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f586cebb640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 0x7f58500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 -- 172.21.15.77:0/2568425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 msgr2=0x561ec16c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 -- 172.21.15.77:0/2568425465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 msgr2=0x561ec16c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58677fe640 1 -- 172.21.15.77:0/2568425465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec15b53a0 con 0x561ec16ce370 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f586cebb640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f5867fff640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.875+0000 7f58657fa640 1 -- 172.21.15.77:0/2616487206 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f58580d02c0 con 0x7f5868101de0 2024-09-15T22:17:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f58677fe640 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f585c005ab0 tx=0x7f585c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b660c832be6ec002 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f584e7fc640 1 -- 172.21.15.77:0/2568425465 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c03f070 con 0x561ec16ce370 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f584e7fc640 1 -- 172.21.15.77:0/2568425465 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c031a20 con 0x561ec16ce370 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f586cebb640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 0x7f58500a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5860007310 tx=0x7f58600072a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f584e7fc640 1 -- 172.21.15.77:0/2568425465 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c035070 con 0x561ec16ce370 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f5870cf7740 1 -- 172.21.15.77:0/2568425465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 msgr2=0x561ec16d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f585c005ab0 tx=0x7f585c031630 comp rx=0 tx=0).stop 2024-09-15T22:17:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f5870cf7740 1 -- 172.21.15.77:0/2568425465 shutdown_connections 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.876+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16d0760 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2568425465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- 172.21.15.77:0/2568425465 >> 172.21.15.77:0/2568425465 conn(0x561ec168c310 msgr2=0x561ec16bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- 172.21.15.77:0/2568425465 shutdown_connections 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- 172.21.15.77:0/2568425465 wait complete. 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- start start 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 Processor -- start 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16ce370 0x561ec16cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561ec16d1010 con 0x561ec16c4b40 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561ec15e4060 con 0x561ec16ce370 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561ec16d1190 con 0x561ec16c35d0 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f5870cf7740 10 client.0 _ll_register_callbacks cb 0x561ec13378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f58677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f58677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47242/0 (socket says 172.21.15.77:47242) 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.877+0000 7f58677fe640 1 -- 172.21.15.77:0/1559107022 learned_addr learned my addr 172.21.15.77:0/1559107022 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f5867fff640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 -- 172.21.15.77:0/1559107022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 msgr2=0x561ec16cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f586cebb640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16ce370 0x561ec16cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 -- 172.21.15.77:0/1559107022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16ce370 msgr2=0x561ec16cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16ce370 0x561ec16cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 -- 172.21.15.77:0/1559107022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec139a3c0 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58677fe640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f585c031550 tx=0x7f585c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835431030b6d061f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58477fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c035070 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f585c004030 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c031a20 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f585c055050 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f5870cf7740 10 client.16896 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561ec16dc9e0 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.878+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c031a20 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f585c03b430 con 0x561ec16c4b40 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 10 client.16896 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 10 client.16896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 20 client.16896 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:43.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec16ddd30 0x561ec16e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 4 client.16896 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ec1636860 con 0x561ec16ddd30 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5867fff640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec16ddd30 0x561ec16e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f58477fe640 10 client.16896 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.879+0000 7f5867fff640 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec16ddd30 0x561ec16e0120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88427b41684c7cbf server_cookie=fbbf3db6a1e05044 in_seq=0 out_seq=0 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f58477fe640 1 -- 172.21.15.77:0/1559107022 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ec1636860 con 0x561ec16ddd30 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f58477fe640 10 client.16896 handle_command_reply: tid=0 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 client.16896 shutdown 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 msgr2=0x561ec16cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f585c031550 tx=0x7f585c004a50 comp rx=0 tx=0).stop 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 shutdown_connections 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec16ddd30 0x561ec16e0120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16ce370 0x561ec16cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16c4b40 0x561ec16cafe0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f58477fe640 0 client.16896 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 --2- 172.21.15.77:0/1559107022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 >> 172.21.15.77:0/1559107022 conn(0x561ec168c310 msgr2=0x561ec16c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 shutdown_connections 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 1 -- 172.21.15.77:0/1559107022 wait complete. 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.880+0000 7f5870cf7740 20 client.16896 trim_cache size 0 max 16384 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 Processor -- start 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 -- start start 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58681aed50 con 0x561ec16ce370 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b0890 con 0x561ec16c35d0 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f5870cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58681af420 con 0x561ec16c4b40 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f586cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f586cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47244/0 (socket says 172.21.15.77:47244) 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f586cebb640 1 -- 172.21.15.77:0/919193430 learned_addr learned my addr 172.21.15.77:0/919193430 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.893+0000 7f58677fe640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 -- 172.21.15.77:0/919193430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 msgr2=0x561ec16e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 -- 172.21.15.77:0/919193430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 msgr2=0x561ec16da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f5867fff640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16da2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 -- 172.21.15.77:0/919193430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec16e0490 con 0x561ec16ce370 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f58677fe640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f586cebb640 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5860010700 tx=0x7f5860012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcbd352560eab83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f584dffb640 1 -- 172.21.15.77:0/919193430 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004d20 con 0x561ec16ce370 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f584dffb640 1 -- 172.21.15.77:0/919193430 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586001cb30 con 0x561ec16ce370 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f584dffb640 1 -- 172.21.15.77:0/919193430 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600058e0 con 0x561ec16ce370 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f5870cf7740 1 -- 172.21.15.77:0/919193430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 msgr2=0x561ec16e5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.894+0000 7f5870cf7740 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5860010700 tx=0x7f5860012070 comp rx=0 tx=0).stop 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- 172.21.15.77:0/919193430 shutdown_connections 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec16e5c00 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c4b40 0x561ec16e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.942 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- 172.21.15.77:0/919193430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c35d0 0x561ec16da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- 172.21.15.77:0/919193430 >> 172.21.15.77:0/919193430 conn(0x561ec168c310 msgr2=0x561ec16ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- 172.21.15.77:0/919193430 shutdown_connections 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- 172.21.15.77:0/919193430 wait complete. 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- start start 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 Processor -- start 2024-09-15T22:17:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 0x561ec16d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b01c0 con 0x561ec16ce370 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58681afaf0 con 0x561ec16c4b40 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58681acd30 con 0x561ec16c35d0 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f5870cf7740 10 client.0 _ll_register_callbacks cb 0x7f585c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.895+0000 7f586cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47254/0 (socket says 172.21.15.77:47254) 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 -- 172.21.15.77:0/2745889630 learned_addr learned my addr 172.21.15.77:0/2745889630 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f5867fff640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58677fe640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 0x561ec16d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 -- 172.21.15.77:0/2745889630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 msgr2=0x561ec16d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 -- 172.21.15.77:0/2745889630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 msgr2=0x561ec16d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 0x561ec16d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 -- 172.21.15.77:0/2745889630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec174a4d0 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58677fe640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 0x561ec16d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f586cebb640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5860007e00 tx=0x7f586001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc746260a818138 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58477fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860033060 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f5860004a40 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586001cb30 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f586003a050 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f5870cf7740 10 client.16902 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.896+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561ec17517a0 con 0x561ec16ce370 2024-09-15T22:17:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586001cb30 con 0x561ec16ce370 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f586002d070 con 0x561ec16ce370 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 10 client.16902 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 10 client.16902 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 20 client.16902 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec1751e10 0x561ec1754230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 4 client.16902 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ec1630d70 con 0x561ec1751e10 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5867fff640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec1751e10 0x561ec1754230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f58477fe640 10 client.16902 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5867fff640 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec1751e10 0x561ec1754230 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10753cfd4019ac50 server_cookie=728e2871dbcbc4b4 in_seq=0 out_seq=0 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f58477fe640 1 -- 172.21.15.77:0/2745889630 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ec1630d70 con 0x561ec1751e10 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f58477fe640 10 client.16902 handle_command_reply: tid=0 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.897+0000 7f5870cf7740 1 client.16902 shutdown 2024-09-15T22:17:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 msgr2=0x561ec1749ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5860007e00 tx=0x7f586001c880 comp rx=0 tx=0).stop 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 shutdown_connections 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561ec1751e10 0x561ec1754230 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561ec16ce370 0x561ec1749ee0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561ec16c4b40 0x561ec16d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 --2- 172.21.15.77:0/2745889630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ec16c35d0 0x561ec16d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 >> 172.21.15.77:0/2745889630 conn(0x561ec168c310 msgr2=0x561ec16cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f58477fe640 0 client.16902 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 shutdown_connections 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 1 -- 172.21.15.77:0/2745889630 wait complete. 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f5870cf7740 20 client.16902 trim_cache size 0 max 16384 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 msgr2=0x7f58500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 0x7f58500a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5860007310 tx=0x7f58600072a0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 msgr2=0x7f5868198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f585800c920 tx=0x7f585800cde0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 shutdown_connections 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f58500a33b0 0x7f58500a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5868106f10 0x7f586819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5868102790 0x7f5868198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 --2- 172.21.15.77:0/2616487206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5868101de0 0x7f5868198310 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.898+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 >> 172.21.15.77:0/2616487206 conn(0x7f58680fd8c0 msgr2=0x7f58680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.899+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 shutdown_connections 2024-09-15T22:17:43.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:43.899+0000 7f586e91e640 1 -- 172.21.15.77:0/2616487206 wait complete. 2024-09-15T22:17:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:44 smithi077 ceph-mon[30980]: pgmap v162: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:44.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.47 MB/sec execute 113 sec latency 9425.518 ms 2024-09-15T22:17:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:44 smithi196 ceph-mon[35109]: pgmap v162: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:44 smithi117 ceph-mon[35021]: pgmap v162: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:44.819 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:17:45.063 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:45.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.39 MB/sec execute 114 sec latency 10425.584 ms 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 -- 172.21.15.77:0/3005604279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f0101e10 msgr2=0x7f61f01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/3005604279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f0101e10 0x7f61f01021f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009900 tx=0x7f61e002a410 comp rx=0 tx=0).stop 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 -- 172.21.15.77:0/3005604279 shutdown_connections 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/3005604279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0106f40 0x7f61f010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/3005604279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f01027c0 0x7f61f0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/3005604279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f0101e10 0x7f61f01021f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 -- 172.21.15.77:0/3005604279 >> 172.21.15.77:0/3005604279 conn(0x7f61f00fd8d0 msgr2=0x7f61f00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 -- 172.21.15.77:0/3005604279 shutdown_connections 2024-09-15T22:17:45.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.546+0000 7f61f5eb7640 1 -- 172.21.15.77:0/3005604279 wait complete. 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 Processor -- start 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 -- start start 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0101e10 0x7f61f01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f0106f40 0x7f61f019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0112040 con 0x7f61f01027c0 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0111ec0 con 0x7f61f0101e10 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61f5eb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01121c0 con 0x7f61f0106f40 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47276/0 (socket says 172.21.15.77:47276) 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61eeffd640 1 -- 172.21.15.77:0/2915745020 learned_addr learned my addr 172.21.15.77:0/2915745020 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.547+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0101e10 0x7f61f01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 -- 172.21.15.77:0/2915745020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f0106f40 msgr2=0x7f61f019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f0106f40 0x7f61f019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 -- 172.21.15.77:0/2915745020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0101e10 msgr2=0x7f61f01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0101e10 0x7f61f01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 -- 172.21.15.77:0/2915745020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f019d550 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61eeffd640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f61e400b4a0 tx=0x7f61e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61ecff9640 1 -- 172.21.15.77:0/2915745020 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4014070 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61ecff9640 1 -- 172.21.15.77:0/2915745020 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4010af0 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f019d840 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61ecff9640 1 -- 172.21.15.77:0/2915745020 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4017d10 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.548+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a5120 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.550+0000 7f61ecff9640 1 -- 172.21.15.77:0/2915745020 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61e4017070 con 0x7f61f01027c0 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.550+0000 7f61f8290740 1 Processor -- start 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.550+0000 7f61f8290740 1 -- start start 2024-09-15T22:17:45.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 0x56446402f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 0x56446403c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564463f952f0 con 0x56446402f5d0 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564463f3ccf0 con 0x564464030b40 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61f8290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564463f912b0 con 0x56446403a370 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47280/0 (socket says 172.21.15.77:47280) 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 -- 172.21.15.77:0/3351442256 learned_addr learned my addr 172.21.15.77:0/3351442256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61effff640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 0x56446402f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61eeffd640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 0x56446403c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 -- 172.21.15.77:0/3351442256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 msgr2=0x56446403c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 0x56446403c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 -- 172.21.15.77:0/3351442256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 msgr2=0x56446402f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 0x56446402f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 -- 172.21.15.77:0/3351442256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564463f213a0 con 0x56446402f5d0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61eeffd640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 0x56446403c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61effff640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 0x56446402f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ecff9640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1290e47802cbc762 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.551+0000 7f61effff640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61ecff9640 1 -- 172.21.15.77:0/2915745020 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61e40cf3c0 con 0x7f61f01027c0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61d5ffb640 1 -- 172.21.15.77:0/3351442256 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e003f070 con 0x56446402f5d0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61d5ffb640 1 -- 172.21.15.77:0/3351442256 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e00317b0 con 0x56446402f5d0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61d5ffb640 1 -- 172.21.15.77:0/3351442256 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0035070 con 0x56446402f5d0 2024-09-15T22:17:45.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61f8290740 1 -- 172.21.15.77:0/3351442256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 msgr2=0x56446402fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61f8290740 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61effff640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009ec0 tx=0x7f61e8009450 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61f8290740 1 -- 172.21.15.77:0/3351442256 shutdown_connections 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.552+0000 7f61f8290740 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446403a370 0x56446403c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x56446402fa10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 --2- 172.21.15.77:0/3351442256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564464030b40 0x56446402f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- 172.21.15.77:0/3351442256 >> 172.21.15.77:0/3351442256 conn(0x564463ff8310 msgr2=0x56446402bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- 172.21.15.77:0/3351442256 shutdown_connections 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- 172.21.15.77:0/3351442256 wait complete. 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- start start 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 Processor -- start 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464036f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x5644640374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56446403d010 con 0x56446402f5d0 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564463f50060 con 0x56446403a370 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56446403d190 con 0x564464030b40 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61f8290740 10 client.0 _ll_register_callbacks cb 0x564463ca38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47286/0 (socket says 172.21.15.77:47286) 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.553+0000 7f61ef7fe640 1 -- 172.21.15.77:0/1231031863 learned_addr learned my addr 172.21.15.77:0/1231031863 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61effff640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x5644640374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 -- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 msgr2=0x564464036f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464036f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 -- 172.21.15.77:0/1231031863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 msgr2=0x5644640374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x5644640374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 -- 172.21.15.77:0/1231031863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564463d063c0 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f61e00316a0 tx=0x7f61e00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a4cba363d83cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e003f030 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e00317b0 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61e0004460 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61e0054050 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61f8290740 10 client.16914 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564464048990 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.554+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e00317b0 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f61e003b730 con 0x56446402f5d0 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 10 client.16914 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 10 client.16914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:17:45.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 20 client.16914 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x56446404c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 4 client.16914 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564463fa2860 con 0x564464049ce0 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61eeffd640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x56446404c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61ceffd640 10 client.16914 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.555+0000 7f61eeffd640 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x56446404c0d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96f15eaea53b51f6 server_cookie=de0329e5cb5e42b4 in_seq=0 out_seq=0 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61ceffd640 1 -- 172.21.15.77:0/1231031863 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564463fa2860 con 0x564464049ce0 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61ceffd640 10 client.16914 handle_command_reply: tid=0 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 client.16914 shutdown 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 msgr2=0x564464039cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f61e00316a0 tx=0x7f61e00316d0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 shutdown_connections 2024-09-15T22:17:45.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x56446404c0d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61ceffd640 0 client.16914 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x5644640374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464036f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 --2- 172.21.15.77:0/1231031863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446402f5d0 0x564464039cc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.556+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 >> 172.21.15.77:0/1231031863 conn(0x564463ff8310 msgr2=0x564464031960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.557+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 shutdown_connections 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.557+0000 7f61f8290740 1 -- 172.21.15.77:0/1231031863 wait complete. 2024-09-15T22:17:45.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.557+0000 7f61f8290740 20 client.16914 trim_cache size 0 max 16384 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 Processor -- start 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 -- start start 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464035f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01aefd0 con 0x56446403a370 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b0b10 con 0x56446402f5d0 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.570+0000 7f61f8290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01af6a0 con 0x564464030b40 2024-09-15T22:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34040/0 (socket says 172.21.15.77:34040) 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61ef7fe640 1 -- 172.21.15.77:0/1924129231 learned_addr learned my addr 172.21.15.77:0/1924129231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47298/0 (socket says 172.21.15.77:47298) 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61eeffd640 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464035f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 -- 172.21.15.77:0/1924129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 msgr2=0x564464035f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464035f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 -- 172.21.15.77:0/1924129231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 msgr2=0x5644640462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 -- 172.21.15.77:0/1924129231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564464039ff0 con 0x56446403a370 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61effff640 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f61e8017730 tx=0x7f61e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758971fdcf11ae3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.571+0000 7f61d57fa640 1 -- 172.21.15.77:0/1924129231 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8017e60 con 0x56446403a370 2024-09-15T22:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61d57fa640 1 -- 172.21.15.77:0/1924129231 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e8027070 con 0x56446403a370 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61d57fa640 1 -- 172.21.15.77:0/1924129231 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8020880 con 0x56446403a370 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 -- 172.21.15.77:0/1924129231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 msgr2=0x564464051670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f61e8017730 tx=0x7f61e8015000 comp rx=0 tx=0).stop 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 -- 172.21.15.77:0/1924129231 shutdown_connections 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56446403a370 0x564464051670 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564464030b40 0x564464035f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 --2- 172.21.15.77:0/1924129231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446402f5d0 0x5644640462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 -- 172.21.15.77:0/1924129231 >> 172.21.15.77:0/1924129231 conn(0x564463ff8310 msgr2=0x564464035860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 -- 172.21.15.77:0/1924129231 shutdown_connections 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.572+0000 7f61f8290740 1 -- 172.21.15.77:0/1924129231 wait complete. 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 -- start start 2024-09-15T22:17:45.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 Processor -- start 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446402f5d0 0x564464042b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x564464034220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b0440 con 0x564464030b40 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01afd70 con 0x56446403a370 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.573+0000 7f61f8290740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01acfb0 con 0x56446402f5d0 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61f8290740 10 client.0 _ll_register_callbacks cb 0x7f61e0032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47308/0 (socket says 172.21.15.77:47308) 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 -- 172.21.15.77:0/3494531783 learned_addr learned my addr 172.21.15.77:0/3494531783 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446402f5d0 0x564464042b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61effff640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x564464034220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 -- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446402f5d0 msgr2=0x564464042b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446402f5d0 0x564464042b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 -- 172.21.15.77:0/3494531783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 msgr2=0x564464034220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x564464034220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 -- 172.21.15.77:0/3494531783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564464034a50 con 0x564464030b40 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61effff640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x564464034220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61eeffd640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f61e40cc990 tx=0x7f61e40d18a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286ffcaefec2cee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e401e040 con 0x564464030b40 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61e4017360 con 0x564464030b40 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e8027070 con 0x564464030b40 2024-09-15T22:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61e40a40e0 con 0x564464030b40 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61f8290740 10 client.16920 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.574+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5644640b4730 con 0x564464030b40 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e8027070 con 0x564464030b40 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f61e40cca50 con 0x564464030b40 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 10 client.16920 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 10 client.16920 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 20 client.16920 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x5644640b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 4 client.16920 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564463f9cd70 con 0x564464049ce0 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.575+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x5644640b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61ceffd640 10 client.16920 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61ef7fe640 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x5644640b6e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=adb5232b675e2c72 server_cookie=4f965ff474933269 in_seq=0 out_seq=0 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61ceffd640 1 -- 172.21.15.77:0/3494531783 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564463f9cd70 con 0x564464049ce0 2024-09-15T22:17:45.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61ceffd640 10 client.16920 handle_command_reply: tid=0 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 client.16920 shutdown 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 msgr2=0x564464033cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f61e40cc990 tx=0x7f61e40d18a0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 shutdown_connections 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564464049ce0 0x5644640b6e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56446403a370 0x564464034220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564464030b40 0x564464033cc0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 --2- 172.21.15.77:0/3494531783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56446402f5d0 0x564464042b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 >> 172.21.15.77:0/3494531783 conn(0x564463ff8310 msgr2=0x564464036080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61ceffd640 0 client.16920 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 shutdown_connections 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 1 -- 172.21.15.77:0/3494531783 wait complete. 2024-09-15T22:17:45.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.576+0000 7f61f8290740 20 client.16920 trim_cache size 0 max 16384 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 msgr2=0x7f61d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009ec0 tx=0x7f61e8009450 comp rx=0 tx=0).stop 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 msgr2=0x7f61f0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f61e400b4a0 tx=0x7f61e400b960 comp rx=0 tx=0).stop 2024-09-15T22:17:45.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 shutdown_connections 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f0106f40 0x7f61f019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f01027c0 0x7f61f0198ae0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 --2- 172.21.15.77:0/2915745020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61f0101e10 0x7f61f01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 >> 172.21.15.77:0/2915745020 conn(0x7f61f00fd8d0 msgr2=0x7f61f00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 shutdown_connections 2024-09-15T22:17:45.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:45.577+0000 7f61f5eb7640 1 -- 172.21.15.77:0/2915745020 wait complete. 2024-09-15T22:17:46.407 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:17:46.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.32 MB/sec execute 115 sec latency 11425.613 ms 2024-09-15T22:17:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:46 smithi077 ceph-mon[30980]: pgmap v163: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:46 smithi196 ceph-mon[35109]: pgmap v163: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:46 smithi117 ceph-mon[35021]: pgmap v163: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-15T22:17:46.663 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 -- 172.21.15.77:0/824009702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248102790 msgr2=0x7fb2481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 --2- 172.21.15.77:0/824009702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248102790 0x7fb2481067e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009900 tx=0x7fb23c02a410 comp rx=0 tx=0).stop 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 -- 172.21.15.77:0/824009702 shutdown_connections 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 --2- 172.21.15.77:0/824009702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248106f10 0x7fb24810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 --2- 172.21.15.77:0/824009702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248102790 0x7fb2481067e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 --2- 172.21.15.77:0/824009702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248101de0 0x7fb2481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 -- 172.21.15.77:0/824009702 >> 172.21.15.77:0/824009702 conn(0x7fb2480fd8c0 msgr2=0x7fb2480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 -- 172.21.15.77:0/824009702 shutdown_connections 2024-09-15T22:17:47.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.164+0000 7fb24d59b640 1 -- 172.21.15.77:0/824009702 wait complete. 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 Processor -- start 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 -- start start 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 0x7fb2481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248106f10 0x7fb24819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb248111f70 con 0x7fb248101de0 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb248111df0 con 0x7fb248106f10 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb24d59b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481120f0 con 0x7fb248102790 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.165+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47330/0 (socket says 172.21.15.77:47330) 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 -- 172.21.15.77:0/2372383918 learned_addr learned my addr 172.21.15.77:0/2372383918 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb2467fc640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 0x7fb2481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb2477fe640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248106f10 0x7fb24819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 -- 172.21.15.77:0/2372383918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 msgr2=0x7fb2481989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 0x7fb2481989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 -- 172.21.15.77:0/2372383918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248106f10 msgr2=0x7fb24819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248106f10 0x7fb24819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 -- 172.21.15.77:0/2372383918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24819d440 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb2467fc640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 0x7fb2481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb246ffd640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb23800d950 tx=0x7fb23800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb22ffff640 1 -- 172.21.15.77:0/2372383918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238017070 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24819d730 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb22ffff640 1 -- 172.21.15.77:0/2372383918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb238009c30 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb22ffff640 1 -- 172.21.15.77:0/2372383918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23800f6b0 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.166+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb248074420 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.168+0000 7fb22ffff640 1 -- 172.21.15.77:0/2372383918 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb2380109d0 con 0x7fb248101de0 2024-09-15T22:17:47.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 Processor -- start 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 -- start start 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e675090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e682760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b0e5db2f0 con 0x562b0e6755d0 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562b0e582cf0 con 0x562b0e680370 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb24f974740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b0e5d72b0 con 0x562b0e676b40 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb22ffff640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 0x7fb2300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb2467fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e682760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb2477fe640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 0x7fb2300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47338/0 (socket says 172.21.15.77:47338) 2024-09-15T22:17:47.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb246ffd640 1 -- 172.21.15.77:0/1031282994 learned_addr learned my addr 172.21.15.77:0/1031282994 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.169+0000 7fb2477fe640 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e675090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb22ffff640 1 -- 172.21.15.77:0/2372383918 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb2380cfd80 con 0x7fb248101de0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 -- 172.21.15.77:0/1031282994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 msgr2=0x562b0e675090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e675090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 -- 172.21.15.77:0/1031282994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 msgr2=0x562b0e682760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e682760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 -- 172.21.15.77:0/1031282994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0e5673a0 con 0x562b0e6755d0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb2380986f0 tx=0x7fb23809bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb77f8b294f44a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb2477fe640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 0x7fb2300a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb240009270 tx=0x7fb240004ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb22d7fa640 1 -- 172.21.15.77:0/1031282994 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238020070 con 0x562b0e6755d0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb22d7fa640 1 -- 172.21.15.77:0/1031282994 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2380a00d0 con 0x562b0e6755d0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.170+0000 7fb22d7fa640 1 -- 172.21.15.77:0/1031282994 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23809b370 con 0x562b0e6755d0 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- 172.21.15.77:0/1031282994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 msgr2=0x562b0e675a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb2380986f0 tx=0x7fb23809bc20 comp rx=0 tx=0).stop 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- 172.21.15.77:0/1031282994 shutdown_connections 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e682760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e675a10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- 172.21.15.77:0/1031282994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e675090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- 172.21.15.77:0/1031282994 >> 172.21.15.77:0/1031282994 conn(0x562b0e63e310 msgr2=0x562b0e671c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- 172.21.15.77:0/1031282994 shutdown_connections 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- 172.21.15.77:0/1031282994 wait complete. 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- start start 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 Processor -- start 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e67d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e67d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b0e682fb0 con 0x562b0e6755d0 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562b0e596060 con 0x562b0e680370 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562b0e683130 con 0x562b0e676b40 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.171+0000 7fb24f974740 10 client.0 _ll_register_callbacks cb 0x562b0e2e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47340/0 (socket says 172.21.15.77:47340) 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 -- 172.21.15.77:0/858081426 learned_addr learned my addr 172.21.15.77:0/858081426 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb2477fe640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e67d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb2467fc640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e67d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 -- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 msgr2=0x562b0e67d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e67d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 -- 172.21.15.77:0/858081426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 msgr2=0x562b0e67d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e67d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 -- 172.21.15.77:0/858081426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0e34c3c0 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb2467fc640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e67d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb246ffd640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb2380d0130 tx=0x7fb238010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dc6d6a4bb68a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb22aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238020050 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb23809b380 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2380a00d0 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb238097a70 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb24f974740 10 client.16932 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b0e68eae0 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.172+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2380a00d0 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb2380b3070 con 0x562b0e6755d0 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 10 client.16932 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:47.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 10 client.16932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 20 client.16932 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e68fe30 0x562b0e692220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 4 client.16932 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0e5e8860 con 0x562b0e68fe30 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb2467fc640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e68fe30 0x562b0e692220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb22aff5640 10 client.16932 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.173+0000 7fb2467fc640 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e68fe30 0x562b0e692220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=225aeac6f46c176a server_cookie=2c30d4fc431b3578 in_seq=0 out_seq=0 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb22aff5640 1 -- 172.21.15.77:0/858081426 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b0e5e8860 con 0x562b0e68fe30 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb22aff5640 10 client.16932 handle_command_reply: tid=0 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 client.16932 shutdown 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 msgr2=0x562b0e67fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb2380d0130 tx=0x7fb238010d70 comp rx=0 tx=0).stop 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 shutdown_connections 2024-09-15T22:17:47.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e68fe30 0x562b0e692220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e680370 0x562b0e67d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e676b40 0x562b0e67d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 --2- 172.21.15.77:0/858081426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e67fdf0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 >> 172.21.15.77:0/858081426 conn(0x562b0e63e310 msgr2=0x562b0e677990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb22aff5640 0 client.16932 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 shutdown_connections 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 1 -- 172.21.15.77:0/858081426 wait complete. 2024-09-15T22:17:47.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.174+0000 7fb24f974740 20 client.16932 trim_cache size 0 max 16384 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.187+0000 7fb24f974740 1 Processor -- start 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.187+0000 7fb24f974740 1 -- start start 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 0x562b0e68c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e67bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b1100 con 0x562b0e680370 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b2c40 con 0x562b0e676b40 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb24f974740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b17d0 con 0x562b0e6755d0 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47354/0 (socket says 172.21.15.77:47354) 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 0x562b0e68c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2477fe640 1 -- 172.21.15.77:0/4175299000 learned_addr learned my addr 172.21.15.77:0/4175299000 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2467fc640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e67bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 -- 172.21.15.77:0/4175299000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 msgr2=0x562b0e67bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e67bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 -- 172.21.15.77:0/4175299000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 msgr2=0x562b0e681760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 -- 172.21.15.77:0/4175299000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0e672960 con 0x562b0e6755d0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2477fe640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb2467fc640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e67bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb246ffd640 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 0x562b0e68c3d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2380986f0 tx=0x7fb2380983e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f15d4d9083ad26fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.188+0000 7fb22cff9640 1 -- 172.21.15.77:0/4175299000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238020070 con 0x562b0e6755d0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb22cff9640 1 -- 172.21.15.77:0/4175299000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb238098500 con 0x562b0e6755d0 2024-09-15T22:17:47.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb22cff9640 1 -- 172.21.15.77:0/4175299000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238010460 con 0x562b0e6755d0 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 -- 172.21.15.77:0/4175299000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 msgr2=0x562b0e68c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 0x562b0e68c3d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2380986f0 tx=0x7fb2380983e0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 -- 172.21.15.77:0/4175299000 shutdown_connections 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e680370 0x562b0e681760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e67bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 --2- 172.21.15.77:0/4175299000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e6755d0 0x562b0e68c3d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 -- 172.21.15.77:0/4175299000 >> 172.21.15.77:0/4175299000 conn(0x562b0e63e310 msgr2=0x562b0e67b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 -- 172.21.15.77:0/4175299000 shutdown_connections 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.189+0000 7fb24f974740 1 -- 172.21.15.77:0/4175299000 wait complete. 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 -- start start 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 Processor -- start 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e6956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e680370 0x562b0e679ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b2570 con 0x562b0e6755d0 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b1ea0 con 0x562b0e676b40 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481af0e0 con 0x562b0e680370 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb24f974740 10 client.0 _ll_register_callbacks cb 0x7fb2400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47364/0 (socket says 172.21.15.77:47364) 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 -- 172.21.15.77:0/1812350572 learned_addr learned my addr 172.21.15.77:0/1812350572 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb2467fc640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e6956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 -- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e680370 msgr2=0x562b0e679ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e680370 0x562b0e679ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 -- 172.21.15.77:0/1812350572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 msgr2=0x562b0e6956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e6956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.190+0000 7fb246ffd640 1 -- 172.21.15.77:0/1812350572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0e678120 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb246ffd640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb2380a4460 tx=0x7fb23809b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0304af14452524a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238020070 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb23809ade0 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb238098500 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb2380a00d0 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 10 client.16938 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b0e678120 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb238098500 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb2380b3070 con 0x562b0e6755d0 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 10 client.16938 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 10 client.16938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 20 client.16938 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:47.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e6fa4b0 0x562b0e6fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 4 client.16938 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.191+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b0e5e2d70 con 0x562b0e6fa4b0 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb2467fc640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e6fa4b0 0x562b0e6fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb2467fc640 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e6fa4b0 0x562b0e6fc8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e69aedb67070d8ef server_cookie=6bd8faf73529fcc in_seq=0 out_seq=0 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb22aff5640 10 client.16938 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb22aff5640 1 -- 172.21.15.77:0/1812350572 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b0e5e2d70 con 0x562b0e6fa4b0 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb22aff5640 10 client.16938 handle_command_reply: tid=0 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 client.16938 shutdown 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 msgr2=0x562b0e6951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb2380a4460 tx=0x7fb23809b680 comp rx=0 tx=0).stop 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 shutdown_connections 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562b0e6fa4b0 0x562b0e6fc8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb22aff5640 0 client.16938 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:17:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562b0e680370 0x562b0e679ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b0e676b40 0x562b0e6956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 --2- 172.21.15.77:0/1812350572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b0e6755d0 0x562b0e6951b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 >> 172.21.15.77:0/1812350572 conn(0x562b0e63e310 msgr2=0x562b0e67b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.192+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 shutdown_connections 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24f974740 1 -- 172.21.15.77:0/1812350572 wait complete. 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24f974740 20 client.16938 trim_cache size 0 max 16384 2024-09-15T22:17:47.204 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 msgr2=0x7fb2300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 0x7fb2300a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb240009270 tx=0x7fb240004ab0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 msgr2=0x7fb248198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb23800d950 tx=0x7fb23800de10 comp rx=0 tx=0).stop 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 shutdown_connections 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb2300a33b0 0x7fb2300a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb248106f10 0x7fb24819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb248102790 0x7fb2481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 --2- 172.21.15.77:0/2372383918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb248101de0 0x7fb248198490 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 >> 172.21.15.77:0/2372383918 conn(0x7fb2480fd8c0 msgr2=0x7fb2480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 shutdown_connections 2024-09-15T22:17:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:47.193+0000 7fb24d59b640 1 -- 172.21.15.77:0/2372383918 wait complete. 2024-09-15T22:17:47.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.25 MB/sec execute 116 sec latency 12425.671 ms 2024-09-15T22:17:48.096 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:17:48.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:48 smithi077 ceph-mon[30980]: pgmap v164: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-15T22:17:48.348 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:48.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.18 MB/sec execute 117 sec latency 13425.739 ms 2024-09-15T22:17:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:48 smithi196 ceph-mon[35109]: pgmap v164: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-15T22:17:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:48 smithi117 ceph-mon[35021]: pgmap v164: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-15T22:17:48.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 -- 172.21.15.77:0/1650535085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d0103700 msgr2=0x7fa4d010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 --2- 172.21.15.77:0/1650535085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d0103700 0x7fa4d010bbf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009900 tx=0x7fa4bc02a480 comp rx=0 tx=0).stop 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 -- 172.21.15.77:0/1650535085 shutdown_connections 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 --2- 172.21.15.77:0/1650535085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d010c180 0x7fa4d010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 --2- 172.21.15.77:0/1650535085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d0103700 0x7fa4d010bbf0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 --2- 172.21.15.77:0/1650535085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 0x7fa4d01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 -- 172.21.15.77:0/1650535085 >> 172.21.15.77:0/1650535085 conn(0x7fa4d00fc7a0 msgr2=0x7fa4d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 -- 172.21.15.77:0/1650535085 shutdown_connections 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.840+0000 7fa4d700a640 1 -- 172.21.15.77:0/1650535085 wait complete. 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 Processor -- start 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 -- start start 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 0x7fa4d01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d0103700 0x7fa4d01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0111fb0 con 0x7fa4d010c180 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0111e30 con 0x7fa4d0102de0 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d700a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0112130 con 0x7fa4d0103700 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47380/0 (socket says 172.21.15.77:47380) 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 -- 172.21.15.77:0/2543524545 learned_addr learned my addr 172.21.15.77:0/2543524545 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 0x7fa4d01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d0103700 0x7fa4d01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 -- 172.21.15.77:0/2543524545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d0103700 msgr2=0x7fa4d01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d0103700 0x7fa4d01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 -- 172.21.15.77:0/2543524545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 msgr2=0x7fa4d01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 0x7fa4d01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 -- 172.21.15.77:0/2543524545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d019d460 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.841+0000 7fa4d55a7640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800b970 tx=0x7fa4c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.842+0000 7fa4cdffb640 1 -- 172.21.15.77:0/2543524545 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8019070 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.842+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d019d750 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.842+0000 7fa4cdffb640 1 -- 172.21.15.77:0/2543524545 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c800cb40 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.842+0000 7fa4cdffb640 1 -- 172.21.15.77:0/2543524545 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8007650 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.842+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d01a5030 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4cdffb640 1 -- 172.21.15.77:0/2543524545 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa4c801c070 con 0x7fa4d010c180 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 Processor -- start 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 -- start start 2024-09-15T22:17:48.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a79370 0x55d8a0a7b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8a09d42f0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8a097bcf0 con 0x55d8a0a79370 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.844+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8a09d02b0 con 0x55d8a0a6fb40 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4cdffb640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47384/0 (socket says 172.21.15.77:47384) 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2243891912 learned_addr learned my addr 172.21.15.77:0/2243891912 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a79370 0x55d8a0a7b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d55a7640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2243891912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 msgr2=0x55d8a0a6e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a6e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2243891912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a79370 msgr2=0x55d8a0a7b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a79370 0x55d8a0a7b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4cdffb640 1 -- 172.21.15.77:0/2543524545 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4c80cf540 con 0x7fa4d010c180 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2243891912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a09603a0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400bd10 tx=0x7fa4c40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67622f2320af9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4d55a7640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 0x7fa4c00a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80ccb10 tx=0x7fa4c806a390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4baffd640 1 -- 172.21.15.77:0/2243891912 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004030 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4baffd640 1 -- 172.21.15.77:0/2243891912 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40041d0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.845+0000 7fa4baffd640 1 -- 172.21.15.77:0/2243891912 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40117e0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2243891912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 msgr2=0x55d8a0a6ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400bd10 tx=0x7fa4c40107a0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2243891912 shutdown_connections 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a79370 0x55d8a0a7b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a6ea10 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2243891912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2243891912 >> 172.21.15.77:0/2243891912 conn(0x55d8a0a37310 msgr2=0x55d8a0a6ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2243891912 shutdown_connections 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2243891912 wait complete. 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.846+0000 7fa4d93e3740 1 -- start start 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 Processor -- start 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a76000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8a0a7c070 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8a098f060 con 0x55d8a0a6fb40 2024-09-15T22:17:48.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8a0a7c1f0 con 0x55d8a0a79370 2024-09-15T22:17:48.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d93e3740 10 client.0 _ll_register_callbacks cb 0x55d8a06e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:48.886 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:48.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47392/0 (socket says 172.21.15.77:47392) 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 -- 172.21.15.77:0/3396322544 learned_addr learned my addr 172.21.15.77:0/3396322544 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 -- 172.21.15.77:0/3396322544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 msgr2=0x55d8a0a765d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 -- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 msgr2=0x55d8a0a76000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a76000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a76000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.847+0000 7fa4cffff640 1 -- 172.21.15.77:0/3396322544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a07453c0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4cffff640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02a9a0 tx=0x7fa4bc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9e41110b0c2a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc035040 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4bc0035f0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c40041d0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa4bc0037f0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 10 client.16950 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8a0a87a60 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40041d0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa4bc03a040 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 10 client.16950 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 10 client.16950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 20 client.16950 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0a88f20 0x55d8a0a8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.848+0000 7fa4d93e3740 4 client.16950 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:48.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.849+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8a09e1860 con 0x55d8a0a88f20 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.849+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0a88f20 0x55d8a0a8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.849+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0a88f20 0x55d8a0a8b330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b52e7635261383 server_cookie=31935a8abbee0355 in_seq=0 out_seq=0 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.849+0000 7fa4affff640 10 client.16950 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4affff640 1 -- 172.21.15.77:0/3396322544 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8a09e1860 con 0x55d8a0a88f20 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4affff640 10 client.16950 handle_command_reply: tid=0 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 client.16950 shutdown 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 msgr2=0x55d8a0a78d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02a9a0 tx=0x7fa4bc005e80 comp rx=0 tx=0).stop 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 shutdown_connections 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0a88f20 0x55d8a0a8b330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4affff640 0 client.16950 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:48.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a76000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/3396322544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a78d50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 >> 172.21.15.77:0/3396322544 conn(0x55d8a0a37310 msgr2=0x55d8a0a70960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 shutdown_connections 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 1 -- 172.21.15.77:0/3396322544 wait complete. 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.850+0000 7fa4d93e3740 20 client.16950 trim_cache size 0 max 16384 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 Processor -- start 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 -- start start 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a71660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01aeee0 con 0x55d8a0a6fb40 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01b0a20 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01af5b0 con 0x55d8a0a79370 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47396/0 (socket says 172.21.15.77:47396) 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2298045982 learned_addr learned my addr 172.21.15.77:0/2298045982 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2298045982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 msgr2=0x55d8a0a71660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a71660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2298045982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 msgr2=0x55d8a0a69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a69c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4d4da6640 1 -- 172.21.15.77:0/2298045982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a0a8b9b0 con 0x55d8a0a6fb40 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.863+0000 7fa4cffff640 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b7f0 tx=0x7fa4c4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c43ef260f0443b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4ba7fc640 1 -- 172.21.15.77:0/2298045982 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4005240 con 0x55d8a0a6fb40 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4ba7fc640 1 -- 172.21.15.77:0/2298045982 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40053e0 con 0x55d8a0a6fb40 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4ba7fc640 1 -- 172.21.15.77:0/2298045982 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4027730 con 0x55d8a0a6fb40 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2298045982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 msgr2=0x55d8a0a71120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b7f0 tx=0x7fa4c4005c50 comp rx=0 tx=0).stop 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2298045982 shutdown_connections 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a79370 0x55d8a0a71660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a71120 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/2298045982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2298045982 >> 172.21.15.77:0/2298045982 conn(0x55d8a0a37310 msgr2=0x55d8a0a69500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2298045982 shutdown_connections 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.864+0000 7fa4d93e3740 1 -- 172.21.15.77:0/2298045982 wait complete. 2024-09-15T22:17:48.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 -- start start 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 Processor -- start 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a72e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01b0350 con 0x55d8a0a79370 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01afc80 con 0x55d8a0a6fb40 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01acec0 con 0x55d8a0a6e5d0 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4d93e3740 10 client.0 _ll_register_callbacks cb 0x7fa4bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47398/0 (socket says 172.21.15.77:47398) 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.865+0000 7fa4cffff640 1 -- 172.21.15.77:0/245843060 learned_addr learned my addr 172.21.15.77:0/245843060 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 -- 172.21.15.77:0/245843060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6e5d0 msgr2=0x55d8a0a81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 -- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 msgr2=0x55d8a0a72e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a72e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 -- 172.21.15.77:0/245843060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a0af2bd0 con 0x55d8a0a79370 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4cffff640 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02abb0 tx=0x7fa4bc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364d990067ba36a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc059060 con 0x55d8a0a79370 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa4bc004480 con 0x55d8a0a79370 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c40053e0 con 0x55d8a0a79370 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa4bc060050 con 0x55d8a0a79370 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 10 client.16956 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:48.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8a0af3d20 con 0x55d8a0a79370 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40053e0 con 0x55d8a0a79370 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa4bc054c50 con 0x55d8a0a79370 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 10 client.16956 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 10 client.16956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 20 client.16956 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0af4500 0x55d8a0af6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 4 client.16956 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.866+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8a09dbd70 con 0x55d8a0af4500 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0af4500 0x55d8a0af6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4affff640 10 client.16956 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d4da6640 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0af4500 0x55d8a0af6920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e097644be41958d5 server_cookie=26477831a32abd5f in_seq=0 out_seq=0 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4affff640 1 -- 172.21.15.77:0/245843060 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8a09dbd70 con 0x55d8a0af4500 2024-09-15T22:17:48.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4affff640 10 client.16956 handle_command_reply: tid=0 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 client.16956 shutdown 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 msgr2=0x55d8a0a73620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02abb0 tx=0x7fa4bc005bc0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 shutdown_connections 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d8a0af4500 0x55d8a0af6920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8a0a79370 0x55d8a0a73620 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8a0a6fb40 0x55d8a0a72e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4affff640 0 client.16956 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 --2- 172.21.15.77:0/245843060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a0a6e5d0 0x55d8a0a81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.867+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 >> 172.21.15.77:0/245843060 conn(0x55d8a0a37310 msgr2=0x55d8a0a69d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 shutdown_connections 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d93e3740 1 -- 172.21.15.77:0/245843060 wait complete. 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d93e3740 20 client.16956 trim_cache size 0 max 16384 2024-09-15T22:17:48.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 msgr2=0x7fa4c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 0x7fa4c00a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80ccb10 tx=0x7fa4c806a390 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 msgr2=0x7fa4d019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800b970 tx=0x7fa4c800be30 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 shutdown_connections 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4c00a33b0 0x7fa4c00a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4d010c180 0x7fa4d019cd80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4d0103700 0x7fa4d01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 --2- 172.21.15.77:0/2543524545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4d0102de0 0x7fa4d01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 >> 172.21.15.77:0/2543524545 conn(0x7fa4d00fc7a0 msgr2=0x7fa4d00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 shutdown_connections 2024-09-15T22:17:48.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:48.868+0000 7fa4d700a640 1 -- 172.21.15.77:0/2543524545 wait complete. 2024-09-15T22:17:49.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.11 MB/sec execute 118 sec latency 14425.801 ms 2024-09-15T22:17:49.783 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:17:50.030 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:50.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 8.04 MB/sec execute 119 sec latency 15425.871 ms 2024-09-15T22:17:50.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:50 smithi077 ceph-mon[30980]: pgmap v165: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-09-15T22:17:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:50 smithi196 ceph-mon[35109]: pgmap v165: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-09-15T22:17:50.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 -- 172.21.15.77:0/2131127667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 msgr2=0x7f61701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 --2- 172.21.15.77:0/2131127667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f61701031b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6160009930 tx=0x7f616002a440 comp rx=0 tx=0).stop 2024-09-15T22:17:50.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 -- 172.21.15.77:0/2131127667 shutdown_connections 2024-09-15T22:17:50.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 --2- 172.21.15.77:0/2131127667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170108300 0x7f617010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 --2- 172.21.15.77:0/2131127667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170103780 0x7f6170107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 --2- 172.21.15.77:0/2131127667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f61701031b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 -- 172.21.15.77:0/2131127667 >> 172.21.15.77:0/2131127667 conn(0x7f61700fc7d0 msgr2=0x7f61700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 -- 172.21.15.77:0/2131127667 shutdown_connections 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 -- 172.21.15.77:0/2131127667 wait complete. 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.576+0000 7f6175b67640 1 Processor -- start 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 -- start start 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170103780 0x7f6170198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170108300 0x7f617019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61701120f0 con 0x7f6170102dd0 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6170111f70 con 0x7f6170103780 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f6175b67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112270 con 0x7f6170108300 2024-09-15T22:17:50.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39288/0 (socket says 172.21.15.77:39288) 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 -- 172.21.15.77:0/4093600471 learned_addr learned my addr 172.21.15.77:0/4093600471 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616effd640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170103780 0x7f6170198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 -- 172.21.15.77:0/4093600471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170108300 msgr2=0x7f617019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616ffff640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170108300 0x7f617019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170108300 0x7f617019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 -- 172.21.15.77:0/4093600471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170103780 msgr2=0x7f6170198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170103780 0x7f6170198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 -- 172.21.15.77:0/4093600471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617019d5e0 con 0x7f6170102dd0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616f7fe640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6160009230 tx=0x7f6160004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.577+0000 7f616cff9640 1 -- 172.21.15.77:0/4093600471 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616003f030 con 0x7f6170102dd0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.578+0000 7f616cff9640 1 -- 172.21.15.77:0/4093600471 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6160003ad0 con 0x7f6170102dd0 2024-09-15T22:17:50.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.578+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617019d870 con 0x7f6170102dd0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.578+0000 7f616cff9640 1 -- 172.21.15.77:0/4093600471 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160035070 con 0x7f6170102dd0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.578+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a51b0 con 0x7f6170102dd0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f616cff9640 1 -- 172.21.15.77:0/4093600471 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6160042030 con 0x7f6170102dd0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 Processor -- start 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 -- start start 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 0x555e4a4baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e4a4202f0 con 0x555e4a4bbb40 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e4a3c7cf0 con 0x555e4a4ba5d0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.580+0000 7f6177f40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e4a41c2b0 con 0x555e4a4c5370 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616cff9640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 0x7f61580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39304/0 (socket says 172.21.15.77:39304) 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 -- 172.21.15.77:0/1421115850 learned_addr learned my addr 172.21.15.77:0/1421115850 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 0x7f61580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616effd640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616cff9640 1 -- 172.21.15.77:0/4093600471 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61600f0410 con 0x7f6170102dd0 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616f7fe640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 0x555e4a4baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 -- 172.21.15.77:0/1421115850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 msgr2=0x555e4a4c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 -- 172.21.15.77:0/1421115850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 msgr2=0x555e4a4baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 0x555e4a4baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 -- 172.21.15.77:0/1421115850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4a3ac3a0 con 0x555e4a4bbb40 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616f7fe640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 0x555e4a4baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:50.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f616ffff640 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6168015840 tx=0x7f6168015d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8e3e69f50efaab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f6155ffb640 1 -- 172.21.15.77:0/1421115850 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168022070 con 0x555e4a4bbb40 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f6155ffb640 1 -- 172.21.15.77:0/1421115850 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6168009cc0 con 0x555e4a4bbb40 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.581+0000 7f6155ffb640 1 -- 172.21.15.77:0/1421115850 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616801c9a0 con 0x555e4a4bbb40 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- 172.21.15.77:0/1421115850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 msgr2=0x555e4a4ba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6168015840 tx=0x7f6168015d00 comp rx=0 tx=0).stop 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f616ffff640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 0x7f61580a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f616801f000 tx=0x7f616801c000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- 172.21.15.77:0/1421115850 shutdown_connections 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4ba5d0 0x555e4a4baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 --2- 172.21.15.77:0/1421115850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4ba090 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- 172.21.15.77:0/1421115850 >> 172.21.15.77:0/1421115850 conn(0x555e4a483310 msgr2=0x555e4a4b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- 172.21.15.77:0/1421115850 shutdown_connections 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- 172.21.15.77:0/1421115850 wait complete. 2024-09-15T22:17:50.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.582+0000 7f6177f40740 1 -- start start 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 Processor -- start 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4c5370 0x555e4a4c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e4a4c8070 con 0x555e4a4bbb40 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e4a3db060 con 0x555e4a4c5370 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e4a4c81f0 con 0x555e4a4ba5d0 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f6177f40740 10 client.0 _ll_register_callbacks cb 0x555e4a12e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39318/0 (socket says 172.21.15.77:39318) 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 -- 172.21.15.77:0/2490362245 learned_addr learned my addr 172.21.15.77:0/2490362245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616ffff640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4c5370 0x555e4a4c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 -- 172.21.15.77:0/2490362245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4ba5d0 msgr2=0x555e4a4c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 -- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4c5370 msgr2=0x555e4a4c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4c5370 0x555e4a4c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.583+0000 7f616effd640 1 -- 172.21.15.77:0/2490362245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4a1913c0 con 0x555e4a4bbb40 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f616effd640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f616400ab60 tx=0x7f6164011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285f0b234e2ccdd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616401a070 con 0x555e4a4bbb40 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6164016650 con 0x555e4a4bbb40 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6168009cc0 con 0x555e4a4bbb40 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6164026050 con 0x555e4a4bbb40 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 10 client.16968 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e4a4d3b30 con 0x555e4a4bbb40 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6168009cc0 con 0x555e4a4bbb40 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f616401e730 con 0x555e4a4bbb40 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 10 client.16968 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 20 client.16968 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a4d7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 4 client.16968 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.584+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e4a42d860 con 0x555e4a4d4fe0 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.585+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a4d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.585+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a4d7400 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de491017245e21cb server_cookie=dea6abeea62c74f6 in_seq=0 out_seq=0 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.585+0000 7f614effd640 10 client.16968 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f614effd640 1 -- 172.21.15.77:0/2490362245 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e4a42d860 con 0x555e4a4d4fe0 2024-09-15T22:17:50.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f614effd640 10 client.16968 handle_command_reply: tid=0 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 client.16968 shutdown 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 msgr2=0x555e4a4c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f616400ab60 tx=0x7f6164011580 comp rx=0 tx=0).stop 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 shutdown_connections 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a4d7400 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f614effd640 0 client.16968 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4c5370 0x555e4a4c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4bbb40 0x555e4a4c1fe0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 --2- 172.21.15.77:0/2490362245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 >> 172.21.15.77:0/2490362245 conn(0x555e4a483310 msgr2=0x555e4a4bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 shutdown_connections 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 1 -- 172.21.15.77:0/2490362245 wait complete. 2024-09-15T22:17:50.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.586+0000 7f6177f40740 20 client.16968 trim_cache size 0 max 16384 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 Processor -- start 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 -- start start 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61701aef40 con 0x555e4a4ba5d0 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b0a80 con 0x555e4a4bbb40 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f6177f40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61701af610 con 0x555e4a4c5370 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39334/0 (socket says 172.21.15.77:39334) 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 -- 172.21.15.77:0/2156848033 learned_addr learned my addr 172.21.15.77:0/2156848033 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616effd640 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 -- 172.21.15.77:0/2156848033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 msgr2=0x555e4a4dccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 -- 172.21.15.77:0/2156848033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 msgr2=0x555e4a4c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.599+0000 7f616f7fe640 1 -- 172.21.15.77:0/2156848033 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e4a4b64f0 con 0x555e4a4ba5d0 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f61600bc150 tx=0x7f61600b8730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17453a6d271667d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f61557fa640 1 -- 172.21.15.77:0/2156848033 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160049040 con 0x555e4a4ba5d0 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f61557fa640 1 -- 172.21.15.77:0/2156848033 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61600c0800 con 0x555e4a4ba5d0 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f61557fa640 1 -- 172.21.15.77:0/2156848033 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61600b8ac0 con 0x555e4a4ba5d0 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 -- 172.21.15.77:0/2156848033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 msgr2=0x555e4a4d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f61600bc150 tx=0x7f61600b8730 comp rx=0 tx=0).stop 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 -- 172.21.15.77:0/2156848033 shutdown_connections 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 --2- 172.21.15.77:0/2156848033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4d1310 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 -- 172.21.15.77:0/2156848033 >> 172.21.15.77:0/2156848033 conn(0x555e4a483310 msgr2=0x555e4a4c08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 -- 172.21.15.77:0/2156848033 shutdown_connections 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.600+0000 7f6177f40740 1 -- 172.21.15.77:0/2156848033 wait complete. 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 -- start start 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 Processor -- start 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b03b0 con 0x555e4a4ba5d0 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61701afce0 con 0x555e4a4bbb40 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61701acf20 con 0x555e4a4c5370 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f6177f40740 10 client.0 _ll_register_callbacks cb 0x7f6164009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39336/0 (socket says 172.21.15.77:39336) 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f616f7fe640 1 -- 172.21.15.77:0/2983881324 learned_addr learned my addr 172.21.15.77:0/2983881324 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.601+0000 7f616ffff640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616effd640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 -- 172.21.15.77:0/2983881324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 msgr2=0x555e4a4c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 -- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 msgr2=0x555e4a4c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 -- 172.21.15.77:0/2983881324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e4a53ed50 con 0x555e4a4ba5d0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616effd640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f616f7fe640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6160005e90 tx=0x7f61600bcbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fdeea6ce877af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61600d4030 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f61600ba750 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61600c0800 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6160049070 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 10 client.16974 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e4a540010 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61600c0800 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f61600d5270 con 0x555e4a4ba5d0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 10 client.16974 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.602+0000 7f6177f40740 20 client.16974 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a542630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f6177f40740 4 client.16974 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e4a427d70 con 0x555e4a4d4fe0 2024-09-15T22:17:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f616effd640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a542630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f614effd640 10 client.16974 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f616effd640 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a542630 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a949d8403903fd56 server_cookie=65e1c4417ab1f02f in_seq=0 out_seq=0 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f614effd640 1 -- 172.21.15.77:0/2983881324 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e4a427d70 con 0x555e4a4d4fe0 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f614effd640 10 client.16974 handle_command_reply: tid=0 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.603+0000 7f6177f40740 1 client.16974 shutdown 2024-09-15T22:17:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 msgr2=0x555e4a4c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6160005e90 tx=0x7f61600bcbd0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 shutdown_connections 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555e4a4d4fe0 0x555e4a542630 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e4a4c5370 0x555e4a4c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f614effd640 0 client.16974 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e4a4bbb40 0x555e4a4c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 --2- 172.21.15.77:0/2983881324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e4a4ba5d0 0x555e4a4c92c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 >> 172.21.15.77:0/2983881324 conn(0x555e4a483310 msgr2=0x555e4a4c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 shutdown_connections 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 1 -- 172.21.15.77:0/2983881324 wait complete. 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6177f40740 20 client.16974 trim_cache size 0 max 16384 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 msgr2=0x7f61580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 0x7f61580a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f616801f000 tx=0x7f616801c000 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 msgr2=0x7f6170198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6160009230 tx=0x7f6160004270 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 shutdown_connections 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61580a3430 0x7f61580a58f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6170108300 0x7f617019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6170103780 0x7f6170198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 --2- 172.21.15.77:0/4093600471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6170102dd0 0x7f6170198630 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 >> 172.21.15.77:0/4093600471 conn(0x7f61700fc7d0 msgr2=0x7f6170105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 shutdown_connections 2024-09-15T22:17:50.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:50.604+0000 7f6175b67640 1 -- 172.21.15.77:0/4093600471 wait complete. 2024-09-15T22:17:50.626 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:50 smithi117 ceph-mon[35021]: pgmap v165: 129 pgs: 129 active+clean; 243 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 31 op/s 2024-09-15T22:17:51.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.97 MB/sec execute 120 sec latency 16425.903 ms 2024-09-15T22:17:52.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:17:52.420 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:17:52.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.91 MB/sec execute 121 sec latency 17425.969 ms 2024-09-15T22:17:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:52 smithi077 ceph-mon[30980]: pgmap v166: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2024-09-15T22:17:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:52 smithi196 ceph-mon[35109]: pgmap v166: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2024-09-15T22:17:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:52 smithi117 ceph-mon[35021]: pgmap v166: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2024-09-15T22:17:52.680 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:53.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 -- 172.21.15.77:0/2954021786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac102790 msgr2=0x7f4fac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2954021786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac102790 0x7f4fac1067e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009900 tx=0x7f4f9402a410 comp rx=0 tx=0).stop 2024-09-15T22:17:53.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 -- 172.21.15.77:0/2954021786 shutdown_connections 2024-09-15T22:17:53.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2954021786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fac106f10 0x7f4fac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2954021786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac102790 0x7f4fac1067e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2954021786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac101de0 0x7f4fac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 -- 172.21.15.77:0/2954021786 >> 172.21.15.77:0/2954021786 conn(0x7f4fac0fd8c0 msgr2=0x7f4fac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 -- 172.21.15.77:0/2954021786 shutdown_connections 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.166+0000 7f4fb178f640 1 -- 172.21.15.77:0/2954021786 wait complete. 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 Processor -- start 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 -- start start 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac102790 0x7f4fac198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fac106f10 0x7f4fac19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac111f90 con 0x7f4fac101de0 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac111e10 con 0x7f4fac102790 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fb178f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac112110 con 0x7f4fac106f10 2024-09-15T22:17:53.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39362/0 (socket says 172.21.15.77:39362) 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 -- 172.21.15.77:0/2822681786 learned_addr learned my addr 172.21.15.77:0/2822681786 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac102790 0x7f4fac198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 -- 172.21.15.77:0/2822681786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fac106f10 msgr2=0x7f4fac19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fac106f10 0x7f4fac19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 -- 172.21.15.77:0/2822681786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac102790 msgr2=0x7f4fac198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac102790 0x7f4fac198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.167+0000 7f4faaffd640 1 -- 172.21.15.77:0/2822681786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac19d520 con 0x7f4fac101de0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4faaffd640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d950 tx=0x7f4f9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4fa8ff9640 1 -- 172.21.15.77:0/2822681786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c017070 con 0x7f4fac101de0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4fa8ff9640 1 -- 172.21.15.77:0/2822681786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c009c30 con 0x7f4fac101de0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4fa8ff9640 1 -- 172.21.15.77:0/2822681786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00f6b0 con 0x7f4fac101de0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fac19d810 con 0x7f4fac101de0 2024-09-15T22:17:53.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.168+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fac1a50f0 con 0x7f4fac101de0 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.170+0000 7f4fa8ff9640 1 -- 172.21.15.77:0/2822681786 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f9c0109d0 con 0x7f4fac101de0 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.170+0000 7f4fb3b68740 1 Processor -- start 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 -- start start 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fa8ff9640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 0x7f4f980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 0x7f4f980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fa8ff9640 1 -- 172.21.15.77:0/2822681786 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f9c0cfd10 con 0x7f4fac101de0 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3de2360 0x5638b3df0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3df1150 0x5638b3df3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638b3d47280 con 0x5638b3de1a40 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5638b3ceeb10 con 0x5638b3df1150 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638b3d43240 con 0x5638b3de2360 2024-09-15T22:17:53.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39374/0 (socket says 172.21.15.77:39374) 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 -- 172.21.15.77:0/1435485528 learned_addr learned my addr 172.21.15.77:0/1435485528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 0x7f4f980a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400aba0 tx=0x7f4fa4009250 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 -- 172.21.15.77:0/1435485528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3de2360 msgr2=0x5638b3df0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3de2360 0x5638b3df0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 -- 172.21.15.77:0/1435485528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3df1150 msgr2=0x5638b3df3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3df1150 0x5638b3df3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.171+0000 7f4faaffd640 1 -- 172.21.15.77:0/1435485528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b3cfb1f0 con 0x5638b3de1a40 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4faaffd640 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0a43d0 tx=0x7f4f9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e3e56fc6495f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4fa17fa640 1 -- 172.21.15.77:0/1435485528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c020040 con 0x5638b3de1a40 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4fa17fa640 1 -- 172.21.15.77:0/1435485528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c0a0040 con 0x5638b3de1a40 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4fa17fa640 1 -- 172.21.15.77:0/1435485528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c09b340 con 0x5638b3de1a40 2024-09-15T22:17:53.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4fb3b68740 1 -- 172.21.15.77:0/1435485528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 msgr2=0x5638b3de1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.172+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0a43d0 tx=0x7f4f9c005f90 comp rx=0 tx=0).stop 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- 172.21.15.77:0/1435485528 shutdown_connections 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3df1150 0x5638b3df3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3de2360 0x5638b3df0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/1435485528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3de1e20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- 172.21.15.77:0/1435485528 >> 172.21.15.77:0/1435485528 conn(0x5638b3dde0f0 msgr2=0x5638b3dde4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- 172.21.15.77:0/1435485528 shutdown_connections 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- 172.21.15.77:0/1435485528 wait complete. 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- start start 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 Processor -- start 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 0x5638b3dee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3deebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638b3df3cc0 con 0x5638b3de2360 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5638b3d01f10 con 0x5638b3de1a40 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5638b3df3e40 con 0x5638b3df1150 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fb3b68740 10 client.0 _ll_register_callbacks cb 0x5638b3ccaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.173+0000 7f4fa3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39388/0 (socket says 172.21.15.77:39388) 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 -- 172.21.15.77:0/718750268 learned_addr learned my addr 172.21.15.77:0/718750268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3deebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4faaffd640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 0x5638b3dee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 -- 172.21.15.77:0/718750268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 msgr2=0x5638b3deebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3deebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 -- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 msgr2=0x5638b3dee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 0x5638b3dee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 -- 172.21.15.77:0/718750268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b3ab86a0 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4faaffd640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 0x5638b3dee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009900 tx=0x7f4f94031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bbd5c3d4b1249a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94040070 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f94004240 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c0a0040 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f94035040 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fb3b68740 10 client.16986 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5638b3dffa10 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.174+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c0a0040 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f9403c6a0 con 0x5638b3de2360 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 10 client.16986 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 20 client.16986 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e00ed0 0x5638b3e03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 4 client.16986 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b3d547f0 con 0x5638b3e00ed0 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4faaffd640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e00ed0 0x5638b3e03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4faaffd640 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e00ed0 0x5638b3e03320 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5788245bfa638d server_cookie=e5d4932b36c54950 in_seq=0 out_seq=0 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.175+0000 7f4f927fc640 10 client.16986 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4f927fc640 1 -- 172.21.15.77:0/718750268 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638b3d547f0 con 0x5638b3e00ed0 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4f927fc640 10 client.16986 handle_command_reply: tid=0 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 client.16986 shutdown 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 msgr2=0x5638b3dee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009900 tx=0x7f4f94031850 comp rx=0 tx=0).stop 2024-09-15T22:17:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 shutdown_connections 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e00ed0 0x5638b3e03320 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3deebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de2360 0x5638b3dee680 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4f927fc640 0 client.16986 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/718750268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de1a40 0x5638b3dee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 >> 172.21.15.77:0/718750268 conn(0x5638b3dde0f0 msgr2=0x5638b3de3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 shutdown_connections 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 1 -- 172.21.15.77:0/718750268 wait complete. 2024-09-15T22:17:53.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.176+0000 7f4fb3b68740 20 client.16986 trim_cache size 0 max 16384 2024-09-15T22:17:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 Processor -- start 2024-09-15T22:17:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 -- start start 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3e08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3e09170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1acd50 con 0x5638b3de1a40 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1ae890 con 0x5638b3de2360 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.194+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1ad420 con 0x5638b3df1150 2024-09-15T22:17:53.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39400/0 (socket says 172.21.15.77:39400) 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 -- 172.21.15.77:0/15112482 learned_addr learned my addr 172.21.15.77:0/15112482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3e08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3e09170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 -- 172.21.15.77:0/15112482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 msgr2=0x5638b3e09170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3e09170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 -- 172.21.15.77:0/15112482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 msgr2=0x5638b3e08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3e08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 -- 172.21.15.77:0/15112482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b3dddb90 con 0x5638b3de1a40 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3e09170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4faaffd640 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009610 tx=0x7f4f9c084f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dff1900b58a244e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fa0ff9640 1 -- 172.21.15.77:0/15112482 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c020040 con 0x5638b3de1a40 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fa0ff9640 1 -- 172.21.15.77:0/15112482 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c099f90 con 0x5638b3de1a40 2024-09-15T22:17:53.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.195+0000 7f4fa0ff9640 1 -- 172.21.15.77:0/15112482 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c09adb0 con 0x5638b3de1a40 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- 172.21.15.77:0/15112482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 msgr2=0x5638b3deffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009610 tx=0x7f4f9c084f90 comp rx=0 tx=0).stop 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- 172.21.15.77:0/15112482 shutdown_connections 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3e09170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3e08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/15112482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3deffa0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- 172.21.15.77:0/15112482 >> 172.21.15.77:0/15112482 conn(0x5638b3dde0f0 msgr2=0x5638b3def890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- 172.21.15.77:0/15112482 shutdown_connections 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- 172.21.15.77:0/15112482 wait complete. 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.196+0000 7f4fb3b68740 1 -- start start 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 Processor -- start 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3df4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3df5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1ae1c0 con 0x5638b3de1a40 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1adaf0 con 0x5638b3de2360 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1aad30 con 0x5638b3df1150 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39416/0 (socket says 172.21.15.77:39416) 2024-09-15T22:17:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 -- 172.21.15.77:0/16981133 learned_addr learned my addr 172.21.15.77:0/16981133 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fb3b68740 10 client.0 _ll_register_callbacks cb 0x7f4f94005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3df4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3df5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 -- 172.21.15.77:0/16981133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 msgr2=0x5638b3df5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3df5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 -- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 msgr2=0x5638b3df4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3df4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 -- 172.21.15.77:0/16981133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b3df5850 con 0x5638b3de1a40 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3df4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4fab7fe640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3df5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4faaffd640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0058c0 tx=0x7f4f9c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93eee6a2033e36f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.197+0000 7f4f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c09b390 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f9c098300 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c099f90 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c0cd380 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 10 client.16992 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5638b3e6bb60 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c099f90 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f9c086280 con 0x5638b3de1a40 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 10 client.16992 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 20 client.16992 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e6bdd0 0x5638b3e6e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:53.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 4 client.16992 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638b3d4ed00 con 0x5638b3e6bdd0 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.198+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e6bdd0 0x5638b3e6e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.199+0000 7f4f927fc640 10 client.16992 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.199+0000 7f4fa3fff640 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e6bdd0 0x5638b3e6e1c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e09a120fc681d3 server_cookie=b3e51d2dbe2bbf04 in_seq=0 out_seq=0 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4f927fc640 1 -- 172.21.15.77:0/16981133 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638b3d4ed00 con 0x5638b3e6bdd0 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4f927fc640 10 client.16992 handle_command_reply: tid=0 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 client.16992 shutdown 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 msgr2=0x5638b3df97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0058c0 tx=0x7f4f9c00bf60 comp rx=0 tx=0).stop 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 shutdown_connections 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5638b3e6bdd0 0x5638b3e6e1c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5638b3df1150 0x5638b3df5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5638b3de2360 0x5638b3df4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4f927fc640 0 client.16992 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 --2- 172.21.15.77:0/16981133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638b3de1a40 0x5638b3df97e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 >> 172.21.15.77:0/16981133 conn(0x5638b3dde0f0 msgr2=0x5638b3ddd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 shutdown_connections 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 1 -- 172.21.15.77:0/16981133 wait complete. 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.200+0000 7f4fb3b68740 20 client.16992 trim_cache size 0 max 16384 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "84984d2f-b6d0-4e15-852a-46de1d029d02", 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:17:53.207 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 msgr2=0x7f4f980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 0x7f4f980a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400aba0 tx=0x7f4fa4009250 comp rx=0 tx=0).stop 2024-09-15T22:17:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 msgr2=0x7f4fac198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d950 tx=0x7f4f9c00de10 comp rx=0 tx=0).stop 2024-09-15T22:17:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 shutdown_connections 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f980a33b0 0x7f4f980a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fac106f10 0x7f4fac19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fac102790 0x7f4fac198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 --2- 172.21.15.77:0/2822681786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fac101de0 0x7f4fac198570 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 >> 172.21.15.77:0/2822681786 conn(0x7f4fac0fd8c0 msgr2=0x7f4fac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 shutdown_connections 2024-09-15T22:17:53.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:53.201+0000 7f4fb178f640 1 -- 172.21.15.77:0/2822681786 wait complete. 2024-09-15T22:17:53.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.84 MB/sec execute 122 sec latency 18426.003 ms 2024-09-15T22:17:54.111 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:17:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: pgmap v167: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:17:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:17:54.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:17:54.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.365 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:54.470 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.78 MB/sec execute 123 sec latency 19426.068 ms 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: pgmap v167: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: pgmap v167: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:17:54.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:17:54.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:17:54.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 -- 172.21.15.77:0/2030703542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58106a80 msgr2=0x7f8e58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/2030703542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58106a80 0x7f8e58106e60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009930 tx=0x7f8e4802a420 comp rx=0 tx=0).stop 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 -- 172.21.15.77:0/2030703542 shutdown_connections 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/2030703542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58101420 0x7f8e5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/2030703542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/2030703542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58106a80 0x7f8e58106e60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 -- 172.21.15.77:0/2030703542 >> 172.21.15.77:0/2030703542 conn(0x7f8e580fc7a0 msgr2=0x7f8e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 -- 172.21.15.77:0/2030703542 shutdown_connections 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 -- 172.21.15.77:0/2030703542 wait complete. 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.845+0000 7f8e5cf25640 1 Processor -- start 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 -- start start 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58106a80 0x7f8e5819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58110e80 con 0x7f8e58101420 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58110d00 con 0x7f8e58100a80 2024-09-15T22:17:54.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58111000 con 0x7f8e58106a80 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39442/0 (socket says 172.21.15.77:39442) 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 -- 172.21.15.77:0/1865495911 learned_addr learned my addr 172.21.15.77:0/1865495911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5659c640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58106a80 0x7f8e5819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 -- 172.21.15.77:0/1865495911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58106a80 msgr2=0x7f8e5819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58106a80 0x7f8e5819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 -- 172.21.15.77:0/1865495911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 msgr2=0x7f8e58197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 -- 172.21.15.77:0/1865495911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5819c3d0 con 0x7f8e58101420 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5659c640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:54.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4f7fe640 1 -- 172.21.15.77:0/1865495911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40017070 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5819c6c0 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.846+0000 7f8e4f7fe640 1 -- 172.21.15.77:0/1865495911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40009c30 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.847+0000 7f8e4f7fe640 1 -- 172.21.15.77:0/1865495911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000f6b0 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.847+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a3fa0 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.849+0000 7f8e4f7fe640 1 -- 172.21.15.77:0/1865495911 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8e400109d0 con 0x7f8e58101420 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 Processor -- start 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 -- start start 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x55698914ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569890b02e0 con 0x556989157890 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556989057ce0 con 0x55698914c6b0 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5569890ac2a0 con 0x55698914cfd0 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39444/0 (socket says 172.21.15.77:39444) 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e4ffff640 1 -- 172.21.15.77:0/2239842629 learned_addr learned my addr 172.21.15.77:0/2239842629 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e4f7fe640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 0x7f8e440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.850+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x55698914ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e5659c640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 0x7f8e440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4f7fe640 1 -- 172.21.15.77:0/1865495911 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8e400cfd80 con 0x7f8e58101420 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 -- 172.21.15.77:0/2239842629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 msgr2=0x556989157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 -- 172.21.15.77:0/2239842629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 msgr2=0x55698914ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x55698914ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 -- 172.21.15.77:0/2239842629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55698903c440 con 0x556989157890 2024-09-15T22:17:54.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x55698914ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8e4009bcb0 tx=0x7f8e4000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1414f37986a5e3df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 0x7f8e440a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e50008fd0 tx=0x7f8e50004ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4cff9640 1 -- 172.21.15.77:0/2239842629 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40020070 con 0x556989157890 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4cff9640 1 -- 172.21.15.77:0/2239842629 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40009de0 con 0x556989157890 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.851+0000 7f8e4cff9640 1 -- 172.21.15.77:0/2239842629 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40098390 con 0x556989157890 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2239842629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 msgr2=0x556989159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8e4009bcb0 tx=0x7f8e4000bf60 comp rx=0 tx=0).stop 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2239842629 shutdown_connections 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x556989159c80 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2239842629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x55698914ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2239842629 >> 172.21.15.77:0/2239842629 conn(0x556989148c40 msgr2=0x556989149020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2239842629 shutdown_connections 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2239842629 wait complete. 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 -- start start 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.852+0000 7f8e5f2fe740 1 Processor -- start 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989154570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x556989154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55698915a400 con 0x55698914c6b0 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55698906b050 con 0x556989157890 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55698915a580 con 0x55698914cfd0 2024-09-15T22:17:54.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 10 client.0 _ll_register_callbacks cb 0x5569890ab160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39456/0 (socket says 172.21.15.77:39456) 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 -- 172.21.15.77:0/86021186 learned_addr learned my addr 172.21.15.77:0/86021186 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x556989154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 -- 172.21.15.77:0/86021186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 msgr2=0x556989154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 -- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 msgr2=0x556989154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x556989154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 -- 172.21.15.77:0/86021186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556988e213c0 con 0x55698914c6b0 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x556989154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:54.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5659c640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009930 tx=0x7f8e480319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b5ada19e613079 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48035070 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8e48004240 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e40009de0 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.853+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e48004460 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 10 client.17004 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556989165ea0 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40009de0 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8e48004c70 con 0x55698914c6b0 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 10 client.17004 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:54.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 20 client.17004 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556989167360 0x556989169790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 4 client.17004 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569890bd850 con 0x556989167360 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.854+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556989167360 0x556989169790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e3dffb640 10 client.17004 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556989167360 0x556989169790 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bae2145fb79768 server_cookie=937f51c049a585fc in_seq=0 out_seq=0 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e3dffb640 1 -- 172.21.15.77:0/86021186 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569890bd850 con 0x556989167360 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e3dffb640 10 client.17004 handle_command_reply: tid=0 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 client.17004 shutdown 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 msgr2=0x5569891572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009930 tx=0x7f8e480319a0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 shutdown_connections 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556989167360 0x556989169790 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x556989154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x556989154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/86021186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x5569891572c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e3dffb640 0 client.17004 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 >> 172.21.15.77:0/86021186 conn(0x556989148c40 msgr2=0x55698914e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 shutdown_connections 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.855+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/86021186 wait complete. 2024-09-15T22:17:54.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.856+0000 7f8e5f2fe740 20 client.17004 trim_cache size 0 max 16384 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.872+0000 7f8e5f2fe740 1 Processor -- start 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.872+0000 7f8e5f2fe740 1 -- start start 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x556989152e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698916f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581acd10 con 0x556989157890 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581ae850 con 0x55698914c6b0 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581ad3e0 con 0x55698914cfd0 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39462/0 (socket says 172.21.15.77:39462) 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 -- 172.21.15.77:0/301205400 learned_addr learned my addr 172.21.15.77:0/301205400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5659c640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x556989152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 -- 172.21.15.77:0/301205400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 msgr2=0x55698916f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698916f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698916f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 -- 172.21.15.77:0/301205400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 msgr2=0x556989152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x556989152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 -- 172.21.15.77:0/301205400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989153520 con 0x556989157890 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e5659c640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x556989152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e50012ef0 tx=0x7f8e50008670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab979f1031e492 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e3ffff640 1 -- 172.21.15.77:0/301205400 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50018070 con 0x556989157890 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e3ffff640 1 -- 172.21.15.77:0/301205400 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e50023070 con 0x556989157890 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.873+0000 7f8e3ffff640 1 -- 172.21.15.77:0/301205400 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5000fb20 con 0x556989157890 2024-09-15T22:17:54.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/301205400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 msgr2=0x55698916f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e50012ef0 tx=0x7f8e50008670 comp rx=0 tx=0).stop 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/301205400 shutdown_connections 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556989157890 0x55698916f5a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698916f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/301205400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55698914c6b0 0x556989152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/301205400 >> 172.21.15.77:0/301205400 conn(0x556989148c40 msgr2=0x55698914fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/301205400 shutdown_connections 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/301205400 wait complete. 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.874+0000 7f8e5f2fe740 1 -- start start 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 Processor -- start 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698915c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x5569891d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581ae180 con 0x55698914c6b0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581adab0 con 0x556989157890 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581aacf0 con 0x55698914cfd0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5f2fe740 10 client.0 _ll_register_callbacks cb 0x7f8e48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39464/0 (socket says 172.21.15.77:39464) 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 -- 172.21.15.77:0/2248561192 learned_addr learned my addr 172.21.15.77:0/2248561192 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e56d9d640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x5569891d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698915c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 -- 172.21.15.77:0/2248561192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 msgr2=0x55698915c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698915c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 -- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 msgr2=0x5569891d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x5569891d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 -- 172.21.15.77:0/2248561192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698914ce00 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e5659c640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005940 tx=0x7f8e48031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8857cd0e5787600 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.875+0000 7f8e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48056060 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f8e48003f50 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e50023070 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8e4805d050 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 10 client.17010 fetch_fsmap learned FSMap version 34 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569891d2670 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e50023070 con 0x55698914c6b0 2024-09-15T22:17:54.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8e48004930 con 0x55698914c6b0 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 10 client.17010 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 20 client.17010 populate_metadata read hostname 'smithi077' 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5569891d2d90 0x5569891d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 4 client.17010 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569890b7d60 con 0x5569891d2d90 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.876+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5569891d2d90 0x5569891d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e4ffff640 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5569891d2d90 0x5569891d51b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff1cdfa704cf34bd server_cookie=b46b23a67a3947e2 in_seq=0 out_seq=0 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e3dffb640 10 client.17010 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e3dffb640 1 -- 172.21.15.77:0/2248561192 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5569890b7d60 con 0x5569891d2d90 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e3dffb640 10 client.17010 handle_command_reply: tid=0 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 client.17010 shutdown 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 msgr2=0x55698915bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005940 tx=0x7f8e48031c40 comp rx=0 tx=0).stop 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 shutdown_connections 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5569891d2d90 0x5569891d51b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556989157890 0x5569891d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55698914cfd0 0x55698915c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e3dffb640 0 client.17010 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 --2- 172.21.15.77:0/2248561192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55698914c6b0 0x55698915bbd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 >> 172.21.15.77:0/2248561192 conn(0x556989148c40 msgr2=0x556989145270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 shutdown_connections 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 1 -- 172.21.15.77:0/2248561192 wait complete. 2024-09-15T22:17:54.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.877+0000 7f8e5f2fe740 20 client.17010 trim_cache size 0 max 16384 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "84984d2f-b6d0-4e15-852a-46de1d029d02": { 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "84984d2f-b6d0-4e15-852a-46de1d029d02", 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:17:54.889 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 msgr2=0x7f8e440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 0x7f8e440a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e50008fd0 tx=0x7f8e50004ab0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 msgr2=0x7f8e58197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000de10 comp rx=0 tx=0).stop 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 shutdown_connections 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e440a33b0 0x7f8e440a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e58106a80 0x7f8e5819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e58101420 0x7f8e58197960 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 --2- 172.21.15.77:0/1865495911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58100a80 0x7f8e58197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:54.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 >> 172.21.15.77:0/1865495911 conn(0x7f8e580fc7a0 msgr2=0x7f8e580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:54.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 shutdown_connections 2024-09-15T22:17:54.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:54.878+0000 7f8e5cf25640 1 -- 172.21.15.77:0/1865495911 wait complete. 2024-09-15T22:17:55.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.72 MB/sec execute 124 sec latency 20426.116 ms 2024-09-15T22:17:55.683 INFO:tasks.cephfs.filesystem:scrub status for tag:84984d2f-b6d0-4e15-852a-46de1d029d02 - {'path': '/', 'tag': '84984d2f-b6d0-4e15-852a-46de1d029d02', 'options': 'recursive,force'} 2024-09-15T22:17:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:56 smithi077 ceph-mon[30980]: pgmap v168: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:56.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.66 MB/sec execute 125 sec latency 21426.199 ms 2024-09-15T22:17:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:56 smithi196 ceph-mon[35109]: pgmap v168: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:56 smithi117 ceph-mon[35021]: pgmap v168: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 937 KiB/s wr, 40 op/s 2024-09-15T22:17:57.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.60 MB/sec execute 126 sec latency 22426.300 ms 2024-09-15T22:17:57.622 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:57.889 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:58.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.352+0000 7f6b330b0640 1 -- 172.21.15.77:0/2441674986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 msgr2=0x7f6b2c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:58.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.352+0000 7f6b330b0640 1 --2- 172.21.15.77:0/2441674986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c106e60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009930 tx=0x7f6b1c02a420 comp rx=0 tx=0).stop 2024-09-15T22:17:58.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- 172.21.15.77:0/2441674986 shutdown_connections 2024-09-15T22:17:58.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- 172.21.15.77:0/2441674986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 0x7f6b2c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- 172.21.15.77:0/2441674986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 0x7f6b2c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- 172.21.15.77:0/2441674986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c106e60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- 172.21.15.77:0/2441674986 >> 172.21.15.77:0/2441674986 conn(0x7f6b2c0fc7a0 msgr2=0x7f6b2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- 172.21.15.77:0/2441674986 shutdown_connections 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- 172.21.15.77:0/2441674986 wait complete. 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 Processor -- start 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- start start 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 0x7f6b2c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 0x7f6b2c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c110e80 con 0x7f6b2c106a80 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c110d00 con 0x7f6b2c100a80 2024-09-15T22:17:58.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.353+0000 7f6b330b0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c111000 con 0x7f6b2c101420 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39476/0 (socket says 172.21.15.77:39476) 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 -- 172.21.15.77:0/191506903 learned_addr learned my addr 172.21.15.77:0/191506903 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b30e4c640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 0x7f6b2c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b2bfff640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 0x7f6b2c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 -- 172.21.15.77:0/191506903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 msgr2=0x7f6b2c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 0x7f6b2c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 -- 172.21.15.77:0/191506903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 msgr2=0x7f6b2c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 0x7f6b2c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 -- 172.21.15.77:0/191506903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c19c270 con 0x7f6b2c106a80 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b3164d640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6b24007900 tx=0x7f6b24007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400e6f0 con 0x7f6b2c106a80 2024-09-15T22:17:58.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c19c560 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b2400ed70 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400ad10 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.354+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c06a790 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.355+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c100ee0 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.357+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6b2400e890 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.357+0000 7f6b29ffb640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 0x7f6b140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.357+0000 7f6b30e4c640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 0x7f6b140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.357+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6b240d19b0 con 0x7f6b2c106a80 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.358+0000 7f6b30e4c640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 0x7f6b140a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009900 tx=0x7f6b1c0058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:58.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.361+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b2409e050 con 0x7f6b2c106a80 2024-09-15T22:17:58.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.54 MB/sec execute 127 sec latency 23426.372 ms 2024-09-15T22:17:58.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.561+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b2c06b1b0 con 0x7f6b2c106a80 2024-09-15T22:17:58.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:58 smithi077 ceph-mon[30980]: pgmap v169: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 952 KiB/s wr, 45 op/s 2024-09-15T22:17:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.562+0000 7f6b29ffb640 1 -- 172.21.15.77:0/191506903 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6b2409d950 con 0x7f6b2c106a80 2024-09-15T22:17:58.564 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:17:58.564 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:17:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:17:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 msgr2=0x7f6b140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 0x7f6b140a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009900 tx=0x7f6b1c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 msgr2=0x7f6b2c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6b24007900 tx=0x7f6b24007dc0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 shutdown_connections 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b140a33b0 0x7f6b140a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b2c106a80 0x7f6b2c19bb90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b2c101420 0x7f6b2c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 --2- 172.21.15.77:0/191506903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c100a80 0x7f6b2c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 >> 172.21.15.77:0/191506903 conn(0x7f6b2c0fc7a0 msgr2=0x7f6b2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 shutdown_connections 2024-09-15T22:17:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:17:58.565+0000 7f6b330b0640 1 -- 172.21.15.77:0/191506903 wait complete. 2024-09-15T22:17:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:58 smithi196 ceph-mon[35109]: pgmap v169: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 952 KiB/s wr, 45 op/s 2024-09-15T22:17:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:58 smithi117 ceph-mon[35021]: pgmap v169: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 952 KiB/s wr, 45 op/s 2024-09-15T22:17:59.377 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id fbaeebd6 --release --await' 2024-09-15T22:17:59.377 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id fbaeebd6 --release --await 2024-09-15T22:17:59.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391186 7.48 MB/sec execute 128 sec latency 24426.426 ms 2024-09-15T22:17:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:17:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/191506903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:17:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/191506903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:17:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/191506903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:59.623 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/2404764364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a4106f50 msgr2=0x7fb5a410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/2404764364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a4106f50 0x7fb5a410eaf0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009cd0 tx=0x7fb59c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/2404764364 shutdown_connections 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/2404764364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a4106f50 0x7fb5a410eaf0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/2404764364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 0x7fb5a4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/2404764364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a41002f0 0x7fb5a41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/2404764364 >> 172.21.15.77:0/2404764364 conn(0x7fb5a4077210 msgr2=0x7fb5a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/2404764364 shutdown_connections 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.137+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/2404764364 wait complete. 2024-09-15T22:18:00.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 Processor -- start 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 -- start start 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 0x7fb5a4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 0x7fb5a419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112020 con 0x7fb5a41002f0 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4111ea0 con 0x7fb5a4106f50 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a8e5b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41121a0 con 0x7fb5a4100ca0 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54728/0 (socket says 172.21.15.77:54728) 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 -- 172.21.15.77:0/1662403409 learned_addr learned my addr 172.21.15.77:0/1662403409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb59bfff640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 0x7fb5a4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 0x7fb5a419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 -- 172.21.15.77:0/1662403409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 msgr2=0x7fb5a4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 0x7fb5a4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 -- 172.21.15.77:0/1662403409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 msgr2=0x7fb5a419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 0x7fb5a419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a259c640 1 -- 172.21.15.77:0/1662403409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419d490 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.138+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 0x7fb5a419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb5a259c640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb59400b4a0 tx=0x7fb59400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb59b7fe640 1 -- 172.21.15.77:0/1662403409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594014070 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb59b7fe640 1 -- 172.21.15.77:0/1662403409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594010af0 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419d780 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb59b7fe640 1 -- 172.21.15.77:0/1662403409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594017d10 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.139+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a5060 con 0x7fb5a41002f0 2024-09-15T22:18:00.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb59b7fe640 1 -- 172.21.15.77:0/1662403409 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb594017070 con 0x7fb5a41002f0 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 Processor -- start 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 -- start start 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596040c3620 con 0x55960415df40 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55960406ae50 con 0x55960415e860 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5ab234740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596040bf5e0 con 0x55960416d650 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54732/0 (socket says 172.21.15.77:54732) 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/730812528 learned_addr learned my addr 172.21.15.77:0/730812528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.142+0000 7fb59bfff640 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a259c640 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/730812528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 msgr2=0x55960416fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/730812528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 msgr2=0x55960416d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/730812528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960405bd40 con 0x55960415df40 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb59b7fe640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 0x7fb5900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005e30 tx=0x7fb59c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43545c2021884ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb5a259c640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 0x7fb5900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb59b7fe640 1 -- 172.21.15.77:0/1662403409 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb5940cf240 con 0x7fb5a41002f0 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb598ff9640 1 -- 172.21.15.77:0/730812528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0318c0 con 0x55960415df40 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb598ff9640 1 -- 172.21.15.77:0/730812528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c036040 con 0x55960415df40 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.143+0000 7fb598ff9640 1 -- 172.21.15.77:0/730812528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c004630 con 0x55960415df40 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- 172.21.15.77:0/730812528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 msgr2=0x55960415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5a259c640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 0x7fb5900a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5940cc810 tx=0x7fb59400bf90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005e30 tx=0x7fb59c009a20 comp rx=0 tx=0).stop 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- 172.21.15.77:0/730812528 shutdown_connections 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 --2- 172.21.15.77:0/730812528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960415e320 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- 172.21.15.77:0/730812528 >> 172.21.15.77:0/730812528 conn(0x55960415a5f0 msgr2=0x55960415a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- 172.21.15.77:0/730812528 shutdown_connections 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- 172.21.15.77:0/730812528 wait complete. 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 -- start start 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.144+0000 7fb5ab234740 1 Processor -- start 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559604170220 con 0x55960415df40 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55960404bf10 con 0x55960415e860 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596041703a0 con 0x55960416d650 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 10 client.0 _ll_register_callbacks cb 0x55960406fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54742/0 (socket says 172.21.15.77:54742) 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/322068450 learned_addr learned my addr 172.21.15.77:0/322068450 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb59bfff640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/322068450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 msgr2=0x55960416afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 msgr2=0x55960416aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/322068450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559603dd2580 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb59c02a920 tx=0x7fb59c004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe6fdbdec9efbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c053060 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb59c003b70 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59c036040 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb59c05a050 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 10 client.17028 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.145+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559603e9c960 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c036040 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb59c030070 con 0x55960415df40 2024-09-15T22:18:00.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 10 client.17028 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 10 client.17028 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 20 client.17028 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55960417d0a0 0x55960417f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 4 client.17028 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596040d0b90 con 0x55960417d0a0 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.146+0000 7fb59bfff640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55960417d0a0 0x55960417f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb589ffb640 10 client.17028 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb59bfff640 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55960417d0a0 0x55960417f490 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb3a93a5b899228 server_cookie=e1fc890f0d260547 in_seq=0 out_seq=0 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb589ffb640 1 -- 172.21.15.77:0/322068450 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596040d0b90 con 0x55960417d0a0 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb589ffb640 10 client.17028 handle_command_reply: tid=0 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 client.17028 shutdown 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 msgr2=0x55960416a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb59c02a920 tx=0x7fb59c004210 comp rx=0 tx=0).stop 2024-09-15T22:18:00.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 shutdown_connections 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55960417d0a0 0x55960417f490 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x55960416afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x55960416aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 --2- 172.21.15.77:0/322068450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x55960416a550 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 >> 172.21.15.77:0/322068450 conn(0x55960415a5f0 msgr2=0x559604160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb589ffb640 0 client.17028 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 shutdown_connections 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.147+0000 7fb5ab234740 1 -- 172.21.15.77:0/322068450 wait complete. 2024-09-15T22:18:00.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.148+0000 7fb5ab234740 20 client.17028 trim_cache size 0 max 16384 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 Processor -- start 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 -- start start 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 0x559604168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41acde0 con 0x55960415e860 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41ae920 con 0x55960415df40 2024-09-15T22:18:00.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.164+0000 7fb5ab234740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41ad4b0 con 0x55960416d650 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54754/0 (socket says 172.21.15.77:54754) 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 -- 172.21.15.77:0/376882581 learned_addr learned my addr 172.21.15.77:0/376882581 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 -- 172.21.15.77:0/376882581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 msgr2=0x5596041677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 -- 172.21.15.77:0/376882581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 msgr2=0x559604168e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 0x559604168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 0x559604168e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 -- 172.21.15.77:0/376882581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960416c490 con 0x55960415e860 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 0x559604168e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb59bfff640 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4104830 tx=0x7fb58c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9691471fdd66553d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb58bfff640 1 -- 172.21.15.77:0/376882581 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c003de0 con 0x55960415e860 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb58bfff640 1 -- 172.21.15.77:0/376882581 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c003f80 con 0x55960415e860 2024-09-15T22:18:00.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.165+0000 7fb58bfff640 1 -- 172.21.15.77:0/376882581 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0119a0 con 0x55960415e860 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- 172.21.15.77:0/376882581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 msgr2=0x559604159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4104830 tx=0x7fb58c00bd80 comp rx=0 tx=0).stop 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- 172.21.15.77:0/376882581 shutdown_connections 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415e860 0x559604159640 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- 172.21.15.77:0/376882581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415df40 0x559604168e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- 172.21.15.77:0/376882581 >> 172.21.15.77:0/376882581 conn(0x55960415a5f0 msgr2=0x559604158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- 172.21.15.77:0/376882581 shutdown_connections 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- 172.21.15.77:0/376882581 wait complete. 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 -- start start 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 Processor -- start 2024-09-15T22:18:00.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.166+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x559604180c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041e09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41ae250 con 0x55960415df40 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41adb80 con 0x55960415e860 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41aadc0 con 0x55960416d650 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 10 client.0 _ll_register_callbacks cb 0x7fb59c031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54758/0 (socket says 172.21.15.77:54758) 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/1148882419 learned_addr learned my addr 172.21.15.77:0/1148882419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb59bfff640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x559604180c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/1148882419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 msgr2=0x5596041e09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041e09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 msgr2=0x559604180c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x559604180c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 -- 172.21.15.77:0/1148882419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559604169500 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb59bfff640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x559604180c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5a2d9d640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0049d0 tx=0x7fb59c0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680b683de9a58ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c053070 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fb59c02e040 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58c003f80 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb59c065050 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 10 client.17034 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.167+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5596041e8150 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c003f80 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb59c045cb0 con 0x55960415df40 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 10 client.17034 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:00.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 20 client.17034 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5596041e8790 0x5596041eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 4 client.17034 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fbaeebd6", "release": true, "await": true}] 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fbaeebd6", "release": true, "await": true}) -- 0x5596040cb0a0 con 0x5596041e8790 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb59bfff640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5596041e8790 0x5596041eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb59bfff640 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5596041e8790 0x5596041eabb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d916ab725a4c95c4 server_cookie=d3932760f5b36e8d in_seq=0 out_seq=0 2024-09-15T22:18:00.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.168+0000 7fb589ffb640 10 client.17034 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:00.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.178+0000 7fb589ffb640 1 -- 172.21.15.77:0/1148882419 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5596040cb0a0 con 0x5596041e8790 2024-09-15T22:18:00.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.178+0000 7fb589ffb640 10 client.17034 handle_command_reply: tid=0 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.178+0000 7fb5ab234740 1 client.17034 shutdown 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 msgr2=0x5596041806f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0049d0 tx=0x7fb59c0098c0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 shutdown_connections 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5596041e8790 0x5596041eabb0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55960416d650 0x5596041e09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55960415e860 0x559604180c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 --2- 172.21.15.77:0/1148882419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55960415df40 0x5596041806f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 >> 172.21.15.77:0/1148882419 conn(0x55960415a5f0 msgr2=0x55960415fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb589ffb640 0 client.17034 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 shutdown_connections 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 1 -- 172.21.15.77:0/1148882419 wait complete. 2024-09-15T22:18:00.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5ab234740 20 client.17034 trim_cache size 0 max 16384 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 20, 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "fbaeebd6": { 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 20, 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:00.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 145.3, 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:00.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 msgr2=0x7fb5900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 0x7fb5900a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5940cc810 tx=0x7fb59400bf90 comp rx=0 tx=0).stop 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 msgr2=0x7fb5a41984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:00.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb59400b4a0 tx=0x7fb59400b960 comp rx=0 tx=0).stop 2024-09-15T22:18:00.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 shutdown_connections 2024-09-15T22:18:00.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5900a3430 0x7fb5900a58f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5a4106f50 0x7fb5a419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5a4100ca0 0x7fb5a4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.179+0000 7fb5a8e5b640 1 --2- 172.21.15.77:0/1662403409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5a41002f0 0x7fb5a41984e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.180+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 >> 172.21.15.77:0/1662403409 conn(0x7fb5a4077210 msgr2=0x7fb5a4105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.180+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 shutdown_connections 2024-09-15T22:18:00.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:00.180+0000 7fb5a8e5b640 1 -- 172.21.15.77:0/1662403409 wait complete. 2024-09-15T22:18:00.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 391781 7.45 MB/sec execute 129 sec latency 25133.961 ms 2024-09-15T22:18:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:00 smithi077 ceph-mon[30980]: pgmap v170: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-15T22:18:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:00 smithi196 ceph-mon[35109]: pgmap v170: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-15T22:18:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:00 smithi117 ceph-mon[35021]: pgmap v170: 129 pgs: 129 active+clean; 232 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-09-15T22:18:00.907 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fbaeebd6', total seconds elapsed: 28.6 2024-09-15T22:18:00.907 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-15T22:18:00.907 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:01.179 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:01.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 392661 7.41 MB/sec execute 130 sec latency 60.219 ms 2024-09-15T22:18:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:18:01.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:18:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:18:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:18:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:18:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:18:01.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 -- 172.21.15.77:0/223023114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac106f10 msgr2=0x7f83ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:01.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 --2- 172.21.15.77:0/223023114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac106f10 0x7f83ac10ea90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009900 tx=0x7f83a402a410 comp rx=0 tx=0).stop 2024-09-15T22:18:01.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 -- 172.21.15.77:0/223023114 shutdown_connections 2024-09-15T22:18:01.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 --2- 172.21.15.77:0/223023114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac106f10 0x7f83ac10ea90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 --2- 172.21.15.77:0/223023114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 0x7f83ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 --2- 172.21.15.77:0/223023114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac101de0 0x7f83ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 -- 172.21.15.77:0/223023114 >> 172.21.15.77:0/223023114 conn(0x7f83ac0fd8c0 msgr2=0x7f83ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:01.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 -- 172.21.15.77:0/223023114 shutdown_connections 2024-09-15T22:18:01.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 -- 172.21.15.77:0/223023114 wait complete. 2024-09-15T22:18:01.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.682+0000 7f83b3893640 1 Processor -- start 2024-09-15T22:18:01.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 -- start start 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 0x7f83ac1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac106f10 0x7f83ac19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111f70 con 0x7f83ac101de0 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111df0 con 0x7f83ac102790 2024-09-15T22:18:01.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b3893640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1120f0 con 0x7f83ac106f10 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54782/0 (socket says 172.21.15.77:54782) 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 -- 172.21.15.77:0/3848445419 learned_addr learned my addr 172.21.15.77:0/3848445419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b1e30640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac106f10 0x7f83ac19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b0e2e640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 0x7f83ac1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:01.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 -- 172.21.15.77:0/3848445419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac106f10 msgr2=0x7f83ac19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac106f10 0x7f83ac19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 -- 172.21.15.77:0/3848445419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 msgr2=0x7f83ac1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 0x7f83ac1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 -- 172.21.15.77:0/3848445419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac19d360 con 0x7f83ac101de0 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83b162f640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f839c00eab0 tx=0x7f839c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:01.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.683+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c00ce40 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.684+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839c011950 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.684+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac19d650 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.684+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c010ae0 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.684+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac1a4f30 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.685+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac1021c0 con 0x7f83ac101de0 2024-09-15T22:18:01.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.686+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f839c011b00 con 0x7f83ac101de0 2024-09-15T22:18:01.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.686+0000 7f83aa7fc640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 0x7f83940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:01.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.686+0000 7f83b0e2e640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 0x7f83940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:01.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.686+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f839c0d0620 con 0x7f83ac101de0 2024-09-15T22:18:01.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.687+0000 7f83b0e2e640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 0x7f83940a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83ac1088e0 tx=0x7f83a0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:01.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.691+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f839c016030 con 0x7f83ac101de0 2024-09-15T22:18:01.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.916+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83ac02d7c0 con 0x7f83ac101de0 2024-09-15T22:18:01.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.917+0000 7f83aa7fc640 1 -- 172.21.15.77:0/3848445419 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f839c09ccc0 con 0x7f83ac101de0 2024-09-15T22:18:01.919 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:18:01.919 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:18:01.919 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:18:01.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 msgr2=0x7f83940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:01.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 0x7f83940a58f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83ac1088e0 tx=0x7f83a0005f90 comp rx=0 tx=0).stop 2024-09-15T22:18:01.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 msgr2=0x7f83ac1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:01.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f839c00eab0 tx=0x7f839c00ef70 comp rx=0 tx=0).stop 2024-09-15T22:18:01.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 shutdown_connections 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83940a3430 0x7f83940a58f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ac106f10 0x7f83ac19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83ac102790 0x7f83ac1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 --2- 172.21.15.77:0/3848445419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83ac101de0 0x7f83ac1983b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.919+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 >> 172.21.15.77:0/3848445419 conn(0x7f83ac0fd8c0 msgr2=0x7f83ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.920+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 shutdown_connections 2024-09-15T22:18:01.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:01.920+0000 7f83b3893640 1 -- 172.21.15.77:0/3848445419 wait complete. 2024-09-15T22:18:02.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 394679 7.45 MB/sec execute 131 sec latency 19.865 ms 2024-09-15T22:18:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:02 smithi077 ceph-mon[30980]: pgmap v171: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-15T22:18:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:02 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3848445419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:02 smithi196 ceph-mon[35109]: pgmap v171: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-15T22:18:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:02 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3848445419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:02 smithi117 ceph-mon[35021]: pgmap v171: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-15T22:18:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:02 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3848445419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:02.669 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-15T22:18:02.670 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-15T22:18:02.962 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:03.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 396777 7.48 MB/sec execute 132 sec latency 131.608 ms 2024-09-15T22:18:03.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 -- 172.21.15.77:0/649384665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df0101270 msgr2=0x7f2df010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 --2- 172.21.15.77:0/649384665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df0101270 0x7f2df010d7a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009cd0 tx=0x7f2de802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 -- 172.21.15.77:0/649384665 shutdown_connections 2024-09-15T22:18:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 --2- 172.21.15.77:0/649384665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df0101270 0x7f2df010d7a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 --2- 172.21.15.77:0/649384665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df01008d0 0x7f2df0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 --2- 172.21.15.77:0/649384665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01068d0 0x7f2df0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 -- 172.21.15.77:0/649384665 >> 172.21.15.77:0/649384665 conn(0x7f2df00fc610 msgr2=0x7f2df00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 -- 172.21.15.77:0/649384665 shutdown_connections 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 -- 172.21.15.77:0/649384665 wait complete. 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.518+0000 7f2df5e83640 1 Processor -- start 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 -- start start 2024-09-15T22:18:03.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0101270 0x7f2df0197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df01068d0 0x7f2df0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0110cd0 con 0x7f2df01068d0 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0110b50 con 0x7f2df01008d0 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5e83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0110e50 con 0x7f2df0101270 2024-09-15T22:18:03.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36110/0 (socket says 172.21.15.77:36110) 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 -- 172.21.15.77:0/3108190970 learned_addr learned my addr 172.21.15.77:0/3108190970 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2deffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0101270 0x7f2df0197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df5682640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df01068d0 0x7f2df0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 -- 172.21.15.77:0/3108190970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0101270 msgr2=0x7f2df0197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0101270 0x7f2df0197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 -- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df01068d0 msgr2=0x7f2df0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df01068d0 0x7f2df0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 -- 172.21.15.77:0/3108190970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a6740 con 0x7f2df01008d0 2024-09-15T22:18:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.519+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9f0 tx=0x7f2de400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.520+0000 7f2dedffb640 1 -- 172.21.15.77:0/3108190970 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400cd80 con 0x7f2df01008d0 2024-09-15T22:18:03.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.520+0000 7f2dedffb640 1 -- 172.21.15.77:0/3108190970 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de4010910 con 0x7f2df01008d0 2024-09-15T22:18:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.520+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a6a30 con 0x7f2df01008d0 2024-09-15T22:18:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.520+0000 7f2dedffb640 1 -- 172.21.15.77:0/3108190970 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40188f0 con 0x7f2df01008d0 2024-09-15T22:18:03.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.520+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a6f70 con 0x7f2df01008d0 2024-09-15T22:18:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.522+0000 7f2dedffb640 1 -- 172.21.15.77:0/3108190970 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2de4010ac0 con 0x7f2df01008d0 2024-09-15T22:18:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 Processor -- start 2024-09-15T22:18:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 -- start start 2024-09-15T22:18:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d9317f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 0x55d931805ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d93175d160 con 0x55d9317f9340 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d931704720 con 0x55d931803c00 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df94be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d931759120 con 0x55d9317f8a20 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2dedffb640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 0x7f2de00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54800/0 (socket says 172.21.15.77:54800) 2024-09-15T22:18:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df4e81640 1 -- 172.21.15.77:0/3038533438 learned_addr learned my addr 172.21.15.77:0/3038533438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2df5682640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d9317f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.523+0000 7f2deffff640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 0x55d931805ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df5682640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 0x7f2de00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2dedffb640 1 -- 172.21.15.77:0/3108190970 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2de40d03a0 con 0x7f2df01008d0 2024-09-15T22:18:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 -- 172.21.15.77:0/3038533438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 msgr2=0x55d9317f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d9317f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 -- 172.21.15.77:0/3038533438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 msgr2=0x55d931805ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 0x55d931805ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 -- 172.21.15.77:0/3038533438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9314ceba0 con 0x55d9317f9340 2024-09-15T22:18:03.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2deffff640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 0x55d931805ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:03.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2de4098850 tx=0x7f2de409c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb05fad8bf2d653 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2ddaffd640 1 -- 172.21.15.77:0/3038533438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de409c3f0 con 0x55d9317f9340 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2ddaffd640 1 -- 172.21.15.77:0/3038533438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de40a06f0 con 0x55d9317f9340 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.524+0000 7f2ddaffd640 1 -- 172.21.15.77:0/3038533438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a4b20 con 0x55d9317f9340 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- 172.21.15.77:0/3038533438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 msgr2=0x55d9318036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2de4098850 tx=0x7f2de409c060 comp rx=0 tx=0).stop 2024-09-15T22:18:03.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df5682640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 0x7f2de00a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2de8004ab0 tx=0x7f2de8004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- 172.21.15.77:0/3038533438 shutdown_connections 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d931803c00 0x55d931805ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d9318036c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 --2- 172.21.15.77:0/3038533438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d9317f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- 172.21.15.77:0/3038533438 >> 172.21.15.77:0/3038533438 conn(0x55d9317f4fb0 msgr2=0x55d9317f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- 172.21.15.77:0/3038533438 shutdown_connections 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- 172.21.15.77:0/3038533438 wait complete. 2024-09-15T22:18:03.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 -- start start 2024-09-15T22:18:03.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.525+0000 7f2df94be740 1 Processor -- start 2024-09-15T22:18:03.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 0x55d931801070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 0x55d9318015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9318067d0 con 0x55d931803c00 2024-09-15T22:18:03.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9316e5900 con 0x55d9317f8a20 2024-09-15T22:18:03.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d931806950 con 0x55d9317f9340 2024-09-15T22:18:03.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df94be740 10 client.0 _ll_register_callbacks cb 0x55d9316e3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:03.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 0x55d9318015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36130/0 (socket says 172.21.15.77:36130) 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 0x55d9318015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54804/0 (socket says 172.21.15.77:54804) 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 -- 172.21.15.77:0/3955444792 learned_addr learned my addr 172.21.15.77:0/3955444792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2deffff640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 0x55d931801070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 -- 172.21.15.77:0/3955444792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 msgr2=0x55d931801070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 0x55d931801070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 -- 172.21.15.77:0/3955444792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 msgr2=0x55d9318015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 0x55d9318015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 -- 172.21.15.77:0/3955444792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d931725c50 con 0x55d9317f8a20 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2deffff640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 0x55d931801070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:03.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2df4e81640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9c0 tx=0x7f2de409c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3645f6b17d76dec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.526+0000 7f2dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de409c3f0 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2de40a4b20 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de40a06f0 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2de40af050 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 10 client.26252 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d9318124d0 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de40a06f0 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2de409b840 con 0x55d9317f8a20 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 10 client.26252 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 10 client.26252 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:03.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 20 client.26252 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.527+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931813820 0x55d931815c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.528+0000 7f2df94be740 4 client.26252 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.528+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93176a890 con 0x55d931813820 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.528+0000 7f2deffff640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931813820 0x55d931815c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.528+0000 7f2deffff640 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931813820 0x55d931815c10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9dd055a73dbe0f server_cookie=fffce512e856c935 in_seq=0 out_seq=0 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.528+0000 7f2dcffff640 10 client.26252 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2dcffff640 1 -- 172.21.15.77:0/3955444792 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d93176a890 con 0x55d931813820 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2dcffff640 10 client.26252 handle_command_reply: tid=0 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 client.26252 shutdown 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 msgr2=0x55d931800b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9c0 tx=0x7f2de409c270 comp rx=0 tx=0).stop 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 shutdown_connections 2024-09-15T22:18:03.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931813820 0x55d931815c10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d931803c00 0x55d9318015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f9340 0x55d931801070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2dcffff640 0 client.26252 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 --2- 172.21.15.77:0/3955444792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d931800b30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 >> 172.21.15.77:0/3955444792 conn(0x55d9317f4fb0 msgr2=0x55d9317fac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 shutdown_connections 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 1 -- 172.21.15.77:0/3955444792 wait complete. 2024-09-15T22:18:03.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.529+0000 7f2df94be740 20 client.26252 trim_cache size 0 max 16384 2024-09-15T22:18:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 Processor -- start 2024-09-15T22:18:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 -- start start 2024-09-15T22:18:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 0x55d9317f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d9317f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b0f80 con 0x55d9317f9340 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b2ac0 con 0x55d9317f8a20 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df94be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b1650 con 0x55d9317f4fb0 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54812/0 (socket says 172.21.15.77:54812) 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 -- 172.21.15.77:0/2469061495 learned_addr learned my addr 172.21.15.77:0/2469061495 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df4e81640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 0x55d9317f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2deffff640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d9317f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 -- 172.21.15.77:0/2469061495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 msgr2=0x55d9317f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 0x55d9317f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 -- 172.21.15.77:0/2469061495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 msgr2=0x55d9317f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d9317f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df5682640 1 -- 172.21.15.77:0/2469061495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d931806350 con 0x55d9317f9340 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2deffff640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d9317f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.550+0000 7f2df4e81640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 0x55d9317f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.551+0000 7f2df5682640 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035000 tx=0x7f2de80093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c0e3c1de4f2894 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.551+0000 7f2dda7fc640 1 -- 172.21.15.77:0/2469061495 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8003a30 con 0x55d9317f9340 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.551+0000 7f2dda7fc640 1 -- 172.21.15.77:0/2469061495 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8036040 con 0x55d9317f9340 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.551+0000 7f2dda7fc640 1 -- 172.21.15.77:0/2469061495 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8030690 con 0x55d9317f9340 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- 172.21.15.77:0/2469061495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 msgr2=0x55d93181b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2de8035000 tx=0x7f2de80093a0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- 172.21.15.77:0/2469061495 shutdown_connections 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d93181b200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f8a20 0x55d9317f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 --2- 172.21.15.77:0/2469061495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f4fb0 0x55d9317f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- 172.21.15.77:0/2469061495 >> 172.21.15.77:0/2469061495 conn(0x55d9317f00e0 msgr2=0x55d9317f3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- 172.21.15.77:0/2469061495 shutdown_connections 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- 172.21.15.77:0/2469061495 wait complete. 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 -- start start 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.552+0000 7f2df94be740 1 Processor -- start 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 0x55d93180bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d93180c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d931808110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b23f0 con 0x55d9317f9340 2024-09-15T22:18:03.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b1d20 con 0x55d9317f4fb0 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01aef60 con 0x55d9317f8a20 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df94be740 10 client.0 _ll_register_callbacks cb 0x7f2ddc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df4e81640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 0x55d93180bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d931808110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2deffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d93180c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df5682640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d931808110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54820/0 (socket says 172.21.15.77:54820) 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2df5682640 1 -- 172.21.15.77:0/2013512415 learned_addr learned my addr 172.21.15.77:0/2013512415 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2deffff640 1 -- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 msgr2=0x55d93180bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2deffff640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 0x55d93180bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2deffff640 1 -- 172.21.15.77:0/2013512415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 msgr2=0x55d931808110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.553+0000 7f2deffff640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d931808110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2deffff640 1 -- 172.21.15.77:0/2013512415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d931808910 con 0x55d9317f8a20 2024-09-15T22:18:03.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2df4e81640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 0x55d93180bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2deffff640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d93180c1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d9318161a0 tx=0x7f2ddc0044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ef9ecfed8856023 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc010630 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f2ddc0107d0 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de8036040 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc02e050 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2df94be740 10 client.26158 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d93187eca0 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.554+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8036040 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ddc005020 con 0x55d9317f8a20 2024-09-15T22:18:03.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 10 client.26158 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 10 client.26158 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 20 client.26158 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931803c00 0x55d9318814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 4 client.26158 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55d931764be0 con 0x55d931803c00 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df4e81640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931803c00 0x55d9318814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2dcffff640 10 client.26158 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:03.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.555+0000 7f2df4e81640 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931803c00 0x55d9318814a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e7f4d1735555b9 server_cookie=6a4321731187397c in_seq=0 out_seq=0 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.657+0000 7f2dcffff640 1 -- 172.21.15.77:0/2013512415 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d931764be0 con 0x55d931803c00 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.657+0000 7f2dcffff640 10 client.26158 handle_command_reply: tid=0 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.657+0000 7f2df94be740 1 client.26158 shutdown 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 msgr2=0x55d93180c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d93180c1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d9318161a0 tx=0x7f2ddc0044d0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 shutdown_connections 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d931803c00 0x55d9318814a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9317f9340 0x55d931808110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9317f8a20 0x55d93180c1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2dcffff640 0 client.26158 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 --2- 172.21.15.77:0/2013512415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9317f4fb0 0x55d93180bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 >> 172.21.15.77:0/2013512415 conn(0x55d9317f00e0 msgr2=0x55d9318012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 shutdown_connections 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 1 -- 172.21.15.77:0/2013512415 wait complete. 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df94be740 20 client.26158 trim_cache size 0 max 16384 2024-09-15T22:18:03.661 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 22, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "c1bbebc4": { 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 22, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:18:03.662 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 145.3, 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:03.663 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:03.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 msgr2=0x7f2de00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 0x7f2de00a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2de8004ab0 tx=0x7f2de8004a40 comp rx=0 tx=0).stop 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 msgr2=0x7f2df0196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.658+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9f0 tx=0x7f2de400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 shutdown_connections 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2de00a33b0 0x7f2de00a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2df01068d0 0x7f2df0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2df0101270 0x7f2df0197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 --2- 172.21.15.77:0/3108190970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2df01008d0 0x7f2df0196ed0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:03.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 >> 172.21.15.77:0/3108190970 conn(0x7f2df00fc610 msgr2=0x7f2df00fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:03.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 shutdown_connections 2024-09-15T22:18:03.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:03.659+0000 7f2df5e83640 1 -- 172.21.15.77:0/3108190970 wait complete. 2024-09-15T22:18:04.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.44 MB/sec execute 133 sec latency 862.108 ms 2024-09-15T22:18:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:04 smithi077 ceph-mon[30980]: pgmap v172: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:04 smithi196 ceph-mon[35109]: pgmap v172: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:04 smithi117 ceph-mon[35021]: pgmap v172: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:04.642 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c1bbebc4', quiesce took 3.7 seconds. Will release after: 21.650000000000002 2024-09-15T22:18:05.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.39 MB/sec execute 134 sec latency 1862.191 ms 2024-09-15T22:18:06.471 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.33 MB/sec execute 135 sec latency 2862.288 ms 2024-09-15T22:18:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:06 smithi196 ceph-mon[35109]: pgmap v173: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:06 smithi117 ceph-mon[35021]: pgmap v173: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:06.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:06 smithi077 ceph-mon[30980]: pgmap v173: 129 pgs: 129 active+clean; 233 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 101 KiB/s wr, 31 op/s 2024-09-15T22:18:07.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.28 MB/sec execute 136 sec latency 3862.398 ms 2024-09-15T22:18:08.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.22 MB/sec execute 137 sec latency 4862.491 ms 2024-09-15T22:18:08.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:08 smithi196 ceph-mon[35109]: pgmap v174: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-15T22:18:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:08 smithi117 ceph-mon[35021]: pgmap v174: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-15T22:18:08.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:08 smithi077 ceph-mon[30980]: pgmap v174: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-09-15T22:18:09.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.17 MB/sec execute 138 sec latency 5862.595 ms 2024-09-15T22:18:10.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.12 MB/sec execute 139 sec latency 6862.696 ms 2024-09-15T22:18:10.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:10 smithi196 ceph-mon[35109]: pgmap v175: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-09-15T22:18:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:10 smithi117 ceph-mon[35021]: pgmap v175: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-09-15T22:18:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:10 smithi077 ceph-mon[30980]: pgmap v175: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-09-15T22:18:11.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.07 MB/sec execute 140 sec latency 7862.778 ms 2024-09-15T22:18:12.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 7.02 MB/sec execute 141 sec latency 8862.865 ms 2024-09-15T22:18:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:12 smithi196 ceph-mon[35109]: pgmap v176: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-09-15T22:18:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:12 smithi117 ceph-mon[35021]: pgmap v176: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-09-15T22:18:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:12 smithi077 ceph-mon[30980]: pgmap v176: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-09-15T22:18:13.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.97 MB/sec execute 142 sec latency 9862.958 ms 2024-09-15T22:18:14.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.92 MB/sec execute 143 sec latency 10863.075 ms 2024-09-15T22:18:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:14 smithi196 ceph-mon[35109]: pgmap v177: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:14 smithi117 ceph-mon[35021]: pgmap v177: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:14.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:14 smithi077 ceph-mon[30980]: pgmap v177: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:15.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.87 MB/sec execute 144 sec latency 11863.145 ms 2024-09-15T22:18:16.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.83 MB/sec execute 145 sec latency 12863.286 ms 2024-09-15T22:18:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:16 smithi117 ceph-mon[35021]: pgmap v178: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:16 smithi077 ceph-mon[30980]: pgmap v178: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:16 smithi196 ceph-mon[35109]: pgmap v178: 129 pgs: 129 active+clean; 239 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-15T22:18:17.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.78 MB/sec execute 146 sec latency 13863.378 ms 2024-09-15T22:18:18.472 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.73 MB/sec execute 147 sec latency 14863.465 ms 2024-09-15T22:18:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:18 smithi117 ceph-mon[35021]: pgmap v179: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-15T22:18:18.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:18 smithi077 ceph-mon[30980]: pgmap v179: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-15T22:18:18.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:18 smithi196 ceph-mon[35109]: pgmap v179: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-15T22:18:19.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.69 MB/sec execute 148 sec latency 15863.547 ms 2024-09-15T22:18:20.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.64 MB/sec execute 149 sec latency 16863.651 ms 2024-09-15T22:18:20.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:20 smithi077 ceph-mon[30980]: pgmap v180: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-09-15T22:18:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:20 smithi196 ceph-mon[35109]: pgmap v180: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-09-15T22:18:20.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:20 smithi117 ceph-mon[35021]: pgmap v180: 129 pgs: 129 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-09-15T22:18:21.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.60 MB/sec execute 150 sec latency 17863.722 ms 2024-09-15T22:18:22.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.55 MB/sec execute 151 sec latency 18863.810 ms 2024-09-15T22:18:22.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:22 smithi077 ceph-mon[30980]: pgmap v181: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 326 KiB/s wr, 26 op/s 2024-09-15T22:18:22.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:22 smithi196 ceph-mon[35109]: pgmap v181: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 326 KiB/s wr, 26 op/s 2024-09-15T22:18:22.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:22 smithi117 ceph-mon[35021]: pgmap v181: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 326 KiB/s wr, 26 op/s 2024-09-15T22:18:23.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.51 MB/sec execute 152 sec latency 19863.914 ms 2024-09-15T22:18:24.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.47 MB/sec execute 153 sec latency 20864.014 ms 2024-09-15T22:18:24.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:24 smithi077 ceph-mon[30980]: pgmap v182: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:24.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:24 smithi196 ceph-mon[35109]: pgmap v182: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:24 smithi117 ceph-mon[35021]: pgmap v182: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:25.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.43 MB/sec execute 154 sec latency 21864.134 ms 2024-09-15T22:18:25.684 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:18:25.947 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:26.294 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 -- 172.21.15.77:0/336366331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84102ef0 msgr2=0x7f4f8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/336366331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84102ef0 0x7f4f8410da40 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ca0 tx=0x7f4f7c02a950 comp rx=0 tx=0).stop 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 -- 172.21.15.77:0/336366331 shutdown_connections 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/336366331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84102ef0 0x7f4f8410da40 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/336366331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102570 0x7f4f841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/336366331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84100e20 0x7f4f84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 -- 172.21.15.77:0/336366331 >> 172.21.15.77:0/336366331 conn(0x7f4f840fc7d0 msgr2=0x7f4f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 -- 172.21.15.77:0/336366331 shutdown_connections 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.435+0000 7f4f89ee6640 1 -- 172.21.15.77:0/336366331 wait complete. 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 Processor -- start 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 -- start start 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84102570 0x7f4f84197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102ef0 0x7f4f8419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84111020 con 0x7f4f84100e20 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.436+0000 7f4f89ee6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84110ea0 con 0x7f4f84102ef0 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f89ee6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841111a0 con 0x7f4f84102570 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40142/0 (socket says 172.21.15.77:40142) 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 -- 172.21.15.77:0/3430959298 learned_addr learned my addr 172.21.15.77:0/3430959298 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84102570 0x7f4f84197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f83fff640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102ef0 0x7f4f8419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 -- 172.21.15.77:0/3430959298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84102570 msgr2=0x7f4f84197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84102570 0x7f4f84197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 -- 172.21.15.77:0/3430959298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102ef0 msgr2=0x7f4f8419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102ef0 0x7f4f8419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 -- 172.21.15.77:0/3430959298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8419c490 con 0x7f4f84100e20 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b4a0 tx=0x7f4f7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f80ff9640 1 -- 172.21.15.77:0/3430959298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74015020 con 0x7f4f84100e20 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8419c780 con 0x7f4f84100e20 2024-09-15T22:18:26.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f80ff9640 1 -- 172.21.15.77:0/3430959298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f74011d10 con 0x7f4f84100e20 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.437+0000 7f4f80ff9640 1 -- 172.21.15.77:0/3430959298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400f2e0 con 0x7f4f84100e20 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.438+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f84069f70 con 0x7f4f84100e20 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f80ff9640 1 -- 172.21.15.77:0/3430959298 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f74018020 con 0x7f4f84100e20 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 Processor -- start 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 -- start start 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f80ff9640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 0x7f4f6c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 0x558dba2d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dba22b2e0 con 0x558dba2c54a0 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dba1d2ce0 con 0x558dba2d0680 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.440+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558dba2272a0 con 0x558dba2c5dc0 2024-09-15T22:18:26.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40150/0 (socket says 172.21.15.77:40150) 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f83fff640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 0x7f4f6c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 -- 172.21.15.77:0/3604318540 learned_addr learned my addr 172.21.15.77:0/3604318540 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f80ff9640 1 -- 172.21.15.77:0/3430959298 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f740cf7d0 con 0x7f4f84100e20 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 0x558dba2d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f83fff640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 -- 172.21.15.77:0/3604318540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 msgr2=0x558dba2d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 0x558dba2d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 -- 172.21.15.77:0/3604318540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 msgr2=0x558dba2d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 -- 172.21.15.77:0/3604318540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dba1b7440 con 0x558dba2c54a0 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 0x558dba2d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:26.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f83fff640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 0x7f4f6c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098d0 tx=0x7f4f7c02fae0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4f740cccf0 tx=0x7f4f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78042316cf28ad00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f69ffb640 1 -- 172.21.15.77:0/3604318540 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74078c00 con 0x558dba2c54a0 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f69ffb640 1 -- 172.21.15.77:0/3604318540 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7409bf20 con 0x558dba2c54a0 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.441+0000 7f4f69ffb640 1 -- 172.21.15.77:0/3604318540 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740092b0 con 0x558dba2c54a0 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3604318540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 msgr2=0x558dba2c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4f740cccf0 tx=0x7f4f740046c0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3604318540 shutdown_connections 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c5dc0 0x558dba2d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3604318540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2c5880 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3604318540 >> 172.21.15.77:0/3604318540 conn(0x558dba2c1a30 msgr2=0x558dba2c1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3604318540 shutdown_connections 2024-09-15T22:18:26.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.442+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3604318540 wait complete. 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 -- start start 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 Processor -- start 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2d0680 0x558dba2cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dba2d3250 con 0x558dba2c54a0 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dba1e6050 con 0x558dba2c5dc0 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558dba2d33d0 con 0x558dba2d0680 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f8c2bf740 10 client.0 _ll_register_callbacks cb 0x558dba226160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40156/0 (socket says 172.21.15.77:40156) 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f837fe640 1 -- 172.21.15.77:0/3051562458 learned_addr learned my addr 172.21.15.77:0/3051562458 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:26.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f83fff640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2d0680 0x558dba2cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.443+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 -- 172.21.15.77:0/3051562458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2d0680 msgr2=0x558dba2cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2d0680 0x558dba2cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 -- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 msgr2=0x558dba2cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 -- 172.21.15.77:0/3051562458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db9f9c3c0 con 0x558dba2c54a0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f837fe640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400bfd0 tx=0x7f4f7409fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfff77e83a3774d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b4060 con 0x558dba2c54a0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f74010510 con 0x558dba2c54a0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7409bf20 con 0x558dba2c54a0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f74078a20 con 0x558dba2c54a0 2024-09-15T22:18:26.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f8c2bf740 10 client.17070 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558dba2d3250 con 0x558dba2c54a0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.444+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7409bf20 con 0x558dba2c54a0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f740cf380 con 0x558dba2c54a0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 20 client.17070 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba2dfda0 0x558dba2e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 4 client.17070 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dba238850 con 0x558dba2dfda0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba2dfda0 0x558dba2e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f62ffd640 10 client.17070 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.445+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba2dfda0 0x558dba2e2190 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86a11be75c05682 server_cookie=b8aba324b61be250 in_seq=0 out_seq=0 2024-09-15T22:18:26.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f62ffd640 1 -- 172.21.15.77:0/3051562458 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dba238850 con 0x558dba2dfda0 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f62ffd640 10 client.17070 handle_command_reply: tid=0 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 client.17070 shutdown 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 msgr2=0x558dba2cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400bfd0 tx=0x7f4f7409fca0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 shutdown_connections 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba2dfda0 0x558dba2e2190 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2d0680 0x558dba2cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f62ffd640 0 client.17070 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/3051562458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c54a0 0x558dba2cffd0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 >> 172.21.15.77:0/3051562458 conn(0x558dba2c1a30 msgr2=0x558dba2c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 shutdown_connections 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/3051562458 wait complete. 2024-09-15T22:18:26.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.446+0000 7f4f8c2bf740 20 client.17070 trim_cache size 0 max 16384 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 Processor -- start 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 -- start start 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841aefb0 con 0x558dba2c5dc0 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841b0af0 con 0x558dba2d0680 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841af680 con 0x558dba2c54a0 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40164/0 (socket says 172.21.15.77:40164) 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 -- 172.21.15.77:0/543176498 learned_addr learned my addr 172.21.15.77:0/543176498 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f83fff640 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f837fe640 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 -- 172.21.15.77:0/543176498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 msgr2=0x558dba2dc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2dc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 -- 172.21.15.77:0/543176498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 msgr2=0x558dba2c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.464+0000 7f4f82ffd640 1 -- 172.21.15.77:0/543176498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dba2c5bf0 con 0x558dba2c5dc0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.465+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009a90 tx=0x7f4f78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6cb9a9d0706965 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.465+0000 7f4f697fa640 1 -- 172.21.15.77:0/543176498 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7801d310 con 0x558dba2c5dc0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.465+0000 7f4f697fa640 1 -- 172.21.15.77:0/543176498 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7801d990 con 0x558dba2c5dc0 2024-09-15T22:18:26.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.465+0000 7f4f697fa640 1 -- 172.21.15.77:0/543176498 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78015c40 con 0x558dba2c5dc0 2024-09-15T22:18:26.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/543176498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 msgr2=0x558dba2cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009a90 tx=0x7f4f78008040 comp rx=0 tx=0).stop 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/543176498 shutdown_connections 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2d0680 0x558dba2c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2c5dc0 0x558dba2cbb50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/543176498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/543176498 >> 172.21.15.77:0/543176498 conn(0x558dba2c1a30 msgr2=0x558dba2c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/543176498 shutdown_connections 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/543176498 wait complete. 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 -- start start 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.466+0000 7f4f8c2bf740 1 Processor -- start 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841b0420 con 0x558dba2d0680 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841afd50 con 0x558dba2c5dc0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841acf90 con 0x558dba2c54a0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f8c2bf740 10 client.0 _ll_register_callbacks cb 0x7f4f7409b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40180/0 (socket says 172.21.15.77:40180) 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 -- 172.21.15.77:0/2879136863 learned_addr learned my addr 172.21.15.77:0/2879136863 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f837fe640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f82ffd640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 -- 172.21.15.77:0/2879136863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 msgr2=0x558dba2d44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2d44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 -- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 msgr2=0x558dba2d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 -- 172.21.15.77:0/2879136863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dba349a00 con 0x558dba2d0680 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f837fe640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f83fff640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ca0 tx=0x7f4f7c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f50e76e73d6fad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.467+0000 7f4f62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:26.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c058060 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f7801d990 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f4f7c045040 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f7c05f050 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 10 client.17076 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558dba34ad20 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f7801d990 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f7c0307e0 con 0x558dba2d0680 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 10 client.17076 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 20 client.17076 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba34b450 0x558dba34d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 4 client.17076 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:18:26.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.468+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dba232d60 con 0x558dba34b450 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f837fe640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba34b450 0x558dba34d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f62ffd640 10 client.17076 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f837fe640 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba34b450 0x558dba34d870 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234a4023b2495c42 server_cookie=28f592e281270a49 in_seq=0 out_seq=0 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f62ffd640 1 -- 172.21.15.77:0/2879136863 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558dba232d60 con 0x558dba34b450 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f62ffd640 10 client.17076 handle_command_reply: tid=0 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 client.17076 shutdown 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 msgr2=0x558dba2d4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009ca0 tx=0x7f4f7c005c50 comp rx=0 tx=0).stop 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 shutdown_connections 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558dba34b450 0x558dba34d870 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dba2d0680 0x558dba2d4f40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dba2c5dc0 0x558dba2d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 --2- 172.21.15.77:0/2879136863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558dba2c54a0 0x558dba2d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 >> 172.21.15.77:0/2879136863 conn(0x558dba2c1a30 msgr2=0x558dba2ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.469+0000 7f4f62ffd640 0 client.17076 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 shutdown_connections 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f8c2bf740 1 -- 172.21.15.77:0/2879136863 wait complete. 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f8c2bf740 20 client.17076 trim_cache size 0 max 16384 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:18:26.479 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 msgr2=0x7f4f6c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 0x7f4f6c0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098d0 tx=0x7f4f7c02fae0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 msgr2=0x7f4f841974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400b4a0 tx=0x7f4f7400b8a0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 shutdown_connections 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f6c0a3430 0x7f4f6c0a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f84102ef0 0x7f4f8419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84102570 0x7f4f84197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 --2- 172.21.15.77:0/3430959298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f84100e20 0x7f4f841974e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:26.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.470+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 >> 172.21.15.77:0/3430959298 conn(0x7f4f840fc7d0 msgr2=0x7f4f840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:26.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.471+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 shutdown_connections 2024-09-15T22:18:26.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.39 MB/sec execute 155 sec latency 22864.170 ms 2024-09-15T22:18:26.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:26.471+0000 7f4f89ee6640 1 -- 172.21.15.77:0/3430959298 wait complete. 2024-09-15T22:18:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:26 smithi077 ceph-mon[30980]: pgmap v183: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:26.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:26 smithi196 ceph-mon[35109]: pgmap v183: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:26 smithi117 ceph-mon[35021]: pgmap v183: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-09-15T22:18:27.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:18:27.277 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:27.388 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:27.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.34 MB/sec execute 156 sec latency 23864.197 ms 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- 172.21.15.77:0/44974391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102ef0 msgr2=0x7f5afc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 --2- 172.21.15.77:0/44974391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102ef0 0x7f5afc10da40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009ca0 tx=0x7f5af402a950 comp rx=0 tx=0).stop 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- 172.21.15.77:0/44974391 shutdown_connections 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 --2- 172.21.15.77:0/44974391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102ef0 0x7f5afc10da40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 --2- 172.21.15.77:0/44974391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc102570 0x7f5afc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:27.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 --2- 172.21.15.77:0/44974391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc100e20 0x7f5afc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- 172.21.15.77:0/44974391 >> 172.21.15.77:0/44974391 conn(0x7f5afc0fc7d0 msgr2=0x7f5afc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- 172.21.15.77:0/44974391 shutdown_connections 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- 172.21.15.77:0/44974391 wait complete. 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 Processor -- start 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.880+0000 7f5b02d84640 1 -- start start 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 0x7f5afc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 0x7f5afc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1120a0 con 0x7f5afc102570 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc111f20 con 0x7f5afc102ef0 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc112220 con 0x7f5afc100e20 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:27.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40202/0 (socket says 172.21.15.77:40202) 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 -- 172.21.15.77:0/3009982893 learned_addr learned my addr 172.21.15.77:0/3009982893 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b00b20640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 0x7f5afc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b01321640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 0x7f5afc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 -- 172.21.15.77:0/3009982893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 msgr2=0x7f5afc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 0x7f5afc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 -- 172.21.15.77:0/3009982893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 msgr2=0x7f5afc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 0x7f5afc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 -- 172.21.15.77:0/3009982893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc19d3e0 con 0x7f5afc102570 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b00b20640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 0x7f5afc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b01321640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 0x7f5afc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5afbfff640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800c920 tx=0x7f5ae800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8007c70 con 0x7f5afc102570 2024-09-15T22:18:27.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8007e10 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc19d6d0 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.881+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8018890 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.882+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc1a4fb0 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.883+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc101200 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.884+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5ae8018a70 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.884+0000 7f5af9ffb640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 0x7f5aec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.884+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f5ae80d0260 con 0x7f5afc102570 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.884+0000 7f5b00b20640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 0x7f5aec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.885+0000 7f5b00b20640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 0x7f5aec0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009930 tx=0x7f5af0008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:27.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:27.889+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ae8014030 con 0x7f5afc102570 2024-09-15T22:18:28.057 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:28.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.105+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5afc1a5240 con 0x7f5afc102570 2024-09-15T22:18:28.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.106+0000 7f5af9ffb640 1 -- 172.21.15.77:0/3009982893 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f5ae800de20 con 0x7f5afc102570 2024-09-15T22:18:28.109 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:18:28.109 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:18:28.109 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 msgr2=0x7f5aec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 0x7f5aec0a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009930 tx=0x7f5af0008040 comp rx=0 tx=0).stop 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 msgr2=0x7f5afc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800c920 tx=0x7f5ae800cde0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 shutdown_connections 2024-09-15T22:18:28.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5aec0a33b0 0x7f5aec0a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5afc102ef0 0x7f5afc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5afc102570 0x7f5afc198970 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 --2- 172.21.15.77:0/3009982893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5afc100e20 0x7f5afc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 >> 172.21.15.77:0/3009982893 conn(0x7f5afc0fc7d0 msgr2=0x7f5afc106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 shutdown_connections 2024-09-15T22:18:28.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.108+0000 7f5b02d84640 1 -- 172.21.15.77:0/3009982893 wait complete. 2024-09-15T22:18:28.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.30 MB/sec execute 157 sec latency 24864.308 ms 2024-09-15T22:18:28.698 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:28 smithi077 ceph-mon[30980]: pgmap v184: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-15T22:18:28.699 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:28 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3009982893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:28.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 -- 172.21.15.77:0/4197514632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 msgr2=0x7fe7841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 --2- 172.21.15.77:0/4197514632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe7841029b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009900 tx=0x7fe76c02a480 comp rx=0 tx=0).stop 2024-09-15T22:18:28.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 -- 172.21.15.77:0/4197514632 shutdown_connections 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 --2- 172.21.15.77:0/4197514632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784102ef0 0x7fe78410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 --2- 172.21.15.77:0/4197514632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe7841029b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 --2- 172.21.15.77:0/4197514632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784100e20 0x7fe784101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 -- 172.21.15.77:0/4197514632 >> 172.21.15.77:0/4197514632 conn(0x7fe7840fc7d0 msgr2=0x7fe7840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 -- 172.21.15.77:0/4197514632 shutdown_connections 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.696+0000 7fe788b80640 1 -- 172.21.15.77:0/4197514632 wait complete. 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 Processor -- start 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 -- start start 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784100e20 0x7fe7841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:28.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 0x7fe78419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe784110f80 con 0x7fe784102570 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe784110e00 con 0x7fe784100e20 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe788b80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe784111100 con 0x7fe784102ef0 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40212/0 (socket says 172.21.15.77:40212) 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 -- 172.21.15.77:0/1902420875 learned_addr learned my addr 172.21.15.77:0/1902420875 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe782d9d640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 0x7fe78419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe78259c640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784100e20 0x7fe7841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 -- 172.21.15.77:0/1902420875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 msgr2=0x7fe78419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 0x7fe78419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 -- 172.21.15.77:0/1902420875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784100e20 msgr2=0x7fe7841985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784100e20 0x7fe7841985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe77bfff640 1 -- 172.21.15.77:0/1902420875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78419d550 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.697+0000 7fe782d9d640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 0x7fe78419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe77bfff640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe76c006010 tx=0x7fe76c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c03f070 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78419d7e0 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c031c30 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c035030 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.698+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a5120 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.699+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe784101200 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.700+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe76c042070 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.701+0000 7fe77b7fe640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 0x7fe7700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.701+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe76c0f0620 con 0x7fe784102570 2024-09-15T22:18:28.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.701+0000 7fe78259c640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 0x7fe7700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:28.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.701+0000 7fe78259c640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 0x7fe7700a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe774005fd0 tx=0x7fe774005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:28.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.705+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe76c031de0 con 0x7fe784102570 2024-09-15T22:18:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:28 smithi196 ceph-mon[35109]: pgmap v184: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-15T22:18:28.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:28 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3009982893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:28 smithi117 ceph-mon[35021]: pgmap v184: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-09-15T22:18:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:28 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3009982893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.909+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe78410a0a0 con 0x7fe784102570 2024-09-15T22:18:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.910+0000 7fe77b7fe640 1 -- 172.21.15.77:0/1902420875 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fe76c0bccc0 con 0x7fe784102570 2024-09-15T22:18:28.913 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:18:28.913 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:18:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:18:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 msgr2=0x7fe7700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 0x7fe7700a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe774005fd0 tx=0x7fe774005850 comp rx=0 tx=0).stop 2024-09-15T22:18:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 msgr2=0x7fe784198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe76c006010 tx=0x7fe76c003c10 comp rx=0 tx=0).stop 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 shutdown_connections 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7700a33b0 0x7fe7700a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe784102ef0 0x7fe78419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe784102570 0x7fe784198ae0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 --2- 172.21.15.77:0/1902420875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe784100e20 0x7fe7841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 >> 172.21.15.77:0/1902420875 conn(0x7fe7840fc7d0 msgr2=0x7fe78406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 shutdown_connections 2024-09-15T22:18:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:28.912+0000 7fe788b80640 1 -- 172.21.15.77:0/1902420875 wait complete. 2024-09-15T22:18:29.352 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id c1bbebc4 --release --await' 2024-09-15T22:18:29.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id c1bbebc4 --release --await 2024-09-15T22:18:29.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.26 MB/sec execute 158 sec latency 25864.354 ms 2024-09-15T22:18:29.688 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1902420875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:29.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:29 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1902420875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:29.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:29 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1902420875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:30.241 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:30.245 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:18:30.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 397130 6.22 MB/sec execute 159 sec latency 26864.433 ms 2024-09-15T22:18:30.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:30 smithi077 ceph-mon[30980]: pgmap v185: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 -- 172.21.15.77:0/4109164036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690106a80 msgr2=0x7fc690106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 --2- 172.21.15.77:0/4109164036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690106a80 0x7fc690106e60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc684009930 tx=0x7fc68402a420 comp rx=0 tx=0).stop 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 -- 172.21.15.77:0/4109164036 shutdown_connections 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 --2- 172.21.15.77:0/4109164036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690101420 0x7fc69010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 --2- 172.21.15.77:0/4109164036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 --2- 172.21.15.77:0/4109164036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690106a80 0x7fc690106e60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 -- 172.21.15.77:0/4109164036 >> 172.21.15.77:0/4109164036 conn(0x7fc6900fc7a0 msgr2=0x7fc6900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 -- 172.21.15.77:0/4109164036 shutdown_connections 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.727+0000 7fc696f45640 1 -- 172.21.15.77:0/4109164036 wait complete. 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 Processor -- start 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 -- start start 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690106a80 0x7fc6901030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc690110f20 con 0x7fc690101420 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc690110da0 con 0x7fc690106a80 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc696f45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901110a0 con 0x7fc690100a80 2024-09-15T22:18:30.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41306/0 (socket says 172.21.15.77:41306) 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 -- 172.21.15.77:0/133926905 learned_addr learned my addr 172.21.15.77:0/133926905 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc694ce1640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc6954e2640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690106a80 0x7fc6901030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 -- 172.21.15.77:0/133926905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 msgr2=0x7fc690102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 -- 172.21.15.77:0/133926905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690106a80 msgr2=0x7fc6901030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690106a80 0x7fc6901030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc68ffff640 1 -- 172.21.15.77:0/133926905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6901a6aa0 con 0x7fc690101420 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.728+0000 7fc694ce1640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc68ffff640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b4a0 tx=0x7fc67c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc68dffb640 1 -- 172.21.15.77:0/133926905 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c014070 con 0x7fc690101420 2024-09-15T22:18:30.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc68dffb640 1 -- 172.21.15.77:0/133926905 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c010af0 con 0x7fc690101420 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6901a6d90 con 0x7fc690101420 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc68dffb640 1 -- 172.21.15.77:0/133926905 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c017d10 con 0x7fc690101420 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.729+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6901a7220 con 0x7fc690101420 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.731+0000 7fc68dffb640 1 -- 172.21.15.77:0/133926905 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc67c017070 con 0x7fc690101420 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 Processor -- start 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 -- start start 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917641d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x555917644630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55591759c620 con 0x555917637060 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555917543e50 con 0x555917642240 2024-09-15T22:18:30.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc69931e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5559175985e0 con 0x555917637980 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41312/0 (socket says 172.21.15.77:41312) 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 -- 172.21.15.77:0/2625948127 learned_addr learned my addr 172.21.15.77:0/2625948127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc68dffb640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 0x7fc6800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 -- 172.21.15.77:0/2625948127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 msgr2=0x555917641d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917641d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc68ffff640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x555917644630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917641d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 -- 172.21.15.77:0/2625948127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 msgr2=0x555917644630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc68dffb640 1 -- 172.21.15.77:0/133926905 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc67c0cf3d0 con 0x7fc690101420 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x555917644630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc6954e2640 1 -- 172.21.15.77:0/2625948127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555917534d40 con 0x555917637060 2024-09-15T22:18:30.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.732+0000 7fc68ffff640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x555917644630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.733+0000 7fc6954e2640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 0x7fc6800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.733+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc688007f10 tx=0x7fc68800c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16938198021f6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.733+0000 7fc67affd640 1 -- 172.21.15.77:0/2625948127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688017040 con 0x555917637060 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.733+0000 7fc67affd640 1 -- 172.21.15.77:0/2625948127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688013e50 con 0x555917637060 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.733+0000 7fc67affd640 1 -- 172.21.15.77:0/2625948127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68801dde0 con 0x555917637060 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- 172.21.15.77:0/2625948127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 msgr2=0x555917637440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc688007f10 tx=0x7fc68800c560 comp rx=0 tx=0).stop 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc6954e2640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 0x7fc6800a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc68801d000 tx=0x7fc68801e000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- 172.21.15.77:0/2625948127 shutdown_connections 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x555917644630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917641d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- 172.21.15.77:0/2625948127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637060 0x555917637440 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- 172.21.15.77:0/2625948127 >> 172.21.15.77:0/2625948127 conn(0x5559176335f0 msgr2=0x5559176339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- 172.21.15.77:0/2625948127 shutdown_connections 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- 172.21.15.77:0/2625948127 wait complete. 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- start start 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 Processor -- start 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637060 0x55591763f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x55591763fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555917644e10 con 0x555917637980 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555917524f10 con 0x555917642240 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.734+0000 7fc69931e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555917644f90 con 0x555917637060 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc69931e740 10 client.0 _ll_register_callbacks cb 0x555917548db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41324/0 (socket says 172.21.15.77:41324) 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 -- 172.21.15.77:0/505086438 learned_addr learned my addr 172.21.15.77:0/505086438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc694ce1640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637060 0x55591763f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc6954e2640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x55591763fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 -- 172.21.15.77:0/505086438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637060 msgr2=0x55591763f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637060 0x55591763f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 -- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 msgr2=0x55591763fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x55591763fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 -- 172.21.15.77:0/505086438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559172ab580 con 0x555917637980 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc6954e2640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x55591763fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc68ffff640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009420 tx=0x7fc67c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da56e21912d396ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc66ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:30.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c09b480 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc67c0cca60 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc688013e50 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc67c09f830 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc69931e740 10 client.17100 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555917375960 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.735+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688013e50 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc67c060220 con 0x555917637980 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 10 client.17100 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 10 client.17100 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 20 client.17100 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555917651d30 0x555917654160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 4 client.17100 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559175a9b90 con 0x555917651d30 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.736+0000 7fc694ce1640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555917651d30 0x555917654160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc694ce1640 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555917651d30 0x555917654160 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a613cbf02330193 server_cookie=dbc88851a5bccfa3 in_seq=0 out_seq=0 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc66ffff640 10 client.17100 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc66ffff640 1 -- 172.21.15.77:0/505086438 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559175a9b90 con 0x555917651d30 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc66ffff640 10 client.17100 handle_command_reply: tid=0 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 client.17100 shutdown 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 msgr2=0x55591763f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009420 tx=0x7fc67c01c000 comp rx=0 tx=0).stop 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 shutdown_connections 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555917651d30 0x555917654160 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917642240 0x55591763fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917637980 0x55591763f690 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc66ffff640 0 client.17100 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 --2- 172.21.15.77:0/505086438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637060 0x55591763f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.737+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 >> 172.21.15.77:0/505086438 conn(0x5559176335f0 msgr2=0x5559176392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.738+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 shutdown_connections 2024-09-15T22:18:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.738+0000 7fc69931e740 1 -- 172.21.15.77:0/505086438 wait complete. 2024-09-15T22:18:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.738+0000 7fc69931e740 20 client.17100 trim_cache size 0 max 16384 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.755+0000 7fc69931e740 1 Processor -- start 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.755+0000 7fc69931e740 1 -- start start 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x555917639ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917632640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901b11e0 con 0x555917642240 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901b2d20 con 0x555917637060 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc69931e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901b18b0 con 0x555917637980 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41332/0 (socket says 172.21.15.77:41332) 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 -- 172.21.15.77:0/2025732204 learned_addr learned my addr 172.21.15.77:0/2025732204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x555917639ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc68ffff640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917632640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 -- 172.21.15.77:0/2025732204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 msgr2=0x555917632640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917632640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 -- 172.21.15.77:0/2025732204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 msgr2=0x555917639ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x555917639ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc6954e2640 1 -- 172.21.15.77:0/2025732204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559176377b0 con 0x555917642240 2024-09-15T22:18:30.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.756+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x555917639ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.757+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc688007520 tx=0x7fc68800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62059f21b60fd0f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.757+0000 7fc67a7fc640 1 -- 172.21.15.77:0/2025732204 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68802c070 con 0x555917642240 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.757+0000 7fc67a7fc640 1 -- 172.21.15.77:0/2025732204 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688013e50 con 0x555917642240 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.757+0000 7fc67a7fc640 1 -- 172.21.15.77:0/2025732204 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688017040 con 0x555917642240 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 -- 172.21.15.77:0/2025732204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 msgr2=0x555917659a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc688007520 tx=0x7fc68800f000 comp rx=0 tx=0).stop 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 -- 172.21.15.77:0/2025732204 shutdown_connections 2024-09-15T22:18:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917659a30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917632640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 --2- 172.21.15.77:0/2025732204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x555917639ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 -- 172.21.15.77:0/2025732204 >> 172.21.15.77:0/2025732204 conn(0x5559176335f0 msgr2=0x555917631f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 -- 172.21.15.77:0/2025732204 shutdown_connections 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.758+0000 7fc69931e740 1 -- 172.21.15.77:0/2025732204 wait complete. 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 -- start start 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 Processor -- start 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x5559176461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917646710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901b2650 con 0x555917642240 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901b1f80 con 0x555917637060 2024-09-15T22:18:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901af1c0 con 0x555917637980 2024-09-15T22:18:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc69931e740 10 client.0 _ll_register_callbacks cb 0x7fc67c09b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc6954e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41340/0 (socket says 172.21.15.77:41340) 2024-09-15T22:18:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.759+0000 7fc6954e2640 1 -- 172.21.15.77:0/2995820268 learned_addr learned my addr 172.21.15.77:0/2995820268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x5559176461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc68ffff640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917646710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 -- 172.21.15.77:0/2995820268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 msgr2=0x555917646710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917646710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 -- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 msgr2=0x5559176461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x5559176461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 -- 172.21.15.77:0/2995820268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559176bbc00 con 0x555917642240 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x5559176461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc6954e2640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc68800ca60 tx=0x7fc688007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3776bd1dcbabed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc66ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688035070 con 0x555917642240 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fc688028ae0 con 0x555917642240 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.760+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc688013e50 con 0x555917642240 2024-09-15T22:18:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc688017070 con 0x555917642240 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 10 client.17106 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5559176bcf20 con 0x555917642240 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688013e50 con 0x555917642240 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc688025070 con 0x555917642240 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 10 client.17106 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 10 client.17106 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 20 client.17106 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559176bd4a0 0x5559176bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 4 client.17106 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c1bbebc4", "release": true, "await": true}] 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c1bbebc4", "release": true, "await": true}) -- 0x5559175a40a0 con 0x5559176bd4a0 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.761+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559176bd4a0 0x5559176bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.762+0000 7fc66ffff640 10 client.17106 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.762+0000 7fc694ce1640 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559176bd4a0 0x5559176bf890 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68a5dd272048e5e server_cookie=84d8cd7a57d32713 in_seq=0 out_seq=0 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc66ffff640 1 -- 172.21.15.77:0/2995820268 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5559175a40a0 con 0x5559176bd4a0 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc66ffff640 10 client.17106 handle_command_reply: tid=0 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 client.17106 shutdown 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 msgr2=0x555917646c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc68800ca60 tx=0x7fc688007680 comp rx=0 tx=0).stop 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 shutdown_connections 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559176bd4a0 0x5559176bf890 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555917642240 0x555917646c50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555917637980 0x555917646710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 --2- 172.21.15.77:0/2995820268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555917637060 0x5559176461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc66ffff640 0 client.17106 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 >> 172.21.15.77:0/2995820268 conn(0x5559176335f0 msgr2=0x55591763f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 shutdown_connections 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 1 -- 172.21.15.77:0/2995820268 wait complete. 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.772+0000 7fc69931e740 20 client.17106 trim_cache size 0 max 16384 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 24, 2024-09-15T22:18:30.776 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "c1bbebc4": { 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 24, 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 145.3, 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:18:30.777 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:30.778 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:30.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 msgr2=0x7fc6800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 0x7fc6800a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc68801d000 tx=0x7fc68801e000 comp rx=0 tx=0).stop 2024-09-15T22:18:30.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 msgr2=0x7fc690102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:30.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b4a0 tx=0x7fc67c00b960 comp rx=0 tx=0).stop 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 shutdown_connections 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6800a33b0 0x7fc6800a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc690106a80 0x7fc6901030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc690101420 0x7fc690102b90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 --2- 172.21.15.77:0/133926905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc690100a80 0x7fc690102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 >> 172.21.15.77:0/133926905 conn(0x7fc6900fc7a0 msgr2=0x7fc690104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 shutdown_connections 2024-09-15T22:18:30.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:30.773+0000 7fc696f45640 1 -- 172.21.15.77:0/133926905 wait complete. 2024-09-15T22:18:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:30.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:30 smithi196 ceph-mon[35109]: pgmap v185: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-09-15T22:18:30.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:30 smithi117 ceph-mon[35021]: pgmap v185: 129 pgs: 129 active+clean; 236 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-09-15T22:18:31.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 398099 6.22 MB/sec execute 160 sec latency 27166.968 ms 2024-09-15T22:18:31.607 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c1bbebc4', total seconds elapsed: 30.7 2024-09-15T22:18:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 -- 172.21.15.77:0/428223901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4100e20 msgr2=0x7f6ce4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 --2- 172.21.15.77:0/428223901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4100e20 0x7f6ce4101200 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009900 tx=0x7f6cd402a480 comp rx=0 tx=0).stop 2024-09-15T22:18:32.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 -- 172.21.15.77:0/428223901 shutdown_connections 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 --2- 172.21.15.77:0/428223901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102ef0 0x7f6ce410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 --2- 172.21.15.77:0/428223901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4102570 0x7f6ce41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 --2- 172.21.15.77:0/428223901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4100e20 0x7f6ce4101200 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 -- 172.21.15.77:0/428223901 >> 172.21.15.77:0/428223901 conn(0x7f6ce40fc7d0 msgr2=0x7f6ce40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 -- 172.21.15.77:0/428223901 shutdown_connections 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.036+0000 7f6ce909c640 1 -- 172.21.15.77:0/428223901 wait complete. 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.037+0000 7f6ce909c640 1 Processor -- start 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.037+0000 7f6ce909c640 1 -- start start 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 0x7f6ce4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4110f80 con 0x7f6ce4102ef0 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4110e00 con 0x7f6ce4100e20 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce909c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce4111100 con 0x7f6ce4102570 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 0x7f6ce4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41354/0 (socket says 172.21.15.77:41354) 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49448/0 (socket says 172.21.15.77:49448) 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 -- 172.21.15.77:0/853585584 learned_addr learned my addr 172.21.15.77:0/853585584 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 -- 172.21.15.77:0/853585584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 msgr2=0x7f6ce4199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 0x7f6ce4199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 -- 172.21.15.77:0/853585584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 msgr2=0x7f6ce4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 -- 172.21.15.77:0/853585584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce41a48f0 con 0x7f6ce4102ef0 2024-09-15T22:18:32.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce259c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 0x7f6ce4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.038+0000 7f6ce359e640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00eb10 tx=0x7f6cdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.039+0000 7f6ccbfff640 1 -- 172.21.15.77:0/853585584 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc004b20 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.039+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce41a4ac0 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.039+0000 7f6ccbfff640 1 -- 172.21.15.77:0/853585584 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc004cc0 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.039+0000 7f6ccbfff640 1 -- 172.21.15.77:0/853585584 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc010760 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.039+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41a5000 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.041+0000 7f6ccbfff640 1 -- 172.21.15.77:0/853585584 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6cdc005020 con 0x7f6ce4102ef0 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.041+0000 7f6ceb475740 1 Processor -- start 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.041+0000 7f6ceb475740 1 -- start start 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5623419ed2f0 con 0x562341a88b40 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562341994cf0 con 0x562341a875d0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ceb475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623419e92b0 con 0x562341a92370 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ccbfff640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41356/0 (socket says 172.21.15.77:41356) 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 -- 172.21.15.77:0/620809094 learned_addr learned my addr 172.21.15.77:0/620809094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ccbfff640 1 -- 172.21.15.77:0/853585584 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6cdc0d0690 con 0x7f6ce4102ef0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce359e640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 -- 172.21.15.77:0/620809094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 msgr2=0x562341a94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 -- 172.21.15.77:0/620809094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 msgr2=0x562341a87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a87a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 -- 172.21.15.77:0/620809094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623419793a0 con 0x562341a88b40 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.042+0000 7f6ce259c640 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8013c00 tx=0x7f6cd8009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628ea297fa707547 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6cc97fa640 1 -- 172.21.15.77:0/620809094 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8013db0 con 0x562341a88b40 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6cc97fa640 1 -- 172.21.15.77:0/620809094 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8013170 con 0x562341a88b40 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6cc97fa640 1 -- 172.21.15.77:0/620809094 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004a00 con 0x562341a88b40 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6ceb475740 1 -- 172.21.15.77:0/620809094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 msgr2=0x562341a87090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6ceb475740 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8013c00 tx=0x7f6cd8009570 comp rx=0 tx=0).stop 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.043+0000 7f6ce359e640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0cdb50 tx=0x7f6cdc00efd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- 172.21.15.77:0/620809094 shutdown_connections 2024-09-15T22:18:32.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- 172.21.15.77:0/620809094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a87090 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- 172.21.15.77:0/620809094 >> 172.21.15.77:0/620809094 conn(0x562341a50310 msgr2=0x562341a83c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- 172.21.15.77:0/620809094 shutdown_connections 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- 172.21.15.77:0/620809094 wait complete. 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- start start 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 Processor -- start 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a91ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562341a95010 con 0x562341a88b40 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5623419a8060 con 0x562341a875d0 2024-09-15T22:18:32.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562341a95190 con 0x562341a92370 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ceb475740 10 client.0 _ll_register_callbacks cb 0x5623416fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41370/0 (socket says 172.21.15.77:41370) 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ce2d9d640 1 -- 172.21.15.77:0/3732050310 learned_addr learned my addr 172.21.15.77:0/3732050310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.044+0000 7f6ce259c640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a91ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce259c640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 -- 172.21.15.77:0/3732050310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 msgr2=0x562341a8f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a8f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 -- 172.21.15.77:0/3732050310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 msgr2=0x562341a91ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a91ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 -- 172.21.15.77:0/3732050310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234175e3c0 con 0x562341a88b40 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40314e0 tx=0x7f6cd403d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b136dd3d4d152f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd404c060 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd8013170 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6cd403dda0 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6cd4053050 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ceb475740 10 client.17118 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562341aa0ab0 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8013170 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.045+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6cd4046070 con 0x562341a88b40 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 10 client.17118 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 10 client.17118 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 20 client.17118 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341aa1ec0 0x562341aa42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 4 client.17118 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623419fa860 con 0x562341aa1ec0 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ce259c640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341aa1ec0 0x562341aa42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6cc6ff5640 10 client.17118 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.046+0000 7f6ce259c640 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341aa1ec0 0x562341aa42b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca761209f15fb50e server_cookie=f87056c6cce4bb7f in_seq=0 out_seq=0 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/3732050310 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623419fa860 con 0x562341aa1ec0 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6cc6ff5640 10 client.17118 handle_command_reply: tid=0 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 client.17118 shutdown 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 msgr2=0x562341a8efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40314e0 tx=0x7f6cd403d760 comp rx=0 tx=0).stop 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 shutdown_connections 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341aa1ec0 0x562341aa42b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a92370 0x562341a8f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a88b40 0x562341a8efe0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 --2- 172.21.15.77:0/3732050310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6cc6ff5640 0 client.17118 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 >> 172.21.15.77:0/3732050310 conn(0x562341a50310 msgr2=0x562341a89930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 shutdown_connections 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 1 -- 172.21.15.77:0/3732050310 wait complete. 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.047+0000 7f6ceb475740 20 client.17118 trim_cache size 0 max 16384 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 Processor -- start 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 -- start start 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a92370 0x562341a8a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41acd80 con 0x562341a875d0 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41ae8c0 con 0x562341a92370 2024-09-15T22:18:32.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ceb475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41ad450 con 0x562341a88b40 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41382/0 (socket says 172.21.15.77:41382) 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 -- 172.21.15.77:0/4012272064 learned_addr learned my addr 172.21.15.77:0/4012272064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a8a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 -- 172.21.15.77:0/4012272064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 msgr2=0x562341a8a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a8a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 -- 172.21.15.77:0/4012272064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a92370 msgr2=0x562341a8a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a92370 0x562341a8a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.063+0000 7f6ce259c640 1 -- 172.21.15.77:0/4012272064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562341a8df90 con 0x562341a875d0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.064+0000 7f6ce259c640 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80093e0 tx=0x7f6cd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a50d0d3964b7b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.064+0000 7f6cc8ff9640 1 -- 172.21.15.77:0/4012272064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8013170 con 0x562341a875d0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.064+0000 7f6cc8ff9640 1 -- 172.21.15.77:0/4012272064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd80035f0 con 0x562341a875d0 2024-09-15T22:18:32.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.064+0000 7f6cc8ff9640 1 -- 172.21.15.77:0/4012272064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80206c0 con 0x562341a875d0 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- 172.21.15.77:0/4012272064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 msgr2=0x562341a8e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80093e0 tx=0x7f6cd8009260 comp rx=0 tx=0).stop 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- 172.21.15.77:0/4012272064 shutdown_connections 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a92370 0x562341a8a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a8a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- 172.21.15.77:0/4012272064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a875d0 0x562341a8e8a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- 172.21.15.77:0/4012272064 >> 172.21.15.77:0/4012272064 conn(0x562341a50310 msgr2=0x562341a91060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- 172.21.15.77:0/4012272064 shutdown_connections 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- 172.21.15.77:0/4012272064 wait complete. 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- start start 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 Processor -- start 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a96390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41ae1f0 con 0x562341a92370 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41adb20 con 0x562341a875d0 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41aad60 con 0x562341a88b40 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.065+0000 7f6ceb475740 10 client.0 _ll_register_callbacks cb 0x7f6cd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce2d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41384/0 (socket says 172.21.15.77:41384) 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 -- 172.21.15.77:0/1257538758 learned_addr learned my addr 172.21.15.77:0/1257538758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 -- 172.21.15.77:0/1257538758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 msgr2=0x562341a968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 -- 172.21.15.77:0/1257538758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 msgr2=0x562341a96390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:32.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a96390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 -- 172.21.15.77:0/1257538758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562341a92050 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ce259c640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8013c00 tx=0x7f6cd8009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e1a11eeaaed8c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6cc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80050b0 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f6cd8005250 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd80035f0 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6cd8005490 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ceb475740 10 client.17124 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.066+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562341b0ce60 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd80035f0 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6cd8020950 con 0x562341a92370 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 10 client.17124 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 10 client.17124 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:18:32.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 20 client.17124 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341b0d2d0 0x562341b0f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 4 client.17124 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623419f4d70 con 0x562341b0d2d0 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341b0d2d0 0x562341b0f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ce2d9d640 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341b0d2d0 0x562341b0f6c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f288fadd33fb65cc server_cookie=49733a893244588a in_seq=0 out_seq=0 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6cc6ff5640 10 client.17124 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6cc6ff5640 1 -- 172.21.15.77:0/1257538758 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623419f4d70 con 0x562341b0d2d0 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6cc6ff5640 10 client.17124 handle_command_reply: tid=0 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.067+0000 7f6ceb475740 1 client.17124 shutdown 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 msgr2=0x562341a96e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8013c00 tx=0x7f6cd8009770 comp rx=0 tx=0).stop 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 shutdown_connections 2024-09-15T22:18:32.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562341b0d2d0 0x562341b0f6c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6cc6ff5640 0 client.17124 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562341a92370 0x562341a96e10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562341a88b40 0x562341a968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 --2- 172.21.15.77:0/1257538758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562341a875d0 0x562341a96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 >> 172.21.15.77:0/1257538758 conn(0x562341a50310 msgr2=0x562341a824e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 shutdown_connections 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 1 -- 172.21.15.77:0/1257538758 wait complete. 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ceb475740 20 client.17124 trim_cache size 0 max 16384 2024-09-15T22:18:32.097 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 msgr2=0x7f6ccc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.068+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0cdb50 tx=0x7f6cdc00efd0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 msgr2=0x7f6ce41a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00eb10 tx=0x7f6cdc00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 shutdown_connections 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ccc0a33b0 0x7f6ccc0a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce4102ef0 0x7f6ce41a43b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ce4102570 0x7f6ce4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 --2- 172.21.15.77:0/853585584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce4100e20 0x7f6ce4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 >> 172.21.15.77:0/853585584 conn(0x7f6ce40fc7d0 msgr2=0x7f6ce40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 shutdown_connections 2024-09-15T22:18:32.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:32.069+0000 7f6ce909c640 1 -- 172.21.15.77:0/853585584 wait complete. 2024-09-15T22:18:32.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 400531 6.26 MB/sec execute 161 sec latency 66.109 ms 2024-09-15T22:18:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:32 smithi077 ceph-mon[30980]: pgmap v186: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 545 KiB/s wr, 23 op/s 2024-09-15T22:18:32.826 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:18:32.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:32 smithi196 ceph-mon[35109]: pgmap v186: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 545 KiB/s wr, 23 op/s 2024-09-15T22:18:32.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:32 smithi117 ceph-mon[35021]: pgmap v186: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 545 KiB/s wr, 23 op/s 2024-09-15T22:18:33.119 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:33.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 402311 6.29 MB/sec execute 162 sec latency 34.909 ms 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 -- 172.21.15.77:0/323826331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c101df0 msgr2=0x7fad9c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 --2- 172.21.15.77:0/323826331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c101df0 0x7fad9c1021d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009900 tx=0x7fad8c02a410 comp rx=0 tx=0).stop 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 -- 172.21.15.77:0/323826331 shutdown_connections 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 --2- 172.21.15.77:0/323826331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c106f20 0x7fad9c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 --2- 172.21.15.77:0/323826331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c1027a0 0x7fad9c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 --2- 172.21.15.77:0/323826331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c101df0 0x7fad9c1021d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 -- 172.21.15.77:0/323826331 >> 172.21.15.77:0/323826331 conn(0x7fad9c0fd8b0 msgr2=0x7fad9c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 -- 172.21.15.77:0/323826331 shutdown_connections 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 -- 172.21.15.77:0/323826331 wait complete. 2024-09-15T22:18:33.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.606+0000 7fada1245640 1 Processor -- start 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 -- start start 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 0x7fad9c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 0x7fad9c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c111f80 con 0x7fad9c1027a0 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c111e00 con 0x7fad9c106f20 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fada1245640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c112100 con 0x7fad9c101df0 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 0x7fad9c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41412/0 (socket says 172.21.15.77:41412) 2024-09-15T22:18:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 -- 172.21.15.77:0/3563276317 learned_addr learned my addr 172.21.15.77:0/3563276317 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 0x7fad9c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 -- 172.21.15.77:0/3563276317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 msgr2=0x7fad9c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 0x7fad9c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 -- 172.21.15.77:0/3563276317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 msgr2=0x7fad9c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 0x7fad9c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9a59c640 1 -- 172.21.15.77:0/3563276317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c19d410 con 0x7fad9c1027a0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 0x7fad9c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.607+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 0x7fad9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fad9a59c640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e9f0 tx=0x7fad9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fad83fff640 1 -- 172.21.15.77:0/3563276317 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9000cd80 con 0x7fad9c1027a0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fad83fff640 1 -- 172.21.15.77:0/3563276317 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad90010910 con 0x7fad9c1027a0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c19d700 con 0x7fad9c1027a0 2024-09-15T22:18:33.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fad83fff640 1 -- 172.21.15.77:0/3563276317 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90018990 con 0x7fad9c1027a0 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.608+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1a4fe0 con 0x7fad9c1027a0 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fad83fff640 1 -- 172.21.15.77:0/3563276317 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fad90010ac0 con 0x7fad9c1027a0 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 Processor -- start 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 -- start start 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f254a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f265b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f1ba2f0 con 0x55bf7f2545d0 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f161cf0 con 0x55bf7f263780 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fada361e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f1b62b0 con 0x55bf7f255b40 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.611+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f254a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45260/0 (socket says 172.21.15.77:45260) 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328022014 learned_addr learned my addr 172.21.15.77:0/3328022014 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9a59c640 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f265b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328022014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 msgr2=0x55bf7f265b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f265b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad83fff640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 0x7fad840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328022014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 msgr2=0x55bf7f254a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f254a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328022014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf7f1463a0 con 0x55bf7f255b40 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f254a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad83fff640 1 -- 172.21.15.77:0/3563276317 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fad900d0370 con 0x7fad9c1027a0 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 0x7fad840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad94011790 tx=0x7fad94011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52c6fd8fc181cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad817fa640 1 -- 172.21.15.77:0/3328022014 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94004050 con 0x55bf7f255b40 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad817fa640 1 -- 172.21.15.77:0/3328022014 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad940046d0 con 0x55bf7f255b40 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.612+0000 7fad817fa640 1 -- 172.21.15.77:0/3328022014 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9401cac0 con 0x55bf7f255b40 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 -- 172.21.15.77:0/3328022014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 msgr2=0x55bf7f254090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 0x7fad840a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad94020000 tx=0x7fad94021000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad94011790 tx=0x7fad94011c50 comp rx=0 tx=0).stop 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 -- 172.21.15.77:0/3328022014 shutdown_connections 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f265b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f254a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 --2- 172.21.15.77:0/3328022014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f254090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 -- 172.21.15.77:0/3328022014 >> 172.21.15.77:0/3328022014 conn(0x55bf7f21d310 msgr2=0x55bf7f250c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 -- 172.21.15.77:0/3328022014 shutdown_connections 2024-09-15T22:18:33.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.613+0000 7fada361e740 1 -- 172.21.15.77:0/3328022014 wait complete. 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 -- start start 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 Processor -- start 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f2603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f260a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f266480 con 0x55bf7f2545d0 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f175060 con 0x55bf7f263780 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bf7f266600 con 0x55bf7f255b40 2024-09-15T22:18:33.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fada361e740 10 client.0 _ll_register_callbacks cb 0x55bf7eec88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f2603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f260a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41422/0 (socket says 172.21.15.77:41422) 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9ad9d640 1 -- 172.21.15.77:0/2823070438 learned_addr learned my addr 172.21.15.77:0/2823070438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 -- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 msgr2=0x55bf7f2603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f2603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 -- 172.21.15.77:0/2823070438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 msgr2=0x55bf7f2630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9b59e640 1 -- 172.21.15.77:0/2823070438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7ef2b3c0 con 0x55bf7f263780 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.614+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad9b59e640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f260a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad9400f790 tx=0x7fad94003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895fdd13c3d93a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9401ca30 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad940046d0 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fad9402e070 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad9403f050 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fada361e740 10 client.26312 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf7f271ed0 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad940046d0 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.615+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fad940289a0 con 0x55bf7f263780 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 10 client.26312 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 10 client.26312 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 20 client.26312 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:33.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f273220 0x55bf7f275610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 4 client.26312 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf7f1c7860 con 0x55bf7f273220 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f273220 0x55bf7f275610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fad7eff5640 10 client.26312 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.616+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f273220 0x55bf7f275610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=592b43d16a37311f server_cookie=c449725bd2c33ac7 in_seq=0 out_seq=0 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fad7eff5640 1 -- 172.21.15.77:0/2823070438 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf7f1c7860 con 0x55bf7f273220 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fad7eff5640 10 client.26312 handle_command_reply: tid=0 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 client.26312 shutdown 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 msgr2=0x55bf7f260a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f260a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad9400f790 tx=0x7fad94003e20 comp rx=0 tx=0).stop 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 shutdown_connections 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f273220 0x55bf7f275610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f263780 0x55bf7f260a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f2603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 --2- 172.21.15.77:0/2823070438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f2545d0 0x55bf7f2630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fad7eff5640 0 client.26312 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 >> 172.21.15.77:0/2823070438 conn(0x55bf7f21d310 msgr2=0x55bf7f256960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 shutdown_connections 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 1 -- 172.21.15.77:0/2823070438 wait complete. 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.617+0000 7fada361e740 20 client.26312 trim_cache size 0 max 16384 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.630+0000 7fada361e740 1 Processor -- start 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.630+0000 7fada361e740 1 -- start start 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f255b40 0x55bf7f262360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1acce0 con 0x55bf7f263780 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ae820 con 0x55bf7f255b40 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fada361e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ad3b0 con 0x55bf7f2545d0 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41430/0 (socket says 172.21.15.77:41430) 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45276/0 (socket says 172.21.15.77:45276) 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328994249 learned_addr learned my addr 172.21.15.77:0/3328994249 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9a59c640 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f255b40 0x55bf7f262360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328994249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 msgr2=0x55bf7f26f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328994249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f255b40 msgr2=0x55bf7f262360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f255b40 0x55bf7f262360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 -- 172.21.15.77:0/3328994249 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf7f251960 con 0x55bf7f263780 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad9b59e640 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fad94028da0 tx=0x7fad94011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8a3a157baf71f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad80ff9640 1 -- 172.21.15.77:0/3328994249 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94026430 con 0x55bf7f263780 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad80ff9640 1 -- 172.21.15.77:0/3328994249 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad94036070 con 0x55bf7f263780 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.631+0000 7fad80ff9640 1 -- 172.21.15.77:0/3328994249 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94005290 con 0x55bf7f263780 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- 172.21.15.77:0/3328994249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 msgr2=0x55bf7f27ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fad94028da0 tx=0x7fad94011c50 comp rx=0 tx=0).stop 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- 172.21.15.77:0/3328994249 shutdown_connections 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f27ae20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f255b40 0x55bf7f262360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 --2- 172.21.15.77:0/3328994249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f2545d0 0x55bf7f26f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- 172.21.15.77:0/3328994249 >> 172.21.15.77:0/3328994249 conn(0x55bf7f21d310 msgr2=0x55bf7f261c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- 172.21.15.77:0/3328994249 shutdown_connections 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- 172.21.15.77:0/3328994249 wait complete. 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.632+0000 7fada361e740 1 -- start start 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 Processor -- start 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 0x55bf7f26bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f267320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ae150 con 0x55bf7f263780 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ada80 con 0x55bf7f2545d0 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1aacc0 con 0x55bf7f255b40 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fada361e740 10 client.0 _ll_register_callbacks cb 0x7fad9401f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:33.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41438/0 (socket says 172.21.15.77:41438) 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 -- 172.21.15.77:0/1647072364 learned_addr learned my addr 172.21.15.77:0/1647072364 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9a59c640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f267320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 0x55bf7f26bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 -- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 msgr2=0x55bf7f267320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f267320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 -- 172.21.15.77:0/1647072364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 msgr2=0x55bf7f26bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 0x55bf7f26bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 -- 172.21.15.77:0/1647072364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7f268040 con 0x55bf7f263780 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 0x55bf7f26bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad9b59e640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fad94009f80 tx=0x7fad9401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6124903eca635974 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.633+0000 7fad7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:33.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94011280 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fad94005210 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad94036070 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad94051050 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 10 client.17142 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf7f2dde00 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad94036070 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fad94010430 con 0x55bf7f263780 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 10 client.17142 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 10 client.17142 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 20 client.17142 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f2de2d0 0x55bf7f2e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 4 client.17142 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:33.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf7f1c1d70 con 0x55bf7f2de2d0 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.634+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f2de2d0 0x55bf7f2e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fad7eff5640 10 client.17142 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fad9ad9d640 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f2de2d0 0x55bf7f2e06c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3409db47960fb1d8 server_cookie=c26709a0ebb941ca in_seq=0 out_seq=0 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fad7eff5640 1 -- 172.21.15.77:0/1647072364 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf7f1c1d70 con 0x55bf7f2de2d0 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fad7eff5640 10 client.17142 handle_command_reply: tid=0 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 client.17142 shutdown 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 msgr2=0x55bf7f267860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fad94009f80 tx=0x7fad9401cc60 comp rx=0 tx=0).stop 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 shutdown_connections 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55bf7f2de2d0 0x55bf7f2e06c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bf7f263780 0x55bf7f267860 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf7f255b40 0x55bf7f267320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 --2- 172.21.15.77:0/1647072364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf7f2545d0 0x55bf7f26bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 >> 172.21.15.77:0/1647072364 conn(0x55bf7f21d310 msgr2=0x55bf7f262470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fad7eff5640 0 client.17142 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:18:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 shutdown_connections 2024-09-15T22:18:33.675 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:18:33.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 1 -- 172.21.15.77:0/1647072364 wait complete. 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.635+0000 7fada361e740 20 client.17142 trim_cache size 0 max 16384 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 msgr2=0x7fad840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 0x7fad840a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad94020000 tx=0x7fad94021000 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 msgr2=0x7fad9c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e9f0 tx=0x7fad9000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 shutdown_connections 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fad840a33b0 0x7fad840a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9c106f20 0x7fad9c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad9c1027a0 0x7fad9c1989a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 --2- 172.21.15.77:0/3563276317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c101df0 0x7fad9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 >> 172.21.15.77:0/3563276317 conn(0x7fad9c0fd8b0 msgr2=0x7fad9c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 shutdown_connections 2024-09-15T22:18:33.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:33.636+0000 7fada1245640 1 -- 172.21.15.77:0/3563276317 wait complete. 2024-09-15T22:18:34.449 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:18:34.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 404827 6.33 MB/sec execute 163 sec latency 15.936 ms 2024-09-15T22:18:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:34.760 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:34 smithi077 ceph-mon[30980]: pgmap v187: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:34.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:34 smithi196 ceph-mon[35109]: pgmap v187: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:34.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:34 smithi117 ceph-mon[35021]: pgmap v187: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- 172.21.15.77:0/2534269513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778105de0 msgr2=0x7f977810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 --2- 172.21.15.77:0/2534269513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778105de0 0x7f977810d960 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ca0 tx=0x7f977002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- 172.21.15.77:0/2534269513 shutdown_connections 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 --2- 172.21.15.77:0/2534269513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778105de0 0x7f977810d960 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 --2- 172.21.15.77:0/2534269513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f97781056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 --2- 172.21.15.77:0/2534269513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778101470 0x7f9778101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- 172.21.15.77:0/2534269513 >> 172.21.15.77:0/2534269513 conn(0x7f9778077230 msgr2=0x7f97780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- 172.21.15.77:0/2534269513 shutdown_connections 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- 172.21.15.77:0/2534269513 wait complete. 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 Processor -- start 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.252+0000 7f977d844640 1 -- start start 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 0x7f977810eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778105de0 0x7f977810f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9778111f40 con 0x7f9778101470 2024-09-15T22:18:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9778111dc0 con 0x7f9778105de0 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f977d844640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97781120c0 con 0x7f9778101e20 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 0x7f977810eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 0x7f977810eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41456/0 (socket says 172.21.15.77:41456) 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f9776ffd640 1 -- 172.21.15.77:0/2836697993 learned_addr learned my addr 172.21.15.77:0/2836697993 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45308/0 (socket says 172.21.15.77:45308) 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f97777fe640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778105de0 0x7f977810f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 -- 172.21.15.77:0/2836697993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778105de0 msgr2=0x7f977810f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778105de0 0x7f977810f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 -- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 msgr2=0x7f977810eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 0x7f977810eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.253+0000 7f976e7fc640 1 -- 172.21.15.77:0/2836697993 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97781a68a0 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f976000e9f0 tx=0x7f976000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f977c842640 1 -- 172.21.15.77:0/2836697993 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976000ccc0 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97781a6b90 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f977c842640 1 -- 172.21.15.77:0/2836697993 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760005020 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f977c842640 1 -- 172.21.15.77:0/2836697993 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600049e0 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.254+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97781a7080 con 0x7f9778101e20 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 Processor -- start 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 -- start start 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e77c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b0e6d52f0 con 0x557b0e77a370 2024-09-15T22:18:35.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b0e67ccf0 con 0x557b0e76f5d0 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977fc1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b0e6d12b0 con 0x557b0e770b40 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f977c842640 1 -- 172.21.15.77:0/2836697993 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9760005560 con 0x7f9778101e20 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f97777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f97777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45322/0 (socket says 172.21.15.77:45322) 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49550/0 (socket says 172.21.15.77:49550) 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.257+0000 7f97777fe640 1 -- 172.21.15.77:0/1833115465 learned_addr learned my addr 172.21.15.77:0/1833115465 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f976e7fc640 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e77c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 -- 172.21.15.77:0/1833115465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 msgr2=0x557b0e76fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 -- 172.21.15.77:0/1833115465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 msgr2=0x557b0e77c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e77c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 -- 172.21.15.77:0/1833115465 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0e6613a0 con 0x557b0e770b40 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f9776ffd640 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f97777fe640 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ca0 tx=0x7f9770005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e304fd0c2cf20478 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f976dffb640 1 -- 172.21.15.77:0/1833115465 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770031db0 con 0x557b0e770b40 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f976dffb640 1 -- 172.21.15.77:0/1833115465 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770036040 con 0x557b0e770b40 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f976dffb640 1 -- 172.21.15.77:0/1833115465 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97700042a0 con 0x557b0e770b40 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.258+0000 7f977c842640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 0x7f97640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 0x7f97640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977c842640 1 -- 172.21.15.77:0/2836697993 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f97600cff90 con 0x7f9778101e20 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 0x7f97640a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x557b0e76cac0 tx=0x7f9768005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 -- 172.21.15.77:0/1833115465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 msgr2=0x557b0e76f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9770009ca0 tx=0x7f9770005f10 comp rx=0 tx=0).stop 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 -- 172.21.15.77:0/1833115465 shutdown_connections 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e77c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e76f5d0 0x557b0e76fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 --2- 172.21.15.77:0/1833115465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e770b40 0x557b0e76f090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 -- 172.21.15.77:0/1833115465 >> 172.21.15.77:0/1833115465 conn(0x557b0e738310 msgr2=0x557b0e76bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 -- 172.21.15.77:0/1833115465 shutdown_connections 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.259+0000 7f977fc1d740 1 -- 172.21.15.77:0/1833115465 wait complete. 2024-09-15T22:18:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.260+0000 7f977fc1d740 1 -- start start 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.260+0000 7f977fc1d740 1 Processor -- start 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 0x557b0e776fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 0x557b0e777530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b0e77ce90 con 0x557b0e770b40 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b0e690060 con 0x557b0e77a370 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b0e77d010 con 0x557b0e76f5d0 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f977fc1d740 10 client.0 _ll_register_callbacks cb 0x557b0e3e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 0x557b0e776fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f97777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 0x557b0e777530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45338/0 (socket says 172.21.15.77:45338) 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f9776ffd640 1 -- 172.21.15.77:0/2319367184 learned_addr learned my addr 172.21.15.77:0/2319367184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 -- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 msgr2=0x557b0e779d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 -- 172.21.15.77:0/2319367184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 msgr2=0x557b0e777530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 0x557b0e777530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f976e7fc640 1 -- 172.21.15.77:0/2319367184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0e4463c0 con 0x557b0e770b40 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f97777fe640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 0x557b0e777530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.261+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 0x557b0e776fb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f97600cd340 tx=0x7f976009ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c3c16ac4926c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f9756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976001c050 con 0x557b0e770b40 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f97600a46b0 con 0x557b0e770b40 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9770036040 con 0x557b0e770b40 2024-09-15T22:18:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f97600cd7d0 con 0x557b0e770b40 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f977fc1d740 10 client.17154 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b0e788970 con 0x557b0e770b40 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.262+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770036040 con 0x557b0e770b40 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9760004bc0 con 0x557b0e770b40 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 10 client.17154 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 20 client.17154 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e789dd0 0x557b0e78c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 4 client.17154 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b0e6e2860 con 0x557b0e789dd0 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e789dd0 0x557b0e78c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f9756ffd640 10 client.17154 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:18:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.263+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e789dd0 0x557b0e78c1c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73c322a4d12431af server_cookie=cf20e40b3436f121 in_seq=0 out_seq=0 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f9756ffd640 1 -- 172.21.15.77:0/2319367184 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b0e6e2860 con 0x557b0e789dd0 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f9756ffd640 10 client.17154 handle_command_reply: tid=0 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 client.17154 shutdown 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 msgr2=0x557b0e776fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 0x557b0e776fb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f97600cd340 tx=0x7f976009ba10 comp rx=0 tx=0).stop 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 shutdown_connections 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e789dd0 0x557b0e78c1c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e77a370 0x557b0e777530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e770b40 0x557b0e776fb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.264+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2319367184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e779d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.265+0000 7f9756ffd640 0 client.17154 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.265+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 >> 172.21.15.77:0/2319367184 conn(0x557b0e738310 msgr2=0x557b0e771960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.265+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 shutdown_connections 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.265+0000 7f977fc1d740 1 -- 172.21.15.77:0/2319367184 wait complete. 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.265+0000 7f977fc1d740 20 client.17154 trim_cache size 0 max 16384 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 Processor -- start 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 -- start start 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e78bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e791a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b0f80 con 0x557b0e77a370 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b2ac0 con 0x557b0e775f90 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.279+0000 7f977fc1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b1650 con 0x557b0e76f5d0 2024-09-15T22:18:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f976e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e78bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45348/0 (socket says 172.21.15.77:45348) 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 -- 172.21.15.77:0/3662277925 learned_addr learned my addr 172.21.15.77:0/3662277925 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f97777fe640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e791a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 -- 172.21.15.77:0/3662277925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 msgr2=0x557b0e791a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e791a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 -- 172.21.15.77:0/3662277925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 msgr2=0x557b0e78bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e78bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 -- 172.21.15.77:0/3662277925 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0e7703a0 con 0x557b0e76f5d0 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f976e7fc640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e78bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f97777fe640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e791a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f9776ffd640 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9768009fd0 tx=0x7f976801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973b986eb63d43c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f976d7fa640 1 -- 172.21.15.77:0/3662277925 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976802b4e0 con 0x557b0e76f5d0 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f976d7fa640 1 -- 172.21.15.77:0/3662277925 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976802bb60 con 0x557b0e76f5d0 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.280+0000 7f976d7fa640 1 -- 172.21.15.77:0/3662277925 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976801f400 con 0x557b0e76f5d0 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 -- 172.21.15.77:0/3662277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 msgr2=0x557b0e786130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9768009fd0 tx=0x7f976801b000 comp rx=0 tx=0).stop 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 -- 172.21.15.77:0/3662277925 shutdown_connections 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e791a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e77a370 0x557b0e78bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 --2- 172.21.15.77:0/3662277925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e76f5d0 0x557b0e786130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 -- 172.21.15.77:0/3662277925 >> 172.21.15.77:0/3662277925 conn(0x557b0e738310 msgr2=0x557b0e775880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 -- 172.21.15.77:0/3662277925 shutdown_connections 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.281+0000 7f977fc1d740 1 -- 172.21.15.77:0/3662277925 wait complete. 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 -- start start 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 Processor -- start 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e76f5d0 0x557b0e77e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e77e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b23f0 con 0x557b0e76f5d0 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f977fc1d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b1d20 con 0x557b0e775f90 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f977fc1d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97781aef60 con 0x557b0e77a370 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f977fc1d740 10 client.0 _ll_register_callbacks cb 0x7f97600053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45352/0 (socket says 172.21.15.77:45352) 2024-09-15T22:18:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 -- 172.21.15.77:0/2524848476 learned_addr learned my addr 172.21.15.77:0/2524848476 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f976e7fc640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e77e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.282+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e76f5d0 0x557b0e77e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 -- 172.21.15.77:0/2524848476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 msgr2=0x557b0e77e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e77e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 -- 172.21.15.77:0/2524848476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e76f5d0 msgr2=0x557b0e77e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e76f5d0 0x557b0e77e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 -- 172.21.15.77:0/2524848476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0e7f3cb0 con 0x557b0e77a370 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976802bb60 con 0x557b0e77a370 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f97777fe640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9770031940 tx=0x7f97700318d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95d0b1cf1987cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f9756ffd640 10 client.26242 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.283+0000 7f9756ffd640 1 -- 172.21.15.77:0/2524848476 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977004a040 con 0x557b0e77a370 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f9756ffd640 1 -- 172.21.15.77:0/2524848476 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f9770003b80 con 0x557b0e77a370 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f9756ffd640 1 -- 172.21.15.77:0/2524848476 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9770032ed0 con 0x557b0e77a370 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 10 client.26242 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=34}) -- 0x557b0e7f4eb0 con 0x557b0e77a370 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f9756ffd640 1 -- 172.21.15.77:0/2524848476 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9770043070 con 0x557b0e77a370 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 10 client.26242 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 10 client.26242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 20 client.26242 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e7f5240 0x557b0e7f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 4 client.26242 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b0e6dcd70 con 0x557b0e7f5240 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.284+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e7f5240 0x557b0e7f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.285+0000 7f9756ffd640 10 client.26242 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.285+0000 7f9776ffd640 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e7f5240 0x557b0e7f7630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=602cbd1331b10b63 server_cookie=d00df54fc8943f07 in_seq=0 out_seq=0 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f9756ffd640 1 -- 172.21.15.77:0/2524848476 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b0e6dcd70 con 0x557b0e7f5240 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f9756ffd640 10 client.26242 handle_command_reply: tid=0 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 client.26242 shutdown 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 msgr2=0x557b0e77ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9770031940 tx=0x7f97700318d0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.311 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 shutdown_connections 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b0e7f5240 0x557b0e7f7630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b0e77a370 0x557b0e77ed50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b0e775f90 0x557b0e77e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 --2- 172.21.15.77:0/2524848476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b0e76f5d0 0x557b0e77e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 >> 172.21.15.77:0/2524848476 conn(0x557b0e738310 msgr2=0x557b0e778810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f9756ffd640 0 client.26242 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 shutdown_connections 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 1 -- 172.21.15.77:0/2524848476 wait complete. 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.286+0000 7f977fc1d740 20 client.26242 trim_cache size 0 max 16384 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 msgr2=0x7f97640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 0x7f97640a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x557b0e76cac0 tx=0x7f9768005f90 comp rx=0 tx=0).stop 2024-09-15T22:18:35.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 msgr2=0x7f977810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f976000e9f0 tx=0x7f976000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 shutdown_connections 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97640a33b0 0x7f97640a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9778105de0 0x7f977810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9778101e20 0x7f977810f3f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 --2- 172.21.15.77:0/2836697993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9778101470 0x7f977810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 >> 172.21.15.77:0/2836697993 conn(0x7f9778077230 msgr2=0x7f97781045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 shutdown_connections 2024-09-15T22:18:35.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:35.287+0000 7f977d844640 1 -- 172.21.15.77:0/2836697993 wait complete. 2024-09-15T22:18:35.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 407174 6.36 MB/sec execute 164 sec latency 30.327 ms 2024-09-15T22:18:36.023 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:18:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:36.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 409359 6.40 MB/sec execute 165 sec latency 15.419 ms 2024-09-15T22:18:36.638 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:36 smithi077 ceph-mon[30980]: pgmap v188: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:36.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:36 smithi196 ceph-mon[35109]: pgmap v188: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:36.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:36 smithi117 ceph-mon[35021]: pgmap v188: 129 pgs: 129 active+clean; 242 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 504 KiB/s wr, 17 op/s 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 -- 172.21.15.77:0/2701964481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c102d90 msgr2=0x7fe57c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 --2- 172.21.15.77:0/2701964481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c102d90 0x7fe57c103170 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009930 tx=0x7fe56c02a420 comp rx=0 tx=0).stop 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 -- 172.21.15.77:0/2701964481 shutdown_connections 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 --2- 172.21.15.77:0/2701964481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 0x7fe57c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 --2- 172.21.15.77:0/2701964481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c103740 0x7fe57c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 --2- 172.21.15.77:0/2701964481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c102d90 0x7fe57c103170 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 -- 172.21.15.77:0/2701964481 >> 172.21.15.77:0/2701964481 conn(0x7fe57c0fc7d0 msgr2=0x7fe57c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 -- 172.21.15.77:0/2701964481 shutdown_connections 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.854+0000 7fe5828a0640 1 -- 172.21.15.77:0/2701964481 wait complete. 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 Processor -- start 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 -- start start 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c102d90 0x7fe57c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 0x7fe57c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c112090 con 0x7fe57c103740 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c111f10 con 0x7fe57c108290 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe5828a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c112210 con 0x7fe57c102d90 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c102d90 0x7fe57c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41514/0 (socket says 172.21.15.77:41514) 2024-09-15T22:18:36.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 -- 172.21.15.77:0/1962547061 learned_addr learned my addr 172.21.15.77:0/1962547061 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 0x7fe57c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 -- 172.21.15.77:0/1962547061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c102d90 msgr2=0x7fe57c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c102d90 0x7fe57c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 -- 172.21.15.77:0/1962547061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 msgr2=0x7fe57c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 0x7fe57c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.855+0000 7fe573fff640 1 -- 172.21.15.77:0/1962547061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c19d520 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe573fff640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9f0 tx=0x7fe56400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe579ffb640 1 -- 172.21.15.77:0/1962547061 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400ccc0 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe579ffb640 1 -- 172.21.15.77:0/1962547061 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564005500 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c19d810 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe579ffb640 1 -- 172.21.15.77:0/1962547061 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564010690 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.856+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c1a50f0 con 0x7fe57c103740 2024-09-15T22:18:36.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.858+0000 7fe579ffb640 1 -- 172.21.15.77:0/1962547061 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe5640046c0 con 0x7fe57c103740 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 Processor -- start 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 -- start start 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d501090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe579ffb640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 0x7fe5680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56203d4672f0 con 0x56203d50c370 2024-09-15T22:18:36.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56203d40ecf0 con 0x56203d502b40 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe584c79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56203d4632b0 con 0x56203d5015d0 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe580e3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d501090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe573fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45378/0 (socket says 172.21.15.77:45378) 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe580e3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d501090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49614/0 (socket says 172.21.15.77:49614) 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 -- 172.21.15.77:0/2779960951 learned_addr learned my addr 172.21.15.77:0/2779960951 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 0x7fe5680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 -- 172.21.15.77:0/2779960951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 msgr2=0x56203d501090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d501090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 -- 172.21.15.77:0/2779960951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 msgr2=0x56203d50e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe57bfff640 1 -- 172.21.15.77:0/2779960951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203d3f33a0 con 0x56203d5015d0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.859+0000 7fe579ffb640 1 -- 172.21.15.77:0/1962547061 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe5640d0330 con 0x7fe57c103740 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.860+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 0x7fe5680a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe574014bb0 tx=0x7fe574007520 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.860+0000 7fe57bfff640 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009930 tx=0x7fe56c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deaefe02ea946ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.860+0000 7fe5727fc640 1 -- 172.21.15.77:0/2779960951 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c040070 con 0x56203d5015d0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.860+0000 7fe5727fc640 1 -- 172.21.15.77:0/2779960951 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c0319f0 con 0x56203d5015d0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.860+0000 7fe5727fc640 1 -- 172.21.15.77:0/2779960951 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c035040 con 0x56203d5015d0 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- 172.21.15.77:0/2779960951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 msgr2=0x56203d501a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009930 tx=0x7fe56c005e80 comp rx=0 tx=0).stop 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- 172.21.15.77:0/2779960951 shutdown_connections 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d501a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- 172.21.15.77:0/2779960951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d501090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- 172.21.15.77:0/2779960951 >> 172.21.15.77:0/2779960951 conn(0x56203d4ca310 msgr2=0x56203d4fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- 172.21.15.77:0/2779960951 shutdown_connections 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- 172.21.15.77:0/2779960951 wait complete. 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 -- start start 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 Processor -- start 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d508f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.861+0000 7fe584c79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d5094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe584c79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56203d513420 con 0x56203d5015d0 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe584c79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56203d422060 con 0x56203d502b40 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe584c79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56203d5135a0 con 0x56203d50c370 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe584c79740 10 client.0 _ll_register_callbacks cb 0x56203d1758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41536/0 (socket says 172.21.15.77:41536) 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe57bfff640 1 -- 172.21.15.77:0/1628452820 learned_addr learned my addr 172.21.15.77:0/1628452820 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d508f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d5094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 -- 172.21.15.77:0/1628452820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 msgr2=0x56203d5094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d5094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 -- 172.21.15.77:0/1628452820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 msgr2=0x56203d50bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 -- 172.21.15.77:0/1628452820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203d1d83c0 con 0x56203d502b40 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe573fff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d508f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5640cd740 tx=0x7fe56409bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0752653547d455d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe55b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:18:36.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640d0170 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe5640a4ab0 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56c0319f0 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.862+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5640ac340 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 10 client.26345 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56203d50edd0 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c0319f0 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe5640b5120 con 0x56203d502b40 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 10 client.26345 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 10 client.26345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 20 client.26345 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d510120 0x56203d512570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 4 client.26345 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56203d474860 con 0x56203d510120 2024-09-15T22:18:36.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.863+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d510120 0x56203d512570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.864+0000 7fe55b7fe640 10 client.26345 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:18:36.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.864+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d510120 0x56203d512570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81fb7a9fc273617a server_cookie=b05cd8cdfdcad7a4 in_seq=0 out_seq=0 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.864+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1628452820 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56203d474860 con 0x56203d510120 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.864+0000 7fe55b7fe640 10 client.26345 handle_command_reply: tid=0 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.864+0000 7fe584c79740 1 client.26345 shutdown 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 msgr2=0x56203d508f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d508f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5640cd740 tx=0x7fe56409bd60 comp rx=0 tx=0).stop 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 shutdown_connections 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d510120 0x56203d512570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d5094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d508f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe55b7fe640 0 client.26345 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:18:36.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 --2- 172.21.15.77:0/1628452820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d5015d0 0x56203d50bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 >> 172.21.15.77:0/1628452820 conn(0x56203d4ca310 msgr2=0x56203d503960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 shutdown_connections 2024-09-15T22:18:36.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 1 -- 172.21.15.77:0/1628452820 wait complete. 2024-09-15T22:18:36.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.865+0000 7fe584c79740 20 client.26345 trim_cache size 0 max 16384 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.878+0000 7fe584c79740 1 Processor -- start 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.878+0000 7fe584c79740 1 -- start start 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d508850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d507f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1acc60 con 0x56203d50c370 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1ae7a0 con 0x56203d502b40 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe584c79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1ad330 con 0x56203d5015d0 2024-09-15T22:18:36.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d508850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41542/0 (socket says 172.21.15.77:41542) 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d508850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45398/0 (socket says 172.21.15.77:45398) 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 -- 172.21.15.77:0/1312081034 learned_addr learned my addr 172.21.15.77:0/1312081034 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 -- 172.21.15.77:0/1312081034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 msgr2=0x56203d508850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d508850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 -- 172.21.15.77:0/1312081034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 msgr2=0x56203d507f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe573fff640 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d507f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d507f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.879+0000 7fe580e3d640 1 -- 172.21.15.77:0/1312081034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203d4fe960 con 0x56203d50c370 2024-09-15T22:18:36.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.880+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe574003c60 tx=0x7fe5740078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24b9b20fa4022bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.880+0000 7fe571ffb640 1 -- 172.21.15.77:0/1312081034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574026070 con 0x56203d50c370 2024-09-15T22:18:36.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.880+0000 7fe571ffb640 1 -- 172.21.15.77:0/1312081034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5740047b0 con 0x56203d50c370 2024-09-15T22:18:36.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.880+0000 7fe571ffb640 1 -- 172.21.15.77:0/1312081034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57400fd90 con 0x56203d50c370 2024-09-15T22:18:36.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- 172.21.15.77:0/1312081034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 msgr2=0x56203d50d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe574003c60 tx=0x7fe5740078b0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- 172.21.15.77:0/1312081034 shutdown_connections 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d50c370 0x56203d50d760 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d502b40 0x56203d507f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 --2- 172.21.15.77:0/1312081034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d5015d0 0x56203d508850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- 172.21.15.77:0/1312081034 >> 172.21.15.77:0/1312081034 conn(0x56203d4ca310 msgr2=0x56203d507860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- 172.21.15.77:0/1312081034 shutdown_connections 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- 172.21.15.77:0/1312081034 wait complete. 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 -- start start 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 Processor -- start 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.881+0000 7fe584c79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d502b40 0x56203d524390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d51f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1ae0d0 con 0x56203d502b40 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1ada00 con 0x56203d5015d0 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c1aac40 con 0x56203d50c370 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 10 client.0 _ll_register_callbacks cb 0x7fe564004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49650/0 (socket says 172.21.15.77:49650) 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 -- 172.21.15.77:0/1473081494 learned_addr learned my addr 172.21.15.77:0/1473081494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe580e3d640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d51f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe573fff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d502b40 0x56203d524390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 -- 172.21.15.77:0/1473081494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 msgr2=0x56203d51f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d51f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 -- 172.21.15.77:0/1473081494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d502b40 msgr2=0x56203d524390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d502b40 0x56203d524390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 -- 172.21.15.77:0/1473081494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203d51faf0 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe57bfff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe56c031550 tx=0x7fe56c005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b359ef72dd89d567 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe55b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c040040 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fe56c0052a0 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.882+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5740047b0 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe56c067050 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 10 client.26348 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56203d586870 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5740047b0 con 0x56203d5015d0 2024-09-15T22:18:36.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe56c004770 con 0x56203d5015d0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 10 client.26348 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 10 client.26348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 20 client.26348 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d586bb0 0x56203d588fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 4 client.26348 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56203d46ed70 con 0x56203d586bb0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.883+0000 7fe573fff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d586bb0 0x56203d588fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe573fff640 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d586bb0 0x56203d588fa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2365f7aae9367e64 server_cookie=23c19785667f836b in_seq=0 out_seq=0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe55b7fe640 10 client.26348 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe55b7fe640 1 -- 172.21.15.77:0/1473081494 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56203d46ed70 con 0x56203d586bb0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe55b7fe640 10 client.26348 handle_command_reply: tid=0 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 client.26348 shutdown 2024-09-15T22:18:36.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 msgr2=0x56203d523e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe56c031550 tx=0x7fe56c005640 comp rx=0 tx=0).stop 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 shutdown_connections 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x56203d586bb0 0x56203d588fa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56203d50c370 0x56203d51f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56203d502b40 0x56203d524390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe55b7fe640 0 client.26348 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 --2- 172.21.15.77:0/1473081494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56203d5015d0 0x56203d523e50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 >> 172.21.15.77:0/1473081494 conn(0x56203d4ca310 msgr2=0x56203d50a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 shutdown_connections 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 1 -- 172.21.15.77:0/1473081494 wait complete. 2024-09-15T22:18:36.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.884+0000 7fe584c79740 20 client.26348 trim_cache size 0 max 16384 2024-09-15T22:18:36.892 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:18:36.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 msgr2=0x7fe5680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 0x7fe5680a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe574014bb0 tx=0x7fe574007520 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 msgr2=0x7fe57c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9f0 tx=0x7fe56400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 shutdown_connections 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe5680a3430 0x7fe5680a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c108290 0x7fe57c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe57c103740 0x7fe57c198ab0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 --2- 172.21.15.77:0/1962547061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe57c102d90 0x7fe57c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 >> 172.21.15.77:0/1962547061 conn(0x7fe57c0fc7d0 msgr2=0x7fe57c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 shutdown_connections 2024-09-15T22:18:36.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:36.885+0000 7fe5828a0640 1 -- 172.21.15.77:0/1962547061 wait complete. 2024-09-15T22:18:37.473 INFO:tasks.workunit.client.0.smithi077.stdout: 1 411795 6.45 MB/sec execute 166 sec latency 46.196 ms 2024-09-15T22:18:37.665 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:18:37.925 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:38.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 -- 172.21.15.77:0/1196829393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 msgr2=0x7f603010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 --2- 172.21.15.77:0/1196829393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f603010df90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6024009900 tx=0x7f602402a410 comp rx=0 tx=0).stop 2024-09-15T22:18:38.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 -- 172.21.15.77:0/1196829393 shutdown_connections 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 --2- 172.21.15.77:0/1196829393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f603010e4d0 0x7f60301108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 --2- 172.21.15.77:0/1196829393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f603010df90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 --2- 172.21.15.77:0/1196829393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60301009b0 0x7f6030100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 -- 172.21.15.77:0/1196829393 >> 172.21.15.77:0/1196829393 conn(0x7f60300fc7d0 msgr2=0x7f60300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 -- 172.21.15.77:0/1196829393 shutdown_connections 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.402+0000 7f60360a8640 1 -- 172.21.15.77:0/1196829393 wait complete. 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 Processor -- start 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 -- start start 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 0x7f603019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6030114220 con 0x7f6030101360 2024-09-15T22:18:38.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60301140a0 con 0x7f603010e4d0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f60360a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60301143a0 con 0x7f60301009b0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45432/0 (socket says 172.21.15.77:45432) 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602f7fe640 1 -- 172.21.15.77:0/2812780847 learned_addr learned my addr 172.21.15.77:0/2812780847 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41568/0 (socket says 172.21.15.77:41568) 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.403+0000 7f602ffff640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 0x7f603019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 -- 172.21.15.77:0/2812780847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 msgr2=0x7f60301a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 -- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 msgr2=0x7f60301a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 -- 172.21.15.77:0/2812780847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019ad00 con 0x7f603010e4d0 2024-09-15T22:18:38.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602f7fe640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602effd640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602ffff640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 0x7f603019a5d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fd0 tx=0x7f602800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602cff9640 1 -- 172.21.15.77:0/2812780847 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602800ce20 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602cff9640 1 -- 172.21.15.77:0/2812780847 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6028005020 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603019aff0 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f602cff9640 1 -- 172.21.15.77:0/2812780847 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028010720 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.404+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a72e0 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.406+0000 7f602cff9640 1 -- 172.21.15.77:0/2812780847 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6028005540 con 0x7f603010e4d0 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 Processor -- start 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 -- start start 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602cff9640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 0x7f60180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49866a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c49873760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c497cc2f0 con 0x560c49867b40 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c49773cf0 con 0x560c49871370 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f6038481740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c497c82b0 con 0x560c498665d0 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602cff9640 1 -- 172.21.15.77:0/2812780847 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f60280d05b0 con 0x7f603010e4d0 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41574/0 (socket says 172.21.15.77:41574) 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 -- 172.21.15.77:0/734836744 learned_addr learned my addr 172.21.15.77:0/734836744 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602ffff640 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c49873760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602effd640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 0x7f60180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 -- 172.21.15.77:0/734836744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 msgr2=0x560c49866a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:18:38.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49866a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 -- 172.21.15.77:0/734836744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 msgr2=0x560c49873760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c49873760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.407+0000 7f602f7fe640 1 -- 172.21.15.77:0/734836744 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c497583a0 con 0x560c49867b40 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f602f7fe640 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6020011650 tx=0x7f6020009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506f3c5dac0ca4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f6015ffb640 1 -- 172.21.15.77:0/734836744 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602001e070 con 0x560c49867b40 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f602effd640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 0x7f60180a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6024009230 tx=0x7f60240058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f6015ffb640 1 -- 172.21.15.77:0/734836744 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602000fba0 con 0x560c49867b40 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f6015ffb640 1 -- 172.21.15.77:0/734836744 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020019d00 con 0x560c49867b40 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f6038481740 1 -- 172.21.15.77:0/734836744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 msgr2=0x560c49866090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.408+0000 7f6038481740 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6020011650 tx=0x7f6020009f90 comp rx=0 tx=0).stop 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- 172.21.15.77:0/734836744 shutdown_connections 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c49873760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49866a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- 172.21.15.77:0/734836744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c49866090 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- 172.21.15.77:0/734836744 >> 172.21.15.77:0/734836744 conn(0x560c4982f310 msgr2=0x560c49862c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- 172.21.15.77:0/734836744 shutdown_connections 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- 172.21.15.77:0/734836744 wait complete. 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- start start 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 Processor -- start 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c498665d0 0x560c49870d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c4986e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c49874070 con 0x560c49867b40 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c49787060 con 0x560c498665d0 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.409+0000 7f6038481740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c498741f0 con 0x560c49871370 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f6038481740 10 client.0 _ll_register_callbacks cb 0x560c494da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:38.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c498665d0 0x560c49870d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41582/0 (socket says 172.21.15.77:41582) 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 -- 172.21.15.77:0/1550829029 learned_addr learned my addr 172.21.15.77:0/1550829029 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602ffff640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c4986e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 -- 172.21.15.77:0/1550829029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 msgr2=0x560c4986e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c4986e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 -- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c498665d0 msgr2=0x560c49870d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c498665d0 0x560c49870d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 -- 172.21.15.77:0/1550829029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4953d3c0 con 0x560c49867b40 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602ffff640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c4986e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f602effd640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6024009900 tx=0x7f6024009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6d20839b32731c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f600effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:38.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602404c030 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f60240043d0 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602000fba0 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6024059050 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f6038481740 10 client.17190 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.410+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c4987faa0 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602000fba0 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6024048750 con 0x560c49867b40 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 10 client.17190 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 20 client.17190 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:38.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c49880fb0 0x560c49883410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 4 client.17190 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c497d9860 con 0x560c49880fb0 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f602f7fe640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c49880fb0 0x560c49883410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f600effd640 10 client.17190 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.411+0000 7f602f7fe640 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c49880fb0 0x560c49883410 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4322119bd8215b server_cookie=9eee3cd5da01866d in_seq=0 out_seq=0 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f600effd640 1 -- 172.21.15.77:0/1550829029 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c497d9860 con 0x560c49880fb0 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f600effd640 10 client.17190 handle_command_reply: tid=0 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 client.17190 shutdown 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 msgr2=0x560c4986dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6024009900 tx=0x7f6024009260 comp rx=0 tx=0).stop 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 shutdown_connections 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c49880fb0 0x560c49883410 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c4986e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986dfb0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f600effd640 0 client.17190 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 --2- 172.21.15.77:0/1550829029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c498665d0 0x560c49870d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 >> 172.21.15.77:0/1550829029 conn(0x560c4982f310 msgr2=0x560c49868960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 shutdown_connections 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.412+0000 7f6038481740 1 -- 172.21.15.77:0/1550829029 wait complete. 2024-09-15T22:18:38.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.413+0000 7f6038481740 20 client.17190 trim_cache size 0 max 16384 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.426+0000 7f6038481740 1 Processor -- start 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.426+0000 7f6038481740 1 -- start start 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49867b40 0x560c49869660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c49888ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b1120 con 0x560c498665d0 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b2c60 con 0x560c49867b40 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f6038481740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b17f0 con 0x560c49871370 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41598/0 (socket says 172.21.15.77:41598) 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 -- 172.21.15.77:0/4179345979 learned_addr learned my addr 172.21.15.77:0/4179345979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602effd640 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49867b40 0x560c49869660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602ffff640 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c49888ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 -- 172.21.15.77:0/4179345979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 msgr2=0x560c49888ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c49888ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 -- 172.21.15.77:0/4179345979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49867b40 msgr2=0x560c49869660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49867b40 0x560c49869660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 -- 172.21.15.77:0/4179345979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c49863950 con 0x560c498665d0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.427+0000 7f602f7fe640 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6020005fd0 tx=0x7f60200196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666f17667533173 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f60157fa640 1 -- 172.21.15.77:0/4179345979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602001e040 con 0x560c498665d0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f60157fa640 1 -- 172.21.15.77:0/4179345979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6020019830 con 0x560c498665d0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f60157fa640 1 -- 172.21.15.77:0/4179345979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000f920 con 0x560c498665d0 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 -- 172.21.15.77:0/4179345979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 msgr2=0x560c49869120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6020005fd0 tx=0x7f60200196c0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 -- 172.21.15.77:0/4179345979 shutdown_connections 2024-09-15T22:18:38.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c49871370 0x560c49888ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49867b40 0x560c49869660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 --2- 172.21.15.77:0/4179345979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c498665d0 0x560c49869120 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 -- 172.21.15.77:0/4179345979 >> 172.21.15.77:0/4179345979 conn(0x560c4982f310 msgr2=0x560c49870030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 -- 172.21.15.77:0/4179345979 shutdown_connections 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.428+0000 7f6038481740 1 -- 172.21.15.77:0/4179345979 wait complete. 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 -- start start 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 Processor -- start 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c4986b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b2590 con 0x560c49867b40 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b1ec0 con 0x560c49871370 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60301af100 con 0x560c498665d0 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.429+0000 7f6038481740 10 client.0 _ll_register_callbacks cb 0x7f6024040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:38.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45462/0 (socket says 172.21.15.77:45462) 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602f7fe640 1 -- 172.21.15.77:0/9428063 learned_addr learned my addr 172.21.15.77:0/9428063 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c4986b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602effd640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 -- 172.21.15.77:0/9428063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 msgr2=0x560c49879d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 -- 172.21.15.77:0/9428063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 msgr2=0x560c4986ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 -- 172.21.15.77:0/9428063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c498eacb0 con 0x560c49871370 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602f7fe640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f602ffff640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c4986b5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60280cd960 tx=0x7f602809bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5bf9ef217f4355 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:38.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f600effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280d0390 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7f602809c750 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6020019830 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f60280a08a0 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f6038481740 10 client.26366 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.430+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c498ebe60 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6020019830 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f60280b5200 con 0x560c49871370 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 10 client.26366 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 20 client.26366 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c498ec580 0x560c498ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 4 client.26366 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:38.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c497d3d70 con 0x560c498ec580 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f602f7fe640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c498ec580 0x560c498ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f600effd640 10 client.26366 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.431+0000 7f602f7fe640 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c498ec580 0x560c498ee970 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d08c8b043574f0 server_cookie=9064f33565a50d9f in_seq=0 out_seq=0 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f600effd640 1 -- 172.21.15.77:0/9428063 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c497d3d70 con 0x560c498ec580 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f600effd640 10 client.26366 handle_command_reply: tid=0 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 client.26366 shutdown 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 msgr2=0x560c4986b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c4986b5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60280cd960 tx=0x7f602809bf10 comp rx=0 tx=0).stop 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 shutdown_connections 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560c498ec580 0x560c498ee970 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c49871370 0x560c4986b5f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c49867b40 0x560c4986ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f600effd640 0 client.26366 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 --2- 172.21.15.77:0/9428063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c498665d0 0x560c49879d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 >> 172.21.15.77:0/9428063 conn(0x560c4982f310 msgr2=0x560c498614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 shutdown_connections 2024-09-15T22:18:38.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 1 -- 172.21.15.77:0/9428063 wait complete. 2024-09-15T22:18:38.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.432+0000 7f6038481740 20 client.26366 trim_cache size 0 max 16384 2024-09-15T22:18:38.447 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 msgr2=0x7f60180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 0x7f60180a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6024009230 tx=0x7f60240058d0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 msgr2=0x7f603019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 0x7f603019a5d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6028009fd0 tx=0x7f602800ef10 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 shutdown_connections 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f60180a3430 0x7f60180a58f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f603010e4d0 0x7f603019a5d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6030101360 0x7f60301a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 --2- 172.21.15.77:0/2812780847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60301009b0 0x7f60301a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 >> 172.21.15.77:0/2812780847 conn(0x7f60300fc7d0 msgr2=0x7f60300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 shutdown_connections 2024-09-15T22:18:38.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:38.433+0000 7f60360a8640 1 -- 172.21.15.77:0/2812780847 wait complete. 2024-09-15T22:18:38.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 413848 6.45 MB/sec execute 167 sec latency 50.559 ms 2024-09-15T22:18:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:38 smithi077 ceph-mon[30980]: pgmap v189: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-09-15T22:18:38.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:38 smithi196 ceph-mon[35109]: pgmap v189: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-09-15T22:18:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:38 smithi117 ceph-mon[35021]: pgmap v189: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 97 op/s 2024-09-15T22:18:39.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 415762 6.49 MB/sec execute 168 sec latency 78.750 ms 2024-09-15T22:18:39.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:39 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:39.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:39 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:39 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:40.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:18:40.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:18:40.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 418416 6.54 MB/sec execute 169 sec latency 18.199 ms 2024-09-15T22:18:40.704 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:40.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:40 smithi077 ceph-mon[30980]: pgmap v190: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-09-15T22:18:40.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:40.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:40.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:18:40.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:40 smithi196 ceph-mon[35109]: pgmap v190: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-09-15T22:18:40.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:18:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:40 smithi117 ceph-mon[35021]: pgmap v190: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-09-15T22:18:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:18:41.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.196+0000 7fa02c517640 1 -- 172.21.15.77:0/4126438307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024100e20 msgr2=0x7fa024101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.196+0000 7fa02c517640 1 --2- 172.21.15.77:0/4126438307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024100e20 0x7fa024101200 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa020009900 tx=0x7fa02002a480 comp rx=0 tx=0).stop 2024-09-15T22:18:41.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- 172.21.15.77:0/4126438307 shutdown_connections 2024-09-15T22:18:41.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- 172.21.15.77:0/4126438307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 0x7fa02410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- 172.21.15.77:0/4126438307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024102570 0x7fa0241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- 172.21.15.77:0/4126438307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024100e20 0x7fa024101200 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- 172.21.15.77:0/4126438307 >> 172.21.15.77:0/4126438307 conn(0x7fa0240fc7d0 msgr2=0x7fa0240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- 172.21.15.77:0/4126438307 shutdown_connections 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- 172.21.15.77:0/4126438307 wait complete. 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 Processor -- start 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- start start 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 0x7fa0240fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 0x7fa0240fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 0x7fa0240fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa024110f50 con 0x7fa024102570 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa024110dd0 con 0x7fa024100e20 2024-09-15T22:18:41.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.197+0000 7fa02c517640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241110d0 con 0x7fa024102ef0 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 0x7fa0240fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 0x7fa0240fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 0x7fa0240fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:45128/0 (socket says 172.21.15.77:45128) 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 0x7fa0240fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02a2b3640 1 -- 172.21.15.77:0/310488777 learned_addr learned my addr 172.21.15.77:0/310488777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 -- 172.21.15.77:0/310488777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 msgr2=0x7fa0240fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 0x7fa0240fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 -- 172.21.15.77:0/310488777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 msgr2=0x7fa0240fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 0x7fa0240fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 -- 172.21.15.77:0/310488777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0241a6980 con 0x7fa024102570 2024-09-15T22:18:41.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa029ab2640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 0x7fa0240fd1c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c970 tx=0x7fa01400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa01b7fe640 1 -- 172.21.15.77:0/310488777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014007bb0 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0241a6c70 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa01b7fe640 1 -- 172.21.15.77:0/310488777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014007d50 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa01b7fe640 1 -- 172.21.15.77:0/310488777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014018820 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.198+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0241a71b0 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.201+0000 7fa01b7fe640 1 -- 172.21.15.77:0/310488777 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa0140189c0 con 0x7fa024102570 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.201+0000 7fa02e8f0740 1 Processor -- start 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 -- start start 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa01b7fe640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 0x7fa0100a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3beef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bef480 0x559aa3bf9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559aa3b54280 con 0x559aa3bef480 2024-09-15T22:18:41.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559aa3afbb10 con 0x559aa3beeb60 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559aa3b50240 con 0x559aa3bf9d40 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02aab4640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 0x7fa0100a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35334/0 (socket says 172.21.15.77:35334) 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02aab4640 1 -- 172.21.15.77:0/1996879508 learned_addr learned my addr 172.21.15.77:0/1996879508 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa01b7fe640 1 -- 172.21.15.77:0/310488777 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa0140cffc0 con 0x7fa024102570 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3beef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.202+0000 7fa029ab2640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bef480 0x559aa3bf9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1996879508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 msgr2=0x559aa3bfc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1996879508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bef480 msgr2=0x559aa3bf9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bef480 0x559aa3bf9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1996879508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa3b081f0 con 0x559aa3beeb60 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02aab4640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02aab4640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 0x7fa0100a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa01c012c00 tx=0x7fa01c009e00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3beef40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa020009230 tx=0x7fa020005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6103c148f733714 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa018ff9640 1 -- 172.21.15.77:0/1996879508 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020040070 con 0x559aa3beeb60 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa018ff9640 1 -- 172.21.15.77:0/1996879508 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa020005510 con 0x559aa3beeb60 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.203+0000 7fa018ff9640 1 -- 172.21.15.77:0/1996879508 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020035040 con 0x559aa3beeb60 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1996879508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 msgr2=0x559aa3beef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3beef40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa020009230 tx=0x7fa020005e80 comp rx=0 tx=0).stop 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1996879508 shutdown_connections 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bef480 0x559aa3bf9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1996879508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3beef40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1996879508 >> 172.21.15.77:0/1996879508 conn(0x559aa3beb0f0 msgr2=0x559aa3beb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1996879508 shutdown_connections 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.204+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1996879508 wait complete. 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 -- start start 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 Processor -- start 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3bf6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bf7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559aa3bfc910 con 0x559aa3beeb60 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559aa3b0ef10 con 0x559aa3bef480 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.205+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559aa3bfca90 con 0x559aa3bf9d40 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02e8f0740 10 client.0 _ll_register_callbacks cb 0x559aa3ad7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bf7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa029ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3bf6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bf7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35336/0 (socket says 172.21.15.77:35336) 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa029ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:45146/0 (socket says 172.21.15.77:45146) 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02aab4640 1 -- 172.21.15.77:0/1536182863 learned_addr learned my addr 172.21.15.77:0/1536182863 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1536182863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 msgr2=0x559aa3bf7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bf7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 msgr2=0x559aa3bf7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1536182863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa38c56a0 con 0x559aa3beeb60 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa029ab2640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3bf6bc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa020006010 tx=0x7fa0200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add3d358a4642093 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa00dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02004c020 con 0x559aa3beeb60 2024-09-15T22:18:41.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa020005060 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa020005510 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.206+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa020057050 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 10 client.17208 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559aa3c084f0 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa020005510 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa020031540 con 0x559aa3beeb60 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 10 client.17208 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 20 client.17208 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c09840 0x559aa3c0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 4 client.17208 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aa3b617f0 con 0x559aa3c09840 2024-09-15T22:18:41.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa029ab2640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c09840 0x559aa3c0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa029ab2640 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c09840 0x559aa3c0bc30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2d51f6ea0e8ac9 server_cookie=23d6610b8fa197e2 in_seq=0 out_seq=0 2024-09-15T22:18:41.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.207+0000 7fa00dffb640 10 client.17208 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.208+0000 7fa00dffb640 1 -- 172.21.15.77:0/1536182863 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559aa3b617f0 con 0x559aa3c09840 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.208+0000 7fa00dffb640 10 client.17208 handle_command_reply: tid=0 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.208+0000 7fa02e8f0740 1 client.17208 shutdown 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.208+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 msgr2=0x559aa3bf6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.208+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3bf6bc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa020006010 tx=0x7fa0200048f0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 shutdown_connections 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c09840 0x559aa3c0bc30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3bf7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3bf7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1536182863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3bf6bc0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa00dffb640 0 client.17208 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:41.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 >> 172.21.15.77:0/1536182863 conn(0x559aa3beb0f0 msgr2=0x559aa3bf0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 shutdown_connections 2024-09-15T22:18:41.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1536182863 wait complete. 2024-09-15T22:18:41.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.209+0000 7fa02e8f0740 20 client.17208 trim_cache size 0 max 16384 2024-09-15T22:18:41.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 Processor -- start 2024-09-15T22:18:41.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 -- start start 2024-09-15T22:18:41.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bef480 0x559aa3bf64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bf9d40 0x559aa3c0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241acbd0 con 0x559aa3bf9d40 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241ae710 con 0x559aa3beeb60 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241ad2a0 con 0x559aa3bef480 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:45152/0 (socket says 172.21.15.77:45152) 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1843327668 learned_addr learned my addr 172.21.15.77:0/1843327668 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa02aab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bf9d40 0x559aa3c0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.228+0000 7fa029ab2640 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bef480 0x559aa3bf64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1843327668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bef480 msgr2=0x559aa3bf64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bef480 0x559aa3bf64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1843327668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bf9d40 msgr2=0x559aa3c0ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bf9d40 0x559aa3c0ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 -- 172.21.15.77:0/1843327668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa3beaf20 con 0x559aa3beeb60 2024-09-15T22:18:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa020009850 tx=0x7fa020005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b48a87dd151449 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa00ffff640 1 -- 172.21.15.77:0/1843327668 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02004c040 con 0x559aa3beeb60 2024-09-15T22:18:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa00ffff640 1 -- 172.21.15.77:0/1843327668 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa020031d90 con 0x559aa3beeb60 2024-09-15T22:18:41.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.229+0000 7fa00ffff640 1 -- 172.21.15.77:0/1843327668 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020040070 con 0x559aa3beeb60 2024-09-15T22:18:41.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1843327668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 msgr2=0x559aa3c057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa020009850 tx=0x7fa020005a20 comp rx=0 tx=0).stop 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1843327668 shutdown_connections 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3bf9d40 0x559aa3c0ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bef480 0x559aa3bf64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/1843327668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3beeb60 0x559aa3c057e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1843327668 >> 172.21.15.77:0/1843327668 conn(0x559aa3beb0f0 msgr2=0x559aa3bea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1843327668 shutdown_connections 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- 172.21.15.77:0/1843327668 wait complete. 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 -- start start 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.230+0000 7fa02e8f0740 1 Processor -- start 2024-09-15T22:18:41.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3c0c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241ae040 con 0x559aa3beeb60 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241ad970 con 0x559aa3bef480 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0241aabb0 con 0x559aa3bf9d40 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02e8f0740 10 client.0 _ll_register_callbacks cb 0x7fa014007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa029ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36986/0 (socket says 172.21.15.77:36986) 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 -- 172.21.15.77:0/2578960338 learned_addr learned my addr 172.21.15.77:0/2578960338 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa029ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:45156/0 (socket says 172.21.15.77:45156) 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02aab4640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3c0c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 -- 172.21.15.77:0/2578960338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 msgr2=0x559aa3c0c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3c0c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 -- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 msgr2=0x559aa3c0c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 -- 172.21.15.77:0/2578960338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa3c0cf20 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa029ab2640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa02a2b3640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa020031990 tx=0x7fa020005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f6cbd0f1a5a020 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa00dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.231+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020040040 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fa0200051a0 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559aa3b5bd00 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa02004c040 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 10 client.17214 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559aa3c73f20 con 0x559aa3beeb60 2024-09-15T22:18:41.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559aa3b5bd00 con 0x559aa3beeb60 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa02003c440 con 0x559aa3beeb60 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 10 client.17214 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 20 client.17214 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c743a0 0x559aa3c76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 4 client.17214 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559aa3ad8110 con 0x559aa3c743a0 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.232+0000 7fa029ab2640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c743a0 0x559aa3c76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.233+0000 7fa00dffb640 10 client.17214 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:41.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.233+0000 7fa029ab2640 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c743a0 0x559aa3c76790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890a48185fdfb8b2 server_cookie=eab986d2c9a27340 in_seq=0 out_seq=0 2024-09-15T22:18:41.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa00dffb640 1 -- 172.21.15.77:0/2578960338 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559aa3ad8110 con 0x559aa3c743a0 2024-09-15T22:18:41.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa00dffb640 10 client.17214 handle_command_reply: tid=0 2024-09-15T22:18:41.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 client.17214 shutdown 2024-09-15T22:18:41.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 msgr2=0x559aa3c0f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa020031990 tx=0x7fa020005c90 comp rx=0 tx=0).stop 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 shutdown_connections 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559aa3c743a0 0x559aa3c76790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559aa3bf9d40 0x559aa3c0c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559aa3bef480 0x559aa3c0c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 --2- 172.21.15.77:0/2578960338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559aa3beeb60 0x559aa3c0f120 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 >> 172.21.15.77:0/2578960338 conn(0x559aa3beb0f0 msgr2=0x559aa3bf0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa00dffb640 0 client.17214 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 shutdown_connections 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 1 -- 172.21.15.77:0/2578960338 wait complete. 2024-09-15T22:18:41.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.234+0000 7fa02e8f0740 20 client.17214 trim_cache size 0 max 16384 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "9dd76b70-7029-4b57-8cc7-6502a247a224", 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 msgr2=0x7fa0100a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 0x7fa0100a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa01c012c00 tx=0x7fa01c009e00 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 msgr2=0x7fa0240fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 0x7fa0240fd1c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c970 tx=0x7fa01400ce30 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 shutdown_connections 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0100a3430 0x7fa0100a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024102ef0 0x7fa0240fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa024102570 0x7fa0240fd1c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 --2- 172.21.15.77:0/310488777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa024100e20 0x7fa0240fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 >> 172.21.15.77:0/310488777 conn(0x7fa0240fc7d0 msgr2=0x7fa02406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 shutdown_connections 2024-09-15T22:18:41.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:41.235+0000 7fa02c517640 1 -- 172.21.15.77:0/310488777 wait complete. 2024-09-15T22:18:41.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 420102 6.55 MB/sec execute 170 sec latency 44.475 ms 2024-09-15T22:18:42.118 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:18:42.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:42.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 422605 6.60 MB/sec execute 171 sec latency 12.393 ms 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.683 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:42 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:42 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:18:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:42 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:42.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 -- 172.21.15.77:0/415345910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 msgr2=0x7fac58107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 --2- 172.21.15.77:0/415345910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58107b80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fac48009930 tx=0x7fac4802a420 comp rx=0 tx=0).stop 2024-09-15T22:18:42.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 -- 172.21.15.77:0/415345910 shutdown_connections 2024-09-15T22:18:42.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 --2- 172.21.15.77:0/415345910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 0x7fac5810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 --2- 172.21.15.77:0/415345910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58107b80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 --2- 172.21.15.77:0/415345910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 -- 172.21.15.77:0/415345910 >> 172.21.15.77:0/415345910 conn(0x7fac580fc7d0 msgr2=0x7fac580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 -- 172.21.15.77:0/415345910 shutdown_connections 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.920+0000 7fac5f36f640 1 -- 172.21.15.77:0/415345910 wait complete. 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 Processor -- start 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 -- start start 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 0x7fac5819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac58112090 con 0x7fac58103740 2024-09-15T22:18:42.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac58111f10 con 0x7fac58108290 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5f36f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac58112210 con 0x7fac58102d90 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37002/0 (socket says 172.21.15.77:37002) 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5d10b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5c90a640 1 -- 172.21.15.77:0/698082694 learned_addr learned my addr 172.21.15.77:0/698082694 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.921+0000 7fac5d90c640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 0x7fac5819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 -- 172.21.15.77:0/698082694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 msgr2=0x7fac5819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 0x7fac5819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 -- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 msgr2=0x7fac58198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 -- 172.21.15.77:0/698082694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5819d5c0 con 0x7fac58102d90 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5c90a640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:42.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5d10b640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58198610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9f0 tx=0x7fac5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac4e7fc640 1 -- 172.21.15.77:0/698082694 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5400ccc0 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5819d8b0 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac4e7fc640 1 -- 172.21.15.77:0/698082694 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac54005020 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac4e7fc640 1 -- 172.21.15.77:0/698082694 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540049e0 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.922+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac581a5190 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 Processor -- start 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac4e7fc640 1 -- 172.21.15.77:0/698082694 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fac54005560 con 0x7fac58102d90 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 -- start start 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d99537140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d99539a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d994922e0 con 0x557d9952c4a0 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d99439ce0 con 0x557d9952cdc0 2024-09-15T22:18:42.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.925+0000 7fac61748740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d9948e2a0 con 0x557d99537680 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d10b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d99537140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37012/0 (socket says 172.21.15.77:37012) 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 -- 172.21.15.77:0/632211420 learned_addr learned my addr 172.21.15.77:0/632211420 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5c90a640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d99539a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 -- 172.21.15.77:0/632211420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 msgr2=0x557d99539a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d99539a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 -- 172.21.15.77:0/632211420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 msgr2=0x557d99537140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d99537140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 -- 172.21.15.77:0/632211420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9941e440 con 0x557d9952c4a0 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac4e7fc640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 0x7fac440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d10b640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d99537140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5c90a640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d99539a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d10b640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 0x7fac440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac5d90c640 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fac50009f70 tx=0x7fac50010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f776864d2f196c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac4e7fc640 1 -- 172.21.15.77:0/698082694 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fac540cffd0 con 0x7fac58102d90 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.926+0000 7fac437fe640 1 -- 172.21.15.77:0/632211420 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50016070 con 0x557d9952c4a0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac437fe640 1 -- 172.21.15.77:0/632211420 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50004d20 con 0x557d9952c4a0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac437fe640 1 -- 172.21.15.77:0/632211420 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50012910 con 0x557d9952c4a0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 -- 172.21.15.77:0/632211420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 msgr2=0x557d9952c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fac50009f70 tx=0x7fac50010800 comp rx=0 tx=0).stop 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac5d10b640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 0x7fac440a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fac540cd490 tx=0x7fac540d3b20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 -- 172.21.15.77:0/632211420 shutdown_connections 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d99539a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d99537140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 --2- 172.21.15.77:0/632211420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9952c880 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.927+0000 7fac61748740 1 -- 172.21.15.77:0/632211420 >> 172.21.15.77:0/632211420 conn(0x557d99528a30 msgr2=0x557d99528e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- 172.21.15.77:0/632211420 shutdown_connections 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- 172.21.15.77:0/632211420 wait complete. 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- start start 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 Processor -- start 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952c4a0 0x557d99537060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d9953a250 con 0x557d99537680 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d9944d050 con 0x557d9952c4a0 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d9953a3d0 con 0x557d9952cdc0 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.928+0000 7fac61748740 10 client.0 _ll_register_callbacks cb 0x557d9948d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:42.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35396/0 (socket says 172.21.15.77:35396) 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37022/0 (socket says 172.21.15.77:37022) 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5c90a640 1 -- 172.21.15.77:0/318742950 learned_addr learned my addr 172.21.15.77:0/318742950 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952c4a0 0x557d99537060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 -- 172.21.15.77:0/318742950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 msgr2=0x557d99534310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 -- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952c4a0 msgr2=0x557d99537060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952c4a0 0x557d99537060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 -- 172.21.15.77:0/318742950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d992033c0 con 0x557d99537680 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5c90a640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac5d90c640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac500125f0 tx=0x7fac500124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361d394898c6b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50016030 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fac50003970 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac50004d20 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fac50030050 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac61748740 10 client.17226 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.929+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557d99545c90 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50004d20 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fac5000f040 con 0x557d99537680 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 10 client.17226 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 20 client.17226 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d99549540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 4 client.17226 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d9949f850 con 0x557d99547150 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac5c90a640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d99549540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.930+0000 7fac40ff9640 10 client.17226 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac5c90a640 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d99549540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66669fe69d7edd69 server_cookie=73a0646dffc310a5 in_seq=0 out_seq=0 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac40ff9640 1 -- 172.21.15.77:0/318742950 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d9949f850 con 0x557d99547150 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac40ff9640 10 client.17226 handle_command_reply: tid=0 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 client.17226 shutdown 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 msgr2=0x557d995348e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac500125f0 tx=0x7fac500124f0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 shutdown_connections 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d99549540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d99537680 0x557d995348e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.931+0000 7fac40ff9640 0 client.17226 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d99534310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 1 --2- 172.21.15.77:0/318742950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952c4a0 0x557d99537060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 >> 172.21.15.77:0/318742950 conn(0x557d99528a30 msgr2=0x557d9952e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 shutdown_connections 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 1 -- 172.21.15.77:0/318742950 wait complete. 2024-09-15T22:18:42.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.932+0000 7fac61748740 20 client.17226 trim_cache size 0 max 16384 2024-09-15T22:18:42.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 Processor -- start 2024-09-15T22:18:42.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 -- start start 2024-09-15T22:18:42.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d9954edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d9954f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac581af040 con 0x557d9952c4a0 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac581b0b80 con 0x557d9952cdc0 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac61748740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac581af710 con 0x557d99537680 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5d90c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37026/0 (socket says 172.21.15.77:37026) 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d9954edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5d90c640 1 -- 172.21.15.77:0/1977692270 learned_addr learned my addr 172.21.15.77:0/1977692270 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5d90c640 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d9954f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 -- 172.21.15.77:0/1977692270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 msgr2=0x557d9954f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d9954f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 -- 172.21.15.77:0/1977692270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 msgr2=0x557d995434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5c90a640 1 -- 172.21.15.77:0/1977692270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9952cbf0 con 0x557d9952cdc0 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.950+0000 7fac5d90c640 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.951+0000 7fac5c90a640 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d9954edc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557d99535950 tx=0x7fac48032a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763b46d60263bd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.951+0000 7fac42ffd640 1 -- 172.21.15.77:0/1977692270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48035040 con 0x557d9952cdc0 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.951+0000 7fac42ffd640 1 -- 172.21.15.77:0/1977692270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac48004240 con 0x557d9952cdc0 2024-09-15T22:18:42.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.951+0000 7fac42ffd640 1 -- 172.21.15.77:0/1977692270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4804a620 con 0x557d9952cdc0 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- 172.21.15.77:0/1977692270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 msgr2=0x557d9954edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d9954edc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557d99535950 tx=0x7fac48032a10 comp rx=0 tx=0).stop 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- 172.21.15.77:0/1977692270 shutdown_connections 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d99537680 0x557d9954f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d9952cdc0 0x557d9954edc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 --2- 172.21.15.77:0/1977692270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d995434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- 172.21.15.77:0/1977692270 >> 172.21.15.77:0/1977692270 conn(0x557d99528a30 msgr2=0x557d99532be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- 172.21.15.77:0/1977692270 shutdown_connections 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- 172.21.15.77:0/1977692270 wait complete. 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 -- start start 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 Processor -- start 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.952+0000 7fac61748740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9953f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 0x557d995306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac581b04b0 con 0x557d9952c4a0 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac581afde0 con 0x557d99537680 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac581ad020 con 0x557d9952cdc0 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 10 client.0 _ll_register_callbacks cb 0x7fac540d39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35416/0 (socket says 172.21.15.77:35416) 2024-09-15T22:18:42.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 -- 172.21.15.77:0/3311198193 learned_addr learned my addr 172.21.15.77:0/3311198193 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5d90c640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 0x557d995306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5d90c640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9953f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 -- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 msgr2=0x557d995306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 0x557d995306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 -- 172.21.15.77:0/3311198193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 msgr2=0x557d9953f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9953f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 -- 172.21.15.77:0/3311198193 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d99530c00 con 0x557d9952cdc0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5d90c640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 0x557d995306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac5c90a640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac480329e0 tx=0x7fac48009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3bca2bffbf532f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48035040 con 0x557d9952cdc0 2024-09-15T22:18:42.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac48004240 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.953+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9350+0+0 (secure 0 0 0) 0x7fac48005250 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fac48064050 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 10 client.26311 fetch_fsmap learned FSMap version 34 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557d995b1380 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac48004240 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fac48031920 con 0x557d9952cdc0 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 10 client.26311 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 10 client.26311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 20 client.26311 populate_metadata read hostname 'smithi077' 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d995b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 4 client.26311 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:18:42.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d99499d60 con 0x557d99547150 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.954+0000 7fac5d90c640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d995b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac40ff9640 10 client.26311 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac5d90c640 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d995b3ac0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ef9f0983a62845 server_cookie=f8620b20f80ceb74 in_seq=0 out_seq=0 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac40ff9640 1 -- 172.21.15.77:0/3311198193 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557d99499d60 con 0x557d99547150 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac40ff9640 10 client.26311 handle_command_reply: tid=0 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 client.26311 shutdown 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 msgr2=0x557d9953fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac480329e0 tx=0x7fac48009260 comp rx=0 tx=0).stop 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 shutdown_connections 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d99547150 0x557d995b3ac0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d99537680 0x557d995306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d9952cdc0 0x557d9953fb60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 --2- 172.21.15.77:0/3311198193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d9952c4a0 0x557d9953f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac40ff9640 0 client.26311 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 >> 172.21.15.77:0/3311198193 conn(0x557d99528a30 msgr2=0x557d99527350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 shutdown_connections 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 1 -- 172.21.15.77:0/3311198193 wait complete. 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.955+0000 7fac61748740 20 client.26311 trim_cache size 0 max 16384 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "9dd76b70-7029-4b57-8cc7-6502a247a224": { 2024-09-15T22:18:42.963 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:18:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "9dd76b70-7029-4b57-8cc7-6502a247a224", 2024-09-15T22:18:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:18:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:42.964 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:18:42.964 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 msgr2=0x7fac440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 0x7fac440a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fac540cd490 tx=0x7fac540d3b20 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 msgr2=0x7fac58198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58198610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9f0 tx=0x7fac5400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 shutdown_connections 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fac440a33b0 0x7fac440a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac58108290 0x7fac5819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac58103740 0x7fac58198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 --2- 172.21.15.77:0/698082694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac58102d90 0x7fac58198610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 >> 172.21.15.77:0/698082694 conn(0x7fac580fc7d0 msgr2=0x7fac58105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 shutdown_connections 2024-09-15T22:18:42.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:18:42.956+0000 7fac5f36f640 1 -- 172.21.15.77:0/698082694 wait complete. 2024-09-15T22:18:43.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 424703 6.61 MB/sec execute 172 sec latency 27.276 ms 2024-09-15T22:18:43.890 INFO:tasks.cephfs.filesystem:scrub status for tag:9dd76b70-7029-4b57-8cc7-6502a247a224 - {'path': '/', 'tag': '9dd76b70-7029-4b57-8cc7-6502a247a224', 'options': 'recursive,force'} 2024-09-15T22:18:44.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 426857 6.64 MB/sec execute 173 sec latency 16.840 ms 2024-09-15T22:18:44.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:44 smithi196 ceph-mon[35109]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:44.663 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:44 smithi117 ceph-mon[35021]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:44.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:44 smithi077 ceph-mon[30980]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:45.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 429492 6.70 MB/sec execute 174 sec latency 14.623 ms 2024-09-15T22:18:46.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 431670 6.71 MB/sec execute 175 sec latency 16.765 ms 2024-09-15T22:18:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:46 smithi077 ceph-mon[30980]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:46.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:46 smithi196 ceph-mon[35109]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:46 smithi117 ceph-mon[35021]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 160 op/s 2024-09-15T22:18:47.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 433889 6.75 MB/sec execute 176 sec latency 12.428 ms 2024-09-15T22:18:48.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 436572 6.80 MB/sec execute 177 sec latency 14.574 ms 2024-09-15T22:18:48.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:48 smithi077 ceph-mon[30980]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2024-09-15T22:18:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:48 smithi196 ceph-mon[35109]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2024-09-15T22:18:48.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:48 smithi117 ceph-mon[35021]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 236 op/s 2024-09-15T22:18:49.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 438709 6.81 MB/sec execute 178 sec latency 45.832 ms 2024-09-15T22:18:50.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 440835 6.84 MB/sec execute 179 sec latency 10.745 ms 2024-09-15T22:18:50.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:50 smithi077 ceph-mon[30980]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-15T22:18:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:50 smithi196 ceph-mon[35109]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-15T22:18:50.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:50 smithi117 ceph-mon[35021]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-15T22:18:51.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 443639 6.90 MB/sec execute 180 sec latency 16.903 ms 2024-09-15T22:18:51.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:18:51.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:18:51.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:18:51.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:18:51.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:18:51.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:18:52.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 445764 6.91 MB/sec execute 181 sec latency 13.438 ms 2024-09-15T22:18:52.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:52 smithi077 ceph-mon[30980]: pgmap v196: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:52 smithi196 ceph-mon[35109]: pgmap v196: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:52.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:52 smithi117 ceph-mon[35021]: pgmap v196: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:53.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 447854 6.94 MB/sec execute 182 sec latency 12.374 ms 2024-09-15T22:18:54.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 450602 6.99 MB/sec execute 183 sec latency 15.518 ms 2024-09-15T22:18:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:54 smithi077 ceph-mon[30980]: pgmap v197: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:54.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:54 smithi196 ceph-mon[35109]: pgmap v197: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:54 smithi117 ceph-mon[35021]: pgmap v197: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:55.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 452369 6.99 MB/sec execute 184 sec latency 18.271 ms 2024-09-15T22:18:56.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 454770 7.03 MB/sec execute 185 sec latency 16.393 ms 2024-09-15T22:18:56.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:56 smithi077 ceph-mon[30980]: pgmap v198: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:56.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:56 smithi196 ceph-mon[35109]: pgmap v198: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:56.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:56 smithi117 ceph-mon[35021]: pgmap v198: 129 pgs: 129 active+clean; 273 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-15T22:18:57.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 457611 7.08 MB/sec execute 186 sec latency 15.902 ms 2024-09-15T22:18:58.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 458805 7.14 MB/sec execute 187 sec latency 58.138 ms 2024-09-15T22:18:58.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:18:58 smithi077 ceph-mon[30980]: pgmap v199: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:58.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:18:58 smithi196 ceph-mon[35109]: pgmap v199: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:18:58 smithi117 ceph-mon[35021]: pgmap v199: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:18:59.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 461220 7.19 MB/sec execute 188 sec latency 82.624 ms 2024-09-15T22:19:00.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 463175 7.21 MB/sec execute 189 sec latency 24.665 ms 2024-09-15T22:19:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:00 smithi077 ceph-mon[30980]: pgmap v200: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-09-15T22:19:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:00 smithi196 ceph-mon[35109]: pgmap v200: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-09-15T22:19:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:00 smithi117 ceph-mon[35021]: pgmap v200: 129 pgs: 129 active+clean; 283 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-09-15T22:19:01.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 464199 7.19 MB/sec execute 190 sec latency 57.416 ms 2024-09-15T22:19:02.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 466358 7.22 MB/sec execute 191 sec latency 15.858 ms 2024-09-15T22:19:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:02 smithi077 ceph-mon[30980]: pgmap v201: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 255 op/s 2024-09-15T22:19:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:02 smithi196 ceph-mon[35109]: pgmap v201: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 255 op/s 2024-09-15T22:19:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:02 smithi117 ceph-mon[35021]: pgmap v201: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.8 MiB/s wr, 255 op/s 2024-09-15T22:19:03.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 469246 7.28 MB/sec execute 192 sec latency 15.139 ms 2024-09-15T22:19:04.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 471478 7.28 MB/sec execute 193 sec latency 14.706 ms 2024-09-15T22:19:04.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:04 smithi077 ceph-mon[30980]: pgmap v202: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:04 smithi196 ceph-mon[35109]: pgmap v202: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:04 smithi117 ceph-mon[35021]: pgmap v202: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:05.474 INFO:tasks.workunit.client.0.smithi077.stdout: 1 473542 7.31 MB/sec execute 194 sec latency 17.007 ms 2024-09-15T22:19:06.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 476354 7.36 MB/sec execute 195 sec latency 15.161 ms 2024-09-15T22:19:06.837 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:06 smithi077 ceph-mon[30980]: pgmap v203: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:06.837 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:06 smithi077 ceph-mon[30980]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:19:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:06 smithi196 ceph-mon[35109]: pgmap v203: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:06 smithi196 ceph-mon[35109]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:19:06.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:06 smithi117 ceph-mon[35021]: pgmap v203: 129 pgs: 129 active+clean; 293 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-09-15T22:19:06.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:06 smithi117 ceph-mon[35021]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:19:07.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 476785 7.33 MB/sec execute 196 sec latency 336.381 ms 2024-09-15T22:19:08.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 478230 7.32 MB/sec execute 197 sec latency 297.836 ms 2024-09-15T22:19:08.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:08 smithi077 ceph-mon[30980]: pgmap v205: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:08 smithi196 ceph-mon[35109]: pgmap v205: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:08 smithi117 ceph-mon[35021]: pgmap v205: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:09.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 480096 7.35 MB/sec execute 198 sec latency 15.012 ms 2024-09-15T22:19:10.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 482569 7.37 MB/sec execute 199 sec latency 20.950 ms 2024-09-15T22:19:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:10 smithi077 ceph-mon[30980]: pgmap v206: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:10 smithi196 ceph-mon[35109]: pgmap v206: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:10 smithi117 ceph-mon[35021]: pgmap v206: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 302 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-09-15T22:19:11.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 484643 7.40 MB/sec execute 200 sec latency 21.427 ms 2024-09-15T22:19:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:11 smithi077 ceph-mon[30980]: pgmap v207: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:11 smithi196 ceph-mon[35109]: pgmap v207: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:11 smithi117 ceph-mon[35021]: pgmap v207: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:12.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 487166 7.43 MB/sec execute 201 sec latency 16.392 ms 2024-09-15T22:19:13.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 489535 7.45 MB/sec execute 202 sec latency 27.833 ms 2024-09-15T22:19:13.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:13 smithi077 ceph-mon[30980]: pgmap v208: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:13.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:13 smithi196 ceph-mon[35109]: pgmap v208: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:13.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:13 smithi117 ceph-mon[35021]: pgmap v208: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 405 op/s 2024-09-15T22:19:13.891 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:19:14.163 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:14.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 491475 7.46 MB/sec execute 203 sec latency 30.758 ms 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- 172.21.15.77:0/759775652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 msgr2=0x7f82841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 --2- 172.21.15.77:0/759775652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841029b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002a480 comp rx=0 tx=0).stop 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- 172.21.15.77:0/759775652 shutdown_connections 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 --2- 172.21.15.77:0/759775652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 0x7f828410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 --2- 172.21.15.77:0/759775652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841029b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 --2- 172.21.15.77:0/759775652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f8284101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- 172.21.15.77:0/759775652 >> 172.21.15.77:0/759775652 conn(0x7f82840fc7d0 msgr2=0x7f82840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- 172.21.15.77:0/759775652 shutdown_connections 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- 172.21.15.77:0/759775652 wait complete. 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 Processor -- start 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.661+0000 7f828ab1f640 1 -- start start 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 0x7f828419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8284110f80 con 0x7f8284102570 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8284110e00 con 0x7f8284100e20 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8284111100 con 0x7f8284102ef0 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f82888bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f82888bb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39422/0 (socket says 172.21.15.77:39422) 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f82888bb640 1 -- 172.21.15.77:0/1700312077 learned_addr learned my addr 172.21.15.77:0/1700312077 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47988/0 (socket says 172.21.15.77:47988) 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f82890bc640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 0x7f828419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 -- 172.21.15.77:0/1700312077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 msgr2=0x7f828419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 0x7f828419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 -- 172.21.15.77:0/1700312077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 msgr2=0x7f82841984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 -- 172.21.15.77:0/1700312077 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828419d460 con 0x7f8284102570 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f82888bb640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8283fff640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8270005940 tx=0x7f8270003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8281ffb640 1 -- 172.21.15.77:0/1700312077 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827003f070 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8281ffb640 1 -- 172.21.15.77:0/1700312077 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8270031c30 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828419d6f0 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.662+0000 7f8281ffb640 1 -- 172.21.15.77:0/1700312077 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270035030 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.663+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82841a5030 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.665+0000 7f8281ffb640 1 -- 172.21.15.77:0/1700312077 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8270042070 con 0x7f8284102570 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.665+0000 7f828cef8740 1 Processor -- start 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.665+0000 7f828cef8740 1 -- start start 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f8281ffb640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 0x7f82740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c277140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c279a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f38c1d22e0 con 0x55f38c26cdc0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f38c179ce0 con 0x55f38c277680 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f828cef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f38c1ce2a0 con 0x55f38c26c4a0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f8281ffb640 1 -- 172.21.15.77:0/1700312077 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f82700f0610 con 0x7f8284102570 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42010/0 (socket says 172.21.15.77:42010) 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 -- 172.21.15.77:0/1611119572 learned_addr learned my addr 172.21.15.77:0/1611119572 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f8283fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c277140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82890bc640 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c279a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82890bc640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 0x7f82740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 -- 172.21.15.77:0/1611119572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 msgr2=0x55f38c279a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c279a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 -- 172.21.15.77:0/1611119572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 msgr2=0x55f38c277140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c277140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 -- 172.21.15.77:0/1611119572 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38c15e440 con 0x55f38c26c4a0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82888bb640 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8278011650 tx=0x7f8278009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b75041d10efce43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.666+0000 7f82890bc640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 0x7f82740a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55f38c269db0 tx=0x7f827c009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f826effd640 1 -- 172.21.15.77:0/1611119572 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827801e070 con 0x55f38c26c4a0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f826effd640 1 -- 172.21.15.77:0/1611119572 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827800fba0 con 0x55f38c26c4a0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f826effd640 1 -- 172.21.15.77:0/1611119572 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278019d40 con 0x55f38c26c4a0 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 -- 172.21.15.77:0/1611119572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 msgr2=0x55f38c26c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8278011650 tx=0x7f8278009f90 comp rx=0 tx=0).stop 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 -- 172.21.15.77:0/1611119572 shutdown_connections 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c279a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c277140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 --2- 172.21.15.77:0/1611119572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26c4a0 0x55f38c26c880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 -- 172.21.15.77:0/1611119572 >> 172.21.15.77:0/1611119572 conn(0x55f38c268a30 msgr2=0x55f38c268e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 -- 172.21.15.77:0/1611119572 shutdown_connections 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.667+0000 7f828cef8740 1 -- 172.21.15.77:0/1611119572 wait complete. 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 -- start start 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 Processor -- start 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 0x55f38c274270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c2747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f38c27a250 con 0x55f38c26c4a0 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f38c18d050 con 0x55f38c277680 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f38c27a3d0 con 0x55f38c26cdc0 2024-09-15T22:19:14.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f828cef8740 10 client.0 _ll_register_callbacks cb 0x55f38c1cd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.668+0000 7f8283fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 0x55f38c274270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82890bc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c2747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48004/0 (socket says 172.21.15.77:48004) 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 -- 172.21.15.77:0/3112953629 learned_addr learned my addr 172.21.15.77:0/3112953629 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 -- 172.21.15.77:0/3112953629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 msgr2=0x55f38c274270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 0x55f38c274270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 -- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 msgr2=0x55f38c2747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c2747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 -- 172.21.15.77:0/3112953629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38bf433c0 con 0x55f38c26c4a0 2024-09-15T22:19:14.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f8283fff640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 0x55f38c274270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f82888bb640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8278019880 tx=0x7f82780056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43c3a7beb5a2881 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f8263fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827801e040 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f8278009cc0 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827800fba0 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8278030050 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f828cef8740 10 client.17244 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.669+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f38c285c00 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827800fba0 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8278021070 con 0x55f38c26c4a0 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 10 client.17244 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:14.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 20 client.17244 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c287000 0x55f38c2893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 4 client.17244 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38c1df850 con 0x55f38c287000 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f8283fff640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c287000 0x55f38c2893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f8283fff640 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c287000 0x55f38c2893f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad418df2a098de81 server_cookie=8eccc591104cacef in_seq=0 out_seq=0 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.670+0000 7f8263fff640 10 client.17244 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f8263fff640 1 -- 172.21.15.77:0/3112953629 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f38c1df850 con 0x55f38c287000 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f8263fff640 10 client.17244 handle_command_reply: tid=0 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 client.17244 shutdown 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 msgr2=0x55f38c276fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8278019880 tx=0x7f82780056d0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 shutdown_connections 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c287000 0x55f38c2893f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c2747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f8263fff640 0 client.17244 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c26cdc0 0x55f38c274270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 --2- 172.21.15.77:0/3112953629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c276fc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 >> 172.21.15.77:0/3112953629 conn(0x55f38c268a30 msgr2=0x55f38c267b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 shutdown_connections 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.671+0000 7f828cef8740 1 -- 172.21.15.77:0/3112953629 wait complete. 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.672+0000 7f828cef8740 20 client.17244 trim_cache size 0 max 16384 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 Processor -- start 2024-09-15T22:19:14.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 -- start start 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 0x55f38c267370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c277680 0x55f38c28eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82841aef70 con 0x55f38c26c4a0 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b0ab0 con 0x55f38c26cdc0 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f828cef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82841af640 con 0x55f38c277680 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48012/0 (socket says 172.21.15.77:48012) 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 -- 172.21.15.77:0/3713123309 learned_addr learned my addr 172.21.15.77:0/3713123309 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f8283fff640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 0x55f38c267370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82890bc640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c277680 0x55f38c28eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 -- 172.21.15.77:0/3713123309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c277680 msgr2=0x55f38c28eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c277680 0x55f38c28eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 -- 172.21.15.77:0/3713123309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 msgr2=0x55f38c267370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 0x55f38c267370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f82888bb640 1 -- 172.21.15.77:0/3713123309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38c2727c0 con 0x55f38c26c4a0 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.690+0000 7f8283fff640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 0x55f38c267370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.691+0000 7f82888bb640 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f82780059f0 tx=0x7f82780037b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7633d390b074ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.691+0000 7f826e7fc640 1 -- 172.21.15.77:0/3713123309 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827801e040 con 0x55f38c26c4a0 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.691+0000 7f826e7fc640 1 -- 172.21.15.77:0/3713123309 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278027070 con 0x55f38c26c4a0 2024-09-15T22:19:14.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.691+0000 7f826e7fc640 1 -- 172.21.15.77:0/3713123309 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800fe20 con 0x55f38c26c4a0 2024-09-15T22:19:14.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- 172.21.15.77:0/3713123309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 msgr2=0x55f38c2834b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f82780059f0 tx=0x7f82780037b0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- 172.21.15.77:0/3713123309 shutdown_connections 2024-09-15T22:19:14.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c277680 0x55f38c28eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c26cdc0 0x55f38c267370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 --2- 172.21.15.77:0/3713123309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26c4a0 0x55f38c2834b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- 172.21.15.77:0/3713123309 >> 172.21.15.77:0/3713123309 conn(0x55f38c268a30 msgr2=0x55f38c2762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- 172.21.15.77:0/3713123309 shutdown_connections 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- 172.21.15.77:0/3713123309 wait complete. 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 -- start start 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.692+0000 7f828cef8740 1 Processor -- start 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c27cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c287000 0x55f38c27d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b03e0 con 0x55f38c26cdc0 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82841afd10 con 0x55f38c277680 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82841acf70 con 0x55f38c287000 2024-09-15T22:19:14.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f828cef8740 10 client.0 _ll_register_callbacks cb 0x7f827801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48014/0 (socket says 172.21.15.77:48014) 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f8283fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c27cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 -- 172.21.15.77:0/2533105267 learned_addr learned my addr 172.21.15.77:0/2533105267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82890bc640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c287000 0x55f38c27d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 -- 172.21.15.77:0/2533105267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c287000 msgr2=0x55f38c27d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c287000 0x55f38c27d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 -- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 msgr2=0x55f38c27cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c27cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.693+0000 7f82888bb640 1 -- 172.21.15.77:0/2533105267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38c27da60 con 0x55f38c26cdc0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f82888bb640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8278005820 tx=0x7f82780196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd8a4f4d30bcc4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f8263fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827801e040 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f8278003c10 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8278027070 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8278044050 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f828cef8740 10 client.17250 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f38c2f5df0 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.694+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278027070 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8278021070 con 0x55f38c26cdc0 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 10 client.17250 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 20 client.17250 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c2f6400 0x55f38c2f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 4 client.17250 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:19:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f38c274b40 con 0x55f38c2f6400 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f8283fff640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c2f6400 0x55f38c2f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f8263fff640 10 client.17250 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.695+0000 7f8283fff640 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c2f6400 0x55f38c2f8820 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5754835f7e6c5f server_cookie=79613ea4d92c4f72 in_seq=0 out_seq=0 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f8263fff640 1 -- 172.21.15.77:0/2533105267 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f38c274b40 con 0x55f38c2f6400 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f8263fff640 10 client.17250 handle_command_reply: tid=0 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 client.17250 shutdown 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 msgr2=0x55f38c27fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8278005820 tx=0x7f82780196d0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 shutdown_connections 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f38c2f6400 0x55f38c2f8820 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f38c287000 0x55f38c27d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f38c277680 0x55f38c27cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 --2- 172.21.15.77:0/2533105267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f38c26cdc0 0x55f38c27fb90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 >> 172.21.15.77:0/2533105267 conn(0x55f38c268a30 msgr2=0x55f38c2824a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f8263fff640 0 client.17250 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 shutdown_connections 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 1 -- 172.21.15.77:0/2533105267 wait complete. 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.696+0000 7f828cef8740 20 client.17250 trim_cache size 0 max 16384 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:19:14.714 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:19:14.715 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:19:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 msgr2=0x7f82740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 0x7f82740a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55f38c269db0 tx=0x7f827c009500 comp rx=0 tx=0).stop 2024-09-15T22:19:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 msgr2=0x7f82841989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8270005940 tx=0x7f8270003c10 comp rx=0 tx=0).stop 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 shutdown_connections 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82740a33b0 0x7f82740a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8284102ef0 0x7f828419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8284102570 0x7f82841989f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 --2- 172.21.15.77:0/1700312077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8284100e20 0x7f82841984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 >> 172.21.15.77:0/1700312077 conn(0x7f82840fc7d0 msgr2=0x7f828406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 shutdown_connections 2024-09-15T22:19:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:14.697+0000 7f828ab1f640 1 -- 172.21.15.77:0/1700312077 wait complete. 2024-09-15T22:19:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:14 smithi077 ceph-mon[30980]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:19:15.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:14 smithi196 ceph-mon[35109]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:19:15.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:14 smithi117 ceph-mon[35021]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:19:15.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 494146 7.51 MB/sec execute 204 sec latency 38.826 ms 2024-09-15T22:19:15.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:19:15.839 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:19:16.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:15 smithi077 ceph-mon[30980]: pgmap v210: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-15T22:19:16.101 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:16.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:15 smithi196 ceph-mon[35109]: pgmap v210: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-15T22:19:16.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:15 smithi117 ceph-mon[35021]: pgmap v210: 129 pgs: 129 active+clean; 322 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-15T22:19:16.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 496220 7.51 MB/sec execute 205 sec latency 25.302 ms 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 -- 172.21.15.77:0/2277163779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8100e20 msgr2=0x7f34f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 --2- 172.21.15.77:0/2277163779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8100e20 0x7f34f8101200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009890 tx=0x7f34e802a410 comp rx=0 tx=0).stop 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 -- 172.21.15.77:0/2277163779 shutdown_connections 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 --2- 172.21.15.77:0/2277163779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8102ef0 0x7f34f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 --2- 172.21.15.77:0/2277163779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 --2- 172.21.15.77:0/2277163779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8100e20 0x7f34f8101200 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 -- 172.21.15.77:0/2277163779 >> 172.21.15.77:0/2277163779 conn(0x7f34f80fc7d0 msgr2=0x7f34f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 -- 172.21.15.77:0/2277163779 shutdown_connections 2024-09-15T22:19:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.599+0000 7f34fe79d640 1 -- 172.21.15.77:0/2277163779 wait complete. 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 Processor -- start 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 -- start start 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8102ef0 0x7f34f819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8110f80 con 0x7f34f8102570 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8110e00 con 0x7f34f8102ef0 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34fe79d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8111100 con 0x7f34f8100e20 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34f7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34f7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42054/0 (socket says 172.21.15.77:42054) 2024-09-15T22:19:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.600+0000 7f34f7fff640 1 -- 172.21.15.77:0/882495085 learned_addr learned my addr 172.21.15.77:0/882495085 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34fcd3a640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8102ef0 0x7f34f819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 -- 172.21.15.77:0/882495085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8102ef0 msgr2=0x7f34f819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8102ef0 0x7f34f819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 -- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 msgr2=0x7f34f8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 -- 172.21.15.77:0/882495085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f819d510 con 0x7f34f8100e20 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f77fe640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f7fff640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005940 tx=0x7f34e8030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8040020 con 0x7f34f8100e20 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f819d7a0 con 0x7f34f8100e20 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e8005550 con 0x7f34f8100e20 2024-09-15T22:19:16.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8031aa0 con 0x7f34f8100e20 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.601+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81a50e0 con 0x7f34f8100e20 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.602+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8101200 con 0x7f34f8100e20 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.604+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f34e8043020 con 0x7f34f8100e20 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.604+0000 7f34f57fa640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 0x7f34e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.604+0000 7f34f77fe640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 0x7f34e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.604+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f34e80f1370 con 0x7f34f8100e20 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.605+0000 7f34f77fe640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 0x7f34e00a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f34f80fef70 tx=0x7f34ec009260 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:16.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.608+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34e804a020 con 0x7f34f8100e20 2024-09-15T22:19:16.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.839+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f810a0a0 con 0x7f34f8100e20 2024-09-15T22:19:16.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.840+0000 7f34f57fa640 1 -- 172.21.15.77:0/882495085 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f34e80bd9c0 con 0x7f34f8100e20 2024-09-15T22:19:16.843 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:19:16.843 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:19:16.843 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 msgr2=0x7f34e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 0x7f34e00a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f34f80fef70 tx=0x7f34ec009260 comp rx=0 tx=0).stop 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 msgr2=0x7f34f8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005940 tx=0x7f34e8030650 comp rx=0 tx=0).stop 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 shutdown_connections 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34e00a33b0 0x7f34e00a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34f8102ef0 0x7f34f819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34f8102570 0x7f34f8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 --2- 172.21.15.77:0/882495085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34f8100e20 0x7f34f8198560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 >> 172.21.15.77:0/882495085 conn(0x7f34f80fc7d0 msgr2=0x7f34f806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 shutdown_connections 2024-09-15T22:19:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:16.842+0000 7f34fe79d640 1 -- 172.21.15.77:0/882495085 wait complete. 2024-09-15T22:19:17.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 498219 7.53 MB/sec execute 206 sec latency 25.120 ms 2024-09-15T22:19:17.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:19:17.928 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:18.051 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:17 smithi077 ceph-mon[30980]: pgmap v211: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:18.051 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:17 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/882495085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:18.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:17 smithi196 ceph-mon[35109]: pgmap v211: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:18.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:17 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/882495085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:18.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:17 smithi117 ceph-mon[35021]: pgmap v211: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:18.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:17 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/882495085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:18.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 -- 172.21.15.77:0/3959857938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 msgr2=0x7f315410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 --2- 172.21.15.77:0/3959857938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f315410da40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f314c009ca0 tx=0x7f314c02a950 comp rx=0 tx=0).stop 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 -- 172.21.15.77:0/3959857938 shutdown_connections 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 --2- 172.21.15.77:0/3959857938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f315410da40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 --2- 172.21.15.77:0/3959857938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 --2- 172.21.15.77:0/3959857938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 -- 172.21.15.77:0/3959857938 >> 172.21.15.77:0/3959857938 conn(0x7f31540fc7d0 msgr2=0x7f31540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 -- 172.21.15.77:0/3959857938 shutdown_connections 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.428+0000 7f315c58a640 1 -- 172.21.15.77:0/3959857938 wait complete. 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 Processor -- start 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 -- start start 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3154110f80 con 0x7f3154102ef0 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3154110e00 con 0x7f3154100e20 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315c58a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3154111100 con 0x7f3154102570 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315ab27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315a326640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f3159b25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315ab27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48068/0 (socket says 172.21.15.77:48068) 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315ab27640 1 -- 172.21.15.77:0/1669435258 learned_addr learned my addr 172.21.15.77:0/1669435258 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.429+0000 7f315a326640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39506/0 (socket says 172.21.15.77:39506) 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 -- 172.21.15.77:0/1669435258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 msgr2=0x7f31541998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 -- 172.21.15.77:0/1669435258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 msgr2=0x7f3154199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 -- 172.21.15.77:0/1669435258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31541a4930 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f3159b25640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315a326640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315ab27640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f314c004af0 tx=0x7f314c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f314b7fe640 1 -- 172.21.15.77:0/1669435258 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c034040 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31541a4b00 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f314b7fe640 1 -- 172.21.15.77:0/1669435258 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314c030920 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f314b7fe640 1 -- 172.21.15.77:0/1669435258 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c004820 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.430+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31541a4fb0 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.433+0000 7f314b7fe640 1 -- 172.21.15.77:0/1669435258 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f314c037020 con 0x7f3154102ef0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.434+0000 7f314b7fe640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 0x7f31400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.434+0000 7f315e963740 1 Processor -- start 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.434+0000 7f315a326640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 0x7f31400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.434+0000 7f314b7fe640 1 -- 172.21.15.77:0/1669435258 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f314c0f0e50 con 0x7f3154102ef0 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 -- start start 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb885ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561eb87b32f0 con 0x561eb8857400 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561eb875acf0 con 0x561eb8857db0 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315e963740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561eb87af2b0 con 0x561eb885c970 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f3159b25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48076/0 (socket says 172.21.15.77:48076) 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 -- 172.21.15.77:0/1007499211 learned_addr learned my addr 172.21.15.77:0/1007499211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 0x7f31400a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31500058c0 tx=0x7f315000a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315ab27640 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb885ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 -- 172.21.15.77:0/1007499211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 msgr2=0x561eb885ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb885ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 -- 172.21.15.77:0/1007499211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 msgr2=0x561eb885c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f315a326640 1 -- 172.21.15.77:0/1007499211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb873f3a0 con 0x561eb8857400 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.435+0000 7f3159b25640 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.436+0000 7f315a326640 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3150010750 tx=0x7f3150012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9195172cdff71ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.436+0000 7f3148ff9640 1 -- 172.21.15.77:0/1007499211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150016070 con 0x561eb8857400 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.436+0000 7f3148ff9640 1 -- 172.21.15.77:0/1007499211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315002d430 con 0x561eb8857400 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.436+0000 7f3148ff9640 1 -- 172.21.15.77:0/1007499211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31500195e0 con 0x561eb8857400 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 -- 172.21.15.77:0/1007499211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 msgr2=0x561eb88577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3150010750 tx=0x7f3150012000 comp rx=0 tx=0).stop 2024-09-15T22:19:18.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 -- 172.21.15.77:0/1007499211 shutdown_connections 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb885ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 --2- 172.21.15.77:0/1007499211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb88577e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.437+0000 7f315e963740 1 -- 172.21.15.77:0/1007499211 >> 172.21.15.77:0/1007499211 conn(0x561eb8816310 msgr2=0x561eb8849680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.438+0000 7f315e963740 1 -- 172.21.15.77:0/1007499211 shutdown_connections 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.438+0000 7f315e963740 1 -- 172.21.15.77:0/1007499211 wait complete. 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.438+0000 7f315e963740 1 -- start start 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.438+0000 7f315e963740 1 Processor -- start 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb8862cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb88604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561eb8863890 con 0x561eb885c970 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f3159b25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315ab27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb88604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f3159b25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39530/0 (socket says 172.21.15.77:39530) 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f3159b25640 1 -- 172.21.15.77:0/3446318119 learned_addr learned my addr 172.21.15.77:0/3446318119 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315a326640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb8862cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561eb876e060 con 0x561eb8857db0 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561eb8863a10 con 0x561eb8857400 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.439+0000 7f315e963740 10 client.0 _ll_register_callbacks cb 0x561eb84c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 -- 172.21.15.77:0/3446318119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 msgr2=0x561eb8862cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb8862cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 -- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 msgr2=0x561eb88604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb88604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb85243c0 con 0x561eb8857db0 2024-09-15T22:19:18.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f315ab27640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb88604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f315a326640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb8862cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f3159b25640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3144012d20 tx=0x7f314400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3db2a2fc42e0fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.440+0000 7f313dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440120d0 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f3144010430 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315002d430 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3144012270 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f315e963740 10 client.26432 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561eb886f230 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.441+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315002d430 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f314401c780 con 0x561eb8857db0 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 10 client.26432 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 10 client.26432 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:19:18.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 20 client.26432 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb8870740 0x561eb8872ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 4 client.26432 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb87c0860 con 0x561eb8870740 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.442+0000 7f315a326640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb8870740 0x561eb8872ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.443+0000 7f313dffb640 10 client.26432 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:19:18.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.443+0000 7f315a326640 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb8870740 0x561eb8872ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab3cd3c41ff53f7e server_cookie=deded5a1f3bf1bed in_seq=0 out_seq=0 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f313dffb640 1 -- 172.21.15.77:0/3446318119 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561eb87c0860 con 0x561eb8870740 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f313dffb640 10 client.26432 handle_command_reply: tid=0 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 client.26432 shutdown 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 msgr2=0x561eb885ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3144012d20 tx=0x7f314400f790 comp rx=0 tx=0).stop 2024-09-15T22:19:18.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 shutdown_connections 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb8870740 0x561eb8872ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb88604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885ff80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f313dffb640 0 client.26432 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 --2- 172.21.15.77:0/3446318119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb8862cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 >> 172.21.15.77:0/3446318119 conn(0x561eb8816310 msgr2=0x561eb8856a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 shutdown_connections 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 1 -- 172.21.15.77:0/3446318119 wait complete. 2024-09-15T22:19:18.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.444+0000 7f315e963740 20 client.26432 trim_cache size 0 max 16384 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 Processor -- start 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 -- start start 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb886cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31540736e0 con 0x561eb885c970 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31541a60a0 con 0x561eb8857db0 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315e963740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3154073db0 con 0x561eb8857400 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315ab27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315ab27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48086/0 (socket says 172.21.15.77:48086) 2024-09-15T22:19:18.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315ab27640 1 -- 172.21.15.77:0/391626338 learned_addr learned my addr 172.21.15.77:0/391626338 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:18.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f315a326640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb886cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.458+0000 7f3159b25640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 -- 172.21.15.77:0/391626338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 msgr2=0x561eb886cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb886cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 -- 172.21.15.77:0/391626338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 msgr2=0x561eb885c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 -- 172.21.15.77:0/391626338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb8863a60 con 0x561eb885c970 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f3159b25640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315a326640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb886cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f315ab27640 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f314c0b9820 tx=0x7f314c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f26e5c1e8f4b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f313ffff640 1 -- 172.21.15.77:0/391626338 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c02c040 con 0x561eb885c970 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f313ffff640 1 -- 172.21.15.77:0/391626338 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314c0ee020 con 0x561eb885c970 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.459+0000 7f313ffff640 1 -- 172.21.15.77:0/391626338 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c0c5ac0 con 0x561eb885c970 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- 172.21.15.77:0/391626338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 msgr2=0x561eb885e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f314c0b9820 tx=0x7f314c003d80 comp rx=0 tx=0).stop 2024-09-15T22:19:18.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- 172.21.15.77:0/391626338 shutdown_connections 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb885c970 0x561eb885e720 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb885c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 --2- 172.21.15.77:0/391626338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb8857400 0x561eb886cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- 172.21.15.77:0/391626338 >> 172.21.15.77:0/391626338 conn(0x561eb8816310 msgr2=0x561eb8856a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- 172.21.15.77:0/391626338 shutdown_connections 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- 172.21.15.77:0/391626338 wait complete. 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.460+0000 7f315e963740 1 -- start start 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 Processor -- start 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb8877b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb88780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31541a59d0 con 0x561eb8857400 2024-09-15T22:19:18.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31541a5320 con 0x561eb8857db0 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31540716e0 con 0x561eb885c970 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315e963740 10 client.0 _ll_register_callbacks cb 0x7f3144009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315ab27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb88780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48094/0 (socket says 172.21.15.77:48094) 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 -- 172.21.15.77:0/3706721008 learned_addr learned my addr 172.21.15.77:0/3706721008 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f3159b25640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb8877b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 -- 172.21.15.77:0/3706721008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 msgr2=0x561eb88780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb88780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 -- 172.21.15.77:0/3706721008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 msgr2=0x561eb8877b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb8877b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315a326640 1 -- 172.21.15.77:0/3706721008 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb8878630 con 0x561eb8857400 2024-09-15T22:19:18.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.461+0000 7f315ab27640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb88780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315a326640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f315002de30 tx=0x7f315000fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1ac3befdea1e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150016040 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f3150005240 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f314c0ee020 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3150052050 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 10 client.17274 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561eb88db200 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314c0ee020 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3150004660 con 0x561eb8857400 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 10 client.17274 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 10 client.17274 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:19:18.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 20 client.17274 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb88db650 0x561eb88dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 4 client.17274 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.462+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561eb87bad70 con 0x561eb88db650 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f3159b25640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb88db650 0x561eb88dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f313dffb640 10 client.17274 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f3159b25640 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb88db650 0x561eb88dda70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=343cdfe6402c6503 server_cookie=e5373e5106df33ab in_seq=0 out_seq=0 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f313dffb640 1 -- 172.21.15.77:0/3706721008 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561eb87bad70 con 0x561eb88db650 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f313dffb640 10 client.17274 handle_command_reply: tid=0 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 client.17274 shutdown 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 msgr2=0x561eb8869190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f315002de30 tx=0x7f315000fd50 comp rx=0 tx=0).stop 2024-09-15T22:19:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 shutdown_connections 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561eb88db650 0x561eb88dda70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561eb885c970 0x561eb88780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561eb8857db0 0x561eb8877b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 --2- 172.21.15.77:0/3706721008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561eb8857400 0x561eb8869190 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 >> 172.21.15.77:0/3706721008 conn(0x561eb8816310 msgr2=0x561eb88484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.463+0000 7f313dffb640 0 client.17274 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 shutdown_connections 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315e963740 1 -- 172.21.15.77:0/3706721008 wait complete. 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315e963740 20 client.17274 trim_cache size 0 max 16384 2024-09-15T22:19:18.472 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 msgr2=0x7f31400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 0x7f31400a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31500058c0 tx=0x7f315000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 msgr2=0x7f31541a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f314c004af0 tx=0x7f314c02f5b0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 shutdown_connections 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31400a33b0 0x7f31400a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3154102ef0 0x7f31541a43f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3154102570 0x7f31541998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 --2- 172.21.15.77:0/1669435258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3154100e20 0x7f3154199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.464+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 >> 172.21.15.77:0/1669435258 conn(0x7f31540fc7d0 msgr2=0x7f3154106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.465+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 shutdown_connections 2024-09-15T22:19:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:18.465+0000 7f315c58a640 1 -- 172.21.15.77:0/1669435258 wait complete. 2024-09-15T22:19:18.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 500775 7.56 MB/sec execute 207 sec latency 25.739 ms 2024-09-15T22:19:19.221 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:19:19.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 502447 7.57 MB/sec execute 208 sec latency 70.023 ms 2024-09-15T22:19:19.488 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:19.914 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:19 smithi077 ceph-mon[30980]: pgmap v212: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:20.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.996+0000 7fefde103640 1 -- 172.21.15.77:0/3077025312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102570 msgr2=0x7fefd81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.996+0000 7fefde103640 1 --2- 172.21.15.77:0/3077025312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102570 0x7fefd81029b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009900 tx=0x7fefc002a480 comp rx=0 tx=0).stop 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 -- 172.21.15.77:0/3077025312 shutdown_connections 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 --2- 172.21.15.77:0/3077025312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8102ef0 0x7fefd810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 --2- 172.21.15.77:0/3077025312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102570 0x7fefd81029b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 --2- 172.21.15.77:0/3077025312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8100e20 0x7fefd8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 -- 172.21.15.77:0/3077025312 >> 172.21.15.77:0/3077025312 conn(0x7fefd80fc7d0 msgr2=0x7fefd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 -- 172.21.15.77:0/3077025312 shutdown_connections 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 -- 172.21.15.77:0/3077025312 wait complete. 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 Processor -- start 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 -- start start 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.997+0000 7fefde103640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8100e20 0x7fefd81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 0x7fefd8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8110fd0 con 0x7fefd8102ef0 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8110e50 con 0x7fefd8100e20 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8111150 con 0x7fefd8102570 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 0x7fefd8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8100e20 0x7fefd81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34222/0 (socket says 172.21.15.77:34222) 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd7fff640 1 -- 172.21.15.77:0/3741777029 learned_addr learned my addr 172.21.15.77:0/3741777029 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 -- 172.21.15.77:0/3741777029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8100e20 msgr2=0x7fefd81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8100e20 0x7fefd81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 -- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 msgr2=0x7fefd819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 -- 172.21.15.77:0/3741777029 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd819d480 con 0x7fefd8102570 2024-09-15T22:19:20.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd7fff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefcffff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 0x7fefd8198a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefc002a9a0 tx=0x7fefc0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefd57fa640 1 -- 172.21.15.77:0/3741777029 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc003f070 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.998+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd819d710 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.999+0000 7fefd57fa640 1 -- 172.21.15.77:0/3741777029 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc0031c20 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.999+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd81a5050 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:19.999+0000 7fefd57fa640 1 -- 172.21.15.77:0/3741777029 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0035030 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefd57fa640 1 -- 172.21.15.77:0/3741777029 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fefc0042070 con 0x7fefd8102570 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 Processor -- start 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 -- start start 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31de6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 0x556e31df3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e31d4c2f0 con 0x556e31de7b40 2024-09-15T22:19:20.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e31cf3cf0 con 0x556e31df1370 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.001+0000 7fefe04dc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e31d482b0 con 0x556e31de65d0 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefcffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 0x556e31df3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34232/0 (socket says 172.21.15.77:34232) 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd7fff640 1 -- 172.21.15.77:0/578998761 learned_addr learned my addr 172.21.15.77:0/578998761 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31de6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 -- 172.21.15.77:0/578998761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 msgr2=0x556e31df3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 0x556e31df3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 -- 172.21.15.77:0/578998761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 msgr2=0x556e31de6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd57fa640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 0x7fefc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 -- 172.21.15.77:0/578998761 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e31cd83a0 con 0x556e31de65d0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd7fff640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefcffff640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 0x556e31df3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd57fa640 1 -- 172.21.15.77:0/3741777029 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fefc00f03f0 con 0x7fefd8102570 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd7fff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 0x7fefc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefd77fe640 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31de6a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009c00 tx=0x7fefc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3dff9587b5a18f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.002+0000 7fefcdffb640 1 -- 172.21.15.77:0/578998761 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004230 con 0x556e31de65d0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefcdffb640 1 -- 172.21.15.77:0/578998761 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc80043d0 con 0x556e31de65d0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefcdffb640 1 -- 172.21.15.77:0/578998761 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004dd0 con 0x556e31de65d0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 -- 172.21.15.77:0/578998761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 msgr2=0x556e31de6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefd7fff640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 0x7fefc40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefd00094f0 tx=0x7fefd0004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31de6a10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009c00 tx=0x7fefc8011810 comp rx=0 tx=0).stop 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 -- 172.21.15.77:0/578998761 shutdown_connections 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31df1370 0x556e31df3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31de6a10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 --2- 172.21.15.77:0/578998761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31de7b40 0x556e31de6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 -- 172.21.15.77:0/578998761 >> 172.21.15.77:0/578998761 conn(0x556e31daf310 msgr2=0x556e31de2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.003+0000 7fefe04dc740 1 -- 172.21.15.77:0/578998761 shutdown_connections 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 -- 172.21.15.77:0/578998761 wait complete. 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 -- start start 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 Processor -- start 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31df0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e31df4070 con 0x556e31df1370 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e31d07060 con 0x556e31de65d0 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556e31df41f0 con 0x556e31de7b40 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefe04dc740 10 client.0 _ll_register_callbacks cb 0x556e31a5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:20.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.004+0000 7fefcffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47208/0 (socket says 172.21.15.77:47208) 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 -- 172.21.15.77:0/3817056696 learned_addr learned my addr 172.21.15.77:0/3817056696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34246/0 (socket says 172.21.15.77:34246) 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefd77fe640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31df0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 -- 172.21.15.77:0/3817056696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 msgr2=0x556e31df0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31df0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 -- 172.21.15.77:0/3817056696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 msgr2=0x556e31dee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 -- 172.21.15.77:0/3817056696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e31abd3c0 con 0x556e31de7b40 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefd7fff640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefcffff640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefc00bc380 tx=0x7fefc003b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44fc52170f2ecec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:19:20.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0049070 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fefc00eda30 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc80043d0 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fefc00bbf30 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefe04dc740 10 client.26365 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.005+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e31df40d0 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc80043d0 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fefc004c060 con 0x556e31de7b40 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 10 client.26365 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 10 client.26365 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 20 client.26365 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e05300 0x556e31e076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 4 client.26365 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:20.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e31d59860 con 0x556e31e05300 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefd77fe640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e05300 0x556e31e076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefb6ffd640 10 client.26365 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.006+0000 7fefd77fe640 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e05300 0x556e31e076f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7c5f548ebc00bcc server_cookie=8c051c5616fc8e07 in_seq=0 out_seq=0 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefb6ffd640 1 -- 172.21.15.77:0/3817056696 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e31d59860 con 0x556e31e05300 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefb6ffd640 10 client.26365 handle_command_reply: tid=0 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 client.26365 shutdown 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 msgr2=0x556e31dedfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefc00bc380 tx=0x7fefc003b790 comp rx=0 tx=0).stop 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 shutdown_connections 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e05300 0x556e31e076f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31dee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31dedfd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 --2- 172.21.15.77:0/3817056696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31df0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefb6ffd640 0 client.26365 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 >> 172.21.15.77:0/3817056696 conn(0x556e31daf310 msgr2=0x556e31de8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 shutdown_connections 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 1 -- 172.21.15.77:0/3817056696 wait complete. 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.007+0000 7fefe04dc740 20 client.26365 trim_cache size 0 max 16384 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 Processor -- start 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 -- start start 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31de2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31df2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81acc00 con 0x556e31df1370 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81ae740 con 0x556e31de65d0 2024-09-15T22:19:20.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefe04dc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81ad2d0 con 0x556e31de7b40 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34254/0 (socket says 172.21.15.77:34254) 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31de2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 -- 172.21.15.77:0/964792339 learned_addr learned my addr 172.21.15.77:0/964792339 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefcffff640 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31df2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 -- 172.21.15.77:0/964792339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 msgr2=0x556e31df2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31df2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 -- 172.21.15.77:0/964792339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 msgr2=0x556e31de2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31de2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd7fff640 1 -- 172.21.15.77:0/964792339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e31e01b60 con 0x556e31df1370 2024-09-15T22:19:20.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.021+0000 7fefd77fe640 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31de2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.022+0000 7fefd7fff640 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fefd00143f0 tx=0x7fefd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23654e17123eff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.022+0000 7fefcd7fa640 1 -- 172.21.15.77:0/964792339 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0012e20 con 0x556e31df1370 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.022+0000 7fefcd7fa640 1 -- 172.21.15.77:0/964792339 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd0020900 con 0x556e31df1370 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.022+0000 7fefcd7fa640 1 -- 172.21.15.77:0/964792339 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd001d9b0 con 0x556e31df1370 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- 172.21.15.77:0/964792339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 msgr2=0x556e31df2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fefd00143f0 tx=0x7fefd0017000 comp rx=0 tx=0).stop 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- 172.21.15.77:0/964792339 shutdown_connections 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31df2ca0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de7b40 0x556e31df2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 --2- 172.21.15.77:0/964792339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de65d0 0x556e31de2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- 172.21.15.77:0/964792339 >> 172.21.15.77:0/964792339 conn(0x556e31daf310 msgr2=0x556e31df33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- 172.21.15.77:0/964792339 shutdown_connections 2024-09-15T22:19:20.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- 172.21.15.77:0/964792339 wait complete. 2024-09-15T22:19:20.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 -- start start 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.023+0000 7fefe04dc740 1 Processor -- start 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31df9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81ae070 con 0x556e31df1370 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81ad9a0 con 0x556e31de7b40 2024-09-15T22:19:20.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81aac00 con 0x556e31de65d0 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 10 client.0 _ll_register_callbacks cb 0x7fefc00f3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefd77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31df9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53186/0 (socket says 172.21.15.77:53186) 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34260/0 (socket says 172.21.15.77:34260) 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 -- 172.21.15.77:0/2874461704 learned_addr learned my addr 172.21.15.77:0/2874461704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 -- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 msgr2=0x556e31df9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31df9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 -- 172.21.15.77:0/2874461704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 msgr2=0x556e31deb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 -- 172.21.15.77:0/2874461704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e31deb890 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefd77fe640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31df9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefd7fff640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefcffff640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fefc00ba6a0 tx=0x7fefc002a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a0d844e329e2fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0049040 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fefc00bba40 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.024+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd0020900 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fefc00bbd50 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 10 client.26456 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e31e6ffb0 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd0020900 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fefc00d50a0 con 0x556e31de7b40 2024-09-15T22:19:20.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 10 client.26456 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 10 client.26456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 20 client.26456 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e703c0 0x556e31e727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 4 client.26456 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e31d53d70 con 0x556e31e703c0 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.025+0000 7fefd77fe640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e703c0 0x556e31e727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefb6ffd640 10 client.26456 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefd77fe640 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e703c0 0x556e31e727e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63c57c9f65dd118b server_cookie=5d286b811cb3be17 in_seq=0 out_seq=0 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefb6ffd640 1 -- 172.21.15.77:0/2874461704 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e31d53d70 con 0x556e31e703c0 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefb6ffd640 10 client.26456 handle_command_reply: tid=0 2024-09-15T22:19:20.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 client.26456 shutdown 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 msgr2=0x556e31deab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fefc00ba6a0 tx=0x7fefc002a480 comp rx=0 tx=0).stop 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 shutdown_connections 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x556e31e703c0 0x556e31e727e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e31df1370 0x556e31deb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e31de7b40 0x556e31deab50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefb6ffd640 0 client.26456 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 --2- 172.21.15.77:0/2874461704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556e31de65d0 0x556e31df9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 >> 172.21.15.77:0/2874461704 conn(0x556e31daf310 msgr2=0x556e31dec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 shutdown_connections 2024-09-15T22:19:20.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 1 -- 172.21.15.77:0/2874461704 wait complete. 2024-09-15T22:19:20.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.026+0000 7fefe04dc740 20 client.26456 trim_cache size 0 max 16384 2024-09-15T22:19:20.046 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 msgr2=0x7fefc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 0x7fefc40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefd00094f0 tx=0x7fefd0004a40 comp rx=0 tx=0).stop 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 msgr2=0x7fefd8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 0x7fefd8198a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefc002a9a0 tx=0x7fefc0003b90 comp rx=0 tx=0).stop 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 shutdown_connections 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fefc40a33b0 0x7fefc40a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd8102ef0 0x7fefd819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd8102570 0x7fefd8198a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 --2- 172.21.15.77:0/3741777029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8100e20 0x7fefd81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 >> 172.21.15.77:0/3741777029 conn(0x7fefd80fc7d0 msgr2=0x7fefd806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 shutdown_connections 2024-09-15T22:19:20.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:20.027+0000 7fefde103640 1 -- 172.21.15.77:0/3741777029 wait complete. 2024-09-15T22:19:20.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:19 smithi196 ceph-mon[35109]: pgmap v212: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:20.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:19 smithi117 ceph-mon[35021]: pgmap v212: 129 pgs: 129 active+clean; 332 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 389 op/s 2024-09-15T22:19:20.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 504876 7.60 MB/sec execute 209 sec latency 41.039 ms 2024-09-15T22:19:20.800 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:19:21.061 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:21.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 506837 7.60 MB/sec execute 210 sec latency 31.029 ms 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 -- 172.21.15.77:0/3155828013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 msgr2=0x7f38ac10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 --2- 172.21.15.77:0/3155828013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac10bb60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f389c009900 tx=0x7f389c02a480 comp rx=0 tx=0).stop 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 -- 172.21.15.77:0/3155828013 shutdown_connections 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 --2- 172.21.15.77:0/3155828013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 0x7f38ac10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 --2- 172.21.15.77:0/3155828013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac10bb60 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 --2- 172.21.15.77:0/3155828013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 0x7f38ac103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 -- 172.21.15.77:0/3155828013 >> 172.21.15.77:0/3155828013 conn(0x7f38ac0fc7d0 msgr2=0x7f38ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 -- 172.21.15.77:0/3155828013 shutdown_connections 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.543+0000 7f38b3d84640 1 -- 172.21.15.77:0/3155828013 wait complete. 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 Processor -- start 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 -- start start 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 0x7f38ac1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 0x7f38ac101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111f80 con 0x7f38ac103670 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111e00 con 0x7f38ac102d50 2024-09-15T22:19:21.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b3d84640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac112100 con 0x7f38ac10c180 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b131f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 0x7f38ac1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b131f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34278/0 (socket says 172.21.15.77:34278) 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b131f640 1 -- 172.21.15.77:0/10394018 learned_addr learned my addr 172.21.15.77:0/10394018 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.544+0000 7f38b2321640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 0x7f38ac101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 -- 172.21.15.77:0/10394018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 msgr2=0x7f38ac101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 0x7f38ac101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 -- 172.21.15.77:0/10394018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 msgr2=0x7f38ac1009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 0x7f38ac1009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 -- 172.21.15.77:0/10394018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac1a6920 con 0x7f38ac103670 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b131f640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f389c006010 tx=0x7f389c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38a2ffd640 1 -- 172.21.15.77:0/10394018 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c03f030 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38a2ffd640 1 -- 172.21.15.77:0/10394018 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c004440 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38a2ffd640 1 -- 172.21.15.77:0/10394018 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c035070 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac1a6bb0 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.545+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a7090 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38a2ffd640 1 -- 172.21.15.77:0/10394018 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f389c042030 con 0x7f38ac103670 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 Processor -- start 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 -- start start 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027087ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 0x560270888760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5602707e12f0 con 0x56027087cb40 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560270788cf0 con 0x56027087b5d0 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b615d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5602707dd2b0 con 0x560270886370 2024-09-15T22:19:21.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38b2321640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.548+0000 7f38a2ffd640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 0x7f38980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34280/0 (socket says 172.21.15.77:34280) 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 -- 172.21.15.77:0/2361889787 learned_addr learned my addr 172.21.15.77:0/2361889787 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b131f640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 0x560270888760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b1b20640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027087ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 0x7f38980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 -- 172.21.15.77:0/2361889787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 msgr2=0x560270888760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 0x560270888760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 -- 172.21.15.77:0/2361889787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 msgr2=0x56027087ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027087ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38a2ffd640 1 -- 172.21.15.77:0/10394018 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f389c0f03f0 con 0x7f38ac103670 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 -- 172.21.15.77:0/2361889787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027076d3a0 con 0x56027087cb40 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b131f640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 0x560270888760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005c40 tx=0x7f38a4013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5fcdc6d8001b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f3897fff640 1 -- 172.21.15.77:0/2361889787 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4022020 con 0x56027087cb40 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f38b2321640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 0x7f38980a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38a4023000 tx=0x7f38a4021000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f3897fff640 1 -- 172.21.15.77:0/2361889787 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a40134e0 con 0x56027087cb40 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.549+0000 7f3897fff640 1 -- 172.21.15.77:0/2361889787 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4010bb0 con 0x56027087cb40 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 -- 172.21.15.77:0/2361889787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 msgr2=0x56027087b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005c40 tx=0x7f38a4013f50 comp rx=0 tx=0).stop 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 -- 172.21.15.77:0/2361889787 shutdown_connections 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560270886370 0x560270888760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027087ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 --2- 172.21.15.77:0/2361889787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x56027087b090 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 -- 172.21.15.77:0/2361889787 >> 172.21.15.77:0/2361889787 conn(0x560270844310 msgr2=0x560270877c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 -- 172.21.15.77:0/2361889787 shutdown_connections 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.550+0000 7f38b615d740 1 -- 172.21.15.77:0/2361889787 wait complete. 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 -- start start 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 Processor -- start 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087b5d0 0x560270885dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x560270883810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560270889070 con 0x56027087cb40 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56027079c060 con 0x560270886370 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5602708891f0 con 0x56027087b5d0 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b615d740 10 client.0 _ll_register_callbacks cb 0x5602704ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b131f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087b5d0 0x560270885dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b131f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34288/0 (socket says 172.21.15.77:34288) 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b131f640 1 -- 172.21.15.77:0/4164486585 learned_addr learned my addr 172.21.15.77:0/4164486585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.551+0000 7f38b2321640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x560270883810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 -- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087b5d0 msgr2=0x560270885dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087b5d0 0x560270885dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 -- 172.21.15.77:0/4164486585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 msgr2=0x560270883810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x560270883810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 -- 172.21.15.77:0/4164486585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602705523c0 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b131f640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b8850 tx=0x7f389c0daf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505fdfd6cb431231 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38957fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c049070 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f389c003cd0 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602707ee860 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f389c0c4fb0 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b615d740 10 client.17304 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560270894b90 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.552+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c0bc3f0 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f389c031bd0 con 0x56027087cb40 2024-09-15T22:19:21.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 10 client.17304 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 10 client.17304 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 20 client.17304 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x5602708982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 4 client.17304 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602707ee860 con 0x560270895ee0 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b1b20640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x5602708982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38957fa640 10 client.17304 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.553+0000 7f38b1b20640 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x5602708982d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2944be649b4d242e server_cookie=19a215f55a175569 in_seq=0 out_seq=0 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38957fa640 1 -- 172.21.15.77:0/4164486585 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602707ee860 con 0x560270895ee0 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38957fa640 10 client.17304 handle_command_reply: tid=0 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 client.17304 shutdown 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 msgr2=0x560270883080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f389c0b8850 tx=0x7f389c0daf80 comp rx=0 tx=0).stop 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 shutdown_connections 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x5602708982d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x560270883810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087cb40 0x560270883080 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38957fa640 0 client.17304 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 --2- 172.21.15.77:0/4164486585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087b5d0 0x560270885dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.554+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 >> 172.21.15.77:0/4164486585 conn(0x560270844310 msgr2=0x56027087d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.555+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 shutdown_connections 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.555+0000 7f38b615d740 1 -- 172.21.15.77:0/4164486585 wait complete. 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.555+0000 7f38b615d740 20 client.17304 trim_cache size 0 max 16384 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.567+0000 7f38b615d740 1 Processor -- start 2024-09-15T22:19:21.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.567+0000 7f38b615d740 1 -- start start 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027087e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x56027087e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1acd00 con 0x56027087b5d0 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ae840 con 0x560270886370 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b615d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ad3d0 con 0x56027087cb40 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34298/0 (socket says 172.21.15.77:34298) 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 -- 172.21.15.77:0/3797942446 learned_addr learned my addr 172.21.15.77:0/3797942446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b131f640 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027087e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b2321640 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x56027087e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 -- 172.21.15.77:0/3797942446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 msgr2=0x56027087e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027087e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 -- 172.21.15.77:0/3797942446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 msgr2=0x56027087e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x56027087e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 -- 172.21.15.77:0/3797942446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560270878960 con 0x56027087b5d0 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38b1b20640 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005fd0 tx=0x7f38a800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61416db1980d35d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.568+0000 7f38977fe640 1 -- 172.21.15.77:0/3797942446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004220 con 0x56027087b5d0 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38977fe640 1 -- 172.21.15.77:0/3797942446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a80043c0 con 0x56027087b5d0 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38977fe640 1 -- 172.21.15.77:0/3797942446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8003dd0 con 0x56027087b5d0 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 -- 172.21.15.77:0/3797942446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 msgr2=0x560270884ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005fd0 tx=0x7f38a800b410 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 -- 172.21.15.77:0/3797942446 shutdown_connections 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560270886370 0x56027087e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027087e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 --2- 172.21.15.77:0/3797942446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56027087b5d0 0x560270884ff0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 -- 172.21.15.77:0/3797942446 >> 172.21.15.77:0/3797942446 conn(0x560270844310 msgr2=0x5602708848e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 -- 172.21.15.77:0/3797942446 shutdown_connections 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.569+0000 7f38b615d740 1 -- 172.21.15.77:0/3797942446 wait complete. 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 -- start start 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 Processor -- start 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 0x56027088ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ae170 con 0x560270886370 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1adaa0 con 0x56027087b5d0 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1aad00 con 0x56027087cb40 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b615d740 10 client.0 _ll_register_callbacks cb 0x7f389c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b2321640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 0x56027088ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b131f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b131f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47276/0 (socket says 172.21.15.77:47276) 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b1b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53246/0 (socket says 172.21.15.77:53246) 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.570+0000 7f38b131f640 1 -- 172.21.15.77:0/1070171493 learned_addr learned my addr 172.21.15.77:0/1070171493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 -- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 msgr2=0x56027088a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 -- 172.21.15.77:0/1070171493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 msgr2=0x56027088ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 0x56027088ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 -- 172.21.15.77:0/1070171493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602708ffd50 con 0x56027087b5d0 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b2321640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 0x56027088ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b131f640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:21.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b1b20640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005d00 tx=0x7f38a8005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0c81e5ce195c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004220 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f38a8003b50 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a80043c0 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f38a8003d50 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b615d740 10 client.26474 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.571+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560270901010 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a80043c0 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f38a8005440 con 0x56027087b5d0 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 10 client.26474 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 10 client.26474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 20 client.26474 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:21.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x560270903680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 4 client.26474 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602707e8d70 con 0x560270895ee0 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b131f640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x560270903680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38957fa640 10 client.26474 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.572+0000 7f38b131f640 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x560270903680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0ca83531bc127b5 server_cookie=675e3e541b24b28e in_seq=0 out_seq=0 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38957fa640 1 -- 172.21.15.77:0/1070171493 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602707e8d70 con 0x560270895ee0 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38957fa640 10 client.26474 handle_command_reply: tid=0 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 client.26474 shutdown 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 msgr2=0x56027088a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005d00 tx=0x7f38a8005c10 comp rx=0 tx=0).stop 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 shutdown_connections 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x560270895ee0 0x560270903680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560270886370 0x56027088ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56027087cb40 0x56027088a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38957fa640 0 client.26474 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 --2- 172.21.15.77:0/1070171493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56027087b5d0 0x56027088a410 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 >> 172.21.15.77:0/1070171493 conn(0x560270844310 msgr2=0x560270885100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 shutdown_connections 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 1 -- 172.21.15.77:0/1070171493 wait complete. 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b615d740 20 client.26474 trim_cache size 0 max 16384 2024-09-15T22:19:21.592 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:19:21.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.573+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 msgr2=0x7f38980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 0x7f38980a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38a4023000 tx=0x7f38a4021000 comp rx=0 tx=0).stop 2024-09-15T22:19:21.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 msgr2=0x7f38ac100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:21.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f389c006010 tx=0x7f389c004040 comp rx=0 tx=0).stop 2024-09-15T22:19:21.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 shutdown_connections 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38980a33b0 0x7f38980a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38ac10c180 0x7f38ac101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac103670 0x7f38ac100ef0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 --2- 172.21.15.77:0/10394018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ac102d50 0x7f38ac1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 >> 172.21.15.77:0/10394018 conn(0x7f38ac0fc7d0 msgr2=0x7f38ac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:21.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 shutdown_connections 2024-09-15T22:19:21.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:21.574+0000 7f38b3d84640 1 -- 172.21.15.77:0/10394018 wait complete. 2024-09-15T22:19:22.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:21 smithi077 ceph-mon[30980]: pgmap v213: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:22.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:21 smithi196 ceph-mon[35109]: pgmap v213: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:22.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:21 smithi117 ceph-mon[35021]: pgmap v213: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:22.296 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:19:22.475 INFO:tasks.workunit.client.0.smithi077.stdout: 1 508956 7.63 MB/sec execute 211 sec latency 11.182 ms 2024-09-15T22:19:22.560 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:23.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- 172.21.15.77:0/3300782787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 msgr2=0x7f500c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 --2- 172.21.15.77:0/3300782787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c10da40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5004009ca0 tx=0x7f500402a940 comp rx=0 tx=0).stop 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- 172.21.15.77:0/3300782787 shutdown_connections 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 --2- 172.21.15.77:0/3300782787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c10da40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 --2- 172.21.15.77:0/3300782787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c102570 0x7f500c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 --2- 172.21.15.77:0/3300782787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c100e20 0x7f500c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- 172.21.15.77:0/3300782787 >> 172.21.15.77:0/3300782787 conn(0x7f500c0fc7d0 msgr2=0x7f500c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- 172.21.15.77:0/3300782787 shutdown_connections 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- 172.21.15.77:0/3300782787 wait complete. 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 Processor -- start 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.082+0000 7f50122dc640 1 -- start start 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c102570 0x7f500c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500c110f80 con 0x7f500c102ef0 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f500c110e00 con 0x7f500c100e20 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f50122dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f500c111100 con 0x7f500c102570 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f5010879640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53276/0 (socket says 172.21.15.77:53276) 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c102570 0x7f500c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f5010879640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34316/0 (socket says 172.21.15.77:34316) 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 -- 172.21.15.77:0/298001 learned_addr learned my addr 172.21.15.77:0/298001 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 -- 172.21.15.77:0/298001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c102570 msgr2=0x7f500c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c102570 0x7f500c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 -- 172.21.15.77:0/298001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 msgr2=0x7f500c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f500bfff640 1 -- 172.21.15.77:0/298001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c19d3e0 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.083+0000 7f5010879640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f500bfff640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d950 tx=0x7f4ffc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f50097fa640 1 -- 172.21.15.77:0/298001 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc017030 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c19d6d0 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f50097fa640 1 -- 172.21.15.77:0/298001 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ffc004020 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f50097fa640 1 -- 172.21.15.77:0/298001 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0047a0 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.084+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c1a4fb0 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.086+0000 7f50097fa640 1 -- 172.21.15.77:0/298001 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ffc0041d0 con 0x7f500c100e20 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.086+0000 7f50146b5740 1 Processor -- start 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 -- start start 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629d9ca62f0 con 0x5629d9d4b370 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629d9c4dcf0 con 0x5629d9d41b40 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50146b5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629d9ca22b0 con 0x5629d9d405d0 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f5010879640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f5010879640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53286/0 (socket says 172.21.15.77:53286) 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f5010879640 1 -- 172.21.15.77:0/3817986049 learned_addr learned my addr 172.21.15.77:0/3817986049 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50097fa640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 0x7f4ff40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 -- 172.21.15.77:0/3817986049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 msgr2=0x5629d9d40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 -- 172.21.15.77:0/3817986049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 msgr2=0x5629d9d40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 -- 172.21.15.77:0/3817986049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d9c323a0 con 0x5629d9d4b370 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f5010879640 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f5010879640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 0x7f4ff40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f50097fa640 1 -- 172.21.15.77:0/298001 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4ffc0cfa60 con 0x7f500c100e20 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.087+0000 7f500b7fe640 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d4d760 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5000009bf0 tx=0x7f5000011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0790f4015cf7e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f4ff27fc640 1 -- 172.21.15.77:0/3817986049 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000004050 con 0x5629d9d4b370 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f4ff27fc640 1 -- 172.21.15.77:0/3817986049 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000005500 con 0x5629d9d4b370 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f4ff27fc640 1 -- 172.21.15.77:0/3817986049 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000004d70 con 0x5629d9d4b370 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 -- 172.21.15.77:0/3817986049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 msgr2=0x5629d9d4d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f5010879640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 0x7f4ff40a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5004009ca0 tx=0x7f50040095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d4d760 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5000009bf0 tx=0x7f5000011810 comp rx=0 tx=0).stop 2024-09-15T22:19:23.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 -- 172.21.15.77:0/3817986049 shutdown_connections 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d4d760 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 --2- 172.21.15.77:0/3817986049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.088+0000 7f50146b5740 1 -- 172.21.15.77:0/3817986049 >> 172.21.15.77:0/3817986049 conn(0x5629d9d09310 msgr2=0x5629d9d3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- 172.21.15.77:0/3817986049 shutdown_connections 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- 172.21.15.77:0/3817986049 wait complete. 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- start start 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 Processor -- start 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629d9d4e070 con 0x5629d9d4b370 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5629d9c61060 con 0x5629d9d41b40 2024-09-15T22:19:23.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5629d9d4e1f0 con 0x5629d9d405d0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f50146b5740 10 client.0 _ll_register_callbacks cb 0x5629d99b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.089+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f5010879640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47314/0 (socket says 172.21.15.77:47314) 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 -- 172.21.15.77:0/814171999 learned_addr learned my addr 172.21.15.77:0/814171999 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 -- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 msgr2=0x5629d9d47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 -- 172.21.15.77:0/814171999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 msgr2=0x5629d9d484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 -- 172.21.15.77:0/814171999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d9a173c0 con 0x5629d9d405d0 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500b7fe640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:23.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f500bfff640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cce80 tx=0x7f4ffc0d31a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338ddb46fe376fb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f4feb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc09b310 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5000005500 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4ffc0cd0a0 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ffc09c0b0 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f50146b5740 10 client.26401 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.090+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629d9d59ae0 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000005500 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4ffc09cce0 con 0x5629d9d405d0 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 10 client.26401 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:23.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 10 client.26401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 20 client.26401 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9d5afa0 0x5629d9d5d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 4 client.26401 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629d9cb3860 con 0x5629d9d5afa0 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f500b7fe640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9d5afa0 0x5629d9d5d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f4feb7fe640 10 client.26401 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.091+0000 7f500b7fe640 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9d5afa0 0x5629d9d5d390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c75779e062aa795c server_cookie=4cb5f947cb2afa73 in_seq=0 out_seq=0 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f4feb7fe640 1 -- 172.21.15.77:0/814171999 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629d9cb3860 con 0x5629d9d5afa0 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f4feb7fe640 10 client.26401 handle_command_reply: tid=0 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 client.26401 shutdown 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 msgr2=0x5629d9d4acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cce80 tx=0x7f4ffc0d31a0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 shutdown_connections 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9d5afa0 0x5629d9d5d390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d4b370 0x5629d9d484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d41b40 0x5629d9d47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 --2- 172.21.15.77:0/814171999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d4acc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 >> 172.21.15.77:0/814171999 conn(0x5629d9d09310 msgr2=0x5629d9d42960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f4feb7fe640 0 client.26401 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 shutdown_connections 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 1 -- 172.21.15.77:0/814171999 wait complete. 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.092+0000 7f50146b5740 20 client.26401 trim_cache size 0 max 16384 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.105+0000 7f50146b5740 1 Processor -- start 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 -- start start 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1aef70 con 0x5629d9d41b40 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1b0ab0 con 0x5629d9d4b370 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f50146b5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1af640 con 0x5629d9d405d0 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34344/0 (socket says 172.21.15.77:34344) 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47324/0 (socket says 172.21.15.77:47324) 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 -- 172.21.15.77:0/628116290 learned_addr learned my addr 172.21.15.77:0/628116290 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f5010879640 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 -- 172.21.15.77:0/628116290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 msgr2=0x5629d9d57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 -- 172.21.15.77:0/628116290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 msgr2=0x5629d9d4cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d4cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 -- 172.21.15.77:0/628116290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d9d5da10 con 0x5629d9d41b40 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500bfff640 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.106+0000 7f500b7fe640 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5000009b40 tx=0x7f50000049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2fb2b86442599 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f4ff1ffb640 1 -- 172.21.15.77:0/628116290 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000033070 con 0x5629d9d41b40 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f4ff1ffb640 1 -- 172.21.15.77:0/628116290 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000004840 con 0x5629d9d41b40 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f4ff1ffb640 1 -- 172.21.15.77:0/628116290 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000003850 con 0x5629d9d41b40 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 -- 172.21.15.77:0/628116290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 msgr2=0x5629d9d4c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5000009b40 tx=0x7f50000049f0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 -- 172.21.15.77:0/628116290 shutdown_connections 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d4cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d4c760 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 --2- 172.21.15.77:0/628116290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 -- 172.21.15.77:0/628116290 >> 172.21.15.77:0/628116290 conn(0x5629d9d09310 msgr2=0x5629d9d46840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 -- 172.21.15.77:0/628116290 shutdown_connections 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.107+0000 7f50146b5740 1 -- 172.21.15.77:0/628116290 wait complete. 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 -- start start 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 Processor -- start 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d63200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1b03e0 con 0x5629d9d41b40 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1afd10 con 0x5629d9d4b370 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1acf70 con 0x5629d9d405d0 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.108+0000 7f50146b5740 10 client.0 _ll_register_callbacks cb 0x7f4ffc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f5010879640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d63200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47326/0 (socket says 172.21.15.77:47326) 2024-09-15T22:19:23.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 -- 172.21.15.77:0/317327205 learned_addr learned my addr 172.21.15.77:0/317327205 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f5010879640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d63200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53306/0 (socket says 172.21.15.77:53306) 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500b7fe640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 -- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 msgr2=0x5629d9d63200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d63200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 -- 172.21.15.77:0/317327205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 msgr2=0x5629d9d62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d62cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 -- 172.21.15.77:0/317327205 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d9d51900 con 0x5629d9d405d0 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f500bfff640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc097e10 tx=0x7f4ffc09ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81bfdd7fafd095fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:23.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f4feb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:19:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc09b310 con 0x5629d9d405d0 2024-09-15T22:19:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f4ffc097900 con 0x5629d9d405d0 2024-09-15T22:19:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.109+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5000004840 con 0x5629d9d405d0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ffc0a4620 con 0x5629d9d405d0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 10 client.26407 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629d9dc5e40 con 0x5629d9d405d0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000004840 con 0x5629d9d405d0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4ffc09cce0 con 0x5629d9d405d0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 10 client.26407 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 10 client.26407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 20 client.26407 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9dc65b0 0x5629d9dc89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 4 client.26407 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629d9cadd70 con 0x5629d9dc65b0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.110+0000 7f500b7fe640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9dc65b0 0x5629d9dc89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f4feb7fe640 10 client.26407 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f500b7fe640 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9dc65b0 0x5629d9dc89d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a373523d4293223 server_cookie=ef8c19de6fc01dab in_seq=0 out_seq=0 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f4feb7fe640 1 -- 172.21.15.77:0/317327205 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629d9cadd70 con 0x5629d9dc65b0 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f4feb7fe640 10 client.26407 handle_command_reply: tid=0 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 client.26407 shutdown 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 msgr2=0x5629d9d62780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc097e10 tx=0x7f4ffc09ac90 comp rx=0 tx=0).stop 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 shutdown_connections 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5629d9dc65b0 0x5629d9dc89d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629d9d4b370 0x5629d9d63200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629d9d41b40 0x5629d9d62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 --2- 172.21.15.77:0/317327205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629d9d405d0 0x5629d9d62780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.111+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 >> 172.21.15.77:0/317327205 conn(0x5629d9d09310 msgr2=0x5629d9d497d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.112+0000 7f4feb7fe640 0 client.26407 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:19:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.112+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 shutdown_connections 2024-09-15T22:19:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.112+0000 7f50146b5740 1 -- 172.21.15.77:0/317327205 wait complete. 2024-09-15T22:19:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.112+0000 7f50146b5740 20 client.26407 trim_cache size 0 max 16384 2024-09-15T22:19:23.129 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 msgr2=0x7f4ff40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 0x7f4ff40a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5004009ca0 tx=0x7f50040095c0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 msgr2=0x7f500c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d950 tx=0x7f4ffc00de10 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 shutdown_connections 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4ff40a33b0 0x7f4ff40a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f500c102ef0 0x7f500c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500c102570 0x7f500c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 --2- 172.21.15.77:0/298001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f500c100e20 0x7f500c198430 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 >> 172.21.15.77:0/298001 conn(0x7f500c0fc7d0 msgr2=0x7f500c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:23.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 shutdown_connections 2024-09-15T22:19:23.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:23.114+0000 7f50122dc640 1 -- 172.21.15.77:0/298001 wait complete. 2024-09-15T22:19:23.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 511641 7.67 MB/sec execute 212 sec latency 63.008 ms 2024-09-15T22:19:24.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:23 smithi077 ceph-mon[30980]: pgmap v214: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:24.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:23 smithi196 ceph-mon[35109]: pgmap v214: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:24.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:23 smithi117 ceph-mon[35021]: pgmap v214: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:19:24.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 513441 7.67 MB/sec execute 213 sec latency 25.782 ms 2024-09-15T22:19:24.952 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:19:25.226 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:25.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 515808 7.70 MB/sec execute 214 sec latency 13.507 ms 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 -- 172.21.15.77:0/1662182982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 msgr2=0x7f24a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/1662182982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a8101200 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2498009900 tx=0x7f249802a480 comp rx=0 tx=0).stop 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 -- 172.21.15.77:0/1662182982 shutdown_connections 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/1662182982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/1662182982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 0x7f24a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/1662182982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a8101200 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 -- 172.21.15.77:0/1662182982 >> 172.21.15.77:0/1662182982 conn(0x7f24a80fc7d0 msgr2=0x7f24a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 -- 172.21.15.77:0/1662182982 shutdown_connections 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.749+0000 7f24ad1d6640 1 -- 172.21.15.77:0/1662182982 wait complete. 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.750+0000 7f24ad1d6640 1 Processor -- start 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.750+0000 7f24ad1d6640 1 -- start start 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 0x7f24a8196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810fe90 con 0x7f24a8100e20 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810fd10 con 0x7f24a8102ef0 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24ad1d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8110010 con 0x7f24a8102570 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34370/0 (socket says 172.21.15.77:34370) 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 -- 172.21.15.77:0/3947809236 learned_addr learned my addr 172.21.15.77:0/3947809236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a659c640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 0x7f24a8196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a759e640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 -- 172.21.15.77:0/3947809236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 msgr2=0x7f24a8196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 0x7f24a8196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 -- 172.21.15.77:0/3947809236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 msgr2=0x7f24a819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 -- 172.21.15.77:0/3947809236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a819b270 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a759e640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2498006010 tx=0x7f2498003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.751+0000 7f248ffff640 1 -- 172.21.15.77:0/3947809236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249803f070 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.752+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a819b500 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.752+0000 7f248ffff640 1 -- 172.21.15.77:0/3947809236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498031c30 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.752+0000 7f248ffff640 1 -- 172.21.15.77:0/3947809236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498035030 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.752+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a2e40 con 0x7f24a8100e20 2024-09-15T22:19:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f248ffff640 1 -- 172.21.15.77:0/3947809236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2498042070 con 0x7f24a8100e20 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 Processor -- start 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 -- start start 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5621c50c82f0 con 0x5621c516d370 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5621c506fcf0 con 0x5621c5163b40 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24af5af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621c50c42b0 con 0x5621c51625d0 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34386/0 (socket says 172.21.15.77:34386) 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53332/0 (socket says 172.21.15.77:53332) 2024-09-15T22:19:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a659c640 1 -- 172.21.15.77:0/324540445 learned_addr learned my addr 172.21.15.77:0/324540445 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 -- 172.21.15.77:0/324540445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 msgr2=0x5621c5162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 -- 172.21.15.77:0/324540445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 msgr2=0x5621c516f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f248ffff640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 0x7f24900a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.755+0000 7f24a759e640 1 -- 172.21.15.77:0/324540445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c50543a0 con 0x5621c5163b40 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f24a659c640 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 0x7f24900a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f24a759e640 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009f00 tx=0x7f24a000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d020498a2efa45dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f248ffff640 1 -- 172.21.15.77:0/3947809236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f24980f0910 con 0x7f24a8100e20 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f248d7fa640 1 -- 172.21.15.77:0/324540445 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0015070 con 0x5621c5163b40 2024-09-15T22:19:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f248d7fa640 1 -- 172.21.15.77:0/324540445 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a0011920 con 0x5621c5163b40 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.756+0000 7f248d7fa640 1 -- 172.21.15.77:0/324540445 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0010910 con 0x5621c5163b40 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- 172.21.15.77:0/324540445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 msgr2=0x5621c5162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24a6d9d640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 0x7f24900a7a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24980edd80 tx=0x7f24980f6550 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009f00 tx=0x7f24a000f730 comp rx=0 tx=0).stop 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- 172.21.15.77:0/324540445 shutdown_connections 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- 172.21.15.77:0/324540445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c5162090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- 172.21.15.77:0/324540445 >> 172.21.15.77:0/324540445 conn(0x5621c512b310 msgr2=0x5621c515ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- 172.21.15.77:0/324540445 shutdown_connections 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- 172.21.15.77:0/324540445 wait complete. 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- start start 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 Processor -- start 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c516cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c516a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5621c5170070 con 0x5621c516d370 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5621c5083060 con 0x5621c5163b40 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.757+0000 7f24af5af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621c51701f0 con 0x5621c51625d0 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24af5af740 10 client.0 _ll_register_callbacks cb 0x5621c4dd68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34390/0 (socket says 172.21.15.77:34390) 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 -- 172.21.15.77:0/4243239844 learned_addr learned my addr 172.21.15.77:0/4243239844 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a659c640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c516a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c516cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 -- 172.21.15.77:0/4243239844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 msgr2=0x5621c516cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c516cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 -- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 msgr2=0x5621c516a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c516a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 -- 172.21.15.77:0/4243239844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c4e393c0 con 0x5621c516d370 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24a759e640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f24a000fc30 tx=0x7f24a0010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10129cfe2bb7ad8f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f248aff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0015040 con 0x5621c516d370 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f24a0011ce0 con 0x5621c516d370 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a0011920 con 0x5621c516d370 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24a0033050 con 0x5621c516d370 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24af5af740 10 client.17340 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.758+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621c517bb10 con 0x5621c516d370 2024-09-15T22:19:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a0011920 con 0x5621c516d370 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f24a0028020 con 0x5621c516d370 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 10 client.17340 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 10 client.17340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 20 client.17340 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c517f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 4 client.17340 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c50d5860 con 0x5621c517cfd0 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.759+0000 7f24a659c640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c517f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f248aff5640 10 client.17340 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24a659c640 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c517f3c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79dc4d0275184f77 server_cookie=e3e5dc0f801146e2 in_seq=0 out_seq=0 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f248aff5640 1 -- 172.21.15.77:0/4243239844 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621c50d5860 con 0x5621c517cfd0 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f248aff5640 10 client.17340 handle_command_reply: tid=0 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 client.17340 shutdown 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 msgr2=0x5621c516a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f24a000fc30 tx=0x7f24a0010470 comp rx=0 tx=0).stop 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 shutdown_connections 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c517f3c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c516a600 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f248aff5640 0 client.17340 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:25.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c516a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 --2- 172.21.15.77:0/4243239844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c516cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.760+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 >> 172.21.15.77:0/4243239844 conn(0x5621c512b310 msgr2=0x5621c5164960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.761+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 shutdown_connections 2024-09-15T22:19:25.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.761+0000 7f24af5af740 1 -- 172.21.15.77:0/4243239844 wait complete. 2024-09-15T22:19:25.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.761+0000 7f24af5af740 20 client.17340 trim_cache size 0 max 16384 2024-09-15T22:19:25.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 Processor -- start 2024-09-15T22:19:25.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 -- start start 2024-09-15T22:19:25.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5179310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c5165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81accd0 con 0x5621c516d370 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ae810 con 0x5621c5163b40 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24af5af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ad3a0 con 0x5621c51625d0 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c5165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c5165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34396/0 (socket says 172.21.15.77:34396) 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53356/0 (socket says 172.21.15.77:53356) 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a759e640 1 -- 172.21.15.77:0/974960245 learned_addr learned my addr 172.21.15.77:0/974960245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.774+0000 7f24a759e640 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5179310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 -- 172.21.15.77:0/974960245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 msgr2=0x5621c5179310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5179310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 -- 172.21.15.77:0/974960245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 msgr2=0x5621c5165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c5165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 -- 172.21.15.77:0/974960245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c5163760 con 0x5621c5163b40 2024-09-15T22:19:25.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f24a659c640 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5621c516b650 tx=0x7f249c00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27f255848e02b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f248cff9640 1 -- 172.21.15.77:0/974960245 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c004220 con 0x5621c5163b40 2024-09-15T22:19:25.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f248cff9640 1 -- 172.21.15.77:0/974960245 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c0043c0 con 0x5621c5163b40 2024-09-15T22:19:25.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.775+0000 7f248cff9640 1 -- 172.21.15.77:0/974960245 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c01d980 con 0x5621c5163b40 2024-09-15T22:19:25.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- 172.21.15.77:0/974960245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 msgr2=0x5621c51698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5621c516b650 tx=0x7f249c00b410 comp rx=0 tx=0).stop 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- 172.21.15.77:0/974960245 shutdown_connections 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c516d370 0x5621c5165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c5163b40 0x5621c51698d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 --2- 172.21.15.77:0/974960245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c51625d0 0x5621c5179310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- 172.21.15.77:0/974960245 >> 172.21.15.77:0/974960245 conn(0x5621c512b310 msgr2=0x5621c51688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- 172.21.15.77:0/974960245 shutdown_connections 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- 172.21.15.77:0/974960245 wait complete. 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 -- start start 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.776+0000 7f24af5af740 1 Processor -- start 2024-09-15T22:19:25.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c516d370 0x5621c5166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ae140 con 0x5621c5163b40 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ada70 con 0x5621c51625d0 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81aacd0 con 0x5621c516d370 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 10 client.0 _ll_register_callbacks cb 0x7f249c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34400/0 (socket says 172.21.15.77:34400) 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a659c640 1 -- 172.21.15.77:0/1634957216 learned_addr learned my addr 172.21.15.77:0/1634957216 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53368/0 (socket says 172.21.15.77:53368) 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c516d370 0x5621c5166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 -- 172.21.15.77:0/1634957216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c516d370 msgr2=0x5621c5166280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c516d370 0x5621c5166280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 -- 172.21.15.77:0/1634957216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 msgr2=0x5621c51759c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 -- 172.21.15.77:0/1634957216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c5167ef0 con 0x5621c51625d0 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a659c640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24a759e640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24a0014040 tx=0x7f24a00103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a89063269dc100 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f248aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0043060 con 0x5621c51625d0 2024-09-15T22:19:25.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249c0043c0 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.777+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f24a0010c60 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24a004a050 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 10 client.26510 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621c51e6fa0 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c0043c0 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f24a0028040 con 0x5621c51625d0 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 10 client.26510 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 20 client.26510 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:25.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c51e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 4 client.26510 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621c50cfd70 con 0x5621c517cfd0 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.778+0000 7f24a659c640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c51e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f248aff5640 10 client.26510 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24a659c640 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c51e9670 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efea9d336846939e server_cookie=3cec153c6b1cdac7 in_seq=0 out_seq=0 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f248aff5640 1 -- 172.21.15.77:0/1634957216 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621c50cfd70 con 0x5621c517cfd0 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f248aff5640 10 client.26510 handle_command_reply: tid=0 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 client.26510 shutdown 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 msgr2=0x5621c5175480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24a0014040 tx=0x7f24a00103d0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 shutdown_connections 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5621c517cfd0 0x5621c51e9670 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621c516d370 0x5621c5166280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f248aff5640 0 client.26510 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5621c5163b40 0x5621c51759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 --2- 172.21.15.77:0/1634957216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621c51625d0 0x5621c5175480 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 >> 172.21.15.77:0/1634957216 conn(0x5621c512b310 msgr2=0x5621c516c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 shutdown_connections 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 1 -- 172.21.15.77:0/1634957216 wait complete. 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.779+0000 7f24af5af740 20 client.26510 trim_cache size 0 max 16384 2024-09-15T22:19:25.788 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 msgr2=0x7f24900a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 0x7f24900a7a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24980edd80 tx=0x7f24980f6550 comp rx=0 tx=0).stop 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 msgr2=0x7f24a81962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2498006010 tx=0x7f2498003c10 comp rx=0 tx=0).stop 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 shutdown_connections 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24900a55c0 0x7f24900a7a80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24a8102ef0 0x7f24a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8102570 0x7f24a8196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 --2- 172.21.15.77:0/3947809236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24a8100e20 0x7f24a81962c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 >> 172.21.15.77:0/3947809236 conn(0x7f24a80fc7d0 msgr2=0x7f24a80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 shutdown_connections 2024-09-15T22:19:25.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:25.780+0000 7f24ad1d6640 1 -- 172.21.15.77:0/3947809236 wait complete. 2024-09-15T22:19:26.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:25 smithi077 ceph-mon[30980]: pgmap v215: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:19:26.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:25 smithi196 ceph-mon[35109]: pgmap v215: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:19:26.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:25 smithi117 ceph-mon[35021]: pgmap v215: 129 pgs: 129 active+clean; 342 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:19:26.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 518330 7.72 MB/sec execute 215 sec latency 27.666 ms 2024-09-15T22:19:27.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 520232 7.74 MB/sec execute 216 sec latency 13.999 ms 2024-09-15T22:19:27.591 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:19:27.591 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:19:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:27.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:27 smithi077 ceph-mon[30980]: pgmap v216: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:19:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:27 smithi196 ceph-mon[35109]: pgmap v216: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:19:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:27 smithi117 ceph-mon[35021]: pgmap v216: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:19:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 -- 172.21.15.77:0/1645158124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490102570 msgr2=0x7f74901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 --2- 172.21.15.77:0/1645158124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490102570 0x7f74901029b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7478009900 tx=0x7f747802a480 comp rx=0 tx=0).stop 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 -- 172.21.15.77:0/1645158124 shutdown_connections 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 --2- 172.21.15.77:0/1645158124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f749010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 --2- 172.21.15.77:0/1645158124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490102570 0x7f74901029b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 --2- 172.21.15.77:0/1645158124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490100e20 0x7f7490101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 -- 172.21.15.77:0/1645158124 >> 172.21.15.77:0/1645158124 conn(0x7f74900fc7d0 msgr2=0x7f74900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 -- 172.21.15.77:0/1645158124 shutdown_connections 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.392+0000 7f74956f0640 1 -- 172.21.15.77:0/1645158124 wait complete. 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 Processor -- start 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 -- start start 2024-09-15T22:19:28.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 0x7f7490199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f74901a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7490110f80 con 0x7f7490100e20 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7490110e00 con 0x7f7490102570 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f74956f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7490111100 con 0x7f7490102ef0 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.393+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34422/0 (socket says 172.21.15.77:34422) 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 -- 172.21.15.77:0/123607379 learned_addr learned my addr 172.21.15.77:0/123607379 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748f7fe640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f74901a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748e7fc640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 0x7f7490199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 -- 172.21.15.77:0/123607379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 msgr2=0x7f74901a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f74901a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 -- 172.21.15.77:0/123607379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 msgr2=0x7f7490199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 0x7f7490199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 -- 172.21.15.77:0/123607379 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901a48d0 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748f7fe640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f74901a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748e7fc640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 0x7f7490199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f748effd640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7488009d50 tx=0x7f748800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f7487fff640 1 -- 172.21.15.77:0/123607379 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488003b70 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f7487fff640 1 -- 172.21.15.77:0/123607379 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488003d10 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901a4aa0 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f7487fff640 1 -- 172.21.15.77:0/123607379 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488010a20 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.394+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a4f60 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.396+0000 7f7487fff640 1 -- 172.21.15.77:0/123607379 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7488003ec0 con 0x7f7490100e20 2024-09-15T22:19:28.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7497ac9740 1 Processor -- start 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7497ac9740 1 -- start start 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7487fff640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 0x7f747c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f748f7fe640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 0x7f747c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7487fff640 1 -- 172.21.15.77:0/123607379 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f74880d0760 con 0x7f7490100e20 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.397+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 0x558e37deb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f7497ac9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e37d43280 con 0x558e37de8d40 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f7497ac9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e37ceab10 con 0x558e37dddb60 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 0x558e37deb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 0x558e37deb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34434/0 (socket says 172.21.15.77:34434) 2024-09-15T22:19:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748f7fe640 1 -- 172.21.15.77:0/758924980 learned_addr learned my addr 172.21.15.77:0/758924980 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53402/0 (socket says 172.21.15.77:53402) 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e37d3f240 con 0x558e37dde480 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748f7fe640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 0x7f747c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7480009f90 tx=0x7f7480011810 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 -- 172.21.15.77:0/758924980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 msgr2=0x558e37de8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 -- 172.21.15.77:0/758924980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 msgr2=0x558e37deb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 0x558e37deb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.398+0000 7f748effd640 1 -- 172.21.15.77:0/758924980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e37cf71f0 con 0x558e37dddb60 2024-09-15T22:19:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.399+0000 7f748effd640 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f74880cdb20 tx=0x7f748809c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfad7a452dee4c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.399+0000 7f74857fa640 1 -- 172.21.15.77:0/758924980 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880b4070 con 0x558e37dddb60 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.399+0000 7f74857fa640 1 -- 172.21.15.77:0/758924980 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748809c6b0 con 0x558e37dddb60 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.399+0000 7f74857fa640 1 -- 172.21.15.77:0/758924980 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880cdf30 con 0x558e37dddb60 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 msgr2=0x558e37dddf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f74880cdb20 tx=0x7f748809c2b0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 shutdown_connections 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37de8d40 0x558e37deb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 --2- 172.21.15.77:0/758924980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37dddb60 0x558e37dddf40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 >> 172.21.15.77:0/758924980 conn(0x558e37dda0f0 msgr2=0x558e37dda4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 shutdown_connections 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- 172.21.15.77:0/758924980 wait complete. 2024-09-15T22:19:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 -- start start 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.400+0000 7f7497ac9740 1 Processor -- start 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37de6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e37deb910 con 0x558e37dddb60 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e37cfdf10 con 0x558e37de8d40 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558e37deba90 con 0x558e37dde480 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 10 client.0 _ll_register_callbacks cb 0x558e37cc6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34444/0 (socket says 172.21.15.77:34444) 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 -- 172.21.15.77:0/857670750 learned_addr learned my addr 172.21.15.77:0/857670750 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748e7fc640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748f7fe640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37de6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 -- 172.21.15.77:0/857670750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 msgr2=0x558e37de6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 -- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 msgr2=0x558e37de6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37de6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 -- 172.21.15.77:0/857670750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e37ab46a0 con 0x558e37dddb60 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f748effd640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f74880a4df0 tx=0x7f74880104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc419efbff55b723 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f74767fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880bb070 con 0x558e37dddb60 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748809c6b0 con 0x558e37dddb60 2024-09-15T22:19:28.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.401+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f74880d0380 con 0x558e37dddb60 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74880b4070 con 0x558e37dddb60 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 10 client.17358 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e37df74e0 con 0x558e37dddb60 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748809c6b0 con 0x558e37dddb60 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f74880bc040 con 0x558e37dddb60 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 10 client.17358 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 20 client.17358 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37df8830 0x558e37dfac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 4 client.17358 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e37d507f0 con 0x558e37df8830 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.402+0000 7f748e7fc640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37df8830 0x558e37dfac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f74767fc640 10 client.17358 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:28.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f748e7fc640 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37df8830 0x558e37dfac20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e3d598e22b7ad4 server_cookie=ba4c54b5e96c6ae5 in_seq=0 out_seq=0 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f74767fc640 1 -- 172.21.15.77:0/857670750 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e37d507f0 con 0x558e37df8830 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f74767fc640 10 client.17358 handle_command_reply: tid=0 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 client.17358 shutdown 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 msgr2=0x558e37de5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f74880a4df0 tx=0x7f74880104c0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 shutdown_connections 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37df8830 0x558e37dfac20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37de6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37de6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f74767fc640 0 client.17358 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 --2- 172.21.15.77:0/857670750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37de5be0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 >> 172.21.15.77:0/857670750 conn(0x558e37dda0f0 msgr2=0x558e37dd9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 shutdown_connections 2024-09-15T22:19:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.403+0000 7f7497ac9740 1 -- 172.21.15.77:0/857670750 wait complete. 2024-09-15T22:19:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.404+0000 7f7497ac9740 20 client.17358 trim_cache size 0 max 16384 2024-09-15T22:19:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.422+0000 7f7497ac9740 1 Processor -- start 2024-09-15T22:19:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.422+0000 7f7497ac9740 1 -- start start 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37dfdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37dfe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74901acde0 con 0x558e37dddb60 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ae920 con 0x558e37de8d40 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f7497ac9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ad4b0 con 0x558e37dde480 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34456/0 (socket says 172.21.15.77:34456) 2024-09-15T22:19:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 -- 172.21.15.77:0/3511995778 learned_addr learned my addr 172.21.15.77:0/3511995778 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37dfdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748f7fe640 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37dfe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 -- 172.21.15.77:0/3511995778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 msgr2=0x558e37dfdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37dfdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 -- 172.21.15.77:0/3511995778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 msgr2=0x558e37dfe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37dfe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 -- 172.21.15.77:0/3511995778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e37dd9b90 con 0x558e37dddb60 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748e7fc640 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37dfdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.423+0000 7f748effd640 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f74880ce120 tx=0x7f74880cdbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e2a79ee34c31b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.424+0000 7f7484ff9640 1 -- 172.21.15.77:0/3511995778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880bb050 con 0x558e37dddb60 2024-09-15T22:19:28.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.424+0000 7f7484ff9640 1 -- 172.21.15.77:0/3511995778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74880b0960 con 0x558e37dddb60 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.424+0000 7f7484ff9640 1 -- 172.21.15.77:0/3511995778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880a0a60 con 0x558e37dddb60 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- 172.21.15.77:0/3511995778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 msgr2=0x558e37df43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f74880ce120 tx=0x7f74880cdbd0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- 172.21.15.77:0/3511995778 shutdown_connections 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37dfe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dde480 0x558e37dfdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 --2- 172.21.15.77:0/3511995778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dddb60 0x558e37df43f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- 172.21.15.77:0/3511995778 >> 172.21.15.77:0/3511995778 conn(0x558e37dda0f0 msgr2=0x558e37df3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- 172.21.15.77:0/3511995778 shutdown_connections 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- 172.21.15.77:0/3511995778 wait complete. 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 -- start start 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.425+0000 7f7497ac9740 1 Processor -- start 2024-09-15T22:19:28.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 0x558e37dfc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dde480 0x558e37dfc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37e62170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ae250 con 0x558e37dde480 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74901adb80 con 0x558e37de8d40 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74901aade0 con 0x558e37dddb60 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 10 client.0 _ll_register_callbacks cb 0x7f7480009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37e62170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 0x558e37dfc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dde480 0x558e37dfc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 0x558e37dfc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47440/0 (socket says 172.21.15.77:47440) 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748effd640 1 -- 172.21.15.77:0/726802552 learned_addr learned my addr 172.21.15.77:0/726802552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 -- 172.21.15.77:0/726802552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 msgr2=0x558e37dfc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 0x558e37dfc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 -- 172.21.15.77:0/726802552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dde480 msgr2=0x558e37dfc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dde480 0x558e37dfc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 -- 172.21.15.77:0/726802552 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e37e626b0 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f748f7fe640 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37e62170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7480011d10 tx=0x7f7480004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7685019653dcff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f74767fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480017040 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74880b0960 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.426+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f74800214c0 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74800216c0 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 10 client.26528 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e37e639b0 con 0x558e37de8d40 2024-09-15T22:19:28.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74880b0960 con 0x558e37de8d40 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f748001d600 con 0x558e37de8d40 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 10 client.26528 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 20 client.26528 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37e640e0 0x558e37e664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 4 client.26528 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e37d4ad00 con 0x558e37e640e0 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f748effd640 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37e640e0 0x558e37e664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.427+0000 7f74767fc640 10 client.26528 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:28.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.428+0000 7f748effd640 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37e640e0 0x558e37e664d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d659443da6f827 server_cookie=2e2ee5c4d922ec9e in_seq=0 out_seq=0 2024-09-15T22:19:28.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.428+0000 7f74767fc640 1 -- 172.21.15.77:0/726802552 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558e37d4ad00 con 0x558e37e640e0 2024-09-15T22:19:28.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.428+0000 7f74767fc640 10 client.26528 handle_command_reply: tid=0 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.428+0000 7f7497ac9740 1 client.26528 shutdown 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 msgr2=0x558e37e62170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37e62170 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7480011d10 tx=0x7f7480004730 comp rx=0 tx=0).stop 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 shutdown_connections 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558e37e640e0 0x558e37e664d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e37de8d40 0x558e37e62170 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e37dde480 0x558e37dfc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f74767fc640 0 client.26528 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 --2- 172.21.15.77:0/726802552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558e37dddb60 0x558e37dfc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 >> 172.21.15.77:0/726802552 conn(0x558e37dda0f0 msgr2=0x558e37ddfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 shutdown_connections 2024-09-15T22:19:28.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 1 -- 172.21.15.77:0/726802552 wait complete. 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.429+0000 7f7497ac9740 20 client.26528 trim_cache size 0 max 16384 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "824a5fce-1e9b-4d64-9752-3347c0ba6d43", 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:19:28.436 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:19:28.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 msgr2=0x7f747c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 0x7f747c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7480009f90 tx=0x7f7480011810 comp rx=0 tx=0).stop 2024-09-15T22:19:28.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 msgr2=0x7f7490199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7488009d50 tx=0x7f748800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 shutdown_connections 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f747c0a33b0 0x7f747c0a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7490102ef0 0x7f74901a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490102570 0x7f7490199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 --2- 172.21.15.77:0/123607379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7490100e20 0x7f7490199310 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 >> 172.21.15.77:0/123607379 conn(0x7f74900fc7d0 msgr2=0x7f74900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 shutdown_connections 2024-09-15T22:19:28.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:28.430+0000 7f74956f0640 1 -- 172.21.15.77:0/123607379 wait complete. 2024-09-15T22:19:28.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 522633 7.76 MB/sec execute 217 sec latency 37.413 ms 2024-09-15T22:19:29.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:28 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:19:29.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:28 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:28 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:19:29.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:28 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:28 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:19:29.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:28 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:28 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:19:29.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:28 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:28 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:19:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:28 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:28 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:19:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:28 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:29.120 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:19:29.401 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:29.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 524796 7.77 MB/sec execute 218 sec latency 33.592 ms 2024-09-15T22:19:29.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.945+0000 7f0372cb4640 1 -- 172.21.15.77:0/36873557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 msgr2=0x7f036c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.945+0000 7f0372cb4640 1 --2- 172.21.15.77:0/36873557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c101200 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0360009900 tx=0x7f036002a480 comp rx=0 tx=0).stop 2024-09-15T22:19:29.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 -- 172.21.15.77:0/36873557 shutdown_connections 2024-09-15T22:19:29.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- 172.21.15.77:0/36873557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102ef0 0x7f036c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- 172.21.15.77:0/36873557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102570 0x7f036c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- 172.21.15.77:0/36873557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c101200 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 -- 172.21.15.77:0/36873557 >> 172.21.15.77:0/36873557 conn(0x7f036c0fc7d0 msgr2=0x7f036c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 -- 172.21.15.77:0/36873557 shutdown_connections 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 -- 172.21.15.77:0/36873557 wait complete. 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 Processor -- start 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 -- start start 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.946+0000 7f0372cb4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 0x7f036c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0372cb4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f036c110f80 con 0x7f036c100e20 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0372cb4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c110e00 con 0x7f036c102570 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0372cb4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f036c111100 con 0x7f036c102ef0 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f036bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f036bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59088/0 (socket says 172.21.15.77:59088) 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0370a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f036bfff640 1 -- 172.21.15.77:0/872417978 learned_addr learned my addr 172.21.15.77:0/872417978 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 0x7f036c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 -- 172.21.15.77:0/872417978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 msgr2=0x7f036c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 -- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 msgr2=0x7f036c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 -- 172.21.15.77:0/872417978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c19d4d0 con 0x7f036c102ef0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f036bfff640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0371251640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 0x7f036c19cdf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f036400e940 tx=0x7f036400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.947+0000 7f0369ffb640 1 -- 172.21.15.77:0/872417978 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364004d20 con 0x7f036c102ef0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.948+0000 7f0369ffb640 1 -- 172.21.15.77:0/872417978 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0364005020 con 0x7f036c102ef0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.948+0000 7f0369ffb640 1 -- 172.21.15.77:0/872417978 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03640116b0 con 0x7f036c102ef0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.948+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c19d7c0 con 0x7f036c102ef0 2024-09-15T22:19:29.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.948+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1a50a0 con 0x7f036c102ef0 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.950+0000 7f0369ffb640 1 -- 172.21.15.77:0/872417978 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f03640054d0 con 0x7f036c102ef0 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 Processor -- start 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 -- start start 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f0369ffb640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 0x7f035c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472cc80 0x55da9473b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da946922e0 con 0x55da9472cc80 2024-09-15T22:19:29.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da94639ce0 con 0x55da9472c360 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f037508d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da9468e2a0 con 0x55da9473ba90 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f036bfff640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 0x7f035c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f0369ffb640 1 -- 172.21.15.77:0/872417978 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f03640d0140 con 0x7f036c102ef0 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f0370a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f0370a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59096/0 (socket says 172.21.15.77:59096) 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.951+0000 7f0370a50640 1 -- 172.21.15.77:0/2719931644 learned_addr learned my addr 172.21.15.77:0/2719931644 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f036bfff640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472cc80 0x55da9473b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 -- 172.21.15.77:0/2719931644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 msgr2=0x55da9472c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 -- 172.21.15.77:0/2719931644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472cc80 msgr2=0x55da9473b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472cc80 0x55da9473b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 -- 172.21.15.77:0/2719931644 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da9461e440 con 0x55da9473ba90 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f036bfff640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 0x7f035c0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0358009fd0 tx=0x7f0358013d50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0370a50640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0371251640 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473de80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03640cd500 tx=0x7f036409c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e89b4f0236569d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0356ffd640 1 -- 172.21.15.77:0/2719931644 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036401f070 con 0x55da9473ba90 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0356ffd640 1 -- 172.21.15.77:0/2719931644 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03640bc0d0 con 0x55da9473ba90 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.952+0000 7f0356ffd640 1 -- 172.21.15.77:0/2719931644 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03640cd910 con 0x55da9473ba90 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 -- 172.21.15.77:0/2719931644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 msgr2=0x55da9473de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473de80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03640cd500 tx=0x7f036409c140 comp rx=0 tx=0).stop 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 -- 172.21.15.77:0/2719931644 shutdown_connections 2024-09-15T22:19:29.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473de80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472cc80 0x55da9473b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 --2- 172.21.15.77:0/2719931644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9472c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 -- 172.21.15.77:0/2719931644 >> 172.21.15.77:0/2719931644 conn(0x55da94728a30 msgr2=0x55da94728e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 -- 172.21.15.77:0/2719931644 shutdown_connections 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.953+0000 7f037508d740 1 -- 172.21.15.77:0/2719931644 wait complete. 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 -- start start 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 Processor -- start 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da94738760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da9473e660 con 0x55da9473ba90 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da9464d050 con 0x55da9472c360 2024-09-15T22:19:29.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da9473e7e0 con 0x55da9472cc80 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f037508d740 10 client.0 _ll_register_callbacks cb 0x55da9468d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f0371251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f0370a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f036bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da94738760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f0370a50640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:59112/0 (socket says 172.21.15.77:59112) 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f0371251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34120/0 (socket says 172.21.15.77:34120) 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.954+0000 7f0370a50640 1 -- 172.21.15.77:0/2032100313 learned_addr learned my addr 172.21.15.77:0/2032100313 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 -- 172.21.15.77:0/2032100313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 msgr2=0x55da94738760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da94738760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 -- 172.21.15.77:0/2032100313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 msgr2=0x55da94738d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 -- 172.21.15.77:0/2032100313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da944033c0 con 0x55da9472c360 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0371251640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:29.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f0370a50640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0360005940 tx=0x7f03600043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712cb4ca9773285 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f034bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360035030 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03640bc0d0 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f0360005220 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0360005420 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f037508d740 10 client.26540 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.955+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da9474a130 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03640bc0d0 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0360004b50 con 0x55da9472c360 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 10 client.26540 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:29.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 20 client.26540 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da9474b5f0 0x55da9474da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 4 client.26540 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da9469f850 con 0x55da9474b5f0 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f036bfff640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da9474b5f0 0x55da9474da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f034bfff640 10 client.26540 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:29.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.956+0000 7f036bfff640 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da9474b5f0 0x55da9474da10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8cd9fba846ce16 server_cookie=4b913e922b022050 in_seq=0 out_seq=0 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f034bfff640 1 -- 172.21.15.77:0/2032100313 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da9469f850 con 0x55da9474b5f0 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f034bfff640 10 client.26540 handle_command_reply: tid=0 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 client.26540 shutdown 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 msgr2=0x55da9473b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0360005940 tx=0x7f03600043b0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 shutdown_connections 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da9474b5f0 0x55da9474da10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9473ba90 0x55da94738d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da94738760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 --2- 172.21.15.77:0/2032100313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472c360 0x55da9473b4b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f034bfff640 0 client.26540 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:29.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 >> 172.21.15.77:0/2032100313 conn(0x55da94728a30 msgr2=0x55da9472e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 shutdown_connections 2024-09-15T22:19:29.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.957+0000 7f037508d740 1 -- 172.21.15.77:0/2032100313 wait complete. 2024-09-15T22:19:29.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.958+0000 7f037508d740 20 client.26540 trim_cache size 0 max 16384 2024-09-15T22:19:29.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 Processor -- start 2024-09-15T22:19:29.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 -- start start 2024-09-15T22:19:29.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da947532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9473ba90 0x55da94753820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1acd60 con 0x55da9472c360 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1ae8a0 con 0x55da9473ba90 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f037508d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1ad430 con 0x55da9472cc80 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f036bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da947532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34128/0 (socket says 172.21.15.77:34128) 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 -- 172.21.15.77:0/791663719 learned_addr learned my addr 172.21.15.77:0/791663719 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0371251640 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9473ba90 0x55da94753820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 -- 172.21.15.77:0/791663719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 msgr2=0x55da947532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da947532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 -- 172.21.15.77:0/791663719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9473ba90 msgr2=0x55da94753820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9473ba90 0x55da94753820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.976+0000 7f0370a50640 1 -- 172.21.15.77:0/791663719 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da947390e0 con 0x55da9472c360 2024-09-15T22:19:29.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.977+0000 7f0370a50640 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f036002a9a0 tx=0x7f0360001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f928e391f2494fea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.977+0000 7f03567fc640 1 -- 172.21.15.77:0/791663719 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360035040 con 0x55da9472c360 2024-09-15T22:19:29.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.977+0000 7f03567fc640 1 -- 172.21.15.77:0/791663719 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0360004ba0 con 0x55da9472c360 2024-09-15T22:19:29.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.977+0000 7f03567fc640 1 -- 172.21.15.77:0/791663719 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360031db0 con 0x55da9472c360 2024-09-15T22:19:29.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- 172.21.15.77:0/791663719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 msgr2=0x55da94747900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f036002a9a0 tx=0x7f0360001e70 comp rx=0 tx=0).stop 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- 172.21.15.77:0/791663719 shutdown_connections 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9473ba90 0x55da94753820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9472cc80 0x55da947532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- 172.21.15.77:0/791663719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94747900 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- 172.21.15.77:0/791663719 >> 172.21.15.77:0/791663719 conn(0x55da94728a30 msgr2=0x55da94733d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- 172.21.15.77:0/791663719 shutdown_connections 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- 172.21.15.77:0/791663719 wait complete. 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 -- start start 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 Processor -- start 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 0x55da9473f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.978+0000 7f037508d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f037508d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1ae1d0 con 0x55da9472c360 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f037508d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1adb00 con 0x55da9472cc80 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f037508d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1aad60 con 0x55da9473ba90 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f037508d740 10 client.0 _ll_register_callbacks cb 0x7f0360009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34138/0 (socket says 172.21.15.77:34138) 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 -- 172.21.15.77:0/2048490245 learned_addr learned my addr 172.21.15.77:0/2048490245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f036bfff640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 0x55da9473f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0371251640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 -- 172.21.15.77:0/2048490245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 msgr2=0x55da9473f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 -- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 msgr2=0x55da9473f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 0x55da9473f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 -- 172.21.15.77:0/2048490245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da9473fdd0 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f036bfff640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 0x55da9473f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f0370a50640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f03600314e0 tx=0x7f0360031540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dc1aec87682e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f034bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.979+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036005e060 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0360004ba0 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f03600048c0 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0360065050 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 10 client.17382 fetch_fsmap learned FSMap version 34 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da947b6050 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0360004ba0 con 0x55da9472c360 2024-09-15T22:19:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0360059440 con 0x55da9472c360 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 10 client.17382 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 20 client.17382 populate_metadata read hostname 'smithi077' 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da947b6660 0x55da947b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 4 client.17382 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.980+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da94699d60 con 0x55da947b6660 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f036bfff640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da947b6660 0x55da947b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f034bfff640 10 client.17382 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f036bfff640 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da947b6660 0x55da947b8a50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c523000de541aa5 server_cookie=990eda1157e4661a in_seq=0 out_seq=0 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f034bfff640 1 -- 172.21.15.77:0/2048490245 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da94699d60 con 0x55da947b6660 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f034bfff640 10 client.17382 handle_command_reply: tid=0 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 client.17382 shutdown 2024-09-15T22:19:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 msgr2=0x55da94743f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f03600314e0 tx=0x7f0360031540 comp rx=0 tx=0).stop 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 shutdown_connections 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55da947b6660 0x55da947b8a50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da9473ba90 0x55da9473f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da9472cc80 0x55da9473f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 --2- 172.21.15.77:0/2048490245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da9472c360 0x55da94743f50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f034bfff640 0 client.17382 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.981+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 >> 172.21.15.77:0/2048490245 conn(0x55da94728a30 msgr2=0x55da9472de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 shutdown_connections 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f037508d740 1 -- 172.21.15.77:0/2048490245 wait complete. 2024-09-15T22:19:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f037508d740 20 client.17382 trim_cache size 0 max 16384 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "824a5fce-1e9b-4d64-9752-3347c0ba6d43": { 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "824a5fce-1e9b-4d64-9752-3347c0ba6d43", 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:19:29.989 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 msgr2=0x7f035c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 0x7f035c0a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0358009fd0 tx=0x7f0358013d50 comp rx=0 tx=0).stop 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 msgr2=0x7f036c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 0x7f036c19cdf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f036400e940 tx=0x7f036400ee00 comp rx=0 tx=0).stop 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 shutdown_connections 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f035c0a33b0 0x7f035c0a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036c102ef0 0x7f036c19cdf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102570 0x7f036c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.982+0000 7f0372cb4640 1 --2- 172.21.15.77:0/872417978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f036c100e20 0x7f036c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.983+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 >> 172.21.15.77:0/872417978 conn(0x7f036c0fc7d0 msgr2=0x7f036c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.983+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 shutdown_connections 2024-09-15T22:19:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:19:29.983+0000 7f0372cb4640 1 -- 172.21.15.77:0/872417978 wait complete. 2024-09-15T22:19:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:29 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:19:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:29 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:29 smithi077 ceph-mon[30980]: pgmap v217: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:19:30.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:29 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:19:30.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:29 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:30.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:29 smithi196 ceph-mon[35109]: pgmap v217: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:19:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:29 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:19:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:29 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:29 smithi117 ceph-mon[35021]: pgmap v217: 129 pgs: 129 active+clean; 354 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:19:30.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 526872 7.79 MB/sec execute 219 sec latency 19.831 ms 2024-09-15T22:19:30.814 INFO:tasks.cephfs.filesystem:scrub status for tag:824a5fce-1e9b-4d64-9752-3347c0ba6d43 - {'path': '/', 'tag': '824a5fce-1e9b-4d64-9752-3347c0ba6d43', 'options': 'recursive,force'} 2024-09-15T22:19:31.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 529601 7.83 MB/sec execute 220 sec latency 15.204 ms 2024-09-15T22:19:32.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:31 smithi077 ceph-mon[30980]: pgmap v218: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:19:32.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:31 smithi196 ceph-mon[35109]: pgmap v218: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:19:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:31 smithi117 ceph-mon[35021]: pgmap v218: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-09-15T22:19:32.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 531795 7.83 MB/sec execute 221 sec latency 23.924 ms 2024-09-15T22:19:33.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 533867 7.85 MB/sec execute 222 sec latency 11.728 ms 2024-09-15T22:19:34.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:33 smithi077 ceph-mon[30980]: pgmap v219: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:34.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:33 smithi196 ceph-mon[35109]: pgmap v219: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:34.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:33 smithi117 ceph-mon[35021]: pgmap v219: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:34.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 536654 7.90 MB/sec execute 223 sec latency 12.978 ms 2024-09-15T22:19:35.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 538691 7.89 MB/sec execute 224 sec latency 14.354 ms 2024-09-15T22:19:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:35 smithi077 ceph-mon[30980]: pgmap v220: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:35 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:19:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:35 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:19:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:35 smithi196 ceph-mon[35109]: pgmap v220: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:35 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:19:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:35 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:19:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:35 smithi117 ceph-mon[35021]: pgmap v220: 129 pgs: 129 active+clean; 364 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-15T22:19:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:35 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:19:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:35 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:19:36.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 540903 7.92 MB/sec execute 225 sec latency 11.907 ms 2024-09-15T22:19:37.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 543681 7.96 MB/sec execute 226 sec latency 17.303 ms 2024-09-15T22:19:38.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:37 smithi077 ceph-mon[30980]: pgmap v221: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-15T22:19:38.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:37 smithi196 ceph-mon[35109]: pgmap v221: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-15T22:19:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:37 smithi117 ceph-mon[35021]: pgmap v221: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-15T22:19:38.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 545720 7.95 MB/sec execute 227 sec latency 28.373 ms 2024-09-15T22:19:39.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 547809 7.98 MB/sec execute 228 sec latency 15.239 ms 2024-09-15T22:19:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:39 smithi077 ceph-mon[30980]: pgmap v222: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-15T22:19:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:39 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:40.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:39 smithi196 ceph-mon[35109]: pgmap v222: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-15T22:19:40.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:39 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:40.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:39 smithi117 ceph-mon[35021]: pgmap v222: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-09-15T22:19:40.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:39 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:40.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 550574 8.02 MB/sec execute 229 sec latency 17.044 ms 2024-09-15T22:19:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:41.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:40 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:19:41.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:41.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:41.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:40 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:19:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:41.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:40 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:19:41.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 552317 8.01 MB/sec execute 230 sec latency 17.457 ms 2024-09-15T22:19:42.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 554780 8.04 MB/sec execute 231 sec latency 15.063 ms 2024-09-15T22:19:42.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:42 smithi077 ceph-mon[30980]: pgmap v223: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-15T22:19:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:42 smithi196 ceph-mon[35109]: pgmap v223: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-15T22:19:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:42 smithi117 ceph-mon[35021]: pgmap v223: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 236 op/s 2024-09-15T22:19:43.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 556658 8.04 MB/sec execute 232 sec latency 51.999 ms 2024-09-15T22:19:44.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 558744 8.06 MB/sec execute 233 sec latency 31.113 ms 2024-09-15T22:19:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:44 smithi077 ceph-mon[30980]: pgmap v224: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:44 smithi196 ceph-mon[35109]: pgmap v224: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:44 smithi117 ceph-mon[35021]: pgmap v224: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:45.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 560632 8.07 MB/sec execute 234 sec latency 15.345 ms 2024-09-15T22:19:46.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 562689 8.09 MB/sec execute 235 sec latency 18.051 ms 2024-09-15T22:19:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:46 smithi077 ceph-mon[30980]: pgmap v225: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:46 smithi196 ceph-mon[35109]: pgmap v225: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:46 smithi117 ceph-mon[35021]: pgmap v225: 129 pgs: 129 active+clean; 385 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-15T22:19:47.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 565198 8.11 MB/sec execute 236 sec latency 14.796 ms 2024-09-15T22:19:48.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 566761 8.10 MB/sec execute 237 sec latency 66.296 ms 2024-09-15T22:19:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:48 smithi077 ceph-mon[30980]: pgmap v226: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-09-15T22:19:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:48 smithi196 ceph-mon[35109]: pgmap v226: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-09-15T22:19:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:48 smithi117 ceph-mon[35021]: pgmap v226: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.7 MiB/s wr, 261 op/s 2024-09-15T22:19:49.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 569086 8.13 MB/sec execute 238 sec latency 17.101 ms 2024-09-15T22:19:50.476 INFO:tasks.workunit.client.0.smithi077.stdout: 1 571893 8.16 MB/sec execute 239 sec latency 13.811 ms 2024-09-15T22:19:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:50 smithi077 ceph-mon[30980]: pgmap v227: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:19:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:50 smithi196 ceph-mon[35109]: pgmap v227: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:19:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:50 smithi117 ceph-mon[35021]: pgmap v227: 129 pgs: 129 active+clean; 394 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:19:51.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 573664 8.16 MB/sec execute 240 sec latency 15.101 ms 2024-09-15T22:19:52.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 576098 8.18 MB/sec execute 241 sec latency 14.495 ms 2024-09-15T22:19:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:52 smithi077 ceph-mon[30980]: pgmap v228: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 288 op/s 2024-09-15T22:19:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:52 smithi196 ceph-mon[35109]: pgmap v228: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 288 op/s 2024-09-15T22:19:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:52 smithi117 ceph-mon[35021]: pgmap v228: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.4 MiB/s wr, 288 op/s 2024-09-15T22:19:53.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 578010 8.18 MB/sec execute 242 sec latency 125.789 ms 2024-09-15T22:19:54.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 580113 8.20 MB/sec execute 243 sec latency 41.820 ms 2024-09-15T22:19:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:54 smithi077 ceph-mon[30980]: pgmap v229: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:54 smithi196 ceph-mon[35109]: pgmap v229: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:54 smithi117 ceph-mon[35021]: pgmap v229: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:55.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 582832 8.24 MB/sec execute 244 sec latency 13.844 ms 2024-09-15T22:19:56.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 584877 8.23 MB/sec execute 245 sec latency 39.141 ms 2024-09-15T22:19:56.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:56 smithi077 ceph-mon[30980]: pgmap v230: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:56 smithi196 ceph-mon[35109]: pgmap v230: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:56 smithi117 ceph-mon[35021]: pgmap v230: 129 pgs: 129 active+clean; 403 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:19:57.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 586955 8.25 MB/sec execute 246 sec latency 16.989 ms 2024-09-15T22:19:58.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 589043 8.27 MB/sec execute 247 sec latency 124.445 ms 2024-09-15T22:19:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:19:58 smithi077 ceph-mon[30980]: pgmap v231: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 313 op/s 2024-09-15T22:19:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:19:58 smithi196 ceph-mon[35109]: pgmap v231: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 313 op/s 2024-09-15T22:19:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:19:58 smithi117 ceph-mon[35021]: pgmap v231: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 313 op/s 2024-09-15T22:19:59.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 591055 8.28 MB/sec execute 248 sec latency 13.490 ms 2024-09-15T22:20:00.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 593572 8.30 MB/sec execute 249 sec latency 14.171 ms 2024-09-15T22:20:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:00 smithi077 ceph-mon[30980]: pgmap v232: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:20:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:00 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:20:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:00 smithi196 ceph-mon[35109]: pgmap v232: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:20:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:00 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:20:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:00 smithi117 ceph-mon[35021]: pgmap v232: 129 pgs: 129 active+clean; 412 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-09-15T22:20:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:00 smithi117 ceph-mon[35021]: overall HEALTH_OK 2024-09-15T22:20:00.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:20:01.118 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:01.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 594195 8.28 MB/sec execute 250 sec latency 114.872 ms 2024-09-15T22:20:02.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:02 smithi077 ceph-mon[30980]: pgmap v233: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-15T22:20:02.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 596057 8.28 MB/sec execute 251 sec latency 117.668 ms 2024-09-15T22:20:02.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 -- 172.21.15.77:0/3717095311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 msgr2=0x7f322410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 --2- 172.21.15.77:0/3717095311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322410eac0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f321c009ca0 tx=0x7f321c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 -- 172.21.15.77:0/3717095311 shutdown_connections 2024-09-15T22:20:02.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 --2- 172.21.15.77:0/3717095311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322410eac0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 --2- 172.21.15.77:0/3717095311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 --2- 172.21.15.77:0/3717095311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 0x7f32241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 -- 172.21.15.77:0/3717095311 >> 172.21.15.77:0/3717095311 conn(0x7f32240fd8d0 msgr2=0x7f32240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 -- 172.21.15.77:0/3717095311 shutdown_connections 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.468+0000 7f322c14d640 1 -- 172.21.15.77:0/3717095311 wait complete. 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 Processor -- start 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 -- start start 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 0x7f32241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3224111ff0 con 0x7f3224106f40 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3224111e70 con 0x7f3224101e10 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322c14d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3224112170 con 0x7f32241027c0 2024-09-15T22:20:02.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f322a6ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f32296e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f3229ee9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 0x7f32241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f32296e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43176/0 (socket says 172.21.15.77:43176) 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.469+0000 7f32296e8640 1 -- 172.21.15.77:0/3835714385 learned_addr learned my addr 172.21.15.77:0/3835714385 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 -- 172.21.15.77:0/3835714385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 msgr2=0x7f3224198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 -- 172.21.15.77:0/3835714385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 msgr2=0x7f32241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 0x7f32241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 -- 172.21.15.77:0/3835714385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322419d3a0 con 0x7f3224106f40 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322a6ea640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322419ccc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e30 tx=0x7f321c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f321affd640 1 -- 172.21.15.77:0/3835714385 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c031650 con 0x7f3224106f40 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f321affd640 1 -- 172.21.15.77:0/3835714385 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c0317f0 con 0x7f3224106f40 2024-09-15T22:20:02.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322419d630 con 0x7f3224106f40 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f321affd640 1 -- 172.21.15.77:0/3835714385 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c02fa30 con 0x7f3224106f40 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.470+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32241a4f70 con 0x7f3224106f40 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.472+0000 7f321affd640 1 -- 172.21.15.77:0/3835714385 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f321c02fbd0 con 0x7f3224106f40 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.473+0000 7f321affd640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 0x7f32100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.473+0000 7f3229ee9640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 0x7f32100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.473+0000 7f321affd640 1 -- 172.21.15.77:0/3835714385 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f321c0f1220 con 0x7f3224106f40 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.473+0000 7f322e526740 1 Processor -- start 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 -- start start 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97540720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97551e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f974a62e0 con 0x564f97540c60 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564f9744dce0 con 0x564f9754fa90 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322e526740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f974a22a0 con 0x564f97540340 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f3229ee9640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 0x7f32100a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f322000bbe0 tx=0x7f322000b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55816/0 (socket says 172.21.15.77:55816) 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f3229ee9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97540720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 -- 172.21.15.77:0/343784550 learned_addr learned my addr 172.21.15.77:0/343784550 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322a6ea640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97551e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 -- 172.21.15.77:0/343784550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 msgr2=0x564f97540720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97540720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 -- 172.21.15.77:0/343784550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 msgr2=0x564f97551e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97551e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f32296e8640 1 -- 172.21.15.77:0/343784550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f97432440 con 0x564f97540c60 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f3229ee9640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.474+0000 7f322a6ea640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97551e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.475+0000 7f32296e8640 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3214009d00 tx=0x7f3214011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0663c99cea4c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.475+0000 7f320ffff640 1 -- 172.21.15.77:0/343784550 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321401e070 con 0x564f97540c60 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.475+0000 7f320ffff640 1 -- 172.21.15.77:0/343784550 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214019910 con 0x564f97540c60 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.475+0000 7f320ffff640 1 -- 172.21.15.77:0/343784550 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321400fa70 con 0x564f97540c60 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- 172.21.15.77:0/343784550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 msgr2=0x564f9754f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3214009d00 tx=0x7f3214011810 comp rx=0 tx=0).stop 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- 172.21.15.77:0/343784550 shutdown_connections 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97551e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f9754f550 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 --2- 172.21.15.77:0/343784550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- 172.21.15.77:0/343784550 >> 172.21.15.77:0/343784550 conn(0x564f9753ca30 msgr2=0x564f9753ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- 172.21.15.77:0/343784550 shutdown_connections 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- 172.21.15.77:0/343784550 wait complete. 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 -- start start 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.476+0000 7f322e526740 1 Processor -- start 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540340 0x564f9754f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540c60 0x564f9754c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564f97552660 con 0x564f9754fa90 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564f97461050 con 0x564f97540340 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564f975527e0 con 0x564f97540c60 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55824/0 (socket says 172.21.15.77:55824) 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 -- 172.21.15.77:0/289464857 learned_addr learned my addr 172.21.15.77:0/289464857 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:02.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f3229ee9640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540340 0x564f9754f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f32296e8640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540c60 0x564f9754c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 10 client.0 _ll_register_callbacks cb 0x564f974a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 -- 172.21.15.77:0/289464857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540c60 msgr2=0x564f9754c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540c60 0x564f9754c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 -- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540340 msgr2=0x564f9754f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540340 0x564f9754f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 -- 172.21.15.77:0/289464857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f972173c0 con 0x564f9754fa90 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322a6ea640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f321c0ee750 tx=0x7f321c0bccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adafce392b643ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f320d7fa640 10 client.17394 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3214019910 con 0x564f9754fa90 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f320d7fa640 1 -- 172.21.15.77:0/289464857 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0d6060 con 0x564f9754fa90 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f320d7fa640 1 -- 172.21.15.77:0/289464857 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f321c0ee860 con 0x564f9754fa90 2024-09-15T22:20:02.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.477+0000 7f320d7fa640 1 -- 172.21.15.77:0/289464857 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f321c0bd280 con 0x564f9754fa90 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 10 client.17394 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=34}) -- 0x564f97552660 con 0x564f9754fa90 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f320d7fa640 1 -- 172.21.15.77:0/289464857 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f321c0b98f0 con 0x564f9754fa90 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 10 client.17394 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 20 client.17394 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f9755f320 0x564f975617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 4 client.17394 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f974b3850 con 0x564f9755f320 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f3229ee9640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f9755f320 0x564f975617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f320d7fa640 10 client.17394 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.478+0000 7f3229ee9640 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f9755f320 0x564f975617e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7e829d436a43e server_cookie=b0c6bd434c4f0288 in_seq=0 out_seq=0 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f320d7fa640 1 -- 172.21.15.77:0/289464857 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f974b3850 con 0x564f9755f320 2024-09-15T22:20:02.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f320d7fa640 10 client.17394 handle_command_reply: tid=0 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 client.17394 shutdown 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 msgr2=0x564f9754cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f321c0ee750 tx=0x7f321c0bccb0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 shutdown_connections 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f9755f320 0x564f975617e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f9754fa90 0x564f9754cc90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540c60 0x564f9754c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 --2- 172.21.15.77:0/289464857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540340 0x564f9754f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 >> 172.21.15.77:0/289464857 conn(0x564f9753ca30 msgr2=0x564f97542790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f320d7fa640 0 client.17394 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 shutdown_connections 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 1 -- 172.21.15.77:0/289464857 wait complete. 2024-09-15T22:20:02.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.479+0000 7f322e526740 20 client.17394 trim_cache size 0 max 16384 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 Processor -- start 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 -- start start 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540c60 0x564f975670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f9754fa90 0x564f97567630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32241acc80 con 0x564f97540340 2024-09-15T22:20:02.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ae7c0 con 0x564f97540c60 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322e526740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ad350 con 0x564f9754fa90 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55830/0 (socket says 172.21.15.77:55830) 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f32296e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540c60 0x564f975670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 -- 172.21.15.77:0/2204206370 learned_addr learned my addr 172.21.15.77:0/2204206370 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f322a6ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f9754fa90 0x564f97567630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 -- 172.21.15.77:0/2204206370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f9754fa90 msgr2=0x564f97567630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f9754fa90 0x564f97567630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 -- 172.21.15.77:0/2204206370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540c60 msgr2=0x564f975670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540c60 0x564f975670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.499+0000 7f3229ee9640 1 -- 172.21.15.77:0/2204206370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9754ac40 con 0x564f97540340 2024-09-15T22:20:02.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.500+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3220010ce0 tx=0x7f3220013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba280af44d4763c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.500+0000 7f320f7fe640 1 -- 172.21.15.77:0/2204206370 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220005020 con 0x564f97540340 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.500+0000 7f320f7fe640 1 -- 172.21.15.77:0/2204206370 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32200047a0 con 0x564f97540340 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.500+0000 7f320f7fe640 1 -- 172.21.15.77:0/2204206370 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220021960 con 0x564f97540340 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- 172.21.15.77:0/2204206370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 msgr2=0x564f9755b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3220010ce0 tx=0x7f3220013000 comp rx=0 tx=0).stop 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- 172.21.15.77:0/2204206370 shutdown_connections 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f9754fa90 0x564f97567630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f97540c60 0x564f975670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 --2- 172.21.15.77:0/2204206370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540340 0x564f9755b900 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- 172.21.15.77:0/2204206370 >> 172.21.15.77:0/2204206370 conn(0x564f9753ca30 msgr2=0x564f9754afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- 172.21.15.77:0/2204206370 shutdown_connections 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- 172.21.15.77:0/2204206370 wait complete. 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 -- start start 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.501+0000 7f322e526740 1 Processor -- start 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97546c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97557fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ae0f0 con 0x564f97540c60 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32241ada20 con 0x564f9754fa90 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32241aac80 con 0x564f97540340 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322e526740 10 client.0 _ll_register_callbacks cb 0x7f321c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:02.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55838/0 (socket says 172.21.15.77:55838) 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 -- 172.21.15.77:0/2408142331 learned_addr learned my addr 172.21.15.77:0/2408142331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97546c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322a6ea640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97557fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 -- 172.21.15.77:0/2408142331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 msgr2=0x564f97546c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97546c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 -- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 msgr2=0x564f97557fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97557fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 -- 172.21.15.77:0/2408142331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f97558620 con 0x564f97540c60 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f322a6ea640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97557fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f32296e8640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3214009c50 tx=0x7f3214009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5989863b41a526f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.502+0000 7f320d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:02.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321401e040 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32200047a0 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f3214005040 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3214005240 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 10 client.17400 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f975ce8b0 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32200047a0 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3214004680 con 0x564f97540c60 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 10 client.17400 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 20 client.17400 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f975ced90 0x564f975d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 4 client.17400 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.503+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f974add60 con 0x564f975ced90 2024-09-15T22:20:02.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f975ced90 0x564f975d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f3229ee9640 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f975ced90 0x564f975d1180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e46016652f5a4e6 server_cookie=ed17f8da4d4983ff in_seq=0 out_seq=0 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f320d7fa640 10 client.17400 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f320d7fa640 1 -- 172.21.15.77:0/2408142331 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f974add60 con 0x564f975ced90 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f320d7fa640 10 client.17400 handle_command_reply: tid=0 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 client.17400 shutdown 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 msgr2=0x564f97557a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3214009c50 tx=0x7f3214009a80 comp rx=0 tx=0).stop 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 shutdown_connections 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564f975ced90 0x564f975d1180 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f9754fa90 0x564f97557fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f320d7fa640 0 client.17400 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f97540c60 0x564f97557a70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 --2- 172.21.15.77:0/2408142331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564f97540340 0x564f97546c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.504+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 >> 172.21.15.77:0/2408142331 conn(0x564f9753ca30 msgr2=0x564f9754df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 shutdown_connections 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322e526740 1 -- 172.21.15.77:0/2408142331 wait complete. 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322e526740 20 client.17400 trim_cache size 0 max 16384 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:20:02.512 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 msgr2=0x7f32100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 0x7f32100a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f322000bbe0 tx=0x7f322000b390 comp rx=0 tx=0).stop 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 msgr2=0x7f322419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322419ccc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e30 tx=0x7f321c003f10 comp rx=0 tx=0).stop 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 shutdown_connections 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f32100a33b0 0x7f32100a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3224106f40 0x7f322419ccc0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32241027c0 0x7f3224198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 --2- 172.21.15.77:0/3835714385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3224101e10 0x7f32241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:02.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.505+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 >> 172.21.15.77:0/3835714385 conn(0x7f32240fd8d0 msgr2=0x7f32240fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:02.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.506+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 shutdown_connections 2024-09-15T22:20:02.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:02.506+0000 7f322c14d640 1 -- 172.21.15.77:0/3835714385 wait complete. 2024-09-15T22:20:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:02 smithi196 ceph-mon[35109]: pgmap v233: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-15T22:20:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:02 smithi117 ceph-mon[35021]: pgmap v233: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 317 op/s 2024-09-15T22:20:03.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 597661 8.28 MB/sec execute 252 sec latency 125.491 ms 2024-09-15T22:20:03.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:20:03.634 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:03.907 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:04.452 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:04 smithi077 ceph-mon[30980]: pgmap v234: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 -- 172.21.15.77:0/4139540184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 msgr2=0x7fa84410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 --2- 172.21.15.77:0/4139540184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84410ec50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009cd0 tx=0x7fa83c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 -- 172.21.15.77:0/4139540184 shutdown_connections 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 --2- 172.21.15.77:0/4139540184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84410ec50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 --2- 172.21.15.77:0/4139540184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 0x7fa844107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 --2- 172.21.15.77:0/4139540184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 0x7fa844103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 -- 172.21.15.77:0/4139540184 >> 172.21.15.77:0/4139540184 conn(0x7fa8440fc7d0 msgr2=0x7fa8440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 -- 172.21.15.77:0/4139540184 shutdown_connections 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.449+0000 7fa848953640 1 -- 172.21.15.77:0/4139540184 wait complete. 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 Processor -- start 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 -- start start 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 0x7fa844198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 0x7fa844198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441120b0 con 0x7fa8441082e0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111f30 con 0x7fa844102db0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa848953640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112230 con 0x7fa844103760 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa842d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa842d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55852/0 (socket says 172.21.15.77:55852) 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa842d9d640 1 -- 172.21.15.77:0/437638191 learned_addr learned my addr 172.21.15.77:0/437638191 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa84259c640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 0x7fa844198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.450+0000 7fa83bfff640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 0x7fa844198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 -- 172.21.15.77:0/437638191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 msgr2=0x7fa844198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 0x7fa844198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 -- 172.21.15.77:0/437638191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 msgr2=0x7fa844198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 0x7fa844198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 -- 172.21.15.77:0/437638191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419d500 con 0x7fa8441082e0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa842d9d640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa83c005cc0 tx=0x7fa83c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c003e00 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419d790 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa83c004520 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a50d0 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.451+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c030d60 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.452+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844103190 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.453+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa83c004040 con 0x7fa8441082e0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.454+0000 7fa83b7fe640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 0x7fa8300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.455+0000 7fa84259c640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 0x7fa8300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.455+0000 7fa84259c640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 0x7fa8300a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa834009bd0 tx=0x7fa834005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.455+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fa83c02e030 con 0x7fa8441082e0 2024-09-15T22:20:04.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.459+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa83c04b060 con 0x7fa8441082e0 2024-09-15T22:20:04.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 600349 8.32 MB/sec execute 253 sec latency 19.409 ms 2024-09-15T22:20:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:04 smithi196 ceph-mon[35109]: pgmap v234: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:04 smithi117 ceph-mon[35021]: pgmap v234: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:04.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.685+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8441a5300 con 0x7fa8441082e0 2024-09-15T22:20:04.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.686+0000 7fa83b7fe640 1 -- 172.21.15.77:0/437638191 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fa83c0bdc90 con 0x7fa8441082e0 2024-09-15T22:20:04.689 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:20:04.689 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:20:04.689 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 msgr2=0x7fa8300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 0x7fa8300a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa834009bd0 tx=0x7fa834005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 msgr2=0x7fa84419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa83c005cc0 tx=0x7fa83c009870 comp rx=0 tx=0).stop 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 shutdown_connections 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa8300a33b0 0x7fa8300a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8441082e0 0x7fa84419ce20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa844103760 0x7fa844198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 --2- 172.21.15.77:0/437638191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844102db0 0x7fa844198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:04.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 >> 172.21.15.77:0/437638191 conn(0x7fa8440fc7d0 msgr2=0x7fa844105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:04.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 shutdown_connections 2024-09-15T22:20:04.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:04.688+0000 7fa848953640 1 -- 172.21.15.77:0/437638191 wait complete. 2024-09-15T22:20:05.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 601923 8.31 MB/sec execute 254 sec latency 55.779 ms 2024-09-15T22:20:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:05 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/437638191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:05 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/437638191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:05 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/437638191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:05.765 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.7 2024-09-15T22:20:05.765 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:06.145 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:20:06.215 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:06.402 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:06 smithi077 ceph-mon[30980]: pgmap v235: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:06.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 604396 8.33 MB/sec execute 255 sec latency 12.977 ms 2024-09-15T22:20:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:06 smithi196 ceph-mon[35109]: pgmap v235: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:06 smithi117 ceph-mon[35021]: pgmap v235: 129 pgs: 129 active+clean; 421 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-09-15T22:20:06.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 -- 172.21.15.77:0/3488097033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102570 msgr2=0x7fca241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:06.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 --2- 172.21.15.77:0/3488097033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102570 0x7fca241029b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fca18009900 tx=0x7fca1802a480 comp rx=0 tx=0).stop 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 -- 172.21.15.77:0/3488097033 shutdown_connections 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 --2- 172.21.15.77:0/3488097033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24102ef0 0x7fca2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 --2- 172.21.15.77:0/3488097033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102570 0x7fca241029b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 --2- 172.21.15.77:0/3488097033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24100e20 0x7fca24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 -- 172.21.15.77:0/3488097033 >> 172.21.15.77:0/3488097033 conn(0x7fca240fc7d0 msgr2=0x7fca240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 -- 172.21.15.77:0/3488097033 shutdown_connections 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 -- 172.21.15.77:0/3488097033 wait complete. 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.796+0000 7fca2a79a640 1 Processor -- start 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 -- start start 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 0x7fca241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24102570 0x7fca241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:06.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca24110f80 con 0x7fca24102ef0 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca24110e00 con 0x7fca24100e20 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca2a79a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca24111100 con 0x7fca24102570 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55870/0 (socket says 172.21.15.77:55870) 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 -- 172.21.15.77:0/1229903094 learned_addr learned my addr 172.21.15.77:0/1229903094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca237fe640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24102570 0x7fca241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca23fff640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 0x7fca241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 -- 172.21.15.77:0/1229903094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24102570 msgr2=0x7fca241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24102570 0x7fca241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 -- 172.21.15.77:0/1229903094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 msgr2=0x7fca241984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:06.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.797+0000 7fca28d37640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 0x7fca241984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca28d37640 1 -- 172.21.15.77:0/1229903094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2419d460 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca23fff640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 0x7fca241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca28d37640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eb10 tx=0x7fca1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004b20 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c004cc0 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c010760 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.798+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2419d750 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.799+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241a5030 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.800+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca24101200 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.801+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fca1c005020 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.802+0000 7fca217fa640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 0x7fca0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.802+0000 7fca23fff640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 0x7fca0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.802+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fca1c0d0690 con 0x7fca24102ef0 2024-09-15T22:20:06.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.802+0000 7fca23fff640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 0x7fca0c0a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca14009b50 tx=0x7fca14005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:06.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:06.807+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca1c09cce0 con 0x7fca24102ef0 2024-09-15T22:20:06.863 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:07.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.041+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca2410a0a0 con 0x7fca24102ef0 2024-09-15T22:20:07.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.042+0000 7fca217fa640 1 -- 172.21.15.77:0/1229903094 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fca1c09c5e0 con 0x7fca24102ef0 2024-09-15T22:20:07.044 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:20:07.044 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:20:07.045 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:20:07.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 msgr2=0x7fca0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 0x7fca0c0a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca14009b50 tx=0x7fca14005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 msgr2=0x7fca2419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eb10 tx=0x7fca1c00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 shutdown_connections 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca0c0a3430 0x7fca0c0a58f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca24102ef0 0x7fca2419cd80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca24102570 0x7fca241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 --2- 172.21.15.77:0/1229903094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca24100e20 0x7fca241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 >> 172.21.15.77:0/1229903094 conn(0x7fca240fc7d0 msgr2=0x7fca2406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 shutdown_connections 2024-09-15T22:20:07.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:07.044+0000 7fca2a79a640 1 -- 172.21.15.77:0/1229903094 wait complete. 2024-09-15T22:20:07.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 606723 8.35 MB/sec execute 256 sec latency 42.475 ms 2024-09-15T22:20:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:07 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1229903094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:07 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1229903094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:07 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1229903094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:08.038 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0' 2024-09-15T22:20:08.038 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0 2024-09-15T22:20:08.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 -- 172.21.15.77:0/2429125445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410ae80 msgr2=0x7f2ae4113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/2429125445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410ae80 0x7f2ae4113370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009900 tx=0x7f2ad402a410 comp rx=0 tx=0).stop 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 -- 172.21.15.77:0/2429125445 shutdown_connections 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/2429125445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae41139e0 0x7f2ae4115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/2429125445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410ae80 0x7f2ae4113370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/2429125445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae410a560 0x7f2ae410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 -- 172.21.15.77:0/2429125445 >> 172.21.15.77:0/2429125445 conn(0x7f2ae406e2d0 msgr2=0x7f2ae406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 -- 172.21.15.77:0/2429125445 shutdown_connections 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.353+0000 7f2aebe2d640 1 -- 172.21.15.77:0/2429125445 wait complete. 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 Processor -- start 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 -- start start 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410a560 0x7f2ae41a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 0x7f2ae41abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41196f0 con 0x7f2ae410ae80 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4119570 con 0x7f2ae410a560 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aebe2d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4119870 con 0x7f2ae41139e0 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55888/0 (socket says 172.21.15.77:55888) 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2ae93c8640 1 -- 172.21.15.77:0/3424638854 learned_addr learned my addr 172.21.15.77:0/3424638854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 0x7f2ae41abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.354+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410a560 0x7f2ae41a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 -- 172.21.15.77:0/3424638854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 msgr2=0x7f2ae41abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 0x7f2ae41abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 -- 172.21.15.77:0/3424638854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410a560 msgr2=0x7f2ae41a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410a560 0x7f2ae41a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 -- 172.21.15.77:0/3424638854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41ac2a0 con 0x7f2ae410ae80 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 0x7f2ae41abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2ad402a930 tx=0x7f2ad4003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2adaffd640 1 -- 172.21.15.77:0/3424638854 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad403f070 con 0x7f2ae410ae80 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41ac530 con 0x7f2ae410ae80 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2adaffd640 1 -- 172.21.15.77:0/3424638854 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad4031c90 con 0x7f2ae410ae80 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2adaffd640 1 -- 172.21.15.77:0/3424638854 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035030 con 0x7f2ae410ae80 2024-09-15T22:20:08.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.355+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41b3e70 con 0x7f2ae410ae80 2024-09-15T22:20:08.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.358+0000 7f2adaffd640 1 -- 172.21.15.77:0/3424638854 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2ad4042070 con 0x7f2ae410ae80 2024-09-15T22:20:08.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.358+0000 7f2aee206740 1 Processor -- start 2024-09-15T22:20:08.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 -- start start 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e3348aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e33497760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559e333f02f0 con 0x559e3348bb40 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e33397cf0 con 0x559e3348a5d0 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aee206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e333ec2b0 con 0x559e33495370 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aea3ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aea3ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55896/0 (socket says 172.21.15.77:55896) 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2aea3ca640 1 -- 172.21.15.77:0/816147972 learned_addr learned my addr 172.21.15.77:0/816147972 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:08.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e3348aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.359+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e33497760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/816147972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 msgr2=0x559e33497760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2adaffd640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 0x7f2ad00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e33497760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/816147972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 msgr2=0x559e3348a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/816147972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3337c3a0 con 0x559e3348a5d0 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e33497760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2adaffd640 1 -- 172.21.15.77:0/3424638854 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2ad40f0710 con 0x7f2ae410ae80 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 0x7f2ad00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e3348aa10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009c20 tx=0x7f2ae0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c63766c1a6c966 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2acffff640 1 -- 172.21.15.77:0/816147972 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0004050 con 0x559e3348a5d0 2024-09-15T22:20:08.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2acffff640 1 -- 172.21.15.77:0/816147972 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae0005500 con 0x559e3348a5d0 2024-09-15T22:20:08.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.360+0000 7f2acffff640 1 -- 172.21.15.77:0/816147972 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0004dd0 con 0x559e3348a5d0 2024-09-15T22:20:08.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 -- 172.21.15.77:0/816147972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 msgr2=0x559e3348aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 0x7f2ad00a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009fd0 tx=0x7f2adc004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e3348aa10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009c20 tx=0x7f2ae0011810 comp rx=0 tx=0).stop 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 -- 172.21.15.77:0/816147972 shutdown_connections 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e33497760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e3348aa10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 --2- 172.21.15.77:0/816147972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e3348a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 -- 172.21.15.77:0/816147972 >> 172.21.15.77:0/816147972 conn(0x559e33453310 msgr2=0x559e33486c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.361+0000 7f2aee206740 1 -- 172.21.15.77:0/816147972 shutdown_connections 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.362+0000 7f2aee206740 1 -- 172.21.15.77:0/816147972 wait complete. 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.362+0000 7f2aee206740 1 -- start start 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.362+0000 7f2aee206740 1 Processor -- start 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e33491fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 0x559e33492560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559e33498070 con 0x559e3348bb40 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e333ab060 con 0x559e33495370 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559e334981f0 con 0x559e3348a5d0 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43276/0 (socket says 172.21.15.77:43276) 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/3835040167 learned_addr learned my addr 172.21.15.77:0/3835040167 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 0x559e33492560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e33491fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2aee206740 10 client.0 _ll_register_callbacks cb 0x559e330fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/3835040167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 msgr2=0x559e33492560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 0x559e33492560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 msgr2=0x559e33491fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e33491fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.363+0000 7f2ae9bc9640 1 -- 172.21.15.77:0/3835040167 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e331613c0 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 0x559e33492560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005df0 tx=0x7f2ae00056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e31c60b92d4105 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0029070 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae0005500 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2ae0004d50 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ae0035050 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2aee206740 10 client.26503 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e334a3aa0 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae0005500 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.364+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ae0003ca0 con 0x559e3348a5d0 2024-09-15T22:20:08.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 10 client.26503 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 10 client.26503 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 20 client.26503 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e334a5170 0x559e334a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 4 client.26503 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e333fd860 con 0x559e334a5170 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e334a5170 0x559e334a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2acd7fa640 10 client.26503 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.365+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e334a5170 0x559e334a7560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1cd52afd6a9e8501 server_cookie=d6bdce8fb394d263 in_seq=0 out_seq=0 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2acd7fa640 1 -- 172.21.15.77:0/3835040167 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e333fd860 con 0x559e334a5170 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2acd7fa640 10 client.26503 handle_command_reply: tid=0 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 client.26503 shutdown 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 msgr2=0x559e33494d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005df0 tx=0x7f2ae00056b0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 shutdown_connections 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e334a5170 0x559e334a7560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e33495370 0x559e33492560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348bb40 0x559e33491fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 --2- 172.21.15.77:0/3835040167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348a5d0 0x559e33494d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2acd7fa640 0 client.26503 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.366+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 >> 172.21.15.77:0/3835040167 conn(0x559e33453310 msgr2=0x559e3348c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.367+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 shutdown_connections 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.367+0000 7f2aee206740 1 -- 172.21.15.77:0/3835040167 wait complete. 2024-09-15T22:20:08.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.367+0000 7f2aee206740 20 client.26503 trim_cache size 0 max 16384 2024-09-15T22:20:08.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 Processor -- start 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 -- start start 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e334aa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41bde70 con 0x559e3348a5d0 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41bf9b0 con 0x559e3348bb40 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aee206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41be540 con 0x559e33495370 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae9bc9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40356/0 (socket says 172.21.15.77:40356) 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 -- 172.21.15.77:0/1466407494 learned_addr learned my addr 172.21.15.77:0/1466407494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:08.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae9bc9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55922/0 (socket says 172.21.15.77:55922) 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e334aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 -- 172.21.15.77:0/1466407494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 msgr2=0x559e334aa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e334aa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 -- 172.21.15.77:0/1466407494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 msgr2=0x559e33486e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.389+0000 7f2ae93c8640 1 -- 172.21.15.77:0/1466407494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e33490510 con 0x559e3348bb40 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.390+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.390+0000 7f2ae93c8640 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40bc3b0 tx=0x7f2ad40f6530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6a0f1cc6649535 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.390+0000 7f2acf7fe640 1 -- 172.21.15.77:0/1466407494 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4049040 con 0x559e3348bb40 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.390+0000 7f2acf7fe640 1 -- 172.21.15.77:0/1466407494 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad40bc6b0 con 0x559e3348bb40 2024-09-15T22:20:08.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.390+0000 7f2acf7fe640 1 -- 172.21.15.77:0/1466407494 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad403b460 con 0x559e3348bb40 2024-09-15T22:20:08.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- 172.21.15.77:0/1466407494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 msgr2=0x559e33491880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40bc3b0 tx=0x7f2ad40f6530 comp rx=0 tx=0).stop 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- 172.21.15.77:0/1466407494 shutdown_connections 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e33495370 0x559e334aa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348bb40 0x559e33491880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- 172.21.15.77:0/1466407494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e3348a5d0 0x559e33486e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- 172.21.15.77:0/1466407494 >> 172.21.15.77:0/1466407494 conn(0x559e33453310 msgr2=0x559e33486700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- 172.21.15.77:0/1466407494 shutdown_connections 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- 172.21.15.77:0/1466407494 wait complete. 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 -- start start 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 Processor -- start 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e334aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 0x559e334aae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.391+0000 7f2aee206740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 0x559e334a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aee206740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41bf2e0 con 0x559e33495370 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aee206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41bec10 con 0x559e3348a5d0 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aee206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41bbe70 con 0x559e3348bb40 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aee206740 10 client.0 _ll_register_callbacks cb 0x7f2adc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 0x559e334a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 0x559e334aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2ae93c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 0x559e334aae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43290/0 (socket says 172.21.15.77:43290) 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2ae93c8640 1 -- 172.21.15.77:0/467026021 learned_addr learned my addr 172.21.15.77:0/467026021 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e334aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 -- 172.21.15.77:0/467026021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 msgr2=0x559e334aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 0x559e334aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 -- 172.21.15.77:0/467026021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 msgr2=0x559e334aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e334aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 -- 172.21.15.77:0/467026021 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e334a8710 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e334aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2aea3ca640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 0x559e334a7ee0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x559e334a50c0 tx=0x7f2adc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f78c8d9e7ed3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.392+0000 7f2acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc02f070 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2adc01b7c0 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad40bc6b0 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2adc03b050 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 10 client.17430 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e3350f780 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc02b440 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2adc0333b0 con 0x559e33495370 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 10 client.17430 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 10 client.17430 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:20:08.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 20 client.17430 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e3350fe80 0x559e335122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 4 client.17430 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.393+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2ad40bc6b0 con 0x559e3350fe80 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e3350fe80 0x559e335122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2acd7fa640 10 client.17430 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2ae9bc9640 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e3350fe80 0x559e335122a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92ac02ae6b1a73ef server_cookie=3c75b894b77c5b63 in_seq=0 out_seq=0 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2acd7fa640 1 -- 172.21.15.77:0/467026021 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2ad40bc6b0 con 0x559e3350fe80 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2acd7fa640 10 client.17430 handle_command_reply: tid=0 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.394+0000 7f2aee206740 1 client.17430 shutdown 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 msgr2=0x559e334a7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 0x559e334a7ee0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x559e334a50c0 tx=0x7f2adc004ab0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 shutdown_connections 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x559e3350fe80 0x559e335122a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559e33495370 0x559e334a7ee0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559e3348bb40 0x559e334aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 --2- 172.21.15.77:0/467026021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3348a5d0 0x559e334aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 >> 172.21.15.77:0/467026021 conn(0x559e33453310 msgr2=0x559e334854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2acd7fa640 0 client.17430 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 shutdown_connections 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 1 -- 172.21.15.77:0/467026021 wait complete. 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.395+0000 7f2aee206740 20 client.17430 trim_cache size 0 max 16384 2024-09-15T22:20:08.402 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 msgr2=0x7f2ad00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 0x7f2ad00a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009fd0 tx=0x7f2adc004a40 comp rx=0 tx=0).stop 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 msgr2=0x7f2ae41a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2ad402a930 tx=0x7f2ad4003b20 comp rx=0 tx=0).stop 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 shutdown_connections 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2ad00a3430 0x7f2ad00a58f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ae41139e0 0x7f2ae41abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ae410ae80 0x7f2ae41a7ae0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 --2- 172.21.15.77:0/3424638854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ae410a560 0x7f2ae41a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:08.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 >> 172.21.15.77:0/3424638854 conn(0x7f2ae406e2d0 msgr2=0x7f2ae4109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:08.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 shutdown_connections 2024-09-15T22:20:08.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:08.396+0000 7f2aebe2d640 1 -- 172.21.15.77:0/3424638854 wait complete. 2024-09-15T22:20:08.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 608268 8.34 MB/sec execute 257 sec latency 127.394 ms 2024-09-15T22:20:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:08 smithi077 ceph-mon[30980]: pgmap v236: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-15T22:20:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:08 smithi196 ceph-mon[35109]: pgmap v236: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-15T22:20:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:08 smithi117 ceph-mon[35021]: pgmap v236: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-15T22:20:08.666 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:09.110 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:20:09.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 610724 8.37 MB/sec execute 258 sec latency 42.241 ms 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1377393900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 msgr2=0x7f2ba8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1377393900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba8103110 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009930 tx=0x7f2ba402a420 comp rx=0 tx=0).stop 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1377393900 shutdown_connections 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1377393900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81082a0 0x7f2ba810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1377393900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81036e0 0x7f2ba8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1377393900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba8103110 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1377393900 >> 172.21.15.77:0/1377393900 conn(0x7f2ba80fc7d0 msgr2=0x7f2ba80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1377393900 shutdown_connections 2024-09-15T22:20:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.612+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1377393900 wait complete. 2024-09-15T22:20:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 Processor -- start 2024-09-15T22:20:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 -- start start 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba81a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81082a0 0x7f2ba819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81120a0 con 0x7f2ba8102d30 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8111f20 con 0x7f2ba81082a0 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bafb5a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8112220 con 0x7f2ba81036e0 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55878/0 (socket says 172.21.15.77:55878) 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.613+0000 7f2bad0f5640 1 -- 172.21.15.77:0/1714820416 learned_addr learned my addr 172.21.15.77:0/1714820416 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bae0f7640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81082a0 0x7f2ba819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba81a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 -- 172.21.15.77:0/1714820416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81082a0 msgr2=0x7f2ba819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81082a0 0x7f2ba819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 -- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 msgr2=0x7f2ba81a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba81a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 -- 172.21.15.77:0/1714820416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba819ad20 con 0x7f2ba81036e0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.614+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800d950 tx=0x7f2b9800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.615+0000 7f2b9effd640 1 -- 172.21.15.77:0/1714820416 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98017070 con 0x7f2ba81036e0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.615+0000 7f2b9effd640 1 -- 172.21.15.77:0/1714820416 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b98009c30 con 0x7f2ba81036e0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.615+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba819b010 con 0x7f2ba81036e0 2024-09-15T22:20:09.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.615+0000 7f2b9effd640 1 -- 172.21.15.77:0/1714820416 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b980047a0 con 0x7f2ba81036e0 2024-09-15T22:20:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.615+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81a7300 con 0x7f2ba81036e0 2024-09-15T22:20:09.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.617+0000 7f2b9effd640 1 -- 172.21.15.77:0/1714820416 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2b98004020 con 0x7f2ba81036e0 2024-09-15T22:20:09.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2b9effd640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 0x7f2b940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 0x7f2b940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2b9effd640 1 -- 172.21.15.77:0/1714820416 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2b980cfcc0 con 0x7f2ba81036e0 2024-09-15T22:20:09.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2bb1f33740 1 Processor -- start 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2bb1f33740 1 -- start start 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.618+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c12d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556591b6b2a0 con 0x556591c06080 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556591b12b10 con 0x556591c10940 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556591b67260 con 0x556591c05760 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 0x7f2b940a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009900 tx=0x7f2ba40058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55882/0 (socket says 172.21.15.77:55882) 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bae0f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c12d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c10400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44716/0 (socket says 172.21.15.77:44716) 2024-09-15T22:20:09.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 -- 172.21.15.77:0/3729061882 learned_addr learned my addr 172.21.15.77:0/3729061882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 -- 172.21.15.77:0/3729061882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 msgr2=0x556591c12d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c12d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 -- 172.21.15.77:0/3729061882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 msgr2=0x556591c10400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c10400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 -- 172.21.15.77:0/3729061882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565918dcb90 con 0x556591c05760 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bae0f7640 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c12d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:09.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009930 tx=0x7f2ba403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f08a301a40410b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.619+0000 7f2b93fff640 1 -- 172.21.15.77:0/3729061882 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba404d070 con 0x556591c05760 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.620+0000 7f2b93fff640 1 -- 172.21.15.77:0/3729061882 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba4031a10 con 0x556591c05760 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.620+0000 7f2b93fff640 1 -- 172.21.15.77:0/3729061882 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4035040 con 0x556591c05760 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- 172.21.15.77:0/3729061882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 msgr2=0x556591c05b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009930 tx=0x7f2ba403d000 comp rx=0 tx=0).stop 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- 172.21.15.77:0/3729061882 shutdown_connections 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c12d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c10400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/3729061882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c05b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- 172.21.15.77:0/3729061882 >> 172.21.15.77:0/3729061882 conn(0x556591c01cf0 msgr2=0x556591c020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- 172.21.15.77:0/3729061882 shutdown_connections 2024-09-15T22:20:09.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- 172.21.15.77:0/3729061882 wait complete. 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 -- start start 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.621+0000 7f2bb1f33740 1 Processor -- start 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556591c13510 con 0x556591c06080 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556591af36e0 con 0x556591c10940 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556591c13690 con 0x556591c05760 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bb1f33740 10 client.0 _ll_register_callbacks cb 0x556591b17a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bad0f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44728/0 (socket says 172.21.15.77:44728) 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55892/0 (socket says 172.21.15.77:55892) 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bad0f5640 1 -- 172.21.15.77:0/692813309 learned_addr learned my addr 172.21.15.77:0/692813309 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.622+0000 7f2bae0f7640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 -- 172.21.15.77:0/692813309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 msgr2=0x556591c0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 -- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 msgr2=0x556591c0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 -- 172.21.15.77:0/692813309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556591c1c9d0 con 0x556591c06080 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800b750 tx=0x7f2b9809b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e6b3b38ca828e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:09.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556591b789d0 con 0x556591c06080 2024-09-15T22:20:09.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2b917fa640 1 -- 172.21.15.77:0/692813309 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9809ffc0 con 0x556591c06080 2024-09-15T22:20:09.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2b917fa640 1 -- 172.21.15.77:0/692813309 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2b98098280 con 0x556591c06080 2024-09-15T22:20:09.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2b917fa640 1 -- 172.21.15.77:0/692813309 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2b980a4880 con 0x556591c06080 2024-09-15T22:20:09.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bb1f33740 10 client.17442 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:09.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.623+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=34}) -- 0x556591c1f380 con 0x556591c06080 2024-09-15T22:20:09.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2b917fa640 1 -- 172.21.15.77:0/692813309 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2b980860b0 con 0x556591c06080 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 10 client.17442 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 10 client.17442 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 20 client.17442 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c20730 0x556591c22b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 4 client.17442 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556591b72d20 con 0x556591c20730 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c20730 0x556591c22b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2b917fa640 10 client.17442 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.624+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c20730 0x556591c22b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62ccc1a3567a948 server_cookie=2fcea8f812347406 in_seq=0 out_seq=0 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2b917fa640 1 -- 172.21.15.77:0/692813309 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556591b72d20 con 0x556591c20730 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2b917fa640 10 client.17442 handle_command_reply: tid=0 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 client.17442 shutdown 2024-09-15T22:20:09.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 msgr2=0x556591c0dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800b750 tx=0x7f2b9809b660 comp rx=0 tx=0).stop 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 shutdown_connections 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c20730 0x556591c22b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c0dd40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/692813309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2b917fa640 0 client.17442 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.625+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 >> 172.21.15.77:0/692813309 conn(0x556591c01cf0 msgr2=0x556591c07a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.626+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 shutdown_connections 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.626+0000 7f2bb1f33740 1 -- 172.21.15.77:0/692813309 wait complete. 2024-09-15T22:20:09.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.626+0000 7f2bb1f33740 20 client.17442 trim_cache size 0 max 16384 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.644+0000 7f2bb1f33740 1 Processor -- start 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.644+0000 7f2bb1f33740 1 -- start start 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c10940 0x556591c0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81b12f0 con 0x556591c06080 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81b2e30 con 0x556591c05760 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81b19c0 con 0x556591c10940 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41618/0 (socket says 172.21.15.77:41618) 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 -- 172.21.15.77:0/25552685 learned_addr learned my addr 172.21.15.77:0/25552685 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bae0f7640 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c10940 0x556591c0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 -- 172.21.15.77:0/25552685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c10940 msgr2=0x556591c0aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c10940 0x556591c0aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 -- 172.21.15.77:0/25552685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 msgr2=0x556591c288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.645+0000 7f2bad8f6640 1 -- 172.21.15.77:0/25552685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556591c05eb0 con 0x556591c05760 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.646+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f047adda252283 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.646+0000 7f2b937fe640 1 -- 172.21.15.77:0/25552685 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba406d070 con 0x556591c05760 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.646+0000 7f2b937fe640 1 -- 172.21.15.77:0/25552685 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba4049e50 con 0x556591c05760 2024-09-15T22:20:09.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.646+0000 7f2b937fe640 1 -- 172.21.15.77:0/25552685 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4040030 con 0x556591c05760 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- 172.21.15.77:0/25552685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 msgr2=0x556591c283a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4000c00 tx=0x7f2ba402a420 comp rx=0 tx=0).stop 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- 172.21.15.77:0/25552685 shutdown_connections 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c10940 0x556591c0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/25552685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c05760 0x556591c283a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- 172.21.15.77:0/25552685 >> 172.21.15.77:0/25552685 conn(0x556591c01cf0 msgr2=0x556591c0f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- 172.21.15.77:0/25552685 shutdown_connections 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- 172.21.15.77:0/25552685 wait complete. 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 -- start start 2024-09-15T22:20:09.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.647+0000 7f2bb1f33740 1 Processor -- start 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81b2760 con 0x556591c06080 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81b2090 con 0x556591c10940 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba81af2f0 con 0x556591c05760 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bb1f33740 10 client.0 _ll_register_callbacks cb 0x7f2b9809b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bad8f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55908/0 (socket says 172.21.15.77:55908) 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bad8f6640 1 -- 172.21.15.77:0/486018437 learned_addr learned my addr 172.21.15.77:0/486018437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:09.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.648+0000 7f2bae0f7640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 -- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 msgr2=0x556591c158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 -- 172.21.15.77:0/486018437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 msgr2=0x556591c19000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c19000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 -- 172.21.15.77:0/486018437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556591c160c0 con 0x556591c05760 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bad8f6640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4004650 tx=0x7f2ba4004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911371c162d746ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba406d030 con 0x556591c05760 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba4049e50 con 0x556591c05760 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7f2ba4068ae0 con 0x556591c05760 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ba407a050 con 0x556591c05760 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bb1f33740 10 client.26524 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:09.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.649+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556591c8be60 con 0x556591c05760 2024-09-15T22:20:09.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba4049e50 con 0x556591c05760 2024-09-15T22:20:09.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ba4003a10 con 0x556591c05760 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 10 client.26524 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 10 client.26524 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 20 client.26524 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c8c150 0x556591c8e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 4 client.26524 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}] 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}) -- 0x556591aeeec0 con 0x556591c8c150 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c8c150 0x556591c8e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2b917fa640 10 client.26524 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.650+0000 7f2bad0f5640 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c8c150 0x556591c8e540 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6252af1b3b831be7 server_cookie=edd31fc9484ed12f in_seq=0 out_seq=0 2024-09-15T22:20:09.656 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:09.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2b917fa640 1 -- 172.21.15.77:0/486018437 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556591aeeec0 con 0x556591c8c150 2024-09-15T22:20:09.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2b917fa640 10 client.26524 handle_command_reply: tid=0 2024-09-15T22:20:09.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 client.26524 shutdown 2024-09-15T22:20:09.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 msgr2=0x556591c18ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4004650 tx=0x7f2ba4004490 comp rx=0 tx=0).stop 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 shutdown_connections 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556591c8c150 0x556591c8e540 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556591c10940 0x556591c158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2b917fa640 0 client.26524 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556591c06080 0x556591c19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 --2- 172.21.15.77:0/486018437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556591c05760 0x556591c18ac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 >> 172.21.15.77:0/486018437 conn(0x556591c01cf0 msgr2=0x556591c07180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 shutdown_connections 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 1 -- 172.21.15.77:0/486018437 wait complete. 2024-09-15T22:20:09.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bb1f33740 20 client.26524 trim_cache size 0 max 16384 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 26, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "fc7daac0": { 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 26, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:20:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 143.7, 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:09.764 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 msgr2=0x7f2b940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 0x7f2b940a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009900 tx=0x7f2ba40058d0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 msgr2=0x7f2ba81a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800d950 tx=0x7f2b9800de10 comp rx=0 tx=0).stop 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 shutdown_connections 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2b940a33b0 0x7f2b940a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba81082a0 0x7f2ba819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba81036e0 0x7f2ba81a0970 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 --2- 172.21.15.77:0/1714820416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8102d30 0x7f2ba81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 >> 172.21.15.77:0/1714820416 conn(0x7f2ba80fc7d0 msgr2=0x7f2ba8100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 shutdown_connections 2024-09-15T22:20:09.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:09.759+0000 7f2bafb5a640 1 -- 172.21.15.77:0/1714820416 wait complete. 2024-09-15T22:20:10.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 -- 172.21.15.77:0/359729284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528101420 msgr2=0x7fd52810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 --2- 172.21.15.77:0/359729284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528101420 0x7fd52810d950 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd520009930 tx=0x7fd52002a420 comp rx=0 tx=0).stop 2024-09-15T22:20:10.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 -- 172.21.15.77:0/359729284 shutdown_connections 2024-09-15T22:20:10.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 --2- 172.21.15.77:0/359729284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528101420 0x7fd52810d950 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 --2- 172.21.15.77:0/359729284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528100a80 0x7fd528100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.350+0000 7fd52f100640 1 --2- 172.21.15.77:0/359729284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 0x7fd528106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- 172.21.15.77:0/359729284 >> 172.21.15.77:0/359729284 conn(0x7fd5280fc7a0 msgr2=0x7fd5280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- 172.21.15.77:0/359729284 shutdown_connections 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- 172.21.15.77:0/359729284 wait complete. 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 Processor -- start 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- start start 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528101420 0x7fd5281997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 0x7fd5281a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd528110e80 con 0x7fd528100a80 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd528110d00 con 0x7fd528101420 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52f100640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd528111000 con 0x7fd528106a80 2024-09-15T22:20:10.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44760/0 (socket says 172.21.15.77:44760) 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52ce9c640 1 -- 172.21.15.77:0/2733730897 learned_addr learned my addr 172.21.15.77:0/2733730897 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.351+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 0x7fd5281a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd527fff640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528101420 0x7fd5281997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 -- 172.21.15.77:0/2733730897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 msgr2=0x7fd5281a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 0x7fd5281a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 -- 172.21.15.77:0/2733730897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528101420 msgr2=0x7fd5281997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528101420 0x7fd5281997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 -- 172.21.15.77:0/2733730897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5281a4850 con 0x7fd528100a80 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd51800d770 tx=0x7fd51800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd525ffb640 1 -- 172.21.15.77:0/2733730897 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518017070 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5281a4a20 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd525ffb640 1 -- 172.21.15.77:0/2733730897 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd518005500 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd525ffb640 1 -- 172.21.15.77:0/2733730897 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518004e00 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.352+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5281a4ee0 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.355+0000 7fd525ffb640 1 -- 172.21.15.77:0/2733730897 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd518004fa0 con 0x7fd528100a80 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.355+0000 7fd5314d9740 1 Processor -- start 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.355+0000 7fd5314d9740 1 -- start start 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa82b40 0x562d5aa81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d5a9e72f0 con 0x562d5aa815d0 2024-09-15T22:20:10.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d5a98ecf0 con 0x562d5aa8c370 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd5314d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562d5a9e32b0 con 0x562d5aa82b40 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd525ffb640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 0x7fd5100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44770/0 (socket says 172.21.15.77:44770) 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52ce9c640 1 -- 172.21.15.77:0/817498152 learned_addr learned my addr 172.21.15.77:0/817498152 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 0x7fd5100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52d69d640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa82b40 0x562d5aa81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd525ffb640 1 -- 172.21.15.77:0/2733730897 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd5180cfa20 con 0x7fd528100a80 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 -- 172.21.15.77:0/817498152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa82b40 msgr2=0x562d5aa81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa82b40 0x562d5aa81090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 -- 172.21.15.77:0/817498152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 msgr2=0x562d5aa81a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 -- 172.21.15.77:0/817498152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5a9733a0 con 0x562d5aa8c370 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 0x7fd5100a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd520006010 tx=0x7fd520031480 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.356+0000 7fd527fff640 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa8e760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0bf95a8810b636 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd50effd640 1 -- 172.21.15.77:0/817498152 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004b20 con 0x562d5aa8c370 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd50effd640 1 -- 172.21.15.77:0/817498152 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c004cc0 con 0x562d5aa8c370 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd50effd640 1 -- 172.21.15.77:0/817498152 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0126b0 con 0x562d5aa8c370 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 -- 172.21.15.77:0/817498152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 msgr2=0x562d5aa8e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa8e760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c003e00 comp rx=0 tx=0).stop 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 -- 172.21.15.77:0/817498152 shutdown_connections 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa8e760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa815d0 0x562d5aa81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 --2- 172.21.15.77:0/817498152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa82b40 0x562d5aa81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 -- 172.21.15.77:0/817498152 >> 172.21.15.77:0/817498152 conn(0x562d5aa4a310 msgr2=0x562d5aa7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 -- 172.21.15.77:0/817498152 shutdown_connections 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.357+0000 7fd5314d9740 1 -- 172.21.15.77:0/817498152 wait complete. 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 -- start start 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 Processor -- start 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa8bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa89530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d5aa93480 con 0x562d5aa82b40 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d5a9a2060 con 0x562d5aa8c370 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562d5aa93600 con 0x562d5aa815d0 2024-09-15T22:20:10.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd5314d9740 10 client.0 _ll_register_callbacks cb 0x562d5a6f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd527fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd527fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44774/0 (socket says 172.21.15.77:44774) 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd527fff640 1 -- 172.21.15.77:0/1531016381 learned_addr learned my addr 172.21.15.77:0/1531016381 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.358+0000 7fd52d69d640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa89530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa8bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 -- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 msgr2=0x562d5aa8bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa8bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 -- 172.21.15.77:0/1531016381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 msgr2=0x562d5aa89530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa89530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 -- 172.21.15.77:0/1531016381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5a7583c0 con 0x562d5aa82b40 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa8bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:10.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd527fff640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000f80 tx=0x7fd51c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92152aeb8f174bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd507fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c005200 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd51c0053a0 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51c004cc0 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd51c012450 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd5314d9740 10 client.17460 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d5aa9eee0 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.359+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c004cc0 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd51c00f040 con 0x562d5aa82b40 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 10 client.17460 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 10 client.17460 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 20 client.17460 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:10.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5aaa0290 0x562d5aaa2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 4 client.17460 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d5a9f4860 con 0x562d5aaa0290 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5aaa0290 0x562d5aaa2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd507fff640 10 client.17460 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.360+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5aaa0290 0x562d5aaa2680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab6ef4ced55b6de0 server_cookie=e72ec165c07e971c in_seq=0 out_seq=0 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd507fff640 1 -- 172.21.15.77:0/1531016381 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d5a9f4860 con 0x562d5aaa0290 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd507fff640 10 client.17460 handle_command_reply: tid=0 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 client.17460 shutdown 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 msgr2=0x562d5aa88fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000f80 tx=0x7fd51c0098a0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 shutdown_connections 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5aaa0290 0x562d5aaa2680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa8c370 0x562d5aa89530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa82b40 0x562d5aa88fb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd507fff640 0 client.17460 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 --2- 172.21.15.77:0/1531016381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa8bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 >> 172.21.15.77:0/1531016381 conn(0x562d5aa4a310 msgr2=0x562d5aa83930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.361+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 shutdown_connections 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.362+0000 7fd5314d9740 1 -- 172.21.15.77:0/1531016381 wait complete. 2024-09-15T22:20:10.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.362+0000 7fd5314d9740 20 client.17460 trim_cache size 0 max 16384 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.374+0000 7fd5314d9740 1 Processor -- start 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.374+0000 7fd5314d9740 1 -- start start 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa7de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa9c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ace70 con 0x562d5aa8c370 2024-09-15T22:20:10.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ae9b0 con 0x562d5aa82b40 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd5314d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ad540 con 0x562d5aa815d0 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44784/0 (socket says 172.21.15.77:44784) 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 -- 172.21.15.77:0/2877989096 learned_addr learned my addr 172.21.15.77:0/2877989096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa7de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd527fff640 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa9c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 -- 172.21.15.77:0/2877989096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 msgr2=0x562d5aa7de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa7de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 -- 172.21.15.77:0/2877989096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 msgr2=0x562d5aa9c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa9c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 -- 172.21.15.77:0/2877989096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5aaa2c40 con 0x562d5aa8c370 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd52d69d640 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd520009230 tx=0x7fd52003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66adcfa09f81680 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd50e7fc640 1 -- 172.21.15.77:0/2877989096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520005220 con 0x562d5aa8c370 2024-09-15T22:20:10.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd50e7fc640 1 -- 172.21.15.77:0/2877989096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd520004c80 con 0x562d5aa8c370 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.375+0000 7fd50e7fc640 1 -- 172.21.15.77:0/2877989096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5200039c0 con 0x562d5aa8c370 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- 172.21.15.77:0/2877989096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 msgr2=0x562d5aa88870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd520009230 tx=0x7fd52003c000 comp rx=0 tx=0).stop 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- 172.21.15.77:0/2877989096 shutdown_connections 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa88870 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa9c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 --2- 172.21.15.77:0/2877989096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa7de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- 172.21.15.77:0/2877989096 >> 172.21.15.77:0/2877989096 conn(0x562d5aa4a310 msgr2=0x562d5aa7d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- 172.21.15.77:0/2877989096 shutdown_connections 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- 172.21.15.77:0/2877989096 wait complete. 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.376+0000 7fd5314d9740 1 -- start start 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 Processor -- start 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa85f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281ae2e0 con 0x562d5aa8c370 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281adc10 con 0x562d5aa82b40 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281aae70 con 0x562d5aa815d0 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd5314d9740 10 client.0 _ll_register_callbacks cb 0x7fd518098090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52d69d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55964/0 (socket says 172.21.15.77:55964) 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 -- 172.21.15.77:0/3437733049 learned_addr learned my addr 172.21.15.77:0/3437733049 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd527fff640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa85f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 -- 172.21.15.77:0/3437733049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 msgr2=0x562d5aa85f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa85f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 -- 172.21.15.77:0/3437733049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 msgr2=0x562d5aa978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52ce9c640 1 -- 172.21.15.77:0/3437733049 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5aa86850 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.377+0000 7fd52d69d640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd52ce9c640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd518005c20 tx=0x7fd518098270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b3303c9f7d91d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51809ba20 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9438+0+0 (secure 0 0 0) 0x7fd518099d20 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd520004c80 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd518020020 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 10 client.26542 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d5aa98f60 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd520004c80 con 0x562d5aa815d0 2024-09-15T22:20:10.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd5180cd060 con 0x562d5aa815d0 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 10 client.26542 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 10 client.26542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 20 client.26542 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5ab0f630 0x562d5ab11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 4 client.26542 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.378+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d5a9eed70 con 0x562d5ab0f630 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd527fff640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5ab0f630 0x562d5ab11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd507fff640 10 client.26542 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd527fff640 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5ab0f630 0x562d5ab11a20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27b083d4cc7c84ef server_cookie=3ea7576bbe1c4cc9 in_seq=0 out_seq=0 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd507fff640 1 -- 172.21.15.77:0/3437733049 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d5a9eed70 con 0x562d5ab0f630 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd507fff640 10 client.26542 handle_command_reply: tid=0 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 client.26542 shutdown 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 msgr2=0x562d5aa85a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd518005c20 tx=0x7fd518098270 comp rx=0 tx=0).stop 2024-09-15T22:20:10.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 shutdown_connections 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x562d5ab0f630 0x562d5ab11a20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d5aa8c370 0x562d5aa978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd507fff640 0 client.26542 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d5aa82b40 0x562d5aa85f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 --2- 172.21.15.77:0/3437733049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d5aa815d0 0x562d5aa85a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 >> 172.21.15.77:0/3437733049 conn(0x562d5aa4a310 msgr2=0x562d5aa8a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 shutdown_connections 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 1 -- 172.21.15.77:0/3437733049 wait complete. 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.379+0000 7fd5314d9740 20 client.26542 trim_cache size 0 max 16384 2024-09-15T22:20:10.391 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:10.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 msgr2=0x7fd5100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 0x7fd5100a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd520006010 tx=0x7fd520031480 comp rx=0 tx=0).stop 2024-09-15T22:20:10.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 msgr2=0x7fd528199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd51800d770 tx=0x7fd51800dc30 comp rx=0 tx=0).stop 2024-09-15T22:20:10.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 shutdown_connections 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5100a33b0 0x7fd5100a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd528106a80 0x7fd5281a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd528101420 0x7fd5281997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 --2- 172.21.15.77:0/2733730897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd528100a80 0x7fd528199290 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 >> 172.21.15.77:0/2733730897 conn(0x7fd5280fc7a0 msgr2=0x7fd5280fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 shutdown_connections 2024-09-15T22:20:10.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:10.380+0000 7fd52f100640 1 -- 172.21.15.77:0/2733730897 wait complete. 2024-09-15T22:20:10.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.35 MB/sec execute 259 sec latency 806.508 ms 2024-09-15T22:20:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:10 smithi077 ceph-mon[30980]: pgmap v237: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-09-15T22:20:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:10 smithi196 ceph-mon[35109]: pgmap v237: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-09-15T22:20:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:10 smithi117 ceph-mon[35021]: pgmap v237: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2024-09-15T22:20:11.300 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fc7daac0', quiesce took 5.5 seconds. Will release after: 18.2 2024-09-15T22:20:11.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.32 MB/sec execute 260 sec latency 1806.535 ms 2024-09-15T22:20:11.891 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:20:12.134 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:12.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.29 MB/sec execute 261 sec latency 2806.569 ms 2024-09-15T22:20:12.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:12 smithi077 ceph-mon[30980]: pgmap v238: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-15T22:20:12.535 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:12 smithi077 ceph-mon[30980]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:20:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:12 smithi196 ceph-mon[35109]: pgmap v238: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-15T22:20:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:12 smithi196 ceph-mon[35109]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:20:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:12 smithi117 ceph-mon[35021]: pgmap v238: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 327 op/s 2024-09-15T22:20:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:12 smithi117 ceph-mon[35021]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 -- 172.21.15.77:0/507870314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720101420 msgr2=0x7fd72010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 --2- 172.21.15.77:0/507870314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720101420 0x7fd72010d950 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd718009cd0 tx=0x7fd71802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 -- 172.21.15.77:0/507870314 shutdown_connections 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 --2- 172.21.15.77:0/507870314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720101420 0x7fd72010d950 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 --2- 172.21.15.77:0/507870314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720100a80 0x7fd720100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 --2- 172.21.15.77:0/507870314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 0x7fd720106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 -- 172.21.15.77:0/507870314 >> 172.21.15.77:0/507870314 conn(0x7fd7200fc7a0 msgr2=0x7fd7200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 -- 172.21.15.77:0/507870314 shutdown_connections 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.614+0000 7fd727ab3640 1 -- 172.21.15.77:0/507870314 wait complete. 2024-09-15T22:20:12.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 Processor -- start 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 -- start start 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101420 0x7fd720198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 0x7fd72019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd720110e80 con 0x7fd720100a80 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd720110d00 con 0x7fd720106a80 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd727ab3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd720111000 con 0x7fd720101420 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44820/0 (socket says 172.21.15.77:44820) 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 -- 172.21.15.77:0/1475771119 learned_addr learned my addr 172.21.15.77:0/1475771119 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd726050640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 0x7fd72019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 -- 172.21.15.77:0/1475771119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101420 msgr2=0x7fd720198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101420 0x7fd720198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 -- 172.21.15.77:0/1475771119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 msgr2=0x7fd72019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 0x7fd72019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 -- 172.21.15.77:0/1475771119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72019d3c0 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.615+0000 7fd72584f640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.616+0000 7fd712ffd640 1 -- 172.21.15.77:0/1475771119 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714017070 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.616+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72019d6b0 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.616+0000 7fd712ffd640 1 -- 172.21.15.77:0/1475771119 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714009c30 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.616+0000 7fd712ffd640 1 -- 172.21.15.77:0/1475771119 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f6b0 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.616+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201a4f90 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd712ffd640 1 -- 172.21.15.77:0/1475771119 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd7140109d0 con 0x7fd720100a80 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 Processor -- start 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 -- start start 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe3407090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3414760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fe336d2f0 con 0x561fe34075d0 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561fe3314cf0 con 0x561fe3412370 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.618+0000 7fd729e8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fe33692b0 con 0x561fe3408b40 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd712ffd640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44828/0 (socket says 172.21.15.77:44828) 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 -- 172.21.15.77:0/749818375 learned_addr learned my addr 172.21.15.77:0/749818375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72504e640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3414760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd726050640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd726050640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe3407090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd712ffd640 1 -- 172.21.15.77:0/1475771119 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7fd7140cfe20 con 0x7fd720100a80 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 -- 172.21.15.77:0/749818375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 msgr2=0x561fe3407090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe3407090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 -- 172.21.15.77:0/749818375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 msgr2=0x561fe3414760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3414760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 -- 172.21.15.77:0/749818375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe32f93a0 con 0x561fe34075d0 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd726050640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe3407090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd72584f640 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd714098680 tx=0x7fd714010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109e103db00183a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd70bfff640 1 -- 172.21.15.77:0/749818375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714020070 con 0x561fe34075d0 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd70bfff640 1 -- 172.21.15.77:0/749818375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71409c3b0 con 0x561fe34075d0 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.619+0000 7fd70bfff640 1 -- 172.21.15.77:0/749818375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714098480 con 0x561fe34075d0 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- 172.21.15.77:0/749818375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 msgr2=0x561fe3407a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd714098680 tx=0x7fd714010d70 comp rx=0 tx=0).stop 2024-09-15T22:20:12.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd726050640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 0x7fd70c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd718009a60 tx=0x7fd7180095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- 172.21.15.77:0/749818375 shutdown_connections 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3414760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3407a10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 --2- 172.21.15.77:0/749818375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe3407090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- 172.21.15.77:0/749818375 >> 172.21.15.77:0/749818375 conn(0x561fe33d0310 msgr2=0x561fe3403c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- 172.21.15.77:0/749818375 shutdown_connections 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- 172.21.15.77:0/749818375 wait complete. 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.620+0000 7fd729e8c740 1 -- start start 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 Processor -- start 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe340f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe340f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fe3415070 con 0x561fe34075d0 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561fe3328060 con 0x561fe3412370 2024-09-15T22:20:12.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fe34151f0 con 0x561fe3408b40 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd729e8c740 10 client.0 _ll_register_callbacks cb 0x561fe307b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44832/0 (socket says 172.21.15.77:44832) 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 -- 172.21.15.77:0/2734987998 learned_addr learned my addr 172.21.15.77:0/2734987998 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72504e640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe340f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 -- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 msgr2=0x561fe340f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe340f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 -- 172.21.15.77:0/2734987998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 msgr2=0x561fe340f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe340f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72584f640 1 -- 172.21.15.77:0/2734987998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe30de3c0 con 0x561fe34075d0 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.621+0000 7fd72504e640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe340f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd72584f640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd714098110 tx=0x7fd71409be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fee4e582d0e522 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714020040 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7fd714099fb0 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71409c3b0 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd7140cd5b0 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 10 client.17478 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fe3420b90 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71409c3b0 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd714010460 con 0x561fe34075d0 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 10 client.17478 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 10 client.17478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 20 client.17478 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe3421ee0 0x561fe34242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 4 client.17478 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe337a860 con 0x561fe3421ee0 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.622+0000 7fd72504e640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe3421ee0 0x561fe34242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.623+0000 7fd7097fa640 10 client.17478 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.623+0000 7fd72504e640 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe3421ee0 0x561fe34242d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=beedb4c05437922e server_cookie=1a5fbf169e76a01 in_seq=0 out_seq=0 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd7097fa640 1 -- 172.21.15.77:0/2734987998 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe337a860 con 0x561fe3421ee0 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd7097fa640 10 client.17478 handle_command_reply: tid=0 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 client.17478 shutdown 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 msgr2=0x561fe3411dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd714098110 tx=0x7fd71409be10 comp rx=0 tx=0).stop 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 shutdown_connections 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe3421ee0 0x561fe34242d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe340f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe340f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2734987998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe3411dd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd7097fa640 0 client.17478 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 >> 172.21.15.77:0/2734987998 conn(0x561fe33d0310 msgr2=0x561fe3409990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 shutdown_connections 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 1 -- 172.21.15.77:0/2734987998 wait complete. 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.624+0000 7fd729e8c740 20 client.17478 trim_cache size 0 max 16384 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 Processor -- start 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 -- start start 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe34272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3427830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201b1370 con 0x561fe34075d0 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201b2eb0 con 0x561fe3412370 2024-09-15T22:20:12.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd729e8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201b1a40 con 0x561fe3408b40 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44834/0 (socket says 172.21.15.77:44834) 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 -- 172.21.15.77:0/202419788 learned_addr learned my addr 172.21.15.77:0/202419788 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72504e640 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe34272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 -- 172.21.15.77:0/202419788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 msgr2=0x561fe34272f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe34272f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 -- 172.21.15.77:0/202419788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 msgr2=0x561fe3427830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3427830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 -- 172.21.15.77:0/202419788 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe340fb40 con 0x561fe34075d0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.637+0000 7fd72584f640 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd714098630 tx=0x7fd714010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bc013d68fb2a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd70b7fe640 1 -- 172.21.15.77:0/202419788 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714020070 con 0x561fe34075d0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd70b7fe640 1 -- 172.21.15.77:0/202419788 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71409c3b0 con 0x561fe34075d0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd70b7fe640 1 -- 172.21.15.77:0/202419788 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7140860b0 con 0x561fe34075d0 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 -- 172.21.15.77:0/202419788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 msgr2=0x561fe340e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd714098630 tx=0x7fd714010d70 comp rx=0 tx=0).stop 2024-09-15T22:20:12.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 -- 172.21.15.77:0/202419788 shutdown_connections 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3412370 0x561fe3427830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3408b40 0x561fe34272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 --2- 172.21.15.77:0/202419788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe34075d0 0x561fe340e060 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 -- 172.21.15.77:0/202419788 >> 172.21.15.77:0/202419788 conn(0x561fe33d0310 msgr2=0x561fe340d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 -- 172.21.15.77:0/202419788 shutdown_connections 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.638+0000 7fd729e8c740 1 -- 172.21.15.77:0/202419788 wait complete. 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 -- start start 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 Processor -- start 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe34075d0 0x561fe3427990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 0x561fe3424ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201b27e0 con 0x561fe3412370 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201b2110 con 0x561fe3408b40 2024-09-15T22:20:12.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201af370 con 0x561fe34075d0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd729e8c740 10 client.0 _ll_register_callbacks cb 0x7fd71c005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44838/0 (socket says 172.21.15.77:44838) 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 -- 172.21.15.77:0/2952328304 learned_addr learned my addr 172.21.15.77:0/2952328304 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd72584f640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe34075d0 0x561fe3427990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd72504e640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 0x561fe3424ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 -- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe34075d0 msgr2=0x561fe3427990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe34075d0 0x561fe3427990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 -- 172.21.15.77:0/2952328304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 msgr2=0x561fe3424ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 0x561fe3424ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd726050640 1 -- 172.21.15.77:0/2952328304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe3425a00 con 0x561fe3412370 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.639+0000 7fd72504e640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 0x561fe3424ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd726050640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd718009cd0 tx=0x7fd718003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efce3f21dbf06df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71804d030 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7fd718004720 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71409c3b0 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd718043070 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 10 client.17484 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fe348c770 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71409c3b0 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd718030070 con 0x561fe3412370 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 10 client.17484 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 20 client.17484 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe348cf40 0x561fe348f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 4 client.17484 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.640+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fe3374d70 con 0x561fe348cf40 2024-09-15T22:20:12.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd72584f640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe348cf40 0x561fe348f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd7097fa640 10 client.17484 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd72584f640 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe348cf40 0x561fe348f360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b5f339f6126a85 server_cookie=95da2c2df5aeb48d in_seq=0 out_seq=0 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd7097fa640 1 -- 172.21.15.77:0/2952328304 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fe3374d70 con 0x561fe348cf40 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd7097fa640 10 client.17484 handle_command_reply: tid=0 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd729e8c740 1 client.17484 shutdown 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 msgr2=0x561fe3425220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.641+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd718009cd0 tx=0x7fd718003d40 comp rx=0 tx=0).stop 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 shutdown_connections 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561fe348cf40 0x561fe348f360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe3412370 0x561fe3425220 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3408b40 0x561fe3424ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 --2- 172.21.15.77:0/2952328304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe34075d0 0x561fe3427990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 >> 172.21.15.77:0/2952328304 conn(0x561fe33d0310 msgr2=0x561fe341d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd7097fa640 0 client.17484 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:12.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 shutdown_connections 2024-09-15T22:20:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 1 -- 172.21.15.77:0/2952328304 wait complete. 2024-09-15T22:20:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd729e8c740 20 client.17484 trim_cache size 0 max 16384 2024-09-15T22:20:12.660 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:12.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 msgr2=0x7fd70c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 0x7fd70c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd718009a60 tx=0x7fd7180095c0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 msgr2=0x7fd720198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400de10 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 shutdown_connections 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd70c0a33b0 0x7fd70c0a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd720106a80 0x7fd72019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101420 0x7fd720198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.642+0000 7fd727ab3640 1 --2- 172.21.15.77:0/1475771119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd720100a80 0x7fd720198410 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.643+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 >> 172.21.15.77:0/1475771119 conn(0x7fd7200fc7a0 msgr2=0x7fd72006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.643+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 shutdown_connections 2024-09-15T22:20:12.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:12.643+0000 7fd727ab3640 1 -- 172.21.15.77:0/1475771119 wait complete. 2024-09-15T22:20:13.362 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:20:13.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.25 MB/sec execute 262 sec latency 3806.633 ms 2024-09-15T22:20:13.637 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 -- 172.21.15.77:0/3357568278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0101dd0 msgr2=0x7f6ef01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 --2- 172.21.15.77:0/3357568278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0101dd0 0x7f6ef01021b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee002a410 comp rx=0 tx=0).stop 2024-09-15T22:20:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 -- 172.21.15.77:0/3357568278 shutdown_connections 2024-09-15T22:20:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 --2- 172.21.15.77:0/3357568278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0106f00 0x7f6ef010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 --2- 172.21.15.77:0/3357568278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0102780 0x7f6ef01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 --2- 172.21.15.77:0/3357568278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0101dd0 0x7f6ef01021b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 -- 172.21.15.77:0/3357568278 >> 172.21.15.77:0/3357568278 conn(0x7f6ef00fd8b0 msgr2=0x7f6ef00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 -- 172.21.15.77:0/3357568278 shutdown_connections 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.153+0000 7f6ef6147640 1 -- 172.21.15.77:0/3357568278 wait complete. 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 Processor -- start 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 -- start start 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0101dd0 0x7f6ef0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0106f00 0x7f6ef019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111f60 con 0x7f6ef0102780 2024-09-15T22:20:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111de0 con 0x7f6ef0101dd0 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6ef6147640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01120e0 con 0x7f6ef0106f00 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44864/0 (socket says 172.21.15.77:44864) 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.154+0000 7f6eeeffd640 1 -- 172.21.15.77:0/4201296126 learned_addr learned my addr 172.21.15.77:0/4201296126 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeffff640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0106f00 0x7f6ef019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0101dd0 0x7f6ef0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 -- 172.21.15.77:0/4201296126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0106f00 msgr2=0x7f6ef019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0106f00 0x7f6ef019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 -- 172.21.15.77:0/4201296126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0101dd0 msgr2=0x7f6ef0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0101dd0 0x7f6ef0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 -- 172.21.15.77:0/4201296126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef019d3d0 con 0x7f6ef0102780 2024-09-15T22:20:14.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d950 tx=0x7f6ee400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eecff9640 1 -- 172.21.15.77:0/4201296126 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4017030 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eecff9640 1 -- 172.21.15.77:0/4201296126 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee4004020 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef019d6c0 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6eecff9640 1 -- 172.21.15.77:0/4201296126 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40047a0 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.155+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a4fa0 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.157+0000 7f6eecff9640 1 -- 172.21.15.77:0/4201296126 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6ee40041d0 con 0x7f6ef0102780 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 Processor -- start 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 -- start start 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b41fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a60b3852f0 con 0x55a60b420b40 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a60b32ccf0 con 0x55a60b41f5d0 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6ef8520740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a60b3812b0 con 0x55a60b42a370 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eecff9640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 0x7f6ed80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44878/0 (socket says 172.21.15.77:44878) 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eeffff640 1 -- 172.21.15.77:0/1918786354 learned_addr learned my addr 172.21.15.77:0/1918786354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eecff9640 1 -- 172.21.15.77:0/4201296126 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f6ee40cf9d0 con 0x7f6ef0102780 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b41fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.158+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 0x7f6ed80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 -- 172.21.15.77:0/1918786354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 msgr2=0x55a60b42c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 -- 172.21.15.77:0/1918786354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 msgr2=0x55a60b41fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b41fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 -- 172.21.15.77:0/1918786354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a60b3113a0 con 0x55a60b420b40 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6eeffff640 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8017dc0 tx=0x7f6ee80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5e5b28fbecb39a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6ed5ffb640 1 -- 172.21.15.77:0/1918786354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8017890 con 0x55a60b420b40 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6ed5ffb640 1 -- 172.21.15.77:0/1918786354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee800f8f0 con 0x55a60b420b40 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.159+0000 7f6ed5ffb640 1 -- 172.21.15.77:0/1918786354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8020740 con 0x55a60b420b40 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- 172.21.15.77:0/1918786354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 msgr2=0x55a60b41f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8017dc0 tx=0x7f6ee80117b0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6eeffff640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 0x7f6ed80a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8024000 tx=0x7f6ee8022000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- 172.21.15.77:0/1918786354 shutdown_connections 2024-09-15T22:20:14.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b41fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1918786354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b41f090 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- 172.21.15.77:0/1918786354 >> 172.21.15.77:0/1918786354 conn(0x55a60b3e8310 msgr2=0x55a60b41bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- 172.21.15.77:0/1918786354 shutdown_connections 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- 172.21.15.77:0/1918786354 wait complete. 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.160+0000 7f6ef8520740 1 -- start start 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 Processor -- start 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b426fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b427570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a60b42d070 con 0x55a60b41f5d0 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a60b340060 con 0x55a60b420b40 2024-09-15T22:20:14.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a60b42d1f0 con 0x55a60b42a370 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6ef8520740 10 client.0 _ll_register_callbacks cb 0x55a60b0938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44886/0 (socket says 172.21.15.77:44886) 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 -- 172.21.15.77:0/4135934410 learned_addr learned my addr 172.21.15.77:0/4135934410 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b426fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eeffff640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b427570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 -- 172.21.15.77:0/4135934410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 msgr2=0x55a60b427570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b427570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 -- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 msgr2=0x55a60b426fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b426fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eef7fe640 1 -- 172.21.15.77:0/4135934410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a60b0f63c0 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eeffff640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b427570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.161+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b426fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6ee002a930 tx=0x7f6ee0031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcbddc2cf6c46f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0035040 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f6ee0004240 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee800f8f0 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee004e050 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 10 client.17496 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a60b438ac0 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee800f8f0 con 0x55a60b41f5d0 2024-09-15T22:20:14.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6ee0004b80 con 0x55a60b41f5d0 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 10 client.17496 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 20 client.17496 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b439f80 0x55a60b43c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 4 client.17496 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.162+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a60b392860 con 0x55a60b439f80 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.163+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b439f80 0x55a60b43c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.163+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b439f80 0x55a60b43c370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=375d2fecb077e44 server_cookie=2623f3766bfaae2d in_seq=0 out_seq=0 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.163+0000 7f6eceffd640 10 client.17496 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6eceffd640 1 -- 172.21.15.77:0/4135934410 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a60b392860 con 0x55a60b439f80 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6eceffd640 10 client.17496 handle_command_reply: tid=0 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 client.17496 shutdown 2024-09-15T22:20:14.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 msgr2=0x55a60b429d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f6ee002a930 tx=0x7f6ee0031920 comp rx=0 tx=0).stop 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 shutdown_connections 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b439f80 0x55a60b43c370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b427570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b426fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 --2- 172.21.15.77:0/4135934410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b429d20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 >> 172.21.15.77:0/4135934410 conn(0x55a60b3e8310 msgr2=0x55a60b421930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6eceffd640 0 client.17496 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 shutdown_connections 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 1 -- 172.21.15.77:0/4135934410 wait complete. 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.164+0000 7f6ef8520740 20 client.17496 trim_cache size 0 max 16384 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 Processor -- start 2024-09-15T22:20:14.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 -- start start 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b428f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b422120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01acef0 con 0x55a60b41f5d0 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01aea30 con 0x55a60b420b40 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6ef8520740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01ad5c0 con 0x55a60b42a370 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44902/0 (socket says 172.21.15.77:44902) 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.177+0000 7f6eef7fe640 1 -- 172.21.15.77:0/860565585 learned_addr learned my addr 172.21.15.77:0/860565585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b428f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 -- 172.21.15.77:0/860565585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 msgr2=0x55a60b422120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b422120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 -- 172.21.15.77:0/860565585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 msgr2=0x55a60b428f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b428f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 -- 172.21.15.77:0/860565585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a60b425f80 con 0x55a60b41f5d0 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b428f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaf4c3d2ab041c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6ed57fa640 1 -- 172.21.15.77:0/860565585 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0000e60 con 0x55a60b41f5d0 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6ed57fa640 1 -- 172.21.15.77:0/860565585 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0004c30 con 0x55a60b41f5d0 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.178+0000 7f6ed57fa640 1 -- 172.21.15.77:0/860565585 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0035030 con 0x55a60b41f5d0 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- 172.21.15.77:0/860565585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 msgr2=0x55a60b4258a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee002a410 comp rx=0 tx=0).stop 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- 172.21.15.77:0/860565585 shutdown_connections 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b422120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b420b40 0x55a60b428f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 --2- 172.21.15.77:0/860565585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b41f5d0 0x55a60b4258a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- 172.21.15.77:0/860565585 >> 172.21.15.77:0/860565585 conn(0x55a60b3e8310 msgr2=0x55a60b428830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- 172.21.15.77:0/860565585 shutdown_connections 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- 172.21.15.77:0/860565585 wait complete. 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.179+0000 7f6ef8520740 1 -- start start 2024-09-15T22:20:14.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 Processor -- start 2024-09-15T22:20:14.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b42e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01ae360 con 0x55a60b420b40 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01adc90 con 0x55a60b41f5d0 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01aaef0 con 0x55a60b42a370 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6ef8520740 10 client.0 _ll_register_callbacks cb 0x7f6ee0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44908/0 (socket says 172.21.15.77:44908) 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 -- 172.21.15.77:0/1717278378 learned_addr learned my addr 172.21.15.77:0/1717278378 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b42e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeffff640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 -- 172.21.15.77:0/1717278378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 msgr2=0x55a60b42eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 -- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 msgr2=0x55a60b42e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b42e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eeeffd640 1 -- 172.21.15.77:0/1717278378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a60b4a3bf0 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.180+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b42e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eeeffd640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4097d20 tx=0x7f6ee409b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a075f2cabd587fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40b3070 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f6ee4097f30 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee0004c30 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee409fc70 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 10 client.17502 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a60b4a4e30 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0004c30 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6ee4084c50 con 0x55a60b420b40 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 10 client.17502 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:14.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 20 client.17502 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b4a5560 0x55a60b4a7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 4 client.17502 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.181+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a60b38cd70 con 0x55a60b4a5560 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b4a5560 0x55a60b4a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6eef7fe640 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b4a5560 0x55a60b4a7980 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2d6ce027d858872 server_cookie=734bdbc92567219c in_seq=0 out_seq=0 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6eceffd640 10 client.17502 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6eceffd640 1 -- 172.21.15.77:0/1717278378 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a60b38cd70 con 0x55a60b4a5560 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6eceffd640 10 client.17502 handle_command_reply: tid=0 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.182+0000 7f6ef8520740 1 client.17502 shutdown 2024-09-15T22:20:14.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 msgr2=0x55a60b42e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4097d20 tx=0x7f6ee409b7b0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 shutdown_connections 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55a60b4a5560 0x55a60b4a7980 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a60b42a370 0x55a60b42eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a60b420b40 0x55a60b42e660 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 --2- 172.21.15.77:0/1717278378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a60b41f5d0 0x55a60b42e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 >> 172.21.15.77:0/1717278378 conn(0x55a60b3e8310 msgr2=0x55a60b428830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6eceffd640 0 client.17502 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 shutdown_connections 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 1 -- 172.21.15.77:0/1717278378 wait complete. 2024-09-15T22:20:14.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef8520740 20 client.17502 trim_cache size 0 max 16384 2024-09-15T22:20:14.202 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:14.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 msgr2=0x7f6ed80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 0x7f6ed80a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8024000 tx=0x7f6ee8022000 comp rx=0 tx=0).stop 2024-09-15T22:20:14.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 msgr2=0x7f6ef0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.183+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d950 tx=0x7f6ee400de10 comp rx=0 tx=0).stop 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 shutdown_connections 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ed80a3430 0x7f6ed80a58f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef0106f00 0x7f6ef019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0102780 0x7f6ef0198960 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 --2- 172.21.15.77:0/4201296126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0101dd0 0x7f6ef0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 >> 172.21.15.77:0/4201296126 conn(0x7f6ef00fd8b0 msgr2=0x7f6ef00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 shutdown_connections 2024-09-15T22:20:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:14.184+0000 7f6ef6147640 1 -- 172.21.15.77:0/4201296126 wait complete. 2024-09-15T22:20:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:14 smithi077 ceph-mon[30980]: pgmap v240: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:14.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.22 MB/sec execute 263 sec latency 4806.687 ms 2024-09-15T22:20:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:14 smithi196 ceph-mon[35109]: pgmap v240: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:14 smithi117 ceph-mon[35021]: pgmap v240: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:14.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:20:15.174 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:15.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.19 MB/sec execute 264 sec latency 5806.717 ms 2024-09-15T22:20:15.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 -- 172.21.15.77:0/4225460397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40102570 msgr2=0x7f7c401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 --2- 172.21.15.77:0/4225460397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40102570 0x7f7c401029b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009900 tx=0x7f7c2c02a480 comp rx=0 tx=0).stop 2024-09-15T22:20:15.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 -- 172.21.15.77:0/4225460397 shutdown_connections 2024-09-15T22:20:15.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 --2- 172.21.15.77:0/4225460397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102ef0 0x7f7c4010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 --2- 172.21.15.77:0/4225460397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40102570 0x7f7c401029b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 --2- 172.21.15.77:0/4225460397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40100e20 0x7f7c40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 -- 172.21.15.77:0/4225460397 >> 172.21.15.77:0/4225460397 conn(0x7f7c400fc7d0 msgr2=0x7f7c400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 -- 172.21.15.77:0/4225460397 shutdown_connections 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.685+0000 7f7c46de2640 1 -- 172.21.15.77:0/4225460397 wait complete. 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 Processor -- start 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 -- start start 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 0x7f7c40197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 0x7f7c4019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40110f50 con 0x7f7c40100e20 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40110dd0 con 0x7f7c40102570 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c46de2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401110d0 con 0x7f7c40102ef0 2024-09-15T22:20:15.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44928/0 (socket says 172.21.15.77:44928) 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 -- 172.21.15.77:0/3549136429 learned_addr learned my addr 172.21.15.77:0/3549136429 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 0x7f7c40197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 0x7f7c4019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 -- 172.21.15.77:0/3549136429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 msgr2=0x7f7c4019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 0x7f7c4019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 -- 172.21.15.77:0/3549136429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 msgr2=0x7f7c40197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 0x7f7c40197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 -- 172.21.15.77:0/3549136429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4019c3e0 con 0x7f7c40100e20 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 0x7f7c4019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 0x7f7c40197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.686+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d950 tx=0x7f7c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.687+0000 7f7c3dffb640 1 -- 172.21.15.77:0/3549136429 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34017030 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.687+0000 7f7c3dffb640 1 -- 172.21.15.77:0/3549136429 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34004020 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.687+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4019c6d0 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.687+0000 7f7c3dffb640 1 -- 172.21.15.77:0/3549136429 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340047a0 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.687+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c401a3fb0 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.689+0000 7f7c3dffb640 1 -- 172.21.15.77:0/3549136429 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c340041d0 con 0x7f7c40100e20 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 Processor -- start 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 -- start start 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8dac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e83e2f0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e7e5cf0 con 0x55ab5e8da7e0 2024-09-15T22:20:15.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c491bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e83a2b0 con 0x55ab5e8e5580 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44930/0 (socket says 172.21.15.77:44930) 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c4537f640 1 -- 172.21.15.77:0/3542817293 learned_addr learned my addr 172.21.15.77:0/3542817293 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c3dffb640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 0x7f7c300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8dac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c4537f640 1 -- 172.21.15.77:0/3542817293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 msgr2=0x55ab5e8e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.690+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c4537f640 1 -- 172.21.15.77:0/3542817293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 msgr2=0x55ab5e8dac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8dac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c4537f640 1 -- 172.21.15.77:0/3542817293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5e7ca3a0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 0x7f7c300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c3dffb640 1 -- 172.21.15.77:0/3549136429 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f7c340cfbb0 con 0x7f7c40100e20 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800f3e0 tx=0x7f7c3800f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bf84818c2b38e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c2affd640 1 -- 172.21.15.77:0/3542817293 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801c400 con 0x55ab5e8dbd50 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c2affd640 1 -- 172.21.15.77:0/3542817293 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3801ca80 con 0x55ab5e8dbd50 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.691+0000 7f7c2affd640 1 -- 172.21.15.77:0/3542817293 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c380106d0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- 172.21.15.77:0/3542817293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 msgr2=0x55ab5e8da2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 0x7f7c300a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7c38020000 tx=0x7f7c3801d000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800f3e0 tx=0x7f7c3800f7e0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- 172.21.15.77:0/3542817293 shutdown_connections 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3542817293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8da2a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- 172.21.15.77:0/3542817293 >> 172.21.15.77:0/3542817293 conn(0x55ab5e8a1310 msgr2=0x55ab5e8d6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- 172.21.15.77:0/3542817293 shutdown_connections 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- 172.21.15.77:0/3542817293 wait complete. 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 -- start start 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.692+0000 7f7c491bb740 1 Processor -- start 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e8e8280 con 0x55ab5e8e5580 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e7f9060 con 0x55ab5e8dbd50 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab5e8e8400 con 0x55ab5e8da7e0 2024-09-15T22:20:15.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c491bb740 10 client.0 _ll_register_callbacks cb 0x55ab5e54c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c3ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44942/0 (socket says 172.21.15.77:44942) 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c44b7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 -- 172.21.15.77:0/2748712428 learned_addr learned my addr 172.21.15.77:0/2748712428 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 -- 172.21.15.77:0/2748712428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8da7e0 msgr2=0x55ab5e8e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 -- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8dbd50 msgr2=0x55ab5e8e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 -- 172.21.15.77:0/2748712428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5e5af3c0 con 0x55ab5e8e5580 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c4537f640 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7c3801cf10 tx=0x7f7c38009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52168485408b9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.693+0000 7f7c1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:15.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38033060 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3801ca80 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f7c380109e0 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c380106c0 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 10 client.17514 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ab5e8f3d20 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3801ca80 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c38003bd0 con 0x55ab5e8e5580 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 10 client.17514 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 20 client.17514 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e8f5130 0x55ab5e8f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 4 client.17514 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.694+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab5e84b860 con 0x55ab5e8f5130 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.695+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e8f5130 0x55ab5e8f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.695+0000 7f7c1ffff640 10 client.17514 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.695+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e8f5130 0x55ab5e8f7520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83440e52c502ad89 server_cookie=51a5d0e273d3a352 in_seq=0 out_seq=0 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c1ffff640 1 -- 172.21.15.77:0/2748712428 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab5e84b860 con 0x55ab5e8f5130 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c1ffff640 10 client.17514 handle_command_reply: tid=0 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 client.17514 shutdown 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 msgr2=0x55ab5e8e2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7c3801cf10 tx=0x7f7c38009aa0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 shutdown_connections 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e8f5130 0x55ab5e8f7520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8e2820 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c1ffff640 0 client.17514 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 --2- 172.21.15.77:0/2748712428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 >> 172.21.15.77:0/2748712428 conn(0x55ab5e8a1310 msgr2=0x55ab5e8dcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 shutdown_connections 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 1 -- 172.21.15.77:0/2748712428 wait complete. 2024-09-15T22:20:15.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.696+0000 7f7c491bb740 20 client.17514 trim_cache size 0 max 16384 2024-09-15T22:20:15.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 Processor -- start 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 -- start start 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8dd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401acfa0 con 0x55ab5e8e5580 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401aeae0 con 0x55ab5e8da7e0 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c491bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401ad670 con 0x55ab5e8dbd50 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44944/0 (socket says 172.21.15.77:44944) 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 -- 172.21.15.77:0/3250465836 learned_addr learned my addr 172.21.15.77:0/3250465836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8dd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 -- 172.21.15.77:0/3250465836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 msgr2=0x55ab5e8dd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8dd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 -- 172.21.15.77:0/3250465836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 msgr2=0x55ab5e8e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 -- 172.21.15.77:0/3250465836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5e8e0780 con 0x55ab5e8e5580 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8dd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.709+0000 7f7c4537f640 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7c38004310 tx=0x7f7c3800f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e80f90370bc651 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c2a7fc640 1 -- 172.21.15.77:0/3250465836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38044070 con 0x55ab5e8e5580 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c2a7fc640 1 -- 172.21.15.77:0/3250465836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3802a040 con 0x55ab5e8e5580 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c2a7fc640 1 -- 172.21.15.77:0/3250465836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38004840 con 0x55ab5e8e5580 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 -- 172.21.15.77:0/3250465836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 msgr2=0x55ab5e8dd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7c38004310 tx=0x7f7c3800f7e0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 -- 172.21.15.77:0/3250465836 shutdown_connections 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8e5580 0x55ab5e8dd870 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8dd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 --2- 172.21.15.77:0/3250465836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 -- 172.21.15.77:0/3250465836 >> 172.21.15.77:0/3250465836 conn(0x55ab5e8a1310 msgr2=0x55ab5e8e42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 -- 172.21.15.77:0/3250465836 shutdown_connections 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.710+0000 7f7c491bb740 1 -- 172.21.15.77:0/3250465836 wait complete. 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 -- start start 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 Processor -- start 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e95dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401ae410 con 0x55ab5e8dbd50 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401add40 con 0x55ab5e8da7e0 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401aafa0 con 0x55ab5e8e5580 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c491bb740 10 client.0 _ll_register_callbacks cb 0x7f7c3801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44948/0 (socket says 172.21.15.77:44948) 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c3ffff640 1 -- 172.21.15.77:0/59940709 learned_addr learned my addr 172.21.15.77:0/59940709 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c4537f640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e95dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.711+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 -- 172.21.15.77:0/59940709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 msgr2=0x55ab5e95dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e95dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 -- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 msgr2=0x55ab5e8e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:15.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 -- 172.21.15.77:0/59940709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5e95e1c0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c3ffff640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0314e0 tx=0x7f7c2c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9340a44da4c5897d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c035040 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f7c2c0048c0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3802a040 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c2c0055f0 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c491bb740 10 client.17520 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ab5e95f440 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3802a040 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.712+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c2c043020 con 0x55ab5e8dbd50 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 10 client.17520 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:15.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 20 client.17520 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e95fbf0 0x55ab5e962010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 4 client.17520 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab5e845d70 con 0x55ab5e95fbf0 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e95fbf0 0x55ab5e962010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c44b7e640 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e95fbf0 0x55ab5e962010 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fb562ea5c46dda server_cookie=9ef09da9123dbe13 in_seq=0 out_seq=0 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c1ffff640 10 client.17520 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c1ffff640 1 -- 172.21.15.77:0/59940709 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab5e845d70 con 0x55ab5e95fbf0 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c1ffff640 10 client.17520 handle_command_reply: tid=0 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.713+0000 7f7c491bb740 1 client.17520 shutdown 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 msgr2=0x55ab5e8e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0314e0 tx=0x7f7c2c005270 comp rx=0 tx=0).stop 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 shutdown_connections 2024-09-15T22:20:15.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ab5e95fbf0 0x55ab5e962010 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab5e8e5580 0x55ab5e95dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab5e8dbd50 0x55ab5e8e9180 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c1ffff640 0 client.17520 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 --2- 172.21.15.77:0/59940709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab5e8da7e0 0x55ab5e8e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 >> 172.21.15.77:0/59940709 conn(0x55ab5e8a1310 msgr2=0x55ab5e8d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 shutdown_connections 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 1 -- 172.21.15.77:0/59940709 wait complete. 2024-09-15T22:20:15.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c491bb740 20 client.17520 trim_cache size 0 max 16384 2024-09-15T22:20:15.729 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:15.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 msgr2=0x7f7c300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 0x7f7c300a58f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7c38020000 tx=0x7f7c3801d000 comp rx=0 tx=0).stop 2024-09-15T22:20:15.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 msgr2=0x7f7c40197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:15.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.714+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d950 tx=0x7f7c3400de10 comp rx=0 tx=0).stop 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 shutdown_connections 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c300a3430 0x7f7c300a58f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c40102ef0 0x7f7c4019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c40102570 0x7f7c40197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 --2- 172.21.15.77:0/3549136429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c40100e20 0x7f7c40197430 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 >> 172.21.15.77:0/3549136429 conn(0x7f7c400fc7d0 msgr2=0x7f7c400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 shutdown_connections 2024-09-15T22:20:15.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:15.715+0000 7f7c46de2640 1 -- 172.21.15.77:0/3549136429 wait complete. 2024-09-15T22:20:16.477 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.16 MB/sec execute 265 sec latency 6806.822 ms 2024-09-15T22:20:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:16 smithi077 ceph-mon[30980]: pgmap v241: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:16 smithi196 ceph-mon[35109]: pgmap v241: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:16 smithi117 ceph-mon[35021]: pgmap v241: 129 pgs: 129 active+clean; 438 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-09-15T22:20:17.475 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:20:17.475 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:20:17.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.13 MB/sec execute 266 sec latency 7806.867 ms 2024-09-15T22:20:17.718 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- 172.21.15.77:0/1855496553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 msgr2=0x7f84c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- 172.21.15.77:0/1855496553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c8100ee0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009930 tx=0x7f84bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- 172.21.15.77:0/1855496553 shutdown_connections 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- 172.21.15.77:0/1855496553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c8101420 0x7f84c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- 172.21.15.77:0/1855496553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c8100ee0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- 172.21.15.77:0/1855496553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8106a80 0x7f84c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- 172.21.15.77:0/1855496553 >> 172.21.15.77:0/1855496553 conn(0x7f84c80fc7a0 msgr2=0x7f84c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- 172.21.15.77:0/1855496553 shutdown_connections 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- 172.21.15.77:0/1855496553 wait complete. 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 Processor -- start 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- start start 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8101420 0x7f84c819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c8106a80 0x7f84c819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8110ef0 con 0x7f84c8100a80 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8110d70 con 0x7f84c8101420 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.191+0000 7f84d022f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8111070 con 0x7f84c8106a80 2024-09-15T22:20:18.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44964/0 (socket says 172.21.15.77:44964) 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 -- 172.21.15.77:0/392697540 learned_addr learned my addr 172.21.15.77:0/392697540 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 -- 172.21.15.77:0/392697540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c8106a80 msgr2=0x7f84c819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c8106a80 0x7f84c819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8101420 0x7f84c819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 -- 172.21.15.77:0/392697540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8101420 msgr2=0x7f84c819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8101420 0x7f84c819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 -- 172.21.15.77:0/392697540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c819f6d0 con 0x7f84c8100a80 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eab0 tx=0x7f84b800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84c6ffd640 1 -- 172.21.15.77:0/392697540 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800ce40 con 0x7f84c8100a80 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c819f9c0 con 0x7f84c8100a80 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84c6ffd640 1 -- 172.21.15.77:0/392697540 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8011950 con 0x7f84c8100a80 2024-09-15T22:20:18.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84c6ffd640 1 -- 172.21.15.77:0/392697540 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010ae0 con 0x7f84c8100a80 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.192+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c81a72a0 con 0x7f84c8100a80 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.194+0000 7f84c6ffd640 1 -- 172.21.15.77:0/392697540 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f84b8011b00 con 0x7f84c8100a80 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 Processor -- start 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 -- start start 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84c6ffd640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 0x7f84b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f990130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55704f8e8280 con 0x55704f982b60 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55704f88fb10 con 0x55704f98dd40 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84d2608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55704f8e4240 con 0x55704f983480 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 0x7f84b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84c6ffd640 1 -- 172.21.15.77:0/392697540 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f84b80d0900 con 0x7f84c8100a80 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44980/0 (socket says 172.21.15.77:44980) 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.195+0000 7f84cdfcb640 1 -- 172.21.15.77:0/366811273 learned_addr learned my addr 172.21.15.77:0/366811273 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f990130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 -- 172.21.15.77:0/366811273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 msgr2=0x55704f98d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 -- 172.21.15.77:0/366811273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 msgr2=0x55704f990130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f990130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 -- 172.21.15.77:0/366811273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704f89c1f0 con 0x55704f982b60 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f990130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c700 tx=0x7f84b8098490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe2b8c25417bda3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 0x7f84b00a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55704f980470 tx=0x7f84c00091c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84affff640 1 -- 172.21.15.77:0/366811273 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8079bd0 con 0x55704f982b60 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84affff640 1 -- 172.21.15.77:0/366811273 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b809cef0 con 0x55704f982b60 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.196+0000 7f84affff640 1 -- 172.21.15.77:0/366811273 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80799f0 con 0x55704f982b60 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- 172.21.15.77:0/366811273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 msgr2=0x55704f982f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c700 tx=0x7f84b8098490 comp rx=0 tx=0).stop 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- 172.21.15.77:0/366811273 shutdown_connections 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f990130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 --2- 172.21.15.77:0/366811273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f982f40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- 172.21.15.77:0/366811273 >> 172.21.15.77:0/366811273 conn(0x55704f97f0f0 msgr2=0x55704f97f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- 172.21.15.77:0/366811273 shutdown_connections 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- 172.21.15.77:0/366811273 wait complete. 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 -- start start 2024-09-15T22:20:18.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.197+0000 7f84d2608740 1 Processor -- start 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f982b60 0x55704f98abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55704f990910 con 0x55704f98dd40 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55704f8a2f10 con 0x55704f982b60 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55704f990a90 con 0x55704f983480 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84d2608740 10 client.0 _ll_register_callbacks cb 0x55704f86ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44982/0 (socket says 172.21.15.77:44982) 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 -- 172.21.15.77:0/3310261835 learned_addr learned my addr 172.21.15.77:0/3310261835 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f982b60 0x55704f98abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 -- 172.21.15.77:0/3310261835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 msgr2=0x55704f98b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 -- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f982b60 msgr2=0x55704f98abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f982b60 0x55704f98abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 -- 172.21.15.77:0/3310261835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704f6596a0 con 0x55704f98dd40 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f84c00194f0 tx=0x7f84c00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54be0b963edeba87 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.198+0000 7f84ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c000f040 con 0x55704f98dd40 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f84c0004020 con 0x55704f98dd40 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b809cef0 con 0x55704f98dd40 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84c002c050 con 0x55704f98dd40 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 10 client.17532 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:18.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55704f99c470 con 0x55704f98dd40 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b809cef0 con 0x55704f98dd40 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84c00037b0 con 0x55704f98dd40 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 10 client.17532 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 20 client.17532 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704f99d930 0x55704f99fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 4 client.17532 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55704f8f57f0 con 0x55704f99d930 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.199+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704f99d930 0x55704f99fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.200+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704f99d930 0x55704f99fd60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74fa6973f497e1c server_cookie=72a96c865783f03d in_seq=0 out_seq=0 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.200+0000 7f84ad7fa640 10 client.17532 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:18.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84ad7fa640 1 -- 172.21.15.77:0/3310261835 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55704f8f57f0 con 0x55704f99d930 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84ad7fa640 10 client.17532 handle_command_reply: tid=0 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 client.17532 shutdown 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 msgr2=0x55704f98b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f84c00194f0 tx=0x7f84c00047d0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 shutdown_connections 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704f99d930 0x55704f99fd60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f98b630 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 --2- 172.21.15.77:0/3310261835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f982b60 0x55704f98abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84ad7fa640 0 client.17532 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 >> 172.21.15.77:0/3310261835 conn(0x55704f97f0f0 msgr2=0x55704f984e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 shutdown_connections 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 1 -- 172.21.15.77:0/3310261835 wait complete. 2024-09-15T22:20:18.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.201+0000 7f84d2608740 20 client.17532 trim_cache size 0 max 16384 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 Processor -- start 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 -- start start 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f9a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81b1410 con 0x55704f982b60 2024-09-15T22:20:18.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81b2f50 con 0x55704f98dd40 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84d2608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81b1ae0 con 0x55704f983480 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44986/0 (socket says 172.21.15.77:44986) 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 -- 172.21.15.77:0/1732208437 learned_addr learned my addr 172.21.15.77:0/1732208437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 -- 172.21.15.77:0/1732208437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 msgr2=0x55704f98ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 -- 172.21.15.77:0/1732208437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 msgr2=0x55704f9a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f9a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.218+0000 7f84cdfcb640 1 -- 172.21.15.77:0/1732208437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704f98d480 con 0x55704f982b60 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ce260 tx=0x7f84b8010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f08787d8e4289b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84af7fe640 1 -- 172.21.15.77:0/1732208437 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b809a890 con 0x55704f982b60 2024-09-15T22:20:18.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84af7fe640 1 -- 172.21.15.77:0/1732208437 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b809aaf0 con 0x55704f982b60 2024-09-15T22:20:18.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84af7fe640 1 -- 172.21.15.77:0/1732208437 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80202b0 con 0x55704f982b60 2024-09-15T22:20:18.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 -- 172.21.15.77:0/1732208437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 msgr2=0x55704f9855c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ce260 tx=0x7f84b8010540 comp rx=0 tx=0).stop 2024-09-15T22:20:18.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 -- 172.21.15.77:0/1732208437 shutdown_connections 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f98dd40 0x55704f9a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f983480 0x55704f98ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 --2- 172.21.15.77:0/1732208437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f982b60 0x55704f9855c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 -- 172.21.15.77:0/1732208437 >> 172.21.15.77:0/1732208437 conn(0x55704f97f0f0 msgr2=0x55704f98c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 -- 172.21.15.77:0/1732208437 shutdown_connections 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.219+0000 7f84d2608740 1 -- 172.21.15.77:0/1732208437 wait complete. 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 -- start start 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 Processor -- start 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 0x55704f991a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 0x55704f991fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81b2880 con 0x55704f98dd40 2024-09-15T22:20:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81b21b0 con 0x55704f983480 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81af410 con 0x55704f982b60 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84d2608740 10 client.0 _ll_register_callbacks cb 0x7f84c000dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84ce7cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84ce7cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44990/0 (socket says 172.21.15.77:44990) 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84ce7cc640 1 -- 172.21.15.77:0/1844908917 learned_addr learned my addr 172.21.15.77:0/1844908917 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 0x55704f991a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.220+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 0x55704f991fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 -- 172.21.15.77:0/1844908917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 msgr2=0x55704f991a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 0x55704f991a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 -- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 msgr2=0x55704f991fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 0x55704f991fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 -- 172.21.15.77:0/1844908917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704fa076a0 con 0x55704f98dd40 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 0x55704f991a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84cd7ca640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 0x55704f991fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ce7cc640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f84c000d570 tx=0x7f84c0003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107a086db596cab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0004250 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9526+0+0 (secure 0 0 0) 0x7f84c00043f0 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b809aaf0 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84c0019d80 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84d2608740 10 client.17538 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55704fa089a0 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b809aaf0 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84c0006020 con 0x55704f98dd40 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84d2608740 10 client.17538 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.221+0000 7f84d2608740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84d2608740 20 client.17538 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704fa090e0 0x55704fa0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84d2608740 4 client.17538 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55704f8efd00 con 0x55704fa090e0 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704fa090e0 0x55704fa0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84ad7fa640 10 client.17538 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.222+0000 7f84cdfcb640 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704fa090e0 0x55704fa0b500 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a675ce8b66c1cec3 server_cookie=d31ba6912c3dc1f2 in_seq=0 out_seq=0 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.223+0000 7f84ad7fa640 1 -- 172.21.15.77:0/1844908917 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55704f8efd00 con 0x55704fa090e0 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.223+0000 7f84ad7fa640 10 client.17538 handle_command_reply: tid=0 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.223+0000 7f84d2608740 1 client.17538 shutdown 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.223+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 msgr2=0x55704f9924f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.223+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f84c000d570 tx=0x7f84c0003920 comp rx=0 tx=0).stop 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 shutdown_connections 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55704fa090e0 0x55704fa0b500 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704f98dd40 0x55704f9924f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55704f983480 0x55704f991fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 --2- 172.21.15.77:0/1844908917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55704f982b60 0x55704f991a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84ad7fa640 0 client.17538 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 >> 172.21.15.77:0/1844908917 conn(0x55704f97f0f0 msgr2=0x55704f97e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 shutdown_connections 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 1 -- 172.21.15.77:0/1844908917 wait complete. 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d2608740 20 client.17538 trim_cache size 0 max 16384 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "5f23ca9c-a1ef-4b13-9f4d-96dcdc85d990", 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:20:18.230 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:18.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 msgr2=0x7f84b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 0x7f84b00a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55704f980470 tx=0x7f84c00091c0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 msgr2=0x7f84c819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eab0 tx=0x7f84b800ef70 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 shutdown_connections 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84b00a33b0 0x7f84b00a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84c8106a80 0x7f84c819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84c8101420 0x7f84c819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 --2- 172.21.15.77:0/392697540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c8100a80 0x7f84c819a720 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.224+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 >> 172.21.15.77:0/392697540 conn(0x7f84c80fc7a0 msgr2=0x7f84c810c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.225+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 shutdown_connections 2024-09-15T22:20:18.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:18.225+0000 7f84d022f640 1 -- 172.21.15.77:0/392697540 wait complete. 2024-09-15T22:20:18.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.10 MB/sec execute 267 sec latency 8806.903 ms 2024-09-15T22:20:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:18 smithi077 ceph-mon[30980]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:18 smithi196 ceph-mon[35109]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:18 smithi117 ceph-mon[35021]: pgmap v242: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:19.143 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:20:19.411 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:19.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.07 MB/sec execute 268 sec latency 9807.011 ms 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:19 smithi077 ceph-mon[30980]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:20:19.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:19 smithi196 ceph-mon[35109]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:19.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:19 smithi117 ceph-mon[35021]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:20:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 -- 172.21.15.77:0/853847960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 msgr2=0x7fa47c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 --2- 172.21.15.77:0/853847960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c10d950 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa474009cd0 tx=0x7fa47402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 -- 172.21.15.77:0/853847960 shutdown_connections 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 --2- 172.21.15.77:0/853847960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c10d950 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 --2- 172.21.15.77:0/853847960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 0x7fa47c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 --2- 172.21.15.77:0/853847960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 0x7fa47c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 -- 172.21.15.77:0/853847960 >> 172.21.15.77:0/853847960 conn(0x7fa47c0fc7a0 msgr2=0x7fa47c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 -- 172.21.15.77:0/853847960 shutdown_connections 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.865+0000 7fa482ab7640 1 -- 172.21.15.77:0/853847960 wait complete. 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 Processor -- start 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 -- start start 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 0x7fa47c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 0x7fa47c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c111fa0 con 0x7fa47c101420 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c111e20 con 0x7fa47c106a80 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa482ab7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c112120 con 0x7fa47c100a80 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54976/0 (socket says 172.21.15.77:54976) 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa473fff640 1 -- 172.21.15.77:0/307831544 learned_addr learned my addr 172.21.15.77:0/307831544 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.866+0000 7fa480853640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 0x7fa47c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa481054640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 0x7fa47c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 -- 172.21.15.77:0/307831544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 msgr2=0x7fa47c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 0x7fa47c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 -- 172.21.15.77:0/307831544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 msgr2=0x7fa47c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 0x7fa47c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 -- 172.21.15.77:0/307831544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c19d3d0 con 0x7fa47c101420 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa473fff640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00d770 tx=0x7fa46c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa471ffb640 1 -- 172.21.15.77:0/307831544 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c017070 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c19d6c0 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa471ffb640 1 -- 172.21.15.77:0/307831544 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c005500 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa471ffb640 1 -- 172.21.15.77:0/307831544 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c004e00 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.867+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a4fa0 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.869+0000 7fa471ffb640 1 -- 172.21.15.77:0/307831544 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa46c004fa0 con 0x7fa47c101420 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.870+0000 7fa484e90740 1 Processor -- start 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.870+0000 7fa484e90740 1 -- start start 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.870+0000 7fa471ffb640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 0x7fa4640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.870+0000 7fa484e90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 0x561deffb6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.870+0000 7fa484e90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa484e90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa484e90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561deff1c2e0 con 0x561deffb6dc0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa484e90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561defec3ce0 con 0x561deffb64a0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa484e90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561deff182a0 con 0x561deffc1680 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa481054640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 0x7fa4640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa471ffb640 1 -- 172.21.15.77:0/307831544 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fa46c0cfb90 con 0x7fa47c101420 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54992/0 (socket says 172.21.15.77:54992) 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 -- 172.21.15.77:0/4207662392 learned_addr learned my addr 172.21.15.77:0/4207662392 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa480853640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 0x561deffb6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa481054640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 -- 172.21.15.77:0/4207662392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 msgr2=0x561deffc3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffc3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 -- 172.21.15.77:0/4207662392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 msgr2=0x561deffb6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 0x561deffb6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 -- 172.21.15.77:0/4207662392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561defea8440 con 0x561deffb6dc0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa480853640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 0x561deffb6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa481054640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa473fff640 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0ccf50 tx=0x7fa46c097e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583ac33b92e542cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa481054640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 0x7fa4640a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa474006010 tx=0x7fa474031450 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa462ffd640 1 -- 172.21.15.77:0/4207662392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c020070 con 0x561deffb6dc0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa462ffd640 1 -- 172.21.15.77:0/4207662392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c094c90 con 0x561deffb6dc0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.871+0000 7fa462ffd640 1 -- 172.21.15.77:0/4207662392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c009cc0 con 0x561deffb6dc0 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- 172.21.15.77:0/4207662392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 msgr2=0x561deffc1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0ccf50 tx=0x7fa46c097e50 comp rx=0 tx=0).stop 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- 172.21.15.77:0/4207662392 shutdown_connections 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb6dc0 0x561deffc1140 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 --2- 172.21.15.77:0/4207662392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb64a0 0x561deffb6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- 172.21.15.77:0/4207662392 >> 172.21.15.77:0/4207662392 conn(0x561deffb2a30 msgr2=0x561deffb2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- 172.21.15.77:0/4207662392 shutdown_connections 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- 172.21.15.77:0/4207662392 wait complete. 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.872+0000 7fa484e90740 1 -- start start 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 Processor -- start 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffbe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffbe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561deffc41f0 con 0x561deffb64a0 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561defed7050 con 0x561deffb6dc0 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561deffc4370 con 0x561deffc1680 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa484e90740 10 client.0 _ll_register_callbacks cb 0x561deff17160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:19.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54998/0 (socket says 172.21.15.77:54998) 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 -- 172.21.15.77:0/275453933 learned_addr learned my addr 172.21.15.77:0/275453933 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa473fff640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffbe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa481054640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffbe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 -- 172.21.15.77:0/275453933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 msgr2=0x561deffbe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffbe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 -- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 msgr2=0x561deffbe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffbe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa480853640 1 -- 172.21.15.77:0/275453933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561defc8d3c0 con 0x561deffb64a0 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.873+0000 7fa473fff640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa480853640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa47800b940 tx=0x7fa47800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d246168b25cf0d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478003ea0 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fa47801b640 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c094c90 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa47801b840 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 10 client.17550 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561deffcfb90 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c094c90 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa478019020 con 0x561deffb64a0 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 10 client.17550 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 20 client.17550 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561deffd1050 0x561deffd3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 4 client.17550 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.874+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561deff29850 con 0x561deffd1050 2024-09-15T22:20:19.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.875+0000 7fa473fff640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561deffd1050 0x561deffd3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.875+0000 7fa473fff640 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561deffd1050 0x561deffd3480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32f186ae51e94eb server_cookie=634e81e7612c6fe1 in_seq=0 out_seq=0 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.875+0000 7fa45bfff640 10 client.17550 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa45bfff640 1 -- 172.21.15.77:0/275453933 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561deff29850 con 0x561deffd1050 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa45bfff640 10 client.17550 handle_command_reply: tid=0 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 client.17550 shutdown 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 msgr2=0x561deffc0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa47800b940 tx=0x7fa47800be00 comp rx=0 tx=0).stop 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 shutdown_connections 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561deffd1050 0x561deffd3480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffc1680 0x561deffbe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 --2- 172.21.15.77:0/275453933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffc0fd0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa45bfff640 0 client.17550 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:19.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 >> 172.21.15.77:0/275453933 conn(0x561deffb2a30 msgr2=0x561deffb8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 shutdown_connections 2024-09-15T22:20:19.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 1 -- 172.21.15.77:0/275453933 wait complete. 2024-09-15T22:20:19.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.876+0000 7fa484e90740 20 client.17550 trim_cache size 0 max 16384 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 Processor -- start 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 -- start start 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 0x561deffd8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffc1680 0x561deffd9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1af080 con 0x561deffb64a0 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b0bc0 con 0x561deffc1680 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa484e90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1af750 con 0x561deffb6dc0 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa480853640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa480853640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55010/0 (socket says 172.21.15.77:55010) 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa480853640 1 -- 172.21.15.77:0/83212371 learned_addr learned my addr 172.21.15.77:0/83212371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.893+0000 7fa473fff640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 0x561deffd8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa481054640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffc1680 0x561deffd9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 -- 172.21.15.77:0/83212371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 msgr2=0x561deffd8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 0x561deffd8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 -- 172.21.15.77:0/83212371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffc1680 msgr2=0x561deffd9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffc1680 0x561deffd9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 -- 172.21.15.77:0/83212371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561deffbd230 con 0x561deffb64a0 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa473fff640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 0x561deffd8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa480853640 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa478011ca0 tx=0x7fa478003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab62d81b9f2b4770 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa4627fc640 1 -- 172.21.15.77:0/83212371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47801b440 con 0x561deffb64a0 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa4627fc640 1 -- 172.21.15.77:0/83212371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47802d070 con 0x561deffb64a0 2024-09-15T22:20:19.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.894+0000 7fa4627fc640 1 -- 172.21.15.77:0/83212371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478011440 con 0x561deffb64a0 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- 172.21.15.77:0/83212371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 msgr2=0x561deffcd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa478011ca0 tx=0x7fa478003ea0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- 172.21.15.77:0/83212371 shutdown_connections 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffc1680 0x561deffd9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb6dc0 0x561deffd8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 --2- 172.21.15.77:0/83212371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffb64a0 0x561deffcd450 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- 172.21.15.77:0/83212371 >> 172.21.15.77:0/83212371 conn(0x561deffb2a30 msgr2=0x561deffb9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- 172.21.15.77:0/83212371 shutdown_connections 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- 172.21.15.77:0/83212371 wait complete. 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 -- start start 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.895+0000 7fa484e90740 1 Processor -- start 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 0x561deffc5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1b04f0 con 0x561deffc1680 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1afe20 con 0x561deffb6dc0 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c1ad080 con 0x561deffb64a0 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 10 client.0 _ll_register_callbacks cb 0x7fa478004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55014/0 (socket says 172.21.15.77:55014) 2024-09-15T22:20:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 -- 172.21.15.77:0/3911841466 learned_addr learned my addr 172.21.15.77:0/3911841466 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa480853640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 0x561deffc5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa473fff640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 -- 172.21.15.77:0/3911841466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 msgr2=0x561deffc5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 0x561deffc5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 -- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 msgr2=0x561deffc5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffc5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 -- 172.21.15.77:0/3911841466 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df0043630 con 0x561deffc1680 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa480853640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 0x561deffc5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa473fff640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa481054640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa474009cd0 tx=0x7fa47402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a882f3aab88d68d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa45bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47404f070 con 0x561deffc1680 2024-09-15T22:20:19.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fa4740305a0 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.896+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47802d070 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa47405c050 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 10 client.17556 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561df0044920 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47802d070 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa474030070 con 0x561deffc1680 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 10 client.17556 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 20 client.17556 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561df00450a0 0x561df00474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 4 client.17556 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561deff23d60 con 0x561df00450a0 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.897+0000 7fa480853640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561df00450a0 0x561df00474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa480853640 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561df00450a0 0x561df00474c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806d8dc52f38d7a server_cookie=25c8e2b3b8140c71 in_seq=0 out_seq=0 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa45bfff640 10 client.17556 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa45bfff640 1 -- 172.21.15.77:0/3911841466 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561deff23d60 con 0x561df00450a0 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa45bfff640 10 client.17556 handle_command_reply: tid=0 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 client.17556 shutdown 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 msgr2=0x561deffc5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa474009cd0 tx=0x7fa47402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 shutdown_connections 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561df00450a0 0x561df00474c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561deffc1680 0x561deffc5ea0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa45bfff640 0 client.17556 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561deffb6dc0 0x561deffc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 --2- 172.21.15.77:0/3911841466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561deffb64a0 0x561deffc5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 >> 172.21.15.77:0/3911841466 conn(0x561deffb2a30 msgr2=0x561deffb1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 shutdown_connections 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 1 -- 172.21.15.77:0/3911841466 wait complete. 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.898+0000 7fa484e90740 20 client.17556 trim_cache size 0 max 16384 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "5f23ca9c-a1ef-4b13-9f4d-96dcdc85d990": { 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "5f23ca9c-a1ef-4b13-9f4d-96dcdc85d990", 2024-09-15T22:20:19.906 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 msgr2=0x7fa4640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 0x7fa4640a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa474006010 tx=0x7fa474031450 comp rx=0 tx=0).stop 2024-09-15T22:20:19.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 msgr2=0x7fa47c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00d770 tx=0x7fa46c00dc30 comp rx=0 tx=0).stop 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 shutdown_connections 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4640a33b0 0x7fa4640a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa47c106a80 0x7fa47c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa47c101420 0x7fa47c198960 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 --2- 172.21.15.77:0/307831544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa47c100a80 0x7fa47c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 >> 172.21.15.77:0/307831544 conn(0x7fa47c0fc7a0 msgr2=0x7fa47c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 shutdown_connections 2024-09-15T22:20:19.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:19.899+0000 7fa482ab7640 1 -- 172.21.15.77:0/307831544 wait complete. 2024-09-15T22:20:20.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.04 MB/sec execute 269 sec latency 10807.095 ms 2024-09-15T22:20:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:20 smithi077 ceph-mon[30980]: pgmap v243: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:20 smithi196 ceph-mon[35109]: pgmap v243: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:20 smithi117 ceph-mon[35021]: pgmap v243: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 139 op/s 2024-09-15T22:20:20.711 INFO:tasks.cephfs.filesystem:scrub status for tag:5f23ca9c-a1ef-4b13-9f4d-96dcdc85d990 - {'path': '/', 'tag': '5f23ca9c-a1ef-4b13-9f4d-96dcdc85d990', 'options': 'recursive,force'} 2024-09-15T22:20:21.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 8.01 MB/sec execute 270 sec latency 11807.188 ms 2024-09-15T22:20:22.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.98 MB/sec execute 271 sec latency 12807.314 ms 2024-09-15T22:20:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:22 smithi077 ceph-mon[30980]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-09-15T22:20:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:22 smithi196 ceph-mon[35109]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-09-15T22:20:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:22 smithi117 ceph-mon[35021]: pgmap v245: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-09-15T22:20:23.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.95 MB/sec execute 272 sec latency 13807.420 ms 2024-09-15T22:20:24.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.92 MB/sec execute 273 sec latency 14807.507 ms 2024-09-15T22:20:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:24 smithi077 ceph-mon[30980]: pgmap v246: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:24 smithi196 ceph-mon[35109]: pgmap v246: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:24 smithi117 ceph-mon[35021]: pgmap v246: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:25.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.89 MB/sec execute 274 sec latency 15807.580 ms 2024-09-15T22:20:26.478 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.86 MB/sec execute 275 sec latency 16807.662 ms 2024-09-15T22:20:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:26 smithi077 ceph-mon[30980]: pgmap v247: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:26 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:20:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:26 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:20:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:26 smithi196 ceph-mon[35109]: pgmap v247: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:26 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:20:26.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:26 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:20:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:26 smithi117 ceph-mon[35021]: pgmap v247: 129 pgs: 129 active+clean; 441 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 2.2 MiB/s wr, 71 op/s 2024-09-15T22:20:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:26 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:20:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:26 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:20:27.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.84 MB/sec execute 276 sec latency 17807.733 ms 2024-09-15T22:20:28.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.81 MB/sec execute 277 sec latency 18807.829 ms 2024-09-15T22:20:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:28 smithi077 ceph-mon[30980]: pgmap v248: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:28 smithi196 ceph-mon[35109]: pgmap v248: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:28 smithi117 ceph-mon[35021]: pgmap v248: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:29.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.78 MB/sec execute 278 sec latency 19807.938 ms 2024-09-15T22:20:29.502 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:29.747 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- 172.21.15.77:0/4226869561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950101420 msgr2=0x7fe95010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 --2- 172.21.15.77:0/4226869561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950101420 0x7fe95010d950 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe948009cd0 tx=0x7fe94802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- 172.21.15.77:0/4226869561 shutdown_connections 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 --2- 172.21.15.77:0/4226869561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950101420 0x7fe95010d950 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 --2- 172.21.15.77:0/4226869561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950100a80 0x7fe950100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 --2- 172.21.15.77:0/4226869561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950106a80 0x7fe950106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- 172.21.15.77:0/4226869561 >> 172.21.15.77:0/4226869561 conn(0x7fe9500fc7a0 msgr2=0x7fe9500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- 172.21.15.77:0/4226869561 shutdown_connections 2024-09-15T22:20:30.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- 172.21.15.77:0/4226869561 wait complete. 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 Processor -- start 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.208+0000 7fe955ccd640 1 -- start start 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 0x7fe950198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950101420 0x7fe9501988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe950111fa0 con 0x7fe950106a80 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe950111e20 con 0x7fe950101420 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe955ccd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe950112120 con 0x7fe950100a80 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57172/0 (socket says 172.21.15.77:57172) 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 -- 172.21.15.77:0/3110672566 learned_addr learned my addr 172.21.15.77:0/3110672566 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94f7fe640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 0x7fe950198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe946ffd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950101420 0x7fe9501988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:30.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 -- 172.21.15.77:0/3110672566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 msgr2=0x7fe950198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 0x7fe950198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 -- 172.21.15.77:0/3110672566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950101420 msgr2=0x7fe9501988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950101420 0x7fe9501988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 -- 172.21.15.77:0/3110672566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95019d340 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94f7fe640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 0x7fe950198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94ffff640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe948005cc0 tx=0x7fe948003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.209+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948004040 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.210+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95019d5d0 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.210+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9480046c0 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.210+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948046480 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.210+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501a4f10 con 0x7fe950106a80 2024-09-15T22:20:30.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.211+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950100ee0 con 0x7fe950106a80 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.212+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe9480305a0 con 0x7fe950106a80 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.212+0000 7fe94d7fa640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 0x7fe93c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.213+0000 7fe94f7fe640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 0x7fe93c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.213+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fe94802e030 con 0x7fe950106a80 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.213+0000 7fe94f7fe640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 0x7fe93c0a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe940005fd0 tx=0x7fe94000b5d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:30.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.216+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe94804b060 con 0x7fe950106a80 2024-09-15T22:20:30.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.418+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9501a50e0 con 0x7fe950106a80 2024-09-15T22:20:30.420 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:30 smithi077 ceph-mon[30980]: pgmap v249: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:30.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.419+0000 7fe94d7fa640 1 -- 172.21.15.77:0/3110672566 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fe9480be010 con 0x7fe950106a80 2024-09-15T22:20:30.421 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:20:30.421 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:20:30.421 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:20:30.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 msgr2=0x7fe93c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:30.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 0x7fe93c0a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe940005fd0 tx=0x7fe94000b5d0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 msgr2=0x7fe95019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe948005cc0 tx=0x7fe948003970 comp rx=0 tx=0).stop 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 shutdown_connections 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe93c0a33b0 0x7fe93c0a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe950106a80 0x7fe95019cc60 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe950101420 0x7fe9501988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 --2- 172.21.15.77:0/3110672566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe950100a80 0x7fe950198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 >> 172.21.15.77:0/3110672566 conn(0x7fe9500fc7a0 msgr2=0x7fe9500fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 shutdown_connections 2024-09-15T22:20:30.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:30.422+0000 7fe955ccd640 1 -- 172.21.15.77:0/3110672566 wait complete. 2024-09-15T22:20:30.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.75 MB/sec execute 279 sec latency 20807.989 ms 2024-09-15T22:20:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:30 smithi196 ceph-mon[35109]: pgmap v249: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:30 smithi117 ceph-mon[35021]: pgmap v249: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 82 op/s 2024-09-15T22:20:31.204 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id fc7daac0 --release --await' 2024-09-15T22:20:31.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id fc7daac0 --release --await 2024-09-15T22:20:31.452 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:31.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611130 7.72 MB/sec execute 280 sec latency 21808.032 ms 2024-09-15T22:20:31.572 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:31 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3110672566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:31 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3110672566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:31 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3110672566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:31.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 -- 172.21.15.77:0/436158720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674101420 msgr2=0x7f667410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 --2- 172.21.15.77:0/436158720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674101420 0x7f667410d950 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f666c009cd0 tx=0x7f666c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 -- 172.21.15.77:0/436158720 shutdown_connections 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 --2- 172.21.15.77:0/436158720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674101420 0x7f667410d950 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 --2- 172.21.15.77:0/436158720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674100a80 0x7f6674100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 --2- 172.21.15.77:0/436158720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 0x7f6674106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 -- 172.21.15.77:0/436158720 >> 172.21.15.77:0/436158720 conn(0x7f66740fc7a0 msgr2=0x7f66740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 -- 172.21.15.77:0/436158720 shutdown_connections 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.943+0000 7f66795df640 1 -- 172.21.15.77:0/436158720 wait complete. 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 Processor -- start 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 -- start start 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 0x7f66741977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 0x7f667419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6674110e80 con 0x7f6674100a80 2024-09-15T22:20:31.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6674110d00 con 0x7f6674101420 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f66795df640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6674111000 con 0x7f6674106a80 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57198/0 (socket says 172.21.15.77:57198) 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.944+0000 7f6672ffd640 1 -- 172.21.15.77:0/855059193 learned_addr learned my addr 172.21.15.77:0/855059193 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f66737fe640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 0x7f667419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f666a7fc640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 0x7f66741977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 -- 172.21.15.77:0/855059193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 msgr2=0x7f667419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 0x7f667419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 -- 172.21.15.77:0/855059193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 msgr2=0x7f66741977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 0x7f66741977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 -- 172.21.15.77:0/855059193 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667419c260 con 0x7f6674100a80 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f666a7fc640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 0x7f66741977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6672ffd640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f666400d770 tx=0x7f666400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6670ff9640 1 -- 172.21.15.77:0/855059193 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664017070 con 0x7f6674100a80 2024-09-15T22:20:31.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6670ff9640 1 -- 172.21.15.77:0/855059193 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664005500 con 0x7f6674100a80 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667419c550 con 0x7f6674100a80 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f6670ff9640 1 -- 172.21.15.77:0/855059193 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664004e00 con 0x7f6674100a80 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.945+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741a3e30 con 0x7f6674100a80 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.947+0000 7f6670ff9640 1 -- 172.21.15.77:0/855059193 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6664004fa0 con 0x7f6674100a80 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 Processor -- start 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 -- start start 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea1856240 0x55bea1858630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bea17b0620 con 0x55bea184b980 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bea1757e50 con 0x55bea1856240 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f667b9b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bea17ac5e0 con 0x55bea184b060 2024-09-15T22:20:31.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f6670ff9640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57208/0 (socket says 172.21.15.77:57208) 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f6672ffd640 1 -- 172.21.15.77:0/1189342691 learned_addr learned my addr 172.21.15.77:0/1189342691 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f66737fe640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f666a7fc640 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea1856240 0x55bea1858630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f66737fe640 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.948+0000 7f6670ff9640 1 -- 172.21.15.77:0/855059193 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f66640cfb80 con 0x7f6674100a80 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 -- 172.21.15.77:0/1189342691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 msgr2=0x55bea184b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 -- 172.21.15.77:0/1189342691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea1856240 msgr2=0x55bea1858630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea1856240 0x55bea1858630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 -- 172.21.15.77:0/1189342691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea1748d40 con 0x55bea184b980 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f666400d740 tx=0x7f66640048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4ea9db65d2758 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f66737fe640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 0x7f66600a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f666c009a60 tx=0x7f666c032f40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f66697fa640 1 -- 172.21.15.77:0/1189342691 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664020030 con 0x55bea184b980 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f66697fa640 1 -- 172.21.15.77:0/1189342691 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666409bad0 con 0x55bea184b980 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.949+0000 7f66697fa640 1 -- 172.21.15.77:0/1189342691 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664005360 con 0x55bea184b980 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- 172.21.15.77:0/1189342691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 msgr2=0x55bea1855d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f666400d740 tx=0x7f66640048a0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- 172.21.15.77:0/1189342691 shutdown_connections 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea1856240 0x55bea1858630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b980 0x55bea1855d00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1189342691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- 172.21.15.77:0/1189342691 >> 172.21.15.77:0/1189342691 conn(0x55bea18475f0 msgr2=0x55bea18479d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- 172.21.15.77:0/1189342691 shutdown_connections 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- 172.21.15.77:0/1189342691 wait complete. 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 -- start start 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 Processor -- start 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea18531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.950+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1853730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bea1858e10 con 0x55bea1856240 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bea1738f10 con 0x55bea184b980 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bea1858f90 con 0x55bea184b060 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 10 client.0 _ll_register_callbacks cb 0x55bea175cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57218/0 (socket says 172.21.15.77:57218) 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 -- 172.21.15.77:0/1979344921 learned_addr learned my addr 172.21.15.77:0/1979344921 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea18531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f666a7fc640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1853730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 -- 172.21.15.77:0/1979344921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 msgr2=0x55bea18531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea18531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 -- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 msgr2=0x55bea1853730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1853730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 -- 172.21.15.77:0/1979344921 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea14bf580 con 0x55bea1856240 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f66737fe640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f666c003ba0 tx=0x7f666c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753059f49e03dea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f665a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c051070 con 0x55bea1856240 2024-09-15T22:20:31.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f666c004640 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666409bad0 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f666c004840 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 10 client.17574 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.951+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bea1589960 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666409bad0 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f666c030070 con 0x55bea1856240 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 10 client.17574 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 10 client.17574 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 20 client.17574 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea1865e10 0x55bea1868290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 4 client.17574 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea17bdb90 con 0x55bea1865e10 2024-09-15T22:20:31.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.952+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea1865e10 0x55bea1868290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f6672ffd640 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea1865e10 0x55bea1868290 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=effce68ec9199aa4 server_cookie=9b2dd33d4e59f2ff in_seq=0 out_seq=0 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f665a7fc640 10 client.17574 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f665a7fc640 1 -- 172.21.15.77:0/1979344921 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bea17bdb90 con 0x55bea1865e10 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f665a7fc640 10 client.17574 handle_command_reply: tid=0 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 client.17574 shutdown 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 msgr2=0x55bea1853c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f666c003ba0 tx=0x7f666c045000 comp rx=0 tx=0).stop 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 shutdown_connections 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea1865e10 0x55bea1868290 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea1853c70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1853730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f665a7fc640 0 client.17574 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:31.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.953+0000 7f667b9b8740 1 --2- 172.21.15.77:0/1979344921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea18531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.954+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 >> 172.21.15.77:0/1979344921 conn(0x55bea18475f0 msgr2=0x55bea184d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.954+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 shutdown_connections 2024-09-15T22:20:31.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.954+0000 7f667b9b8740 1 -- 172.21.15.77:0/1979344921 wait complete. 2024-09-15T22:20:31.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.954+0000 7f667b9b8740 20 client.17574 trim_cache size 0 max 16384 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.970+0000 7f667b9b8740 1 Processor -- start 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.970+0000 7f667b9b8740 1 -- start start 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1855160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ace30 con 0x55bea1856240 2024-09-15T22:20:31.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ae970 con 0x55bea184b980 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f667b9b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ad500 con 0x55bea184b060 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57232/0 (socket says 172.21.15.77:57232) 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 -- 172.21.15.77:0/2128539041 learned_addr learned my addr 172.21.15.77:0/2128539041 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1855160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 -- 172.21.15.77:0/2128539041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 msgr2=0x55bea184e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 -- 172.21.15.77:0/2128539041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 msgr2=0x55bea1855160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1855160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 -- 172.21.15.77:0/2128539041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea1853890 con 0x55bea1856240 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1855160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.971+0000 7f66737fe640 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca4202a03e78ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f6668ff9640 1 -- 172.21.15.77:0/2128539041 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c051040 con 0x55bea1856240 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f6668ff9640 1 -- 172.21.15.77:0/2128539041 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666c036040 con 0x55bea1856240 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f6668ff9640 1 -- 172.21.15.77:0/2128539041 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c030800 con 0x55bea1856240 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 -- 172.21.15.77:0/2128539041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 msgr2=0x55bea186db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c0317c0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 -- 172.21.15.77:0/2128539041 shutdown_connections 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea1856240 0x55bea186db60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea1855160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2128539041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea184b060 0x55bea184e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.972+0000 7f667b9b8740 1 -- 172.21.15.77:0/2128539041 >> 172.21.15.77:0/2128539041 conn(0x55bea18475f0 msgr2=0x55bea1854a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- 172.21.15.77:0/2128539041 shutdown_connections 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- 172.21.15.77:0/2128539041 wait complete. 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- start start 2024-09-15T22:20:31.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 Processor -- start 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea185a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea1856240 0x55bea185ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66741ae2a0 con 0x55bea184b060 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66741adbd0 con 0x55bea184b980 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66741aae30 con 0x55bea1856240 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f667b9b8740 10 client.0 _ll_register_callbacks cb 0x7f666c031d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57236/0 (socket says 172.21.15.77:57236) 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.973+0000 7f6672ffd640 1 -- 172.21.15.77:0/2793956830 learned_addr learned my addr 172.21.15.77:0/2793956830 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f66737fe640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea1856240 0x55bea185ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea185a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 -- 172.21.15.77:0/2793956830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea1856240 msgr2=0x55bea185ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea1856240 0x55bea185ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 -- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 msgr2=0x55bea185a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea185a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 -- 172.21.15.77:0/2793956830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea185b3a0 con 0x55bea184b060 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea185a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f6672ffd640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f666409b7d0 tx=0x7f6664009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9d7f41dda9439 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664020040 con 0x55bea184b060 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f6664005290 con 0x55bea184b060 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666c036040 con 0x55bea184b060 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f666409bad0 con 0x55bea184b060 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 10 client.17580 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:31.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bea18d1830 con 0x55bea184b060 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666c036040 con 0x55bea184b060 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f66640cd1c0 con 0x55bea184b060 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 10 client.17580 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 10 client.17580 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 20 client.17580 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.974+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea18d1b70 0x55bea18d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.975+0000 7f667b9b8740 4 client.17580 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fc7daac0", "release": true, "await": true}] 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.975+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fc7daac0", "release": true, "await": true}) -- 0x55bea1854000 con 0x55bea18d1b70 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.975+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea18d1b70 0x55bea18d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.975+0000 7f665a7fc640 10 client.17580 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:31.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.975+0000 7f666a7fc640 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea18d1b70 0x55bea18d3f60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7517de938fcd917d server_cookie=c76a2faf7b1bf445 in_seq=0 out_seq=0 2024-09-15T22:20:31.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.988+0000 7f665a7fc640 1 -- 172.21.15.77:0/2793956830 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bea1854000 con 0x55bea18d1b70 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.988+0000 7f665a7fc640 10 client.17580 handle_command_reply: tid=0 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.988+0000 7f667b9b8740 1 client.17580 shutdown 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 msgr2=0x55bea185e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f666409b7d0 tx=0x7f6664009740 comp rx=0 tx=0).stop 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 shutdown_connections 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55bea18d1b70 0x55bea18d3f60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bea1856240 0x55bea185ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea184b980 0x55bea185a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 --2- 172.21.15.77:0/2793956830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bea184b060 0x55bea185e5e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 >> 172.21.15.77:0/2793956830 conn(0x55bea18475f0 msgr2=0x55bea1846750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f665a7fc640 0 client.17580 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:31.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 shutdown_connections 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 1 -- 172.21.15.77:0/2793956830 wait complete. 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f667b9b8740 20 client.17580 trim_cache size 0 max 16384 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 28, 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "fc7daac0": { 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 28, 2024-09-15T22:20:31.992 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 143.7, 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:20:31.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:31.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 msgr2=0x7f66600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 0x7f66600a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f666c009a60 tx=0x7f666c032f40 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 msgr2=0x7f66741972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.989+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f666400d770 tx=0x7f666400dc30 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 shutdown_connections 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66600a33b0 0x7f66600a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6674106a80 0x7f667419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6674101420 0x7f66741977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 --2- 172.21.15.77:0/855059193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6674100a80 0x7f66741972b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 >> 172.21.15.77:0/855059193 conn(0x7f66740fc7a0 msgr2=0x7f66740fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 shutdown_connections 2024-09-15T22:20:31.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:31.990+0000 7f66795df640 1 -- 172.21.15.77:0/855059193 wait complete. 2024-09-15T22:20:32.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 611684 7.70 MB/sec execute 281 sec latency 22322.932 ms 2024-09-15T22:20:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:32 smithi077 ceph-mon[30980]: pgmap v250: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-09-15T22:20:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:32 smithi196 ceph-mon[35109]: pgmap v250: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-09-15T22:20:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:32 smithi117 ceph-mon[35021]: pgmap v250: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-09-15T22:20:32.934 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fc7daac0', total seconds elapsed: 27.2 2024-09-15T22:20:33.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 614403 7.73 MB/sec execute 282 sec latency 13.096 ms 2024-09-15T22:20:34.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 615822 7.73 MB/sec execute 283 sec latency 34.859 ms 2024-09-15T22:20:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:34 smithi077 ceph-mon[30980]: pgmap v251: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:34 smithi196 ceph-mon[35109]: pgmap v251: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:34 smithi117 ceph-mon[35021]: pgmap v251: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:35.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 618308 7.75 MB/sec execute 284 sec latency 16.258 ms 2024-09-15T22:20:36.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 620381 7.75 MB/sec execute 285 sec latency 22.083 ms 2024-09-15T22:20:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:36 smithi196 ceph-mon[35109]: pgmap v252: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:36 smithi117 ceph-mon[35021]: pgmap v252: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:36 smithi077 ceph-mon[30980]: pgmap v252: 129 pgs: 129 active+clean; 444 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-09-15T22:20:37.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 622354 7.76 MB/sec execute 286 sec latency 15.048 ms 2024-09-15T22:20:38.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 625235 7.80 MB/sec execute 287 sec latency 12.408 ms 2024-09-15T22:20:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:38 smithi196 ceph-mon[35109]: pgmap v253: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-09-15T22:20:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:38 smithi117 ceph-mon[35021]: pgmap v253: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-09-15T22:20:38.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:38 smithi077 ceph-mon[30980]: pgmap v253: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-09-15T22:20:39.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 626635 7.79 MB/sec execute 288 sec latency 52.863 ms 2024-09-15T22:20:40.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 629084 7.81 MB/sec execute 289 sec latency 18.294 ms 2024-09-15T22:20:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:40 smithi196 ceph-mon[35109]: pgmap v254: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-09-15T22:20:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:40 smithi117 ceph-mon[35021]: pgmap v254: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-09-15T22:20:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:40 smithi077 ceph-mon[30980]: pgmap v254: 129 pgs: 129 active+clean; 413 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 116 op/s 2024-09-15T22:20:41.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 631518 7.83 MB/sec execute 290 sec latency 21.254 ms 2024-09-15T22:20:41.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:41 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:41.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:41 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:41.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:41 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:41.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:41 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:20:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:41 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:41 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:41 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:41 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:20:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:41 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:41 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:41 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:41 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:20:42.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 633424 7.83 MB/sec execute 291 sec latency 16.710 ms 2024-09-15T22:20:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:42 smithi196 ceph-mon[35109]: pgmap v255: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-09-15T22:20:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:42 smithi117 ceph-mon[35021]: pgmap v255: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-09-15T22:20:42.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:42 smithi077 ceph-mon[30980]: pgmap v255: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 218 op/s 2024-09-15T22:20:43.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 636007 7.86 MB/sec execute 292 sec latency 17.361 ms 2024-09-15T22:20:44.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 637910 7.86 MB/sec execute 293 sec latency 34.905 ms 2024-09-15T22:20:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:44 smithi196 ceph-mon[35109]: pgmap v256: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:44 smithi117 ceph-mon[35021]: pgmap v256: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:44.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:44 smithi077 ceph-mon[30980]: pgmap v256: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:45.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 639923 7.88 MB/sec execute 294 sec latency 86.010 ms 2024-09-15T22:20:46.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 641658 7.87 MB/sec execute 295 sec latency 20.093 ms 2024-09-15T22:20:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:46 smithi117 ceph-mon[35021]: pgmap v257: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:46 smithi077 ceph-mon[30980]: pgmap v257: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:46 smithi196 ceph-mon[35109]: pgmap v257: 129 pgs: 129 active+clean; 424 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-09-15T22:20:47.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 643858 7.89 MB/sec execute 296 sec latency 13.541 ms 2024-09-15T22:20:48.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 646557 7.93 MB/sec execute 297 sec latency 17.239 ms 2024-09-15T22:20:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:48 smithi117 ceph-mon[35021]: pgmap v258: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-15T22:20:48.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:48 smithi077 ceph-mon[30980]: pgmap v258: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-15T22:20:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:48 smithi196 ceph-mon[35109]: pgmap v258: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-15T22:20:49.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 648455 7.92 MB/sec execute 298 sec latency 24.302 ms 2024-09-15T22:20:50.479 INFO:tasks.workunit.client.0.smithi077.stdout: 1 650589 7.94 MB/sec execute 299 sec latency 15.385 ms 2024-09-15T22:20:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:50 smithi117 ceph-mon[35021]: pgmap v259: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-09-15T22:20:50.712 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:20:50.736 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:50 smithi077 ceph-mon[30980]: pgmap v259: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-09-15T22:20:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:50 smithi196 ceph-mon[35109]: pgmap v259: 129 pgs: 129 active+clean; 435 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-09-15T22:20:50.979 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:51.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 653066 7.95 MB/sec execute 300 sec latency 61.832 ms 2024-09-15T22:20:51.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- 172.21.15.77:0/3373097485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081082a0 msgr2=0x7fc50810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/3373097485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081082a0 0x7fc50810ec40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc500009cd0 tx=0x7fc50002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- 172.21.15.77:0/3373097485 shutdown_connections 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/3373097485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081082a0 0x7fc50810ec40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/3373097485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5081036e0 0x7fc508107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/3373097485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc508102d30 0x7fc508103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- 172.21.15.77:0/3373097485 >> 172.21.15.77:0/3373097485 conn(0x7fc5080fc7d0 msgr2=0x7fc5080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- 172.21.15.77:0/3373097485 shutdown_connections 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- 172.21.15.77:0/3373097485 wait complete. 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 Processor -- start 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.500+0000 7fc50e4a8640 1 -- start start 2024-09-15T22:20:51.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 0x7fc508100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5081082a0 0x7fc508101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081120a0 con 0x7fc5081036e0 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111f20 con 0x7fc5081082a0 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc508112220 con 0x7fc508102d30 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33224/0 (socket says 172.21.15.77:33224) 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 -- 172.21.15.77:0/891311640 learned_addr learned my addr 172.21.15.77:0/891311640 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc507fff640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 0x7fc508100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50ca45640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5081082a0 0x7fc508101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 -- 172.21.15.77:0/891311640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 msgr2=0x7fc508100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 0x7fc508100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 -- 172.21.15.77:0/891311640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5081082a0 msgr2=0x7fc508101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5081082a0 0x7fc508101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 -- 172.21.15.77:0/891311640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081a6a20 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc507fff640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 0x7fc508100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc4fffff640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9c0 tx=0x7fc4f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc505ffb640 1 -- 172.21.15.77:0/891311640 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000ccc0 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc505ffb640 1 -- 172.21.15.77:0/891311640 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0005500 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081a6d10 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc505ffb640 1 -- 172.21.15.77:0/891311640 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0010690 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.501+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a71a0 con 0x7fc5081036e0 2024-09-15T22:20:51.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.504+0000 7fc505ffb640 1 -- 172.21.15.77:0/891311640 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc4f0004e80 con 0x7fc5081036e0 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.504+0000 7fc510881740 1 Processor -- start 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.504+0000 7fc510881740 1 -- start start 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a39a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c399922e0 con 0x563c39a2cdc0 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c39939ce0 con 0x563c39a37680 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc510881740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c3998e2a0 con 0x563c39a2c4a0 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc50ca45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33230/0 (socket says 172.21.15.77:33230) 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 -- 172.21.15.77:0/1149282645 learned_addr learned my addr 172.21.15.77:0/1149282645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc50ca45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37230/0 (socket says 172.21.15.77:37230) 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a39a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc505ffb640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 0x7fc4f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 -- 172.21.15.77:0/1149282645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 msgr2=0x563c39a2c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 -- 172.21.15.77:0/1149282645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 msgr2=0x563c39a39a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a39a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 -- 172.21.15.77:0/1149282645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3991e440 con 0x563c39a2cdc0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc50ca45640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 0x7fc4f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc507fff640 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80115f0 tx=0x7fc4f8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f2d238f7c44ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.506+0000 7fc4fe7fc640 1 -- 172.21.15.77:0/1149282645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f801e070 con 0x563c39a2cdc0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.506+0000 7fc4fe7fc640 1 -- 172.21.15.77:0/1149282645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f800fb20 con 0x563c39a2cdc0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.506+0000 7fc4fe7fc640 1 -- 172.21.15.77:0/1149282645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8019c40 con 0x563c39a2cdc0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.505+0000 7fc505ffb640 1 -- 172.21.15.77:0/891311640 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fc4f00d0500 con 0x7fc5081036e0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 -- 172.21.15.77:0/1149282645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 msgr2=0x563c39a37140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc50ca45640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 0x7fc4f40a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc500005e30 tx=0x7fc50003f040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80115f0 tx=0x7fc4f8011ab0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 -- 172.21.15.77:0/1149282645 shutdown_connections 2024-09-15T22:20:51.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a39a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2cdc0 0x563c39a37140 secure :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80115f0 tx=0x7fc4f8011ab0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 --2- 172.21.15.77:0/1149282645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a2c4a0 0x563c39a2c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.507+0000 7fc510881740 1 -- 172.21.15.77:0/1149282645 >> 172.21.15.77:0/1149282645 conn(0x563c39a28a30 msgr2=0x563c39a28e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.508+0000 7fc510881740 1 -- 172.21.15.77:0/1149282645 shutdown_connections 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.508+0000 7fc510881740 1 -- 172.21.15.77:0/1149282645 wait complete. 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.508+0000 7fc510881740 1 -- start start 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 Processor -- start 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a34470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c39a3a250 con 0x563c39a37680 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563c3994d050 con 0x563c39a2c4a0 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c39a3a3d0 con 0x563c39a34ef0 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc510881740 10 client.0 _ll_register_callbacks cb 0x563c3998d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a34470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a34470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36724/0 (socket says 172.21.15.77:36724) 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc507fff640 1 -- 172.21.15.77:0/1839162367 learned_addr learned my addr 172.21.15.77:0/1839162367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.509+0000 7fc4fffff640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 -- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 msgr2=0x563c39a34470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a34470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 -- 172.21.15.77:0/1839162367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 msgr2=0x563c39a349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 -- 172.21.15.77:0/1839162367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c397033c0 con 0x563c39a34ef0 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a436f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc500006010 tx=0x7fc500005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fffd459699df351 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.510+0000 7fc4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.511+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500004450 con 0x563c39a34ef0 2024-09-15T22:20:51.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.511+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fc5000045f0 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.511+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f800fb20 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.511+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc5000098c0 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc510881740 10 client.26635 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563c39a46200 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50004c070 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc500030070 con 0x563c39a34ef0 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc510881740 10 client.26635 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc510881740 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.512+0000 7fc510881740 20 client.26635 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.513+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39a476c0 0x563c39a49af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.513+0000 7fc510881740 4 client.26635 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:51.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.513+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc4f800fb20 con 0x563c39a476c0 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.513+0000 7fc507fff640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39a476c0 0x563c39a49af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.513+0000 7fc4e77fe640 10 client.26635 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.514+0000 7fc507fff640 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39a476c0 0x563c39a49af0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4188a7ecdf6fd9ae server_cookie=41c500dfd265e92a in_seq=0 out_seq=0 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc4e77fe640 1 -- 172.21.15.77:0/1839162367 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc4f800fb20 con 0x563c39a476c0 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc4e77fe640 10 client.26635 handle_command_reply: tid=0 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 client.26635 shutdown 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 msgr2=0x563c39a436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a436f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc500006010 tx=0x7fc500005c50 comp rx=0 tx=0).stop 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 shutdown_connections 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39a476c0 0x563c39a49af0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc4e77fe640 0 client.26635 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a436f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 --2- 172.21.15.77:0/1839162367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a34470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 >> 172.21.15.77:0/1839162367 conn(0x563c39a28a30 msgr2=0x563c39a2e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 shutdown_connections 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 1 -- 172.21.15.77:0/1839162367 wait complete. 2024-09-15T22:20:51.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.515+0000 7fc510881740 20 client.26635 trim_cache size 0 max 16384 2024-09-15T22:20:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 Processor -- start 2024-09-15T22:20:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 -- start start 2024-09-15T22:20:51.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2c4a0 0x563c39a32d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b12e0 con 0x563c39a2c4a0 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b2e20 con 0x563c39a37680 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc510881740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b19b0 con 0x563c39a34ef0 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37242/0 (socket says 172.21.15.77:37242) 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.538+0000 7fc4fffff640 1 -- 172.21.15.77:0/1233420989 learned_addr learned my addr 172.21.15.77:0/1233420989 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc507fff640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2c4a0 0x563c39a32d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 -- 172.21.15.77:0/1233420989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 msgr2=0x563c39a434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 -- 172.21.15.77:0/1233420989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2c4a0 msgr2=0x563c39a32d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2c4a0 0x563c39a32d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 -- 172.21.15.77:0/1233420989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c39a39de0 con 0x563c39a37680 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc4fffff640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc50ca45640 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a43a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc500005cc0 tx=0x7fc500009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d55b833b83db5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc4fdffb640 1 -- 172.21.15.77:0/1233420989 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500060070 con 0x563c39a37680 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc4fdffb640 1 -- 172.21.15.77:0/1233420989 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50004c040 con 0x563c39a37680 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.539+0000 7fc4fdffb640 1 -- 172.21.15.77:0/1233420989 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000305a0 con 0x563c39a37680 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- 172.21.15.77:0/1233420989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 msgr2=0x563c39a43a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a43a20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc500005cc0 tx=0x7fc500009c20 comp rx=0 tx=0).stop 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- 172.21.15.77:0/1233420989 shutdown_connections 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a37680 0x563c39a43a20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 --2- 172.21.15.77:0/1233420989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a2c4a0 0x563c39a32d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- 172.21.15.77:0/1233420989 >> 172.21.15.77:0/1233420989 conn(0x563c39a28a30 msgr2=0x563c39a364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- 172.21.15.77:0/1233420989 shutdown_connections 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- 172.21.15.77:0/1233420989 wait complete. 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.540+0000 7fc510881740 1 -- start start 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 Processor -- start 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a3bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b2750 con 0x563c39a37680 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b2080 con 0x563c39a2c4a0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081af2e0 con 0x563c39a34ef0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc510881740 10 client.0 _ll_register_callbacks cb 0x7fc4f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc507fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36750/0 (socket says 172.21.15.77:36750) 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc507fff640 1 -- 172.21.15.77:0/173539065 learned_addr learned my addr 172.21.15.77:0/173539065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc50ca45640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a3bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 -- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 msgr2=0x563c39a3b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 -- 172.21.15.77:0/173539065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 msgr2=0x563c39a3bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a3bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc4fffff640 1 -- 172.21.15.77:0/173539065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c39ab1460 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc50ca45640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a3bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.541+0000 7fc507fff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4fffff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a3b9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00cd8c0 tx=0x7fc4f0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530e9d647cc54858 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000ce60 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fc4f0005170 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50004c040 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc4f009caf0 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 10 client.26641 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563c39ab27d0 con 0x563c39a34ef0 2024-09-15T22:20:51.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f009c3f0 con 0x563c39a34ef0 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc4f00a4bd0 con 0x563c39a34ef0 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 10 client.26641 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.542+0000 7fc510881740 20 client.26641 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39ab2f00 0x563c39ab5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc510881740 4 client.26641 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc50004c040 con 0x563c39ab2f00 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc507fff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39ab2f00 0x563c39ab5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc4e77fe640 10 client.26641 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc507fff640 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39ab2f00 0x563c39ab5320 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba3ed477fcedd3b server_cookie=ff79e23eb8057b67 in_seq=0 out_seq=0 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc4e77fe640 1 -- 172.21.15.77:0/173539065 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc50004c040 con 0x563c39ab2f00 2024-09-15T22:20:51.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc4e77fe640 10 client.26641 handle_command_reply: tid=0 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.543+0000 7fc510881740 1 client.26641 shutdown 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 msgr2=0x563c39a3b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a3b9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00cd8c0 tx=0x7fc4f0009310 comp rx=0 tx=0).stop 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 shutdown_connections 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563c39ab2f00 0x563c39ab5320 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c39a37680 0x563c39a3bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c39a34ef0 0x563c39a3b9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 --2- 172.21.15.77:0/173539065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c39a2c4a0 0x563c39a3b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 >> 172.21.15.77:0/173539065 conn(0x563c39a28a30 msgr2=0x563c39a27350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc4e77fe640 0 client.26641 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 shutdown_connections 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 1 -- 172.21.15.77:0/173539065 wait complete. 2024-09-15T22:20:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.544+0000 7fc510881740 20 client.26641 trim_cache size 0 max 16384 2024-09-15T22:20:51.558 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:20:51.558 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:20:51.558 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:20:51.558 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:20:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 msgr2=0x7fc4f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 0x7fc4f40a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc500005e30 tx=0x7fc50003f040 comp rx=0 tx=0).stop 2024-09-15T22:20:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 msgr2=0x7fc508100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:51.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9c0 tx=0x7fc4f000ee80 comp rx=0 tx=0).stop 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 shutdown_connections 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4f40a33b0 0x7fc4f40a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5081082a0 0x7fc508101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5081036e0 0x7fc508100ed0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 --2- 172.21.15.77:0/891311640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc508102d30 0x7fc508100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 >> 172.21.15.77:0/891311640 conn(0x7fc5080fc7d0 msgr2=0x7fc508105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 shutdown_connections 2024-09-15T22:20:51.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:51.545+0000 7fc50e4a8640 1 -- 172.21.15.77:0/891311640 wait complete. 2024-09-15T22:20:52.250 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:20:52.250 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:52.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 654933 7.96 MB/sec execute 301 sec latency 14.133 ms 2024-09-15T22:20:52.578 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:52 smithi077 ceph-mon[30980]: pgmap v260: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-15T22:20:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:52 smithi117 ceph-mon[35021]: pgmap v260: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-15T22:20:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:52 smithi196 ceph-mon[35109]: pgmap v260: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- 172.21.15.77:0/4182092003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc4101df0 msgr2=0x7fccc41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/4182092003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc4101df0 0x7fccc41021d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009900 tx=0x7fccb402a410 comp rx=0 tx=0).stop 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- 172.21.15.77:0/4182092003 shutdown_connections 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/4182092003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4106f20 0x7fccc410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/4182092003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc41027a0 0x7fccc41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/4182092003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc4101df0 0x7fccc41021d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- 172.21.15.77:0/4182092003 >> 172.21.15.77:0/4182092003 conn(0x7fccc40fd8f0 msgr2=0x7fccc40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- 172.21.15.77:0/4182092003 shutdown_connections 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- 172.21.15.77:0/4182092003 wait complete. 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 Processor -- start 2024-09-15T22:20:53.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.221+0000 7fccc9cc3640 1 -- start start 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4101df0 0x7fccc4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc4106f20 0x7fccc419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4111f40 con 0x7fccc41027a0 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4111dc0 con 0x7fccc4101df0 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc9cc3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41120c0 con 0x7fccc4106f20 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33262/0 (socket says 172.21.15.77:33262) 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4101df0 0x7fccc4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 -- 172.21.15.77:0/2858825341 learned_addr learned my addr 172.21.15.77:0/2858825341 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc3fff640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc4106f20 0x7fccc419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 -- 172.21.15.77:0/2858825341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc4106f20 msgr2=0x7fccc419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc4106f20 0x7fccc419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 -- 172.21.15.77:0/2858825341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4101df0 msgr2=0x7fccc4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4101df0 0x7fccc4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 -- 172.21.15.77:0/2858825341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc419d3b0 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc2ffd640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fccb800d950 tx=0x7fccb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.222+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8017030 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.223+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc419d6a0 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.223+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb8004020 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.223+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80047a0 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.223+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc41a4f80 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.224+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc41021d0 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.226+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fccb80041d0 con 0x7fccc41027a0 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.226+0000 7fccc0ff9640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 0x7fccac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.227+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fccb80d0030 con 0x7fccc41027a0 2024-09-15T22:20:53.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.227+0000 7fccc37fe640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 0x7fccac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.227+0000 7fccc37fe640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 0x7fccac0a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009230 tx=0x7fccb40058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.230+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccb809c700 con 0x7fccc41027a0 2024-09-15T22:20:53.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.465+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc4109b70 con 0x7fccc41027a0 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.466+0000 7fccc0ff9640 1 -- 172.21.15.77:0/2858825341 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fccb809c000 con 0x7fccc41027a0 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 msgr2=0x7fccac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 0x7fccac0a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009230 tx=0x7fccb40058d0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 msgr2=0x7fccc4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fccb800d950 tx=0x7fccb800de10 comp rx=0 tx=0).stop 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 shutdown_connections 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fccac0a3430 0x7fccac0a58f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccc4106f20 0x7fccc419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccc41027a0 0x7fccc4198940 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 --2- 172.21.15.77:0/2858825341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc4101df0 0x7fccc4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.468+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 >> 172.21.15.77:0/2858825341 conn(0x7fccc40fd8f0 msgr2=0x7fccc40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.469+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 shutdown_connections 2024-09-15T22:20:53.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:53.469+0000 7fccc9cc3640 1 -- 172.21.15.77:0/2858825341 wait complete. 2024-09-15T22:20:53.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 657303 7.98 MB/sec execute 302 sec latency 18.316 ms 2024-09-15T22:20:54.182 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:20:54.466 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:54.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 659401 7.98 MB/sec execute 303 sec latency 19.681 ms 2024-09-15T22:20:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:54 smithi077 ceph-mon[30980]: pgmap v261: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:54 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2858825341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:54 smithi117 ceph-mon[35021]: pgmap v261: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:54 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2858825341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:54.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:54 smithi196 ceph-mon[35109]: pgmap v261: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:54.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:54 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2858825341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:54.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 -- 172.21.15.77:0/1644964611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d4101df0 msgr2=0x7f51d41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 --2- 172.21.15.77:0/1644964611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d4101df0 0x7f51d41021d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009900 tx=0x7f51c402a410 comp rx=0 tx=0).stop 2024-09-15T22:20:54.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 -- 172.21.15.77:0/1644964611 shutdown_connections 2024-09-15T22:20:54.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 --2- 172.21.15.77:0/1644964611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4106f20 0x7f51d410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 --2- 172.21.15.77:0/1644964611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d41027a0 0x7f51d41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 --2- 172.21.15.77:0/1644964611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d4101df0 0x7f51d41021d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.951+0000 7f51d954a640 1 -- 172.21.15.77:0/1644964611 >> 172.21.15.77:0/1644964611 conn(0x7f51d40fd8f0 msgr2=0x7f51d40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- 172.21.15.77:0/1644964611 shutdown_connections 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- 172.21.15.77:0/1644964611 wait complete. 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 Processor -- start 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- start start 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 0x7f51d419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111f80 con 0x7f51d41027a0 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111e00 con 0x7f51d4106f20 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.952+0000 7f51d954a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4112100 con 0x7f51d4101df0 2024-09-15T22:20:54.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37284/0 (socket says 172.21.15.77:37284) 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d2ffd640 1 -- 172.21.15.77:0/725912353 learned_addr learned my addr 172.21.15.77:0/725912353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33266/0 (socket says 172.21.15.77:33266) 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d37fe640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 0x7f51d419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 -- 172.21.15.77:0/725912353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 msgr2=0x7f51d4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 -- 172.21.15.77:0/725912353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 msgr2=0x7f51d419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 0x7f51d419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 -- 172.21.15.77:0/725912353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d419d4f0 con 0x7f51d41027a0 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d37fe640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 0x7f51d419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51cbfff640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9f0 tx=0x7f51bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d0ff9640 1 -- 172.21.15.77:0/725912353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00ccc0 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d419d7e0 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d0ff9640 1 -- 172.21.15.77:0/725912353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc010910 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d0ff9640 1 -- 172.21.15.77:0/725912353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc018960 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.953+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a50c0 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.956+0000 7f51d0ff9640 1 -- 172.21.15.77:0/725912353 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f51bc010ac0 con 0x7f51d41027a0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.956+0000 7f51db923740 1 Processor -- start 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.956+0000 7f51db923740 1 -- start start 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f7f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f805b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a8f75a2f0 con 0x561a8f7f89e0 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a8f701cf0 con 0x561a8f803780 2024-09-15T22:20:54.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51db923740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a8f7562b0 con 0x561a8f7f9f50 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33274/0 (socket says 172.21.15.77:33274) 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2337494942 learned_addr learned my addr 172.21.15.77:0/2337494942 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f805b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d37fe640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f7f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d0ff9640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 0x7f51c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2337494942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 msgr2=0x561a8f7f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f7f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2337494942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 msgr2=0x561a8f805b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f805b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2337494942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8f6e63a0 con 0x561a8f7f89e0 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f805b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d37fe640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f7f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d37fe640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 0x7f51c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.957+0000 7f51d0ff9640 1 -- 172.21.15.77:0/725912353 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f51bc0d04a0 con 0x7f51d41027a0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.958+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009a20 tx=0x7f51c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee647001ce5d2d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.958+0000 7f51c97fa640 1 -- 172.21.15.77:0/2337494942 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4040070 con 0x561a8f7f89e0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.958+0000 7f51c97fa640 1 -- 172.21.15.77:0/2337494942 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c4031a20 con 0x561a8f7f89e0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.958+0000 7f51c97fa640 1 -- 172.21.15.77:0/2337494942 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4035040 con 0x561a8f7f89e0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- 172.21.15.77:0/2337494942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 msgr2=0x561a8f7f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009a20 tx=0x7f51c4005e80 comp rx=0 tx=0).stop 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51d37fe640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 0x7f51c00a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51cc005fd0 tx=0x7f51cc004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- 172.21.15.77:0/2337494942 shutdown_connections 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f805b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f7f8e20 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 --2- 172.21.15.77:0/2337494942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f7f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- 172.21.15.77:0/2337494942 >> 172.21.15.77:0/2337494942 conn(0x561a8f7bd310 msgr2=0x561a8f7f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- 172.21.15.77:0/2337494942 shutdown_connections 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- 172.21.15.77:0/2337494942 wait complete. 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 -- start start 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.959+0000 7f51db923740 1 Processor -- start 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f803190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f800a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a8f806420 con 0x561a8f7f9f50 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a8f715060 con 0x561a8f7f89e0 2024-09-15T22:20:54.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a8f8065a0 con 0x561a8f803780 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 10 client.0 _ll_register_callbacks cb 0x561a8f4688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51d37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f800a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33278/0 (socket says 172.21.15.77:33278) 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 -- 172.21.15.77:0/2626624373 learned_addr learned my addr 172.21.15.77:0/2626624373 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f803190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 -- 172.21.15.77:0/2626624373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 msgr2=0x561a8f800a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f800a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 -- 172.21.15.77:0/2626624373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 msgr2=0x561a8f803190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f803190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 -- 172.21.15.77:0/2626624373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8f4cb3c0 con 0x561a8f7f9f50 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51d37fe640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f800a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0cd860 tx=0x7f51bc0980e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5fa8c27105c349 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc09bda0 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f51bc09b890 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c4031a20 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f51bc0cdc90 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.960+0000 7f51db923740 10 client.17616 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a8f811f40 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c4031a20 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f51bc018c20 con 0x561a8f7f9f50 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 10 client.17616 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 10 client.17616 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 20 client.17616 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:54.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f813450 0x561a8f815840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 4 client.17616 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a8f767860 con 0x561a8f813450 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f813450 0x561a8f815840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51ba7fc640 10 client.17616 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.961+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f813450 0x561a8f815840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c3da5895587e6ed server_cookie=38b74b87b0a32aca in_seq=0 out_seq=0 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2626624373 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a8f767860 con 0x561a8f813450 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51ba7fc640 10 client.17616 handle_command_reply: tid=0 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 client.17616 shutdown 2024-09-15T22:20:54.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 msgr2=0x561a8f800440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0cd860 tx=0x7f51bc0980e0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 shutdown_connections 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f813450 0x561a8f815840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f800a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f800440 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51ba7fc640 0 client.17616 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 --2- 172.21.15.77:0/2626624373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f803190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 >> 172.21.15.77:0/2626624373 conn(0x561a8f7bd310 msgr2=0x561a8f7fad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 shutdown_connections 2024-09-15T22:20:54.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 1 -- 172.21.15.77:0/2626624373 wait complete. 2024-09-15T22:20:54.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.962+0000 7f51db923740 20 client.17616 trim_cache size 0 max 16384 2024-09-15T22:20:54.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 Processor -- start 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 -- start start 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f804b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41acd70 con 0x561a8f7f9f50 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41ae8b0 con 0x561a8f7f89e0 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51db923740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41ad440 con 0x561a8f803780 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33288/0 (socket says 172.21.15.77:33288) 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36826/0 (socket says 172.21.15.77:36826) 2024-09-15T22:20:54.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51cbfff640 1 -- 172.21.15.77:0/766125275 learned_addr learned my addr 172.21.15.77:0/766125275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.976+0000 7f51d37fe640 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f804b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 -- 172.21.15.77:0/766125275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 msgr2=0x561a8f804b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f804b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 -- 172.21.15.77:0/766125275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 msgr2=0x561a8f80f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 -- 172.21.15.77:0/766125275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8f815bb0 con 0x561a8f7f9f50 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51cbfff640 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0988a0 tx=0x7f51bc098280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa17ae90694cf29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51c8ff9640 1 -- 172.21.15.77:0/766125275 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0b4070 con 0x561a8f7f9f50 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51c8ff9640 1 -- 172.21.15.77:0/766125275 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc09c440 con 0x561a8f7f9f50 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.977+0000 7f51c8ff9640 1 -- 172.21.15.77:0/766125275 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc09baf0 con 0x561a8f7f9f50 2024-09-15T22:20:54.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- 172.21.15.77:0/766125275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 msgr2=0x561a8f7ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0988a0 tx=0x7f51bc098280 comp rx=0 tx=0).stop 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- 172.21.15.77:0/766125275 shutdown_connections 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f803780 0x561a8f804b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f9f50 0x561a8f7ffd00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 --2- 172.21.15.77:0/766125275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f7f89e0 0x561a8f80f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- 172.21.15.77:0/766125275 >> 172.21.15.77:0/766125275 conn(0x561a8f7bd310 msgr2=0x561a8f7fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- 172.21.15.77:0/766125275 shutdown_connections 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- 172.21.15.77:0/766125275 wait complete. 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 -- start start 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.978+0000 7f51db923740 1 Processor -- start 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f81b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f81b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41ae1e0 con 0x561a8f7f89e0 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41adb10 con 0x561a8f803780 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41aad70 con 0x561a8f7f9f50 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51db923740 10 client.0 _ll_register_callbacks cb 0x7f51bc00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f81b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f81b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33298/0 (socket says 172.21.15.77:33298) 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2080688708 learned_addr learned my addr 172.21.15.77:0/2080688708 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f81b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37322/0 (socket says 172.21.15.77:37322) 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2080688708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 msgr2=0x561a8f81b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f81b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2080688708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 msgr2=0x561a8f81b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f81b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 -- 172.21.15.77:0/2080688708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8f809cb0 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d37fe640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f81b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51d2ffd640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005570 tx=0x7f51c4005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0bb955853b04b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4035040 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.979+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f51c4004240 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc09c440 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f51c405e050 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 10 client.17622 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a8f87e350 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc09c440 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f51c404d040 con 0x561a8f7f89e0 2024-09-15T22:20:54.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 10 client.17622 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 10 client.17622 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 20 client.17622 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f87e640 0x561a8f880a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 4 client.17622 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a8f761d70 con 0x561a8f87e640 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.980+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f87e640 0x561a8f880a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51cbfff640 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f87e640 0x561a8f880a30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53f95470dc499866 server_cookie=c3e8ee66535674ed in_seq=0 out_seq=0 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51ba7fc640 10 client.17622 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51ba7fc640 1 -- 172.21.15.77:0/2080688708 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a8f761d70 con 0x561a8f87e640 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51ba7fc640 10 client.17622 handle_command_reply: tid=0 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 client.17622 shutdown 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 msgr2=0x561a8f81ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005570 tx=0x7f51c4005500 comp rx=0 tx=0).stop 2024-09-15T22:20:54.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 shutdown_connections 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561a8f87e640 0x561a8f880a30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a8f803780 0x561a8f81b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a8f7f9f50 0x561a8f81b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 --2- 172.21.15.77:0/2080688708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a8f7f89e0 0x561a8f81ac30 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 >> 172.21.15.77:0/2080688708 conn(0x561a8f7bd310 msgr2=0x561a8f8024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51ba7fc640 0 client.17622 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 shutdown_connections 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 1 -- 172.21.15.77:0/2080688708 wait complete. 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.981+0000 7f51db923740 20 client.17622 trim_cache size 0 max 16384 2024-09-15T22:20:54.991 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:54.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 msgr2=0x7f51c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 0x7f51c00a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51cc005fd0 tx=0x7f51cc004a40 comp rx=0 tx=0).stop 2024-09-15T22:20:54.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 msgr2=0x7f51d4198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:54.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00e9f0 tx=0x7f51bc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 shutdown_connections 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f51c00a33b0 0x7f51c00a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51d4106f20 0x7f51d419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d41027a0 0x7f51d4198a80 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 --2- 172.21.15.77:0/725912353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51d4101df0 0x7f51d4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 >> 172.21.15.77:0/725912353 conn(0x7f51d40fd8f0 msgr2=0x7f51d40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 shutdown_connections 2024-09-15T22:20:54.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:54.982+0000 7f51d954a640 1 -- 172.21.15.77:0/725912353 wait complete. 2024-09-15T22:20:55.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 661330 8.00 MB/sec execute 304 sec latency 35.890 ms 2024-09-15T22:20:55.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:20:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:56.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 664083 8.03 MB/sec execute 305 sec latency 22.187 ms 2024-09-15T22:20:56.505 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:56 smithi077 ceph-mon[30980]: pgmap v262: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 -- 172.21.15.77:0/782434179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 msgr2=0x7f40501031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 --2- 172.21.15.77:0/782434179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f40501031e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4040009900 tx=0x7f404002a480 comp rx=0 tx=0).stop 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 -- 172.21.15.77:0/782434179 shutdown_connections 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 --2- 172.21.15.77:0/782434179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 0x7f405010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 --2- 172.21.15.77:0/782434179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 0x7f405010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 --2- 172.21.15.77:0/782434179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f40501031e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 -- 172.21.15.77:0/782434179 >> 172.21.15.77:0/782434179 conn(0x7f40500fc7a0 msgr2=0x7f40500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 -- 172.21.15.77:0/782434179 shutdown_connections 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.587+0000 7f4055998640 1 -- 172.21.15.77:0/782434179 wait complete. 2024-09-15T22:20:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 Processor -- start 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 -- start start 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 0x7f4050100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 0x7f40501014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4050111fc0 con 0x7f405010c1a0 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4050111e40 con 0x7f4050102e00 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f4055998640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4050112140 con 0x7f4050103720 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f404effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 0x7f40501014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f404effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36850/0 (socket says 172.21.15.77:36850) 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f404effd640 1 -- 172.21.15.77:0/2451727738 learned_addr learned my addr 172.21.15.77:0/2451727738 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.588+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 0x7f4050100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 -- 172.21.15.77:0/2451727738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 msgr2=0x7f4050100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 0x7f4050100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 -- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 msgr2=0x7f40501014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 0x7f40501014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 -- 172.21.15.77:0/2451727738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40501a6900 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f404effd640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4040004770 tx=0x7f4040003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f4054996640 1 -- 172.21.15.77:0/2451727738 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404003f070 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f4054996640 1 -- 172.21.15.77:0/2451727738 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4040031ca0 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501a6b90 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f4054996640 1 -- 172.21.15.77:0/2451727738 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040035030 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.589+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a7070 con 0x7f4050102e00 2024-09-15T22:20:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.591+0000 7f4054996640 1 -- 172.21.15.77:0/2451727738 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4040042070 con 0x7f4050102e00 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 Processor -- start 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 -- start start 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4054996640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 0x7f40380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c27d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f52c1e32f0 con 0x55f52c27d5d0 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f52c18acf0 con 0x55f52c27eb40 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4057d71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f52c1df2b0 con 0x55f52c288370 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404f7fe640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 0x7f40380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f4054996640 1 -- 172.21.15.77:0/2451727738 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f40400f0670 con 0x7f4050102e00 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c27d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33340/0 (socket says 172.21.15.77:33340) 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404e7fc640 1 -- 172.21.15.77:0/363689457 learned_addr learned my addr 172.21.15.77:0/363689457 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.592+0000 7f404f7fe640 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 -- 172.21.15.77:0/363689457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 msgr2=0x55f52c28a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 -- 172.21.15.77:0/363689457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 msgr2=0x55f52c27da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 -- 172.21.15.77:0/363689457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52c16f3a0 con 0x55f52c27eb40 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404e7fc640 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404f7fe640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 0x7f40380a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4048005fd0 tx=0x7f4048011b40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f404effd640 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c27d090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40400eda30 tx=0x7f4040009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf58c6c5bd6cd41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f4035ffb640 1 -- 172.21.15.77:0/363689457 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040049070 con 0x55f52c27eb40 2024-09-15T22:20:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f4035ffb640 1 -- 172.21.15.77:0/363689457 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40400ef070 con 0x55f52c27eb40 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.593+0000 7f4035ffb640 1 -- 172.21.15.77:0/363689457 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40400baa30 con 0x55f52c27eb40 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- 172.21.15.77:0/363689457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 msgr2=0x55f52c27d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c27d090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40400eda30 tx=0x7f4040009260 comp rx=0 tx=0).stop 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- 172.21.15.77:0/363689457 shutdown_connections 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c27da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- 172.21.15.77:0/363689457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c27d090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- 172.21.15.77:0/363689457 >> 172.21.15.77:0/363689457 conn(0x55f52c246310 msgr2=0x55f52c279c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- 172.21.15.77:0/363689457 shutdown_connections 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- 172.21.15.77:0/363689457 wait complete. 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- start start 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 Processor -- start 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f52c28ae30 con 0x55f52c288370 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f52c19e060 con 0x55f52c27d5d0 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.594+0000 7f4057d71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f52c28afb0 con 0x55f52c27eb40 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f4057d71740 10 client.0 _ll_register_callbacks cb 0x55f52bef18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37340/0 (socket says 172.21.15.77:37340) 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36866/0 (socket says 172.21.15.77:36866) 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404e7fc640 1 -- 172.21.15.77:0/1721830389 learned_addr learned my addr 172.21.15.77:0/1721830389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33348/0 (socket says 172.21.15.77:33348) 2024-09-15T22:20:56.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 -- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 msgr2=0x55f52c285030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 -- 172.21.15.77:0/1721830389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 msgr2=0x55f52c285640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 -- 172.21.15.77:0/1721830389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52bf543c0 con 0x55f52c27d5d0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404f7fe640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404e7fc640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f404effd640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f404009f490 tx=0x7f404003b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9c05e1ba2031a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f402effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040049070 con 0x55f52c27d5d0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f40400edcb0 con 0x55f52c27d5d0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40400ef070 con 0x55f52c27d5d0 2024-09-15T22:20:56.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.595+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4040005d50 con 0x55f52c27d5d0 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 10 client.26747 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f52c28ae30 con 0x55f52c27d5d0 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40400ef070 con 0x55f52c27d5d0 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f404003b4f0 con 0x55f52c27d5d0 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 10 client.26747 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 10 client.26747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 20 client.26747 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c297a80 0x55f52c299e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 4 client.26747 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f52c1f0860 con 0x55f52c297a80 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.596+0000 7f404e7fc640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c297a80 0x55f52c299e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.597+0000 7f402effd640 10 client.26747 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.597+0000 7f404e7fc640 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c297a80 0x55f52c299e70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d61c4afcde90903f server_cookie=58d8c0ce9952c8ec in_seq=0 out_seq=0 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.597+0000 7f402effd640 1 -- 172.21.15.77:0/1721830389 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f52c1f0860 con 0x55f52c297a80 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.597+0000 7f402effd640 10 client.26747 handle_command_reply: tid=0 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.597+0000 7f4057d71740 1 client.26747 shutdown 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 msgr2=0x55f52c287d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f404009f490 tx=0x7f404003b890 comp rx=0 tx=0).stop 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 shutdown_connections 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c297a80 0x55f52c299e70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c288370 0x55f52c285640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c27eb40 0x55f52c285030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 --2- 172.21.15.77:0/1721830389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27d5d0 0x55f52c287d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 >> 172.21.15.77:0/1721830389 conn(0x55f52c246310 msgr2=0x55f52c27f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f402effd640 0 client.26747 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 shutdown_connections 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 1 -- 172.21.15.77:0/1721830389 wait complete. 2024-09-15T22:20:56.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.598+0000 7f4057d71740 20 client.26747 trim_cache size 0 max 16384 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 Processor -- start 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 -- start start 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c284010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c29f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b1180 con 0x55f52c27d5d0 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b2cc0 con 0x55f52c27eb40 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f4057d71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b1850 con 0x55f52c288370 2024-09-15T22:20:56.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f404effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f404e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c284010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c29f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f404effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33350/0 (socket says 172.21.15.77:33350) 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.612+0000 7f404effd640 1 -- 172.21.15.77:0/2721011859 learned_addr learned my addr 172.21.15.77:0/2721011859 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 -- 172.21.15.77:0/2721011859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 msgr2=0x55f52c29f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c29f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 -- 172.21.15.77:0/2721011859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 msgr2=0x55f52c294000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 -- 172.21.15.77:0/2721011859 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52c283580 con 0x55f52c27eb40 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404effd640 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f404e7fc640 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c284010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f52c29a6e0 tx=0x7f4044005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7f542b18e3575 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f40357fa640 1 -- 172.21.15.77:0/2721011859 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044005020 con 0x55f52c27eb40 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f40357fa640 1 -- 172.21.15.77:0/2721011859 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404400f810 con 0x55f52c27eb40 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.613+0000 7f40357fa640 1 -- 172.21.15.77:0/2721011859 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044004a00 con 0x55f52c27eb40 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- 172.21.15.77:0/2721011859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 msgr2=0x55f52c284010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c284010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f52c29a6e0 tx=0x7f4044005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- 172.21.15.77:0/2721011859 shutdown_connections 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c29f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c284010 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 --2- 172.21.15.77:0/2721011859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c294000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- 172.21.15.77:0/2721011859 >> 172.21.15.77:0/2721011859 conn(0x55f52c246310 msgr2=0x55f52c283900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- 172.21.15.77:0/2721011859 shutdown_connections 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- 172.21.15.77:0/2721011859 wait complete. 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.614+0000 7f4057d71740 1 -- start start 2024-09-15T22:20:56.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 Processor -- start 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c28b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b25f0 con 0x55f52c27d5d0 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40501b1f20 con 0x55f52c27eb40 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40501af180 con 0x55f52c288370 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f4057d71740 10 client.0 _ll_register_callbacks cb 0x7f4044009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33362/0 (socket says 172.21.15.77:33362) 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 -- 172.21.15.77:0/4090936343 learned_addr learned my addr 172.21.15.77:0/4090936343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404e7fc640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c28b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404f7fe640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 -- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 msgr2=0x55f52c28bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 -- 172.21.15.77:0/4090936343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 msgr2=0x55f52c28b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c28b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.615+0000 7f404effd640 1 -- 172.21.15.77:0/4090936343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52c3008b0 con 0x55f52c27d5d0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f404effd640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f40400b9100 tx=0x7f4040009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b90cf3c11ea481b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040049040 con 0x55f52c27d5d0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404400f810 con 0x55f52c27d5d0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f40400edcb0 con 0x55f52c27d5d0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40400bc730 con 0x55f52c27d5d0 2024-09-15T22:20:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 10 client.17640 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f52c3008b0 con 0x55f52c27d5d0 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404400f810 con 0x55f52c27d5d0 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f40400f0330 con 0x55f52c27d5d0 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 10 client.17640 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 10 client.17640 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 20 client.17640 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c301ed0 0x55f52c3042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 4 client.17640 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.616+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f52c1ead70 con 0x55f52c301ed0 2024-09-15T22:20:56.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f404e7fc640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c301ed0 0x55f52c3042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f402effd640 10 client.17640 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f404e7fc640 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c301ed0 0x55f52c3042c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4127c6a9b02972ed server_cookie=3975153585b34033 in_seq=0 out_seq=0 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f402effd640 1 -- 172.21.15.77:0/4090936343 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f52c1ead70 con 0x55f52c301ed0 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f402effd640 10 client.17640 handle_command_reply: tid=0 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 client.17640 shutdown 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 msgr2=0x55f52c28b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f40400b9100 tx=0x7f4040009260 comp rx=0 tx=0).stop 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 shutdown_connections 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f52c301ed0 0x55f52c3042c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52c288370 0x55f52c28bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f52c27eb40 0x55f52c28b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 --2- 172.21.15.77:0/4090936343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f52c27d5d0 0x55f52c28b0c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 >> 172.21.15.77:0/4090936343 conn(0x55f52c246310 msgr2=0x55f52c284120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.617+0000 7f402effd640 0 client.17640 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 shutdown_connections 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4057d71740 1 -- 172.21.15.77:0/4090936343 wait complete. 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4057d71740 20 client.17640 trim_cache size 0 max 16384 2024-09-15T22:20:56.629 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 msgr2=0x7f40380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 0x7f40380a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4048005fd0 tx=0x7f4048011b40 comp rx=0 tx=0).stop 2024-09-15T22:20:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 msgr2=0x7f4050100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.618+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4040004770 tx=0x7f4040003c10 comp rx=0 tx=0).stop 2024-09-15T22:20:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 shutdown_connections 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40380a33b0 0x7f40380a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f405010c1a0 0x7f40501014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050103720 0x7f4050100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 --2- 172.21.15.77:0/2451727738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4050102e00 0x7f4050100a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 >> 172.21.15.77:0/2451727738 conn(0x7f40500fc7a0 msgr2=0x7f40500fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 shutdown_connections 2024-09-15T22:20:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:56.619+0000 7f4055998640 1 -- 172.21.15.77:0/2451727738 wait complete. 2024-09-15T22:20:56.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:56 smithi196 ceph-mon[35109]: pgmap v262: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:56.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:56 smithi117 ceph-mon[35021]: pgmap v262: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-09-15T22:20:57.366 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:20:57.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 665778 8.02 MB/sec execute 306 sec latency 20.967 ms 2024-09-15T22:20:57.636 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:58.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 -- 172.21.15.77:0/3653885882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea8106f20 msgr2=0x7efea810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 --2- 172.21.15.77:0/3653885882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea8106f20 0x7efea810eaa0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ca0 tx=0x7efea002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 -- 172.21.15.77:0/3653885882 shutdown_connections 2024-09-15T22:20:58.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 --2- 172.21.15.77:0/3653885882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea8106f20 0x7efea810eaa0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 --2- 172.21.15.77:0/3653885882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea81027a0 0x7efea81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 --2- 172.21.15.77:0/3653885882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 0x7efea81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 -- 172.21.15.77:0/3653885882 >> 172.21.15.77:0/3653885882 conn(0x7efea80fd8f0 msgr2=0x7efea80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 -- 172.21.15.77:0/3653885882 shutdown_connections 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 -- 172.21.15.77:0/3653885882 wait complete. 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.128+0000 7efeaca3a640 1 Processor -- start 2024-09-15T22:20:58.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 -- start start 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 0x7efea80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea8106f20 0x7efea8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111f80 con 0x7efea81027a0 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efea8111e00 con 0x7efea8106f20 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efeaca3a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efea8112100 con 0x7efea8101df0 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efea659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 0x7efea80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33392/0 (socket says 172.21.15.77:33392) 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efea6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea8106f20 0x7efea8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 -- 172.21.15.77:0/1762728928 learned_addr learned my addr 172.21.15.77:0/1762728928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 -- 172.21.15.77:0/1762728928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 msgr2=0x7efea80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 0x7efea80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 -- 172.21.15.77:0/1762728928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea8106f20 msgr2=0x7efea8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea8106f20 0x7efea8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 -- 172.21.15.77:0/1762728928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea81a68a0 con 0x7efea81027a0 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.129+0000 7efe9ffff640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9f0 tx=0x7efe9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.130+0000 7efe9f7fe640 1 -- 172.21.15.77:0/1762728928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9000cd80 con 0x7efea81027a0 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.130+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea81a6b90 con 0x7efea81027a0 2024-09-15T22:20:58.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.130+0000 7efe9f7fe640 1 -- 172.21.15.77:0/1762728928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe90010910 con 0x7efea81027a0 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.130+0000 7efe9f7fe640 1 -- 172.21.15.77:0/1762728928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe900188f0 con 0x7efea81027a0 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.130+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea81a7120 con 0x7efea81027a0 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.132+0000 7efe9f7fe640 1 -- 172.21.15.77:0/1762728928 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efe90010ac0 con 0x7efea81027a0 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.132+0000 7efeaee13740 1 Processor -- start 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 -- start start 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f30138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f3020f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596f2f712f0 con 0x5596f3015350 2024-09-15T22:20:58.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596f2f18cf0 con 0x5596f301eb80 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efeaee13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596f2f6d2b0 con 0x5596f3013de0 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f3020f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37402/0 (socket says 172.21.15.77:37402) 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 -- 172.21.15.77:0/1347646614 learned_addr learned my addr 172.21.15.77:0/1347646614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f30138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efe9f7fe640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 0x7efe940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 -- 172.21.15.77:0/1347646614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 msgr2=0x5596f3020f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f3020f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 -- 172.21.15.77:0/1347646614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 msgr2=0x5596f30138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f30138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea659c640 1 -- 172.21.15.77:0/1347646614 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f2efd3a0 con 0x5596f3013de0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.133+0000 7efea6d9d640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 0x7efe940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efe9f7fe640 1 -- 172.21.15.77:0/1762728928 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7efe900d0410 con 0x7efea81027a0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efea659c640 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe98009d20 tx=0x7efe98011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bdc93bead5e32ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efea6d9d640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 0x7efe940a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efea0004af0 tx=0x7efea00095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efe9cff9640 1 -- 172.21.15.77:0/1347646614 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9801e070 con 0x5596f3013de0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efe9cff9640 1 -- 172.21.15.77:0/1347646614 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98019b10 con 0x5596f3013de0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.134+0000 7efe9cff9640 1 -- 172.21.15.77:0/1347646614 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9800ebb0 con 0x5596f3013de0 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- 172.21.15.77:0/1347646614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 msgr2=0x5596f3014220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe98009d20 tx=0x7efe98011810 comp rx=0 tx=0).stop 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- 172.21.15.77:0/1347646614 shutdown_connections 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f3020f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3013de0 0x5596f3014220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 --2- 172.21.15.77:0/1347646614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f30138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- 172.21.15.77:0/1347646614 >> 172.21.15.77:0/1347646614 conn(0x5596f2fd4310 msgr2=0x5596f3010470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- 172.21.15.77:0/1347646614 shutdown_connections 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- 172.21.15.77:0/1347646614 wait complete. 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 -- start start 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.135+0000 7efeaee13740 1 Processor -- start 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 0x5596f301e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f301bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5596f3021880 con 0x5596f3015350 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33410/0 (socket says 172.21.15.77:33410) 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efe9ffff640 1 -- 172.21.15.77:0/474401168 learned_addr learned my addr 172.21.15.77:0/474401168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efea659c640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 0x5596f301e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efea6d9d640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f301bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596f2f2c060 con 0x5596f3013de0 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.136+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596f3021a00 con 0x5596f301eb80 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efeaee13740 10 client.0 _ll_register_callbacks cb 0x5596f2c7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 -- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 msgr2=0x5596f301bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f301bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 -- 172.21.15.77:0/474401168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 msgr2=0x5596f301e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 0x5596f301e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f2ce23c0 con 0x5596f3015350 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efea6d9d640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f301bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efea659c640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 0x5596f301e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe9ffff640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9c0 tx=0x7efe9009c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6c2246d8cf9da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:58.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe900a0710 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe98019b10 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7efe90098a80 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efe9009c410 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efeaee13740 10 client.17652 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.137+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5596f302d300 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98019b10 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efe90098280 con 0x5596f3015350 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 10 client.17652 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 20 client.17652 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f3030b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 4 client.17652 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:58.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f2f7e860 con 0x5596f302e710 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.138+0000 7efea659c640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f3030b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.139+0000 7efea659c640 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f3030b60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=396fa9103dcb0e7c server_cookie=b43de15c44384050 in_seq=0 out_seq=0 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.139+0000 7efe8dffb640 10 client.17652 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efe8dffb640 1 -- 172.21.15.77:0/474401168 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596f2f7e860 con 0x5596f302e710 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efe8dffb640 10 client.17652 handle_command_reply: tid=0 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 client.17652 shutdown 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 msgr2=0x5596f301b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9c0 tx=0x7efe9009c300 comp rx=0 tx=0).stop 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 shutdown_connections 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f3030b60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f301bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3015350 0x5596f301b7e0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 --2- 172.21.15.77:0/474401168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3013de0 0x5596f301e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efe8dffb640 0 client.17652 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:58.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 >> 172.21.15.77:0/474401168 conn(0x5596f2fd4310 msgr2=0x5596f3016170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 shutdown_connections 2024-09-15T22:20:58.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 1 -- 172.21.15.77:0/474401168 wait complete. 2024-09-15T22:20:58.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.140+0000 7efeaee13740 20 client.17652 trim_cache size 0 max 16384 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 Processor -- start 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 -- start start 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3015350 0x5596f301b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f301ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efea81acd80 con 0x5596f3013de0 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efea81ae8c0 con 0x5596f301eb80 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.154+0000 7efeaee13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efea81ad450 con 0x5596f3015350 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33426/0 (socket says 172.21.15.77:33426) 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 -- 172.21.15.77:0/800569025 learned_addr learned my addr 172.21.15.77:0/800569025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efe9ffff640 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3015350 0x5596f301b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea6d9d640 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f301ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 -- 172.21.15.77:0/800569025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3015350 msgr2=0x5596f301b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3015350 0x5596f301b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 -- 172.21.15.77:0/800569025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 msgr2=0x5596f301ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f301ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 -- 172.21.15.77:0/800569025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f301e8a0 con 0x5596f3013de0 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efea659c640 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efe98009c40 tx=0x7efe9800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2317d0ab54d3818 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efe8ffff640 1 -- 172.21.15.77:0/800569025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9802a070 con 0x5596f3013de0 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efe8ffff640 1 -- 172.21.15.77:0/800569025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9801f040 con 0x5596f3013de0 2024-09-15T22:20:58.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.155+0000 7efe8ffff640 1 -- 172.21.15.77:0/800569025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9801e040 con 0x5596f3013de0 2024-09-15T22:20:58.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 -- 172.21.15.77:0/800569025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 msgr2=0x5596f302ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efe98009c40 tx=0x7efe9800ea30 comp rx=0 tx=0).stop 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 -- 172.21.15.77:0/800569025 shutdown_connections 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f301eb80 0x5596f301ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f3015350 0x5596f301b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 --2- 172.21.15.77:0/800569025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f302ab20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 -- 172.21.15.77:0/800569025 >> 172.21.15.77:0/800569025 conn(0x5596f2fd4310 msgr2=0x5596f3016b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 -- 172.21.15.77:0/800569025 shutdown_connections 2024-09-15T22:20:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.156+0000 7efeaee13740 1 -- 172.21.15.77:0/800569025 wait complete. 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 -- start start 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 Processor -- start 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3015350 0x5596f3035960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f3035ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efea81ae1f0 con 0x5596f3013de0 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efea81adb20 con 0x5596f3015350 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efea81aad80 con 0x5596f301eb80 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efeaee13740 10 client.0 _ll_register_callbacks cb 0x7efe9000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efea659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efea659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33442/0 (socket says 172.21.15.77:33442) 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.157+0000 7efea659c640 1 -- 172.21.15.77:0/2595668051 learned_addr learned my addr 172.21.15.77:0/2595668051 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efe9ffff640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3015350 0x5596f3035960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea6d9d640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f3035ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 -- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 msgr2=0x5596f3035ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f3035ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 -- 172.21.15.77:0/2595668051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3015350 msgr2=0x5596f3035960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3015350 0x5596f3035960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 -- 172.21.15.77:0/2595668051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f3025f30 con 0x5596f3013de0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efea659c640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efe98009250 tx=0x7efe98005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83028f36ae089b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efe8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98036030 con 0x5596f3013de0 2024-09-15T22:20:58.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7efe9800ed80 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.158+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9801f040 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efe98040050 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 10 client.17658 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5596f30271f0 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9801f040 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efe98025040 con 0x5596f3013de0 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 10 client.17658 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 20 client.17658 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f309b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 4 client.17658 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596f2f78d70 con 0x5596f302e710 2024-09-15T22:20:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.159+0000 7efe9ffff640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f309b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.160+0000 7efe9ffff640 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f309b410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b43e28f168ee3062 server_cookie=2907d053dbdda005 in_seq=0 out_seq=0 2024-09-15T22:20:58.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.160+0000 7efe8dffb640 10 client.17658 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efe8dffb640 1 -- 172.21.15.77:0/2595668051 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596f2f78d70 con 0x5596f302e710 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efe8dffb640 10 client.17658 handle_command_reply: tid=0 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 client.17658 shutdown 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 msgr2=0x5596f3035420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efe98009250 tx=0x7efe98005d30 comp rx=0 tx=0).stop 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 shutdown_connections 2024-09-15T22:20:58.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5596f302e710 0x5596f309b410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f301eb80 0x5596f3035ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596f3015350 0x5596f3035960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 --2- 172.21.15.77:0/2595668051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5596f3013de0 0x5596f3035420 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 >> 172.21.15.77:0/2595668051 conn(0x5596f2fd4310 msgr2=0x5596f301a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efe8dffb640 0 client.17658 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 shutdown_connections 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 1 -- 172.21.15.77:0/2595668051 wait complete. 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.162+0000 7efeaee13740 20 client.17658 trim_cache size 0 max 16384 2024-09-15T22:20:58.170 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:58.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 msgr2=0x7efe940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 0x7efe940a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efea0004af0 tx=0x7efea00095c0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 msgr2=0x7efea8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9f0 tx=0x7efe9000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 shutdown_connections 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe940a33b0 0x7efe940a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efea8106f20 0x7efea8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efea81027a0 0x7efea8074a30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 --2- 172.21.15.77:0/1762728928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efea8101df0 0x7efea80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 >> 172.21.15.77:0/1762728928 conn(0x7efea80fd8f0 msgr2=0x7efea80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 shutdown_connections 2024-09-15T22:20:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:58.163+0000 7efeaca3a640 1 -- 172.21.15.77:0/1762728928 wait complete. 2024-09-15T22:20:58.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 668084 8.04 MB/sec execute 307 sec latency 36.624 ms 2024-09-15T22:20:58.791 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:20:58 smithi077 ceph-mon[30980]: pgmap v263: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-15T22:20:58.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:20:58 smithi196 ceph-mon[35109]: pgmap v263: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-15T22:20:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:20:58 smithi117 ceph-mon[35021]: pgmap v263: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-09-15T22:20:58.879 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:20:59.146 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:59.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 670071 8.04 MB/sec execute 308 sec latency 27.368 ms 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 -- 172.21.15.77:0/3373982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8101de0 msgr2=0x7fddb81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3373982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8101de0 0x7fddb81021c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802a410 comp rx=0 tx=0).stop 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 -- 172.21.15.77:0/3373982775 shutdown_connections 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3373982775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8106f10 0x7fddb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3373982775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3373982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8101de0 0x7fddb81021c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 -- 172.21.15.77:0/3373982775 >> 172.21.15.77:0/3373982775 conn(0x7fddb80fd8c0 msgr2=0x7fddb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 -- 172.21.15.77:0/3373982775 shutdown_connections 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.801+0000 7fddbdd87640 1 -- 172.21.15.77:0/3373982775 wait complete. 2024-09-15T22:20:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 Processor -- start 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 -- start start 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8111f70 con 0x7fddb8106f10 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8111df0 con 0x7fddb8101de0 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.802+0000 7fddbdd87640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81120f0 con 0x7fddb8102790 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54242/0 (socket says 172.21.15.77:54242) 2024-09-15T22:20:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb77fe640 1 -- 172.21.15.77:0/3735600909 learned_addr learned my addr 172.21.15.77:0/3735600909 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42434/0 (socket says 172.21.15.77:42434) 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 -- 172.21.15.77:0/3735600909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 msgr2=0x7fddb8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 -- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 msgr2=0x7fddb819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 -- 172.21.15.77:0/3735600909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb819d3c0 con 0x7fddb8102790 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb7fff640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb77fe640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb8198950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9f0 tx=0x7fddac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb4ff9640 1 -- 172.21.15.77:0/3735600909 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac00ccc0 con 0x7fddb8102790 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb819d6b0 con 0x7fddb8102790 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb4ff9640 1 -- 172.21.15.77:0/3735600909 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac010910 con 0x7fddb8102790 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.803+0000 7fddb4ff9640 1 -- 172.21.15.77:0/3735600909 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac018960 con 0x7fddb8102790 2024-09-15T22:20:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.804+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a4f90 con 0x7fddb8102790 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.806+0000 7fddb4ff9640 1 -- 172.21.15.77:0/3735600909 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fddac010ac0 con 0x7fddb8102790 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.806+0000 7fddb4ff9640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 0x7fdda00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb77fe640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 0x7fdda00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb4ff9640 1 -- 172.21.15.77:0/3735600909 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fddac0d0630 con 0x7fddb8102790 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 Processor -- start 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 -- start start 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb77fe640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 0x7fdda00a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdda80096c0 tx=0x7fdda8005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffe762b2f0 con 0x55ffe76c55d0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffe75d2cf0 con 0x55ffe76c6b40 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddc0160740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffe76272b0 con 0x55ffe76d0370 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42444/0 (socket says 172.21.15.77:42444) 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb6ffd640 1 -- 172.21.15.77:0/2025527011 learned_addr learned my addr 172.21.15.77:0/2025527011 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb7fff640 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.807+0000 7fddb77fe640 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 -- 172.21.15.77:0/2025527011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 msgr2=0x55ffe76d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 -- 172.21.15.77:0/2025527011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 msgr2=0x55ffe76c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 -- 172.21.15.77:0/2025527011 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe75b73a0 con 0x55ffe76c55d0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fddac0989e0 tx=0x7fddac09cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cea2e8951838836 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fdd9dffb640 1 -- 172.21.15.77:0/2025527011 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0a0930 con 0x55ffe76c55d0 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fdd9dffb640 1 -- 172.21.15.77:0/2025527011 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac0a51f0 con 0x55ffe76c55d0 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.808+0000 7fdd9dffb640 1 -- 172.21.15.77:0/2025527011 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac09bc30 con 0x55ffe76c55d0 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- 172.21.15.77:0/2025527011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 msgr2=0x55ffe76c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fddac0989e0 tx=0x7fddac09cc80 comp rx=0 tx=0).stop 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- 172.21.15.77:0/2025527011 shutdown_connections 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76c5a10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 --2- 172.21.15.77:0/2025527011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- 172.21.15.77:0/2025527011 >> 172.21.15.77:0/2025527011 conn(0x55ffe768e310 msgr2=0x55ffe76c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- 172.21.15.77:0/2025527011 shutdown_connections 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- 172.21.15.77:0/2025527011 wait complete. 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 -- start start 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.809+0000 7fddc0160740 1 Processor -- start 2024-09-15T22:20:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffe76d3010 con 0x55ffe76c55d0 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ffe75e6060 con 0x55ffe76c6b40 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ffe76d3190 con 0x55ffe76d0370 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddc0160740 10 client.0 _ll_register_callbacks cb 0x55ffe73398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54260/0 (socket says 172.21.15.77:54260) 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 -- 172.21.15.77:0/201345685 learned_addr learned my addr 172.21.15.77:0/201345685 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb7fff640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 -- 172.21.15.77:0/201345685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 msgr2=0x55ffe76cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 -- 172.21.15.77:0/201345685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 msgr2=0x55ffe76cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 -- 172.21.15.77:0/201345685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe739c3c0 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb77fe640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fddac0cdff0 tx=0x7fddac0cdea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8a4b17ba6b33a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.810+0000 7fdd96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0b9020 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fddac0b5490 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddac0a51f0 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fddac082050 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 10 client.26783 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffe76deae0 con 0x55ffe76c6b40 2024-09-15T22:20:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac0a51f0 con 0x55ffe76c6b40 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fddac09ba60 con 0x55ffe76c6b40 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 10 client.26783 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 10 client.26783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 20 client.26783 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe76dfe30 0x55ffe76e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 4 client.26783 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.811+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe7638860 con 0x55ffe76dfe30 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.812+0000 7fddb77fe640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe76dfe30 0x55ffe76e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.812+0000 7fddb77fe640 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe76dfe30 0x55ffe76e2220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1094481eabf165d server_cookie=e404aaea4cd2e1aa in_seq=0 out_seq=0 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.812+0000 7fdd96ffd640 10 client.26783 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.812+0000 7fdd96ffd640 1 -- 172.21.15.77:0/201345685 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffe7638860 con 0x55ffe76dfe30 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fdd96ffd640 10 client.26783 handle_command_reply: tid=0 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 client.26783 shutdown 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 msgr2=0x55ffe76cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fddac0cdff0 tx=0x7fddac0cdea0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 shutdown_connections 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe76dfe30 0x55ffe76e2220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c6b40 0x55ffe76cd040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 --2- 172.21.15.77:0/201345685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fdd96ffd640 0 client.26783 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 >> 172.21.15.77:0/201345685 conn(0x55ffe768e310 msgr2=0x55ffe76c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 shutdown_connections 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 1 -- 172.21.15.77:0/201345685 wait complete. 2024-09-15T22:20:59.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.813+0000 7fddc0160740 20 client.26783 trim_cache size 0 max 16384 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 Processor -- start 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 -- start start 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c6b40 0x55ffe76cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81af000 con 0x55ffe76c6b40 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81b0b40 con 0x55ffe76c55d0 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddc0160740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81af6d0 con 0x55ffe76d0370 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c6b40 0x55ffe76cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57466/0 (socket says 172.21.15.77:57466) 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54276/0 (socket says 172.21.15.77:54276) 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb7fff640 1 -- 172.21.15.77:0/781974995 learned_addr learned my addr 172.21.15.77:0/781974995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 -- 172.21.15.77:0/781974995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 msgr2=0x55ffe76e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 -- 172.21.15.77:0/781974995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c6b40 msgr2=0x55ffe76cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.828+0000 7fddb77fe640 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c6b40 0x55ffe76cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.829+0000 7fddb77fe640 1 -- 172.21.15.77:0/781974995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe76cd9c0 con 0x55ffe76c55d0 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.829+0000 7fddb77fe640 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009230 tx=0x7fdda8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7f26b0178fed9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.829+0000 7fdd9d7fa640 1 -- 172.21.15.77:0/781974995 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8035040 con 0x55ffe76c55d0 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.829+0000 7fdd9d7fa640 1 -- 172.21.15.77:0/781974995 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda8058070 con 0x55ffe76c55d0 2024-09-15T22:20:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.829+0000 7fdd9d7fa640 1 -- 172.21.15.77:0/781974995 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8004de0 con 0x55ffe76c55d0 2024-09-15T22:20:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- 172.21.15.77:0/781974995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 msgr2=0x55ffe76dc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009230 tx=0x7fdda8040000 comp rx=0 tx=0).stop 2024-09-15T22:20:59.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- 172.21.15.77:0/781974995 shutdown_connections 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76d0370 0x55ffe76e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c6b40 0x55ffe76cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 --2- 172.21.15.77:0/781974995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76c55d0 0x55ffe76dc430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- 172.21.15.77:0/781974995 >> 172.21.15.77:0/781974995 conn(0x55ffe768e310 msgr2=0x55ffe76cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- 172.21.15.77:0/781974995 shutdown_connections 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- 172.21.15.77:0/781974995 wait complete. 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 -- start start 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.830+0000 7fddc0160740 1 Processor -- start 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76c6b40 0x55ffe7743050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76d0370 0x55ffe7743590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81b0470 con 0x55ffe76c55d0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81afda0 con 0x55ffe76d0370 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81ad000 con 0x55ffe76c6b40 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddc0160740 10 client.0 _ll_register_callbacks cb 0x7fdda8040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42474/0 (socket says 172.21.15.77:42474) 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 -- 172.21.15.77:0/1320576634 learned_addr learned my addr 172.21.15.77:0/1320576634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76c6b40 0x55ffe7743050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb7fff640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76d0370 0x55ffe7743590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 -- 172.21.15.77:0/1320576634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76c6b40 msgr2=0x55ffe7743050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76c6b40 0x55ffe7743050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 -- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76d0370 msgr2=0x55ffe7743590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76d0370 0x55ffe7743590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 -- 172.21.15.77:0/1320576634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7743c80 con 0x55ffe76c55d0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fddb77fe640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdda803f040 tx=0x7fdda80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccffc439a35b87e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.831+0000 7fdd96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8035050 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fdda8004be0 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda8058070 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdda8004de0 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 10 client.17676 fetch_fsmap learned FSMap version 34 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffe774eb70 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdda8058070 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdda8003ba0 con 0x55ffe76c55d0 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 10 client.17676 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 20 client.17676 populate_metadata read hostname 'smithi077' 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe774f150 0x55ffe7751570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 4 client.17676 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.832+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffe7632d70 con 0x55ffe774f150 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe774f150 0x55ffe7751570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fdd96ffd640 10 client.17676 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddb6ffd640 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe774f150 0x55ffe7751570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bb11c6562bbecf6 server_cookie=d84d01c6ee2ec6e2 in_seq=0 out_seq=0 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fdd96ffd640 1 -- 172.21.15.77:0/1320576634 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffe7632d70 con 0x55ffe774f150 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fdd96ffd640 10 client.17676 handle_command_reply: tid=0 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 client.17676 shutdown 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 msgr2=0x55ffe76e5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdda803f040 tx=0x7fdda80040e0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 shutdown_connections 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ffe774f150 0x55ffe7751570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ffe76d0370 0x55ffe7743590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ffe76c6b40 0x55ffe7743050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 --2- 172.21.15.77:0/1320576634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffe76c55d0 0x55ffe76e5800 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 >> 172.21.15.77:0/1320576634 conn(0x55ffe768e310 msgr2=0x55ffe76ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fdd96ffd640 0 client.17676 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 shutdown_connections 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.833+0000 7fddc0160740 1 -- 172.21.15.77:0/1320576634 wait complete. 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddc0160740 20 client.17676 trim_cache size 0 max 16384 2024-09-15T22:20:59.841 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 msgr2=0x7fdda00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 0x7fdda00a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdda80096c0 tx=0x7fdda8005a20 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 msgr2=0x7fddb8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb8198950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9f0 tx=0x7fddac00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 shutdown_connections 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdda00a33b0 0x7fdda00a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddb8106f10 0x7fddb819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fddb8102790 0x7fddb8198950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 --2- 172.21.15.77:0/3735600909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8101de0 0x7fddb8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 >> 172.21.15.77:0/3735600909 conn(0x7fddb80fd8c0 msgr2=0x7fddb80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 shutdown_connections 2024-09-15T22:20:59.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:20:59.834+0000 7fddbdd87640 1 -- 172.21.15.77:0/3735600909 wait complete. 2024-09-15T22:21:00.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 671972 8.05 MB/sec execute 309 sec latency 40.574 ms 2024-09-15T22:21:00.552 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:00 smithi077 ceph-mon[30980]: pgmap v264: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-15T22:21:00.585 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:21:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:00 smithi196 ceph-mon[35109]: pgmap v264: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-15T22:21:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:00 smithi117 ceph-mon[35021]: pgmap v264: 129 pgs: 129 active+clean; 457 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-09-15T22:21:00.891 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 -- 172.21.15.77:0/3051528985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 msgr2=0x7f975c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 --2- 172.21.15.77:0/3051528985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c100ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9750009900 tx=0x7f975002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 -- 172.21.15.77:0/3051528985 shutdown_connections 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 --2- 172.21.15.77:0/3051528985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c101430 0x7f975c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 --2- 172.21.15.77:0/3051528985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c100ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 --2- 172.21.15.77:0/3051528985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c106a90 0x7f975c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 -- 172.21.15.77:0/3051528985 >> 172.21.15.77:0/3051528985 conn(0x7f975c0fc7d0 msgr2=0x7f975c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 -- 172.21.15.77:0/3051528985 shutdown_connections 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.379+0000 7f97625fd640 1 -- 172.21.15.77:0/3051528985 wait complete. 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 Processor -- start 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 -- start start 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c101430 0x7f975c1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f975c10fd80 con 0x7f975c106a90 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f975c10fc00 con 0x7f975c100a90 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.380+0000 7f97625fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f975c10ff00 con 0x7f975c101430 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42486/0 (socket says 172.21.15.77:42486) 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 -- 172.21.15.77:0/687051335 learned_addr learned my addr 172.21.15.77:0/687051335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54314/0 (socket says 172.21.15.77:54314) 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f975b7fe640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c101430 0x7f975c1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 -- 172.21.15.77:0/687051335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c101430 msgr2=0x7f975c1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c101430 0x7f975c1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 -- 172.21.15.77:0/687051335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 msgr2=0x7f975c196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 -- 172.21.15.77:0/687051335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c19b230 con 0x7f975c106a90 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f9760b9a640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9754010fb0 tx=0x7f975400e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f97597fa640 1 -- 172.21.15.77:0/687051335 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975401a070 con 0x7f975c106a90 2024-09-15T22:21:01.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c19b520 con 0x7f975c106a90 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.381+0000 7f97597fa640 1 -- 172.21.15.77:0/687051335 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754003e50 con 0x7f975c106a90 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.382+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1a2e00 con 0x7f975c106a90 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.382+0000 7f97597fa640 1 -- 172.21.15.77:0/687051335 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97540047e0 con 0x7f975c106a90 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.384+0000 7f97597fa640 1 -- 172.21.15.77:0/687051335 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9754014050 con 0x7f975c106a90 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 Processor -- start 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 -- start start 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 0x561985d35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561985c9b2f0 con 0x561985d36b40 2024-09-15T22:21:01.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561985c42cf0 con 0x561985d355d0 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97649d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561985c972b0 con 0x561985d40370 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97597fa640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 0x7f97440a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 0x561985d35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42500/0 (socket says 172.21.15.77:42500) 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f975b7fe640 1 -- 172.21.15.77:0/4121984353 learned_addr learned my addr 172.21.15.77:0/4121984353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f975b7fe640 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f97597fa640 1 -- 172.21.15.77:0/687051335 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f97540d1fd0 con 0x7f975c106a90 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.385+0000 7f9760b9a640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 0x7f97440a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 -- 172.21.15.77:0/4121984353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 msgr2=0x561985d42760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d42760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 -- 172.21.15.77:0/4121984353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 msgr2=0x561985d35a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 0x561985d35a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 -- 172.21.15.77:0/4121984353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561985c273a0 con 0x561985d36b40 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975bfff640 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 0x561985d35a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f9760b9a640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 0x7f97440a9c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97540a6660 tx=0x7f9754004270 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f975b7fe640 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9750031470 tx=0x7f9750031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196084e1b9e66b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f97427fc640 1 -- 172.21.15.77:0/4121984353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750035040 con 0x561985d36b40 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f97427fc640 1 -- 172.21.15.77:0/4121984353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750003ad0 con 0x561985d36b40 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.386+0000 7f97427fc640 1 -- 172.21.15.77:0/4121984353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750031a00 con 0x561985d36b40 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- 172.21.15.77:0/4121984353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 msgr2=0x561985d35090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9750031470 tx=0x7f9750031400 comp rx=0 tx=0).stop 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- 172.21.15.77:0/4121984353 shutdown_connections 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d42760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d355d0 0x561985d35a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 --2- 172.21.15.77:0/4121984353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d36b40 0x561985d35090 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- 172.21.15.77:0/4121984353 >> 172.21.15.77:0/4121984353 conn(0x561985cfe310 msgr2=0x561985d31c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- 172.21.15.77:0/4121984353 shutdown_connections 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- 172.21.15.77:0/4121984353 wait complete. 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.387+0000 7f97649d6740 1 -- start start 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 Processor -- start 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561985d43010 con 0x561985d355d0 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561985c56060 con 0x561985d36b40 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561985d43190 con 0x561985d40370 2024-09-15T22:21:01.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f97649d6740 10 client.0 _ll_register_callbacks cb 0x5619859a98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42504/0 (socket says 172.21.15.77:42504) 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f975b7fe640 1 -- 172.21.15.77:0/23173270 learned_addr learned my addr 172.21.15.77:0/23173270 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f975bfff640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.388+0000 7f975b7fe640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 -- 172.21.15.77:0/23173270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 msgr2=0x561985d3d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 -- 172.21.15.77:0/23173270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 msgr2=0x561985d3fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 -- 172.21.15.77:0/23173270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561985a0c3c0 con 0x561985d36b40 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f975bfff640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d3d090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f974c0126b0 tx=0x7f974c012ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf33343fb8c0166 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c004030 con 0x561985d36b40 2024-09-15T22:21:01.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f974c0041d0 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9750003ad0 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f974c0043d0 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f97649d6740 10 client.26801 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561985d4eba0 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750003ad0 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f974c003980 con 0x561985d36b40 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f97649d6740 10 client.26801 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.389+0000 7f97649d6740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f97649d6740 20 client.26801 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985d50060 0x561985d52490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f97649d6740 4 client.26801 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561985ca8860 con 0x561985d50060 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f975b7fe640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985d50060 0x561985d52490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f973b7fe640 10 client.26801 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.390+0000 7f975b7fe640 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985d50060 0x561985d52490 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e4eca7ad47d1bb server_cookie=e25854fca10980cc in_seq=0 out_seq=0 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f973b7fe640 1 -- 172.21.15.77:0/23173270 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561985ca8860 con 0x561985d50060 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f973b7fe640 10 client.26801 handle_command_reply: tid=0 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 client.26801 shutdown 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 msgr2=0x561985d3d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d3d090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f974c0126b0 tx=0x7f974c012ab0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 shutdown_connections 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985d50060 0x561985d52490 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d3d090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 --2- 172.21.15.77:0/23173270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 >> 172.21.15.77:0/23173270 conn(0x561985cfe310 msgr2=0x561985d37960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f973b7fe640 0 client.26801 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 shutdown_connections 2024-09-15T22:21:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 1 -- 172.21.15.77:0/23173270 wait complete. 2024-09-15T22:21:01.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.391+0000 7f97649d6740 20 client.26801 trim_cache size 0 max 16384 2024-09-15T22:21:01.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 Processor -- start 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 -- start start 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d4c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1acd60 con 0x561985d355d0 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ae8a0 con 0x561985d36b40 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f97649d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ad430 con 0x561985d40370 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.407+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d4c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42510/0 (socket says 172.21.15.77:42510) 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 -- 172.21.15.77:0/3411512457 learned_addr learned my addr 172.21.15.77:0/3411512457 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 -- 172.21.15.77:0/3411512457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 msgr2=0x561985d3c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 -- 172.21.15.77:0/3411512457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 msgr2=0x561985d4c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d4c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 -- 172.21.15.77:0/3411512457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561985d36760 con 0x561985d355d0 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975bfff640 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d4c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f975b7fe640 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500315e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e56eba11fcdf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f9741ffb640 1 -- 172.21.15.77:0/3411512457 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750035070 con 0x561985d355d0 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f9741ffb640 1 -- 172.21.15.77:0/3411512457 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97500054d0 con 0x561985d355d0 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.408+0000 7f9741ffb640 1 -- 172.21.15.77:0/3411512457 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750031c10 con 0x561985d355d0 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 -- 172.21.15.77:0/3411512457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 msgr2=0x561985d3c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9750000c00 tx=0x7f97500315e0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 -- 172.21.15.77:0/3411512457 shutdown_connections 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d40370 0x561985d3c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d4c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 --2- 172.21.15.77:0/3411512457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d355d0 0x561985d3c950 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 -- 172.21.15.77:0/3411512457 >> 172.21.15.77:0/3411512457 conn(0x561985cfe310 msgr2=0x561985d3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 -- 172.21.15.77:0/3411512457 shutdown_connections 2024-09-15T22:21:01.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.409+0000 7f97649d6740 1 -- 172.21.15.77:0/3411512457 wait complete. 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 -- start start 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 Processor -- start 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d355d0 0x561985d43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 0x561985db8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ae1d0 con 0x561985d40370 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1adb00 con 0x561985d36b40 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1aad60 con 0x561985d355d0 2024-09-15T22:21:01.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f97649d6740 10 client.0 _ll_register_callbacks cb 0x7f974c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d355d0 0x561985d43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 0x561985db8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 0x561985db8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42522/0 (socket says 172.21.15.77:42522) 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54336/0 (socket says 172.21.15.77:54336) 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.410+0000 7f975b7fe640 1 -- 172.21.15.77:0/331076024 learned_addr learned my addr 172.21.15.77:0/331076024 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 -- 172.21.15.77:0/331076024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d355d0 msgr2=0x561985d43980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d355d0 0x561985d43980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 -- 172.21.15.77:0/331076024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 msgr2=0x561985db8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 0x561985db8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 -- 172.21.15.77:0/331076024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561985db92f0 con 0x561985d36b40 2024-09-15T22:21:01.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f975bfff640 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f974c003da0 tx=0x7f974c003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d42814cdd651d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f973b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c02e060 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f974c003800 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97500054d0 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f974c003a00 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f97649d6740 10 client.26807 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.411+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561985dba620 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97500054d0 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f974c005270 con 0x561985d36b40 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 10 client.26807 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 20 client.26807 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985dbadd0 0x561985dbd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 4 client.26807 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561985ca2d70 con 0x561985dbadd0 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.412+0000 7f975b7fe640 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985dbadd0 0x561985dbd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f973b7fe640 10 client.26807 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f975b7fe640 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985dbadd0 0x561985dbd1f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd4ea06094d796e server_cookie=a572725fe39ea500 in_seq=0 out_seq=0 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f973b7fe640 1 -- 172.21.15.77:0/331076024 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561985ca2d70 con 0x561985dbadd0 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f973b7fe640 10 client.26807 handle_command_reply: tid=0 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f97649d6740 1 client.26807 shutdown 2024-09-15T22:21:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 msgr2=0x561985d43ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.413+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f974c003da0 tx=0x7f974c003f90 comp rx=0 tx=0).stop 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 shutdown_connections 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561985dbadd0 0x561985dbd1f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561985d40370 0x561985db8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561985d36b40 0x561985d43ec0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f973b7fe640 0 client.26807 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 --2- 172.21.15.77:0/331076024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561985d355d0 0x561985d43980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 >> 172.21.15.77:0/331076024 conn(0x561985cfe310 msgr2=0x561985d3c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 shutdown_connections 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 1 -- 172.21.15.77:0/331076024 wait complete. 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97649d6740 20 client.26807 trim_cache size 0 max 16384 2024-09-15T22:21:01.422 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 msgr2=0x7f97440a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 0x7f97440a9c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97540a6660 tx=0x7f9754004270 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 msgr2=0x7f975c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.414+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9754010fb0 tx=0x7f975400e490 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 shutdown_connections 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97440a77c0 0x7f97440a9c80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f975c106a90 0x7f975c19ab50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975c101430 0x7f975c1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 --2- 172.21.15.77:0/687051335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f975c100a90 0x7f975c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 >> 172.21.15.77:0/687051335 conn(0x7f975c0fc7d0 msgr2=0x7f975c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 shutdown_connections 2024-09-15T22:21:01.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:01.415+0000 7f97625fd640 1 -- 172.21.15.77:0/687051335 wait complete. 2024-09-15T22:21:01.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 672327 8.03 MB/sec execute 310 sec latency 98.004 ms 2024-09-15T22:21:02.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 674896 8.06 MB/sec execute 311 sec latency 56.117 ms 2024-09-15T22:21:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:02 smithi077 ceph-mon[30980]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-09-15T22:21:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:02 smithi196 ceph-mon[35109]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-09-15T22:21:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:02 smithi117 ceph-mon[35021]: pgmap v265: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-09-15T22:21:03.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:21:03.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:21:03.423 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:03.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 676683 8.05 MB/sec execute 312 sec latency 30.673 ms 2024-09-15T22:21:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 -- 172.21.15.77:0/3759390828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102570 msgr2=0x7f0ed81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 --2- 172.21.15.77:0/3759390828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102570 0x7f0ed81029b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009900 tx=0x7f0ec802a480 comp rx=0 tx=0).stop 2024-09-15T22:21:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 -- 172.21.15.77:0/3759390828 shutdown_connections 2024-09-15T22:21:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 --2- 172.21.15.77:0/3759390828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102ef0 0x7f0ed810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 --2- 172.21.15.77:0/3759390828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102570 0x7f0ed81029b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 --2- 172.21.15.77:0/3759390828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 0x7f0ed8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 -- 172.21.15.77:0/3759390828 >> 172.21.15.77:0/3759390828 conn(0x7f0ed80fc7d0 msgr2=0x7f0ed80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 -- 172.21.15.77:0/3759390828 shutdown_connections 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.993+0000 7f0ee008c640 1 -- 172.21.15.77:0/3759390828 wait complete. 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.994+0000 7f0ee008c640 1 Processor -- start 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.994+0000 7f0ee008c640 1 -- start start 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.994+0000 7f0ee008c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 0x7f0ed81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.994+0000 7f0ee008c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102570 0x7f0ed81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.994+0000 7f0ee008c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ee008c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8110f80 con 0x7f0ed8102ef0 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ee008c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8110e00 con 0x7f0ed8100e20 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ee008c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8111100 con 0x7f0ed8102570 2024-09-15T22:21:04.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42550/0 (socket says 172.21.15.77:42550) 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 -- 172.21.15.77:0/578785998 learned_addr learned my addr 172.21.15.77:0/578785998 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0edd627640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102570 0x7f0ed81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0edde28640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 0x7f0ed81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 -- 172.21.15.77:0/578785998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102570 msgr2=0x7f0ed81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102570 0x7f0ed81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 -- 172.21.15.77:0/578785998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 msgr2=0x7f0ed81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 0x7f0ed81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 -- 172.21.15.77:0/578785998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed819d460 con 0x7f0ed8102ef0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ede629640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000eb10 tx=0x7f0ed000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0eceffd640 1 -- 172.21.15.77:0/578785998 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0004b20 con 0x7f0ed8102ef0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed819d750 con 0x7f0ed8102ef0 2024-09-15T22:21:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0eceffd640 1 -- 172.21.15.77:0/578785998 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed0004cc0 con 0x7f0ed8102ef0 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0eceffd640 1 -- 172.21.15.77:0/578785998 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0010760 con 0x7f0ed8102ef0 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.995+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed81a5030 con 0x7f0ed8102ef0 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0eceffd640 1 -- 172.21.15.77:0/578785998 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ed0005020 con 0x7f0ed8102ef0 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 Processor -- start 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 -- start start 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7bf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f17f713280 con 0x55f17f7ae3a0 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f17f6bab10 con 0x55f17f7ada80 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0ee2465740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f17f70f240 con 0x55f17f7bd150 2024-09-15T22:21:04.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.998+0000 7f0edde28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42562/0 (socket says 172.21.15.77:42562) 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 -- 172.21.15.77:0/571042571 learned_addr learned my addr 172.21.15.77:0/571042571 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edd627640 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7bf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0eceffd640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 0x7f0ec40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edd627640 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0ede629640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 0x7f0ec40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 -- 172.21.15.77:0/571042571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 msgr2=0x55f17f7bf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7bf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 -- 172.21.15.77:0/571042571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 msgr2=0x55f17f7ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 -- 172.21.15.77:0/571042571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17f6c71f0 con 0x55f17f7ae3a0 2024-09-15T22:21:04.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0eceffd640 1 -- 172.21.15.77:0/578785998 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f0ed00d0750 con 0x7f0ed8102ef0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0edde28640 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400bd10 tx=0x7f0ed40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c023c49bda498 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0ede629640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 0x7f0ec40a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ed009c630 tx=0x7f0ed0091b30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0ec3fff640 1 -- 172.21.15.77:0/571042571 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004030 con 0x55f17f7ae3a0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:03.999+0000 7f0ec3fff640 1 -- 172.21.15.77:0/571042571 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed40041d0 con 0x55f17f7ae3a0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ec3fff640 1 -- 172.21.15.77:0/571042571 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4011840 con 0x55f17f7ae3a0 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 -- 172.21.15.77:0/571042571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 msgr2=0x55f17f7bcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400bd10 tx=0x7f0ed40107a0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 -- 172.21.15.77:0/571042571 shutdown_connections 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7bf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7bcc10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 --2- 172.21.15.77:0/571042571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 -- 172.21.15.77:0/571042571 >> 172.21.15.77:0/571042571 conn(0x55f17f7aa0f0 msgr2=0x55f17f7aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.000+0000 7f0ee2465740 1 -- 172.21.15.77:0/571042571 shutdown_connections 2024-09-15T22:21:04.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 -- 172.21.15.77:0/571042571 wait complete. 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 -- start start 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 Processor -- start 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7ada80 0x55f17f7ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f17f7bfcc0 con 0x55f17f7bd150 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f17f6cdf10 con 0x55f17f7ae3a0 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f17f7bfe40 con 0x55f17f7ada80 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0ee2465740 10 client.0 _ll_register_callbacks cb 0x55f17f696a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0edd627640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0edd627640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42564/0 (socket says 172.21.15.77:42564) 2024-09-15T22:21:04.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.001+0000 7f0edd627640 1 -- 172.21.15.77:0/714202192 learned_addr learned my addr 172.21.15.77:0/714202192 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7ada80 0x55f17f7ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edde28640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 -- 172.21.15.77:0/714202192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7ada80 msgr2=0x55f17f7ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7ada80 0x55f17f7ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 -- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 msgr2=0x55f17f7ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 -- 172.21.15.77:0/714202192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17f4846a0 con 0x55f17f7bd150 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edde28640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0edd627640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76804946669603ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8035040 con 0x55f17f7bd150 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f0ec8032e00 con 0x55f17f7bd150 2024-09-15T22:21:04.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed40041d0 con 0x55f17f7bd150 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ec80035f0 con 0x55f17f7bd150 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ee2465740 10 client.17706 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.002+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f17f7cb9c0 con 0x55f17f7bd150 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed40041d0 con 0x55f17f7bd150 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0ec803d7a0 con 0x55f17f7bd150 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 10 client.17706 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 20 client.17706 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f7ccdd0 0x55f17f7cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 4 client.17706 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17f7207f0 con 0x55f17f7ccdd0 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0edde28640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f7ccdd0 0x55f17f7cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0ec17fa640 10 client.17706 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:04.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.003+0000 7f0edde28640 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f7ccdd0 0x55f17f7cf1c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d07b220b475752 server_cookie=b5018f279eb75ec6 in_seq=0 out_seq=0 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ec17fa640 1 -- 172.21.15.77:0/714202192 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f17f7207f0 con 0x55f17f7ccdd0 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ec17fa640 10 client.17706 handle_command_reply: tid=0 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 client.17706 shutdown 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 msgr2=0x55f17f7bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec802a480 comp rx=0 tx=0).stop 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 shutdown_connections 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f7ccdd0 0x55f17f7cf1c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7bd150 0x55f17f7bab70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 --2- 172.21.15.77:0/714202192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7ada80 0x55f17f7ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 >> 172.21.15.77:0/714202192 conn(0x55f17f7aa0f0 msgr2=0x55f17f7afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ec17fa640 0 client.17706 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 shutdown_connections 2024-09-15T22:21:04.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 1 -- 172.21.15.77:0/714202192 wait complete. 2024-09-15T22:21:04.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.004+0000 7f0ee2465740 20 client.17706 trim_cache size 0 max 16384 2024-09-15T22:21:04.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 Processor -- start 2024-09-15T22:21:04.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 -- start start 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81acde0 con 0x55f17f7ada80 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ae920 con 0x55f17f7ae3a0 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0ee2465740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ad4b0 con 0x55f17f7bd150 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edd627640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edd627640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42580/0 (socket says 172.21.15.77:42580) 2024-09-15T22:21:04.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edd627640 1 -- 172.21.15.77:0/4053049696 learned_addr learned my addr 172.21.15.77:0/4053049696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edd627640 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 -- 172.21.15.77:0/4053049696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 msgr2=0x55f17f7b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 -- 172.21.15.77:0/4053049696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 msgr2=0x55f17f7d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edde28640 1 -- 172.21.15.77:0/4053049696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17f7b0570 con 0x55f17f7ae3a0 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.023+0000 7f0edd627640 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.024+0000 7f0edde28640 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7d4f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4010ca0 tx=0x7f0ed4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9046caf26e532122 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.024+0000 7f0ec37fe640 1 -- 172.21.15.77:0/4053049696 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40037f0 con 0x55f17f7ae3a0 2024-09-15T22:21:04.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.024+0000 7f0ec37fe640 1 -- 172.21.15.77:0/4053049696 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed4003990 con 0x55f17f7ae3a0 2024-09-15T22:21:04.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.024+0000 7f0ec37fe640 1 -- 172.21.15.77:0/4053049696 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4011bb0 con 0x55f17f7ae3a0 2024-09-15T22:21:04.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- 172.21.15.77:0/4053049696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 msgr2=0x55f17f7d4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7d4f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4010ca0 tx=0x7f0ed4011440 comp rx=0 tx=0).stop 2024-09-15T22:21:04.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- 172.21.15.77:0/4053049696 shutdown_connections 2024-09-15T22:21:04.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ae3a0 0x55f17f7d4f90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 --2- 172.21.15.77:0/4053049696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ada80 0x55f17f7d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- 172.21.15.77:0/4053049696 >> 172.21.15.77:0/4053049696 conn(0x55f17f7aa0f0 msgr2=0x55f17f7bc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- 172.21.15.77:0/4053049696 shutdown_connections 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- 172.21.15.77:0/4053049696 wait complete. 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 -- start start 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 Processor -- start 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.025+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81ae250 con 0x55f17f7ae3a0 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81adb80 con 0x55f17f7ada80 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81aade0 con 0x55f17f7bd150 2024-09-15T22:21:04.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 10 client.0 _ll_register_callbacks cb 0x7f0ec8032880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edde28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edde28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42594/0 (socket says 172.21.15.77:42594) 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:54398/0 (socket says 172.21.15.77:54398) 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edde28640 1 -- 172.21.15.77:0/1076068273 learned_addr learned my addr 172.21.15.77:0/1076068273 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 -- 172.21.15.77:0/1076068273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 msgr2=0x55f17f7c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 -- 172.21.15.77:0/1076068273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 msgr2=0x55f17f7c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7c18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 -- 172.21.15.77:0/1076068273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17f7c23f0 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0edd627640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80327c0 tx=0x7f0ec8009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e5ef9f7e194e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8035020 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f0ec8032230 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.026+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed4003990 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ec8032430 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 10 client.26825 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f17f838800 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed4003990 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0ec8004bc0 con 0x55f17f7ada80 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 10 client.26825 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:04.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 20 client.26825 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f838ec0 0x55f17f83b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 4 client.26825 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f17f71ad00 con 0x55f17f838ec0 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.027+0000 7f0edde28640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f838ec0 0x55f17f83b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ec17fa640 10 client.26825 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0edde28640 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f838ec0 0x55f17f83b2b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e99435c65ff6f5 server_cookie=dde56eb88bfe2094 in_seq=0 out_seq=0 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ec17fa640 1 -- 172.21.15.77:0/1076068273 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f17f71ad00 con 0x55f17f838ec0 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ec17fa640 10 client.26825 handle_command_reply: tid=0 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ee2465740 1 client.26825 shutdown 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 msgr2=0x55f17f7c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.028+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80327c0 tx=0x7f0ec8009830 comp rx=0 tx=0).stop 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 shutdown_connections 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f17f838ec0 0x55f17f83b2b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f17f7bd150 0x55f17f7c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f17f7ae3a0 0x55f17f7c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 --2- 172.21.15.77:0/1076068273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f17f7ada80 0x55f17f7c55b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ec17fa640 0 client.26825 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 >> 172.21.15.77:0/1076068273 conn(0x55f17f7aa0f0 msgr2=0x55f17f7ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 shutdown_connections 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 1 -- 172.21.15.77:0/1076068273 wait complete. 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.029+0000 7f0ee2465740 20 client.26825 trim_cache size 0 max 16384 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "bfd4695a-763e-4fa0-b35c-3ba8a6ffa66b", 2024-09-15T22:21:04.036 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:21:04.037 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 msgr2=0x7f0ec40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 0x7f0ec40a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ed009c630 tx=0x7f0ed0091b30 comp rx=0 tx=0).stop 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 msgr2=0x7f0ed819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000eb10 tx=0x7f0ed000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 shutdown_connections 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0ec40a33b0 0x7f0ec40a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ed8102ef0 0x7f0ed819cd80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed8102570 0x7f0ed81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 --2- 172.21.15.77:0/578785998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed8100e20 0x7f0ed81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 >> 172.21.15.77:0/578785998 conn(0x7f0ed80fc7d0 msgr2=0x7f0ed806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:04.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 shutdown_connections 2024-09-15T22:21:04.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:04.030+0000 7f0ee008c640 1 -- 172.21.15.77:0/578785998 wait complete. 2024-09-15T22:21:04.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 678929 8.07 MB/sec execute 313 sec latency 43.346 ms 2024-09-15T22:21:04.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:04 smithi077 ceph-mon[30980]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:04 smithi196 ceph-mon[35109]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:04 smithi117 ceph-mon[35021]: pgmap v266: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:05.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:21:05.476 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:05.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 681186 8.08 MB/sec execute 314 sec latency 50.011 ms 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:21:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:21:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:21:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:05.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 -- 172.21.15.77:0/1860514817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 msgr2=0x7fb73c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 --2- 172.21.15.77:0/1860514817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1029b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009900 tx=0x7fb72c02a480 comp rx=0 tx=0).stop 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 -- 172.21.15.77:0/1860514817 shutdown_connections 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 --2- 172.21.15.77:0/1860514817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 0x7fb73c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 --2- 172.21.15.77:0/1860514817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1029b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 --2- 172.21.15.77:0/1860514817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 -- 172.21.15.77:0/1860514817 >> 172.21.15.77:0/1860514817 conn(0x7fb73c0fc7d0 msgr2=0x7fb73c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 -- 172.21.15.77:0/1860514817 shutdown_connections 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.974+0000 7fb744229640 1 -- 172.21.15.77:0/1860514817 wait complete. 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 Processor -- start 2024-09-15T22:21:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 -- start start 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 0x7fb73c1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c110f50 con 0x7fb73c102570 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c110dd0 con 0x7fb73c102ef0 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb744229640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1110d0 con 0x7fb73c100e20 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42606/0 (socket says 172.21.15.77:42606) 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb7417c4640 1 -- 172.21.15.77:0/1033159367 learned_addr learned my addr 172.21.15.77:0/1033159367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb7427c6640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 0x7fb73c1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 -- 172.21.15.77:0/1033159367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 msgr2=0x7fb73c1a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 0x7fb73c1a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 -- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 msgr2=0x7fb73c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb741fc5640 1 -- 172.21.15.77:0/1033159367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c1a4960 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.975+0000 7fb7417c4640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb741fc5640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c1993a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb73800e9f0 tx=0x7fb73800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb732ffd640 1 -- 172.21.15.77:0/1033159367 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800ccc0 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb732ffd640 1 -- 172.21.15.77:0/1033159367 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb738010910 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c1a4b30 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb732ffd640 1 -- 172.21.15.77:0/1033159367 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738018960 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.976+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c1a5040 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.978+0000 7fb732ffd640 1 -- 172.21.15.77:0/1033159367 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb738010ac0 con 0x7fb73c100e20 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.978+0000 7fb746602740 1 Processor -- start 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 -- start start 2024-09-15T22:21:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a356c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a356ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5578a34c72e0 con 0x5578a35614a0 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578a346ece0 con 0x5578a3561dc0 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb746602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578a34c32a0 con 0x5578a356c680 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42610/0 (socket says 172.21.15.77:42610) 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 -- 172.21.15.77:0/3875528830 learned_addr learned my addr 172.21.15.77:0/3875528830 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7417c4640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a356ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb741fc5640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a356c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb732ffd640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 0x7fb7280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 -- 172.21.15.77:0/3875528830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 msgr2=0x5578a356ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a356ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 -- 172.21.15.77:0/3875528830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 msgr2=0x5578a356c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a356c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 -- 172.21.15.77:0/3875528830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a3453440 con 0x5578a35614a0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7417c4640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a356ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb7427c6640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 0x7fb7280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.979+0000 7fb732ffd640 1 -- 172.21.15.77:0/1033159367 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb7380d0480 con 0x7fb73c100e20 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb7427c6640 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb734011590 tx=0x7fb734011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9475949d150bb035 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb727fff640 1 -- 172.21.15.77:0/3875528830 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73401f020 con 0x5578a35614a0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb727fff640 1 -- 172.21.15.77:0/3875528830 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73400fd40 con 0x5578a35614a0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb727fff640 1 -- 172.21.15.77:0/3875528830 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734017050 con 0x5578a35614a0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb746602740 1 -- 172.21.15.77:0/3875528830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 msgr2=0x5578a3561880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb7427c6640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 0x7fb7280a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb734020000 tx=0x7fb734023e30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.980+0000 7fb746602740 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb734011590 tx=0x7fb734011a50 comp rx=0 tx=0).stop 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- 172.21.15.77:0/3875528830 shutdown_connections 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a356ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a356c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- 172.21.15.77:0/3875528830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a3561880 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- 172.21.15.77:0/3875528830 >> 172.21.15.77:0/3875528830 conn(0x5578a355da30 msgr2=0x5578a355de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- 172.21.15.77:0/3875528830 shutdown_connections 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- 172.21.15.77:0/3875528830 wait complete. 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- start start 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 Processor -- start 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a356bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5578a356f250 con 0x5578a3561dc0 2024-09-15T22:21:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578a3482050 con 0x5578a35614a0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.981+0000 7fb746602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5578a356f3d0 con 0x5578a356c680 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb746602740 10 client.0 _ll_register_callbacks cb 0x5578a34c2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42626/0 (socket says 172.21.15.77:42626) 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 -- 172.21.15.77:0/2006201392 learned_addr learned my addr 172.21.15.77:0/2006201392 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7427c6640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb741fc5640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a356bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 -- 172.21.15.77:0/2006201392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 msgr2=0x5578a35696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 -- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 msgr2=0x5578a356bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a356bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 -- 172.21.15.77:0/2006201392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a32383c0 con 0x5578a3561dc0 2024-09-15T22:21:06.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7417c4640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb72c000c00 tx=0x7fb72c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bc23f27cc9898b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7257fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c035040 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb72c003e20 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb73400fd40 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb72c031540 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb746602740 10 client.17724 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.982+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5578a357ab20 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73400fd40 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb72c046860 con 0x5578a3561dc0 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 10 client.17724 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:06.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 20 client.17724 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a357e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 4 client.17724 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a34d4850 con 0x5578a357bfe0 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb741fc5640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a357e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb7257fa640 10 client.17724 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.983+0000 7fb741fc5640 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a357e450 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b5a38a3932840d server_cookie=cf5f03c01065bab3 in_seq=0 out_seq=0 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb7257fa640 1 -- 172.21.15.77:0/2006201392 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578a34d4850 con 0x5578a357bfe0 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb7257fa640 10 client.17724 handle_command_reply: tid=0 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 client.17724 shutdown 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 msgr2=0x5578a3569180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb72c000c00 tx=0x7fb72c003820 comp rx=0 tx=0).stop 2024-09-15T22:21:06.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 shutdown_connections 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a357e450 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb7257fa640 0 client.17724 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3569180 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 --2- 172.21.15.77:0/2006201392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a356bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 >> 172.21.15.77:0/2006201392 conn(0x5578a355da30 msgr2=0x5578a3563760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.984+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 shutdown_connections 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.985+0000 7fb746602740 1 -- 172.21.15.77:0/2006201392 wait complete. 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:05.985+0000 7fb746602740 20 client.17724 trim_cache size 0 max 16384 2024-09-15T22:21:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.002+0000 7fb746602740 1 Processor -- start 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 -- start start 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a3583d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1af0a0 con 0x5578a35614a0 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1b0be0 con 0x5578a3561dc0 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb746602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1af770 con 0x5578a356c680 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42634/0 (socket says 172.21.15.77:42634) 2024-09-15T22:21:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 -- 172.21.15.77:0/4130616493 learned_addr learned my addr 172.21.15.77:0/4130616493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb7417c4640 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a3583d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb7427c6640 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 -- 172.21.15.77:0/4130616493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 msgr2=0x5578a35842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 -- 172.21.15.77:0/4130616493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 msgr2=0x5578a3583d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a3583d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 -- 172.21.15.77:0/4130616493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a3561bf0 con 0x5578a35614a0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb741fc5640 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb73809c3b0 tx=0x7fb7380d2940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93cbd86808e5dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb7277fe640 1 -- 172.21.15.77:0/4130616493 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7380a0720 con 0x5578a35614a0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb7277fe640 1 -- 172.21.15.77:0/4130616493 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7380a4fe0 con 0x5578a35614a0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.003+0000 7fb7277fe640 1 -- 172.21.15.77:0/4130616493 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73809baf0 con 0x5578a35614a0 2024-09-15T22:21:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- 172.21.15.77:0/4130616493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 msgr2=0x5578a35784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb73809c3b0 tx=0x7fb7380d2940 comp rx=0 tx=0).stop 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- 172.21.15.77:0/4130616493 shutdown_connections 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a35842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a3561dc0 0x5578a3583d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 --2- 172.21.15.77:0/4130616493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a35614a0 0x5578a35784b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- 172.21.15.77:0/4130616493 >> 172.21.15.77:0/4130616493 conn(0x5578a355da30 msgr2=0x5578a3567a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- 172.21.15.77:0/4130616493 shutdown_connections 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- 172.21.15.77:0/4130616493 wait complete. 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.004+0000 7fb746602740 1 -- start start 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 Processor -- start 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a3570350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a3570dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1b0510 con 0x5578a3561dc0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1afe40 con 0x5578a35614a0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1ad0a0 con 0x5578a356c680 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb746602740 10 client.0 _ll_register_callbacks cb 0x7fb72c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42650/0 (socket says 172.21.15.77:42650) 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 -- 172.21.15.77:0/3748915189 learned_addr learned my addr 172.21.15.77:0/3748915189 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb741fc5640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a3570350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7427c6640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a3570dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 -- 172.21.15.77:0/3748915189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 msgr2=0x5578a3570dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a3570dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 -- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 msgr2=0x5578a3570350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a3570350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 -- 172.21.15.77:0/3748915189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a35e5d30 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7417c4640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb72c004650 tx=0x7fb72c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4393d6612f70e6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.005+0000 7fb7257fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c058060 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb72c046890 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7380a4fe0 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb72c035040 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 10 client.17730 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5578a35e6ff0 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7380a4fe0 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb72c004760 con 0x5578a3561dc0 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 10 client.17730 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 20 client.17730 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a35e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 4 client.17730 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.006+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578a34ced60 con 0x5578a357bfe0 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.007+0000 7fb741fc5640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a35e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb7257fa640 10 client.17730 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb741fc5640 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a35e97e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653714547154cdfe server_cookie=4c9662ae17f8e574 in_seq=0 out_seq=0 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb7257fa640 1 -- 172.21.15.77:0/3748915189 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5578a34ced60 con 0x5578a357bfe0 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb7257fa640 10 client.17730 handle_command_reply: tid=0 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 client.17730 shutdown 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 msgr2=0x5578a3570890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb72c004650 tx=0x7fb72c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 shutdown_connections 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5578a357bfe0 0x5578a35e97e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5578a356c680 0x5578a3570dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb7257fa640 0 client.17730 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578a3561dc0 0x5578a3570890 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 --2- 172.21.15.77:0/3748915189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5578a35614a0 0x5578a3570350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 >> 172.21.15.77:0/3748915189 conn(0x5578a355da30 msgr2=0x5578a356b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 shutdown_connections 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 1 -- 172.21.15.77:0/3748915189 wait complete. 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.008+0000 7fb746602740 20 client.17730 trim_cache size 0 max 16384 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:21:06.027 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: "bfd4695a-763e-4fa0-b35c-3ba8a6ffa66b": { 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "bfd4695a-763e-4fa0-b35c-3ba8a6ffa66b", 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 msgr2=0x7fb7280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 0x7fb7280a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb734020000 tx=0x7fb734023e30 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 msgr2=0x7fb73c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c1993a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb73800e9f0 tx=0x7fb73800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 shutdown_connections 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb7280a33b0 0x7fb7280a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c102ef0 0x7fb73c1a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb73c102570 0x7fb73c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 --2- 172.21.15.77:0/1033159367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb73c100e20 0x7fb73c1993a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 >> 172.21.15.77:0/1033159367 conn(0x7fb73c0fc7d0 msgr2=0x7fb73c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 shutdown_connections 2024-09-15T22:21:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:06.009+0000 7fb744229640 1 -- 172.21.15.77:0/1033159367 wait complete. 2024-09-15T22:21:06.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 682847 8.08 MB/sec execute 315 sec latency 42.739 ms 2024-09-15T22:21:06.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:06 smithi077 ceph-mon[30980]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:06 smithi196 ceph-mon[35109]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:06.859 INFO:tasks.cephfs.filesystem:scrub status for tag:bfd4695a-763e-4fa0-b35c-3ba8a6ffa66b - {'path': '/', 'tag': 'bfd4695a-763e-4fa0-b35c-3ba8a6ffa66b', 'options': 'recursive,force'} 2024-09-15T22:21:06.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:06 smithi117 ceph-mon[35021]: pgmap v267: 129 pgs: 129 active+clean; 468 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-09-15T22:21:07.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 685551 8.11 MB/sec execute 316 sec latency 15.307 ms 2024-09-15T22:21:07.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:07 smithi077 ceph-mon[30980]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:21:07.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:07 smithi196 ceph-mon[35109]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:21:07.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:07 smithi117 ceph-mon[35021]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:21:08.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 687435 8.11 MB/sec execute 317 sec latency 18.995 ms 2024-09-15T22:21:08.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:08 smithi077 ceph-mon[30980]: pgmap v269: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:08 smithi196 ceph-mon[35109]: pgmap v269: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:08 smithi117 ceph-mon[35021]: pgmap v269: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:09.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 689659 8.13 MB/sec execute 318 sec latency 17.771 ms 2024-09-15T22:21:10.480 INFO:tasks.workunit.client.0.smithi077.stdout: 1 691531 8.12 MB/sec execute 319 sec latency 55.846 ms 2024-09-15T22:21:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:10 smithi077 ceph-mon[30980]: pgmap v270: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:10 smithi196 ceph-mon[35109]: pgmap v270: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:10 smithi117 ceph-mon[35021]: pgmap v270: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:21:11.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 693481 8.14 MB/sec execute 320 sec latency 16.785 ms 2024-09-15T22:21:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:21:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:21:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:21:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:21:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:21:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:21:12.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 696176 8.16 MB/sec execute 321 sec latency 17.547 ms 2024-09-15T22:21:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:12 smithi077 ceph-mon[30980]: pgmap v271: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:12.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:12 smithi196 ceph-mon[35109]: pgmap v271: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:12 smithi117 ceph-mon[35021]: pgmap v271: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:13.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 697901 8.16 MB/sec execute 322 sec latency 25.345 ms 2024-09-15T22:21:14.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 700261 8.18 MB/sec execute 323 sec latency 17.898 ms 2024-09-15T22:21:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:14 smithi077 ceph-mon[30980]: pgmap v272: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:14.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:14 smithi077 ceph-mon[30980]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:21:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:14 smithi196 ceph-mon[35109]: pgmap v272: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:14 smithi196 ceph-mon[35109]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:21:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:14 smithi117 ceph-mon[35021]: pgmap v272: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-09-15T22:21:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:14 smithi117 ceph-mon[35021]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:21:15.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 702159 8.18 MB/sec execute 324 sec latency 108.377 ms 2024-09-15T22:21:16.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 704166 8.19 MB/sec execute 325 sec latency 12.056 ms 2024-09-15T22:21:16.663 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:16 smithi117 ceph-mon[35021]: pgmap v274: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-15T22:21:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:16 smithi077 ceph-mon[30980]: pgmap v274: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-15T22:21:16.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:16 smithi196 ceph-mon[35109]: pgmap v274: 129 pgs: 129 active+clean; 489 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-15T22:21:16.949 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:21:16.949 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-15T22:21:16.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:21:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:17.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 706885 8.22 MB/sec execute 326 sec latency 13.608 ms 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 -- 172.21.15.77:0/1791868958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678102dd0 msgr2=0x7f56781031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 --2- 172.21.15.77:0/1791868958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678102dd0 0x7f56781031b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f566c009930 tx=0x7f566c02a420 comp rx=0 tx=0).stop 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 -- 172.21.15.77:0/1791868958 shutdown_connections 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 --2- 172.21.15.77:0/1791868958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678108300 0x7f567810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 --2- 172.21.15.77:0/1791868958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678103780 0x7f5678107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 --2- 172.21.15.77:0/1791868958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678102dd0 0x7f56781031b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:17.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 -- 172.21.15.77:0/1791868958 >> 172.21.15.77:0/1791868958 conn(0x7f56780fc7d0 msgr2=0x7f56780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 -- 172.21.15.77:0/1791868958 shutdown_connections 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.829+0000 7f567eeee640 1 -- 172.21.15.77:0/1791868958 wait complete. 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 Processor -- start 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 -- start start 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 0x7f5678100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103780 0x7f5678100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56781120b0 con 0x7f5678108300 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5678111f30 con 0x7f5678102dd0 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567eeee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5678112230 con 0x7f5678103780 2024-09-15T22:21:17.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45360/0 (socket says 172.21.15.77:45360) 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 -- 172.21.15.77:0/1685377895 learned_addr learned my addr 172.21.15.77:0/1685377895 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f5677fff640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103780 0x7f5678100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567cc8a640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 0x7f5678100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 -- 172.21.15.77:0/1685377895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103780 msgr2=0x7f5678100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103780 0x7f5678100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 -- 172.21.15.77:0/1685377895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 msgr2=0x7f5678100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 0x7f5678100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567d48b640 1 -- 172.21.15.77:0/1685377895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781a6b30 con 0x7f5678108300 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.830+0000 7f567cc8a640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 0x7f5678100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f567d48b640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f567000eb10 tx=0x7f567000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004b20 con 0x7f5678108300 2024-09-15T22:21:17.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781a6e20 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5670004cc0 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670010760 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.831+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a7330 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.832+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567806a7b0 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.833+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5670005020 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.834+0000 7f5675ffb640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 0x7f56680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.834+0000 7f567cc8a640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 0x7f56680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.834+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f56700d0790 con 0x7f5678108300 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.834+0000 7f567cc8a640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 0x7f56680a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f566c000c00 tx=0x7f566c032c70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:17.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:17.838+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f567009cde0 con 0x7f5678108300 2024-09-15T22:21:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.092+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f567810a290 con 0x7f5678108300 2024-09-15T22:21:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.093+0000 7f5675ffb640 1 -- 172.21.15.77:0/1685377895 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f567009c6e0 con 0x7f5678108300 2024-09-15T22:21:18.095 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:21:18.095 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:21:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 msgr2=0x7f56680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 0x7f56680a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f566c000c00 tx=0x7f566c032c70 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 msgr2=0x7f56781014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f567000eb10 tx=0x7f567000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 shutdown_connections 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56680a33b0 0x7f56680a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5678108300 0x7f56781014b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5678103780 0x7f5678100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 --2- 172.21.15.77:0/1685377895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5678102dd0 0x7f5678100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:18.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 >> 172.21.15.77:0/1685377895 conn(0x7f56780fc7d0 msgr2=0x7f5678105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:18.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 shutdown_connections 2024-09-15T22:21:18.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:18.095+0000 7f567eeee640 1 -- 172.21.15.77:0/1685377895 wait complete. 2024-09-15T22:21:18.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 708534 8.21 MB/sec execute 327 sec latency 26.178 ms 2024-09-15T22:21:18.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:18 smithi077 ceph-mon[30980]: pgmap v275: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:18.854 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:18 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1685377895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:18.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:18 smithi196 ceph-mon[35109]: pgmap v275: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:18.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:18 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1685377895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:18 smithi117 ceph-mon[35021]: pgmap v275: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:18 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1685377895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:18.934 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-15T22:21:18.934 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-15T22:21:19.198 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:19.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 710890 8.23 MB/sec execute 328 sec latency 15.574 ms 2024-09-15T22:21:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 -- 172.21.15.77:0/25779747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54102ef0 msgr2=0x7fbc5410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 --2- 172.21.15.77:0/25779747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54102ef0 0x7fbc5410da40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009ca0 tx=0x7fbc4c02a950 comp rx=0 tx=0).stop 2024-09-15T22:21:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 -- 172.21.15.77:0/25779747 shutdown_connections 2024-09-15T22:21:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 --2- 172.21.15.77:0/25779747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54102ef0 0x7fbc5410da40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 --2- 172.21.15.77:0/25779747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 0x7fbc541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 --2- 172.21.15.77:0/25779747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54100e20 0x7fbc54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 -- 172.21.15.77:0/25779747 >> 172.21.15.77:0/25779747 conn(0x7fbc540fc7d0 msgr2=0x7fbc540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 -- 172.21.15.77:0/25779747 shutdown_connections 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.800+0000 7fbc5c530640 1 -- 172.21.15.77:0/25779747 wait complete. 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 Processor -- start 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 -- start start 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 0x7fbc541989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54102ef0 0x7fbc5419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54110f80 con 0x7fbc54100e20 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54110e00 con 0x7fbc54102ef0 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5c530640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54111100 con 0x7fbc54102570 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38940/0 (socket says 172.21.15.77:38940) 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/558377605 learned_addr learned my addr 172.21.15.77:0/558377605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc59acb640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 0x7fbc541989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54102ef0 0x7fbc5419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/558377605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 msgr2=0x7fbc541989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 0x7fbc541989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/558377605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54102ef0 msgr2=0x7fbc5419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54102ef0 0x7fbc5419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.801+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/558377605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5419d410 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d950 tx=0x7fbc4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc477fe640 1 -- 172.21.15.77:0/558377605 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48017030 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5419d700 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc477fe640 1 -- 172.21.15.77:0/558377605 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc48004020 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc477fe640 1 -- 172.21.15.77:0/558377605 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480047a0 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.802+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541a4fe0 con 0x7fbc54100e20 2024-09-15T22:21:19.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.804+0000 7fbc477fe640 1 -- 172.21.15.77:0/558377605 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbc480041d0 con 0x7fbc54100e20 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.804+0000 7fbc5e909740 1 Processor -- start 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 -- start start 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc477fe640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 0x7fbc400a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 0x7fbc400a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x5607546154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc477fe640 1 -- 172.21.15.77:0/558377605 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbc480cfa70 con 0x7fbc54100e20 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754617e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56075456f160 con 0x560754615a30 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560754516720 con 0x56075460b170 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5e909740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56075456b120 con 0x56075460a850 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43498/0 (socket says 172.21.15.77:43498) 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/3157747456 learned_addr learned my addr 172.21.15.77:0/3157747456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x5607546154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754617e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/3157747456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 msgr2=0x5607546154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x5607546154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/3157747456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 msgr2=0x560754617e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754617e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.805+0000 7fbc5a2cc640 1 -- 172.21.15.77:0/3157747456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607542e0ba0 con 0x56075460a850 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754617e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:19.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 0x7fbc400a9c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005e30 tx=0x7fbc4c02ff80 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc48098390 tx=0x7fbc4809c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4011f520b5a1c536 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc44ff9640 1 -- 172.21.15.77:0/3157747456 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4809fd70 con 0x56075460a850 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc44ff9640 1 -- 172.21.15.77:0/3157747456 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc480a4630 con 0x56075460a850 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.806+0000 7fbc44ff9640 1 -- 172.21.15.77:0/3157747456 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4809b070 con 0x56075460a850 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- 172.21.15.77:0/3157747456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 msgr2=0x56075460ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc48098390 tx=0x7fbc4809c0c0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- 172.21.15.77:0/3157747456 shutdown_connections 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754617e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x5607546154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3157747456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x56075460ac30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- 172.21.15.77:0/3157747456 >> 172.21.15.77:0/3157747456 conn(0x560754606de0 msgr2=0x5607546071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- 172.21.15.77:0/3157747456 shutdown_connections 2024-09-15T22:21:19.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- 172.21.15.77:0/3157747456 wait complete. 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 -- start start 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.807+0000 7fbc5e909740 1 Processor -- start 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x5607546129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754613450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560754618600 con 0x560754615a30 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607544f7900 con 0x56075460b170 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560754618780 con 0x56075460a850 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 10 client.0 _ll_register_callbacks cb 0x5607544f5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5aacd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754613450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x5607546129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36308/0 (socket says 172.21.15.77:36308) 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 -- 172.21.15.77:0/3365851366 learned_addr learned my addr 172.21.15.77:0/3365851366 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 -- 172.21.15.77:0/3365851366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 msgr2=0x5607546129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x5607546129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 -- 172.21.15.77:0/3365851366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 msgr2=0x560754613450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754613450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 -- 172.21.15.77:0/3365851366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560754537c50 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc59acb640 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000bf00 tx=0x7fbc50011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41536967110d7661 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004020 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc480a4630 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.808+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbc500041c0 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc50024050 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 10 client.26858 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607546243a0 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc480a4630 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbc500038c0 con 0x56075460b170 2024-09-15T22:21:19.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 10 client.26858 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 10 client.26858 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 20 client.26858 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560754625870 0x560754627cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 4 client.26858 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56075457c890 con 0x560754625870 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.809+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560754625870 0x560754627cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560754625870 0x560754627cb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcaa4898ec8fdc3 server_cookie=f1679124c739ad74 in_seq=0 out_seq=0 2024-09-15T22:21:19.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc3dffb640 10 client.26858 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc3dffb640 1 -- 172.21.15.77:0/3365851366 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56075457c890 con 0x560754625870 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc3dffb640 10 client.26858 handle_command_reply: tid=0 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc5e909740 1 client.26858 shutdown 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.810+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 msgr2=0x560754612f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000bf00 tx=0x7fbc50011810 comp rx=0 tx=0).stop 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 shutdown_connections 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560754625870 0x560754627cb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560754615a30 0x560754613450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460b170 0x560754612f10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 --2- 172.21.15.77:0/3365851366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460a850 0x5607546129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc3dffb640 0 client.26858 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 >> 172.21.15.77:0/3365851366 conn(0x560754606de0 msgr2=0x56075460cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 shutdown_connections 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 1 -- 172.21.15.77:0/3365851366 wait complete. 2024-09-15T22:21:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.811+0000 7fbc5e909740 20 client.26858 trim_cache size 0 max 16384 2024-09-15T22:21:19.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.829+0000 7fbc5e909740 1 Processor -- start 2024-09-15T22:21:19.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.829+0000 7fbc5e909740 1 -- start start 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.829+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 0x5607546132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560754615a30 0x56075462dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5e909740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541acd00 con 0x56075460b170 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5e909740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541ae840 con 0x56075460a850 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5e909740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541ad3d0 con 0x560754615a30 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38960/0 (socket says 172.21.15.77:38960) 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 0x5607546132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 -- 172.21.15.77:0/2815285543 learned_addr learned my addr 172.21.15.77:0/2815285543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560754615a30 0x56075462dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 -- 172.21.15.77:0/2815285543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560754615a30 msgr2=0x56075462dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560754615a30 0x56075462dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 -- 172.21.15.77:0/2815285543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 msgr2=0x5607546132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 0x5607546132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 -- 172.21.15.77:0/2815285543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607546150e0 con 0x56075460b170 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 0x5607546132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbc50003ce0 tx=0x7fbc50011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b182a7872bd0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc3ffff640 1 -- 172.21.15.77:0/2815285543 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5001a3d0 con 0x56075460b170 2024-09-15T22:21:19.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc3ffff640 1 -- 172.21.15.77:0/2815285543 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50003ad0 con 0x56075460b170 2024-09-15T22:21:19.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.830+0000 7fbc3ffff640 1 -- 172.21.15.77:0/2815285543 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50005400 con 0x56075460b170 2024-09-15T22:21:19.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 -- 172.21.15.77:0/2815285543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 msgr2=0x56075462d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbc50003ce0 tx=0x7fbc50011d10 comp rx=0 tx=0).stop 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 -- 172.21.15.77:0/2815285543 shutdown_connections 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560754615a30 0x56075462dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460b170 0x56075462d580 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2815285543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075460a850 0x5607546132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 -- 172.21.15.77:0/2815285543 >> 172.21.15.77:0/2815285543 conn(0x560754606de0 msgr2=0x560754611990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 -- 172.21.15.77:0/2815285543 shutdown_connections 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.831+0000 7fbc5e909740 1 -- 172.21.15.77:0/2815285543 wait complete. 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 -- start start 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 Processor -- start 2024-09-15T22:21:19.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 0x560754619ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 0x56075468f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541ae170 con 0x56075460a850 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541adaa0 con 0x560754615a30 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541aad00 con 0x56075460b170 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5e909740 10 client.0 _ll_register_callbacks cb 0x7fbc5000bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5a2cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 0x560754619ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc5aacd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 0x56075468f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43518/0 (socket says 172.21.15.77:43518) 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 -- 172.21.15.77:0/2600730616 learned_addr learned my addr 172.21.15.77:0/2600730616 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:19.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 -- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 msgr2=0x56075468f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 0x56075468f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 -- 172.21.15.77:0/2600730616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 msgr2=0x560754619ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.832+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 0x560754619ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc59acb640 1 -- 172.21.15.77:0/2600730616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075468fa90 con 0x56075460b170 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc5aacd640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 0x56075468f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 0x560754619ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc59acb640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc50004610 tx=0x7fbc50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=655e0b92ed1d41b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004150 con 0x56075460b170 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbc50005140 con 0x56075460b170 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560754576be0 con 0x56075460b170 2024-09-15T22:21:19.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc50005340 con 0x56075460b170 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc5e909740 10 client.26797 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.833+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560754690bc0 con 0x56075460b170 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560754576be0 con 0x56075460b170 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbc5001a660 con 0x56075460b170 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 10 client.26797 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 10 client.26797 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 20 client.26797 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607546912a0 0x5607546936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 4 client.26797 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x5607544f2f40 con 0x5607546912a0 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.834+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607546912a0 0x5607546936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:19.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.835+0000 7fbc3dffb640 10 client.26797 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:19.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.835+0000 7fbc5a2cc640 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607546912a0 0x5607546936c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f5de801def2169 server_cookie=a21c2ff0c8b0a07e in_seq=0 out_seq=0 2024-09-15T22:21:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc3dffb640 1 -- 172.21.15.77:0/2600730616 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5607544f2f40 con 0x5607546912a0 2024-09-15T22:21:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc3dffb640 10 client.26797 handle_command_reply: tid=0 2024-09-15T22:21:19.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 client.26797 shutdown 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 msgr2=0x56075461a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc50004610 tx=0x7fbc50011810 comp rx=0 tx=0).stop 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 shutdown_connections 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607546912a0 0x5607546936c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560754615a30 0x56075468f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc3dffb640 0 client.26797 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56075460b170 0x56075461a020 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 --2- 172.21.15.77:0/2600730616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56075460a850 0x560754619ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.939+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 >> 172.21.15.77:0/2600730616 conn(0x560754606de0 msgr2=0x5607546056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.940+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 shutdown_connections 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.940+0000 7fbc5e909740 1 -- 172.21.15.77:0/2600730616 wait complete. 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.940+0000 7fbc5e909740 20 client.26797 trim_cache size 0 max 16384 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 30, 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "6869fc35": { 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 30, 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:21:19.943 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.3, 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:21:19.944 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:19.945 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.941+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 msgr2=0x7fbc400a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.941+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 0x7fbc400a9c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005e30 tx=0x7fbc4c02ff80 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.941+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 msgr2=0x7fbc54198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.941+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d950 tx=0x7fbc4800de10 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 shutdown_connections 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbc400a77c0 0x7fbc400a9c80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc54102ef0 0x7fbc5419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc54102570 0x7fbc541989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 --2- 172.21.15.77:0/558377605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc54100e20 0x7fbc54198460 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:19.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.942+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 >> 172.21.15.77:0/558377605 conn(0x7fbc540fc7d0 msgr2=0x7fbc5406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:19.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.943+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 shutdown_connections 2024-09-15T22:21:19.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:19.944+0000 7fbc5c530640 1 -- 172.21.15.77:0/558377605 wait complete. 2024-09-15T22:21:20.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.22 MB/sec execute 329 sec latency 530.596 ms 2024-09-15T22:21:20.724 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:20 smithi077 ceph-mon[30980]: pgmap v276: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:20.750 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6869fc35', quiesce took 3.8 seconds. Will release after: 18.55 2024-09-15T22:21:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:20 smithi196 ceph-mon[35109]: pgmap v276: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:20.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:20 smithi117 ceph-mon[35021]: pgmap v276: 129 pgs: 129 active+clean; 466 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 237 op/s 2024-09-15T22:21:21.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.19 MB/sec execute 330 sec latency 1530.700 ms 2024-09-15T22:21:22.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.17 MB/sec execute 331 sec latency 2530.747 ms 2024-09-15T22:21:22.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:22 smithi077 ceph-mon[30980]: pgmap v277: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:22.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:22 smithi196 ceph-mon[35109]: pgmap v277: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:22.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:22 smithi117 ceph-mon[35021]: pgmap v277: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:23.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.14 MB/sec execute 332 sec latency 3530.802 ms 2024-09-15T22:21:24.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.12 MB/sec execute 333 sec latency 4530.925 ms 2024-09-15T22:21:24.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:24 smithi077 ceph-mon[30980]: pgmap v278: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:24.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:24 smithi196 ceph-mon[35109]: pgmap v278: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:24 smithi117 ceph-mon[35021]: pgmap v278: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-09-15T22:21:25.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.09 MB/sec execute 334 sec latency 5530.971 ms 2024-09-15T22:21:26.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.07 MB/sec execute 335 sec latency 6531.003 ms 2024-09-15T22:21:26.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:26 smithi077 ceph-mon[30980]: pgmap v279: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 219 op/s 2024-09-15T22:21:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:26 smithi196 ceph-mon[35109]: pgmap v279: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 219 op/s 2024-09-15T22:21:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:26 smithi117 ceph-mon[35021]: pgmap v279: 129 pgs: 129 active+clean; 480 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 219 op/s 2024-09-15T22:21:27.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.05 MB/sec execute 336 sec latency 7531.042 ms 2024-09-15T22:21:28.481 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.02 MB/sec execute 337 sec latency 8531.142 ms 2024-09-15T22:21:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:28 smithi077 ceph-mon[30980]: pgmap v280: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-15T22:21:28.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:28 smithi196 ceph-mon[35109]: pgmap v280: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-15T22:21:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:28 smithi117 ceph-mon[35021]: pgmap v280: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-15T22:21:29.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 8.00 MB/sec execute 338 sec latency 9531.216 ms 2024-09-15T22:21:30.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.97 MB/sec execute 339 sec latency 10531.310 ms 2024-09-15T22:21:30.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:30 smithi077 ceph-mon[30980]: pgmap v281: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 129 op/s 2024-09-15T22:21:30.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:30 smithi196 ceph-mon[35109]: pgmap v281: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 129 op/s 2024-09-15T22:21:30.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:30 smithi117 ceph-mon[35021]: pgmap v281: 129 pgs: 129 active+clean; 479 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 129 op/s 2024-09-15T22:21:31.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.95 MB/sec execute 340 sec latency 11531.424 ms 2024-09-15T22:21:32.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.93 MB/sec execute 341 sec latency 12531.469 ms 2024-09-15T22:21:32.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:32 smithi077 ceph-mon[30980]: pgmap v282: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 161 op/s 2024-09-15T22:21:32.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:32 smithi196 ceph-mon[35109]: pgmap v282: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 161 op/s 2024-09-15T22:21:32.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:32 smithi117 ceph-mon[35021]: pgmap v282: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 161 op/s 2024-09-15T22:21:33.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.90 MB/sec execute 342 sec latency 13531.555 ms 2024-09-15T22:21:34.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.88 MB/sec execute 343 sec latency 14531.667 ms 2024-09-15T22:21:34.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:34 smithi077 ceph-mon[30980]: pgmap v283: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:34.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:34 smithi196 ceph-mon[35109]: pgmap v283: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:34.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:34 smithi117 ceph-mon[35021]: pgmap v283: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:35.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.86 MB/sec execute 344 sec latency 15531.756 ms 2024-09-15T22:21:36.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.84 MB/sec execute 345 sec latency 16531.832 ms 2024-09-15T22:21:36.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:36 smithi077 ceph-mon[30980]: pgmap v284: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:36.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:36 smithi196 ceph-mon[35109]: pgmap v284: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:36.860 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:21:36.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:36 smithi117 ceph-mon[35021]: pgmap v284: 129 pgs: 129 active+clean; 478 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 1.6 MiB/s wr, 65 op/s 2024-09-15T22:21:37.117 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:37.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.81 MB/sec execute 346 sec latency 17531.883 ms 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 -- 172.21.15.77:0/116054736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 msgr2=0x7f079c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 --2- 172.21.15.77:0/116054736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c103180 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c02a480 comp rx=0 tx=0).stop 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 -- 172.21.15.77:0/116054736 shutdown_connections 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 --2- 172.21.15.77:0/116054736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c10c180 0x7f079c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 --2- 172.21.15.77:0/116054736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c1036c0 0x7f079c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 --2- 172.21.15.77:0/116054736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c103180 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 -- 172.21.15.77:0/116054736 >> 172.21.15.77:0/116054736 conn(0x7f079c0fc7a0 msgr2=0x7f079c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 -- 172.21.15.77:0/116054736 shutdown_connections 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.600+0000 7f07a089d640 1 -- 172.21.15.77:0/116054736 wait complete. 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 Processor -- start 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 -- start start 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c1036c0 0x7f079c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c10c180 0x7f079c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f079c111f60 con 0x7f079c102da0 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f079c111de0 con 0x7f079c1036c0 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f07a089d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1120e0 con 0x7f079c10c180 2024-09-15T22:21:37.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50970/0 (socket says 172.21.15.77:50970) 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 -- 172.21.15.77:0/1307957154 learned_addr learned my addr 172.21.15.77:0/1307957154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f0793fff640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c1036c0 0x7f079c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079ad9d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c10c180 0x7f079c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 -- 172.21.15.77:0/1307957154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c10c180 msgr2=0x7f079c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c10c180 0x7f079c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.601+0000 7f079a59c640 1 -- 172.21.15.77:0/1307957154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c1036c0 msgr2=0x7f079c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f079a59c640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c1036c0 0x7f079c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f079a59c640 1 -- 172.21.15.77:0/1307957154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c19d3b0 con 0x7f079c102da0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f079a59c640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f07937fe640 1 -- 172.21.15.77:0/1307957154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c03f030 con 0x7f079c102da0 2024-09-15T22:21:37.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f07937fe640 1 -- 172.21.15.77:0/1307957154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c003ce0 con 0x7f079c102da0 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c19d640 con 0x7f079c102da0 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f07937fe640 1 -- 172.21.15.77:0/1307957154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c035070 con 0x7f079c102da0 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.602+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c1a4f80 con 0x7f079c102da0 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.604+0000 7f07937fe640 1 -- 172.21.15.77:0/1307957154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f078c042030 con 0x7f079c102da0 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.604+0000 7f07a2c76740 1 Processor -- start 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 -- start start 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd304a0 0x5628cdd30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 0x5628cdd3b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628cdc962e0 con 0x5628cdd3b680 2024-09-15T22:21:37.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628cdc3dce0 con 0x5628cdd304a0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07a2c76740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628cdc922a0 con 0x5628cdd30dc0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f07937fe640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 0x7f07880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50974/0 (socket says 172.21.15.77:50974) 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 -- 172.21.15.77:0/4269977842 learned_addr learned my addr 172.21.15.77:0/4269977842 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f079a59c640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 0x5628cdd3b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f079ad9d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 0x7f07880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f079ad9d640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd304a0 0x5628cdd30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 -- 172.21.15.77:0/4269977842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 msgr2=0x5628cdd3b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 0x5628cdd3b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 -- 172.21.15.77:0/4269977842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd304a0 msgr2=0x5628cdd30880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd304a0 0x5628cdd30880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f0793fff640 1 -- 172.21.15.77:0/4269977842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cdc22440 con 0x5628cdd3b680 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.605+0000 7f079a59c640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 0x5628cdd3b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f07937fe640 1 -- 172.21.15.77:0/1307957154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f078c0f0720 con 0x7f079c102da0 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f0793fff640 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f078400bcb0 tx=0x7f07840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1220c6ca035dfbf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f079ad9d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 0x7f07880a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0794009fd0 tx=0x7f0794009840 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f0790ff9640 1 -- 172.21.15.77:0/4269977842 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784004030 con 0x5628cdd3b680 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f0790ff9640 1 -- 172.21.15.77:0/4269977842 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07840041d0 con 0x5628cdd3b680 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f0790ff9640 1 -- 172.21.15.77:0/4269977842 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07840117a0 con 0x5628cdd3b680 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f07a2c76740 1 -- 172.21.15.77:0/4269977842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 msgr2=0x5628cdd3da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.606+0000 7f07a2c76740 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f078400bcb0 tx=0x7f07840107a0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- 172.21.15.77:0/4269977842 shutdown_connections 2024-09-15T22:21:37.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd3da70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd30dc0 0x5628cdd3b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- 172.21.15.77:0/4269977842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd304a0 0x5628cdd30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- 172.21.15.77:0/4269977842 >> 172.21.15.77:0/4269977842 conn(0x5628cdd2ca30 msgr2=0x5628cdd2ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- 172.21.15.77:0/4269977842 shutdown_connections 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- 172.21.15.77:0/4269977842 wait complete. 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- start start 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 Processor -- start 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd3af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd38750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628cdd3e250 con 0x5628cdd30dc0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628cdc51050 con 0x5628cdd3b680 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628cdd3e3d0 con 0x5628cdd304a0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f07a2c76740 10 client.0 _ll_register_callbacks cb 0x5628cdc91160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50976/0 (socket says 172.21.15.77:50976) 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.607+0000 7f0793fff640 1 -- 172.21.15.77:0/3824293487 learned_addr learned my addr 172.21.15.77:0/3824293487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f079ad9d640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd38750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f079a59c640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd3af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 -- 172.21.15.77:0/3824293487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 msgr2=0x5628cdd3af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd3af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 -- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 msgr2=0x5628cdd38750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd38750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 -- 172.21.15.77:0/3824293487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cda073c0 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f079ad9d640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0793fff640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0784003df0 tx=0x7f078400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161aa13904331dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078402a060 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f0784004380 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07840041d0 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0784031050 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f07a2c76740 10 client.17766 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5628cdd49bc0 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.608+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07840041d0 con 0x5628cdd30dc0 2024-09-15T22:21:37.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0784004880 con 0x5628cdd30dc0 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 10 client.17766 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 20 client.17766 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cdd4d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 4 client.17766 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628cdca3850 con 0x5628cdd4b080 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f079a59c640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cdd4d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f079a59c640 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cdd4d470 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f91915a9d9a1fd server_cookie=cc622f6752a70c8e in_seq=0 out_seq=0 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.609+0000 7f0781ffb640 10 client.17766 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:37.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f0781ffb640 1 -- 172.21.15.77:0/3824293487 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628cdca3850 con 0x5628cdd4b080 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f0781ffb640 10 client.17766 handle_command_reply: tid=0 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 client.17766 shutdown 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 msgr2=0x5628cdd38210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0784003df0 tx=0x7f078400b8e0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 shutdown_connections 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cdd4d470 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f0781ffb640 0 client.17766 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd38210 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 --2- 172.21.15.77:0/3824293487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd3af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 >> 172.21.15.77:0/3824293487 conn(0x5628cdd2ca30 msgr2=0x5628cdd32710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 shutdown_connections 2024-09-15T22:21:37.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 1 -- 172.21.15.77:0/3824293487 wait complete. 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.610+0000 7f07a2c76740 20 client.17766 trim_cache size 0 max 16384 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 Processor -- start 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 -- start start 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd30dc0 0x5628cdd33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1aeff0 con 0x5628cdd3b680 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b0b30 con 0x5628cdd30dc0 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f07a2c76740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1af6c0 con 0x5628cdd304a0 2024-09-15T22:21:37.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f079ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f079ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50992/0 (socket says 172.21.15.77:50992) 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.627+0000 7f079ad9d640 1 -- 172.21.15.77:0/2986249590 learned_addr learned my addr 172.21.15.77:0/2986249590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f0793fff640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd30dc0 0x5628cdd33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079a59c640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 -- 172.21.15.77:0/2986249590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 msgr2=0x5628cdd474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 -- 172.21.15.77:0/2986249590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd30dc0 msgr2=0x5628cdd33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd30dc0 0x5628cdd33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 -- 172.21.15.77:0/2986249590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628cdd30bf0 con 0x5628cdd3b680 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079a59c640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:37.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f079ad9d640 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0794012ee0 tx=0x7f0794009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f6f09b2e8183f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f0783fff640 1 -- 172.21.15.77:0/2986249590 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079402c480 con 0x5628cdd3b680 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f0783fff640 1 -- 172.21.15.77:0/2986249590 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0794021920 con 0x5628cdd3b680 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.628+0000 7f0783fff640 1 -- 172.21.15.77:0/2986249590 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794020920 con 0x5628cdd3b680 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- 172.21.15.77:0/2986249590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 msgr2=0x5628cdd52a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0794012ee0 tx=0x7f0794009600 comp rx=0 tx=0).stop 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- 172.21.15.77:0/2986249590 shutdown_connections 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd3b680 0x5628cdd52a40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd30dc0 0x5628cdd33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 --2- 172.21.15.77:0/2986249590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- 172.21.15.77:0/2986249590 >> 172.21.15.77:0/2986249590 conn(0x5628cdd2ca30 msgr2=0x5628cdd33050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- 172.21.15.77:0/2986249590 shutdown_connections 2024-09-15T22:21:37.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- 172.21.15.77:0/2986249590 wait complete. 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 -- start start 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.629+0000 7f07a2c76740 1 Processor -- start 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd43bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd3f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1b0460 con 0x5628cdd30dc0 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1afd90 con 0x5628cdd3b680 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f079c1acff0 con 0x5628cdd304a0 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 10 client.0 _ll_register_callbacks cb 0x7f078400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50994/0 (socket says 172.21.15.77:50994) 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 -- 172.21.15.77:0/681823099 learned_addr learned my addr 172.21.15.77:0/681823099 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f079a59c640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f079ad9d640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd3f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 -- 172.21.15.77:0/681823099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 msgr2=0x5628cdd43bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd43bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 -- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 msgr2=0x5628cdd3f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd3f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 -- 172.21.15.77:0/681823099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628cdd3fba0 con 0x5628cdd30dc0 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f079ad9d640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd3f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:37.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f079a59c640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd43bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0793fff640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0784003f90 tx=0x7f0784004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d474f6c2c41887c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784003a10 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f0784022050 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0794021920 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0784042050 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 10 client.17772 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.630+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5628cddb9ca0 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0794021920 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f07840051d0 con 0x5628cdd30dc0 2024-09-15T22:21:37.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 10 client.17772 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 20 client.17772 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cddbc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 4 client.17772 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5628cdc9dd60 con 0x5628cdd4b080 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f079a59c640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cddbc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f0781ffb640 10 client.17772 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.631+0000 7f079a59c640 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cddbc360 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fda5a87517d1c2 server_cookie=e81584e2193bcf86 in_seq=0 out_seq=0 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f0781ffb640 1 -- 172.21.15.77:0/681823099 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5628cdc9dd60 con 0x5628cdd4b080 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f0781ffb640 10 client.17772 handle_command_reply: tid=0 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 client.17772 shutdown 2024-09-15T22:21:37.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 msgr2=0x5628cdd3efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0784003f90 tx=0x7f0784004f00 comp rx=0 tx=0).stop 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 shutdown_connections 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5628cdd4b080 0x5628cddbc360 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628cdd3b680 0x5628cdd3f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628cdd30dc0 0x5628cdd3efc0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 --2- 172.21.15.77:0/681823099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628cdd304a0 0x5628cdd43bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 >> 172.21.15.77:0/681823099 conn(0x5628cdd2ca30 msgr2=0x5628cdd366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f0781ffb640 0 client.17772 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 shutdown_connections 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 1 -- 172.21.15.77:0/681823099 wait complete. 2024-09-15T22:21:37.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a2c76740 20 client.17772 trim_cache size 0 max 16384 2024-09-15T22:21:37.656 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:37.656 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:21:37.656 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:21:37.656 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:37.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 msgr2=0x7f07880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 0x7f07880a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0794009fd0 tx=0x7f0794009840 comp rx=0 tx=0).stop 2024-09-15T22:21:37.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 msgr2=0x7f079c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:37.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.632+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 shutdown_connections 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f07880a33b0 0x7f07880a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f079c10c180 0x7f079c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f079c1036c0 0x7f079c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 --2- 172.21.15.77:0/1307957154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f079c102da0 0x7f079c198400 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 >> 172.21.15.77:0/1307957154 conn(0x7f079c0fc7a0 msgr2=0x7f079c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 shutdown_connections 2024-09-15T22:21:37.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:37.633+0000 7f07a089d640 1 -- 172.21.15.77:0/1307957154 wait complete. 2024-09-15T22:21:38.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:21:38.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:21:38.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.79 MB/sec execute 347 sec latency 18531.919 ms 2024-09-15T22:21:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:38 smithi077 ceph-mon[30980]: pgmap v285: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-15T22:21:38.703 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:38.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:38 smithi196 ceph-mon[35109]: pgmap v285: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-15T22:21:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:38 smithi117 ceph-mon[35021]: pgmap v285: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-09-15T22:21:39.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 -- 172.21.15.77:0/1642641553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04102790 msgr2=0x7f6b041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:39.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/1642641553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04102790 0x7f6b041067e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009900 tx=0x7f6af402a410 comp rx=0 tx=0).stop 2024-09-15T22:21:39.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 -- 172.21.15.77:0/1642641553 shutdown_connections 2024-09-15T22:21:39.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/1642641553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b04106f10 0x7f6b0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/1642641553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04102790 0x7f6b041067e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/1642641553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 0x7f6b041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 -- 172.21.15.77:0/1642641553 >> 172.21.15.77:0/1642641553 conn(0x7f6b040fd8c0 msgr2=0x7f6b040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 -- 172.21.15.77:0/1642641553 shutdown_connections 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.216+0000 7f6b0c13a640 1 -- 172.21.15.77:0/1642641553 wait complete. 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 Processor -- start 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 -- start start 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 0x7f6b040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b04102790 0x7f6b04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04112010 con 0x7f6b04106f10 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04111e90 con 0x7f6b04101de0 2024-09-15T22:21:39.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0c13a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04112190 con 0x7f6b04102790 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0a6d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0a6d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34112/0 (socket says 172.21.15.77:34112) 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b0a6d7640 1 -- 172.21.15.77:0/702313577 learned_addr learned my addr 172.21.15.77:0/702313577 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.217+0000 7f6b09ed6640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 0x7f6b040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 -- 172.21.15.77:0/702313577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b04102790 msgr2=0x7f6b04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b04102790 0x7f6b04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 -- 172.21.15.77:0/702313577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 msgr2=0x7f6b040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 0x7f6b040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 -- 172.21.15.77:0/702313577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b041a6a70 con 0x7f6b04106f10 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0a6d7640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b970 tx=0x7f6afc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc019070 con 0x7f6b04106f10 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6afc00cb40 con 0x7f6b04106f10 2024-09-15T22:21:39.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b041a6d60 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007650 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.218+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b041a72a0 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.219+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b04102240 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.221+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6afc01c070 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.222+0000 7f6afaffd640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 0x7f6af00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.222+0000 7f6b09ed6640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 0x7f6af00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.222+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f6afc0cf630 con 0x7f6b04106f10 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.223+0000 7f6b09ed6640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 0x7f6af00a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009890 tx=0x7f6b00008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:39.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.227+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6afc09bd30 con 0x7f6b04106f10 2024-09-15T22:21:39.302 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:21:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.434+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b0402d7c0 con 0x7f6b04106f10 2024-09-15T22:21:39.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.435+0000 7f6afaffd640 1 -- 172.21.15.77:0/702313577 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6afc09b630 con 0x7f6b04106f10 2024-09-15T22:21:39.438 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:21:39.438 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:21:39.438 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:21:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 msgr2=0x7f6af00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 0x7f6af00a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009890 tx=0x7f6b00008040 comp rx=0 tx=0).stop 2024-09-15T22:21:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 msgr2=0x7f6b04074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00b970 tx=0x7f6afc00be30 comp rx=0 tx=0).stop 2024-09-15T22:21:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 shutdown_connections 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6af00a33b0 0x7f6af00a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b04106f10 0x7f6b04074f70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b04102790 0x7f6b04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 --2- 172.21.15.77:0/702313577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b04101de0 0x7f6b040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 >> 172.21.15.77:0/702313577 conn(0x7f6b040fd8c0 msgr2=0x7f6b040fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 shutdown_connections 2024-09-15T22:21:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:39.437+0000 7f6b0c13a640 1 -- 172.21.15.77:0/702313577 wait complete. 2024-09-15T22:21:39.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.77 MB/sec execute 348 sec latency 19531.942 ms 2024-09-15T22:21:39.794 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:39 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/702313577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:39.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:39 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/702313577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:39.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:39 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/702313577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:40.368 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:21:40.440 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:40.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.75 MB/sec execute 349 sec latency 20531.971 ms 2024-09-15T22:21:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:40 smithi077 ceph-mon[30980]: pgmap v286: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-15T22:21:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:40 smithi196 ceph-mon[35109]: pgmap v286: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-15T22:21:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:40 smithi117 ceph-mon[35021]: pgmap v286: 129 pgs: 129 active+clean; 473 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 -- 172.21.15.77:0/3599924351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb01082a0 msgr2=0x7efeb010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 --2- 172.21.15.77:0/3599924351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb01082a0 0x7efeb010ec40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7efea8009cd0 tx=0x7efea802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 -- 172.21.15.77:0/3599924351 shutdown_connections 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 --2- 172.21.15.77:0/3599924351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb01082a0 0x7efeb010ec40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 --2- 172.21.15.77:0/3599924351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01036e0 0x7efeb0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 --2- 172.21.15.77:0/3599924351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb0102d30 0x7efeb0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 -- 172.21.15.77:0/3599924351 >> 172.21.15.77:0/3599924351 conn(0x7efeb00fc7d0 msgr2=0x7efeb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:40.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 -- 172.21.15.77:0/3599924351 shutdown_connections 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.962+0000 7efeb80a3640 1 -- 172.21.15.77:0/3599924351 wait complete. 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 Processor -- start 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 -- start start 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 0x7efeb0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 0x7efeb019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efeb01120a0 con 0x7efeb0102d30 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0111f20 con 0x7efeb01036e0 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb80a3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112220 con 0x7efeb01082a0 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb5e3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb5e3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34134/0 (socket says 172.21.15.77:34134) 2024-09-15T22:21:40.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb5e3f640 1 -- 172.21.15.77:0/1773528647 learned_addr learned my addr 172.21.15.77:0/1773528647 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb563e640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 0x7efeb0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.963+0000 7efeb6640640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 0x7efeb019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 -- 172.21.15.77:0/1773528647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 msgr2=0x7efeb019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 0x7efeb019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 -- 172.21.15.77:0/1773528647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 msgr2=0x7efeb0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 0x7efeb0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 -- 172.21.15.77:0/1773528647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb019d500 con 0x7efeb0102d30 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb563e640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 0x7efeb0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb6640640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 0x7efeb019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb5e3f640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efeac009de0 tx=0x7efeac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac01f070 con 0x7efeb0102d30 2024-09-15T22:21:40.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac011750 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac01a850 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb019d7f0 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.964+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb01008c0 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.965+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0103110 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.967+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efeac022070 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.968+0000 7efea6ffd640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 0x7efe9c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.968+0000 7efeb563e640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 0x7efe9c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.968+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7efeac0d4050 con 0x7efeb0102d30 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.968+0000 7efeb563e640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 0x7efe9c0a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efeb00feda0 tx=0x7efea0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:40.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:40.971+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efeac09fff0 con 0x7efeb0102d30 2024-09-15T22:21:41.042 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:41.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.185+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb01998b0 con 0x7efeb0102d30 2024-09-15T22:21:41.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.186+0000 7efea6ffd640 1 -- 172.21.15.77:0/1773528647 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7efeac0a42f0 con 0x7efeb0102d30 2024-09-15T22:21:41.189 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:21:41.189 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:21:41.189 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 msgr2=0x7efe9c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 0x7efe9c0a58f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efeb00feda0 tx=0x7efea0005f90 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 msgr2=0x7efeb0198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efeac009de0 tx=0x7efeac00c6a0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 shutdown_connections 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efe9c0a3430 0x7efe9c0a58f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efeb01082a0 0x7efeb019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efeb01036e0 0x7efeb0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 --2- 172.21.15.77:0/1773528647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb0102d30 0x7efeb0198550 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 >> 172.21.15.77:0/1773528647 conn(0x7efeb00fc7d0 msgr2=0x7efeb0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 shutdown_connections 2024-09-15T22:21:41.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.188+0000 7efeb80a3640 1 -- 172.21.15.77:0/1773528647 wait complete. 2024-09-15T22:21:41.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.72 MB/sec execute 350 sec latency 21532.024 ms 2024-09-15T22:21:41.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:41 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:41.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:41 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1773528647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- 172.21.15.77:0/4082467164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0102570 msgr2=0x7f1ae01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/4082467164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0102570 0x7f1ae01029b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009900 tx=0x7f1acc02a480 comp rx=0 tx=0).stop 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- 172.21.15.77:0/4082467164 shutdown_connections 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/4082467164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102ef0 0x7f1ae010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/4082467164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0102570 0x7f1ae01029b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/4082467164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0100e20 0x7f1ae0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- 172.21.15.77:0/4082467164 >> 172.21.15.77:0/4082467164 conn(0x7f1ae00fc7d0 msgr2=0x7f1ae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- 172.21.15.77:0/4082467164 shutdown_connections 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- 172.21.15.77:0/4082467164 wait complete. 2024-09-15T22:21:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 Processor -- start 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.735+0000 7f1ae6b02640 1 -- start start 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 0x7f1ae0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 0x7f1ae019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0110f50 con 0x7f1ae0100e20 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0110dd0 con 0x7f1ae0102ef0 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01110d0 con 0x7f1ae0102570 2024-09-15T22:21:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34152/0 (socket says 172.21.15.77:34152) 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 -- 172.21.15.77:0/3731466917 learned_addr learned my addr 172.21.15.77:0/3731466917 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1adffff640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 0x7f1ae0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae509f640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 0x7f1ae019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 -- 172.21.15.77:0/3731466917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 msgr2=0x7f1ae0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 0x7f1ae0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 -- 172.21.15.77:0/3731466917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 msgr2=0x7f1ae019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 0x7f1ae019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 -- 172.21.15.77:0/3731466917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae019c300 con 0x7f1ae0100e20 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1adffff640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 0x7f1ae0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae509f640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 0x7f1ae019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae489e640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c920 tx=0x7f1ad400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1addffb640 1 -- 172.21.15.77:0/3731466917 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4007c70 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1addffb640 1 -- 172.21.15.77:0/3731466917 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad4007e10 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae019c5f0 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.736+0000 7f1addffb640 1 -- 172.21.15.77:0/3731466917 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4018890 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.737+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a3ed0 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.739+0000 7f1addffb640 1 -- 172.21.15.77:0/3731466917 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1ad4018a70 con 0x7f1ae0100e20 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 Processor -- start 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 -- start start 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c4a20 0x56459a9c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 0x56459a9d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56459a9292f0 con 0x56459a9c35d0 2024-09-15T22:21:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56459a8d0cf0 con 0x56459a9c4a20 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56459a9252b0 con 0x56459a9d2970 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34162/0 (socket says 172.21.15.77:34162) 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae489e640 1 -- 172.21.15.77:0/4220038416 learned_addr learned my addr 172.21.15.77:0/4220038416 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1addffb640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 0x7f1ad00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1adffff640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 0x56459a9d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae509f640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c4a20 0x56459a9c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae509f640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 0x7f1ad00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.740+0000 7f1ae489e640 1 -- 172.21.15.77:0/4220038416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 msgr2=0x56459a9d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae489e640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 0x56459a9d4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae489e640 1 -- 172.21.15.77:0/4220038416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c4a20 msgr2=0x56459a9c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae489e640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c4a20 0x56459a9c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1addffb640 1 -- 172.21.15.77:0/3731466917 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1ad40d02b0 con 0x7f1ae0100e20 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae489e640 1 -- 172.21.15.77:0/4220038416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459a8b53a0 con 0x56459a9c35d0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1adffff640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 0x56459a9d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae489e640 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ad409bf00 tx=0x7f1ad4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47625fcbf9a8a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1ae509f640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 0x7f1ad00a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009270 tx=0x7f1ad8005bd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1acaffd640 1 -- 172.21.15.77:0/4220038416 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad409c900 con 0x56459a9c35d0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1acaffd640 1 -- 172.21.15.77:0/4220038416 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad400de20 con 0x56459a9c35d0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.741+0000 7f1acaffd640 1 -- 172.21.15.77:0/4220038416 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad404b070 con 0x56459a9c35d0 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- 172.21.15.77:0/4220038416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 msgr2=0x56459a9c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ad409bf00 tx=0x7f1ad4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- 172.21.15.77:0/4220038416 shutdown_connections 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9d2970 0x56459a9d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c35d0 0x56459a9c3a10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/4220038416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c4a20 0x56459a9c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- 172.21.15.77:0/4220038416 >> 172.21.15.77:0/4220038416 conn(0x56459a98c310 msgr2=0x56459a9bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- 172.21.15.77:0/4220038416 shutdown_connections 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- 172.21.15.77:0/4220038416 wait complete. 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 -- start start 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.742+0000 7f1ae8edb740 1 Processor -- start 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 0x56459a9d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c4a20 0x56459a9cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56459a9d5480 con 0x56459a9d2970 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56459a8e4060 con 0x56459a9c35d0 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56459a9d5600 con 0x56459a9c4a20 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 10 client.0 _ll_register_callbacks cb 0x56459a6378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:41.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34174/0 (socket says 172.21.15.77:34174) 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 -- 172.21.15.77:0/1568268014 learned_addr learned my addr 172.21.15.77:0/1568268014 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae489e640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 0x56459a9d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1adffff640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c4a20 0x56459a9cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 -- 172.21.15.77:0/1568268014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c4a20 msgr2=0x56459a9cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c4a20 0x56459a9cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 -- 172.21.15.77:0/1568268014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 msgr2=0x56459a9d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 0x56459a9d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 -- 172.21.15.77:0/1568268014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459a69a3c0 con 0x56459a9d2970 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae489e640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 0x56459a9d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae509f640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800fa10 tx=0x7f1ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d530399d4d6fab32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1abffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8023070 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1ad8011b70 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad400de20 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ad802f050 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.743+0000 7f1ae8edb740 10 client.17796 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56459a9e0f50 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad400de20 con 0x56459a9d2970 2024-09-15T22:21:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ad801e040 con 0x56459a9d2970 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 10 client.17796 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 10 client.17796 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 20 client.17796 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459a9e2360 0x56459a9e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 4 client.17796 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56459a936860 con 0x56459a9e2360 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae489e640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459a9e2360 0x56459a9e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1ae489e640 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459a9e2360 0x56459a9e47c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9343ff645d97283 server_cookie=3326cf15253052d6 in_seq=0 out_seq=0 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.744+0000 7f1abffff640 10 client.17796 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1abffff640 1 -- 172.21.15.77:0/1568268014 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56459a936860 con 0x56459a9e2360 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1abffff640 10 client.17796 handle_command_reply: tid=0 2024-09-15T22:21:41.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 client.17796 shutdown 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 msgr2=0x56459a9cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800fa10 tx=0x7f1ad8004a40 comp rx=0 tx=0).stop 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 shutdown_connections 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459a9e2360 0x56459a9e47c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9d2970 0x56459a9cfbe0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c4a20 0x56459a9cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/1568268014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9c35d0 0x56459a9d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 >> 172.21.15.77:0/1568268014 conn(0x56459a98c310 msgr2=0x56459a9c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1abffff640 0 client.17796 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 shutdown_connections 2024-09-15T22:21:41.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 1 -- 172.21.15.77:0/1568268014 wait complete. 2024-09-15T22:21:41.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.745+0000 7f1ae8edb740 20 client.17796 trim_cache size 0 max 16384 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 Processor -- start 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 -- start start 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01acd00 con 0x56459a9c4a20 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01ae840 con 0x56459a9d2970 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.758+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01ad3d0 con 0x56459a9c35d0 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34190/0 (socket says 172.21.15.77:34190) 2024-09-15T22:21:41.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 -- 172.21.15.77:0/2542117474 learned_addr learned my addr 172.21.15.77:0/2542117474 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1ae509f640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1ae489e640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 -- 172.21.15.77:0/2542117474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 msgr2=0x56459a9c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 -- 172.21.15.77:0/2542117474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 msgr2=0x56459a9ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 -- 172.21.15.77:0/2542117474 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459a9d1f60 con 0x56459a9c4a20 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1ae489e640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1adffff640 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0314e0 tx=0x7f1acc004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ca1688f62f5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1aca7fc640 1 -- 172.21.15.77:0/2542117474 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc035040 con 0x56459a9c4a20 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1aca7fc640 1 -- 172.21.15.77:0/2542117474 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0046c0 con 0x56459a9c4a20 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.759+0000 7f1aca7fc640 1 -- 172.21.15.77:0/2542117474 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003850 con 0x56459a9c4a20 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- 172.21.15.77:0/2542117474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 msgr2=0x56459a9de6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0314e0 tx=0x7f1acc004310 comp rx=0 tx=0).stop 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- 172.21.15.77:0/2542117474 shutdown_connections 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9de6e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/2542117474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- 172.21.15.77:0/2542117474 >> 172.21.15.77:0/2542117474 conn(0x56459a98c310 msgr2=0x56459a9cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- 172.21.15.77:0/2542117474 shutdown_connections 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- 172.21.15.77:0/2542117474 wait complete. 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 -- start start 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.760+0000 7f1ae8edb740 1 Processor -- start 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01ae170 con 0x56459a9c4a20 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01adaa0 con 0x56459a9d2970 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae01aad00 con 0x56459a9c35d0 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 10 client.0 _ll_register_callbacks cb 0x7f1ad8009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34194/0 (socket says 172.21.15.77:34194) 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae509f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 -- 172.21.15.77:0/952067079 learned_addr learned my addr 172.21.15.77:0/952067079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae489e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9daa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39662/0 (socket says 172.21.15.77:39662) 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 -- 172.21.15.77:0/952067079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 msgr2=0x56459a9daa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9daa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 -- 172.21.15.77:0/952067079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 msgr2=0x56459a9e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:41.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 -- 172.21.15.77:0/952067079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459a9e9b80 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1adffff640 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006010 tx=0x7f1acc005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973833a578ae81c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1abffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc035050 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1acc031740 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.761+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1acc0046c0 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1acc031940 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 10 client.17802 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56459aa4c9a0 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0046c0 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1acc043020 con 0x56459a9c4a20 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 10 client.17802 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:41.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 10 client.17802 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 20 client.17802 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459aa4d060 0x56459aa4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 4 client.17802 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56459a930d70 con 0x56459aa4d060 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae489e640 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459aa4d060 0x56459aa4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1ae489e640 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459aa4d060 0x56459aa4f480 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5c75b3f8f54a681 server_cookie=4172f8dc4095cd53 in_seq=0 out_seq=0 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.762+0000 7f1abffff640 10 client.17802 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1abffff640 1 -- 172.21.15.77:0/952067079 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56459a930d70 con 0x56459aa4d060 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1abffff640 10 client.17802 handle_command_reply: tid=0 2024-09-15T22:21:41.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 client.17802 shutdown 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 msgr2=0x56459a9daf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1acc006010 tx=0x7f1acc005210 comp rx=0 tx=0).stop 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 shutdown_connections 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56459aa4d060 0x56459aa4f480 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56459a9d2970 0x56459a9e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56459a9c4a20 0x56459a9daf70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1abffff640 0 client.17802 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 --2- 172.21.15.77:0/952067079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56459a9c35d0 0x56459a9daa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 >> 172.21.15.77:0/952067079 conn(0x56459a98c310 msgr2=0x56459a9cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 shutdown_connections 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 1 -- 172.21.15.77:0/952067079 wait complete. 2024-09-15T22:21:41.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.763+0000 7f1ae8edb740 20 client.17802 trim_cache size 0 max 16384 2024-09-15T22:21:41.780 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 msgr2=0x7f1ad00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 0x7f1ad00a5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009270 tx=0x7f1ad8005bd0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 msgr2=0x7f1ae0197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c920 tx=0x7f1ad400cde0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 shutdown_connections 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ad00a33b0 0x7f1ad00a5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae0102ef0 0x7f1ae019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ae0102570 0x7f1ae0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 --2- 172.21.15.77:0/3731466917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae0100e20 0x7f1ae0197350 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 >> 172.21.15.77:0/3731466917 conn(0x7f1ae00fc7d0 msgr2=0x7f1ae0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:41.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 shutdown_connections 2024-09-15T22:21:41.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:41.764+0000 7f1ae6b02640 1 -- 172.21.15.77:0/3731466917 wait complete. 2024-09-15T22:21:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:41 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:41.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:41 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1773528647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:41 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:41 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1773528647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:42.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 6869fc35 --release --await' 2024-09-15T22:21:42.396 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 6869fc35 --release --await 2024-09-15T22:21:42.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.70 MB/sec execute 351 sec latency 22532.053 ms 2024-09-15T22:21:42.733 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:42 smithi077 ceph-mon[30980]: pgmap v287: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-09-15T22:21:42.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:42 smithi196 ceph-mon[35109]: pgmap v287: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-09-15T22:21:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:42 smithi117 ceph-mon[35021]: pgmap v287: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-09-15T22:21:43.070 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:21:43.142 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:43.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 711649 7.68 MB/sec execute 352 sec latency 23532.097 ms 2024-09-15T22:21:43.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 -- 172.21.15.77:0/3957596826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c100e20 msgr2=0x7fe29c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3957596826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c100e20 0x7fe29c101200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009900 tx=0x7fe28c02a480 comp rx=0 tx=0).stop 2024-09-15T22:21:43.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 -- 172.21.15.77:0/3957596826 shutdown_connections 2024-09-15T22:21:43.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3957596826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102ef0 0x7fe29c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3957596826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c102570 0x7fe29c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3957596826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c100e20 0x7fe29c101200 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 -- 172.21.15.77:0/3957596826 >> 172.21.15.77:0/3957596826 conn(0x7fe29c0fc7d0 msgr2=0x7fe29c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 -- 172.21.15.77:0/3957596826 shutdown_connections 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.648+0000 7fe2a2220640 1 -- 172.21.15.77:0/3957596826 wait complete. 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 Processor -- start 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 -- start start 2024-09-15T22:21:43.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c100e20 0x7fe29c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 0x7fe29c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1120b0 con 0x7fe29c102ef0 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c111f30 con 0x7fe29c100e20 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe2a2220640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c112230 con 0x7fe29c102570 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe29bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe29bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34208/0 (socket says 172.21.15.77:34208) 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe29bfff640 1 -- 172.21.15.77:0/3303975748 learned_addr learned my addr 172.21.15.77:0/3303975748 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.649+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c100e20 0x7fe29c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe293fff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 0x7fe29c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 -- 172.21.15.77:0/3303975748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 msgr2=0x7fe29c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 0x7fe29c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 -- 172.21.15.77:0/3303975748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c100e20 msgr2=0x7fe29c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c100e20 0x7fe29c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 -- 172.21.15.77:0/3303975748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c19d500 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe293fff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 0x7fe29c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe29bfff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe2997fa640 1 -- 172.21.15.77:0/3303975748 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294019070 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c19d7f0 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe2997fa640 1 -- 172.21.15.77:0/3303975748 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29400cb40 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe2997fa640 1 -- 172.21.15.77:0/3303975748 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294007650 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.650+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c1a50d0 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.652+0000 7fe2997fa640 1 -- 172.21.15.77:0/3303975748 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe29401c070 con 0x7fe29c102ef0 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.652+0000 7fe2a45f9740 1 Processor -- start 2024-09-15T22:21:43.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.652+0000 7fe2a45f9740 1 -- start start 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d59f20 0x562d61d5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61d6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d61cbf620 con 0x562d61d5a840 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d61c66e50 con 0x562d61d59f20 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562d61cbb5e0 con 0x562d61d69650 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2997fa640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 0x7fe2880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34222/0 (socket says 172.21.15.77:34222) 2024-09-15T22:21:43.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 -- 172.21.15.77:0/520876354 learned_addr learned my addr 172.21.15.77:0/520876354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe293fff640 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61d6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29bfff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 0x7fe2880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe293fff640 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d59f20 0x562d61d5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe2997fa640 1 -- 172.21.15.77:0/3303975748 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fe2940cf570 con 0x7fe29c102ef0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 -- 172.21.15.77:0/520876354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 msgr2=0x562d61d6ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61d6ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 -- 172.21.15.77:0/520876354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d59f20 msgr2=0x562d61d5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d59f20 0x562d61d5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 -- 172.21.15.77:0/520876354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d61c57d40 con 0x562d61d5a840 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.653+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009900 tx=0x7fe28c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc90b6832f83f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe29bfff640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 0x7fe2880a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe29409b380 tx=0x7fe29406d400 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe291ffb640 1 -- 172.21.15.77:0/520876354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c035030 con 0x562d61d5a840 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe291ffb640 1 -- 172.21.15.77:0/520876354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28c040070 con 0x562d61d5a840 2024-09-15T22:21:43.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe291ffb640 1 -- 172.21.15.77:0/520876354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0319d0 con 0x562d61d5a840 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 -- 172.21.15.77:0/520876354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 msgr2=0x562d61d69110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009900 tx=0x7fe28c004170 comp rx=0 tx=0).stop 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 -- 172.21.15.77:0/520876354 shutdown_connections 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61d6ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d5a840 0x562d61d69110 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/520876354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d59f20 0x562d61d5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 -- 172.21.15.77:0/520876354 >> 172.21.15.77:0/520876354 conn(0x562d61d565f0 msgr2=0x562d61d569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 -- 172.21.15.77:0/520876354 shutdown_connections 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.654+0000 7fe2a45f9740 1 -- 172.21.15.77:0/520876354 wait complete. 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 -- start start 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 Processor -- start 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d59f20 0x562d61d66590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d66ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d61d6c1c0 con 0x562d61d69650 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562d61c47f10 con 0x562d61d5a840 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562d61d6c340 con 0x562d61d59f20 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe2a45f9740 10 client.0 _ll_register_callbacks cb 0x562d61c6bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34234/0 (socket says 172.21.15.77:34234) 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe293fff640 1 -- 172.21.15.77:0/1523574528 learned_addr learned my addr 172.21.15.77:0/1523574528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe293fff640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d59f20 0x562d61d66590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.655+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d66ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 -- 172.21.15.77:0/1523574528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d59f20 msgr2=0x562d61d66590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d59f20 0x562d61d66590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 -- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 msgr2=0x562d61d66ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d66ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 -- 172.21.15.77:0/1523574528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d619ce580 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe293fff640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe284011430 tx=0x7fe2840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14801ae4facb4be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe27affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284005020 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28c040070 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fe2840051c0 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe284029050 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe2a45f9740 10 client.17814 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d61a98960 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.656+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28c040070 con 0x562d61d69650 2024-09-15T22:21:43.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe28400f040 con 0x562d61d69650 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 10 client.17814 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 10 client.17814 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 20 client.17814 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61d791f0 0x562d61d7b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 4 client.17814 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d61cccb90 con 0x562d61d791f0 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61d791f0 0x562d61d7b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe27affd640 10 client.17814 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.657+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61d791f0 0x562d61d7b5e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e91ee2a430f5b44 server_cookie=3062de139438e111 in_seq=0 out_seq=0 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe27affd640 1 -- 172.21.15.77:0/1523574528 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d61cccb90 con 0x562d61d791f0 2024-09-15T22:21:43.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe27affd640 10 client.17814 handle_command_reply: tid=0 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 client.17814 shutdown 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 msgr2=0x562d61d67010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe284011430 tx=0x7fe2840046c0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 shutdown_connections 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61d791f0 0x562d61d7b5e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d69650 0x562d61d67010 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d66ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/1523574528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d59f20 0x562d61d66590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 >> 172.21.15.77:0/1523574528 conn(0x562d61d565f0 msgr2=0x562d61d5c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe27affd640 0 client.17814 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:43.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 shutdown_connections 2024-09-15T22:21:43.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 1 -- 172.21.15.77:0/1523574528 wait complete. 2024-09-15T22:21:43.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.658+0000 7fe2a45f9740 20 client.17814 trim_cache size 0 max 16384 2024-09-15T22:21:43.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.675+0000 7fe2a45f9740 1 Processor -- start 2024-09-15T22:21:43.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.675+0000 7fe2a45f9740 1 -- start start 2024-09-15T22:21:43.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:43 smithi077 ceph-mon[30980]: pgmap v288: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:43.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:43 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:43.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:43 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:43.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:43 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d5a840 0x562d61d813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d69650 0x562d61d637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1ace70 con 0x562d61d59f20 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1ae9b0 con 0x562d61d69650 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1ad540 con 0x562d61d5a840 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34250/0 (socket says 172.21.15.77:34250) 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 -- 172.21.15.77:0/2313927034 learned_addr learned my addr 172.21.15.77:0/2313927034 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d5a840 0x562d61d813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d69650 0x562d61d637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 -- 172.21.15.77:0/2313927034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d5a840 msgr2=0x562d61d813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d5a840 0x562d61d813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 -- 172.21.15.77:0/2313927034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d69650 msgr2=0x562d61d637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d69650 0x562d61d637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.676+0000 7fe293fff640 1 -- 172.21.15.77:0/2313927034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d61d6bde0 con 0x562d61d59f20 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.677+0000 7fe293fff640 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe284013ba0 tx=0x7fe28400bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ada91349b7e82b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.677+0000 7fe2917fa640 1 -- 172.21.15.77:0/2313927034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28400be50 con 0x562d61d59f20 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.677+0000 7fe2917fa640 1 -- 172.21.15.77:0/2313927034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2840207e0 con 0x562d61d59f20 2024-09-15T22:21:43.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.677+0000 7fe2917fa640 1 -- 172.21.15.77:0/2313927034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840053c0 con 0x562d61d59f20 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- 172.21.15.77:0/2313927034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 msgr2=0x562d61d80e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe284013ba0 tx=0x7fe28400bb30 comp rx=0 tx=0).stop 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- 172.21.15.77:0/2313927034 shutdown_connections 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d69650 0x562d61d637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d5a840 0x562d61d813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/2313927034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d80e70 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- 172.21.15.77:0/2313927034 >> 172.21.15.77:0/2313927034 conn(0x562d61d565f0 msgr2=0x562d61d64e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- 172.21.15.77:0/2313927034 shutdown_connections 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- 172.21.15.77:0/2313927034 wait complete. 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.678+0000 7fe2a45f9740 1 -- start start 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 Processor -- start 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61de2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1ae2e0 con 0x562d61d59f20 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1adc10 con 0x562d61d5a840 2024-09-15T22:21:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c1aae70 con 0x562d61d69650 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe2a45f9740 10 client.0 _ll_register_callbacks cb 0x7fe284013c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34258/0 (socket says 172.21.15.77:34258) 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 -- 172.21.15.77:0/3232638025 learned_addr learned my addr 172.21.15.77:0/3232638025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 -- 172.21.15.77:0/3232638025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 msgr2=0x562d61de2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61de2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 -- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 msgr2=0x562d61d6e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d6e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.679+0000 7fe293fff640 1 -- 172.21.15.77:0/3232638025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d61de3430 con 0x562d61d59f20 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe293fff640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe284005940 tx=0x7fe284020990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a6abd4e41bfe54 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28400be50 con 0x562d61d59f20 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fe284004230 con 0x562d61d59f20 2024-09-15T22:21:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2840207e0 con 0x562d61d59f20 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe284020ac0 con 0x562d61d59f20 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 10 client.17820 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d61de4750 con 0x562d61d59f20 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2840207e0 con 0x562d61d59f20 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe28400f470 con 0x562d61d59f20 2024-09-15T22:21:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 10 client.17820 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 10 client.17820 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.680+0000 7fe2a45f9740 20 client.17820 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61de4db0 0x562d61de71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe2a45f9740 4 client.17820 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6869fc35", "release": true, "await": true}] 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6869fc35", "release": true, "await": true}) -- 0x562d61cc70a0 con 0x562d61de4db0 2024-09-15T22:21:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61de4db0 0x562d61de71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe27affd640 10 client.17820 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.681+0000 7fe29b7fe640 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61de4db0 0x562d61de71d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8b62daa9e2bd57 server_cookie=c63b1a9a0bb5da57 in_seq=0 out_seq=0 2024-09-15T22:21:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.689+0000 7fe27affd640 1 -- 172.21.15.77:0/3232638025 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562d61cc70a0 con 0x562d61de4db0 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.689+0000 7fe27affd640 10 client.17820 handle_command_reply: tid=0 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.689+0000 7fe2a45f9740 1 client.17820 shutdown 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.689+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 msgr2=0x562d61d6db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.689+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe284005940 tx=0x7fe284020990 comp rx=0 tx=0).stop 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 shutdown_connections 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562d61de4db0 0x562d61de71d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe27affd640 0 client.17820 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562d61d69650 0x562d61de2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d61d5a840 0x562d61d6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 --2- 172.21.15.77:0/3232638025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d61d59f20 0x562d61d6db50 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 >> 172.21.15.77:0/3232638025 conn(0x562d61d565f0 msgr2=0x562d61d54f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 shutdown_connections 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 1 -- 172.21.15.77:0/3232638025 wait complete. 2024-09-15T22:21:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a45f9740 20 client.17820 trim_cache size 0 max 16384 2024-09-15T22:21:43.696 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:21:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:21:43.696 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 32, 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "6869fc35": { 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 32, 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:21:43.697 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.3, 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:43.698 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 msgr2=0x7fe2880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 0x7fe2880a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe29409b380 tx=0x7fe29406d400 comp rx=0 tx=0).stop 2024-09-15T22:21:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 msgr2=0x7fe29c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.690+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b970 tx=0x7fe29400be30 comp rx=0 tx=0).stop 2024-09-15T22:21:43.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 shutdown_connections 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe2880a33b0 0x7fe2880a5870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe29c102ef0 0x7fe29c19ce20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c102570 0x7fe29c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 --2- 172.21.15.77:0/3303975748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe29c100e20 0x7fe29c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 >> 172.21.15.77:0/3303975748 conn(0x7fe29c0fc7d0 msgr2=0x7fe29c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 shutdown_connections 2024-09-15T22:21:43.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:43.691+0000 7fe2a2220640 1 -- 172.21.15.77:0/3303975748 wait complete. 2024-09-15T22:21:43.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:43 smithi196 ceph-mon[35109]: pgmap v288: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:43.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:43 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:43.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:43 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:43.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:43 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:21:43.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:43 smithi117 ceph-mon[35021]: pgmap v288: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:43.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:43 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:43.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:43 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:43.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:43 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:21:43.947 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:44.448 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6869fc35', total seconds elapsed: 27.5 2024-09-15T22:21:44.448 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-15T22:21:44.449 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:21:44.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 713728 7.69 MB/sec execute 353 sec latency 23743.757 ms 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- 172.21.15.77:0/3250550459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 msgr2=0x7f54841021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 --2- 172.21.15.77:0/3250550459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54841021f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5480009900 tx=0x7f548002a410 comp rx=0 tx=0).stop 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- 172.21.15.77:0/3250550459 shutdown_connections 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 --2- 172.21.15.77:0/3250550459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 0x7f548410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 --2- 172.21.15.77:0/3250550459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 0x7f5484106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:44.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 --2- 172.21.15.77:0/3250550459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54841021f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- 172.21.15.77:0/3250550459 >> 172.21.15.77:0/3250550459 conn(0x7f54840fd8d0 msgr2=0x7f54840ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- 172.21.15.77:0/3250550459 shutdown_connections 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- 172.21.15.77:0/3250550459 wait complete. 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 Processor -- start 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.986+0000 7f548b9af640 1 -- start start 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 0x7f5484074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 0x7f5484074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5484111f60 con 0x7f5484101e10 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5484111de0 con 0x7f54841027c0 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54841120e0 con 0x7f5484106f40 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34278/0 (socket says 172.21.15.77:34278) 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 -- 172.21.15.77:0/3376380320 learned_addr learned my addr 172.21.15.77:0/3376380320 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f5489f4c640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 0x7f5484074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f5488f4a640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 0x7f5484074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 -- 172.21.15.77:0/3376380320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 msgr2=0x7f5484074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 0x7f5484074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 -- 172.21.15.77:0/3376380320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 msgr2=0x7f5484074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 0x7f5484074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 -- 172.21.15.77:0/3376380320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54841a6920 con 0x7f5484101e10 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548974b640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5480006010 tx=0x7f5480004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f547a7fc640 1 -- 172.21.15.77:0/3376380320 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548003f030 con 0x7f5484101e10 2024-09-15T22:21:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f547a7fc640 1 -- 172.21.15.77:0/3376380320 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5480004440 con 0x7f5484101e10 2024-09-15T22:21:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841a6bb0 con 0x7f5484101e10 2024-09-15T22:21:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.987+0000 7f547a7fc640 1 -- 172.21.15.77:0/3376380320 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480035070 con 0x7f5484101e10 2024-09-15T22:21:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.988+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841a7090 con 0x7f5484101e10 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.990+0000 7f547a7fc640 1 -- 172.21.15.77:0/3376380320 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5480042030 con 0x7f5484101e10 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.990+0000 7f548dd88740 1 Processor -- start 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.990+0000 7f548dd88740 1 -- start start 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1761595d0 0x55b176159a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b176166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b1760bf2f0 con 0x55b176164370 2024-09-15T22:21:45.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b176066cf0 con 0x55b1761595d0 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f548dd88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1760bb2b0 con 0x55b17615ab40 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b176166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39732/0 (socket says 172.21.15.77:39732) 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5489f4c640 1 -- 172.21.15.77:0/110190979 learned_addr learned my addr 172.21.15.77:0/110190979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f547a7fc640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 0x7f54700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5489f4c640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 0x7f54700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f547a7fc640 1 -- 172.21.15.77:0/3376380320 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54800f03f0 con 0x7f5484101e10 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 -- 172.21.15.77:0/110190979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 msgr2=0x55b176159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 -- 172.21.15.77:0/110190979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1761595d0 msgr2=0x55b176159a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1761595d0 0x55b176159a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5488f4a640 1 -- 172.21.15.77:0/110190979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17604b3a0 con 0x55b176164370 2024-09-15T22:21:45.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.991+0000 7f5489f4c640 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.992+0000 7f5488f4a640 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b176166760 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5474009bf0 tx=0x7f5474011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d60c89844ab4b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.992+0000 7f546f7fe640 1 -- 172.21.15.77:0/110190979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474009da0 con 0x55b176164370 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.992+0000 7f546f7fe640 1 -- 172.21.15.77:0/110190979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474005020 con 0x55b176164370 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.992+0000 7f546f7fe640 1 -- 172.21.15.77:0/110190979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474004bc0 con 0x55b176164370 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.992+0000 7f5489f4c640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 0x7f54700a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f547c0049d0 tx=0x7f547c004ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- 172.21.15.77:0/110190979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 msgr2=0x55b176166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b176166760 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5474009bf0 tx=0x7f5474011810 comp rx=0 tx=0).stop 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- 172.21.15.77:0/110190979 shutdown_connections 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b176166760 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1761595d0 0x55b176159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 --2- 172.21.15.77:0/110190979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b17615ab40 0x55b176159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- 172.21.15.77:0/110190979 >> 172.21.15.77:0/110190979 conn(0x55b176122310 msgr2=0x55b176155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- 172.21.15.77:0/110190979 shutdown_connections 2024-09-15T22:21:45.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- 172.21.15.77:0/110190979 wait complete. 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 -- start start 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.993+0000 7f548dd88740 1 Processor -- start 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 0x55b1761615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b176167070 con 0x55b17615ab40 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b17607a060 con 0x55b176164370 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1761671f0 con 0x55b1761595d0 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548dd88740 10 client.0 _ll_register_callbacks cb 0x55b175dcd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5488f4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5488f4a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34298/0 (socket says 172.21.15.77:34298) 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5488f4a640 1 -- 172.21.15.77:0/423813525 learned_addr learned my addr 172.21.15.77:0/423813525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5489f4c640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 0x55b1761615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 -- 172.21.15.77:0/423813525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 msgr2=0x55b1761615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 0x55b1761615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 -- 172.21.15.77:0/423813525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 msgr2=0x55b176160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f548974b640 1 -- 172.21.15.77:0/423813525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b175e303c0 con 0x55b1761595d0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5488f4a640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.994+0000 7f5489f4c640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 0x55b1761615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548974b640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176163d30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54800ed7b0 tx=0x7f54800b8790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffefbd8fa951544a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480049040 con 0x55b1761595d0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5474005020 con 0x55b1761595d0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54800eda30 con 0x55b1761595d0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54800bc3f0 con 0x55b1761595d0 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 10 client.26857 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:45.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b1761670d0 con 0x55b1761595d0 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474005020 con 0x55b1761595d0 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f54800d50d0 con 0x55b1761595d0 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 10 client.26857 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 10 client.26857 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 20 client.26857 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b176174000 0x55b1761763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 4 client.26857 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.995+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1760cc860 con 0x55b176174000 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.996+0000 7f5488f4a640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b176174000 0x55b1761763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.996+0000 7f546cff9640 10 client.26857 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.996+0000 7f5488f4a640 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b176174000 0x55b1761763f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca415b48e89e1e0e server_cookie=23df5c15aab046cf in_seq=0 out_seq=0 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f546cff9640 1 -- 172.21.15.77:0/423813525 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1760cc860 con 0x55b176174000 2024-09-15T22:21:45.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f546cff9640 10 client.26857 handle_command_reply: tid=0 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 client.26857 shutdown 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 msgr2=0x55b176163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176163d30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54800ed7b0 tx=0x7f54800b8790 comp rx=0 tx=0).stop 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 shutdown_connections 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b176174000 0x55b1761763f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b176164370 0x55b1761615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b17615ab40 0x55b176160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 --2- 172.21.15.77:0/423813525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176163d30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 >> 172.21.15.77:0/423813525 conn(0x55b176122310 msgr2=0x55b17615b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f546cff9640 0 client.26857 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 shutdown_connections 2024-09-15T22:21:45.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 1 -- 172.21.15.77:0/423813525 wait complete. 2024-09-15T22:21:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:44.997+0000 7f548dd88740 20 client.26857 trim_cache size 0 max 16384 2024-09-15T22:21:45.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.011+0000 7f548dd88740 1 Processor -- start 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 -- start start 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b176170370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17617bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54841b1070 con 0x55b176164370 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54841b2bb0 con 0x55b17615ab40 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548dd88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54841b1740 con 0x55b1761595d0 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17617bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39758/0 (socket says 172.21.15.77:39758) 2024-09-15T22:21:45.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 -- 172.21.15.77:0/2904756064 learned_addr learned my addr 172.21.15.77:0/2904756064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17617bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34300/0 (socket says 172.21.15.77:34300) 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f5488f4a640 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b176170370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 -- 172.21.15.77:0/2904756064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 msgr2=0x55b176170370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b176170370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 -- 172.21.15.77:0/2904756064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 msgr2=0x55b17617bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17617bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.012+0000 7f548974b640 1 -- 172.21.15.77:0/2904756064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17615a760 con 0x55b1761595d0 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.013+0000 7f548974b640 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54800bc5b0 tx=0x7f54800ba900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db02e613ca962eec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.013+0000 7f546effd640 1 -- 172.21.15.77:0/2904756064 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480049020 con 0x55b1761595d0 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.013+0000 7f546effd640 1 -- 172.21.15.77:0/2904756064 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54800bc650 con 0x55b1761595d0 2024-09-15T22:21:45.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.013+0000 7f546effd640 1 -- 172.21.15.77:0/2904756064 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800c4b20 con 0x55b1761595d0 2024-09-15T22:21:45.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 -- 172.21.15.77:0/2904756064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 msgr2=0x55b176155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54800bc5b0 tx=0x7f54800ba900 comp rx=0 tx=0).stop 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 -- 172.21.15.77:0/2904756064 shutdown_connections 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17617bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b176170370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 --2- 172.21.15.77:0/2904756064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b176155df0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 -- 172.21.15.77:0/2904756064 >> 172.21.15.77:0/2904756064 conn(0x55b176122310 msgr2=0x55b1761556e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 -- 172.21.15.77:0/2904756064 shutdown_connections 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.014+0000 7f548dd88740 1 -- 172.21.15.77:0/2904756064 wait complete. 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.015+0000 7f548dd88740 1 -- start start 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.015+0000 7f548dd88740 1 Processor -- start 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.015+0000 7f548dd88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.015+0000 7f548dd88740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b17616ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.015+0000 7f548dd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17615d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39772/0 (socket says 172.21.15.77:39772) 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f5488f4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b17616ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17615d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f5488f4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b17616ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37080/0 (socket says 172.21.15.77:37080) 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f5489f4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17615d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34308/0 (socket says 172.21.15.77:34308) 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 -- 172.21.15.77:0/2941796178 learned_addr learned my addr 172.21.15.77:0/2941796178 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54841b24e0 con 0x55b176164370 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54841b1e10 con 0x55b17615ab40 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54841af070 con 0x55b1761595d0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 -- 172.21.15.77:0/2941796178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 msgr2=0x55b17616ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548dd88740 10 client.0 _ll_register_callbacks cb 0x7f5480032b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b17616ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 -- 172.21.15.77:0/2941796178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 msgr2=0x55b17615d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17615d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17615db70 con 0x55b1761595d0 2024-09-15T22:21:45.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1760c6d70 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f548974b640 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f54800bc830 tx=0x7f54800eda30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bf6a507ebfa77e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f546cff9640 10 client.26863 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.016+0000 7f546cff9640 1 -- 172.21.15.77:0/2941796178 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480049070 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f546cff9640 1 -- 172.21.15.77:0/2941796178 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54800c4d20 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f546cff9640 1 -- 172.21.15.77:0/2941796178 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54800edbb0 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 10 client.26863 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55b17615db70 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f546cff9640 1 -- 172.21.15.77:0/2941796178 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f54800d02d0 con 0x55b1761595d0 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 10 client.26863 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 10 client.26863 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 20 client.26863 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b1761e6d30 0x55b1761e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 4 client.26863 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.017+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b176043170 con 0x55b1761e6d30 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f5488f4a640 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b1761e6d30 0x55b1761e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f546cff9640 10 client.26863 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f5488f4a640 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b1761e6d30 0x55b1761e9150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=974c637fd301a9bd server_cookie=1964abe95b1289b2 in_seq=0 out_seq=0 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f546cff9640 1 -- 172.21.15.77:0/2941796178 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b176043170 con 0x55b1761e6d30 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f546cff9640 10 client.26863 handle_command_reply: tid=0 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.018+0000 7f548dd88740 1 client.26863 shutdown 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 msgr2=0x55b17616c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f54800bc830 tx=0x7f54800eda30 comp rx=0 tx=0).stop 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 shutdown_connections 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b1761e6d30 0x55b1761e9150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b176164370 0x55b17615d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b17615ab40 0x55b17616ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 --2- 172.21.15.77:0/2941796178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b1761595d0 0x55b17616c4e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 >> 172.21.15.77:0/2941796178 conn(0x55b176122310 msgr2=0x55b1761544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f546cff9640 0 client.26863 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 shutdown_connections 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 1 -- 172.21.15.77:0/2941796178 wait complete. 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.019+0000 7f548dd88740 20 client.26863 trim_cache size 0 max 16384 2024-09-15T22:21:45.026 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 msgr2=0x7f54700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 0x7f54700a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f547c0049d0 tx=0x7f547c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 msgr2=0x7f54840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5480006010 tx=0x7f5480004040 comp rx=0 tx=0).stop 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 shutdown_connections 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54700a33b0 0x7f54700a5870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5484106f40 0x7f5484074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54841027c0 0x7f5484074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 --2- 172.21.15.77:0/3376380320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5484101e10 0x7f54840744f0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 >> 172.21.15.77:0/3376380320 conn(0x7f54840fd8d0 msgr2=0x7f54840fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 shutdown_connections 2024-09-15T22:21:45.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:45.020+0000 7f548b9af640 1 -- 172.21.15.77:0/3376380320 wait complete. 2024-09-15T22:21:45.033 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:45.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 715221 7.69 MB/sec execute 354 sec latency 58.336 ms 2024-09-15T22:21:45.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:45 smithi077 ceph-mon[30980]: pgmap v289: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:45.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:45 smithi117 ceph-mon[35021]: pgmap v289: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:45.946 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:21:46.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:45 smithi196 ceph-mon[35109]: pgmap v289: 129 pgs: 129 active+clean; 477 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-09-15T22:21:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 -- 172.21.15.77:0/2463107110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc100e20 msgr2=0x7f8bbc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 --2- 172.21.15.77:0/2463107110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc100e20 0x7f8bbc101200 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009900 tx=0x7f8bb802a480 comp rx=0 tx=0).stop 2024-09-15T22:21:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 -- 172.21.15.77:0/2463107110 shutdown_connections 2024-09-15T22:21:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 --2- 172.21.15.77:0/2463107110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 --2- 172.21.15.77:0/2463107110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc102570 0x7f8bbc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 --2- 172.21.15.77:0/2463107110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc100e20 0x7f8bbc101200 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 -- 172.21.15.77:0/2463107110 >> 172.21.15.77:0/2463107110 conn(0x7f8bbc0fc7d0 msgr2=0x7f8bbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 -- 172.21.15.77:0/2463107110 shutdown_connections 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.425+0000 7f8bc408b640 1 -- 172.21.15.77:0/2463107110 wait complete. 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.426+0000 7f8bc408b640 1 Processor -- start 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.426+0000 7f8bc408b640 1 -- start start 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.426+0000 7f8bc408b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc100e20 0x7f8bbc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.426+0000 7f8bc408b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102570 0x7f8bbc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:46.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.426+0000 7f8bc408b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc408b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc110f80 con 0x7f8bbc102ef0 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc408b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc110e00 con 0x7f8bbc102570 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc408b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bbc111100 con 0x7f8bbc100e20 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc2628640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc2628640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34316/0 (socket says 172.21.15.77:34316) 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc2628640 1 -- 172.21.15.77:0/254148934 learned_addr learned my addr 172.21.15.77:0/254148934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.427+0000 7f8bc1626640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102570 0x7f8bbc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 -- 172.21.15.77:0/254148934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc100e20 msgr2=0x7f8bbc103750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc100e20 0x7f8bbc103750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 -- 172.21.15.77:0/254148934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102570 msgr2=0x7f8bbc103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102570 0x7f8bbc103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 -- 172.21.15.77:0/254148934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc1a69a0 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bc2628640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400eaa0 tx=0x7f8bb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.428+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400cdb0 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.429+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bbc1a6c90 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.429+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bbc1a71d0 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.429+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4005020 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.429+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40116d0 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.430+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba0005290 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.432+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8bb4004a40 con 0x7f8bbc102ef0 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.433+0000 7f8bb2ffd640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 0x7f8ba80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:46.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.433+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f8bb40d0180 con 0x7f8bbc102ef0 2024-09-15T22:21:46.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.434+0000 7f8bc1e27640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 0x7f8ba80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:46.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.436+0000 7f8bc1e27640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 0x7f8ba80a5970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80314e0 tx=0x7f8bb8000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:46.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.441+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bb4016030 con 0x7f8bbc102ef0 2024-09-15T22:21:46.482 INFO:tasks.workunit.client.0.smithi077.stdout: 1 717343 7.70 MB/sec execute 355 sec latency 70.879 ms 2024-09-15T22:21:46.619 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.739+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba0005850 con 0x7f8bbc102ef0 2024-09-15T22:21:46.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.740+0000 7f8bb2ffd640 1 -- 172.21.15.77:0/254148934 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f8bb409c770 con 0x7f8bbc102ef0 2024-09-15T22:21:46.743 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:21:46.743 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:21:46.743 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:21:46.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.742+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 msgr2=0x7f8ba80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:46.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.742+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 0x7f8ba80a5970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80314e0 tx=0x7f8bb8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.742+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 msgr2=0x7f8bbc1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.742+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400eaa0 tx=0x7f8bb400ef60 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 shutdown_connections 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ba80a34b0 0x7f8ba80a5970 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bbc102ef0 0x7f8bbc1041d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bbc102570 0x7f8bbc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 --2- 172.21.15.77:0/254148934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc100e20 0x7f8bbc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 >> 172.21.15.77:0/254148934 conn(0x7f8bbc0fc7d0 msgr2=0x7f8bbc06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 shutdown_connections 2024-09-15T22:21:46.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:46.743+0000 7f8bc408b640 1 -- 172.21.15.77:0/254148934 wait complete. 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 -- 172.21.15.77:0/3977111322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c101df0 msgr2=0x7fab8c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 --2- 172.21.15.77:0/3977111322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c101df0 0x7fab8c1021d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fab80009900 tx=0x7fab8002a410 comp rx=0 tx=0).stop 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 -- 172.21.15.77:0/3977111322 shutdown_connections 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 --2- 172.21.15.77:0/3977111322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c106f20 0x7fab8c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 --2- 172.21.15.77:0/3977111322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 0x7fab8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 --2- 172.21.15.77:0/3977111322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c101df0 0x7fab8c1021d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 -- 172.21.15.77:0/3977111322 >> 172.21.15.77:0/3977111322 conn(0x7fab8c0fd8b0 msgr2=0x7fab8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 -- 172.21.15.77:0/3977111322 shutdown_connections 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.307+0000 7fab930c9640 1 -- 172.21.15.77:0/3977111322 wait complete. 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 Processor -- start 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 -- start start 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c101df0 0x7fab8c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 0x7fab8c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111f40 con 0x7fab8c106f20 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111dc0 con 0x7fab8c101df0 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab930c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1120c0 con 0x7fab8c1027a0 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab91666640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab91666640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34326/0 (socket says 172.21.15.77:34326) 2024-09-15T22:21:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab90e65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c101df0 0x7fab8c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab91666640 1 -- 172.21.15.77:0/2800959657 learned_addr learned my addr 172.21.15.77:0/2800959657 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.308+0000 7fab8bfff640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 0x7fab8c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 -- 172.21.15.77:0/2800959657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 msgr2=0x7fab8c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 0x7fab8c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 -- 172.21.15.77:0/2800959657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c101df0 msgr2=0x7fab8c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c101df0 0x7fab8c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 -- 172.21.15.77:0/2800959657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c19d3d0 con 0x7fab8c106f20 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab91666640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fab84009fd0 tx=0x7fab8400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab89ffb640 1 -- 172.21.15.77:0/2800959657 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84017070 con 0x7fab8c106f20 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c19d6c0 con 0x7fab8c106f20 2024-09-15T22:21:47.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab89ffb640 1 -- 172.21.15.77:0/2800959657 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab84003e50 con 0x7fab8c106f20 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab89ffb640 1 -- 172.21.15.77:0/2800959657 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840106c0 con 0x7fab8c106f20 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.309+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a4fa0 con 0x7fab8c106f20 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.312+0000 7fab89ffb640 1 -- 172.21.15.77:0/2800959657 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fab84003970 con 0x7fab8c106f20 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.312+0000 7fab954a2740 1 Processor -- start 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.312+0000 7fab954a2740 1 -- start start 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.312+0000 7fab89ffb640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 0x7fab7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab89ffb640 1 -- 172.21.15.77:0/2800959657 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fab840cfd30 con 0x7fab8c106f20 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff76a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab8bfff640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 0x7fab7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624cfedc2f0 con 0x5624cff77b40 2024-09-15T22:21:47.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624cfe83cf0 con 0x5624cff85780 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab954a2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624cfed82b0 con 0x5624cff765d0 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab90e65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab90e65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34332/0 (socket says 172.21.15.77:34332) 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab90e65640 1 -- 172.21.15.77:0/1833791349 learned_addr learned my addr 172.21.15.77:0/1833791349 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab8bfff640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff76a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 -- 172.21.15.77:0/1833791349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 msgr2=0x5624cff76a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff76a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 -- 172.21.15.77:0/1833791349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 msgr2=0x5624cff76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 -- 172.21.15.77:0/1833791349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cfe683a0 con 0x5624cff85780 2024-09-15T22:21:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab90e65640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab8bfff640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 0x7fab7c0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab7800bfd0 tx=0x7fab7800bb40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab91666640 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff87b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab840cd0f0 tx=0x7fab8409b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56c37d531dcd971 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.313+0000 7fab76ffd640 1 -- 172.21.15.77:0/1833791349 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84020040 con 0x5624cff85780 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab76ffd640 1 -- 172.21.15.77:0/1833791349 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab840a48f0 con 0x5624cff85780 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab76ffd640 1 -- 172.21.15.77:0/1833791349 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840cd500 con 0x5624cff85780 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 -- 172.21.15.77:0/1833791349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 msgr2=0x5624cff87b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff87b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab840cd0f0 tx=0x7fab8409b750 comp rx=0 tx=0).stop 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 -- 172.21.15.77:0/1833791349 shutdown_connections 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff87b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff76a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 --2- 172.21.15.77:0/1833791349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 -- 172.21.15.77:0/1833791349 >> 172.21.15.77:0/1833791349 conn(0x5624cff3f310 msgr2=0x5624cff72c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 -- 172.21.15.77:0/1833791349 shutdown_connections 2024-09-15T22:21:47.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.314+0000 7fab954a2740 1 -- 172.21.15.77:0/1833791349 wait complete. 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 -- start start 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 Processor -- start 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff77b40 0x5624cff824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5624cff88420 con 0x5624cff85780 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5624cfe97060 con 0x5624cff77b40 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624cff885a0 con 0x5624cff765d0 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab954a2740 10 client.0 _ll_register_callbacks cb 0x5624cfbea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab91666640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab90e65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab91666640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34346/0 (socket says 172.21.15.77:34346) 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab91666640 1 -- 172.21.15.77:0/3314762145 learned_addr learned my addr 172.21.15.77:0/3314762145 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab90e65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff85210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39816/0 (socket says 172.21.15.77:39816) 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab8bfff640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff77b40 0x5624cff824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab91666640 1 -- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 msgr2=0x5624cff85210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.315+0000 7fab91666640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff85210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab91666640 1 -- 172.21.15.77:0/3314762145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff77b40 msgr2=0x5624cff824c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab91666640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff77b40 0x5624cff824c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab91666640 1 -- 172.21.15.77:0/3314762145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cfc4d3c0 con 0x5624cff85780 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab91666640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fab840cd6f0 tx=0x7fab8409b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1368f39050f9aed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840b8030 con 0x5624cff85780 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fab8409a030 con 0x5624cff85780 2024-09-15T22:21:47.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab840a48f0 con 0x5624cff85780 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab84020050 con 0x5624cff85780 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab954a2740 10 client.17856 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5624cff93f70 con 0x5624cff85780 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab840a48f0 con 0x5624cff85780 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab840b4070 con 0x5624cff85780 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.316+0000 7fab954a2740 10 client.17856 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab954a2740 10 client.17856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab954a2740 20 client.17856 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624cff952c0 0x5624cff976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab954a2740 4 client.17856 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624cfee9860 con 0x5624cff952c0 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab90e65640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624cff952c0 0x5624cff976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab90e65640 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624cff952c0 0x5624cff976b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7723f33055d7dff9 server_cookie=43009a71ad3604a2 in_seq=0 out_seq=0 2024-09-15T22:21:47.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.317+0000 7fab6bfff640 10 client.17856 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab6bfff640 1 -- 172.21.15.77:0/3314762145 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624cfee9860 con 0x5624cff952c0 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab6bfff640 10 client.17856 handle_command_reply: tid=0 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 client.17856 shutdown 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 msgr2=0x5624cff82c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fab840cd6f0 tx=0x7fab8409b7c0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 shutdown_connections 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624cff952c0 0x5624cff976b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff85780 0x5624cff82c50 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff77b40 0x5624cff824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 --2- 172.21.15.77:0/3314762145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff85210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 >> 172.21.15.77:0/3314762145 conn(0x5624cff3f310 msgr2=0x5624cff789c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab6bfff640 0 client.17856 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:21:47.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 shutdown_connections 2024-09-15T22:21:47.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 1 -- 172.21.15.77:0/3314762145 wait complete. 2024-09-15T22:21:47.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.318+0000 7fab954a2740 20 client.17856 trim_cache size 0 max 16384 2024-09-15T22:21:47.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 Processor -- start 2024-09-15T22:21:47.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 -- start start 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1acd30 con 0x5624cff77b40 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ae870 con 0x5624cff85780 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab954a2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ad400 con 0x5624cff765d0 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34348/0 (socket says 172.21.15.77:34348) 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab8bfff640 1 -- 172.21.15.77:0/3184587894 learned_addr learned my addr 172.21.15.77:0/3184587894 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab91666640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.334+0000 7fab90e65640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 -- 172.21.15.77:0/3184587894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 msgr2=0x5624cff72d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff72d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 -- 172.21.15.77:0/3184587894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 msgr2=0x5624cff86b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff86b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 -- 172.21.15.77:0/3184587894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cff833d0 con 0x5624cff77b40 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab91666640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff86b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab8bfff640 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fab78005d00 tx=0x7fab7800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4545cc14be92b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab767fc640 1 -- 172.21.15.77:0/3184587894 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800ecf0 con 0x5624cff77b40 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab767fc640 1 -- 172.21.15.77:0/3184587894 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7801edd0 con 0x5624cff77b40 2024-09-15T22:21:47.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.335+0000 7fab767fc640 1 -- 172.21.15.77:0/3184587894 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78010970 con 0x5624cff77b40 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 -- 172.21.15.77:0/3184587894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 msgr2=0x5624cff84430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fab78005d00 tx=0x7fab7800b410 comp rx=0 tx=0).stop 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 -- 172.21.15.77:0/3184587894 shutdown_connections 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff86b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff77b40 0x5624cff84430 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 --2- 172.21.15.77:0/3184587894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff765d0 0x5624cff72d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 -- 172.21.15.77:0/3184587894 >> 172.21.15.77:0/3184587894 conn(0x5624cff3f310 msgr2=0x5624cff83d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 -- 172.21.15.77:0/3184587894 shutdown_connections 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.336+0000 7fab954a2740 1 -- 172.21.15.77:0/3184587894 wait complete. 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 -- start start 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 Processor -- start 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 0x5624cff9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 0x5624cff7f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff7f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ae1a0 con 0x5624cff765d0 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1adad0 con 0x5624cff85780 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1aad30 con 0x5624cff77b40 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab954a2740 10 client.0 _ll_register_callbacks cb 0x7fab80005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:47.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab90e65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 0x5624cff9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 0x5624cff7f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab8bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 0x5624cff7f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39832/0 (socket says 172.21.15.77:39832) 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.337+0000 7fab8bfff640 1 -- 172.21.15.77:0/1367965244 learned_addr learned my addr 172.21.15.77:0/1367965244 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab91666640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff7f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 -- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 msgr2=0x5624cff7f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 0x5624cff7f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 -- 172.21.15.77:0/1367965244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 msgr2=0x5624cff7f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff7f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 -- 172.21.15.77:0/1367965244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cff7ff70 con 0x5624cff765d0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab91666640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff7f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab90e65640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 0x5624cff9aae0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fab80005c40 tx=0x7fab80031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd17994a5c5da4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:47.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80035040 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fab80005040 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7801edd0 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab80056050 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab954a2740 10 client.17862 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5624cffffc50 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.338+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab80004750 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab80003710 con 0x5624cff765d0 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 10 client.17862 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 10 client.17862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 20 client.17862 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624d0000400 0x5624d0002820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:47.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 4 client.17862 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fab7801edd0 con 0x5624d0000400 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab8bfff640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624d0000400 0x5624d0002820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab6bfff640 10 client.17862 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.339+0000 7fab8bfff640 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624d0000400 0x5624d0002820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c09cfe57038b6036 server_cookie=5e799f128b901c00 in_seq=0 out_seq=0 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab6bfff640 1 -- 172.21.15.77:0/1367965244 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fab7801edd0 con 0x5624d0000400 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab6bfff640 10 client.17862 handle_command_reply: tid=0 2024-09-15T22:21:47.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 client.17862 shutdown 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 msgr2=0x5624cff9aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 0x5624cff9aae0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fab80005c40 tx=0x7fab80031bb0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 shutdown_connections 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5624d0000400 0x5624d0002820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5624cff85780 0x5624cff7f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5624cff77b40 0x5624cff7f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 --2- 172.21.15.77:0/1367965244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5624cff765d0 0x5624cff9aae0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab6bfff640 0 client.17862 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 >> 172.21.15.77:0/1367965244 conn(0x5624cff3f310 msgr2=0x5624cff90830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 shutdown_connections 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.340+0000 7fab954a2740 1 -- 172.21.15.77:0/1367965244 wait complete. 2024-09-15T22:21:47.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab954a2740 20 client.17862 trim_cache size 0 max 16384 2024-09-15T22:21:47.349 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:47.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 msgr2=0x7fab7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 0x7fab7c0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab7800bfd0 tx=0x7fab7800bb40 comp rx=0 tx=0).stop 2024-09-15T22:21:47.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 msgr2=0x7fab8c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:47.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fab84009fd0 tx=0x7fab8400b260 comp rx=0 tx=0).stop 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 shutdown_connections 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fab7c0a33b0 0x7fab7c0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab8c106f20 0x7fab8c19ccf0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c1027a0 0x7fab8c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 --2- 172.21.15.77:0/2800959657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab8c101df0 0x7fab8c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 >> 172.21.15.77:0/2800959657 conn(0x7fab8c0fd8b0 msgr2=0x7fab8c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.341+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 shutdown_connections 2024-09-15T22:21:47.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:47.342+0000 7fab930c9640 1 -- 172.21.15.77:0/2800959657 wait complete. 2024-09-15T22:21:47.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 718819 7.70 MB/sec execute 356 sec latency 35.302 ms 2024-09-15T22:21:47.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:47 smithi117 ceph-mon[35021]: pgmap v290: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-15T22:21:47.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:47 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/254148934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:47.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:47 smithi077 ceph-mon[30980]: pgmap v290: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-15T22:21:47.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:47 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/254148934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:47.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:47 smithi196 ceph-mon[35109]: pgmap v290: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-09-15T22:21:47.905 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:47 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/254148934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:47.931 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-15T22:21:47.931 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-15T22:21:48.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 721282 7.72 MB/sec execute 357 sec latency 13.912 ms 2024-09-15T22:21:48.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:21:48.746 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 -- 172.21.15.77:0/1883262855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20102ef0 msgr2=0x7fbb2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1883262855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20102ef0 0x7fbb2010da40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ca0 tx=0x7fbb1802a950 comp rx=0 tx=0).stop 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 -- 172.21.15.77:0/1883262855 shutdown_connections 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1883262855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20102ef0 0x7fbb2010da40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1883262855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 0x7fbb201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1883262855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20100e20 0x7fbb20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 -- 172.21.15.77:0/1883262855 >> 172.21.15.77:0/1883262855 conn(0x7fbb200fc7d0 msgr2=0x7fbb200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 -- 172.21.15.77:0/1883262855 shutdown_connections 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 -- 172.21.15.77:0/1883262855 wait complete. 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.241+0000 7fbb2818e640 1 Processor -- start 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 -- start start 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 0x7fbb200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20102ef0 0x7fbb200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20110f80 con 0x7fbb20100e20 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20110e00 con 0x7fbb20102ef0 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2818e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111100 con 0x7fbb20102570 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38694/0 (socket says 172.21.15.77:38694) 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 -- 172.21.15.77:0/1662495775 learned_addr learned my addr 172.21.15.77:0/1662495775 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb2672b640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20102ef0 0x7fbb200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25729640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 0x7fbb200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 -- 172.21.15.77:0/1662495775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 msgr2=0x7fbb200fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 0x7fbb200fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 -- 172.21.15.77:0/1662495775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20102ef0 msgr2=0x7fbb200fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20102ef0 0x7fbb200fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 -- 172.21.15.77:0/1662495775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a6910 con 0x7fbb20100e20 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9f0 tx=0x7fbb1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.242+0000 7fbb16ffd640 1 -- 172.21.15.77:0/1662495775 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00cd80 con 0x7fbb20100e20 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.243+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a6c00 con 0x7fbb20100e20 2024-09-15T22:21:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.243+0000 7fbb16ffd640 1 -- 172.21.15.77:0/1662495775 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1c010910 con 0x7fbb20100e20 2024-09-15T22:21:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.243+0000 7fbb16ffd640 1 -- 172.21.15.77:0/1662495775 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0188f0 con 0x7fbb20100e20 2024-09-15T22:21:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.243+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a7190 con 0x7fbb20100e20 2024-09-15T22:21:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.245+0000 7fbb16ffd640 1 -- 172.21.15.77:0/1662495775 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbb1c010ac0 con 0x7fbb20100e20 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.246+0000 7fbb16ffd640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 0x7fbb0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.246+0000 7fbb25729640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 0x7fbb0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.246+0000 7fbb16ffd640 1 -- 172.21.15.77:0/1662495775 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbb1c0d0470 con 0x7fbb20100e20 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 Processor -- start 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb25729640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 0x7fbb0c0a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb200fe710 tx=0x7fbb10005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 -- start start 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7021c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6f86160 con 0x55b6f702ca30 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6f2d720 con 0x55b6f7022170 2024-09-15T22:21:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2a567740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6f82120 con 0x55b6f7021850 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb25729640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb25729640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49642/0 (socket says 172.21.15.77:49642) 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb25729640 1 -- 172.21.15.77:0/2634290327 learned_addr learned my addr 172.21.15.77:0/2634290327 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7021c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.247+0000 7fbb2672b640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 -- 172.21.15.77:0/2634290327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 msgr2=0x55b6f7021c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7021c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 -- 172.21.15.77:0/2634290327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 msgr2=0x55b6f702ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 -- 172.21.15.77:0/2634290327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f6cf7ba0 con 0x55b6f7022170 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb2672b640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb25729640 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005f60 tx=0x7fbb10014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d695696e7c2ac78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb0bfff640 1 -- 172.21.15.77:0/2634290327 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10003d70 con 0x55b6f7022170 2024-09-15T22:21:49.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb0bfff640 1 -- 172.21.15.77:0/2634290327 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb100044d0 con 0x55b6f7022170 2024-09-15T22:21:49.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.248+0000 7fbb0bfff640 1 -- 172.21.15.77:0/2634290327 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1001e420 con 0x55b6f7022170 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- 172.21.15.77:0/2634290327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 msgr2=0x55b6f702c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005f60 tx=0x7fbb10014000 comp rx=0 tx=0).stop 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- 172.21.15.77:0/2634290327 shutdown_connections 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f702c4f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 --2- 172.21.15.77:0/2634290327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7021c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- 172.21.15.77:0/2634290327 >> 172.21.15.77:0/2634290327 conn(0x55b6f701dde0 msgr2=0x55b6f701e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- 172.21.15.77:0/2634290327 shutdown_connections 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- 172.21.15.77:0/2634290327 wait complete. 2024-09-15T22:21:49.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 -- start start 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.249+0000 7fbb2a567740 1 Processor -- start 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7022170 0x55b6f7029db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b6f702f600 con 0x55b6f702ca30 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6f0e900 con 0x55b6f7021850 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b6f702f780 con 0x55b6f7022170 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25729640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7022170 0x55b6f7029db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49648/0 (socket says 172.21.15.77:49648) 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4196214413 learned_addr learned my addr 172.21.15.77:0/4196214413 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2672b640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb2a567740 10 client.0 _ll_register_callbacks cb 0x55b6f6f0c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4196214413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7022170 msgr2=0x55b6f7029db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7022170 0x55b6f7029db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4196214413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 msgr2=0x55b6f702a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.250+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4196214413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f6f4ec50 con 0x55b6f7021850 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c098e40 tx=0x7fbb1c09bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585b8c82635ec674 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c09b8c0 con 0x55b6f7021850 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbb1c09ba60 con 0x55b6f7021850 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb100044d0 con 0x55b6f7021850 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c0b5240 con 0x55b6f7021850 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb2a567740 10 client.26969 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:49.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b6f703b240 con 0x55b6f7021850 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb100044d0 con 0x55b6f7021850 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbb1c018ab0 con 0x55b6f7021850 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb2a567740 10 client.26969 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.251+0000 7fbb2a567740 10 client.26969 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb2a567740 20 client.26969 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f703c650 0x55b6f703eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb2a567740 4 client.26969 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6f6f93890 con 0x55b6f703c650 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb25729640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f703c650 0x55b6f703eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb097fa640 10 client.26969 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.252+0000 7fbb25729640 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f703c650 0x55b6f703eaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e06477e446881a5 server_cookie=3b6e2ced8ed74569 in_seq=0 out_seq=0 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb097fa640 1 -- 172.21.15.77:0/4196214413 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6f6f93890 con 0x55b6f703c650 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb097fa640 10 client.26969 handle_command_reply: tid=0 2024-09-15T22:21:49.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 client.26969 shutdown 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 msgr2=0x55b6f7029870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c098e40 tx=0x7fbb1c09bee0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 shutdown_connections 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f703c650 0x55b6f703eaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f702a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb097fa640 0 client.26969 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7022170 0x55b6f7029db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4196214413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7021850 0x55b6f7029870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 >> 172.21.15.77:0/4196214413 conn(0x55b6f701dde0 msgr2=0x55b6f7023ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 shutdown_connections 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 1 -- 172.21.15.77:0/4196214413 wait complete. 2024-09-15T22:21:49.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.253+0000 7fbb2a567740 20 client.26969 trim_cache size 0 max 16384 2024-09-15T22:21:49.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 Processor -- start 2024-09-15T22:21:49.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 -- start start 2024-09-15T22:21:49.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f70448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f7026ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.273+0000 7fbb2a567740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b1210 con 0x55b6f702ca30 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb2a567740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b2d50 con 0x55b6f7022170 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb2a567740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b18e0 con 0x55b6f7021850 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb2672b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f7026ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36382/0 (socket says 172.21.15.77:36382) 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 -- 172.21.15.77:0/574433109 learned_addr learned my addr 172.21.15.77:0/574433109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 -- 172.21.15.77:0/574433109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 msgr2=0x55b6f70448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f70448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 -- 172.21.15.77:0/574433109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 msgr2=0x55b6f7026ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f7026ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 -- 172.21.15.77:0/574433109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f701dc10 con 0x55b6f7021850 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb2672b640 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f7026ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.274+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9c0 tx=0x7fbb1c09c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4483108cdf931d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.275+0000 7fbb0b7fe640 1 -- 172.21.15.77:0/574433109 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c098ae0 con 0x55b6f7021850 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.275+0000 7fbb0b7fe640 1 -- 172.21.15.77:0/574433109 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1c09b8c0 con 0x55b6f7021850 2024-09-15T22:21:49.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.275+0000 7fbb0b7fe640 1 -- 172.21.15.77:0/574433109 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c098420 con 0x55b6f7021850 2024-09-15T22:21:49.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.275+0000 7fbb2a567740 1 -- 172.21.15.77:0/574433109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 msgr2=0x55b6f7044360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.275+0000 7fbb2a567740 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9c0 tx=0x7fbb1c09c370 comp rx=0 tx=0).stop 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 -- 172.21.15.77:0/574433109 shutdown_connections 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f702ca30 0x55b6f7026ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f7022170 0x55b6f70448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 --2- 172.21.15.77:0/574433109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7044360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 -- 172.21.15.77:0/574433109 >> 172.21.15.77:0/574433109 conn(0x55b6f701dde0 msgr2=0x55b6f702b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 -- 172.21.15.77:0/574433109 shutdown_connections 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 -- 172.21.15.77:0/574433109 wait complete. 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 -- start start 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.276+0000 7fbb2a567740 1 Processor -- start 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7035280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 0x55b6f70315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 0x55b6f7031bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b2680 con 0x55b6f7022170 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201b1fb0 con 0x55b6f702ca30 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201af210 con 0x55b6f7021850 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2a567740 10 client.0 _ll_register_callbacks cb 0x7fbb1c09bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25729640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 0x55b6f70315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2672b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 0x55b6f7031bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7035280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25729640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 0x55b6f70315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38732/0 (socket says 172.21.15.77:38732) 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb2672b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 0x55b6f7031bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49664/0 (socket says 172.21.15.77:49664) 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25729640 1 -- 172.21.15.77:0/4019676450 learned_addr learned my addr 172.21.15.77:0/4019676450 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 msgr2=0x55b6f7031bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 0x55b6f7031bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4019676450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 msgr2=0x55b6f70315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 0x55b6f70315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 -- 172.21.15.77:0/4019676450 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f70246c0 con 0x55b6f7021850 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb25f2a640 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7035280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c098e40 tx=0x7fbb1c018d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afde1a6dfdbcaf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.277+0000 7fbb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0b9070 con 0x55b6f7021850 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c09b8c0 con 0x55b6f7021850 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbb1c010630 con 0x55b6f7021850 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c09c470 con 0x55b6f7021850 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb2a567740 10 client.26902 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:49.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b6f70193f0 con 0x55b6f7021850 2024-09-15T22:21:49.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.278+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1c09b8c0 con 0x55b6f7021850 2024-09-15T22:21:49.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbb1c0a1900 con 0x55b6f7021850 2024-09-15T22:21:49.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 10 client.26902 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:49.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 10 client.26902 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 20 client.26902 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f70b0e10 0x55b6f70b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 4 client.26902 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x55b6f6f8dbe0 con 0x55b6f70b0e10 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb25729640 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f70b0e10 0x55b6f70b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.279+0000 7fbb097fa640 10 client.26902 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:49.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.280+0000 7fbb25729640 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f70b0e10 0x55b6f70b3200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b005a593c56aa31 server_cookie=e1f9bd9b9e3b1b61 in_seq=0 out_seq=0 2024-09-15T22:21:49.287 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:49.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb097fa640 1 -- 172.21.15.77:0/4019676450 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b6f6f8dbe0 con 0x55b6f70b0e10 2024-09-15T22:21:49.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb097fa640 10 client.26902 handle_command_reply: tid=0 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 client.26902 shutdown 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 msgr2=0x55b6f7035280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7035280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c098e40 tx=0x7fbb1c018d40 comp rx=0 tx=0).stop 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 shutdown_connections 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b6f70b0e10 0x55b6f70b3200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6f702ca30 0x55b6f7031bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b6f7022170 0x55b6f70315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb097fa640 0 client.26902 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 --2- 172.21.15.77:0/4019676450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b6f7021850 0x55b6f7035280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 >> 172.21.15.77:0/4019676450 conn(0x55b6f701dde0 msgr2=0x55b6f7029830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.362+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 shutdown_connections 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2a567740 1 -- 172.21.15.77:0/4019676450 wait complete. 2024-09-15T22:21:49.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2a567740 20 client.26902 trim_cache size 0 max 16384 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 34, 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "97d2bb19": { 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 34, 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:21:49.366 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.3, 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:49.367 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 msgr2=0x7fbb0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 0x7fbb0c0a5870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb200fe710 tx=0x7fbb10005f90 comp rx=0 tx=0).stop 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 msgr2=0x7fbb200fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:49.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9f0 tx=0x7fbb1c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 shutdown_connections 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb0c0a33b0 0x7fbb0c0a5870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb20102ef0 0x7fbb200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb20102570 0x7fbb200fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 --2- 172.21.15.77:0/1662495775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb20100e20 0x7fbb200fcc80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 >> 172.21.15.77:0/1662495775 conn(0x7fbb200fc7d0 msgr2=0x7fbb2006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 shutdown_connections 2024-09-15T22:21:49.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:49.363+0000 7fbb2818e640 1 -- 172.21.15.77:0/1662495775 wait complete. 2024-09-15T22:21:49.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.71 MB/sec execute 358 sec latency 104.297 ms 2024-09-15T22:21:49.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:49 smithi077 ceph-mon[30980]: pgmap v291: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-09-15T22:21:49.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:49 smithi117 ceph-mon[35021]: pgmap v291: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-09-15T22:21:50.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:49 smithi196 ceph-mon[35109]: pgmap v291: 129 pgs: 129 active+clean; 473 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-09-15T22:21:50.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 -- 172.21.15.77:0/643479490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20100e20 msgr2=0x7f6c20101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 --2- 172.21.15.77:0/643479490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20100e20 0x7f6c20101200 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009900 tx=0x7f6c1802a480 comp rx=0 tx=0).stop 2024-09-15T22:21:50.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 -- 172.21.15.77:0/643479490 shutdown_connections 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 --2- 172.21.15.77:0/643479490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102ef0 0x7f6c2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 --2- 172.21.15.77:0/643479490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20102570 0x7f6c201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 --2- 172.21.15.77:0/643479490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20100e20 0x7f6c20101200 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 -- 172.21.15.77:0/643479490 >> 172.21.15.77:0/643479490 conn(0x7f6c200fc7d0 msgr2=0x7f6c200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 -- 172.21.15.77:0/643479490 shutdown_connections 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.092+0000 7f6c27fad640 1 -- 172.21.15.77:0/643479490 wait complete. 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 Processor -- start 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 -- start start 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20100e20 0x7f6c201993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102570 0x7f6c20199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20110fa0 con 0x7f6c20102ef0 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20110e20 con 0x7f6c20100e20 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c27fad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20111120 con 0x7f6c20102570 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38742/0 (socket says 172.21.15.77:38742) 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 -- 172.21.15.77:0/2315220183 learned_addr learned my addr 172.21.15.77:0/2315220183 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c25d49640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20100e20 0x7f6c201993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c25548640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102570 0x7f6c20199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 -- 172.21.15.77:0/2315220183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102570 msgr2=0x7f6c20199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102570 0x7f6c20199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 -- 172.21.15.77:0/2315220183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20100e20 msgr2=0x7f6c201993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20100e20 0x7f6c201993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.093+0000 7f6c2654a640 1 -- 172.21.15.77:0/2315220183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c201a49b0 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c2654a640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c970 tx=0x7f6c1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c1effd640 1 -- 172.21.15.77:0/2315220183 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10007bb0 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c1effd640 1 -- 172.21.15.77:0/2315220183 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10007d50 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c201a4b80 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c1effd640 1 -- 172.21.15.77:0/2315220183 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10018800 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.094+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c201a5090 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.096+0000 7f6c1effd640 1 -- 172.21.15.77:0/2315220183 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6c1000fa50 con 0x7f6c20102ef0 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.096+0000 7f6c2a386740 1 Processor -- start 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 -- start start 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e11db40 0x55b82e11c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e11ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b82e0822f0 con 0x55b82e127370 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b82e029cf0 con 0x55b82e11c5d0 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2a386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b82e07e2b0 con 0x55b82e11db40 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c1effd640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 0x7f6c080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38756/0 (socket says 172.21.15.77:38756) 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25548640 1 -- 172.21.15.77:0/3885927971 learned_addr learned my addr 172.21.15.77:0/3885927971 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c2654a640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 0x7f6c080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25548640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e11db40 0x55b82e11c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25d49640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e11ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c1effd640 1 -- 172.21.15.77:0/2315220183 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f6c100d0070 con 0x7f6c20102ef0 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.097+0000 7f6c25548640 1 -- 172.21.15.77:0/3885927971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e11db40 msgr2=0x55b82e11c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25548640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e11db40 0x55b82e11c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25548640 1 -- 172.21.15.77:0/3885927971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 msgr2=0x55b82e11ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25548640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e11ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25548640 1 -- 172.21.15.77:0/3885927971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82e00e3a0 con 0x55b82e127370 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25d49640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e11ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c25548640 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400cd50 tx=0x7f6c140099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c847fa2cb9e0f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c2654a640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 0x7f6c080a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cd4e0 tx=0x7f6c10066900 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c07fff640 1 -- 172.21.15.77:0/3885927971 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400c2d0 con 0x55b82e127370 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c07fff640 1 -- 172.21.15.77:0/3885927971 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c14016040 con 0x55b82e127370 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.098+0000 7f6c07fff640 1 -- 172.21.15.77:0/3885927971 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400abb0 con 0x55b82e127370 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- 172.21.15.77:0/3885927971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 msgr2=0x55b82e129760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400cd50 tx=0x7f6c140099f0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- 172.21.15.77:0/3885927971 shutdown_connections 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e127370 0x55b82e129760 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e11ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 --2- 172.21.15.77:0/3885927971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e11db40 0x55b82e11c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- 172.21.15.77:0/3885927971 >> 172.21.15.77:0/3885927971 conn(0x55b82e0e5310 msgr2=0x55b82e118c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- 172.21.15.77:0/3885927971 shutdown_connections 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- 172.21.15.77:0/3885927971 wait complete. 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 -- start start 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 Processor -- start 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.099+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e124000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e1245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b82e12a010 con 0x55b82e11c5d0 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b82e03d060 con 0x55b82e11db40 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b82e12a190 con 0x55b82e127370 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 10 client.0 _ll_register_callbacks cb 0x55b82dd908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38768/0 (socket says 172.21.15.77:38768) 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 -- 172.21.15.77:0/2668223082 learned_addr learned my addr 172.21.15.77:0/2668223082 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:50.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25d49640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e124000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e1245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 -- 172.21.15.77:0/2668223082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 msgr2=0x55b82e1245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e1245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 -- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 msgr2=0x55b82e124000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e124000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 -- 172.21.15.77:0/2668223082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82ddf33c0 con 0x55b82e11c5d0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c25548640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c4b0 tx=0x7f6c14009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bcb8391cd62630 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400a950 con 0x55b82e11c5d0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c14016040 con 0x55b82e11c5d0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f6c1401c410 con 0x55b82e11c5d0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c1400aaf0 con 0x55b82e11c5d0 2024-09-15T22:21:50.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 10 client.17892 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.100+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b82e135a90 con 0x55b82e11c5d0 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c14016040 con 0x55b82e11c5d0 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6c1401b6c0 con 0x55b82e11c5d0 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 10 client.17892 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 10 client.17892 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 20 client.17892 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e136de0 0x55b82e1391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 4 client.17892 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82e08f860 con 0x55b82e136de0 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c25d49640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e136de0 0x55b82e1391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c25d49640 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e136de0 0x55b82e1391d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=290d9866d2eb2e89 server_cookie=c2bfb96ee7527db1 in_seq=0 out_seq=0 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.101+0000 7f6c057fa640 10 client.17892 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c057fa640 1 -- 172.21.15.77:0/2668223082 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b82e08f860 con 0x55b82e136de0 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c057fa640 10 client.17892 handle_command_reply: tid=0 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 client.17892 shutdown 2024-09-15T22:21:50.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 msgr2=0x55b82e126d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c4b0 tx=0x7f6c14009c70 comp rx=0 tx=0).stop 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 shutdown_connections 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e136de0 0x55b82e1391d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e1245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e124000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c057fa640 0 client.17892 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2668223082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e126d50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 >> 172.21.15.77:0/2668223082 conn(0x55b82e0e5310 msgr2=0x55b82e11e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 shutdown_connections 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 1 -- 172.21.15.77:0/2668223082 wait complete. 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.102+0000 7f6c2a386740 20 client.17892 trim_cache size 0 max 16384 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 Processor -- start 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 -- start start 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e1238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e128760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201aefd0 con 0x55b82e11c5d0 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201b0b10 con 0x55b82e11db40 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c2a386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201af6a0 con 0x55b82e127370 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38770/0 (socket says 172.21.15.77:38770) 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 -- 172.21.15.77:0/2686880051 learned_addr learned my addr 172.21.15.77:0/2686880051 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e128760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 -- 172.21.15.77:0/2686880051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 msgr2=0x55b82e128760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e128760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25d49640 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e1238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 -- 172.21.15.77:0/2686880051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 msgr2=0x55b82e1238c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e1238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 -- 172.21.15.77:0/2686880051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82e124940 con 0x55b82e11c5d0 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.116+0000 7f6c25548640 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400cd50 tx=0x7f6c14009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428e03a0e8e978c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c077fe640 1 -- 172.21.15.77:0/2686880051 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14020040 con 0x55b82e11c5d0 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c077fe640 1 -- 172.21.15.77:0/2686880051 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c14009950 con 0x55b82e11c5d0 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c077fe640 1 -- 172.21.15.77:0/2686880051 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1401b620 con 0x55b82e11c5d0 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 -- 172.21.15.77:0/2686880051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 msgr2=0x55b82e1332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400cd50 tx=0x7f6c14009830 comp rx=0 tx=0).stop 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 -- 172.21.15.77:0/2686880051 shutdown_connections 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e128760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11db40 0x55b82e1238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 --2- 172.21.15.77:0/2686880051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11c5d0 0x55b82e1332b0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 -- 172.21.15.77:0/2686880051 >> 172.21.15.77:0/2686880051 conn(0x55b82e0e5310 msgr2=0x55b82e11f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 -- 172.21.15.77:0/2686880051 shutdown_connections 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.117+0000 7f6c2a386740 1 -- 172.21.15.77:0/2686880051 wait complete. 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 -- start start 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 Processor -- start 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e13d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e13e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201b0440 con 0x55b82e11db40 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201afd70 con 0x55b82e11c5d0 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201acfd0 con 0x55b82e127370 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c2a386740 10 client.0 _ll_register_callbacks cb 0x7f6c1400c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c25d49640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c25d49640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38776/0 (socket says 172.21.15.77:38776) 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c25d49640 1 -- 172.21.15.77:0/1219767370 learned_addr learned my addr 172.21.15.77:0/1219767370 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:50.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.118+0000 7f6c25548640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e13e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 -- 172.21.15.77:0/1219767370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 msgr2=0x55b82e13e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e13e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 -- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 msgr2=0x55b82e13d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e13d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 -- 172.21.15.77:0/1219767370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82e12e450 con 0x55b82e11db40 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c25d49640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009900 tx=0x7f6c18004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d15ffe5ea657a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:50.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18035040 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f6c18005220 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c14009950 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c18052050 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 10 client.17898 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b82e12f7a0 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c14009950 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6c18004960 con 0x55b82e11db40 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 10 client.17898 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 10 client.17898 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:21:50.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 20 client.17898 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e1a1860 0x55b82e1a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 4 client.17898 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.119+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b82e089d70 con 0x55b82e1a1860 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c25548640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e1a1860 0x55b82e1a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c25548640 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e1a1860 0x55b82e1a3c50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5045c65ee8af4d66 server_cookie=99d58d113eed24e6 in_seq=0 out_seq=0 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c057fa640 10 client.17898 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c057fa640 1 -- 172.21.15.77:0/1219767370 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b82e089d70 con 0x55b82e1a1860 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c057fa640 10 client.17898 handle_command_reply: tid=0 2024-09-15T22:21:50.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 client.17898 shutdown 2024-09-15T22:21:50.149 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 msgr2=0x55b82e13ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009900 tx=0x7f6c18004460 comp rx=0 tx=0).stop 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 shutdown_connections 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b82e1a1860 0x55b82e1a3c50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b82e127370 0x55b82e13e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b82e11db40 0x55b82e13ddc0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 --2- 172.21.15.77:0/1219767370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b82e11c5d0 0x55b82e13d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c057fa640 0 client.17898 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.120+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 >> 172.21.15.77:0/1219767370 conn(0x55b82e0e5310 msgr2=0x55b82e1227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 shutdown_connections 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c2a386740 1 -- 172.21.15.77:0/1219767370 wait complete. 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c2a386740 20 client.17898 trim_cache size 0 max 16384 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 msgr2=0x7f6c080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 0x7f6c080a58f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cd4e0 tx=0x7f6c10066900 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 msgr2=0x7f6c201a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c970 tx=0x7f6c1000ce30 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 shutdown_connections 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6c080a3430 0x7f6c080a58f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c20102ef0 0x7f6c201a4470 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c20102570 0x7f6c20199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 --2- 172.21.15.77:0/2315220183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c20100e20 0x7f6c201993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 >> 172.21.15.77:0/2315220183 conn(0x7f6c200fc7d0 msgr2=0x7f6c20105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 shutdown_connections 2024-09-15T22:21:50.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:50.121+0000 7f6c27fad640 1 -- 172.21.15.77:0/2315220183 wait complete. 2024-09-15T22:21:50.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.69 MB/sec execute 359 sec latency 1104.366 ms 2024-09-15T22:21:50.910 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:21:51.385 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '97d2bb19', quiesce took 6.9 seconds. Will release after: 15.450000000000001 2024-09-15T22:21:51.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.67 MB/sec execute 360 sec latency 2104.480 ms 2024-09-15T22:21:51.527 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:51.933 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:51 smithi077 ceph-mon[30980]: pgmap v292: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.014+0000 7f68f033b640 1 -- 172.21.15.77:0/1502533596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 msgr2=0x7f68e8100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.014+0000 7f68f033b640 1 --2- 172.21.15.77:0/1502533596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e8100db0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ca0 tx=0x7f68d802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 -- 172.21.15.77:0/1502533596 shutdown_connections 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 --2- 172.21.15.77:0/1502533596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 0x7f68e8110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 --2- 172.21.15.77:0/1502533596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 0x7f68e810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 --2- 172.21.15.77:0/1502533596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e8100db0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 -- 172.21.15.77:0/1502533596 >> 172.21.15.77:0/1502533596 conn(0x7f68e80fc7d0 msgr2=0x7f68e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 -- 172.21.15.77:0/1502533596 shutdown_connections 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 -- 172.21.15.77:0/1502533596 wait complete. 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 Processor -- start 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.015+0000 7f68f033b640 1 -- start start 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 0x7f68e81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 0x7f68e819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81142c0 con 0x7f68e81009d0 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8114140 con 0x7f68e810e4f0 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8114440 con 0x7f68e8101380 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38798/0 (socket says 172.21.15.77:38798) 2024-09-15T22:21:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 -- 172.21.15.77:0/1290962015 learned_addr learned my addr 172.21.15.77:0/1290962015 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 0x7f68e81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 0x7f68e819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 -- 172.21.15.77:0/1290962015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 msgr2=0x7f68e81a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 0x7f68e81a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 -- 172.21.15.77:0/1290962015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 msgr2=0x7f68e819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 0x7f68e819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 -- 172.21.15.77:0/1290962015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e819af00 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f68d802ade0 tx=0x7f68d80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68e77fe640 1 -- 172.21.15.77:0/1290962015 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d802fa20 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68e77fe640 1 -- 172.21.15.77:0/1290962015 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d802fbc0 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e819b190 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68e77fe640 1 -- 172.21.15.77:0/1290962015 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d803a660 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.016+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e81a7390 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68e77fe640 1 -- 172.21.15.77:0/1290962015 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f68d80305c0 con 0x7f68e81009d0 2024-09-15T22:21:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 Processor -- start 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 -- start start 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f75ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f768760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b10f6c12f0 con 0x55b10f75cb40 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68e77fe640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 0x7f68d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b10f668cf0 con 0x55b10f766370 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.019+0000 7f68f2714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b10f6bd2b0 con 0x55b10f75b5d0 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38806/0 (socket says 172.21.15.77:38806) 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 -- 172.21.15.77:0/3934508326 learned_addr learned my addr 172.21.15.77:0/3934508326 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f75ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 0x7f68d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f768760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68e77fe640 1 -- 172.21.15.77:0/1290962015 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f68d80f1550 con 0x7f68e81009d0 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 -- 172.21.15.77:0/3934508326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 msgr2=0x55b10f75ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f75ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 -- 172.21.15.77:0/3934508326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 msgr2=0x55b10f768760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f768760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 -- 172.21.15.77:0/3934508326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10f64d3a0 con 0x55b10f75cb40 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f68e0014c20 tx=0x7f68e00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace3030c2430c3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 0x7f68d00a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68e0011cb0 tx=0x7f68e0016000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68e4dea640 1 -- 172.21.15.77:0/3934508326 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0014250 con 0x55b10f75cb40 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68e4dea640 1 -- 172.21.15.77:0/3934508326 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e0014620 con 0x55b10f75cb40 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.020+0000 7f68e4dea640 1 -- 172.21.15.77:0/3934508326 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e00047a0 con 0x55b10f75cb40 2024-09-15T22:21:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 -- 172.21.15.77:0/3934508326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 msgr2=0x55b10f75b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f68e0014c20 tx=0x7f68e00117b0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 -- 172.21.15.77:0/3934508326 shutdown_connections 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f768760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f75ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 --2- 172.21.15.77:0/3934508326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75cb40 0x55b10f75b090 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 -- 172.21.15.77:0/3934508326 >> 172.21.15.77:0/3934508326 conn(0x55b10f724310 msgr2=0x55b10f757c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 -- 172.21.15.77:0/3934508326 shutdown_connections 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.021+0000 7f68f2714740 1 -- 172.21.15.77:0/3934508326 wait complete. 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 -- start start 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 Processor -- start 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f763030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f766370 0x55b10f763640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b10f769010 con 0x55b10f75b5d0 2024-09-15T22:21:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b10f67c060 con 0x55b10f75cb40 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b10f769190 con 0x55b10f766370 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68f2714740 10 client.0 _ll_register_callbacks cb 0x55b10f3cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38818/0 (socket says 172.21.15.77:38818) 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 -- 172.21.15.77:0/551013542 learned_addr learned my addr 172.21.15.77:0/551013542 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f766370 0x55b10f763640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f763030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 -- 172.21.15.77:0/551013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f766370 msgr2=0x55b10f763640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f766370 0x55b10f763640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 -- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 msgr2=0x55b10f763030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f763030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 -- 172.21.15.77:0/551013542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10f4323c0 con 0x55b10f75b5d0 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006010 tx=0x7f68d80b99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6719ed5391f206a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.022+0000 7f68cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d80bd180 con 0x55b10f75b5d0 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f68d80f11a0 con 0x55b10f75b5d0 2024-09-15T22:21:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e0014620 con 0x55b10f75b5d0 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f68d80bcfa0 con 0x55b10f75b5d0 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 10 client.17910 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b10f769010 con 0x55b10f75b5d0 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e0014620 con 0x55b10f75b5d0 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f68d8087ff0 con 0x55b10f75b5d0 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 10 client.17910 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 20 client.17910 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f775fe0 0x55b10f7783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 4 client.17910 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10f6ce860 con 0x55b10f775fe0 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.023+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f775fe0 0x55b10f7783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.024+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f775fe0 0x55b10f7783f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6327fb3c9f2042f1 server_cookie=30f07be5b4eaf656 in_seq=0 out_seq=0 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.024+0000 7f68cdffb640 10 client.17910 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.024+0000 7f68cdffb640 1 -- 172.21.15.77:0/551013542 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b10f6ce860 con 0x55b10f775fe0 2024-09-15T22:21:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.024+0000 7f68cdffb640 10 client.17910 handle_command_reply: tid=0 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 client.17910 shutdown 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 msgr2=0x55b10f765d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006010 tx=0x7f68d80b99a0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 shutdown_connections 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f775fe0 0x55b10f7783f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f766370 0x55b10f763640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f763030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 --2- 172.21.15.77:0/551013542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f765d80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68cdffb640 0 client.17910 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 >> 172.21.15.77:0/551013542 conn(0x55b10f724310 msgr2=0x55b10f75d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 shutdown_connections 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 1 -- 172.21.15.77:0/551013542 wait complete. 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.025+0000 7f68f2714740 20 client.17910 trim_cache size 0 max 16384 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 Processor -- start 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 -- start start 2024-09-15T22:21:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 0x55b10f75e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f757510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b1070 con 0x55b10f75b5d0 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b2bb0 con 0x55b10f766370 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68f2714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b1740 con 0x55b10f75cb40 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38822/0 (socket says 172.21.15.77:38822) 2024-09-15T22:21:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 -- 172.21.15.77:0/2488310492 learned_addr learned my addr 172.21.15.77:0/2488310492 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 0x55b10f75e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 -- 172.21.15.77:0/2488310492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 msgr2=0x55b10f75e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 0x55b10f75e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 -- 172.21.15.77:0/2488310492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 msgr2=0x55b10f757510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f757510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 -- 172.21.15.77:0/2488310492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10f764f70 con 0x55b10f75b5d0 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 0x55b10f75e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f68d8000f80 tx=0x7f68d80b9a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef3d885da4be962 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68cffff640 1 -- 172.21.15.77:0/2488310492 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d80bd180 con 0x55b10f75b5d0 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68cffff640 1 -- 172.21.15.77:0/2488310492 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d8036040 con 0x55b10f75b5d0 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.038+0000 7f68cffff640 1 -- 172.21.15.77:0/2488310492 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d80b9390 con 0x55b10f75b5d0 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- 172.21.15.77:0/2488310492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 msgr2=0x55b10f75e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f68d8000f80 tx=0x7f68d80b9a10 comp rx=0 tx=0).stop 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- 172.21.15.77:0/2488310492 shutdown_connections 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f766370 0x55b10f757510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75cb40 0x55b10f75e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 --2- 172.21.15.77:0/2488310492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f75b5d0 0x55b10f75e120 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- 172.21.15.77:0/2488310492 >> 172.21.15.77:0/2488310492 conn(0x55b10f724310 msgr2=0x55b10f7712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- 172.21.15.77:0/2488310492 shutdown_connections 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- 172.21.15.77:0/2488310492 wait complete. 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 -- start start 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.039+0000 7f68f2714740 1 Processor -- start 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f76eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f75fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b24e0 con 0x55b10f766370 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b1e10 con 0x55b10f75cb40 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81af070 con 0x55b10f75b5d0 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68f2714740 10 client.0 _ll_register_callbacks cb 0x7f68d8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38836/0 (socket says 172.21.15.77:38836) 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 -- 172.21.15.77:0/1540353748 learned_addr learned my addr 172.21.15.77:0/1540353748 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ed8d6640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f75fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f76eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 -- 172.21.15.77:0/1540353748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 msgr2=0x55b10f76eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f76eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 -- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 msgr2=0x55b10f75fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f75fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.040+0000 7f68ee8d8640 1 -- 172.21.15.77:0/1540353748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10f760d00 con 0x55b10f766370 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68ee8d8640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f68e00048a0 tx=0x7f68e00048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a30ef4f9c9b79a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0033020 con 0x55b10f766370 2024-09-15T22:21:52.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f68e0014250 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d8036040 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f68e003b050 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68f2714740 10 client.17916 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.041+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b10f7e0d40 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d8036040 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f68e001d070 con 0x55b10f766370 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 10 client.17916 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:52.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 10 client.17916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 20 client.17916 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f7e1240 0x55b10f7e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 4 client.17916 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b10f6c8d70 con 0x55b10f7e1240 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.042+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f7e1240 0x55b10f7e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68ee0d7640 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f7e1240 0x55b10f7e3660 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67165a023caf82b server_cookie=75d7fb3395fbb882 in_seq=0 out_seq=0 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68cdffb640 10 client.17916 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:52.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68cdffb640 1 -- 172.21.15.77:0/1540353748 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b10f6c8d70 con 0x55b10f7e1240 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68cdffb640 10 client.17916 handle_command_reply: tid=0 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 client.17916 shutdown 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 msgr2=0x55b10f7604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f68e00048a0 tx=0x7f68e00048d0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 shutdown_connections 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b10f7e1240 0x55b10f7e3660 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b10f766370 0x55b10f7604d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b10f75cb40 0x55b10f75fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 --2- 172.21.15.77:0/1540353748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b10f75b5d0 0x55b10f76eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 >> 172.21.15.77:0/1540353748 conn(0x55b10f724310 msgr2=0x55b10f771ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68cdffb640 0 client.17916 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:52.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 shutdown_connections 2024-09-15T22:21:52.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 1 -- 172.21.15.77:0/1540353748 wait complete. 2024-09-15T22:21:52.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.043+0000 7f68f2714740 20 client.17916 trim_cache size 0 max 16384 2024-09-15T22:21:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 msgr2=0x7f68d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 0x7f68d00a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f68e0011cb0 tx=0x7f68e0016000 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 msgr2=0x7f68e81a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f68d802ade0 tx=0x7f68d80319f0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 shutdown_connections 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f68d00a33b0 0x7f68d00a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68e810e4f0 0x7f68e819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e8101380 0x7f68e81a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 --2- 172.21.15.77:0/1290962015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68e81009d0 0x7f68e81a0570 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 >> 172.21.15.77:0/1290962015 conn(0x7f68e80fc7d0 msgr2=0x7f68e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 shutdown_connections 2024-09-15T22:21:52.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:52.044+0000 7f68f033b640 1 -- 172.21.15.77:0/1290962015 wait complete. 2024-09-15T22:21:52.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:51 smithi196 ceph-mon[35109]: pgmap v292: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:21:52.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:51 smithi117 ceph-mon[35021]: pgmap v292: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 156 op/s 2024-09-15T22:21:52.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.65 MB/sec execute 361 sec latency 3104.561 ms 2024-09-15T22:21:53.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.63 MB/sec execute 362 sec latency 4104.636 ms 2024-09-15T22:21:54.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:53 smithi077 ceph-mon[30980]: pgmap v293: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:54.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:53 smithi196 ceph-mon[35109]: pgmap v293: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:54.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:53 smithi117 ceph-mon[35021]: pgmap v293: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:54.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:21:54.143 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:21:54.401 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:54.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.61 MB/sec execute 363 sec latency 5104.675 ms 2024-09-15T22:21:54.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 -- 172.21.15.77:0/2929313320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 msgr2=0x7f08d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 --2- 172.21.15.77:0/2929313320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01021c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c002a410 comp rx=0 tx=0).stop 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 -- 172.21.15.77:0/2929313320 shutdown_connections 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 --2- 172.21.15.77:0/2929313320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0106f10 0x7f08d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 --2- 172.21.15.77:0/2929313320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0102790 0x7f08d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 --2- 172.21.15.77:0/2929313320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01021c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 -- 172.21.15.77:0/2929313320 >> 172.21.15.77:0/2929313320 conn(0x7f08d00fd8c0 msgr2=0x7f08d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 -- 172.21.15.77:0/2929313320 shutdown_connections 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.876+0000 7f08d62ae640 1 -- 172.21.15.77:0/2929313320 wait complete. 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 Processor -- start 2024-09-15T22:21:54.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 -- start start 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 0x7f08d0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 0x7f08d019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0111f70 con 0x7f08d0101de0 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0111df0 con 0x7f08d0102790 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d62ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01120f0 con 0x7f08d0106f10 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38854/0 (socket says 172.21.15.77:38854) 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 -- 172.21.15.77:0/886372264 learned_addr learned my addr 172.21.15.77:0/886372264 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08c7fff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 0x7f08d0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08d484b640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 0x7f08d019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 -- 172.21.15.77:0/886372264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 msgr2=0x7f08d019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 0x7f08d019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 -- 172.21.15.77:0/886372264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 msgr2=0x7f08d0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 0x7f08d0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.877+0000 7f08cffff640 1 -- 172.21.15.77:0/886372264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d019d380 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08c7fff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 0x7f08d0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08d484b640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 0x7f08d019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08cffff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005ab0 tx=0x7f08c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08cdffb640 1 -- 172.21.15.77:0/886372264 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c003f030 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08cdffb640 1 -- 172.21.15.77:0/886372264 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c0003ce0 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d019d610 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08cdffb640 1 -- 172.21.15.77:0/886372264 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0035070 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.878+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d01a4f50 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.880+0000 7f08cdffb640 1 -- 172.21.15.77:0/886372264 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f08c0042030 con 0x7f08d0101de0 2024-09-15T22:21:54.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.880+0000 7f08d8687740 1 Processor -- start 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.880+0000 7f08d8687740 1 -- start start 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08cdffb640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 0x7f08bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55619c812280 con 0x55619c8ad480 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55619c7b9b10 con 0x55619c8acb60 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d8687740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55619c80e240 con 0x55619c8b7d40 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08cffff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 0x7f08bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38858/0 (socket says 172.21.15.77:38858) 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 -- 172.21.15.77:0/564188427 learned_addr learned my addr 172.21.15.77:0/564188427 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:54.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d484b640 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08cdffb640 1 -- 172.21.15.77:0/886372264 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f08c00f0ad0 con 0x7f08d0101de0 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08d484b640 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 -- 172.21.15.77:0/564188427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 msgr2=0x55619c8ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 -- 172.21.15.77:0/564188427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 msgr2=0x55619c8acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 -- 172.21.15.77:0/564188427 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55619c7c61f0 con 0x55619c8ad480 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08cffff640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 0x7f08bc0a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08c00bc950 tx=0x7f08c00877a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c7fff640 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f08b800bb40 tx=0x7f08b800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e6f13a50346677 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.881+0000 7f08c67fc640 1 -- 172.21.15.77:0/564188427 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8017420 con 0x55619c8ad480 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08c67fc640 1 -- 172.21.15.77:0/564188427 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8017aa0 con 0x55619c8ad480 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08c67fc640 1 -- 172.21.15.77:0/564188427 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800fc30 con 0x55619c8ad480 2024-09-15T22:21:54.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 -- 172.21.15.77:0/564188427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 msgr2=0x55619c8b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f08b800bb40 tx=0x7f08b800d340 comp rx=0 tx=0).stop 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 -- 172.21.15.77:0/564188427 shutdown_connections 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b7800 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 --2- 172.21.15.77:0/564188427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 -- 172.21.15.77:0/564188427 >> 172.21.15.77:0/564188427 conn(0x55619c8a90f0 msgr2=0x55619c8a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 -- 172.21.15.77:0/564188427 shutdown_connections 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.882+0000 7f08d8687740 1 -- 172.21.15.77:0/564188427 wait complete. 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 -- start start 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 Processor -- start 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55619c8ba910 con 0x55619c8ad480 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55619c7ccf10 con 0x55619c8acb60 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55619c8baa90 con 0x55619c8b7d40 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08d8687740 10 client.0 _ll_register_callbacks cb 0x55619c795a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38862/0 (socket says 172.21.15.77:38862) 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.883+0000 7f08c7fff640 1 -- 172.21.15.77:0/776521682 learned_addr learned my addr 172.21.15.77:0/776521682 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d484b640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d484b640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 -- 172.21.15.77:0/776521682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 msgr2=0x55619c8b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 -- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 msgr2=0x55619c8b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 -- 172.21.15.77:0/776521682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55619c5836a0 con 0x55619c8ad480 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d484b640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08c7fff640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f08b800f850 tx=0x7f08b800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00fa39e4544eae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8026070 con 0x55619c8ad480 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f08b800fdc0 con 0x55619c8ad480 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b8017aa0 con 0x55619c8ad480 2024-09-15T22:21:54.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08b8032050 con 0x55619c8ad480 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d8687740 10 client.17928 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55619c8c6470 con 0x55619c8ad480 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8017aa0 con 0x55619c8ad480 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.884+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f08b8022a80 con 0x55619c8ad480 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 10 client.17928 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 20 client.17928 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c8c7820 0x55619c8c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 4 client.17928 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55619c81f7f0 con 0x55619c8c7820 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d484b640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c8c7820 0x55619c8c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08af7fe640 10 client.17928 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.885+0000 7f08d484b640 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c8c7820 0x55619c8c9c10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fcce2ce295adc6 server_cookie=6d814f16d2db5a38 in_seq=0 out_seq=0 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08af7fe640 1 -- 172.21.15.77:0/776521682 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55619c81f7f0 con 0x55619c8c7820 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08af7fe640 10 client.17928 handle_command_reply: tid=0 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 client.17928 shutdown 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 msgr2=0x55619c8b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f08b800f850 tx=0x7f08b800b880 comp rx=0 tx=0).stop 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 shutdown_connections 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c8c7820 0x55619c8c9c10 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8ad480 0x55619c8b4b50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08af7fe640 0 client.17928 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 --2- 172.21.15.77:0/776521682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8acb60 0x55619c8b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 >> 172.21.15.77:0/776521682 conn(0x55619c8a90f0 msgr2=0x55619c8a8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 shutdown_connections 2024-09-15T22:21:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 1 -- 172.21.15.77:0/776521682 wait complete. 2024-09-15T22:21:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.886+0000 7f08d8687740 20 client.17928 trim_cache size 0 max 16384 2024-09-15T22:21:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.903+0000 7f08d8687740 1 Processor -- start 2024-09-15T22:21:54.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 -- start start 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8ad480 0x55619c8cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01aefa0 con 0x55619c8acb60 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01b0ae0 con 0x55619c8ad480 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d8687740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01af670 con 0x55619c8b7d40 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38864/0 (socket says 172.21.15.77:38864) 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 -- 172.21.15.77:0/2020809015 learned_addr learned my addr 172.21.15.77:0/2020809015 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08c7fff640 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8ad480 0x55619c8cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 -- 172.21.15.77:0/2020809015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 msgr2=0x55619c8b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 -- 172.21.15.77:0/2020809015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8ad480 msgr2=0x55619c8cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8ad480 0x55619c8cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 -- 172.21.15.77:0/2020809015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55619c8ad2b0 con 0x55619c8acb60 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.904+0000 7f08d484b640 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x55619c8ca180 tx=0x7f08c800a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dd94fd434b1cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08c5ffb640 1 -- 172.21.15.77:0/2020809015 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c800f810 con 0x55619c8acb60 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08c5ffb640 1 -- 172.21.15.77:0/2020809015 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c8022070 con 0x55619c8acb60 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08c5ffb640 1 -- 172.21.15.77:0/2020809015 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c801db70 con 0x55619c8acb60 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 -- 172.21.15.77:0/2020809015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 msgr2=0x55619c8cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x55619c8ca180 tx=0x7f08c800a930 comp rx=0 tx=0).stop 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 -- 172.21.15.77:0/2020809015 shutdown_connections 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8b7d40 0x55619c8b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8ad480 0x55619c8cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 --2- 172.21.15.77:0/2020809015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8cf1e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 -- 172.21.15.77:0/2020809015 >> 172.21.15.77:0/2020809015 conn(0x55619c8a90f0 msgr2=0x55619c8b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 -- 172.21.15.77:0/2020809015 shutdown_connections 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.905+0000 7f08d8687740 1 -- 172.21.15.77:0/2020809015 wait complete. 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 -- start start 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 Processor -- start 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 0x55619c8bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8b7d40 0x55619c9311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01b0410 con 0x55619c8acb60 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01afd40 con 0x55619c8b7d40 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01acfa0 con 0x55619c8ad480 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d8687740 10 client.0 _ll_register_callbacks cb 0x7f08c800a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:54.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d484b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d484b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38880/0 (socket says 172.21.15.77:38880) 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08d484b640 1 -- 172.21.15.77:0/3354464360 learned_addr learned my addr 172.21.15.77:0/3354464360 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.906+0000 7f08c7fff640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 0x55619c8bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8b7d40 0x55619c9311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 -- 172.21.15.77:0/3354464360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 msgr2=0x55619c8bc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 0x55619c8bc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 -- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8b7d40 msgr2=0x55619c9311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8b7d40 0x55619c9311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 -- 172.21.15.77:0/3354464360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55619c9317f0 con 0x55619c8acb60 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08c7fff640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 0x55619c8bc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d484b640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f08c800a5e0 tx=0x7f08c8005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbfcfdb16cfd781 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:54.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8004a40 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08c8022070 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f08c8004be0 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08c8039050 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d8687740 10 client.17934 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55619c932af0 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.907+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c8022070 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f08c8029720 con 0x55619c8acb60 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 10 client.17934 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 20 client.17934 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c933130 0x55619c935550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 4 client.17934 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55619c819d00 con 0x55619c933130 2024-09-15T22:21:54.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08c7fff640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c933130 0x55619c935550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08af7fe640 10 client.17934 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.908+0000 7f08c7fff640 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c933130 0x55619c935550 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe5cad5e73af1c9 server_cookie=e1b9cca8d3731be1 in_seq=0 out_seq=0 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.909+0000 7f08af7fe640 1 -- 172.21.15.77:0/3354464360 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55619c819d00 con 0x55619c933130 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.909+0000 7f08af7fe640 10 client.17934 handle_command_reply: tid=0 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.909+0000 7f08d8687740 1 client.17934 shutdown 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.909+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 msgr2=0x55619c8bc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.909+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f08c800a5e0 tx=0x7f08c8005b90 comp rx=0 tx=0).stop 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 shutdown_connections 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55619c933130 0x55619c935550 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55619c8b7d40 0x55619c9311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55619c8ad480 0x55619c8bc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08af7fe640 0 client.17934 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:54.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 --2- 172.21.15.77:0/3354464360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55619c8acb60 0x55619c8bc260 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 >> 172.21.15.77:0/3354464360 conn(0x55619c8a90f0 msgr2=0x55619c8a7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "069f0cec-740a-464d-b503-1166df2917b2", 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:21:54.925 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 shutdown_connections 2024-09-15T22:21:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 1 -- 172.21.15.77:0/3354464360 wait complete. 2024-09-15T22:21:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d8687740 20 client.17934 trim_cache size 0 max 16384 2024-09-15T22:21:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 msgr2=0x7f08bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 0x7f08bc0a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08c00bc950 tx=0x7f08c00877a0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 msgr2=0x7f08d01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f08c0005ab0 tx=0x7f08c00042f0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 shutdown_connections 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f08bc0a33b0 0x7f08bc0a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08d0106f10 0x7f08d019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08d0102790 0x7f08d0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 --2- 172.21.15.77:0/886372264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08d0101de0 0x7f08d01983d0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 >> 172.21.15.77:0/886372264 conn(0x7f08d00fd8c0 msgr2=0x7f08d00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 shutdown_connections 2024-09-15T22:21:54.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:54.910+0000 7f08d62ae640 1 -- 172.21.15.77:0/886372264 wait complete. 2024-09-15T22:21:55.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.59 MB/sec execute 364 sec latency 6104.773 ms 2024-09-15T22:21:55.696 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: pgmap v294: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:55.942 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: pgmap v294: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: pgmap v294: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-15T22:21:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:21:56.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:21:56.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:21:56.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1296739390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 msgr2=0x7fa2bc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1296739390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc10bbb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009900 tx=0x7fa2b002a480 comp rx=0 tx=0).stop 2024-09-15T22:21:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1296739390 shutdown_connections 2024-09-15T22:21:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1296739390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc10c180 0x7fa2bc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1296739390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc10bbb0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1296739390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc102da0 0x7fa2bc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.400+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1296739390 >> 172.21.15.77:0/1296739390 conn(0x7fa2bc0fc7a0 msgr2=0x7fa2bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1296739390 shutdown_connections 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1296739390 wait complete. 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 Processor -- start 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 -- start start 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 0x7fa2bc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.401+0000 7fa2c0bd2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 0x7fa2bc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2c0bd2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112020 con 0x7fa2bc1036c0 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2c0bd2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc111ea0 con 0x7fa2bc10c180 2024-09-15T22:21:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2c0bd2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1121a0 con 0x7fa2bc102da0 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38898/0 (socket says 172.21.15.77:38898) 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/1801025000 learned_addr learned my addr 172.21.15.77:0/1801025000 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 0x7fa2bc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 0x7fa2bc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/1801025000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 msgr2=0x7fa2bc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 0x7fa2bc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/1801025000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 msgr2=0x7fa2bc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 0x7fa2bc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/1801025000 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc19d5b0 con 0x7fa2bc1036c0 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 0x7fa2bc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 0x7fa2bc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005ab0 tx=0x7fa2b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2ab7fe640 1 -- 172.21.15.77:0/1801025000 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b003f030 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2ab7fe640 1 -- 172.21.15.77:0/1801025000 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b0003ce0 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc19d840 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2ab7fe640 1 -- 172.21.15.77:0/1801025000 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0035070 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.402+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a5180 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2ab7fe640 1 -- 172.21.15.77:0/1801025000 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa2b0042030 con 0x7fa2bc1036c0 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 Processor -- start 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 -- start start 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc7302e0 con 0x55f8cc7d5680 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc6d7ce0 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.405+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc72c2a0 con 0x55f8cc7cadc0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2ab7fe640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 0x7fa2a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38912/0 (socket says 172.21.15.77:38912) 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/42133528 learned_addr learned my addr 172.21.15.77:0/42133528 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 0x7fa2a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2ab7fe640 1 -- 172.21.15.77:0/1801025000 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fa2b00f0a30 con 0x7fa2bc1036c0 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/42133528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 msgr2=0x55f8cc7d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/42133528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 msgr2=0x55f8cc7ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 -- 172.21.15.77:0/42133528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cc6bc440 con 0x55f8cc7d5680 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00c50c0 tx=0x7fa2b002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39b44501eda18ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 0x7fa2a40a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009fd0 tx=0x7fa2b4008220 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2a8ff9640 1 -- 172.21.15.77:0/42133528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0049030 con 0x55f8cc7d5680 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2a8ff9640 1 -- 172.21.15.77:0/42133528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b00c55f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.406+0000 7fa2a8ff9640 1 -- 172.21.15.77:0/42133528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b003be20 con 0x55f8cc7d5680 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- 172.21.15.77:0/42133528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 msgr2=0x55f8cc7d7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00c50c0 tx=0x7fa2b002a480 comp rx=0 tx=0).stop 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- 172.21.15.77:0/42133528 shutdown_connections 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d7a70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/42133528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- 172.21.15.77:0/42133528 >> 172.21.15.77:0/42133528 conn(0x55f8cc7c6a30 msgr2=0x55f8cc7c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- 172.21.15.77:0/42133528 shutdown_connections 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- 172.21.15.77:0/42133528 wait complete. 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.407+0000 7fa2c2fab740 1 -- start start 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 Processor -- start 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc7d8010 con 0x55f8cc7d5680 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc6eb050 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f8cc7d8190 con 0x55f8cc7cadc0 2024-09-15T22:21:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2c2fab740 10 client.0 _ll_register_callbacks cb 0x55f8cc72b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38926/0 (socket says 172.21.15.77:38926) 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3051958556 learned_addr learned my addr 172.21.15.77:0/3051958556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3051958556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 msgr2=0x55f8cc7d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 msgr2=0x55f8cc7d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3051958556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cc4a13c0 con 0x55f8cc7d5680 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4015ca0 tx=0x7fa2b4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8869fb8daa8f302 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.408+0000 7fa2a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40253f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b00c55f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fa2b4025590 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa2b40257d0 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 10 client.17946 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8cc7e3990 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b00c55f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa2b401a8d0 con 0x55f8cc7d5680 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 10 client.17946 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 20 client.17946 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc7e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 4 client.17946 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.409+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8cc73d850 con 0x55f8cc7e4e50 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.410+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc7e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.410+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc7e72b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e82280a6c1f60ab server_cookie=e6f4a96d756fc131 in_seq=0 out_seq=0 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.410+0000 7fa2a1ffb640 10 client.17946 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3051958556 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8cc73d850 con 0x55f8cc7e4e50 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2a1ffb640 10 client.17946 handle_command_reply: tid=0 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 client.17946 shutdown 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 msgr2=0x55f8cc7d2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4015ca0 tx=0x7fa2b4004b50 comp rx=0 tx=0).stop 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 shutdown_connections 2024-09-15T22:21:56.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc7e72b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc7d2960 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2a1ffb640 0 client.17946 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3051958556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 >> 172.21.15.77:0/3051958556 conn(0x55f8cc7c6a30 msgr2=0x55f8cc7cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 shutdown_connections 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3051958556 wait complete. 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.411+0000 7fa2c2fab740 20 client.17946 trim_cache size 0 max 16384 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 Processor -- start 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 -- start start 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 0x55f8cc7ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1af090 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1b0bd0 con 0x55f8cc7d5680 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1af760 con 0x55f8cc7cadc0 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2ba59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2ba59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38932/0 (socket says 172.21.15.77:38932) 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.428+0000 7fa2ba59c640 1 -- 172.21.15.77:0/4171665677 learned_addr learned my addr 172.21.15.77:0/4171665677 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 0x55f8cc7ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 -- 172.21.15.77:0/4171665677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 msgr2=0x55f8cc7ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7ecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 -- 172.21.15.77:0/4171665677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 msgr2=0x55f8cc7ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 0x55f8cc7ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 -- 172.21.15.77:0/4171665677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cc7d0c20 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 0x55f8cc7ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac000c00 tx=0x7fa2ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cad5006da0e0c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2a3fff640 1 -- 172.21.15.77:0/4171665677 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac026070 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2a3fff640 1 -- 172.21.15.77:0/4171665677 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac0218f0 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.429+0000 7fa2a3fff640 1 -- 172.21.15.77:0/4171665677 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac019810 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- 172.21.15.77:0/4171665677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 msgr2=0x55f8cc7e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac000c00 tx=0x7fa2ac008040 comp rx=0 tx=0).stop 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- 172.21.15.77:0/4171665677 shutdown_connections 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7d5680 0x55f8cc7ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/4171665677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7e11e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- 172.21.15.77:0/4171665677 >> 172.21.15.77:0/4171665677 conn(0x55f8cc7c6a30 msgr2=0x55f8cc7d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- 172.21.15.77:0/4171665677 shutdown_connections 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- 172.21.15.77:0/4171665677 wait complete. 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- start start 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 Processor -- start 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1b0500 con 0x55f8cc7d5680 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1afe30 con 0x55f8cc7ca4a0 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.430+0000 7fa2c2fab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1ad090 con 0x55f8cc7cadc0 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2c2fab740 10 client.0 _ll_register_callbacks cb 0x7fa2b400e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38936/0 (socket says 172.21.15.77:38936) 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3490662916 learned_addr learned my addr 172.21.15.77:0/3490662916 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:21:56.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2b9d9b640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3490662916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 msgr2=0x55f8cc7d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 msgr2=0x55f8cc7d97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 -- 172.21.15.77:0/3490662916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cc84f0e0 con 0x55f8cc7d5680 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2bad9d640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4015a60 tx=0x7fa2b4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18d5b06a87366b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4025600 con 0x55f8cc7d5680 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fa2b400e9a0 con 0x55f8cc7d5680 2024-09-15T22:21:56.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ac0218f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa2b4025970 con 0x55f8cc7d5680 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2c2fab740 10 client.17952 fetch_fsmap learned FSMap version 34 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.431+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8cc854730 con 0x55f8cc7d5680 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac0218f0 con 0x55f8cc7d5680 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa2b400b660 con 0x55f8cc7d5680 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 10 client.17952 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 20 client.17952 populate_metadata read hostname 'smithi077' 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc856e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 4 client.17952 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8cc737d60 con 0x55f8cc7e4e50 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.432+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc856e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2ba59c640 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc856e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de68f9a8821cd8d5 server_cookie=56f4cf3a68d03634 in_seq=0 out_seq=0 2024-09-15T22:21:56.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2a1ffb640 10 client.17952 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2a1ffb640 1 -- 172.21.15.77:0/3490662916 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f8cc737d60 con 0x55f8cc7e4e50 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2a1ffb640 10 client.17952 handle_command_reply: tid=0 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 client.17952 shutdown 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 msgr2=0x55f8cc84eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4015a60 tx=0x7fa2b4009830 comp rx=0 tx=0).stop 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 shutdown_connections 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f8cc7e4e50 0x55f8cc856e60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8cc7d5680 0x55f8cc84eba0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2a1ffb640 0 client.17952 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f8cc7cadc0 0x55f8cc7d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 --2- 172.21.15.77:0/3490662916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8cc7ca4a0 0x55f8cc7d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 >> 172.21.15.77:0/3490662916 conn(0x55f8cc7c6a30 msgr2=0x55f8cc7d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 shutdown_connections 2024-09-15T22:21:56.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 1 -- 172.21.15.77:0/3490662916 wait complete. 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.433+0000 7fa2c2fab740 20 client.17952 trim_cache size 0 max 16384 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "069f0cec-740a-464d-b503-1166df2917b2": { 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "069f0cec-740a-464d-b503-1166df2917b2", 2024-09-15T22:21:56.453 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 msgr2=0x7fa2a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 0x7fa2a40a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009fd0 tx=0x7fa2b4008220 comp rx=0 tx=0).stop 2024-09-15T22:21:56.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 msgr2=0x7fa2bc198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005ab0 tx=0x7fa2b00042f0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 shutdown_connections 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa2a40a33b0 0x7fa2a40a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2bc10c180 0x7fa2bc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2bc1036c0 0x7fa2bc198b40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 --2- 172.21.15.77:0/1801025000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2bc102da0 0x7fa2bc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 >> 172.21.15.77:0/1801025000 conn(0x7fa2bc0fc7a0 msgr2=0x7fa2bc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 shutdown_connections 2024-09-15T22:21:56.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:21:56.434+0000 7fa2c0bd2640 1 -- 172.21.15.77:0/1801025000 wait complete. 2024-09-15T22:21:56.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.56 MB/sec execute 365 sec latency 7104.800 ms 2024-09-15T22:21:57.249 INFO:tasks.cephfs.filesystem:scrub status for tag:069f0cec-740a-464d-b503-1166df2917b2 - {'path': '/', 'tag': '069f0cec-740a-464d-b503-1166df2917b2', 'options': 'recursive,force'} 2024-09-15T22:21:57.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.54 MB/sec execute 366 sec latency 8104.880 ms 2024-09-15T22:21:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:57 smithi077 ceph-mon[30980]: pgmap v295: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-15T22:21:58.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:57 smithi196 ceph-mon[35109]: pgmap v295: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-15T22:21:58.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:57 smithi117 ceph-mon[35021]: pgmap v295: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 158 op/s 2024-09-15T22:21:58.483 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.52 MB/sec execute 367 sec latency 9104.975 ms 2024-09-15T22:21:59.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.50 MB/sec execute 368 sec latency 10105.073 ms 2024-09-15T22:22:00.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:21:59 smithi077 ceph-mon[30980]: pgmap v296: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-09-15T22:22:00.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:21:59 smithi196 ceph-mon[35109]: pgmap v296: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-09-15T22:22:00.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:21:59 smithi117 ceph-mon[35021]: pgmap v296: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 111 op/s 2024-09-15T22:22:00.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.48 MB/sec execute 369 sec latency 11105.173 ms 2024-09-15T22:22:01.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:00 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:22:01.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:00 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:22:01.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:00 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:22:01.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:00 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:22:01.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:00 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:22:01.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:00 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:22:01.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.46 MB/sec execute 370 sec latency 12105.271 ms 2024-09-15T22:22:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:01 smithi077 ceph-mon[30980]: pgmap v297: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 116 op/s 2024-09-15T22:22:02.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:01 smithi196 ceph-mon[35109]: pgmap v297: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 116 op/s 2024-09-15T22:22:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:01 smithi117 ceph-mon[35021]: pgmap v297: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 116 op/s 2024-09-15T22:22:02.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.44 MB/sec execute 371 sec latency 13105.339 ms 2024-09-15T22:22:03.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.42 MB/sec execute 372 sec latency 14105.417 ms 2024-09-15T22:22:04.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:03 smithi077 ceph-mon[30980]: pgmap v298: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:04.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:03 smithi196 ceph-mon[35109]: pgmap v298: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:04.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:03 smithi117 ceph-mon[35021]: pgmap v298: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:04.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.40 MB/sec execute 373 sec latency 15105.507 ms 2024-09-15T22:22:05.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.38 MB/sec execute 374 sec latency 16105.645 ms 2024-09-15T22:22:06.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:05 smithi077 ceph-mon[30980]: pgmap v299: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:06.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:05 smithi196 ceph-mon[35109]: pgmap v299: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:06.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:05 smithi117 ceph-mon[35021]: pgmap v299: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 727 KiB/s wr, 33 op/s 2024-09-15T22:22:06.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.36 MB/sec execute 375 sec latency 17105.742 ms 2024-09-15T22:22:06.836 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:07.094 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:07.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.34 MB/sec execute 376 sec latency 18105.830 ms 2024-09-15T22:22:07.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/1771814584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102ef0 msgr2=0x7f6a8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:07.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/1771814584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102ef0 0x7f6a8810da40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009ca0 tx=0x7f6a8002a950 comp rx=0 tx=0).stop 2024-09-15T22:22:07.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/1771814584 shutdown_connections 2024-09-15T22:22:07.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/1771814584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102ef0 0x7f6a8810da40 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/1771814584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102570 0x7f6a881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/1771814584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 0x7f6a88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/1771814584 >> 172.21.15.77:0/1771814584 conn(0x7f6a880fc7d0 msgr2=0x7f6a880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/1771814584 shutdown_connections 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.593+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/1771814584 wait complete. 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 Processor -- start 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 -- start start 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 0x7f6a8810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102ef0 0x7f6a8810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881120a0 con 0x7f6a88102570 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88111f20 con 0x7f6a88102ef0 2024-09-15T22:22:07.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8cf4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88112220 con 0x7f6a88100e20 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57622/0 (socket says 172.21.15.77:57622) 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 -- 172.21.15.77:0/3858167993 learned_addr learned my addr 172.21.15.77:0/3858167993 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a86d9d640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102ef0 0x7f6a8810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a8659c640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 0x7f6a8810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 -- 172.21.15.77:0/3858167993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 msgr2=0x7f6a8810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 0x7f6a8810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 -- 172.21.15.77:0/3858167993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102ef0 msgr2=0x7f6a8810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102ef0 0x7f6a8810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 -- 172.21.15.77:0/3858167993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a881a69a0 con 0x7f6a88102570 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.594+0000 7f6a7ffff640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000d950 tx=0x7f6a7000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.595+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70017070 con 0x7f6a88102570 2024-09-15T22:22:07.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.595+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7000de80 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.595+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000f620 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.595+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a881a6c90 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.595+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a881a7180 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.596+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88101200 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.597+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6a70009c30 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.598+0000 7f6a7f7fe640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 0x7f6a740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.598+0000 7f6a8659c640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 0x7f6a740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.598+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f6a700cfc70 con 0x7f6a88102570 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.598+0000 7f6a8659c640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 0x7f6a740a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009990 tx=0x7f6a78009110 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:07.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.602+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a7009c2c0 con 0x7f6a88102570 2024-09-15T22:22:07.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.802+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a881a73b0 con 0x7f6a88102570 2024-09-15T22:22:07.804 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:07 smithi077 ceph-mon[30980]: pgmap v300: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 781 KiB/s wr, 55 op/s 2024-09-15T22:22:07.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.803+0000 7f6a7f7fe640 1 -- 172.21.15.77:0/3858167993 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6a7009bbc0 con 0x7f6a88102570 2024-09-15T22:22:07.805 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:22:07.805 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:22:07.805 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:22:07.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 msgr2=0x7f6a740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:07.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 0x7f6a740a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009990 tx=0x7f6a78009110 comp rx=0 tx=0).stop 2024-09-15T22:22:07.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 msgr2=0x7f6a8810ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000d950 tx=0x7f6a7000dd50 comp rx=0 tx=0).stop 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 shutdown_connections 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a740a33b0 0x7f6a740a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a88102ef0 0x7f6a8810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a88102570 0x7f6a8810ea10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 --2- 172.21.15.77:0/3858167993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a88100e20 0x7f6a8810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 >> 172.21.15.77:0/3858167993 conn(0x7f6a880fc7d0 msgr2=0x7f6a88106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 shutdown_connections 2024-09-15T22:22:07.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:07.805+0000 7f6a8cf4e640 1 -- 172.21.15.77:0/3858167993 wait complete. 2024-09-15T22:22:08.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:07 smithi196 ceph-mon[35109]: pgmap v300: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 781 KiB/s wr, 55 op/s 2024-09-15T22:22:08.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:07 smithi117 ceph-mon[35021]: pgmap v300: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 781 KiB/s wr, 55 op/s 2024-09-15T22:22:08.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722813 7.32 MB/sec execute 377 sec latency 19105.950 ms 2024-09-15T22:22:08.638 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 97d2bb19 --release --await' 2024-09-15T22:22:08.638 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 97d2bb19 --release --await 2024-09-15T22:22:08.885 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:09.006 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:08 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3858167993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:09.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:08 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3858167993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:09.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:08 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3858167993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 -- 172.21.15.77:0/599987737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa18102d40 msgr2=0x7faa18103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 --2- 172.21.15.77:0/599987737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa18102d40 0x7faa18103120 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7faa08009900 tx=0x7faa0802a480 comp rx=0 tx=0).stop 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 -- 172.21.15.77:0/599987737 shutdown_connections 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 --2- 172.21.15.77:0/599987737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1810c170 0x7faa1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 --2- 172.21.15.77:0/599987737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa1810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 --2- 172.21.15.77:0/599987737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa18102d40 0x7faa18103120 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.359+0000 7faa1d898640 1 -- 172.21.15.77:0/599987737 >> 172.21.15.77:0/599987737 conn(0x7faa180fc7a0 msgr2=0x7faa180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- 172.21.15.77:0/599987737 shutdown_connections 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- 172.21.15.77:0/599987737 wait complete. 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 Processor -- start 2024-09-15T22:22:09.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- start start 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa18102d40 0x7faa18198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa18198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111f70 con 0x7faa1810c170 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111df0 con 0x7faa18102d40 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.360+0000 7faa1d898640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa181120f0 con 0x7faa18103660 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41140/0 (socket says 172.21.15.77:41140) 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 -- 172.21.15.77:0/3807275137 learned_addr learned my addr 172.21.15.77:0/3807275137 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa18102d40 0x7faa18198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa167fc640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa18198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 -- 172.21.15.77:0/3807275137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 msgr2=0x7faa18198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa18198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 -- 172.21.15.77:0/3807275137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa18102d40 msgr2=0x7faa18198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa18102d40 0x7faa18198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 -- 172.21.15.77:0/3807275137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819d400 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa167fc640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa18198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa177fe640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7faa1000cad0 tx=0x7faa1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa1c896640 1 -- 172.21.15.77:0/3807275137 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1000ae20 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa1c896640 1 -- 172.21.15.77:0/3807275137 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10005020 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819d6f0 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa1c896640 1 -- 172.21.15.77:0/3807275137 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f6f0 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.361+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a4fd0 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.364+0000 7faa1fc71740 1 Processor -- start 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.364+0000 7faa1c896640 1 -- 172.21.15.77:0/3807275137 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7faa100051d0 con 0x7faa1810c170 2024-09-15T22:22:09.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.364+0000 7faa1fc71740 1 -- start start 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f06440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f13630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1e6b620 con 0x55e9e1f06980 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1e12e50 con 0x55e9e1f11240 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1fc71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1e675e0 con 0x55e9e1f06060 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41150/0 (socket says 172.21.15.77:41150) 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 -- 172.21.15.77:0/2190103188 learned_addr learned my addr 172.21.15.77:0/2190103188 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa167fc640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f13630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa167fc640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f06440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 -- 172.21.15.77:0/2190103188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 msgr2=0x55e9e1f06440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1c896640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 0x7faa000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f06440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 -- 172.21.15.77:0/2190103188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 msgr2=0x55e9e1f13630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f13630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 -- 172.21.15.77:0/2190103188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e1e03d40 con 0x55e9e1f06980 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa167fc640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f06440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa177fe640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 0x7faa000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa16ffd640 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7faa08005ab0 tx=0x7faa08031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01933346c7fbe90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7faa1c896640 1 -- 172.21.15.77:0/3807275137 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7faa100d0190 con 0x7faa1810c170 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.365+0000 7fa9fdffb640 1 -- 172.21.15.77:0/2190103188 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0803f070 con 0x55e9e1f06980 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7fa9fdffb640 1 -- 172.21.15.77:0/2190103188 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa08031a20 con 0x55e9e1f06980 2024-09-15T22:22:09.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7fa9fdffb640 1 -- 172.21.15.77:0/2190103188 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08035070 con 0x55e9e1f06980 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa177fe640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 0x7faa000a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa10098540 tx=0x7faa100c6eb0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 -- 172.21.15.77:0/2190103188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 msgr2=0x55e9e1f10d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7faa08005ab0 tx=0x7faa08031630 comp rx=0 tx=0).stop 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 -- 172.21.15.77:0/2190103188 shutdown_connections 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f10d00 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 --2- 172.21.15.77:0/2190103188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f06440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 -- 172.21.15.77:0/2190103188 >> 172.21.15.77:0/2190103188 conn(0x55e9e1f025f0 msgr2=0x55e9e1f029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 -- 172.21.15.77:0/2190103188 shutdown_connections 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.366+0000 7faa1fc71740 1 -- 172.21.15.77:0/2190103188 wait complete. 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 -- start start 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 Processor -- start 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06060 0x55e9e1f0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1f13e10 con 0x55e9e1f06980 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1df3f10 con 0x55e9e1f06060 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e9e1f13f90 con 0x55e9e1f11240 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa1fc71740 10 client.0 _ll_register_callbacks cb 0x55e9e1e17db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41156/0 (socket says 172.21.15.77:41156) 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 -- 172.21.15.77:0/3597098676 learned_addr learned my addr 172.21.15.77:0/3597098676 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa167fc640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06060 0x55e9e1f0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa167fc640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 -- 172.21.15.77:0/3597098676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 msgr2=0x55e9e1f0ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f0ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 -- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06060 msgr2=0x55e9e1f0e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.367+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06060 0x55e9e1f0e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa16ffd640 1 -- 172.21.15.77:0/3597098676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e1b7a580 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7faa08031550 tx=0x7faa08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b45b3b52d54abb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0804c060 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7faa08004dc0 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa08031a20 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faa08053050 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 10 client.17970 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e9e1c44960 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa08031a20 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faa0803b430 con 0x55e9e1f06980 2024-09-15T22:22:09.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 10 client.17970 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 10 client.17970 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 20 client.17970 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.368+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f20c50 0x55e9e1f23040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.369+0000 7faa1fc71740 4 client.17970 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.369+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e1e78b90 con 0x55e9e1f20c50 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.369+0000 7faa167fc640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f20c50 0x55e9e1f23040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.369+0000 7fa9f6ffd640 10 client.17970 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.369+0000 7faa167fc640 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f20c50 0x55e9e1f23040 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcf9324d64ee51d server_cookie=86b4331ea641b808 in_seq=0 out_seq=0 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3597098676 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9e1e78b90 con 0x55e9e1f20c50 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7fa9f6ffd640 10 client.17970 handle_command_reply: tid=0 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 client.17970 shutdown 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 msgr2=0x55e9e1f0e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7faa08031550 tx=0x7faa08004a40 comp rx=0 tx=0).stop 2024-09-15T22:22:09.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 shutdown_connections 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f20c50 0x55e9e1f23040 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f0ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0e710 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3597098676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06060 0x55e9e1f0e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 >> 172.21.15.77:0/3597098676 conn(0x55e9e1f025f0 msgr2=0x55e9e1f08320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7fa9f6ffd640 0 client.17970 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 shutdown_connections 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 1 -- 172.21.15.77:0/3597098676 wait complete. 2024-09-15T22:22:09.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.370+0000 7faa1fc71740 20 client.17970 trim_cache size 0 max 16384 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 Processor -- start 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 -- start start 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa181af060 con 0x55e9e1f06980 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b0ba0 con 0x55e9e1f11240 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.387+0000 7faa1fc71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa181af730 con 0x55e9e1f06060 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41160/0 (socket says 172.21.15.77:41160) 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 -- 172.21.15.77:0/3725130137 learned_addr learned my addr 172.21.15.77:0/3725130137 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa167fc640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa167fc640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 -- 172.21.15.77:0/3725130137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 msgr2=0x55e9e1f1c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f1c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 -- 172.21.15.77:0/3725130137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 msgr2=0x55e9e1f0b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f0b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 -- 172.21.15.77:0/3725130137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e1f0d180 con 0x55e9e1f06980 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa167fc640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7faa080096c0 tx=0x7faa080047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc12425fb57a8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7fa9fd7fa640 1 -- 172.21.15.77:0/3725130137 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0804c070 con 0x55e9e1f06980 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7fa9fd7fa640 1 -- 172.21.15.77:0/3725130137 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0803b430 con 0x55e9e1f06980 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.388+0000 7fa9fd7fa640 1 -- 172.21.15.77:0/3725130137 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08035070 con 0x55e9e1f06980 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- 172.21.15.77:0/3725130137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 msgr2=0x55e9e1f0b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7faa080096c0 tx=0x7faa080047a0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- 172.21.15.77:0/3725130137 shutdown_connections 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f11240 0x55e9e1f0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06980 0x55e9e1f0b450 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3725130137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f06060 0x55e9e1f1c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- 172.21.15.77:0/3725130137 >> 172.21.15.77:0/3725130137 conn(0x55e9e1f025f0 msgr2=0x55e9e1f1c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- 172.21.15.77:0/3725130137 shutdown_connections 2024-09-15T22:22:09.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- 172.21.15.77:0/3725130137 wait complete. 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 -- start start 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.389+0000 7faa1fc71740 1 Processor -- start 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06980 0x55e9e1f24b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b04d0 con 0x55e9e1f06060 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa181afe00 con 0x55e9e1f06980 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ad060 con 0x55e9e1f11240 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 10 client.0 _ll_register_callbacks cb 0x7faa08005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41170/0 (socket says 172.21.15.77:41170) 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 -- 172.21.15.77:0/3930453523 learned_addr learned my addr 172.21.15.77:0/3930453523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06980 0x55e9e1f24b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 -- 172.21.15.77:0/3930453523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 msgr2=0x55e9e1f25ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f25ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 -- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06980 msgr2=0x55e9e1f24b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06980 0x55e9e1f24b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 -- 172.21.15.77:0/3930453523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e1f26300 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa167fc640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0058c0 tx=0x7faa08005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897cc73998065bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7fa9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c01e7d0 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7faa0c01e970 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.390+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0803b430 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faa0c031050 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 10 client.17976 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e9e1f90590 con 0x55e9e1f06060 2024-09-15T22:22:09.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0803b430 con 0x55e9e1f06060 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faa0c0103d0 con 0x55e9e1f06060 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 10 client.17976 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 10 client.17976 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 20 client.17976 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f90d20 0x55e9e1f93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 4 client.17976 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "97d2bb19", "release": true, "await": true}] 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "97d2bb19", "release": true, "await": true}) -- 0x55e9e1e730a0 con 0x55e9e1f90d20 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.391+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f90d20 0x55e9e1f93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.392+0000 7fa9f6ffd640 10 client.17976 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:09.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.392+0000 7faa16ffd640 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f90d20 0x55e9e1f93140 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36552ce74a9a2c73 server_cookie=7430a1b078cdcdf2 in_seq=0 out_seq=0 2024-09-15T22:22:09.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7fa9f6ffd640 1 -- 172.21.15.77:0/3930453523 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e9e1e730a0 con 0x55e9e1f90d20 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7fa9f6ffd640 10 client.17976 handle_command_reply: tid=0 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 client.17976 shutdown 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 msgr2=0x55e9e1f24630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0058c0 tx=0x7faa08005690 comp rx=0 tx=0).stop 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 shutdown_connections 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e9e1f90d20 0x55e9e1f93140 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e9e1f11240 0x55e9e1f25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9e1f06980 0x55e9e1f24b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7fa9f6ffd640 0 client.17976 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 --2- 172.21.15.77:0/3930453523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9e1f06060 0x55e9e1f24630 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 >> 172.21.15.77:0/3930453523 conn(0x55e9e1f025f0 msgr2=0x55e9e1f1ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 shutdown_connections 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 1 -- 172.21.15.77:0/3930453523 wait complete. 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.408+0000 7faa1fc71740 20 client.17976 trim_cache size 0 max 16384 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 36, 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "97d2bb19": { 2024-09-15T22:22:09.411 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 36, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.3, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:22:09.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:09.413 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 msgr2=0x7faa000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 0x7faa000a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa10098540 tx=0x7faa100c6eb0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 msgr2=0x7faa1819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7faa1000cad0 tx=0x7faa1000cf90 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 shutdown_connections 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faa000a33b0 0x7faa000a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa1810c170 0x7faa1819cd20 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa18103660 0x7faa18198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 --2- 172.21.15.77:0/3807275137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa18102d40 0x7faa18198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:09.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 >> 172.21.15.77:0/3807275137 conn(0x7faa180fc7a0 msgr2=0x7faa180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:09.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 shutdown_connections 2024-09-15T22:22:09.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:09.409+0000 7faa1d898640 1 -- 172.21.15.77:0/3807275137 wait complete. 2024-09-15T22:22:09.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 722998 7.31 MB/sec execute 378 sec latency 20033.202 ms 2024-09-15T22:22:10.009 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:09 smithi077 ceph-mon[30980]: pgmap v301: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-09-15T22:22:10.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:09 smithi196 ceph-mon[35109]: pgmap v301: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-09-15T22:22:10.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:09 smithi117 ceph-mon[35021]: pgmap v301: 129 pgs: 129 active+clean; 479 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-09-15T22:22:10.150 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '97d2bb19', total seconds elapsed: 25.7 2024-09-15T22:22:10.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 723770 7.30 MB/sec execute 379 sec latency 63.748 ms 2024-09-15T22:22:11.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 725588 7.31 MB/sec execute 380 sec latency 15.461 ms 2024-09-15T22:22:12.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:11 smithi077 ceph-mon[30980]: pgmap v302: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 210 KiB/s wr, 54 op/s 2024-09-15T22:22:12.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:11 smithi077 ceph-mon[30980]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:22:12.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:11 smithi196 ceph-mon[35109]: pgmap v302: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 210 KiB/s wr, 54 op/s 2024-09-15T22:22:12.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:11 smithi196 ceph-mon[35109]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:22:12.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:11 smithi117 ceph-mon[35021]: pgmap v302: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 210 KiB/s wr, 54 op/s 2024-09-15T22:22:12.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:11 smithi117 ceph-mon[35021]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:22:12.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 728187 7.33 MB/sec execute 381 sec latency 14.825 ms 2024-09-15T22:22:13.484 INFO:tasks.workunit.client.0.smithi077.stdout: 1 730293 7.33 MB/sec execute 382 sec latency 17.927 ms 2024-09-15T22:22:14.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:13 smithi077 ceph-mon[30980]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:14.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:13 smithi196 ceph-mon[35109]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:14.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:13 smithi117 ceph-mon[35021]: pgmap v304: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:14.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 732286 7.35 MB/sec execute 383 sec latency 12.526 ms 2024-09-15T22:22:15.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 735013 7.37 MB/sec execute 384 sec latency 13.200 ms 2024-09-15T22:22:16.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:15 smithi196 ceph-mon[35109]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:16.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:15 smithi117 ceph-mon[35021]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:16.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:15 smithi077 ceph-mon[30980]: pgmap v305: 129 pgs: 129 active+clean; 480 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 251 KiB/s wr, 59 op/s 2024-09-15T22:22:16.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 736530 7.37 MB/sec execute 385 sec latency 32.212 ms 2024-09-15T22:22:17.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 738704 7.38 MB/sec execute 386 sec latency 23.833 ms 2024-09-15T22:22:18.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:17 smithi196 ceph-mon[35109]: pgmap v306: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:18.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:17 smithi117 ceph-mon[35021]: pgmap v306: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:17 smithi077 ceph-mon[30980]: pgmap v306: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:18.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 740281 7.38 MB/sec execute 387 sec latency 28.748 ms 2024-09-15T22:22:19.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 742274 7.39 MB/sec execute 388 sec latency 55.751 ms 2024-09-15T22:22:20.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:19 smithi196 ceph-mon[35109]: pgmap v307: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:20.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:19 smithi196 ceph-mon[35109]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:22:20.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:19 smithi117 ceph-mon[35021]: pgmap v307: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:20.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:19 smithi117 ceph-mon[35021]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:22:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:19 smithi077 ceph-mon[30980]: pgmap v307: 129 pgs: 129 active+clean; 492 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-15T22:22:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:19 smithi077 ceph-mon[30980]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:22:20.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 744045 7.39 MB/sec execute 389 sec latency 15.051 ms 2024-09-15T22:22:21.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 745937 7.41 MB/sec execute 390 sec latency 47.641 ms 2024-09-15T22:22:22.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:21 smithi196 ceph-mon[35109]: pgmap v309: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-09-15T22:22:22.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:21 smithi117 ceph-mon[35021]: pgmap v309: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-09-15T22:22:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:21 smithi077 ceph-mon[30980]: pgmap v309: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 193 op/s 2024-09-15T22:22:22.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 747658 7.40 MB/sec execute 391 sec latency 15.429 ms 2024-09-15T22:22:23.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 749703 7.42 MB/sec execute 392 sec latency 16.688 ms 2024-09-15T22:22:24.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:23 smithi196 ceph-mon[35109]: pgmap v310: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:24.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:23 smithi117 ceph-mon[35021]: pgmap v310: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:24.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:23 smithi077 ceph-mon[30980]: pgmap v310: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:24.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 752147 7.43 MB/sec execute 393 sec latency 15.649 ms 2024-09-15T22:22:25.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 753525 7.43 MB/sec execute 394 sec latency 23.773 ms 2024-09-15T22:22:26.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:25 smithi196 ceph-mon[35109]: pgmap v311: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:26.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:25 smithi117 ceph-mon[35021]: pgmap v311: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:25 smithi077 ceph-mon[30980]: pgmap v311: 129 pgs: 129 active+clean; 504 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-09-15T22:22:26.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 756131 7.45 MB/sec execute 395 sec latency 22.272 ms 2024-09-15T22:22:27.250 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:22:27.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 757296 7.44 MB/sec execute 396 sec latency 13.568 ms 2024-09-15T22:22:27.524 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:27.953 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:27 smithi077 ceph-mon[30980]: pgmap v312: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 -- 172.21.15.77:0/4215244381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e4102d40 msgr2=0x7fd1e4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 --2- 172.21.15.77:0/4215244381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e4102d40 0x7fd1e4103120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009900 tx=0x7fd1cc02a480 comp rx=0 tx=0).stop 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 -- 172.21.15.77:0/4215244381 shutdown_connections 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 --2- 172.21.15.77:0/4215244381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e410c170 0x7fd1e410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 --2- 172.21.15.77:0/4215244381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4103660 0x7fd1e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 --2- 172.21.15.77:0/4215244381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e4102d40 0x7fd1e4103120 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 -- 172.21.15.77:0/4215244381 >> 172.21.15.77:0/4215244381 conn(0x7fd1e40fc7a0 msgr2=0x7fd1e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 -- 172.21.15.77:0/4215244381 shutdown_connections 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.035+0000 7fd1ea168640 1 -- 172.21.15.77:0/4215244381 wait complete. 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 Processor -- start 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 -- start start 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 0x7fd1e41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4111f70 con 0x7fd1e410c170 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4111df0 con 0x7fd1e4103660 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1ea168640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41120f0 con 0x7fd1e4102d40 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1e37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.036+0000 7fd1e37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:46076/0 (socket says 172.21.15.77:46076) 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45900/0 (socket says 172.21.15.77:45900) 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e37fe640 1 -- 172.21.15.77:0/649030901 learned_addr learned my addr 172.21.15.77:0/649030901 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 0x7fd1e41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 -- 172.21.15.77:0/649030901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 msgr2=0x7fd1e4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 -- 172.21.15.77:0/649030901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 msgr2=0x7fd1e41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 0x7fd1e41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 -- 172.21.15.77:0/649030901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e419d420 con 0x7fd1e410c170 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 0x7fd1e41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e3fff640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009fd0 tx=0x7fd1dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e0ff9640 1 -- 172.21.15.77:0/649030901 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00ce20 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e0ff9640 1 -- 172.21.15.77:0/649030901 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc005020 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e419d710 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1e0ff9640 1 -- 172.21.15.77:0/649030901 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc010720 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.037+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e41a4ff0 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e0ff9640 1 -- 172.21.15.77:0/649030901 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd1dc005540 con 0x7fd1e410c170 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 Processor -- start 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 -- start start 2024-09-15T22:22:28.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa4a9b2e0 con 0x555aa4b354a0 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555aa4a42ce0 con 0x555aa4b40680 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1ec541740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555aa4a972a0 con 0x555aa4b35dc0 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e0ff9640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 0x7fd1d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45914/0 (socket says 172.21.15.77:45914) 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/4168950546 learned_addr learned my addr 172.21.15.77:0/4168950546 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.040+0000 7fd1e3fff640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 0x7fd1d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e0ff9640 1 -- 172.21.15.77:0/649030901 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd1dc0d0730 con 0x7fd1e410c170 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/4168950546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 msgr2=0x555aa4b40140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b40140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/4168950546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 msgr2=0x555aa4b42a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b42a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/4168950546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa4a27440 con 0x555aa4b354a0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4013bc0 tx=0x7fd1d4009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b1b767244fcde server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1e3fff640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 0x7fd1d00a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0cdba0 tx=0x7fd1dc06c2a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1d9ffb640 1 -- 172.21.15.77:0/4168950546 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4013cb0 con 0x555aa4b354a0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1d9ffb640 1 -- 172.21.15.77:0/4168950546 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4013e50 con 0x555aa4b354a0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.041+0000 7fd1d9ffb640 1 -- 172.21.15.77:0/4168950546 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4003c50 con 0x555aa4b354a0 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- 172.21.15.77:0/4168950546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 msgr2=0x555aa4b35880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4013bc0 tx=0x7fd1d4009b40 comp rx=0 tx=0).stop 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- 172.21.15.77:0/4168950546 shutdown_connections 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b40140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 --2- 172.21.15.77:0/4168950546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b35880 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- 172.21.15.77:0/4168950546 >> 172.21.15.77:0/4168950546 conn(0x555aa4b31a30 msgr2=0x555aa4b31e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- 172.21.15.77:0/4168950546 shutdown_connections 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- 172.21.15.77:0/4168950546 wait complete. 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.042+0000 7fd1ec541740 1 -- start start 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 Processor -- start 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b40680 0x555aa4b3d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa4b43250 con 0x555aa4b40680 2024-09-15T22:22:28.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555aa4a56050 con 0x555aa4b354a0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555aa4b433d0 con 0x555aa4b35dc0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1ec541740 10 client.0 _ll_register_callbacks cb 0x555aa4a96160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53894/0 (socket says 172.21.15.77:53894) 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/1032181817 learned_addr learned my addr 172.21.15.77:0/1032181817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b40680 0x555aa4b3d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/1032181817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 msgr2=0x555aa4b3d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/1032181817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b40680 msgr2=0x555aa4b3d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b40680 0x555aa4b3d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/1032181817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa480c3c0 con 0x555aa4b354a0 2024-09-15T22:22:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40037b0 tx=0x7fd1d4015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30981076dc625c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.043+0000 7fd1c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d40207e0 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd1d4020980 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d4013e50 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d4034050 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 10 client.27041 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555aa4b4ec70 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4013e50 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd1d402dc60 con 0x555aa4b354a0 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 10 client.27041 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:28.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 20 client.27041 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4b4ffc0 0x555aa4b523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 4 client.27041 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa4aa8850 con 0x555aa4b4ffc0 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.044+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4b4ffc0 0x555aa4b523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.045+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4b4ffc0 0x555aa4b523b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fdf728e0e366b0a server_cookie=789d886bfa5a772f in_seq=0 out_seq=0 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.045+0000 7fd1c2ffd640 10 client.27041 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:28.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.045+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/1032181817 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555aa4aa8850 con 0x555aa4b4ffc0 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.045+0000 7fd1c2ffd640 10 client.27041 handle_command_reply: tid=0 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.045+0000 7fd1ec541740 1 client.27041 shutdown 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 msgr2=0x555aa4b40040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40037b0 tx=0x7fd1d4015040 comp rx=0 tx=0).stop 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 shutdown_connections 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4b4ffc0 0x555aa4b523b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b40680 0x555aa4b3d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1c2ffd640 0 client.27041 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:28.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 --2- 172.21.15.77:0/1032181817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b354a0 0x555aa4b40040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 >> 172.21.15.77:0/1032181817 conn(0x555aa4b31a30 msgr2=0x555aa4b37760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 shutdown_connections 2024-09-15T22:22:28.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 1 -- 172.21.15.77:0/1032181817 wait complete. 2024-09-15T22:22:28.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.046+0000 7fd1ec541740 20 client.27041 trim_cache size 0 max 16384 2024-09-15T22:22:28.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 Processor -- start 2024-09-15T22:22:28.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 -- start start 2024-09-15T22:22:28.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.065+0000 7fd1ec541740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41aef60 con 0x555aa4b354a0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1ec541740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41b0aa0 con 0x555aa4b40680 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1ec541740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41af630 con 0x555aa4b35dc0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45928/0 (socket says 172.21.15.77:45928) 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3527315991 learned_addr learned my addr 172.21.15.77:0/3527315991 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3527315991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 msgr2=0x555aa4b3bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3527315991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 msgr2=0x555aa4b57ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b57ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3527315991 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa4b3c2a0 con 0x555aa4b354a0 2024-09-15T22:22:28.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003880 tx=0x7fd1d401ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b72a8b09b8bf64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1d97fa640 1 -- 172.21.15.77:0/3527315991 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4009b20 con 0x555aa4b354a0 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1d97fa640 1 -- 172.21.15.77:0/3527315991 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4003e50 con 0x555aa4b354a0 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.066+0000 7fd1d97fa640 1 -- 172.21.15.77:0/3527315991 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4005070 con 0x555aa4b354a0 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- 172.21.15.77:0/3527315991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 msgr2=0x555aa4b4c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003880 tx=0x7fd1d401ffa0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- 172.21.15.77:0/3527315991 shutdown_connections 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4b57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b3bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3527315991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b4c4b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- 172.21.15.77:0/3527315991 >> 172.21.15.77:0/3527315991 conn(0x555aa4b31a30 msgr2=0x555aa4b38900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- 172.21.15.77:0/3527315991 shutdown_connections 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- 172.21.15.77:0/3527315991 wait complete. 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.067+0000 7fd1ec541740 1 -- start start 2024-09-15T22:22:28.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 Processor -- start 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4bb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41b03d0 con 0x555aa4b354a0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41afd00 con 0x555aa4b40680 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e41acf60 con 0x555aa4b35dc0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1ec541740 10 client.0 _ll_register_callbacks cb 0x7fd1d40136b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45936/0 (socket says 172.21.15.77:45936) 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3380162173 learned_addr learned my addr 172.21.15.77:0/3380162173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4bb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3380162173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 msgr2=0x555aa4b44c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b44c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 msgr2=0x555aa4bb9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4bb9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.068+0000 7fd1e2ffd640 1 -- 172.21.15.77:0/3380162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa4b35bf0 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1e2ffd640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003c10 tx=0x7fd1d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc75407b1a07c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4003920 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd1d4009b20 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d4003e50 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d404a050 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 10 client.17994 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555aa4b3dc50 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4003e50 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd1d40396b0 con 0x555aa4b354a0 2024-09-15T22:22:28.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 10 client.17994 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 20 client.17994 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4bbb3e0 0x555aa4bbd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 4 client.17994 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.069+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555aa4aa2d60 con 0x555aa4bbb3e0 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4bbb3e0 0x555aa4bbd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1e37fe640 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4bbb3e0 0x555aa4bbd7d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae6eb59b130dc54 server_cookie=27539e1628119fff in_seq=0 out_seq=0 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1c2ffd640 10 client.17994 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1c2ffd640 1 -- 172.21.15.77:0/3380162173 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555aa4aa2d60 con 0x555aa4bbb3e0 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1c2ffd640 10 client.17994 handle_command_reply: tid=0 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 client.17994 shutdown 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 msgr2=0x555aa4b44700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4003c10 tx=0x7fd1d4005f90 comp rx=0 tx=0).stop 2024-09-15T22:22:28.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 shutdown_connections 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555aa4bbb3e0 0x555aa4bbd7d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aa4b40680 0x555aa4bb9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555aa4b35dc0 0x555aa4b44c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.070+0000 7fd1ec541740 1 --2- 172.21.15.77:0/3380162173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa4b354a0 0x555aa4b44700 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 >> 172.21.15.77:0/3380162173 conn(0x555aa4b31a30 msgr2=0x555aa4b30b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1c2ffd640 0 client.17994 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 shutdown_connections 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ec541740 1 -- 172.21.15.77:0/3380162173 wait complete. 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ec541740 20 client.17994 trim_cache size 0 max 16384 2024-09-15T22:22:28.087 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:22:28.088 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:22:28.088 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:22:28.088 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 msgr2=0x7fd1d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 0x7fd1d00a5870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0cdba0 tx=0x7fd1dc06c2a0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 msgr2=0x7fd1e419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009fd0 tx=0x7fd1dc00ef10 comp rx=0 tx=0).stop 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 shutdown_connections 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1d00a33b0 0x7fd1d00a5870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e410c170 0x7fd1e419cd40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1e4103660 0x7fd1e41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 --2- 172.21.15.77:0/649030901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1e4102d40 0x7fd1e4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:28.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.071+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 >> 172.21.15.77:0/649030901 conn(0x7fd1e40fc7a0 msgr2=0x7fd1e40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:28.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.072+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 shutdown_connections 2024-09-15T22:22:28.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:28.072+0000 7fd1ea168640 1 -- 172.21.15.77:0/649030901 wait complete. 2024-09-15T22:22:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:27 smithi196 ceph-mon[35109]: pgmap v312: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:27 smithi117 ceph-mon[35021]: pgmap v312: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:28.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 759861 7.47 MB/sec execute 397 sec latency 46.637 ms 2024-09-15T22:22:28.920 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:22:28.921 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:29.176 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:29.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 761045 7.46 MB/sec execute 398 sec latency 25.331 ms 2024-09-15T22:22:29.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 -- 172.21.15.77:0/2622308081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298101de0 msgr2=0x7f82981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:29.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 --2- 172.21.15.77:0/2622308081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298101de0 0x7f82981021c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8288009900 tx=0x7f828802a410 comp rx=0 tx=0).stop 2024-09-15T22:22:29.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 -- 172.21.15.77:0/2622308081 shutdown_connections 2024-09-15T22:22:29.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 --2- 172.21.15.77:0/2622308081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298106f10 0x7f829810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 --2- 172.21.15.77:0/2622308081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298102790 0x7f82981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 --2- 172.21.15.77:0/2622308081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298101de0 0x7f82981021c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 -- 172.21.15.77:0/2622308081 >> 172.21.15.77:0/2622308081 conn(0x7f82980fd8e0 msgr2=0x7f82980ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 -- 172.21.15.77:0/2622308081 shutdown_connections 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.656+0000 7f829d040640 1 -- 172.21.15.77:0/2622308081 wait complete. 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 Processor -- start 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 -- start start 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298101de0 0x7f829819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298102790 0x7f829819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8298111f70 con 0x7f8298106f10 2024-09-15T22:22:29.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8298111df0 con 0x7f8298101de0 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.657+0000 7f829d040640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82981120f0 con 0x7f8298102790 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f828ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298102790 0x7f829819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46290/0 (socket says 172.21.15.77:46290) 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 -- 172.21.15.77:0/1991648590 learned_addr learned my addr 172.21.15.77:0/1991648590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f8296d9d640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298101de0 0x7f829819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 -- 172.21.15.77:0/1991648590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298102790 msgr2=0x7f829819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298102790 0x7f829819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 -- 172.21.15.77:0/1991648590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298101de0 msgr2=0x7f829819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298101de0 0x7f829819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 -- 172.21.15.77:0/1991648590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829819f5f0 con 0x7f8298106f10 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829759e640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f829000eaa0 tx=0x7f829000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829000cdb0 con 0x7f8298106f10 2024-09-15T22:22:29.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8290005020 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829819f8e0 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82900116b0 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.658+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981a71c0 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.659+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82981021c0 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.660+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8290005560 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.661+0000 7f8294d99640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 0x7f82840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.661+0000 7f8296d9d640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 0x7f82840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.661+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f82900d0140 con 0x7f8298106f10 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.662+0000 7f8296d9d640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 0x7f82840a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8288009230 tx=0x7f82880058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:29.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.665+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8290016030 con 0x7f8298106f10 2024-09-15T22:22:29.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.887+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82981a7500 con 0x7f8298106f10 2024-09-15T22:22:29.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.887+0000 7f8294d99640 1 -- 172.21.15.77:0/1991648590 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f829009c140 con 0x7f8298106f10 2024-09-15T22:22:29.890 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:22:29.890 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:22:29.891 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:22:29.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 msgr2=0x7f82840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:29.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 0x7f82840a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8288009230 tx=0x7f82880058d0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 msgr2=0x7f829819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f829000eaa0 tx=0x7f829000ef60 comp rx=0 tx=0).stop 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 shutdown_connections 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82840a3430 0x7f82840a58f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8298106f10 0x7f829819ef10 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8298102790 0x7f829819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 --2- 172.21.15.77:0/1991648590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8298101de0 0x7f829819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.889+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 >> 172.21.15.77:0/1991648590 conn(0x7f82980fd8e0 msgr2=0x7f82980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.890+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 shutdown_connections 2024-09-15T22:22:29.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:29.890+0000 7f829d040640 1 -- 172.21.15.77:0/1991648590 wait complete. 2024-09-15T22:22:30.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:29 smithi196 ceph-mon[35109]: pgmap v313: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:29 smithi117 ceph-mon[35021]: pgmap v313: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:30.219 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:29 smithi077 ceph-mon[30980]: pgmap v313: 129 pgs: 129 active+clean; 515 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 201 op/s 2024-09-15T22:22:30.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 763408 7.48 MB/sec execute 399 sec latency 66.307 ms 2024-09-15T22:22:30.570 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:22:30.835 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:31.105 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:30 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1991648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:31.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:30 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1991648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 -- 172.21.15.77:0/861227083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae841002f0 msgr2=0x7fae841006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 --2- 172.21.15.77:0/861227083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae841002f0 0x7fae841006d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fae74009cd0 tx=0x7fae7402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 -- 172.21.15.77:0/861227083 shutdown_connections 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 --2- 172.21.15.77:0/861227083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 0x7fae8410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 --2- 172.21.15.77:0/861227083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae84100ca0 0x7fae84106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 --2- 172.21.15.77:0/861227083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae841002f0 0x7fae841006d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 -- 172.21.15.77:0/861227083 >> 172.21.15.77:0/861227083 conn(0x7fae84077210 msgr2=0x7fae840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 -- 172.21.15.77:0/861227083 shutdown_connections 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.318+0000 7fae8b285640 1 -- 172.21.15.77:0/861227083 wait complete. 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 Processor -- start 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 -- start start 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae841002f0 0x7fae84101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 0x7fae84101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae84111fa0 con 0x7fae84100ca0 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae84111e20 con 0x7fae84106f50 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae8b285640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae84112120 con 0x7fae841002f0 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46308/0 (socket says 172.21.15.77:46308) 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 -- 172.21.15.77:0/4197963556 learned_addr learned my addr 172.21.15.77:0/4197963556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae89822640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 0x7fae84101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae89021640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae841002f0 0x7fae84101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 -- 172.21.15.77:0/4197963556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae841002f0 msgr2=0x7fae84101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae841002f0 0x7fae84101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 -- 172.21.15.77:0/4197963556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 msgr2=0x7fae84101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 0x7fae84101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 -- 172.21.15.77:0/4197963556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841a69c0 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.319+0000 7fae88820640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9f0 tx=0x7fae7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.320+0000 7fae827fc640 1 -- 172.21.15.77:0/4197963556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7800ccc0 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.320+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841a6cb0 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.320+0000 7fae827fc640 1 -- 172.21.15.77:0/4197963556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78005500 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.320+0000 7fae827fc640 1 -- 172.21.15.77:0/4197963556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78010690 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.320+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a71a0 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.322+0000 7fae827fc640 1 -- 172.21.15.77:0/4197963556 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fae780046c0 con 0x7fae84100ca0 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.322+0000 7fae8d65e740 1 Processor -- start 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.322+0000 7fae8d65e740 1 -- start start 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c27ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c289370 0x56148c28b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56148c1e42f0 con 0x56148c27fb40 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56148c18bcf0 con 0x56148c27e5d0 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae8d65e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56148c1e02b0 con 0x56148c289370 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c27ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46314/0 (socket says 172.21.15.77:46314) 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae88820640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c289370 0x56148c28b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89822640 1 -- 172.21.15.77:0/265150653 learned_addr learned my addr 172.21.15.77:0/265150653 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae827fc640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 0x7fae6c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 -- 172.21.15.77:0/265150653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c289370 msgr2=0x56148c28b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c289370 0x56148c28b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 -- 172.21.15.77:0/265150653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 msgr2=0x56148c27e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89021640 1 -- 172.21.15.77:0/265150653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148c1703a0 con 0x56148c27e5d0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89822640 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae89822640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 0x7fae6c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.323+0000 7fae827fc640 1 -- 172.21.15.77:0/4197963556 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fae780d0420 con 0x7fae84100ca0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.324+0000 7fae89021640 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c27ea10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae74005e30 tx=0x7fae74009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c669f27791b6990e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.324+0000 7fae6b7fe640 1 -- 172.21.15.77:0/265150653 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74048070 con 0x56148c27e5d0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.324+0000 7fae6b7fe640 1 -- 172.21.15.77:0/265150653 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae74036070 con 0x56148c27e5d0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.324+0000 7fae6b7fe640 1 -- 172.21.15.77:0/265150653 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74004360 con 0x56148c27e5d0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- 172.21.15.77:0/265150653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 msgr2=0x56148c27ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c27ea10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae74005e30 tx=0x7fae74009a20 comp rx=0 tx=0).stop 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae89822640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 0x7fae6c0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0104f0 tx=0x7fae7c0046c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- 172.21.15.77:0/265150653 shutdown_connections 2024-09-15T22:22:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c289370 0x56148c28b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c27ea10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- 172.21.15.77:0/265150653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c27fb40 0x56148c27e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- 172.21.15.77:0/265150653 >> 172.21.15.77:0/265150653 conn(0x56148c247310 msgr2=0x56148c27ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- 172.21.15.77:0/265150653 shutdown_connections 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- 172.21.15.77:0/265150653 wait complete. 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 -- start start 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 Processor -- start 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.325+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27fb40 0x56148c285fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c286560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56148c28c010 con 0x56148c289370 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56148c19f060 con 0x56148c27e5d0 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56148c28c190 con 0x56148c27fb40 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 10 client.0 _ll_register_callbacks cb 0x56148bef28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38428/0 (socket says 172.21.15.77:38428) 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c286560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 -- 172.21.15.77:0/1648020337 learned_addr learned my addr 172.21.15.77:0/1648020337 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae88820640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27fb40 0x56148c285fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 -- 172.21.15.77:0/1648020337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27fb40 msgr2=0x56148c285fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27fb40 0x56148c285fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 -- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 msgr2=0x56148c286560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c286560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 -- 172.21.15.77:0/1648020337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148bf553c0 con 0x56148c27e5d0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae89021640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae74006010 tx=0x7fae74003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a04363eed26c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74050070 con 0x56148c27e5d0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fae740318c0 con 0x56148c27e5d0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae74036070 con 0x56148c27e5d0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae74048040 con 0x56148c27e5d0 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 10 client.27065 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:31.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.326+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56148c28c010 con 0x56148c27e5d0 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae74036070 con 0x56148c27e5d0 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae7404b050 con 0x56148c27e5d0 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 10 client.27065 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 10 client.27065 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 20 client.27065 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c298be0 0x56148c29afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 4 client.27065 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148c1f1860 con 0x56148c298be0 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae88820640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c298be0 0x56148c29afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae68ff9640 10 client.27065 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:22:31.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.327+0000 7fae88820640 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c298be0 0x56148c29afd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd4be8fb902e22cd server_cookie=91d04191b7c2c9ab in_seq=0 out_seq=0 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae68ff9640 1 -- 172.21.15.77:0/1648020337 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56148c1f1860 con 0x56148c298be0 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae68ff9640 10 client.27065 handle_command_reply: tid=0 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 client.27065 shutdown 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 msgr2=0x56148c288d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae74006010 tx=0x7fae74003c20 comp rx=0 tx=0).stop 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 shutdown_connections 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c298be0 0x56148c29afd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c286560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27fb40 0x56148c285fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 --2- 172.21.15.77:0/1648020337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27e5d0 0x56148c288d10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae68ff9640 0 client.27065 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:22:31.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 >> 172.21.15.77:0/1648020337 conn(0x56148c247310 msgr2=0x56148c280930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 shutdown_connections 2024-09-15T22:22:31.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 1 -- 172.21.15.77:0/1648020337 wait complete. 2024-09-15T22:22:31.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.328+0000 7fae8d65e740 20 client.27065 trim_cache size 0 max 16384 2024-09-15T22:22:31.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 Processor -- start 2024-09-15T22:22:31.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 -- start start 2024-09-15T22:22:31.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c279bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c2a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b1220 con 0x56148c289370 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b2d60 con 0x56148c27fb40 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.345+0000 7fae8d65e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b18f0 con 0x56148c27e5d0 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c2a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33800/0 (socket says 172.21.15.77:33800) 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 -- 172.21.15.77:0/834882390 learned_addr learned my addr 172.21.15.77:0/834882390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c2a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46334/0 (socket says 172.21.15.77:46334) 2024-09-15T22:22:31.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae88820640 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c279bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 -- 172.21.15.77:0/834882390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 msgr2=0x56148c279bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c279bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 -- 172.21.15.77:0/834882390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 msgr2=0x56148c2a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c2a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 -- 172.21.15.77:0/834882390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148c284550 con 0x56148c27e5d0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae89021640 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae74002230 tx=0x7fae74009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5fe680d2b02530 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae6affd640 1 -- 172.21.15.77:0/834882390 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74050040 con 0x56148c27e5d0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae6affd640 1 -- 172.21.15.77:0/834882390 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae74036040 con 0x56148c27e5d0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.346+0000 7fae6affd640 1 -- 172.21.15.77:0/834882390 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74004200 con 0x56148c27e5d0 2024-09-15T22:22:31.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 -- 172.21.15.77:0/834882390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 msgr2=0x56148c2952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae74002230 tx=0x7fae74009a20 comp rx=0 tx=0).stop 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 -- 172.21.15.77:0/834882390 shutdown_connections 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c2a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c279bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 --2- 172.21.15.77:0/834882390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c2952b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 -- 172.21.15.77:0/834882390 >> 172.21.15.77:0/834882390 conn(0x56148c247310 msgr2=0x56148c2794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 -- 172.21.15.77:0/834882390 shutdown_connections 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.347+0000 7fae8d65e740 1 -- 172.21.15.77:0/834882390 wait complete. 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 -- start start 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 Processor -- start 2024-09-15T22:22:31.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c282db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c283310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b2690 con 0x56148c289370 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae841b1fc0 con 0x56148c27fb40 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae841af220 con 0x56148c27e5d0 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae8d65e740 10 client.0 _ll_register_callbacks cb 0x7fae74046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.348+0000 7fae89021640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89822640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c283310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33802/0 (socket says 172.21.15.77:33802) 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 -- 172.21.15.77:0/3773724606 learned_addr learned my addr 172.21.15.77:0/3773724606 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae88820640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c282db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 -- 172.21.15.77:0/3773724606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 msgr2=0x56148c282db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c282db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 -- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 msgr2=0x56148c283310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c283310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 -- 172.21.15.77:0/3773724606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148c283b40 con 0x56148c27e5d0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae89021640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae74005e30 tx=0x7fae74000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afbc35a61c1180 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74050040 con 0x56148c27e5d0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fae740309f0 con 0x56148c27e5d0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae74036040 con 0x56148c27e5d0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.349+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae7406e050 con 0x56148c27e5d0 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 10 client.26998 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:31.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56148c303750 con 0x56148c27e5d0 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae74036040 con 0x56148c27e5d0 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae74069030 con 0x56148c27e5d0 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 10 client.26998 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 10 client.26998 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 20 client.26998 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c303c20 0x56148c306010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 4 client.26998 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.350+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56148c1ebd70 con 0x56148c303c20 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae88820640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c303c20 0x56148c306010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae88820640 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c303c20 0x56148c306010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5421fe12b6d88764 server_cookie=f548211acfce308a in_seq=0 out_seq=0 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae68ff9640 10 client.26998 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae68ff9640 1 -- 172.21.15.77:0/3773724606 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56148c1ebd70 con 0x56148c303c20 2024-09-15T22:22:31.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae68ff9640 10 client.26998 handle_command_reply: tid=0 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae8d65e740 1 client.26998 shutdown 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 msgr2=0x56148c291c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae74005e30 tx=0x7fae74000f40 comp rx=0 tx=0).stop 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.351+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 shutdown_connections 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56148c303c20 0x56148c306010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56148c289370 0x56148c283310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56148c27fb40 0x56148c282db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 --2- 172.21.15.77:0/3773724606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56148c27e5d0 0x56148c291c00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 >> 172.21.15.77:0/3773724606 conn(0x56148c247310 msgr2=0x56148c279d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae68ff9640 0 client.26998 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 shutdown_connections 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 1 -- 172.21.15.77:0/3773724606 wait complete. 2024-09-15T22:22:31.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8d65e740 20 client.26998 trim_cache size 0 max 16384 2024-09-15T22:22:31.361 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:22:31.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:30 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1991648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:31.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 msgr2=0x7fae6c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 0x7fae6c0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0104f0 tx=0x7fae7c0046c0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 msgr2=0x7fae841017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.352+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9f0 tx=0x7fae7800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 shutdown_connections 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fae6c0a33b0 0x7fae6c0a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84106f50 0x7fae84101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae84100ca0 0x7fae841017b0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 --2- 172.21.15.77:0/4197963556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae841002f0 0x7fae84101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 >> 172.21.15.77:0/4197963556 conn(0x7fae84077210 msgr2=0x7fae841052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 shutdown_connections 2024-09-15T22:22:31.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:31.353+0000 7fae8b285640 1 -- 172.21.15.77:0/4197963556 wait complete. 2024-09-15T22:22:31.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 764551 7.47 MB/sec execute 400 sec latency 92.535 ms 2024-09-15T22:22:32.061 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:22:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:31 smithi117 ceph-mon[35021]: pgmap v314: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 246 op/s 2024-09-15T22:22:32.327 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:32.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:31 smithi077 ceph-mon[30980]: pgmap v314: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 246 op/s 2024-09-15T22:22:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:31 smithi196 ceph-mon[35109]: pgmap v314: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.2 MiB/s wr, 246 op/s 2024-09-15T22:22:32.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 767026 7.49 MB/sec execute 401 sec latency 14.487 ms 2024-09-15T22:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- 172.21.15.77:0/2358796851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc101400 msgr2=0x7ff0dc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2358796851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc101400 0x7ff0dc10d930 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009cd0 tx=0x7ff0d802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- 172.21.15.77:0/2358796851 shutdown_connections 2024-09-15T22:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2358796851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc101400 0x7ff0dc10d930 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2358796851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc100a60 0x7ff0dc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2358796851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc106a60 0x7ff0dc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- 172.21.15.77:0/2358796851 >> 172.21.15.77:0/2358796851 conn(0x7ff0dc0fc780 msgr2=0x7ff0dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- 172.21.15.77:0/2358796851 shutdown_connections 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- 172.21.15.77:0/2358796851 wait complete. 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 Processor -- start 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.826+0000 7ff0e4351640 1 -- start start 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc101400 0x7ff0dc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc106a60 0x7ff0dc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc110e60 con 0x7ff0dc100a60 2024-09-15T22:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc110ce0 con 0x7ff0dc106a60 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e4351640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc110fe0 con 0x7ff0dc101400 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46366/0 (socket says 172.21.15.77:46366) 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2177570486 learned_addr learned my addr 172.21.15.77:0/2177570486 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc101400 0x7ff0dc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc106a60 0x7ff0dc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2177570486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc101400 msgr2=0x7ff0dc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc101400 0x7ff0dc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2177570486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc106a60 msgr2=0x7ff0dc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc106a60 0x7ff0dc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2177570486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc19d3c0 con 0x7ff0dc100a60 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9f0 tx=0x7ff0d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.827+0000 7ff0d37fe640 1 -- 172.21.15.77:0/2177570486 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400cd80 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.828+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc19d6b0 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.828+0000 7ff0d37fe640 1 -- 172.21.15.77:0/2177570486 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d4005500 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.828+0000 7ff0d37fe640 1 -- 172.21.15.77:0/2177570486 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4010690 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.828+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a4f90 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.830+0000 7ff0d37fe640 1 -- 172.21.15.77:0/2177570486 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff0d40046c0 con 0x7ff0dc100a60 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.830+0000 7ff0e672a740 1 Processor -- start 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 -- start start 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0d37fe640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561dcfea72f0 con 0x561dcff415d0 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561dcfe4ecf0 con 0x561dcff4c370 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e672a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561dcfea32b0 con 0x561dcff42b40 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff41a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46382/0 (socket says 172.21.15.77:46382) 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e18ec640 1 -- 172.21.15.77:0/4004204466 learned_addr learned my addr 172.21.15.77:0/4004204466 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0d37fe640 1 -- 172.21.15.77:0/2177570486 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7ff0d40d04c0 con 0x7ff0dc100a60 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.831+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 -- 172.21.15.77:0/4004204466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 msgr2=0x561dcff4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 -- 172.21.15.77:0/4004204466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 msgr2=0x561dcff41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 -- 172.21.15.77:0/4004204466 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dcfe333a0 con 0x561dcff42b40 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 0x7ff0c80a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8004ab0 tx=0x7ff0d8009800 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff41090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40cd820 tx=0x7ff0d406c090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf11209cfea6125 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0d0dea640 1 -- 172.21.15.77:0/4004204466 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d406e030 con 0x561dcff42b40 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0d0dea640 1 -- 172.21.15.77:0/4004204466 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d409cab0 con 0x561dcff42b40 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.832+0000 7ff0d0dea640 1 -- 172.21.15.77:0/4004204466 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4098b80 con 0x561dcff42b40 2024-09-15T22:22:32.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- 172.21.15.77:0/4004204466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 msgr2=0x561dcff41090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff41090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40cd820 tx=0x7ff0d406c090 comp rx=0 tx=0).stop 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- 172.21.15.77:0/4004204466 shutdown_connections 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 --2- 172.21.15.77:0/4004204466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff41090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- 172.21.15.77:0/4004204466 >> 172.21.15.77:0/4004204466 conn(0x561dcff0a310 msgr2=0x561dcff3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- 172.21.15.77:0/4004204466 shutdown_connections 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- 172.21.15.77:0/4004204466 wait complete. 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 -- start start 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.833+0000 7ff0e672a740 1 Processor -- start 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561dcff4f070 con 0x561dcff4c370 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561dcfe62060 con 0x561dcff415d0 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561dcff4f1f0 con 0x561dcff42b40 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 10 client.0 _ll_register_callbacks cb 0x561dcfbb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e28ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33820/0 (socket says 172.21.15.77:33820) 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e28ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46388/0 (socket says 172.21.15.77:46388) 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 -- 172.21.15.77:0/2909974418 learned_addr learned my addr 172.21.15.77:0/2909974418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 -- 172.21.15.77:0/2909974418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 msgr2=0x561dcff4bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff4bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 -- 172.21.15.77:0/2909974418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 msgr2=0x561dcff497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 -- 172.21.15.77:0/2909974418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dcfc183c0 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc012740 tx=0x7ff0cc012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a025652b6f2a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc004050 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7ff0cc0041f0 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.834+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d409cab0 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0cc025050 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 10 client.27010 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561dcff5ab30 con 0x561dcff42b40 2024-09-15T22:22:32.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d409cab0 con 0x561dcff42b40 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff0cc003b20 con 0x561dcff42b40 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 10 client.27010 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 10 client.27010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 20 client.27010 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcff5e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 4 client.27010 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dcfeb4860 con 0x561dcff5bff0 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.835+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcff5e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0c5ffb640 10 client.27010 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcff5e450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=248bfb7732253e2f server_cookie=614ce27ef821e01d in_seq=0 out_seq=0 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2909974418 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dcfeb4860 con 0x561dcff5bff0 2024-09-15T22:22:32.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0c5ffb640 10 client.27010 handle_command_reply: tid=0 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 client.27010 shutdown 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 msgr2=0x561dcff49060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc012740 tx=0x7ff0cc012c00 comp rx=0 tx=0).stop 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 shutdown_connections 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcff5e450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff4c370 0x561dcff497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff49060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0c5ffb640 0 client.27010 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2909974418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff4bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 >> 172.21.15.77:0/2909974418 conn(0x561dcff0a310 msgr2=0x561dcff43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 shutdown_connections 2024-09-15T22:22:32.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.836+0000 7ff0e672a740 1 -- 172.21.15.77:0/2909974418 wait complete. 2024-09-15T22:22:32.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.837+0000 7ff0e672a740 20 client.27010 trim_cache size 0 max 16384 2024-09-15T22:22:32.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.849+0000 7ff0e672a740 1 Processor -- start 2024-09-15T22:22:32.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.849+0000 7ff0e672a740 1 -- start start 2024-09-15T22:22:32.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.849+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff4c370 0x561dcff44120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e672a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1aefe0 con 0x561dcff42b40 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e672a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b0b20 con 0x561dcff415d0 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e672a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1af6b0 con 0x561dcff4c370 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38478/0 (socket says 172.21.15.77:38478) 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 -- 172.21.15.77:0/769044967 learned_addr learned my addr 172.21.15.77:0/769044967 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46402/0 (socket says 172.21.15.77:46402) 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff4c370 0x561dcff44120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 -- 172.21.15.77:0/769044967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff4c370 msgr2=0x561dcff44120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff4c370 0x561dcff44120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 -- 172.21.15.77:0/769044967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 msgr2=0x561dcff48920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 -- 172.21.15.77:0/769044967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dcff58310 con 0x561dcff415d0 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4098810 tx=0x7ff0d4098650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1117b71213b447b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0c7fff640 1 -- 172.21.15.77:0/769044967 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40ba070 con 0x561dcff415d0 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0c7fff640 1 -- 172.21.15.77:0/769044967 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d406e040 con 0x561dcff415d0 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.850+0000 7ff0c7fff640 1 -- 172.21.15.77:0/769044967 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40b4030 con 0x561dcff415d0 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- 172.21.15.77:0/769044967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 msgr2=0x561dcff42300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4098810 tx=0x7ff0d4098650 comp rx=0 tx=0).stop 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- 172.21.15.77:0/769044967 shutdown_connections 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff4c370 0x561dcff44120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff42b40 0x561dcff48920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 --2- 172.21.15.77:0/769044967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff415d0 0x561dcff42300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- 172.21.15.77:0/769044967 >> 172.21.15.77:0/769044967 conn(0x561dcff0a310 msgr2=0x561dcff48150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- 172.21.15.77:0/769044967 shutdown_connections 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- 172.21.15.77:0/769044967 wait complete. 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.851+0000 7ff0e672a740 1 -- start start 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 Processor -- start 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff54c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b0450 con 0x561dcff415d0 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1afd80 con 0x561dcff4c370 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1acfe0 con 0x561dcff42b40 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e672a740 10 client.0 _ll_register_callbacks cb 0x7ff0cc009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46414/0 (socket says 172.21.15.77:46414) 2024-09-15T22:22:32.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e18ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff54c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2548312489 learned_addr learned my addr 172.21.15.77:0/2548312489 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e28ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 msgr2=0x561dcff54c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff54c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2548312489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 msgr2=0x561dcff458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e20ed640 1 -- 172.21.15.77:0/2548312489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dcff46120 con 0x561dcff415d0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.852+0000 7ff0e28ee640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.853+0000 7ff0e20ed640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40a4af0 tx=0x7ff0d409bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d31227cf51fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.853+0000 7ff0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40b4070 con 0x561dcff415d0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7ff0d4098a20 con 0x561dcff415d0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d406e040 con 0x561dcff415d0 2024-09-15T22:22:32.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0d40b0050 con 0x561dcff415d0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0e672a740 10 client.18036 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561dcffc6830 con 0x561dcff415d0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.855+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d406e040 con 0x561dcff415d0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff0d40a4b90 con 0x561dcff415d0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 10 client.18036 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 10 client.18036 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 20 client.18036 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcffc8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 4 client.18036 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dcfeaed70 con 0x561dcff5bff0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcffc8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0e18ec640 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcffc8a60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=392133ad010b34e8 server_cookie=b17cdfbf349a2d75 in_seq=0 out_seq=0 2024-09-15T22:22:32.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0c5ffb640 10 client.18036 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0c5ffb640 1 -- 172.21.15.77:0/2548312489 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dcfeaed70 con 0x561dcff5bff0 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.856+0000 7ff0c5ffb640 10 client.18036 handle_command_reply: tid=0 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 client.18036 shutdown 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 msgr2=0x561dcff54710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40a4af0 tx=0x7ff0d409bd90 comp rx=0 tx=0).stop 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 shutdown_connections 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561dcff5bff0 0x561dcffc8a60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dcff4c370 0x561dcff458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561dcff42b40 0x561dcff54c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 --2- 172.21.15.77:0/2548312489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561dcff415d0 0x561dcff54710 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0c5ffb640 0 client.18036 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 >> 172.21.15.77:0/2548312489 conn(0x561dcff0a310 msgr2=0x561dcff4a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 shutdown_connections 2024-09-15T22:22:32.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 1 -- 172.21.15.77:0/2548312489 wait complete. 2024-09-15T22:22:32.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.857+0000 7ff0e672a740 20 client.18036 trim_cache size 0 max 16384 2024-09-15T22:22:32.867 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 msgr2=0x7ff0c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 0x7ff0c80a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8004ab0 tx=0x7ff0d8009800 comp rx=0 tx=0).stop 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 msgr2=0x7ff0dc198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9f0 tx=0x7ff0d400edf0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 shutdown_connections 2024-09-15T22:22:32.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0dc106a60 0x7ff0dc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0dc101400 0x7ff0dc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 --2- 172.21.15.77:0/2177570486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0dc100a60 0x7ff0dc198410 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 >> 172.21.15.77:0/2177570486 conn(0x7ff0dc0fc780 msgr2=0x7ff0dc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 shutdown_connections 2024-09-15T22:22:32.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:32.858+0000 7ff0e4351640 1 -- 172.21.15.77:0/2177570486 wait complete. 2024-09-15T22:22:33.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 768476 7.49 MB/sec execute 402 sec latency 40.447 ms 2024-09-15T22:22:33.688 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:22:33.951 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:34.072 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:33 smithi077 ceph-mon[30980]: pgmap v315: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:34.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:33 smithi117 ceph-mon[35021]: pgmap v315: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:33 smithi196 ceph-mon[35109]: pgmap v315: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 -- 172.21.15.77:0/2455361945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5304101df0 msgr2=0x7f53041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 --2- 172.21.15.77:0/2455361945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5304101df0 0x7f53041021d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009900 tx=0x7f52f402a410 comp rx=0 tx=0).stop 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 -- 172.21.15.77:0/2455361945 shutdown_connections 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 --2- 172.21.15.77:0/2455361945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 0x7f530410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 --2- 172.21.15.77:0/2455361945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53041027a0 0x7f53041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 --2- 172.21.15.77:0/2455361945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5304101df0 0x7f53041021d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 -- 172.21.15.77:0/2455361945 >> 172.21.15.77:0/2455361945 conn(0x7f53040fd8f0 msgr2=0x7f53040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 -- 172.21.15.77:0/2455361945 shutdown_connections 2024-09-15T22:22:34.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 -- 172.21.15.77:0/2455361945 wait complete. 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.467+0000 7f5309f67640 1 Processor -- start 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 -- start start 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5304101df0 0x7f53040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 0x7f5304074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5304111f80 con 0x7f53041027a0 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5304111e00 con 0x7f5304101df0 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5304112100 con 0x7f5304106f20 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46436/0 (socket says 172.21.15.77:46436) 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 -- 172.21.15.77:0/2246134344 learned_addr learned my addr 172.21.15.77:0/2246134344 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f53037fe640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5304101df0 0x7f53040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5303fff640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 0x7f5304074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 -- 172.21.15.77:0/2246134344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 msgr2=0x7f5304074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 0x7f5304074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 -- 172.21.15.77:0/2246134344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5304101df0 msgr2=0x7f53040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5304101df0 0x7f53040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 -- 172.21.15.77:0/2246134344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53041a6960 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5302ffd640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f52f800e9f0 tx=0x7f52f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5300ff9640 1 -- 172.21.15.77:0/2246134344 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f800cd80 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.468+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a6c50 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.469+0000 7f5300ff9640 1 -- 172.21.15.77:0/2246134344 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f8010910 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.469+0000 7f5300ff9640 1 -- 172.21.15.77:0/2246134344 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f80188f0 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.469+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a7190 con 0x7f53041027a0 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.471+0000 7f530c340740 1 Processor -- start 2024-09-15T22:22:34.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 -- start start 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5300ff9640 1 -- 172.21.15.77:0/2246134344 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f52f8010ac0 con 0x7f53041027a0 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee9943a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee9950760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dee98a92f0 con 0x55dee994e370 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dee9850cf0 con 0x55dee9944b40 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f530c340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dee98a52b0 con 0x55dee99435d0 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee9950760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38526/0 (socket says 172.21.15.77:38526) 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f53037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee9943a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 -- 172.21.15.77:0/653293543 learned_addr learned my addr 172.21.15.77:0/653293543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:34.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 -- 172.21.15.77:0/653293543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 msgr2=0x55dee9943a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee9943a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 -- 172.21.15.77:0/653293543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 msgr2=0x55dee9950760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee9950760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 -- 172.21.15.77:0/653293543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee98353a0 con 0x55dee9944b40 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.472+0000 7f5303fff640 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009f50 tx=0x7f52fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e8aca84d9eee18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f5300ff9640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 0x7f52ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f52e9ffb640 1 -- 172.21.15.77:0/653293543 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc004d20 con 0x55dee9944b40 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f53037fe640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 0x7f52ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f52e9ffb640 1 -- 172.21.15.77:0/653293543 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc004050 con 0x55dee9944b40 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f52e9ffb640 1 -- 172.21.15.77:0/653293543 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc005280 con 0x55dee9944b40 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f5300ff9640 1 -- 172.21.15.77:0/2246134344 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f52f80d0420 con 0x7f53041027a0 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f530c340740 1 -- 172.21.15.77:0/653293543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 msgr2=0x55dee9943090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f530c340740 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009f50 tx=0x7f52fc0118d0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.473+0000 7f53037fe640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 0x7f52ec0a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009230 tx=0x7f52f40058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- 172.21.15.77:0/653293543 shutdown_connections 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee9950760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee9943a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- 172.21.15.77:0/653293543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9943090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- 172.21.15.77:0/653293543 >> 172.21.15.77:0/653293543 conn(0x55dee990c310 msgr2=0x55dee993fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- 172.21.15.77:0/653293543 shutdown_connections 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- 172.21.15.77:0/653293543 wait complete. 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- start start 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 Processor -- start 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 0x55dee994afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 0x55dee994b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.474+0000 7f530c340740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dee9951070 con 0x55dee99435d0 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f530c340740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dee9864060 con 0x55dee994e370 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f530c340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dee99511f0 con 0x55dee9944b40 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f530c340740 10 client.0 _ll_register_callbacks cb 0x55dee95b78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46450/0 (socket says 172.21.15.77:46450) 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 -- 172.21.15.77:0/644605125 learned_addr learned my addr 172.21.15.77:0/644605125 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f5302ffd640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 0x55dee994afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f5303fff640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 0x55dee994b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 -- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 msgr2=0x55dee994afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 0x55dee994afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 -- 172.21.15.77:0/644605125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 msgr2=0x55dee994b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 0x55dee994b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 -- 172.21.15.77:0/644605125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee961a3c0 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f5302ffd640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 0x55dee994afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f5303fff640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 0x55dee994b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f53037fe640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009900 tx=0x7f52f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e406c89965e9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f52e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f404e070 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f52f40037f0 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.475+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc004050 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f52f405a050 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 10 client.18048 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dee995cac0 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc004050 con 0x55dee99435d0 2024-09-15T22:22:34.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f52f404ab50 con 0x55dee99435d0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 10 client.18048 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 10 client.18048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 20 client.18048 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee9960330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 4 client.18048 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee98b6860 con 0x55dee995ded0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.476+0000 7f5302ffd640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee9960330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.477+0000 7f5302ffd640 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee9960330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a6194b36f4e2948 server_cookie=6b5fbd8f07aeaeb8 in_seq=0 out_seq=0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.477+0000 7f52e2ffd640 10 client.18048 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f52e2ffd640 1 -- 172.21.15.77:0/644605125 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dee98b6860 con 0x55dee995ded0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f52e2ffd640 10 client.18048 handle_command_reply: tid=0 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 client.18048 shutdown 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 msgr2=0x55dee994dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009900 tx=0x7f52f4009260 comp rx=0 tx=0).stop 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 shutdown_connections 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee9960330 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee994e370 0x55dee994b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f52e2ffd640 0 client.18048 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee9944b40 0x55dee994afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 --2- 172.21.15.77:0/644605125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee99435d0 0x55dee994dd20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 >> 172.21.15.77:0/644605125 conn(0x55dee990c310 msgr2=0x55dee9945960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 shutdown_connections 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 1 -- 172.21.15.77:0/644605125 wait complete. 2024-09-15T22:22:34.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.478+0000 7f530c340740 20 client.18048 trim_cache size 0 max 16384 2024-09-15T22:22:34.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 770619 7.50 MB/sec execute 403 sec latency 27.703 ms 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.494+0000 7f530c340740 1 Processor -- start 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.494+0000 7f530c340740 1 -- start start 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee995a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee994a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b11c0 con 0x55dee994e370 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b2d00 con 0x55dee9944b40 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f530c340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b1890 con 0x55dee99435d0 2024-09-15T22:22:34.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f53037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee995a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5303fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee994a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38548/0 (socket says 172.21.15.77:38548) 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f53037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee995a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33876/0 (socket says 172.21.15.77:33876) 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 -- 172.21.15.77:0/1257824465 learned_addr learned my addr 172.21.15.77:0/1257824465 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 -- 172.21.15.77:0/1257824465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 msgr2=0x55dee995a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee995a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 -- 172.21.15.77:0/1257824465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 msgr2=0x55dee994a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee994a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 -- 172.21.15.77:0/1257824465 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee9946120 con 0x55dee9944b40 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5303fff640 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee994a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f5302ffd640 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f52f809c120 tx=0x7f52f80d2930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6911043016a3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.495+0000 7f52e97fa640 1 -- 172.21.15.77:0/1257824465 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f809c420 con 0x55dee9944b40 2024-09-15T22:22:34.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f52e97fa640 1 -- 172.21.15.77:0/1257824465 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f80a0720 con 0x55dee9944b40 2024-09-15T22:22:34.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f52e97fa640 1 -- 172.21.15.77:0/1257824465 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8018b50 con 0x55dee9944b40 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 -- 172.21.15.77:0/1257824465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 msgr2=0x55dee9944300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f52f809c120 tx=0x7f52f80d2930 comp rx=0 tx=0).stop 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 -- 172.21.15.77:0/1257824465 shutdown_connections 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee994e370 0x55dee994a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee9944b40 0x55dee9944300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 --2- 172.21.15.77:0/1257824465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee99435d0 0x55dee995a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 -- 172.21.15.77:0/1257824465 >> 172.21.15.77:0/1257824465 conn(0x55dee990c310 msgr2=0x55dee99498a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 -- 172.21.15.77:0/1257824465 shutdown_connections 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.496+0000 7f530c340740 1 -- 172.21.15.77:0/1257824465 wait complete. 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 -- start start 2024-09-15T22:22:34.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 Processor -- start 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 0x55dee9956ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee994e370 0x55dee9947830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b2630 con 0x55dee9944b40 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b1f60 con 0x55dee99435d0 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53041af1c0 con 0x55dee994e370 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f530c340740 10 client.0 _ll_register_callbacks cb 0x7f52f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f53037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f5302ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 0x55dee9956ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f53037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38554/0 (socket says 172.21.15.77:38554) 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f5303fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee994e370 0x55dee9947830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.497+0000 7f53037fe640 1 -- 172.21.15.77:0/3452054984 learned_addr learned my addr 172.21.15.77:0/3452054984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 -- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee994e370 msgr2=0x55dee9947830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee994e370 0x55dee9947830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 -- 172.21.15.77:0/3452054984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 msgr2=0x55dee9956660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 -- 172.21.15.77:0/3452054984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee9948060 con 0x55dee9944b40 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f53037fe640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f5302ffd640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 0x55dee9956ba0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f52f809c120 tx=0x7f52f800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a798b74d98198d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f52e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f80a4fe0 con 0x55dee9944b40 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f52f80d00d0 con 0x55dee9944b40 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f80a0720 con 0x55dee9944b40 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f52f809c420 con 0x55dee9944b40 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f530c340740 10 client.18054 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dee99c8660 con 0x55dee9944b40 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.498+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f80a0720 con 0x55dee9944b40 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f52f809b9f0 con 0x55dee9944b40 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 10 client.18054 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 10 client.18054 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 20 client.18054 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee99cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 4 client.18054 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dee98b0d70 con 0x55dee995ded0 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f53037fe640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee99cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f52e2ffd640 10 client.18054 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.499+0000 7f53037fe640 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee99cace0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28bd487222c9f40a server_cookie=bba9d45d26c60937 in_seq=0 out_seq=0 2024-09-15T22:22:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f52e2ffd640 1 -- 172.21.15.77:0/3452054984 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dee98b0d70 con 0x55dee995ded0 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f52e2ffd640 10 client.18054 handle_command_reply: tid=0 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 client.18054 shutdown 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 msgr2=0x55dee9956ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 0x55dee9956ba0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f52f809c120 tx=0x7f52f800cf20 comp rx=0 tx=0).stop 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 shutdown_connections 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55dee995ded0 0x55dee99cace0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dee994e370 0x55dee9947830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dee9944b40 0x55dee9956ba0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 --2- 172.21.15.77:0/3452054984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dee99435d0 0x55dee9956660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.500+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 >> 172.21.15.77:0/3452054984 conn(0x55dee990c310 msgr2=0x55dee9945930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f52e2ffd640 0 client.18054 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:22:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 shutdown_connections 2024-09-15T22:22:34.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f530c340740 1 -- 172.21.15.77:0/3452054984 wait complete. 2024-09-15T22:22:34.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f530c340740 20 client.18054 trim_cache size 0 max 16384 2024-09-15T22:22:34.510 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 msgr2=0x7f52ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 0x7f52ec0a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009230 tx=0x7f52f40058d0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 msgr2=0x7f5304074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f52f800e9f0 tx=0x7f52f800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 shutdown_connections 2024-09-15T22:22:34.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52ec0a33b0 0x7f52ec0a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304106f20 0x7f5304074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53041027a0 0x7f5304074a30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 --2- 172.21.15.77:0/2246134344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5304101df0 0x7f53040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 >> 172.21.15.77:0/2246134344 conn(0x7f53040fd8f0 msgr2=0x7f53040fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 shutdown_connections 2024-09-15T22:22:34.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:34.501+0000 7f5309f67640 1 -- 172.21.15.77:0/2246134344 wait complete. 2024-09-15T22:22:35.237 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:22:35.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 771992 7.50 MB/sec execute 404 sec latency 25.080 ms 2024-09-15T22:22:35.492 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:35.936 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:35 smithi077 ceph-mon[30980]: pgmap v316: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 -- 172.21.15.77:0/3881415075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 msgr2=0x7f66981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 --2- 172.21.15.77:0/3881415075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f66981029b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6688009900 tx=0x7f668802a480 comp rx=0 tx=0).stop 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 -- 172.21.15.77:0/3881415075 shutdown_connections 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 --2- 172.21.15.77:0/3881415075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 0x7f669810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 --2- 172.21.15.77:0/3881415075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f66981029b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 --2- 172.21.15.77:0/3881415075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f6698101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 -- 172.21.15.77:0/3881415075 >> 172.21.15.77:0/3881415075 conn(0x7f66980fc7d0 msgr2=0x7f66980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 -- 172.21.15.77:0/3881415075 shutdown_connections 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.025+0000 7f66a0334640 1 -- 172.21.15.77:0/3881415075 wait complete. 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 Processor -- start 2024-09-15T22:22:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 -- start start 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f66981962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 0x7f669819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f669810fe80 con 0x7f6698102570 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f669810fd00 con 0x7f6698100e20 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.026+0000 7f66a0334640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6698110000 con 0x7f6698102ef0 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f66981962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46462/0 (socket says 172.21.15.77:46462) 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 -- 172.21.15.77:0/2209324317 learned_addr learned my addr 172.21.15.77:0/2209324317 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669e8d1640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 0x7f669819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 -- 172.21.15.77:0/2209324317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 msgr2=0x7f669819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 0x7f669819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 -- 172.21.15.77:0/2209324317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 msgr2=0x7f66981962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f66981962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 -- 172.21.15.77:0/2209324317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819b290 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669e0d0640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f66981962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f669d8cf640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f6688003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f668f7fe640 1 -- 172.21.15.77:0/2209324317 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668803f070 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f668f7fe640 1 -- 172.21.15.77:0/2209324317 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688031c30 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f668f7fe640 1 -- 172.21.15.77:0/2209324317 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688035030 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819b520 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.027+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a2e60 con 0x7f6698102570 2024-09-15T22:22:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.030+0000 7f668f7fe640 1 -- 172.21.15.77:0/2209324317 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6688042070 con 0x7f6698102570 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.030+0000 7f66a270d740 1 Processor -- start 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.030+0000 7f66a270d740 1 -- start start 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 0x5636fdc79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636fdbdf2f0 con 0x5636fdc84370 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636fdb86cf0 con 0x5636fdc7ab40 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f66a270d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636fdbdb2b0 con 0x5636fdc795d0 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f668f7fe640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 0x7f6684003c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 0x5636fdc79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38590/0 (socket says 172.21.15.77:38590) 2024-09-15T22:22:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 -- 172.21.15.77:0/1624350710 learned_addr learned my addr 172.21.15.77:0/1624350710 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 0x7f6684003c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 -- 172.21.15.77:0/1624350710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 msgr2=0x5636fdc79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 0x5636fdc79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 -- 172.21.15.77:0/1624350710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 msgr2=0x5636fdc86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f668f7fe640 1 -- 172.21.15.77:0/2209324317 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f66880f0710 con 0x7f6698102570 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 -- 172.21.15.77:0/1624350710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636fdb6b3a0 con 0x5636fdc7ab40 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 0x5636fdc79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.031+0000 7f669e8d1640 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66900136e0 tx=0x7f6690013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4456a7f298fef04b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f668cdea640 1 -- 172.21.15.77:0/1624350710 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000fcc0 con 0x5636fdc7ab40 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f668cdea640 1 -- 172.21.15.77:0/1624350710 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690010430 con 0x5636fdc7ab40 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f668cdea640 1 -- 172.21.15.77:0/1624350710 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000fe60 con 0x5636fdc7ab40 2024-09-15T22:22:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f66a270d740 1 -- 172.21.15.77:0/1624350710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 msgr2=0x5636fdc79090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f669e8d1640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 0x7f6684003c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6690022000 tx=0x7f669001e030 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.032+0000 7f66a270d740 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66900136e0 tx=0x7f6690013ba0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 -- 172.21.15.77:0/1624350710 shutdown_connections 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc795d0 0x5636fdc79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 --2- 172.21.15.77:0/1624350710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc79090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 -- 172.21.15.77:0/1624350710 >> 172.21.15.77:0/1624350710 conn(0x5636fdc42310 msgr2=0x5636fdc75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 -- 172.21.15.77:0/1624350710 shutdown_connections 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 -- 172.21.15.77:0/1624350710 wait complete. 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 -- start start 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 Processor -- start 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.033+0000 7f66a270d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc83d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdc81530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636fdc8b480 con 0x5636fdc795d0 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5636fdb9a060 con 0x5636fdc7ab40 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5636fdc8b600 con 0x5636fdc84370 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc83d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdc81530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdc81530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33918/0 (socket says 172.21.15.77:33918) 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669e8d1640 1 -- 172.21.15.77:0/1003756093 learned_addr learned my addr 172.21.15.77:0/1003756093 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38594/0 (socket says 172.21.15.77:38594) 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc83d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46488/0 (socket says 172.21.15.77:46488) 2024-09-15T22:22:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 10 client.0 _ll_register_callbacks cb 0x5636fd8ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 -- 172.21.15.77:0/1003756093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 msgr2=0x5636fdc81530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdc81530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 -- 172.21.15.77:0/1003756093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 msgr2=0x5636fdc83d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc83d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 -- 172.21.15.77:0/1003756093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636fd9503c0 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f66880edad0 tx=0x7f6688009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466eed979fa68323 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f6681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688049050 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6690010430 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.034+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f66880b8d80 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f66880bc710 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 10 client.27116 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636fdc96e70 con 0x5636fdc7ab40 2024-09-15T22:22:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690010430 con 0x5636fdc7ab40 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f66880f0360 con 0x5636fdc7ab40 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 10 client.27116 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 10 client.27116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 20 client.27116 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdc98210 0x5636fdc9a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 4 client.27116 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636fdbec860 con 0x5636fdc98210 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.035+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdc98210 0x5636fdc9a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdc98210 0x5636fdc9a640 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=281f10457e4ef10b server_cookie=f0e9ba853e0894e in_seq=0 out_seq=0 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f6681ffb640 10 client.27116 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f6681ffb640 1 -- 172.21.15.77:0/1003756093 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636fdbec860 con 0x5636fdc98210 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f6681ffb640 10 client.27116 handle_command_reply: tid=0 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 client.27116 shutdown 2024-09-15T22:22:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 msgr2=0x5636fdc80fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f66880edad0 tx=0x7f6688009680 comp rx=0 tx=0).stop 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 shutdown_connections 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdc98210 0x5636fdc9a640 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdc81530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc80fb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f6681ffb640 0 client.27116 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 --2- 172.21.15.77:0/1003756093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc83d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 >> 172.21.15.77:0/1003756093 conn(0x5636fdc42310 msgr2=0x5636fdc7b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 shutdown_connections 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.036+0000 7f66a270d740 1 -- 172.21.15.77:0/1003756093 wait complete. 2024-09-15T22:22:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.037+0000 7f66a270d740 20 client.27116 trim_cache size 0 max 16384 2024-09-15T22:22:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 Processor -- start 2024-09-15T22:22:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 -- start start 2024-09-15T22:22:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 0x5636fdc94720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66981acc70 con 0x5636fdc84370 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ae7b0 con 0x5636fdc795d0 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f66a270d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ad340 con 0x5636fdc7ab40 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 0x5636fdc94720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 0x5636fdc94720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38606/0 (socket says 172.21.15.77:38606) 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669e8d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46496/0 (socket says 172.21.15.77:46496) 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33924/0 (socket says 172.21.15.77:33924) 2024-09-15T22:22:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.051+0000 7f669e0d0640 1 -- 172.21.15.77:0/3318468029 learned_addr learned my addr 172.21.15.77:0/3318468029 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 -- 172.21.15.77:0/3318468029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 msgr2=0x5636fdc94720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 0x5636fdc94720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 -- 172.21.15.77:0/3318468029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 msgr2=0x5636fdc7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 -- 172.21.15.77:0/3318468029 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636fdc7f500 con 0x5636fdc7ab40 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669e8d1640 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f669d8cf640 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5636fdc82560 tx=0x7f66880f0ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dbf3e8b364b400e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f6683fff640 1 -- 172.21.15.77:0/3318468029 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688049070 con 0x5636fdc7ab40 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f6683fff640 1 -- 172.21.15.77:0/3318468029 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66880c0a10 con 0x5636fdc7ab40 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.052+0000 7f6683fff640 1 -- 172.21.15.77:0/3318468029 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66880edd50 con 0x5636fdc7ab40 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- 172.21.15.77:0/3318468029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 msgr2=0x5636fdc7c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5636fdc82560 tx=0x7f66880f0ac0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- 172.21.15.77:0/3318468029 shutdown_connections 2024-09-15T22:22:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc84370 0x5636fdc7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc7ab40 0x5636fdc7c120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 --2- 172.21.15.77:0/3318468029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc795d0 0x5636fdc94720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- 172.21.15.77:0/3318468029 >> 172.21.15.77:0/3318468029 conn(0x5636fdc42310 msgr2=0x5636fdc800a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- 172.21.15.77:0/3318468029 shutdown_connections 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- 172.21.15.77:0/3318468029 wait complete. 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 -- start start 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.053+0000 7f66a270d740 1 Processor -- start 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc8cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdd01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ae0e0 con 0x5636fdc795d0 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ada10 con 0x5636fdc7ab40 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66981aac70 con 0x5636fdc84370 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f66a270d740 10 client.0 _ll_register_callbacks cb 0x7f66880f0a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46504/0 (socket says 172.21.15.77:46504) 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669d8cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc8cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 -- 172.21.15.77:0/1120053589 learned_addr learned my addr 172.21.15.77:0/1120053589 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e8d1640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdd01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 -- 172.21.15.77:0/1120053589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 msgr2=0x5636fdd01ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdd01ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 -- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 msgr2=0x5636fdc8cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc8cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 -- 172.21.15.77:0/1120053589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636fdc8d7d0 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc8cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f669e0d0640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x5636fdc9ae80 tx=0x7f6694012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80d7f3d1c05c89d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.054+0000 7f6681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402b060 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f6694010d10 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66880c0a10 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6694032050 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 10 client.18072 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5636fdd03590 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66880c0a10 con 0x5636fdc795d0 2024-09-15T22:22:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6694003b20 con 0x5636fdc795d0 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 10 client.18072 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 10 client.18072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 20 client.18072 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdd03c60 0x5636fdd06080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 4 client.18072 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636fdbe6d70 con 0x5636fdd03c60 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.055+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdd03c60 0x5636fdd06080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f6681ffb640 10 client.18072 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f669d8cf640 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdd03c60 0x5636fdd06080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=338a5c82b3567f09 server_cookie=c5270e30f99087e2 in_seq=0 out_seq=0 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f6681ffb640 1 -- 172.21.15.77:0/1120053589 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636fdbe6d70 con 0x5636fdd03c60 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f6681ffb640 10 client.18072 handle_command_reply: tid=0 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 client.18072 shutdown 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 msgr2=0x5636fdc8c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x5636fdc9ae80 tx=0x7f6694012b80 comp rx=0 tx=0).stop 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 shutdown_connections 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5636fdd03c60 0x5636fdd06080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636fdc84370 0x5636fdd01ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636fdc7ab40 0x5636fdc8cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 --2- 172.21.15.77:0/1120053589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fdc795d0 0x5636fdc8c9b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f6681ffb640 0 client.18072 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 >> 172.21.15.77:0/1120053589 conn(0x5636fdc42310 msgr2=0x5636fdc744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 shutdown_connections 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 1 -- 172.21.15.77:0/1120053589 wait complete. 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.056+0000 7f66a270d740 20 client.18072 trim_cache size 0 max 16384 2024-09-15T22:22:36.067 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:22:36.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 msgr2=0x7f6684003c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 0x7f6684003c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6690022000 tx=0x7f669001e030 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 msgr2=0x7f6698196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f6688003c10 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 shutdown_connections 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66840037e0 0x7f6684003c90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6698102ef0 0x7f669819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6698102570 0x7f6698196820 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 --2- 172.21.15.77:0/2209324317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6698100e20 0x7f66981962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 >> 172.21.15.77:0/2209324317 conn(0x7f66980fc7d0 msgr2=0x7f66981068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 shutdown_connections 2024-09-15T22:22:36.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:36.057+0000 7f66a0334640 1 -- 172.21.15.77:0/2209324317 wait complete. 2024-09-15T22:22:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:35 smithi117 ceph-mon[35021]: pgmap v316: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:36.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:35 smithi196 ceph-mon[35109]: pgmap v316: 129 pgs: 129 active+clean; 486 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-15T22:22:36.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 773677 7.50 MB/sec execute 405 sec latency 39.150 ms 2024-09-15T22:22:36.896 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:22:37.157 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:37.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 774858 7.50 MB/sec execute 406 sec latency 42.212 ms 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 -- 172.21.15.77:0/648957724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28781068d0 msgr2=0x7f2878106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 --2- 172.21.15.77:0/648957724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28781068d0 0x7f2878106cb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2868009930 tx=0x7f286802a420 comp rx=0 tx=0).stop 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 -- 172.21.15.77:0/648957724 shutdown_connections 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 --2- 172.21.15.77:0/648957724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2878101270 0x7f287810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 --2- 172.21.15.77:0/648957724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781008d0 0x7f2878100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 --2- 172.21.15.77:0/648957724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28781068d0 0x7f2878106cb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 -- 172.21.15.77:0/648957724 >> 172.21.15.77:0/648957724 conn(0x7f28780fc610 msgr2=0x7f28780fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 -- 172.21.15.77:0/648957724 shutdown_connections 2024-09-15T22:22:37.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.663+0000 7f287cf9d640 1 -- 172.21.15.77:0/648957724 wait complete. 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 Processor -- start 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 -- start start 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 0x7f2878198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781068d0 0x7f287819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2878110d20 con 0x7f2878101270 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2878110ba0 con 0x7f28781008d0 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f287cf9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2878110ea0 con 0x7f28781068d0 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46522/0 (socket says 172.21.15.77:46522) 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f28777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 0x7f2878198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f2876ffd640 1 -- 172.21.15.77:0/3961545112 learned_addr learned my addr 172.21.15.77:0/3961545112 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:37.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.664+0000 7f2877fff640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781068d0 0x7f287819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 -- 172.21.15.77:0/3961545112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781068d0 msgr2=0x7f287819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781068d0 0x7f287819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 -- 172.21.15.77:0/3961545112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 msgr2=0x7f2878198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 0x7f2878198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 -- 172.21.15.77:0/3961545112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287819d2c0 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f28777fe640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 0x7f2878198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f286c00e9f0 tx=0x7f286c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2874ff9640 1 -- 172.21.15.77:0/3961545112 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c00cd80 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287819d5b0 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2874ff9640 1 -- 172.21.15.77:0/3961545112 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c010910 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f2874ff9640 1 -- 172.21.15.77:0/3961545112 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0188f0 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.665+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781a4e90 con 0x7f2878101270 2024-09-15T22:22:37.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.667+0000 7f2874ff9640 1 -- 172.21.15.77:0/3961545112 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f286c010430 con 0x7f2878101270 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 Processor -- start 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 -- start start 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f731270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f73c550 0x562e4f73e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e4f6972f0 con 0x562e4f73c550 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e4f63ecf0 con 0x562e4f732d20 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28805d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e4f6932b0 con 0x562e4f7317b0 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f2874ff9640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 0x7f28600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.668+0000 7f28777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2877fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f731270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:33974/0 (socket says 172.21.15.77:33974) 2024-09-15T22:22:37.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 -- 172.21.15.77:0/2439361621 learned_addr learned my addr 172.21.15.77:0/2439361621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2876ffd640 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f73c550 0x562e4f73e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2877fff640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 0x7f28600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2874ff9640 1 -- 172.21.15.77:0/3961545112 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f286c0d04c0 con 0x7f2878101270 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 -- 172.21.15.77:0/2439361621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 msgr2=0x562e4f731270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f731270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 -- 172.21.15.77:0/2439361621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f73c550 msgr2=0x562e4f73e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f73c550 0x562e4f73e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 -- 172.21.15.77:0/2439361621 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4f6233a0 con 0x562e4f7317b0 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2877fff640 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f731270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f2877fff640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 0x7f28600a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2870005fd0 tx=0x7f2870009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.669+0000 7f28777fe640 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2868009930 tx=0x7f2868000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5c7448963b58e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f285dffb640 1 -- 172.21.15.77:0/2439361621 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868035040 con 0x562e4f7317b0 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f285dffb640 1 -- 172.21.15.77:0/2439361621 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2868004760 con 0x562e4f7317b0 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f285dffb640 1 -- 172.21.15.77:0/2439361621 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28680038e0 con 0x562e4f7317b0 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 -- 172.21.15.77:0/2439361621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 msgr2=0x562e4f731bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2868009930 tx=0x7f2868000bc0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 -- 172.21.15.77:0/2439361621 shutdown_connections 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f73c550 0x562e4f73e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f731bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 --2- 172.21.15.77:0/2439361621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f731270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.670+0000 7f28805d8740 1 -- 172.21.15.77:0/2439361621 >> 172.21.15.77:0/2439361621 conn(0x562e4f6fa310 msgr2=0x562e4f72de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- 172.21.15.77:0/2439361621 shutdown_connections 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- 172.21.15.77:0/2439361621 wait complete. 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- start start 2024-09-15T22:22:37.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 Processor -- start 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f739190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f739710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e4f73f1f0 con 0x562e4f7317b0 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e4f652060 con 0x562e4f732d20 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e4f73f370 con 0x562e4f73c550 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28805d8740 10 client.0 _ll_register_callbacks cb 0x562e4f3a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28777fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28777fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46528/0 (socket says 172.21.15.77:46528) 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f28777fe640 1 -- 172.21.15.77:0/685235340 learned_addr learned my addr 172.21.15.77:0/685235340 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.671+0000 7f2877fff640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f739710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2876ffd640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f739190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 -- 172.21.15.77:0/685235340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 msgr2=0x562e4f739710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f739710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 -- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 msgr2=0x562e4f739190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f739190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 -- 172.21.15.77:0/685235340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4f4083c0 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2876ffd640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f739190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28777fe640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f286802a940 tx=0x7f2868004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda04a10d63eeb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868053060 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2868004760 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f2868005130 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f286805a050 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28805d8740 10 client.18084 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e4f74ac20 con 0x562e4f7317b0 2024-09-15T22:22:37.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.672+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2868004760 con 0x562e4f7317b0 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28680315b0 con 0x562e4f7317b0 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 10 client.18084 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 20 client.18084 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f74bf70 0x562e4f74e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 4 client.18084 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e4f6a4860 con 0x562e4f74bf70 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f2876ffd640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f74bf70 0x562e4f74e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f2856ffd640 10 client.18084 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.673+0000 7f2876ffd640 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f74bf70 0x562e4f74e360 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5314da0b2f9c03ff server_cookie=44409ab549d8a3ed in_seq=0 out_seq=0 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f2856ffd640 1 -- 172.21.15.77:0/685235340 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e4f6a4860 con 0x562e4f74bf70 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f2856ffd640 10 client.18084 handle_command_reply: tid=0 2024-09-15T22:22:37.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 client.18084 shutdown 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 msgr2=0x562e4f73bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f286802a940 tx=0x7f2868004b40 comp rx=0 tx=0).stop 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 shutdown_connections 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f74bf70 0x562e4f74e360 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f739710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f732d20 0x562e4f739190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f2856ffd640 0 client.18084 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 --2- 172.21.15.77:0/685235340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f7317b0 0x562e4f73bee0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 >> 172.21.15.77:0/685235340 conn(0x562e4f6fa310 msgr2=0x562e4f733b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 shutdown_connections 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.674+0000 7f28805d8740 1 -- 172.21.15.77:0/685235340 wait complete. 2024-09-15T22:22:37.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.675+0000 7f28805d8740 20 client.18084 trim_cache size 0 max 16384 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.689+0000 7f28805d8740 1 Processor -- start 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.689+0000 7f28805d8740 1 -- start start 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 0x562e4f748490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f73de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28781aee00 con 0x562e4f732d20 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28781b0940 con 0x562e4f7317b0 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28805d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28781af4d0 con 0x562e4f73c550 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46540/0 (socket says 172.21.15.77:46540) 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 -- 172.21.15.77:0/80002253 learned_addr learned my addr 172.21.15.77:0/80002253 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28777fe640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 0x562e4f748490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2877fff640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f73de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 -- 172.21.15.77:0/80002253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 msgr2=0x562e4f73de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f73de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 -- 172.21.15.77:0/80002253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 msgr2=0x562e4f748490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 0x562e4f748490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 -- 172.21.15.77:0/80002253 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e4f72eb40 con 0x562e4f732d20 2024-09-15T22:22:37.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f28777fe640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 0x562e4f748490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f2876ffd640 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c6a0 tx=0x7f286c098830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a9a21eb9012320 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.690+0000 7f285d7fa640 1 -- 172.21.15.77:0/80002253 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c026040 con 0x562e4f732d20 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.691+0000 7f285d7fa640 1 -- 172.21.15.77:0/80002253 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c0a5020 con 0x562e4f732d20 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.691+0000 7f285d7fa640 1 -- 172.21.15.77:0/80002253 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0d0280 con 0x562e4f732d20 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.691+0000 7f28805d8740 1 -- 172.21.15.77:0/80002253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 msgr2=0x562e4f73d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.691+0000 7f28805d8740 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c6a0 tx=0x7f286c098830 comp rx=0 tx=0).stop 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 -- 172.21.15.77:0/80002253 shutdown_connections 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f73c550 0x562e4f73de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f73d940 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 --2- 172.21.15.77:0/80002253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f7317b0 0x562e4f748490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 -- 172.21.15.77:0/80002253 >> 172.21.15.77:0/80002253 conn(0x562e4f6fa310 msgr2=0x562e4f73b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 -- 172.21.15.77:0/80002253 shutdown_connections 2024-09-15T22:22:37.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 -- 172.21.15.77:0/80002253 wait complete. 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 -- start start 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.692+0000 7f28805d8740 1 Processor -- start 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 0x562e4f753530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28781b0270 con 0x562e4f732d20 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28781afba0 con 0x562e4f73c550 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28781ace00 con 0x562e4f7317b0 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28805d8740 10 client.0 _ll_register_callbacks cb 0x7f286c098700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46544/0 (socket says 172.21.15.77:46544) 2024-09-15T22:22:37.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 -- 172.21.15.77:0/3528708463 learned_addr learned my addr 172.21.15.77:0/3528708463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28777fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34018/0 (socket says 172.21.15.77:34018) 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2877fff640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 0x562e4f753530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 -- 172.21.15.77:0/3528708463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 msgr2=0x562e4f744ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 -- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 msgr2=0x562e4f753530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 0x562e4f753530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 -- 172.21.15.77:0/3528708463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e4f753af0 con 0x562e4f732d20 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2877fff640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 0x562e4f753530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f28777fe640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.693+0000 7f2876ffd640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f286c0a4af0 tx=0x7f286c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750c62012bbc9798 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c010720 con 0x562e4f732d20 2024-09-15T22:22:37.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f286c098ad0 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c0a5020 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f286c0071c0 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 10 client.18090 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e4f743440 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c0a5020 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f286c0a4b90 con 0x562e4f732d20 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 10 client.18090 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 20 client.18090 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f7b6e50 0x562e4f7b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 4 client.18090 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.694+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e4f69ed70 con 0x562e4f7b6e50 2024-09-15T22:22:37.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28777fe640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f7b6e50 0x562e4f7b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f2856ffd640 10 client.18090 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28777fe640 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f7b6e50 0x562e4f7b9270 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4c3491aaf0678f server_cookie=ef197da928af953e in_seq=0 out_seq=0 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f2856ffd640 1 -- 172.21.15.77:0/3528708463 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e4f69ed70 con 0x562e4f7b6e50 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f2856ffd640 10 client.18090 handle_command_reply: tid=0 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 client.18090 shutdown 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 msgr2=0x562e4f752ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f286c0a4af0 tx=0x7f286c00c840 comp rx=0 tx=0).stop 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 shutdown_connections 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562e4f7b6e50 0x562e4f7b9270 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e4f73c550 0x562e4f753530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f2856ffd640 0 client.18090 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e4f732d20 0x562e4f752ff0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 --2- 172.21.15.77:0/3528708463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e4f7317b0 0x562e4f744ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 >> 172.21.15.77:0/3528708463 conn(0x562e4f6fa310 msgr2=0x562e4f737930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 shutdown_connections 2024-09-15T22:22:37.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.695+0000 7f28805d8740 1 -- 172.21.15.77:0/3528708463 wait complete. 2024-09-15T22:22:37.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f28805d8740 20 client.18090 trim_cache size 0 max 16384 2024-09-15T22:22:37.705 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 msgr2=0x7f28600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 0x7f28600a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2870005fd0 tx=0x7f2870009500 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 msgr2=0x7f2878198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f286c00e9f0 tx=0x7f286c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 shutdown_connections 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28600a33b0 0x7f28600a5870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28781068d0 0x7f287819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2878101270 0x7f2878198850 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 --2- 172.21.15.77:0/3961545112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28781008d0 0x7f2878198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:37.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 >> 172.21.15.77:0/3961545112 conn(0x7f28780fc610 msgr2=0x7f287806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:37.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 shutdown_connections 2024-09-15T22:22:37.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:37.696+0000 7f287cf9d640 1 -- 172.21.15.77:0/3961545112 wait complete. 2024-09-15T22:22:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:37 smithi117 ceph-mon[35021]: pgmap v317: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-09-15T22:22:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:37 smithi077 ceph-mon[30980]: pgmap v317: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-09-15T22:22:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:37 smithi196 ceph-mon[35109]: pgmap v317: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-09-15T22:22:38.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 777536 7.52 MB/sec execute 407 sec latency 58.481 ms 2024-09-15T22:22:39.421 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:22:39.421 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:22:39.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 778503 7.51 MB/sec execute 408 sec latency 15.038 ms 2024-09-15T22:22:39.688 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:39 smithi077 ceph-mon[30980]: pgmap v318: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-15T22:22:40.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:39 smithi117 ceph-mon[35021]: pgmap v318: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-15T22:22:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.173+0000 7f4243ba6640 1 -- 172.21.15.77:0/2615308616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 msgr2=0x7f423c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.173+0000 7f4243ba6640 1 --2- 172.21.15.77:0/2615308616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c107b90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f422c009930 tx=0x7f422c02a420 comp rx=0 tx=0).stop 2024-09-15T22:22:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 -- 172.21.15.77:0/2615308616 shutdown_connections 2024-09-15T22:22:40.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 --2- 172.21.15.77:0/2615308616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 0x7f423c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 --2- 172.21.15.77:0/2615308616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c107b90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 --2- 172.21.15.77:0/2615308616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 -- 172.21.15.77:0/2615308616 >> 172.21.15.77:0/2615308616 conn(0x7f423c0fc7d0 msgr2=0x7f423c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 -- 172.21.15.77:0/2615308616 shutdown_connections 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 -- 172.21.15.77:0/2615308616 wait complete. 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 Processor -- start 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 -- start start 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.174+0000 7f4243ba6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 0x7f423c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1120a0 con 0x7f423c1036e0 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f423c111f20 con 0x7f423c102d30 2024-09-15T22:22:40.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f423c112220 con 0x7f423c1082a0 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241141640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51038/0 (socket says 172.21.15.77:51038) 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241141640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36136/0 (socket says 172.21.15.77:36136) 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 -- 172.21.15.77:0/1977651608 learned_addr learned my addr 172.21.15.77:0/1977651608 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4242143640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 0x7f423c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 -- 172.21.15.77:0/1977651608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 msgr2=0x7f423c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 0x7f423c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 -- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 msgr2=0x7f423c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 -- 172.21.15.77:0/1977651608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c19d5f0 con 0x7f423c102d30 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4241942640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f423800e9f0 tx=0x7f423800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4232ffd640 1 -- 172.21.15.77:0/1977651608 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423800cd80 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4232ffd640 1 -- 172.21.15.77:0/1977651608 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238010910 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19d8e0 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4232ffd640 1 -- 172.21.15.77:0/1977651608 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238018990 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.175+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1008c0 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4232ffd640 1 -- 172.21.15.77:0/1977651608 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4238010ac0 con 0x7f423c102d30 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 Processor -- start 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 -- start start 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb876800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb879130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560cbb7d1280 con 0x560cbb86bb60 2024-09-15T22:22:40.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560cbb778b10 con 0x560cbb86c480 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4245f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560cbb7cd240 con 0x560cbb876d40 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.178+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36150/0 (socket says 172.21.15.77:36150) 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4241942640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb876800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 -- 172.21.15.77:0/2301988449 learned_addr learned my addr 172.21.15.77:0/2301988449 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4232ffd640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4241141640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb879130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4232ffd640 1 -- 172.21.15.77:0/1977651608 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f42380d0440 con 0x7f423c102d30 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 -- 172.21.15.77:0/2301988449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 msgr2=0x560cbb879130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb879130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 -- 172.21.15.77:0/2301988449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 msgr2=0x560cbb876800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb876800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 -- 172.21.15.77:0/2301988449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbb7851f0 con 0x560cbb86bb60 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4241141640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb879130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4241942640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb876800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4234005c40 tx=0x7f4234013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf4a4ddaf1b564 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4242143640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f423400f850 tx=0x7f423401f000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4227fff640 1 -- 172.21.15.77:0/2301988449 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423400fde0 con 0x560cbb86bb60 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4227fff640 1 -- 172.21.15.77:0/2301988449 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4234022020 con 0x560cbb86bb60 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.179+0000 7f4227fff640 1 -- 172.21.15.77:0/2301988449 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234010690 con 0x560cbb86bb60 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 -- 172.21.15.77:0/2301988449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 msgr2=0x560cbb86bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4234005c40 tx=0x7f4234013f50 comp rx=0 tx=0).stop 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 -- 172.21.15.77:0/2301988449 shutdown_connections 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb879130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb876800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2301988449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86bf40 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 -- 172.21.15.77:0/2301988449 >> 172.21.15.77:0/2301988449 conn(0x560cbb8680f0 msgr2=0x560cbb8684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 -- 172.21.15.77:0/2301988449 shutdown_connections 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.180+0000 7f4245f7f740 1 -- 172.21.15.77:0/2301988449 wait complete. 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 -- start start 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 Processor -- start 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 0x560cbb873c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86c480 0x560cbb874170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560cbb879910 con 0x560cbb876d40 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560cbb78bf10 con 0x560cbb86bb60 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560cbb879a90 con 0x560cbb86c480 2024-09-15T22:22:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4245f7f740 10 client.0 _ll_register_callbacks cb 0x560cbb754a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 0x560cbb873c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36160/0 (socket says 172.21.15.77:36160) 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4242143640 1 -- 172.21.15.77:0/2544079147 learned_addr learned my addr 172.21.15.77:0/2544079147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241141640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86c480 0x560cbb874170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 -- 172.21.15.77:0/2544079147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86c480 msgr2=0x560cbb874170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86c480 0x560cbb874170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 -- 172.21.15.77:0/2544079147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 msgr2=0x560cbb8746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4241942640 1 -- 172.21.15.77:0/2544079147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbb5426a0 con 0x560cbb86bb60 2024-09-15T22:22:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.181+0000 7f4242143640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4241942640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 0x560cbb873c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f42380cd7a0 tx=0x7f423809a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d30960f8cf1817 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238018d30 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f423809b830 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4234022020 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f423809ba50 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 10 client.27149 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560cbb885580 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4234022020 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f42380105a0 con 0x560cbb86bb60 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 10 client.27149 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 20 client.27149 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb886990 0x560cbb888dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 4 client.27149 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.182+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cbb7de7f0 con 0x560cbb886990 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.183+0000 7f4241141640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb886990 0x560cbb888dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.183+0000 7f4241141640 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb886990 0x560cbb888dc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16ad9bfd653a936 server_cookie=4fcdd1486397f0ea in_seq=0 out_seq=0 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.183+0000 7f42257fa640 10 client.27149 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f42257fa640 1 -- 172.21.15.77:0/2544079147 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cbb7de7f0 con 0x560cbb886990 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f42257fa640 10 client.27149 handle_command_reply: tid=0 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 client.27149 shutdown 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 msgr2=0x560cbb873c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 0x560cbb873c30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f42380cd7a0 tx=0x7f423809a8f0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 shutdown_connections 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb886990 0x560cbb888dc0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb8746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86c480 0x560cbb874170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 --2- 172.21.15.77:0/2544079147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86bb60 0x560cbb873c30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f42257fa640 0 client.27149 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 >> 172.21.15.77:0/2544079147 conn(0x560cbb8680f0 msgr2=0x560cbb867270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 shutdown_connections 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 1 -- 172.21.15.77:0/2544079147 wait complete. 2024-09-15T22:22:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.184+0000 7f4245f7f740 20 client.27149 trim_cache size 0 max 16384 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 Processor -- start 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 -- start start 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb875a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb88e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b1320 con 0x560cbb86bb60 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b2e60 con 0x560cbb86c480 2024-09-15T22:22:40.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4245f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b19f0 con 0x560cbb876d40 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36162/0 (socket says 172.21.15.77:36162) 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 -- 172.21.15.77:0/4184926777 learned_addr learned my addr 172.21.15.77:0/4184926777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4242143640 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb88e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241141640 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb875a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 -- 172.21.15.77:0/4184926777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 msgr2=0x560cbb88e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb88e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 -- 172.21.15.77:0/4184926777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 msgr2=0x560cbb875a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb875a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.202+0000 7f4241942640 1 -- 172.21.15.77:0/4184926777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbb872be0 con 0x560cbb86bb60 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f4241942640 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f423809c370 tx=0x7f4238000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785a5b1e86c0b603 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f42277fe640 1 -- 172.21.15.77:0/4184926777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380a06e0 con 0x560cbb86bb60 2024-09-15T22:22:40.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f42277fe640 1 -- 172.21.15.77:0/4184926777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380983a0 con 0x560cbb86bb60 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f42277fe640 1 -- 172.21.15.77:0/4184926777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380bc350 con 0x560cbb86bb60 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f4245f7f740 1 -- 172.21.15.77:0/4184926777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 msgr2=0x560cbb86e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.203+0000 7f4245f7f740 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f423809c370 tx=0x7f4238000bc0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- 172.21.15.77:0/4184926777 shutdown_connections 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb876d40 0x560cbb88e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb875a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- 172.21.15.77:0/4184926777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb86bb60 0x560cbb86e5c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- 172.21.15.77:0/4184926777 >> 172.21.15.77:0/4184926777 conn(0x560cbb8680f0 msgr2=0x560cbb875310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- 172.21.15.77:0/4184926777 shutdown_connections 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- 172.21.15.77:0/4184926777 wait complete. 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- start start 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 Processor -- start 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86bb60 0x560cbb870d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b2790 con 0x560cbb876d40 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1b20c0 con 0x560cbb86c480 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.204+0000 7f4245f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1af320 con 0x560cbb86bb60 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4245f7f740 10 client.0 _ll_register_callbacks cb 0x7f4238096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51076/0 (socket says 172.21.15.77:51076) 2024-09-15T22:22:40.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4242143640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36172/0 (socket says 172.21.15.77:36172) 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 -- 172.21.15.77:0/937923975 learned_addr learned my addr 172.21.15.77:0/937923975 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241942640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86bb60 0x560cbb870d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 -- 172.21.15.77:0/937923975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86bb60 msgr2=0x560cbb870d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86bb60 0x560cbb870d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 -- 172.21.15.77:0/937923975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 msgr2=0x560cbb871800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 -- 172.21.15.77:0/937923975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbb8f0750 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4242143640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4241141640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f422c009900 tx=0x7f422c032ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1ef7698f90db3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f42257fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c035040 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f422c004450 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.205+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42380983a0 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f422c054050 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 10 client.27155 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560cbb8f1af0 con 0x560cbb86c480 2024-09-15T22:22:40.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380983a0 con 0x560cbb86c480 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f422c0035f0 con 0x560cbb86c480 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 10 client.27155 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 20 client.27155 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb8f22c0 0x560cbb8f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 4 client.27155 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560cbb7d8d00 con 0x560cbb8f22c0 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4241942640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb8f22c0 0x560cbb8f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f42257fa640 10 client.27155 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.206+0000 7f4241942640 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb8f22c0 0x560cbb8f46e0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15181a336e94950 server_cookie=f6a2d71c9f2992dc in_seq=0 out_seq=0 2024-09-15T22:22:40.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.207+0000 7f42257fa640 1 -- 172.21.15.77:0/937923975 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560cbb7d8d00 con 0x560cbb8f22c0 2024-09-15T22:22:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.207+0000 7f42257fa640 10 client.27155 handle_command_reply: tid=0 2024-09-15T22:22:40.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.207+0000 7f4245f7f740 1 client.27155 shutdown 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 msgr2=0x560cbb8712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f422c009900 tx=0x7f422c032ee0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 shutdown_connections 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560cbb8f22c0 0x560cbb8f46e0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560cbb876d40 0x560cbb871800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cbb86c480 0x560cbb8712c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f42257fa640 0 client.27155 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 --2- 172.21.15.77:0/937923975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560cbb86bb60 0x560cbb870d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 >> 172.21.15.77:0/937923975 conn(0x560cbb8680f0 msgr2=0x560cbb874410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 shutdown_connections 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 1 -- 172.21.15.77:0/937923975 wait complete. 2024-09-15T22:22:40.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.208+0000 7f4245f7f740 20 client.27155 trim_cache size 0 max 16384 2024-09-15T22:22:40.219 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:22:40.219 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:22:40.219 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "4c1f9774-8a53-491c-8447-36172c403e2a", 2024-09-15T22:22:40.219 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:22:40.219 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 msgr2=0x7f42280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f423400f850 tx=0x7f423401f000 comp rx=0 tx=0).stop 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 msgr2=0x7f423c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f423800e9f0 tx=0x7f423800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 shutdown_connections 2024-09-15T22:22:40.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f423c1082a0 0x7f423c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f423c1036e0 0x7f423c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 --2- 172.21.15.77:0/1977651608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f423c102d30 0x7f423c198640 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 >> 172.21.15.77:0/1977651608 conn(0x7f423c0fc7d0 msgr2=0x7f423c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.209+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 shutdown_connections 2024-09-15T22:22:40.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:40.210+0000 7f4243ba6640 1 -- 172.21.15.77:0/1977651608 wait complete. 2024-09-15T22:22:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:39 smithi196 ceph-mon[35109]: pgmap v318: 129 pgs: 129 active+clean; 497 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-15T22:22:40.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 781117 7.53 MB/sec execute 409 sec latency 38.414 ms 2024-09-15T22:22:41.244 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:22:41.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:22:41.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:22:41.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.245 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.274 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:22:41.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:22:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:22:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:22:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:41.485 INFO:tasks.workunit.client.0.smithi077.stdout: 1 782274 7.52 MB/sec execute 410 sec latency 26.437 ms 2024-09-15T22:22:41.545 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:42.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:41 smithi077 ceph-mon[30980]: pgmap v319: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-09-15T22:22:42.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 -- 172.21.15.77:0/1227859319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 msgr2=0x7f95841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 --2- 172.21.15.77:0/1227859319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f95841029b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f956c009900 tx=0x7f956c02a480 comp rx=0 tx=0).stop 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 -- 172.21.15.77:0/1227859319 shutdown_connections 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 --2- 172.21.15.77:0/1227859319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584102ef0 0x7f958410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 --2- 172.21.15.77:0/1227859319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f95841029b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 --2- 172.21.15.77:0/1227859319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584100e20 0x7f9584101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 -- 172.21.15.77:0/1227859319 >> 172.21.15.77:0/1227859319 conn(0x7f95840fc7d0 msgr2=0x7f95840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 -- 172.21.15.77:0/1227859319 shutdown_connections 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.101+0000 7f9589cf5640 1 -- 172.21.15.77:0/1227859319 wait complete. 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 Processor -- start 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 -- start start 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584100e20 0x7f95841993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584102ef0 0x7f95841a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9584110f50 con 0x7f9584102570 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9584110dd0 con 0x7f9584102ef0 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f9589cf5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95841110d0 con 0x7f9584100e20 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.102+0000 7f957bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36182/0 (socket says 172.21.15.77:36182) 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 -- 172.21.15.77:0/163967075 learned_addr learned my addr 172.21.15.77:0/163967075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f95837fe640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584100e20 0x7f95841993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f9583fff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584102ef0 0x7f95841a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 -- 172.21.15.77:0/163967075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584100e20 msgr2=0x7f95841993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584100e20 0x7f95841993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 -- 172.21.15.77:0/163967075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584102ef0 msgr2=0x7f95841a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584102ef0 0x7f95841a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 -- 172.21.15.77:0/163967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95841a4980 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f957bfff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f95817fa640 1 -- 172.21.15.77:0/163967075 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c03f070 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f95817fa640 1 -- 172.21.15.77:0/163967075 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f956c031c30 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95841a4b50 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f95817fa640 1 -- 172.21.15.77:0/163967075 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c035030 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.103+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95841a4fa0 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f95817fa640 1 -- 172.21.15.77:0/163967075 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f956c042070 con 0x7f9584102570 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 Processor -- start 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 -- start start 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6c80 0x55e0359c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e03591c2e0 con 0x55e0359c5a90 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0358c3ce0 con 0x55e0359b6c80 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f958c0ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e0359182a0 con 0x55e0359b6360 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f9583fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f957bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f9583fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47060/0 (socket says 172.21.15.77:47060) 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.106+0000 7f9583fff640 1 -- 172.21.15.77:0/2197522063 learned_addr learned my addr 172.21.15.77:0/2197522063 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f95837fe640 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6c80 0x55e0359c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 -- 172.21.15.77:0/2197522063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6c80 msgr2=0x55e0359c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6c80 0x55e0359c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f95817fa640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 0x7f95700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 -- 172.21.15.77:0/2197522063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 msgr2=0x55e0359c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 -- 172.21.15.77:0/2197522063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0358a8440 con 0x55e0359b6360 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f95817fa640 1 -- 172.21.15.77:0/163967075 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f956c0f0710 con 0x7f9584102570 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f957c009f50 tx=0x7f957c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1873fd3b6c1d006f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9583fff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 0x7f95700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9579ffb640 1 -- 172.21.15.77:0/2197522063 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c01e070 con 0x55e0359b6360 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9579ffb640 1 -- 172.21.15.77:0/2197522063 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f957c004610 con 0x55e0359b6360 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.107+0000 7f9579ffb640 1 -- 172.21.15.77:0/2197522063 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c019430 con 0x55e0359b6360 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- 172.21.15.77:0/2197522063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 msgr2=0x55e0359b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f9583fff640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 0x7f95700a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f957c01b000 tx=0x7f957c019ae0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f957c009f50 tx=0x7f957c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- 172.21.15.77:0/2197522063 shutdown_connections 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6c80 0x55e0359c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 --2- 172.21.15.77:0/2197522063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359b6740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- 172.21.15.77:0/2197522063 >> 172.21.15.77:0/2197522063 conn(0x55e0359b2a30 msgr2=0x55e0359b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- 172.21.15.77:0/2197522063 shutdown_connections 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- 172.21.15.77:0/2197522063 wait complete. 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.108+0000 7f958c0ce740 1 -- start start 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 Processor -- start 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359c5a90 0x55e0359c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0359c8660 con 0x55e0359b6c80 2024-09-15T22:22:42.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0358d7050 con 0x55e0359c5a90 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e0359c87e0 con 0x55e0359b6360 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f958c0ce740 10 client.0 _ll_register_callbacks cb 0x55e035917160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f95837fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f95837fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47068/0 (socket says 172.21.15.77:47068) 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36202/0 (socket says 172.21.15.77:36202) 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f95837fe640 1 -- 172.21.15.77:0/706798106 learned_addr learned my addr 172.21.15.77:0/706798106 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f9583fff640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359c5a90 0x55e0359c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 -- 172.21.15.77:0/706798106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 msgr2=0x55e0359c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 -- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359c5a90 msgr2=0x55e0359c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359c5a90 0x55e0359c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.109+0000 7f957bfff640 1 -- 172.21.15.77:0/706798106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03568d3c0 con 0x55e0359b6c80 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f957bfff640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f956c0edad0 tx=0x7f956c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adaebd5e9f863b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c049070 con 0x55e0359b6c80 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f956c0b8d80 con 0x55e0359b6c80 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f957c004610 con 0x55e0359b6c80 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f956c0bc4e0 con 0x55e0359b6c80 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 10 client.18120 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:42.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0359d4070 con 0x55e0359b6c80 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f957c004610 con 0x55e0359b6c80 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f956c0f0320 con 0x55e0359b6c80 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 10 client.18120 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 20 client.18120 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e0359d5570 0x55e0359d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 4 client.18120 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.110+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e035929850 con 0x55e0359d5570 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.111+0000 7f95837fe640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e0359d5570 0x55e0359d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.111+0000 7f95837fe640 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e0359d5570 0x55e0359d7960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6336fb63d1490e server_cookie=fc3d3f01c0f61173 in_seq=0 out_seq=0 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.111+0000 7f9562ffd640 10 client.18120 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f9562ffd640 1 -- 172.21.15.77:0/706798106 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e035929850 con 0x55e0359d5570 2024-09-15T22:22:42.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f9562ffd640 10 client.18120 handle_command_reply: tid=0 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 client.18120 shutdown 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 msgr2=0x55e0359c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f956c0edad0 tx=0x7f956c005c90 comp rx=0 tx=0).stop 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 shutdown_connections 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e0359d5570 0x55e0359d7960 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359c5a90 0x55e0359c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f9562ffd640 0 client.18120 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359c2690 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 --2- 172.21.15.77:0/706798106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6360 0x55e0359c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 >> 172.21.15.77:0/706798106 conn(0x55e0359b2a30 msgr2=0x55e0359b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 shutdown_connections 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 1 -- 172.21.15.77:0/706798106 wait complete. 2024-09-15T22:22:42.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.112+0000 7f958c0ce740 20 client.18120 trim_cache size 0 max 16384 2024-09-15T22:22:42.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 Processor -- start 2024-09-15T22:22:42.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 -- start start 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6c80 0x55e0359c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95841acd20 con 0x55e0359c5a90 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ae860 con 0x55e0359b6360 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f958c0ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ad3f0 con 0x55e0359b6c80 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f957bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6c80 0x55e0359c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51108/0 (socket says 172.21.15.77:51108) 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 -- 172.21.15.77:0/690914195 learned_addr learned my addr 172.21.15.77:0/690914195 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f9583fff640 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 -- 172.21.15.77:0/690914195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6c80 msgr2=0x55e0359c0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6c80 0x55e0359c0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 -- 172.21.15.77:0/690914195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 msgr2=0x55e0359b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f95837fe640 1 -- 172.21.15.77:0/690914195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0359c1640 con 0x55e0359b6360 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.130+0000 7f9583fff640 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.131+0000 7f95837fe640 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e0359d81e0 tx=0x7f95740096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670dcf015aed0b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.131+0000 7f95797fa640 1 -- 172.21.15.77:0/690914195 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574004230 con 0x55e0359b6360 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.131+0000 7f95797fa640 1 -- 172.21.15.77:0/690914195 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95740043d0 con 0x55e0359b6360 2024-09-15T22:22:42.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.131+0000 7f95797fa640 1 -- 172.21.15.77:0/690914195 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957401da00 con 0x55e0359b6360 2024-09-15T22:22:42.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- 172.21.15.77:0/690914195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 msgr2=0x55e0359d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e0359d81e0 tx=0x7f95740096f0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- 172.21.15.77:0/690914195 shutdown_connections 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359c5a90 0x55e0359b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359b6c80 0x55e0359c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 --2- 172.21.15.77:0/690914195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359d1900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- 172.21.15.77:0/690914195 >> 172.21.15.77:0/690914195 conn(0x55e0359b2a30 msgr2=0x55e0359b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- 172.21.15.77:0/690914195 shutdown_connections 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- 172.21.15.77:0/690914195 wait complete. 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 -- start start 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.132+0000 7f958c0ce740 1 Processor -- start 2024-09-15T22:22:42.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359c5a90 0x55e035a3f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ae190 con 0x55e0359b6c80 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95841adac0 con 0x55e0359b6360 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95841aad20 con 0x55e0359c5a90 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f958c0ce740 10 client.0 _ll_register_callbacks cb 0x7f956c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51114/0 (socket says 172.21.15.77:51114) 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 -- 172.21.15.77:0/774599655 learned_addr learned my addr 172.21.15.77:0/774599655 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f957bfff640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f9583fff640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359c5a90 0x55e035a3f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 -- 172.21.15.77:0/774599655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359c5a90 msgr2=0x55e035a3f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359c5a90 0x55e035a3f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 -- 172.21.15.77:0/774599655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 msgr2=0x55e0359ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 -- 172.21.15.77:0/774599655 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0359ca9b0 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f95837fe640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f957401d6e0 tx=0x7f9574004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d11b2cef59dfc4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.133+0000 7f9562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574005250 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95740043d0 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f95740053f0 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9574005610 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 10 client.27173 fetch_fsmap learned FSMap version 34 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e035a408a0 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95740043d0 con 0x55e0359b6360 2024-09-15T22:22:42.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9574004c60 con 0x55e0359b6360 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 10 client.27173 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 10 client.27173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 20 client.27173 populate_metadata read hostname 'smithi077' 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e035a40fc0 0x55e035a433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 4 client.27173 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e035923d60 con 0x55e035a40fc0 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.134+0000 7f957bfff640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e035a40fc0 0x55e035a433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f9562ffd640 10 client.27173 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f957bfff640 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e035a40fc0 0x55e035a433b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1a3fd1ee424a8d server_cookie=9337dacc231e0585 in_seq=0 out_seq=0 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f9562ffd640 1 -- 172.21.15.77:0/774599655 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e035923d60 con 0x55e035a40fc0 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f9562ffd640 10 client.27173 handle_command_reply: tid=0 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 client.27173 shutdown 2024-09-15T22:22:42.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 msgr2=0x55e0359c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f957401d6e0 tx=0x7f9574004610 comp rx=0 tx=0).stop 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 shutdown_connections 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e035a40fc0 0x55e035a433b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e0359c5a90 0x55e035a3f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0359b6c80 0x55e0359ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f9562ffd640 0 client.27173 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 --2- 172.21.15.77:0/774599655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0359b6360 0x55e0359c9b90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 >> 172.21.15.77:0/774599655 conn(0x55e0359b2a30 msgr2=0x55e0359c3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 shutdown_connections 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 1 -- 172.21.15.77:0/774599655 wait complete. 2024-09-15T22:22:42.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.135+0000 7f958c0ce740 20 client.27173 trim_cache size 0 max 16384 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "4c1f9774-8a53-491c-8447-36172c403e2a": { 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "4c1f9774-8a53-491c-8447-36172c403e2a", 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:22:42.143 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 msgr2=0x7f95700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 0x7f95700a5870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f957c01b000 tx=0x7f957c019ae0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 msgr2=0x7f9584199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c003c10 comp rx=0 tx=0).stop 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 shutdown_connections 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95700a33b0 0x7f95700a5870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9584102ef0 0x7f95841a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9584102570 0x7f9584199900 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 --2- 172.21.15.77:0/163967075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9584100e20 0x7f95841993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 >> 172.21.15.77:0/163967075 conn(0x7f95840fc7d0 msgr2=0x7f95840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 shutdown_connections 2024-09-15T22:22:42.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:22:42.136+0000 7f9589cf5640 1 -- 172.21.15.77:0/163967075 wait complete. 2024-09-15T22:22:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:41 smithi196 ceph-mon[35109]: pgmap v319: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-09-15T22:22:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:41 smithi117 ceph-mon[35021]: pgmap v319: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-09-15T22:22:42.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 784682 7.54 MB/sec execute 411 sec latency 49.497 ms 2024-09-15T22:22:42.953 INFO:tasks.cephfs.filesystem:scrub status for tag:4c1f9774-8a53-491c-8447-36172c403e2a - {'path': '/', 'tag': '4c1f9774-8a53-491c-8447-36172c403e2a', 'options': 'recursive,force'} 2024-09-15T22:22:43.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:42 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:43.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:42 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:43.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:42 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:43.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 785741 7.53 MB/sec execute 412 sec latency 27.976 ms 2024-09-15T22:22:44.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:43 smithi196 ceph-mon[35109]: pgmap v320: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:44.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:43 smithi117 ceph-mon[35021]: pgmap v320: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:43 smithi077 ceph-mon[30980]: pgmap v320: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:44.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 788224 7.55 MB/sec execute 413 sec latency 15.971 ms 2024-09-15T22:22:45.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 789639 7.55 MB/sec execute 414 sec latency 17.031 ms 2024-09-15T22:22:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:45.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:45.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:22:45.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:45.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:45.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:22:46.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 791891 7.56 MB/sec execute 415 sec latency 21.910 ms 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: pgmap v321: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:22:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:22:46.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:22:46.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:22:46.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:22:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: pgmap v321: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:22:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:22:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: pgmap v321: 129 pgs: 129 active+clean; 508 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 131 op/s 2024-09-15T22:22:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:22:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:22:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:22:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:22:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:22:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:22:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:22:46.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:22:47.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 793225 7.56 MB/sec execute 416 sec latency 133.993 ms 2024-09-15T22:22:48.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 795390 7.57 MB/sec execute 417 sec latency 29.868 ms 2024-09-15T22:22:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:48 smithi077 ceph-mon[30980]: pgmap v322: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2024-09-15T22:22:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:48 smithi196 ceph-mon[35109]: pgmap v322: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2024-09-15T22:22:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:48 smithi117 ceph-mon[35021]: pgmap v322: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2024-09-15T22:22:49.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 796866 7.57 MB/sec execute 418 sec latency 16.816 ms 2024-09-15T22:22:50.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 799056 7.58 MB/sec execute 419 sec latency 18.370 ms 2024-09-15T22:22:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:50 smithi077 ceph-mon[30980]: pgmap v323: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:22:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:50 smithi196 ceph-mon[35109]: pgmap v323: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:22:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:50 smithi117 ceph-mon[35021]: pgmap v323: 129 pgs: 129 active+clean; 519 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:22:51.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 800752 7.58 MB/sec execute 420 sec latency 17.446 ms 2024-09-15T22:22:52.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 802736 7.59 MB/sec execute 421 sec latency 88.140 ms 2024-09-15T22:22:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:52 smithi077 ceph-mon[30980]: pgmap v324: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-09-15T22:22:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:52 smithi196 ceph-mon[35109]: pgmap v324: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-09-15T22:22:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:52 smithi117 ceph-mon[35021]: pgmap v324: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-09-15T22:22:53.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 804707 7.59 MB/sec execute 422 sec latency 21.378 ms 2024-09-15T22:22:54.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 806480 7.60 MB/sec execute 423 sec latency 16.163 ms 2024-09-15T22:22:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:54 smithi077 ceph-mon[30980]: pgmap v325: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:54 smithi196 ceph-mon[35109]: pgmap v325: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:54 smithi117 ceph-mon[35021]: pgmap v325: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:55.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 808720 7.61 MB/sec execute 424 sec latency 26.980 ms 2024-09-15T22:22:56.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 810223 7.61 MB/sec execute 425 sec latency 19.965 ms 2024-09-15T22:22:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:56 smithi077 ceph-mon[30980]: pgmap v326: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:56 smithi196 ceph-mon[35109]: pgmap v326: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:56 smithi117 ceph-mon[35021]: pgmap v326: 129 pgs: 129 active+clean; 528 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 170 op/s 2024-09-15T22:22:57.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 812017 7.61 MB/sec execute 426 sec latency 95.176 ms 2024-09-15T22:22:58.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 813502 7.62 MB/sec execute 427 sec latency 27.143 ms 2024-09-15T22:22:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:22:58 smithi077 ceph-mon[30980]: pgmap v327: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 261 op/s 2024-09-15T22:22:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:22:58 smithi196 ceph-mon[35109]: pgmap v327: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 261 op/s 2024-09-15T22:22:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:22:58 smithi117 ceph-mon[35021]: pgmap v327: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.9 MiB/s wr, 261 op/s 2024-09-15T22:22:59.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 815462 7.62 MB/sec execute 428 sec latency 47.466 ms 2024-09-15T22:23:00.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 817151 7.63 MB/sec execute 429 sec latency 14.066 ms 2024-09-15T22:23:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:00 smithi077 ceph-mon[30980]: pgmap v328: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 184 op/s 2024-09-15T22:23:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:00 smithi196 ceph-mon[35109]: pgmap v328: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 184 op/s 2024-09-15T22:23:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:00 smithi117 ceph-mon[35021]: pgmap v328: 129 pgs: 129 active+clean; 539 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 184 op/s 2024-09-15T22:23:01.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 817627 7.61 MB/sec execute 430 sec latency 64.753 ms 2024-09-15T22:23:02.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 819143 7.61 MB/sec execute 431 sec latency 140.115 ms 2024-09-15T22:23:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:02 smithi077 ceph-mon[30980]: pgmap v329: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:02 smithi196 ceph-mon[35109]: pgmap v329: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:02 smithi117 ceph-mon[35021]: pgmap v329: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:03.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 820607 7.62 MB/sec execute 432 sec latency 20.828 ms 2024-09-15T22:23:04.486 INFO:tasks.workunit.client.0.smithi077.stdout: 1 822909 7.63 MB/sec execute 433 sec latency 17.166 ms 2024-09-15T22:23:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:04 smithi077 ceph-mon[30980]: pgmap v330: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:04 smithi196 ceph-mon[35109]: pgmap v330: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:04 smithi117 ceph-mon[35021]: pgmap v330: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:05.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 824400 7.63 MB/sec execute 434 sec latency 26.814 ms 2024-09-15T22:23:06.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 826879 7.64 MB/sec execute 435 sec latency 16.185 ms 2024-09-15T22:23:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:06 smithi077 ceph-mon[30980]: pgmap v331: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:06 smithi196 ceph-mon[35109]: pgmap v331: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:06 smithi117 ceph-mon[35021]: pgmap v331: 129 pgs: 129 active+clean; 549 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:23:07.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 827979 7.64 MB/sec execute 436 sec latency 48.313 ms 2024-09-15T22:23:08.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 830156 7.65 MB/sec execute 437 sec latency 22.712 ms 2024-09-15T22:23:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:08 smithi077 ceph-mon[30980]: pgmap v332: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 295 op/s 2024-09-15T22:23:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:08 smithi196 ceph-mon[35109]: pgmap v332: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 295 op/s 2024-09-15T22:23:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:08 smithi117 ceph-mon[35021]: pgmap v332: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 295 op/s 2024-09-15T22:23:09.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 831517 7.65 MB/sec execute 438 sec latency 13.386 ms 2024-09-15T22:23:10.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 834126 7.66 MB/sec execute 439 sec latency 15.426 ms 2024-09-15T22:23:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:10 smithi077 ceph-mon[30980]: pgmap v333: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-09-15T22:23:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:10 smithi196 ceph-mon[35109]: pgmap v333: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-09-15T22:23:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:10 smithi117 ceph-mon[35021]: pgmap v333: 129 pgs: 129 active+clean; 558 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-09-15T22:23:11.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 835287 7.66 MB/sec execute 440 sec latency 25.082 ms 2024-09-15T22:23:12.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 837915 7.68 MB/sec execute 441 sec latency 29.595 ms 2024-09-15T22:23:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:12 smithi077 ceph-mon[30980]: pgmap v334: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-15T22:23:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:12 smithi077 ceph-mon[30980]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:23:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:12 smithi196 ceph-mon[35109]: pgmap v334: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-15T22:23:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:12 smithi196 ceph-mon[35109]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:23:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:12 smithi117 ceph-mon[35021]: pgmap v334: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-15T22:23:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:12 smithi117 ceph-mon[35021]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:23:12.954 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:23:13.215 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:13.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 838790 7.66 MB/sec execute 442 sec latency 180.496 ms 2024-09-15T22:23:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 -- 172.21.15.77:0/3650648726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 msgr2=0x7fe1a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3650648726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a810ec40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009cd0 tx=0x7fe1a002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 -- 172.21.15.77:0/3650648726 shutdown_connections 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3650648726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a810ec40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3650648726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a81036e0 0x7fe1a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3650648726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a8102d30 0x7fe1a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 -- 172.21.15.77:0/3650648726 >> 172.21.15.77:0/3650648726 conn(0x7fe1a80fc7d0 msgr2=0x7fe1a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 -- 172.21.15.77:0/3650648726 shutdown_connections 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.774+0000 7fe1acbac640 1 -- 172.21.15.77:0/3650648726 wait complete. 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 Processor -- start 2024-09-15T22:23:13.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 -- start start 2024-09-15T22:23:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.775+0000 7fe1acbac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81120a0 con 0x7fe1a81082a0 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38122/0 (socket says 172.21.15.77:38122) 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37458/0 (socket says 172.21.15.77:37458) 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a659c640 1 -- 172.21.15.77:0/3785699513 learned_addr learned my addr 172.21.15.77:0/3785699513 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a8111f20 con 0x7fe1a8102d30 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a8112220 con 0x7fe1a81036e0 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/3785699513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 msgr2=0x7fe1a8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/3785699513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 msgr2=0x7fe1a8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/3785699513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a819d4e0 con 0x7fe1a81082a0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe19ffff640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a659c640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a819ce00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe1a002ae00 tx=0x7fe1a0031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe19f7fe640 1 -- 172.21.15.77:0/3785699513 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0003d80 con 0x7fe1a81082a0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.776+0000 7fe19f7fe640 1 -- 172.21.15.77:0/3785699513 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a0004520 con 0x7fe1a81082a0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.777+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a819d770 con 0x7fe1a81082a0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.777+0000 7fe19f7fe640 1 -- 172.21.15.77:0/3785699513 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0030d00 con 0x7fe1a81082a0 2024-09-15T22:23:13.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.777+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a81008c0 con 0x7fe1a81082a0 2024-09-15T22:23:13.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.779+0000 7fe19f7fe640 1 -- 172.21.15.77:0/3785699513 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe1a00046d0 con 0x7fe1a81082a0 2024-09-15T22:23:13.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.779+0000 7fe1aef85740 1 Processor -- start 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 -- start start 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe19f7fe640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 0x7fe1940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612bd1352e0 con 0x5612bd1da680 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612bd0dcce0 con 0x5612bd1cfdc0 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1aef85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612bd1312a0 con 0x5612bd1cf4a0 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe19f7fe640 1 -- 172.21.15.77:0/3785699513 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fe1a002e030 con 0x7fe1a81082a0 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37466/0 (socket says 172.21.15.77:37466) 2024-09-15T22:23:13.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1a659c640 1 -- 172.21.15.77:0/428161997 learned_addr learned my addr 172.21.15.77:0/428161997 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe19ffff640 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe19ffff640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 0x7fe1940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.780+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 -- 172.21.15.77:0/428161997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 msgr2=0x5612bd1da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 -- 172.21.15.77:0/428161997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 msgr2=0x5612bd1dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 -- 172.21.15.77:0/428161997 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bd0c1440 con 0x5612bd1cf4a0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe1a659c640 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe198013940 tx=0x7fe198013d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3b38c1212b8733 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe19ffff640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 0x7fe1940a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe190009fd0 tx=0x7fe190005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe19cff9640 1 -- 172.21.15.77:0/428161997 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198021070 con 0x5612bd1cf4a0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe19cff9640 1 -- 172.21.15.77:0/428161997 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19800ee70 con 0x5612bd1cf4a0 2024-09-15T22:23:13.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.781+0000 7fe19cff9640 1 -- 172.21.15.77:0/428161997 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19801a310 con 0x5612bd1cf4a0 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- 172.21.15.77:0/428161997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 msgr2=0x5612bd1cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe198013940 tx=0x7fe198013d40 comp rx=0 tx=0).stop 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- 172.21.15.77:0/428161997 shutdown_connections 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 --2- 172.21.15.77:0/428161997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1cf880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- 172.21.15.77:0/428161997 >> 172.21.15.77:0/428161997 conn(0x5612bd1cba30 msgr2=0x5612bd1cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- 172.21.15.77:0/428161997 shutdown_connections 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- 172.21.15.77:0/428161997 wait complete. 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 -- start start 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.782+0000 7fe1aef85740 1 Processor -- start 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cfdc0 0x5612bd1d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612bd1dd250 con 0x5612bd1da680 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5612bd0f0050 con 0x5612bd1cf4a0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612bd1dd3d0 con 0x5612bd1cfdc0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1aef85740 10 client.0 _ll_register_callbacks cb 0x5612bd130160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cfdc0 0x5612bd1d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40780/0 (socket says 172.21.15.77:40780) 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2541854442 learned_addr learned my addr 172.21.15.77:0/2541854442 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2541854442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cfdc0 msgr2=0x5612bd1d71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cfdc0 0x5612bd1d71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 msgr2=0x5612bd1d9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2541854442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bcea63c0 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00ee9f0 tx=0x7fe1a00bd060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775d23872a1f723 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.783+0000 7fe18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00bd520 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fe1a00305a0 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19800ee70 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1a00c18d0 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 10 client.18138 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5612bd1e8b70 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19800ee70 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe1a004e070 con 0x5612bd1da680 2024-09-15T22:23:13.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 10 client.18138 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 20 client.18138 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd1ea000 0x5612bd1ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 4 client.18138 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612bd142850 con 0x5612bd1ea000 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.784+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd1ea000 0x5612bd1ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.785+0000 7fe18dffb640 10 client.18138 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.785+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd1ea000 0x5612bd1ec4c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f085b87f74145c35 server_cookie=3dedcc8d166a4c3a in_seq=0 out_seq=0 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.785+0000 7fe18dffb640 1 -- 172.21.15.77:0/2541854442 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612bd142850 con 0x5612bd1ea000 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.785+0000 7fe18dffb640 10 client.18138 handle_command_reply: tid=0 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 client.18138 shutdown 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 msgr2=0x5612bd1d7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00ee9f0 tx=0x7fe1a00bd060 comp rx=0 tx=0).stop 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 shutdown_connections 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd1ea000 0x5612bd1ec4c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1d7710 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cfdc0 0x5612bd1d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2541854442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 >> 172.21.15.77:0/2541854442 conn(0x5612bd1cba30 msgr2=0x5612bd1d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe18dffb640 0 client.18138 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 shutdown_connections 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 1 -- 172.21.15.77:0/2541854442 wait complete. 2024-09-15T22:23:13.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.786+0000 7fe1aef85740 20 client.18138 trim_cache size 0 max 16384 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 Processor -- start 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 -- start start 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1da680 0x5612bd1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81ace70 con 0x5612bd1cf4a0 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81ae9b0 con 0x5612bd1cfdc0 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.806+0000 7fe1aef85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81ad540 con 0x5612bd1da680 2024-09-15T22:23:13.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40792/0 (socket says 172.21.15.77:40792) 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 -- 172.21.15.77:0/2237273996 learned_addr learned my addr 172.21.15.77:0/2237273996 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1da680 0x5612bd1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 -- 172.21.15.77:0/2237273996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1da680 msgr2=0x5612bd1f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1da680 0x5612bd1f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 -- 172.21.15.77:0/2237273996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 msgr2=0x5612bd1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 -- 172.21.15.77:0/2237273996 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bd1da290 con 0x5612bd1cf4a0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe19ffff640 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe19801ca90 tx=0x7fe19801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b618ba0d4a82a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe18ffff640 1 -- 172.21.15.77:0/2237273996 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198021070 con 0x5612bd1cf4a0 2024-09-15T22:23:13.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe18ffff640 1 -- 172.21.15.77:0/2237273996 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe198022040 con 0x5612bd1cf4a0 2024-09-15T22:23:13.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.807+0000 7fe18ffff640 1 -- 172.21.15.77:0/2237273996 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198005530 con 0x5612bd1cf4a0 2024-09-15T22:23:13.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- 172.21.15.77:0/2237273996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 msgr2=0x5612bd1d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe19801ca90 tx=0x7fe19801cc00 comp rx=0 tx=0).stop 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- 172.21.15.77:0/2237273996 shutdown_connections 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1da680 0x5612bd1f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2237273996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1cf4a0 0x5612bd1d9140 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- 172.21.15.77:0/2237273996 >> 172.21.15.77:0/2237273996 conn(0x5612bd1cba30 msgr2=0x5612bd1d8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- 172.21.15.77:0/2237273996 shutdown_connections 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- 172.21.15.77:0/2237273996 wait complete. 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.808+0000 7fe1aef85740 1 -- start start 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 Processor -- start 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81ae2e0 con 0x5612bd1da680 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81adc10 con 0x5612bd1cfdc0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a81aae70 con 0x5612bd1cf4a0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1aef85740 10 client.0 _ll_register_callbacks cb 0x7fe1a0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40800/0 (socket says 172.21.15.77:40800) 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2833247147 learned_addr learned my addr 172.21.15.77:0/2833247147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe19ffff640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2833247147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 msgr2=0x5612bd1e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 msgr2=0x5612bd1e01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:13.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1e01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 -- 172.21.15.77:0/2833247147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bd1e0ff0 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe1a6d9d640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00eea40 tx=0x7fe1a002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ad196de01293b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.809+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00b97c0 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fe1a00f1160 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe198022040 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1a00bd520 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 10 client.18144 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5612bd1e0ff0 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe198022040 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe1a004e040 con 0x5612bd1da680 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 10 client.18144 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:13.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 20 client.18144 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd255230 0x5612bd257620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 4 client.18144 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.810+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612bd13cd60 con 0x5612bd255230 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd255230 0x5612bd257620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1a659c640 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd255230 0x5612bd257620 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f53682bdc6c5b28 server_cookie=9caad9cbb6d8c537 in_seq=0 out_seq=0 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe18dffb640 10 client.18144 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe18dffb640 1 -- 172.21.15.77:0/2833247147 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5612bd13cd60 con 0x5612bd255230 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe18dffb640 10 client.18144 handle_command_reply: tid=0 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 client.18144 shutdown 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 msgr2=0x5612bd1e07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00eea40 tx=0x7fe1a002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 shutdown_connections 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5612bd255230 0x5612bd257620 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612bd1da680 0x5612bd1e07c0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe18dffb640 0 client.18144 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612bd1cfdc0 0x5612bd1e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 --2- 172.21.15.77:0/2833247147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612bd1cf4a0 0x5612bd1e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.811+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 >> 172.21.15.77:0/2833247147 conn(0x5612bd1cba30 msgr2=0x5612bd1d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 shutdown_connections 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1aef85740 1 -- 172.21.15.77:0/2833247147 wait complete. 2024-09-15T22:23:13.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1aef85740 20 client.18144 trim_cache size 0 max 16384 2024-09-15T22:23:13.820 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:23:13.820 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:23:13.820 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:23:13.820 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 msgr2=0x7fe1940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 0x7fe1940a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe190009fd0 tx=0x7fe190005f90 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 msgr2=0x7fe1a819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a819ce00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe1a002ae00 tx=0x7fe1a0031df0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 shutdown_connections 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe1940a3430 0x7fe1940a58f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1a81082a0 0x7fe1a819ce00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a81036e0 0x7fe1a8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 --2- 172.21.15.77:0/3785699513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1a8102d30 0x7fe1a8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:13.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.812+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 >> 172.21.15.77:0/3785699513 conn(0x7fe1a80fc7d0 msgr2=0x7fe1a8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:13.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.813+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 shutdown_connections 2024-09-15T22:23:13.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:13.813+0000 7fe1acbac640 1 -- 172.21.15.77:0/3785699513 wait complete. 2024-09-15T22:23:14.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 841093 7.67 MB/sec execute 443 sec latency 40.324 ms 2024-09-15T22:23:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:14 smithi077 ceph-mon[30980]: pgmap v336: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:14 smithi196 ceph-mon[35109]: pgmap v336: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:14 smithi117 ceph-mon[35021]: pgmap v336: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:14.705 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:23:14.705 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:14.967 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:15.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 842137 7.67 MB/sec execute 444 sec latency 52.871 ms 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 -- 172.21.15.77:0/3194684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa694102370 msgr2=0x7fa69410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 --2- 172.21.15.77:0/3194684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa694102370 0x7fa69410e8a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009cd0 tx=0x7fa68c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 -- 172.21.15.77:0/3194684528 shutdown_connections 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 --2- 172.21.15.77:0/3194684528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa694102370 0x7fa69410e8a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 --2- 172.21.15.77:0/3194684528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6941019d0 0x7fa694101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 --2- 172.21.15.77:0/3194684528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941079d0 0x7fa694107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 -- 172.21.15.77:0/3194684528 >> 172.21.15.77:0/3194684528 conn(0x7fa6940fd710 msgr2=0x7fa6940ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 -- 172.21.15.77:0/3194684528 shutdown_connections 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.512+0000 7fa69359e640 1 -- 172.21.15.77:0/3194684528 wait complete. 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 Processor -- start 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 -- start start 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa694102370 0x7fa69419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6941079d0 0x7fa69419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa694111dd0 con 0x7fa6941079d0 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa694111c50 con 0x7fa6941019d0 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69359e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa694111f50 con 0x7fa694102370 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa692d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6941079d0 0x7fa69419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38190/0 (socket says 172.21.15.77:38190) 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa69259c640 1 -- 172.21.15.77:0/1850452247 learned_addr learned my addr 172.21.15.77:0/1850452247 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.513+0000 7fa68bfff640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa694102370 0x7fa69419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 -- 172.21.15.77:0/1850452247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa694102370 msgr2=0x7fa69419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa694102370 0x7fa69419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 -- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6941079d0 msgr2=0x7fa69419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6941079d0 0x7fa69419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 -- 172.21.15.77:0/1850452247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69419f4a0 con 0x7fa6941019d0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69259c640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa68400e9f0 tx=0x7fa68400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68400cd80 con 0x7fa6941019d0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa684010910 con 0x7fa6941019d0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69419f790 con 0x7fa6941019d0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6840188f0 con 0x7fa6941019d0 2024-09-15T22:23:15.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.514+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941a7070 con 0x7fa6941019d0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.515+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694101e30 con 0x7fa6941019d0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.517+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa684010ac0 con 0x7fa6941019d0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.517+0000 7fa68b7fe640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 0x7fa6800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.517+0000 7fa68bfff640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 0x7fa6800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.517+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fa6840d06e0 con 0x7fa6941019d0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.518+0000 7fa68bfff640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 0x7fa6800a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa694076480 tx=0x7fa67c005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.521+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa68409c680 con 0x7fa6941019d0 2024-09-15T22:23:15.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.756+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa69419b850 con 0x7fa6941019d0 2024-09-15T22:23:15.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.756+0000 7fa68b7fe640 1 -- 172.21.15.77:0/1850452247 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fa6840a0980 con 0x7fa6941019d0 2024-09-15T22:23:15.759 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:23:15.759 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:23:15.760 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:23:15.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.758+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 msgr2=0x7fa6800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.758+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 0x7fa6800a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa694076480 tx=0x7fa67c005f90 comp rx=0 tx=0).stop 2024-09-15T22:23:15.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.758+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 msgr2=0x7fa69419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.758+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa68400e9f0 tx=0x7fa68400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 shutdown_connections 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa6800a33b0 0x7fa6800a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6941079d0 0x7fa69419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa694102370 0x7fa69419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 --2- 172.21.15.77:0/1850452247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6941019d0 0x7fa69419a4f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 >> 172.21.15.77:0/1850452247 conn(0x7fa6940fd710 msgr2=0x7fa69410d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 shutdown_connections 2024-09-15T22:23:15.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:15.759+0000 7fa69359e640 1 -- 172.21.15.77:0/1850452247 wait complete. 2024-09-15T22:23:16.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 844679 7.68 MB/sec execute 445 sec latency 15.684 ms 2024-09-15T22:23:16.553 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:16 smithi077 ceph-mon[30980]: pgmap v337: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:16.553 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:16 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1850452247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:16 smithi196 ceph-mon[35109]: pgmap v337: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:16 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1850452247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:16 smithi117 ceph-mon[35021]: pgmap v337: 129 pgs: 129 active+clean; 525 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 244 op/s 2024-09-15T22:23:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:16 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1850452247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.898 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:23:17.161 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:17.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 845933 7.68 MB/sec execute 446 sec latency 22.355 ms 2024-09-15T22:23:17.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.692+0000 7fb0af59e640 1 -- 172.21.15.77:0/1058571532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 msgr2=0x7fb0b0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.692+0000 7fb0af59e640 1 --2- 172.21.15.77:0/1058571532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b0102f70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a480 comp rx=0 tx=0).stop 2024-09-15T22:23:17.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.694+0000 7fb0af59e640 1 -- 172.21.15.77:0/1058571532 shutdown_connections 2024-09-15T22:23:17.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.694+0000 7fb0af59e640 1 --2- 172.21.15.77:0/1058571532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b010bfc0 0x7fb0b010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.694+0000 7fb0af59e640 1 --2- 172.21.15.77:0/1058571532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b01034b0 0x7fb0b010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.694+0000 7fb0af59e640 1 --2- 172.21.15.77:0/1058571532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b0102f70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.694+0000 7fb0af59e640 1 -- 172.21.15.77:0/1058571532 >> 172.21.15.77:0/1058571532 conn(0x7fb0b00fc610 msgr2=0x7fb0b00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.695+0000 7fb0af59e640 1 -- 172.21.15.77:0/1058571532 shutdown_connections 2024-09-15T22:23:17.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.696+0000 7fb0af59e640 1 -- 172.21.15.77:0/1058571532 wait complete. 2024-09-15T22:23:17.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.697+0000 7fb0af59e640 1 Processor -- start 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 -- start start 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b01034b0 0x7fb0b0108890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b010bfc0 0x7fb0b0108dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0114010 con 0x7fb0b0102b90 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0113e90 con 0x7fb0b010bfc0 2024-09-15T22:23:17.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.699+0000 7fb0af59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0114190 con 0x7fb0b01034b0 2024-09-15T22:23:17.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.701+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.701+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40848/0 (socket says 172.21.15.77:40848) 2024-09-15T22:23:17.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.701+0000 7fb0ae59c640 1 -- 172.21.15.77:0/373749026 learned_addr learned my addr 172.21.15.77:0/373749026 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:17.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 -- 172.21.15.77:0/373749026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b01034b0 msgr2=0x7fb0b0108890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b01034b0 0x7fb0b0108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 -- 172.21.15.77:0/373749026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b010bfc0 msgr2=0x7fb0b0108dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b010bfc0 0x7fb0b0108dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 -- 172.21.15.77:0/373749026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b0109640 con 0x7fb0b0102b90 2024-09-15T22:23:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.702+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002a9f0 tx=0x7fb0a0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.704+0000 7fb09f7fe640 1 -- 172.21.15.77:0/373749026 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a003f070 con 0x7fb0b0102b90 2024-09-15T22:23:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.704+0000 7fb09f7fe640 1 -- 172.21.15.77:0/373749026 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a0031c90 con 0x7fb0b0102b90 2024-09-15T22:23:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.704+0000 7fb09f7fe640 1 -- 172.21.15.77:0/373749026 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0035030 con 0x7fb0b0102b90 2024-09-15T22:23:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.704+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b0072c90 con 0x7fb0b0102b90 2024-09-15T22:23:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.704+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b0073140 con 0x7fb0b0102b90 2024-09-15T22:23:17.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.709+0000 7fb0b6c89740 1 Processor -- start 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 -- start start 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2040240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2040bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1fa62f0 con 0x55c6c204b520 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1f4dcf0 con 0x55c6c2041cf0 2024-09-15T22:23:17.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.710+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1fa22b0 con 0x55c6c2040780 2024-09-15T22:23:17.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40850/0 (socket says 172.21.15.77:40850) 2024-09-15T22:23:17.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 -- 172.21.15.77:0/3738962736 learned_addr learned my addr 172.21.15.77:0/3738962736 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:17.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2040240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 -- 172.21.15.77:0/3738962736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 msgr2=0x55c6c2040bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:23:17.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2040bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 -- 172.21.15.77:0/3738962736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 msgr2=0x55c6c2040240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.711+0000 7fb0add9b640 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2040240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.712+0000 7fb0add9b640 1 -- 172.21.15.77:0/3738962736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c1f323a0 con 0x55c6c204b520 2024-09-15T22:23:17.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.712+0000 7fb0add9b640 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009ac0 tx=0x7fb0a4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fd8787f9a68a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.714+0000 7fb09cff9640 1 -- 172.21.15.77:0/3738962736 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400ed10 con 0x55c6c204b520 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.714+0000 7fb09f7fe640 1 -- 172.21.15.77:0/373749026 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb0a0042070 con 0x7fb0b0102b90 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.715+0000 7fb09cff9640 1 -- 172.21.15.77:0/3738962736 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4005500 con 0x55c6c204b520 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.716+0000 7fb0b6c89740 1 -- 172.21.15.77:0/3738962736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 msgr2=0x55c6c204d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.716+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009ac0 tx=0x7fb0a4009f80 comp rx=0 tx=0).stop 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.716+0000 7fb09f7fe640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 0x7fb0980a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.716+0000 7fb09f7fe640 1 -- 172.21.15.77:0/373749026 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fb0a00f0a10 con 0x7fb0b0102b90 2024-09-15T22:23:17.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.716+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 0x7fb0980a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.717+0000 7fb0b6c89740 1 -- 172.21.15.77:0/3738962736 shutdown_connections 2024-09-15T22:23:17.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.717+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c204b520 0x55c6c204d910 secure :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009ac0 tx=0x7fb0a4009f80 comp rx=0 tx=0).stop 2024-09-15T22:23:17.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.717+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2040bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.717+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/3738962736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2040240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.717+0000 7fb0b6c89740 1 -- 172.21.15.77:0/3738962736 >> 172.21.15.77:0/3738962736 conn(0x55c6c2009310 msgr2=0x55c6c203ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.718+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 0x7fb0980a5a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005cc0 tx=0x7fb0a800a310 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.720+0000 7fb0b6c89740 1 -- 172.21.15.77:0/3738962736 shutdown_connections 2024-09-15T22:23:17.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.722+0000 7fb0b6c89740 1 -- 172.21.15.77:0/3738962736 wait complete. 2024-09-15T22:23:17.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.722+0000 7fb0b6c89740 1 -- start start 2024-09-15T22:23:17.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.722+0000 7fb0b6c89740 1 Processor -- start 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2048340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2048880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6c204e270 con 0x55c6c2048dc0 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1f61060 con 0x55c6c2041cf0 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c6c204e3f0 con 0x55c6c2040780 2024-09-15T22:23:17.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.724+0000 7fb0b6c89740 10 client.0 _ll_register_callbacks cb 0x55c6c1cb48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:17.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40864/0 (socket says 172.21.15.77:40864) 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 -- 172.21.15.77:0/1547944456 learned_addr learned my addr 172.21.15.77:0/1547944456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0add9b640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2048880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2048340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 -- 172.21.15.77:0/1547944456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 msgr2=0x55c6c2048340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2048340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 -- 172.21.15.77:0/1547944456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 msgr2=0x55c6c2048880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2048880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.725+0000 7fb0aed9d640 1 -- 172.21.15.77:0/1547944456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c1d173c0 con 0x55c6c2048dc0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8010e20 tx=0x7fb0a8005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d13aeedda35b075 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8021070 con 0x55c6c2048dc0 2024-09-15T22:23:17.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fb0a801d450 con 0x55c6c2048dc0 2024-09-15T22:23:17.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a4005500 con 0x55c6c2048dc0 2024-09-15T22:23:17.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb0a802d050 con 0x55c6c2048dc0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb0b6c89740 10 client.18162 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.726+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6c205a340 con 0x55c6c2048dc0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4005500 con 0x55c6c2048dc0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a8025570 con 0x55c6c2048dc0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 10 client.18162 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 10 client.18162 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 20 client.18162 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c205b7a0 0x55c6c205dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 4 client.18162 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c1fb3860 con 0x55c6c205b7a0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.727+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c205b7a0 0x55c6c205dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.728+0000 7fb095ffb640 10 client.18162 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:23:17.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.728+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c205b7a0 0x55c6c205dbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce4b8fd64c75bb9a server_cookie=b7f77bb8d1dc385f in_seq=0 out_seq=0 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb095ffb640 1 -- 172.21.15.77:0/1547944456 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6c1fb3860 con 0x55c6c205b7a0 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb095ffb640 10 client.18162 handle_command_reply: tid=0 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 client.18162 shutdown 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 msgr2=0x55c6c20577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8010e20 tx=0x7fb0a8005c10 comp rx=0 tx=0).stop 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 shutdown_connections 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c205b7a0 0x55c6c205dbd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2048dc0 0x55c6c20577a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2048880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1547944456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2040780 0x55c6c2048340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 >> 172.21.15.77:0/1547944456 conn(0x55c6c2009310 msgr2=0x55c6c2042b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb095ffb640 0 client.18162 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 shutdown_connections 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1547944456 wait complete. 2024-09-15T22:23:17.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.729+0000 7fb0b6c89740 20 client.18162 trim_cache size 0 max 16384 2024-09-15T22:23:17.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 Processor -- start 2024-09-15T22:23:17.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 -- start start 2024-09-15T22:23:17.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2057590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c2057ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c6580 con 0x55c6c2040780 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c80c0 con 0x55c6c2041cf0 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c6c50 con 0x55c6c2048dc0 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40878/0 (socket says 172.21.15.77:40878) 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0ae59c640 1 -- 172.21.15.77:0/1894646456 learned_addr learned my addr 172.21.15.77:0/1894646456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c2057ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0add9b640 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2057590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.743+0000 7fb0ae59c640 1 -- 172.21.15.77:0/1894646456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 msgr2=0x55c6c2057ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c2057ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb0ae59c640 1 -- 172.21.15.77:0/1894646456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 msgr2=0x55c6c2057590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2057590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb0ae59c640 1 -- 172.21.15.77:0/1894646456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c205e250 con 0x55c6c2040780 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00bc890 tx=0x7fb0a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d9e219a7aebbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb097fff640 1 -- 172.21.15.77:0/1894646456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00d4070 con 0x55c6c2040780 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb097fff640 1 -- 172.21.15.77:0/1894646456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a00bac00 con 0x55c6c2040780 2024-09-15T22:23:17.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.744+0000 7fb097fff640 1 -- 172.21.15.77:0/1894646456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00433b0 con 0x55c6c2040780 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1894646456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 msgr2=0x55c6c203bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00bc890 tx=0x7fb0a0009260 comp rx=0 tx=0).stop 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1894646456 shutdown_connections 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c2057ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c2057590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/1894646456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c203bda0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1894646456 >> 172.21.15.77:0/1894646456 conn(0x55c6c2009310 msgr2=0x55c6c203b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1894646456 shutdown_connections 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- 172.21.15.77:0/1894646456 wait complete. 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- start start 2024-09-15T22:23:17.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 Processor -- start 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c204f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c204f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c79f0 con 0x55c6c2040780 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c7320 con 0x55c6c2041cf0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01c4580 con 0x55c6c2048dc0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.745+0000 7fb0b6c89740 10 client.0 _ll_register_callbacks cb 0x7fb0a8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37562/0 (socket says 172.21.15.77:37562) 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 -- 172.21.15.77:0/2690975983 learned_addr learned my addr 172.21.15.77:0/2690975983 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c204f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0add9b640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c204f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 -- 172.21.15.77:0/2690975983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 msgr2=0x55c6c204f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c204f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 -- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 msgr2=0x55c6c204f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c204f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 -- 172.21.15.77:0/2690975983 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c20c5470 con 0x55c6c2048dc0 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0add9b640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c204f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0aed9d640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8025940 tx=0x7fb0a800afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9bd82b6caae8bad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8021040 con 0x55c6c2048dc0 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a00bac00 con 0x55c6c2048dc0 2024-09-15T22:23:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9526+0+0 (secure 0 0 0) 0x7fb0a80255b0 con 0x55c6c2048dc0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb0a8041050 con 0x55c6c2048dc0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0b6c89740 10 client.27142 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.746+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6c20c6690 con 0x55c6c2048dc0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a00bac00 con 0x55c6c2048dc0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a8004430 con 0x55c6c2048dc0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 10 client.27142 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 10 client.27142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 20 client.27142 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c20c6de0 0x55c6c20c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 4 client.27142 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6c1fadd70 con 0x55c6c20c6de0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c20c6de0 0x55c6c20c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb0ae59c640 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c20c6de0 0x55c6c20c9200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2547667e5016c3f5 server_cookie=9707783e7ab32407 in_seq=0 out_seq=0 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.747+0000 7fb095ffb640 10 client.27142 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb095ffb640 1 -- 172.21.15.77:0/2690975983 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6c1fadd70 con 0x55c6c20c6de0 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb095ffb640 10 client.27142 handle_command_reply: tid=0 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 client.27142 shutdown 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 msgr2=0x55c6c204fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8025940 tx=0x7fb0a800afb0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 shutdown_connections 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c6c20c6de0 0x55c6c20c9200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c6c2048dc0 0x55c6c204fe10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6c2041cf0 0x55c6c204f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 --2- 172.21.15.77:0/2690975983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c2040780 0x55c6c204f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 >> 172.21.15.77:0/2690975983 conn(0x55c6c2009310 msgr2=0x55c6c203beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb095ffb640 0 client.27142 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:23:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 shutdown_connections 2024-09-15T22:23:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 1 -- 172.21.15.77:0/2690975983 wait complete. 2024-09-15T22:23:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.748+0000 7fb0b6c89740 20 client.27142 trim_cache size 0 max 16384 2024-09-15T22:23:17.759 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:23:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 msgr2=0x7fb0980a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 0x7fb0980a5a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005cc0 tx=0x7fb0a800a310 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 msgr2=0x7fb0b010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002a9f0 tx=0x7fb0a0003b90 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 shutdown_connections 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0980a3540 0x7fb0980a5a00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0b010bfc0 0x7fb0b0108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0b01034b0 0x7fb0b0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 --2- 172.21.15.77:0/373749026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0b0102b90 0x7fb0b010b7e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 >> 172.21.15.77:0/373749026 conn(0x7fb0b00fc610 msgr2=0x7fb0b010eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 shutdown_connections 2024-09-15T22:23:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:17.749+0000 7fb0af59e640 1 -- 172.21.15.77:0/373749026 wait complete. 2024-09-15T22:23:18.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 848395 7.69 MB/sec execute 447 sec latency 54.335 ms 2024-09-15T22:23:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:18 smithi196 ceph-mon[35109]: pgmap v338: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:18 smithi117 ceph-mon[35021]: pgmap v338: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:18.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:18 smithi077 ceph-mon[30980]: pgmap v338: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:18.829 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:23:19.107 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:19.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 849296 7.69 MB/sec execute 448 sec latency 49.487 ms 2024-09-15T22:23:19.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 -- 172.21.15.77:0/3778012171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 msgr2=0x7fa1741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 --2- 172.21.15.77:0/3778012171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 0x7fa1741029b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa164009900 tx=0x7fa16402a480 comp rx=0 tx=0).stop 2024-09-15T22:23:19.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 -- 172.21.15.77:0/3778012171 shutdown_connections 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 --2- 172.21.15.77:0/3778012171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 0x7fa17410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 --2- 172.21.15.77:0/3778012171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 0x7fa1741029b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 --2- 172.21.15.77:0/3778012171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa174101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 -- 172.21.15.77:0/3778012171 >> 172.21.15.77:0/3778012171 conn(0x7fa1740fc7d0 msgr2=0x7fa1740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 -- 172.21.15.77:0/3778012171 shutdown_connections 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.620+0000 7fa17b86b640 1 -- 172.21.15.77:0/3778012171 wait complete. 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 Processor -- start 2024-09-15T22:23:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 -- start start 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 0x7fa17419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 0x7fa17419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa174110f80 con 0x7fa174102570 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa174110e00 con 0x7fa174102ef0 2024-09-15T22:23:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa17b86b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa174111100 con 0x7fa174100e20 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.621+0000 7fa179607640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37784/0 (socket says 172.21.15.77:37784) 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 -- 172.21.15.77:0/1406430312 learned_addr learned my addr 172.21.15.77:0/1406430312 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179e08640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 0x7fa17419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 -- 172.21.15.77:0/1406430312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 msgr2=0x7fa17419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 0x7fa17419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 -- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 msgr2=0x7fa17419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 0x7fa17419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.622+0000 7fa179607640 1 -- 172.21.15.77:0/1406430312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419f660 con 0x7fa174100e20 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.623+0000 7fa179607640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e9f0 tx=0x7fa17000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.624+0000 7fa16a7fc640 1 -- 172.21.15.77:0/1406430312 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000cd80 con 0x7fa174100e20 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.624+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419f950 con 0x7fa174100e20 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.624+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a7230 con 0x7fa174100e20 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.624+0000 7fa16a7fc640 1 -- 172.21.15.77:0/1406430312 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa170005500 con 0x7fa174100e20 2024-09-15T22:23:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.624+0000 7fa16a7fc640 1 -- 172.21.15.77:0/1406430312 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170010690 con 0x7fa174100e20 2024-09-15T22:23:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.627+0000 7fa16a7fc640 1 -- 172.21.15.77:0/1406430312 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa170010830 con 0x7fa174100e20 2024-09-15T22:23:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 Processor -- start 2024-09-15T22:23:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 -- start start 2024-09-15T22:23:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 0x565235d90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 0x565235d9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565235cf62f0 con 0x565235d9b370 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565235c9dcf0 con 0x565235d91b40 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.628+0000 7fa17dc44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565235cf22b0 con 0x565235d905d0 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa178e06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 0x565235d9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa178e06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 0x565235d9d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60630/0 (socket says 172.21.15.77:60630) 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa178e06640 1 -- 172.21.15.77:0/1731760969 learned_addr learned my addr 172.21.15.77:0/1731760969 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 0x565235d90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa16a7fc640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 0x7fa1600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179e08640 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 -- 172.21.15.77:0/1731760969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 msgr2=0x565235d90090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d90090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 -- 172.21.15.77:0/1731760969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 msgr2=0x565235d9d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 0x565235d9d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179607640 1 -- 172.21.15.77:0/1731760969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565235c823a0 con 0x565235d905d0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.629+0000 7fa179e08640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 0x7fa1600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.630+0000 7fa179607640 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 0x565235d90a10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1700cd760 tx=0x7fa17009bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b2cf59c5ce69b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.630+0000 7fa15f7fe640 1 -- 172.21.15.77:0/1731760969 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700b1be0 con 0x565235d905d0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.630+0000 7fa15f7fe640 1 -- 172.21.15.77:0/1731760969 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1700855f0 con 0x565235d905d0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.630+0000 7fa15f7fe640 1 -- 172.21.15.77:0/1731760969 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1700b05b0 con 0x565235d905d0 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.630+0000 7fa16a7fc640 1 -- 172.21.15.77:0/1406430312 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa1700d03a0 con 0x7fa174100e20 2024-09-15T22:23:19.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.631+0000 7fa179e08640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 0x7fa1600a58f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005fd0 tx=0x7fa16c013040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 -- 172.21.15.77:0/1731760969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 msgr2=0x565235d90a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 0x565235d90a10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1700cd760 tx=0x7fa17009bd10 comp rx=0 tx=0).stop 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 -- 172.21.15.77:0/1731760969 shutdown_connections 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d9b370 0x565235d9d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d905d0 0x565235d90a10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1731760969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d90090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 -- 172.21.15.77:0/1731760969 >> 172.21.15.77:0/1731760969 conn(0x565235d59310 msgr2=0x565235d8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 -- 172.21.15.77:0/1731760969 shutdown_connections 2024-09-15T22:23:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.632+0000 7fa17dc44740 1 -- 172.21.15.77:0/1731760969 wait complete. 2024-09-15T22:23:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 -- start start 2024-09-15T22:23:19.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 Processor -- start 2024-09-15T22:23:19.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d98600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565235d9e070 con 0x565235d905d0 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565235cb1060 con 0x565235d91b40 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565235d9e1f0 con 0x565235d9b370 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa17dc44740 10 client.0 _ll_register_callbacks cb 0x565235a048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa178e06640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa178e06640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34646/0 (socket says 172.21.15.77:34646) 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa178e06640 1 -- 172.21.15.77:0/970472803 learned_addr learned my addr 172.21.15.77:0/970472803 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.633+0000 7fa179e08640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa179607640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 -- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 msgr2=0x565235d98600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d98600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 -- 172.21.15.77:0/970472803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 msgr2=0x565235d9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d9ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 -- 172.21.15.77:0/970472803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565235a673c0 con 0x565235d91b40 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.634+0000 7fa178e06640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1640099f0 tx=0x7fa164004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140f6b90b080a125 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.635+0000 7fa15cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.635+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164040070 con 0x565235d91b40 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.635+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa164003a00 con 0x565235d91b40 2024-09-15T22:23:19.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.635+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1700855f0 con 0x565235d91b40 2024-09-15T22:23:19.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa16404e050 con 0x565235d91b40 2024-09-15T22:23:19.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 10 client.27215 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:19.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565235da9b70 con 0x565235d91b40 2024-09-15T22:23:19.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1700855f0 con 0x565235d91b40 2024-09-15T22:23:19.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa164031ca0 con 0x565235d91b40 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 10 client.27215 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 10 client.27215 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 20 client.27215 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235dab0d0 0x565235dad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 4 client.27215 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.636+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565235d03860 con 0x565235dab0d0 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.637+0000 7fa179607640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235dab0d0 0x565235dad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.637+0000 7fa15cff9640 10 client.27215 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:23:19.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.637+0000 7fa179607640 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235dab0d0 0x565235dad4c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bc19d29f9654072 server_cookie=e4404d21e2122c72 in_seq=0 out_seq=0 2024-09-15T22:23:19.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa15cff9640 1 -- 172.21.15.77:0/970472803 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565235d03860 con 0x565235dab0d0 2024-09-15T22:23:19.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa15cff9640 10 client.27215 handle_command_reply: tid=0 2024-09-15T22:23:19.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 client.27215 shutdown 2024-09-15T22:23:19.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 msgr2=0x565235d98010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1640099f0 tx=0x7fa164004270 comp rx=0 tx=0).stop 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 shutdown_connections 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235dab0d0 0x565235dad4c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d91b40 0x565235d98010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 --2- 172.21.15.77:0/970472803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 >> 172.21.15.77:0/970472803 conn(0x565235d59310 msgr2=0x565235d92930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.639+0000 7fa15cff9640 0 client.27215 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.640+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 shutdown_connections 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.640+0000 7fa17dc44740 1 -- 172.21.15.77:0/970472803 wait complete. 2024-09-15T22:23:19.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.640+0000 7fa17dc44740 20 client.27215 trim_cache size 0 max 16384 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 Processor -- start 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 -- start start 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d8ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 0x565235da7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d9b370 0x565235db2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa15800ff60 con 0x565235d905d0 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa158011aa0 con 0x565235d9b370 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa17dc44740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa158010630 con 0x565235d91b40 2024-09-15T22:23:19.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa179607640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d8ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa179607640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d8ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60652/0 (socket says 172.21.15.77:60652) 2024-09-15T22:23:19.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa179607640 1 -- 172.21.15.77:0/1954815629 learned_addr learned my addr 172.21.15.77:0/1954815629 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:19.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa178e06640 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 0x565235da7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.662+0000 7fa179e08640 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d9b370 0x565235db2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 -- 172.21.15.77:0/1954815629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d9b370 msgr2=0x565235db2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d9b370 0x565235db2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 -- 172.21.15.77:0/1954815629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 msgr2=0x565235d8ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d8ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 -- 172.21.15.77:0/1954815629 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565235d98970 con 0x565235d91b40 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa178e06640 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 0x565235da7340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x565235d99650 tx=0x7fa16402ab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c7ab2e4e52bb82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa15effd640 1 -- 172.21.15.77:0/1954815629 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164040040 con 0x565235d91b40 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa15effd640 1 -- 172.21.15.77:0/1954815629 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa164009520 con 0x565235d91b40 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.663+0000 7fa15effd640 1 -- 172.21.15.77:0/1954815629 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164035040 con 0x565235d91b40 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 -- 172.21.15.77:0/1954815629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 msgr2=0x565235da7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 0x565235da7340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x565235d99650 tx=0x7fa16402ab00 comp rx=0 tx=0).stop 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 -- 172.21.15.77:0/1954815629 shutdown_connections 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d9b370 0x565235db2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d91b40 0x565235da7340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 --2- 172.21.15.77:0/1954815629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d905d0 0x565235d8ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 -- 172.21.15.77:0/1954815629 >> 172.21.15.77:0/1954815629 conn(0x565235d59310 msgr2=0x565235d8c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 -- 172.21.15.77:0/1954815629 shutdown_connections 2024-09-15T22:23:19.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.664+0000 7fa17dc44740 1 -- 172.21.15.77:0/1954815629 wait complete. 2024-09-15T22:23:19.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.665+0000 7fa17dc44740 1 -- start start 2024-09-15T22:23:19.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.665+0000 7fa17dc44740 1 Processor -- start 2024-09-15T22:23:19.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.665+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 0x565235da3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.665+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 0x565235d94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179607640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 0x565235da3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa17dc44740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa178e06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 0x565235d94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179607640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 0x565235da3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34670/0 (socket says 172.21.15.77:34670) 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179607640 1 -- 172.21.15.77:0/689242063 learned_addr learned my addr 172.21.15.77:0/689242063 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1580113d0 con 0x565235d91b40 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa158010d00 con 0x565235d905d0 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa15800df60 con 0x565235d9b370 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 -- 172.21.15.77:0/689242063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 msgr2=0x565235da3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 0x565235da3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 -- 172.21.15.77:0/689242063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 msgr2=0x565235d94e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 0x565235d94e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa179e08640 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565235d95ba0 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa178e06640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 0x565235d94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.666+0000 7fa17dc44740 10 client.0 _ll_register_callbacks cb 0x7fa170005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa179e08640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d95370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa16c01eb20 tx=0x7fa16c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d90d3370ff2720cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa15cff9640 10 client.27160 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa164009520 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa15cff9640 1 -- 172.21.15.77:0/689242063 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01ce00 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa15cff9640 1 -- 172.21.15.77:0/689242063 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa16c0283f0 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa15cff9640 1 -- 172.21.15.77:0/689242063 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa16c028630 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa17dc44740 10 client.27160 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.667+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=34}) -- 0x565235e15cd0 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa15cff9640 1 -- 172.21.15.77:0/689242063 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa16c0287d0 con 0x565235d9b370 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 10 client.27160 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 10 client.27160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:23:19.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 20 client.27160 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235e16310 0x565235e18770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 4 client.27160 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa164009520 con 0x565235e16310 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa179607640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235e16310 0x565235e18770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa15cff9640 10 client.27160 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.668+0000 7fa179607640 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235e16310 0x565235e18770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc139a6c65b2b56 server_cookie=39dfdfa34a280e6d in_seq=0 out_seq=0 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa15cff9640 1 -- 172.21.15.77:0/689242063 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa164009520 con 0x565235e16310 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa15cff9640 10 client.27160 handle_command_reply: tid=0 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 client.27160 shutdown 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 msgr2=0x565235d95370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d95370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa16c01eb20 tx=0x7fa16c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 shutdown_connections 2024-09-15T22:23:19.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565235e16310 0x565235e18770 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565235d9b370 0x565235d95370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565235d91b40 0x565235d94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 --2- 172.21.15.77:0/689242063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565235d905d0 0x565235da3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 >> 172.21.15.77:0/689242063 conn(0x565235d59310 msgr2=0x565235d9a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa15cff9640 0 client.27160 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 shutdown_connections 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 1 -- 172.21.15.77:0/689242063 wait complete. 2024-09-15T22:23:19.676 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:23:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.669+0000 7fa17dc44740 20 client.27160 trim_cache size 0 max 16384 2024-09-15T22:23:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 msgr2=0x7fa1600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 0x7fa1600a58f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005fd0 tx=0x7fa16c013040 comp rx=0 tx=0).stop 2024-09-15T22:23:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 msgr2=0x7fa17419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:19.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e9f0 tx=0x7fa17000edf0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 shutdown_connections 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1600a3430 0x7fa1600a58f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa174102ef0 0x7fa17419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa174102570 0x7fa17419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 --2- 172.21.15.77:0/1406430312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa174100e20 0x7fa17419a6b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 >> 172.21.15.77:0/1406430312 conn(0x7fa1740fc7d0 msgr2=0x7fa17406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 shutdown_connections 2024-09-15T22:23:19.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:19.670+0000 7fa17b86b640 1 -- 172.21.15.77:0/1406430312 wait complete. 2024-09-15T22:23:20.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:19 smithi196 ceph-mon[35109]: pgmap v339: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:20.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:19 smithi196 ceph-mon[35109]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:23:20.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:19 smithi117 ceph-mon[35021]: pgmap v339: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:20.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:19 smithi117 ceph-mon[35021]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:23:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:19 smithi077 ceph-mon[30980]: pgmap v339: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 208 op/s 2024-09-15T22:23:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:19 smithi077 ceph-mon[30980]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:23:20.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 851573 7.70 MB/sec execute 449 sec latency 40.750 ms 2024-09-15T22:23:20.707 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:23:20.966 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:21.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 852709 7.70 MB/sec execute 450 sec latency 102.000 ms 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.636+0000 7f15d20f2640 1 -- 172.21.15.77:0/115683006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc103650 msgr2=0x7f15cc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.636+0000 7f15d20f2640 1 --2- 172.21.15.77:0/115683006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc103650 0x7f15cc10bb40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009860 tx=0x7f15bc02a3d0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 -- 172.21.15.77:0/115683006 shutdown_connections 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 --2- 172.21.15.77:0/115683006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc10c160 0x7f15cc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 --2- 172.21.15.77:0/115683006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc103650 0x7f15cc10bb40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 --2- 172.21.15.77:0/115683006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 -- 172.21.15.77:0/115683006 >> 172.21.15.77:0/115683006 conn(0x7f15cc0fc7d0 msgr2=0x7f15cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.637+0000 7f15d20f2640 1 -- 172.21.15.77:0/115683006 shutdown_connections 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.638+0000 7f15d20f2640 1 -- 172.21.15.77:0/115683006 wait complete. 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.638+0000 7f15d20f2640 1 Processor -- start 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 -- start start 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 0x7f15cc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc10c160 0x7f15cc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc111f60 con 0x7f15cc103650 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc111de0 con 0x7f15cc102d30 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15d20f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1120e0 con 0x7f15cc10c160 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34696/0 (socket says 172.21.15.77:34696) 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.639+0000 7f15cb7fe640 1 -- 172.21.15.77:0/4272178555 learned_addr learned my addr 172.21.15.77:0/4272178555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15caffd640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 0x7f15cc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 -- 172.21.15.77:0/4272178555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc10c160 msgr2=0x7f15cc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc10c160 0x7f15cc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 -- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 msgr2=0x7f15cc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 0x7f15cc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 -- 172.21.15.77:0/4272178555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc1a69b0 con 0x7f15cc102d30 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15caffd640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 0x7f15cc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.640+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9f0 tx=0x7f15c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.641+0000 7f15c8ff9640 1 -- 172.21.15.77:0/4272178555 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000cd80 con 0x7f15cc102d30 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.641+0000 7f15c8ff9640 1 -- 172.21.15.77:0/4272178555 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0010910 con 0x7f15cc102d30 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.641+0000 7f15c8ff9640 1 -- 172.21.15.77:0/4272178555 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0018990 con 0x7f15cc102d30 2024-09-15T22:23:21.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.641+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc1a6ca0 con 0x7f15cc102d30 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.641+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1a7190 con 0x7f15cc102d30 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.644+0000 7f15d44cb740 1 Processor -- start 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.644+0000 7f15c8ff9640 1 -- 172.21.15.77:0/4272178555 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f15c0010ac0 con 0x7f15cc102d30 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.644+0000 7f15d44cb740 1 -- start start 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f705105d0 0x558f70510a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f70525f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558f704762f0 con 0x558f70511b40 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f7041dcf0 con 0x558f70523b80 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.645+0000 7f15d44cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f704722b0 con 0x558f705105d0 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15c8ff9640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 0x7f15b40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60678/0 (socket says 172.21.15.77:60678) 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 -- 172.21.15.77:0/1128590044 learned_addr learned my addr 172.21.15.77:0/1128590044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15c8ff9640 1 -- 172.21.15.77:0/4272178555 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f15c00d03a0 con 0x7f15cc102d30 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f705105d0 0x558f70510a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 -- 172.21.15.77:0/1128590044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f705105d0 msgr2=0x558f70510a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f705105d0 0x558f70510a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 -- 172.21.15.77:0/1128590044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 msgr2=0x558f70525f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f70525f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 -- 172.21.15.77:0/1128590044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f704023a0 con 0x558f70511b40 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.646+0000 7f15cbfff640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 0x7f15b40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.647+0000 7f15cbfff640 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009e40 tx=0x7f15c4010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8f0722bc02ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.647+0000 7f15cbfff640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 0x7f15b40a5a40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f15c401f000 tx=0x7f15c401c000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.648+0000 7f15b1ffb640 1 -- 172.21.15.77:0/1128590044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400c790 con 0x558f70511b40 2024-09-15T22:23:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.648+0000 7f15b1ffb640 1 -- 172.21.15.77:0/1128590044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c4011540 con 0x558f70511b40 2024-09-15T22:23:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.648+0000 7f15b1ffb640 1 -- 172.21.15.77:0/1128590044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4019da0 con 0x558f70511b40 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 -- 172.21.15.77:0/1128590044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 msgr2=0x558f70510090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009e40 tx=0x7f15c4010780 comp rx=0 tx=0).stop 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 -- 172.21.15.77:0/1128590044 shutdown_connections 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f70525f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f705105d0 0x558f70510a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 --2- 172.21.15.77:0/1128590044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70511b40 0x558f70510090 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 -- 172.21.15.77:0/1128590044 >> 172.21.15.77:0/1128590044 conn(0x558f704d9310 msgr2=0x558f7050cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 -- 172.21.15.77:0/1128590044 shutdown_connections 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.649+0000 7f15d44cb740 1 -- 172.21.15.77:0/1128590044 wait complete. 2024-09-15T22:23:21.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.650+0000 7f15d44cb740 1 -- start start 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.650+0000 7f15d44cb740 1 Processor -- start 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.650+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f705105d0 0x558f70523530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.650+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f70520830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.650+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15d44cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558f705268d0 con 0x558f70523b80 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60690/0 (socket says 172.21.15.77:60690) 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 -- 172.21.15.77:0/620826355 learned_addr learned my addr 172.21.15.77:0/620826355 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15caffd640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f70520830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f70431060 con 0x558f705105d0 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558f70526a50 con 0x558f70511b40 2024-09-15T22:23:21.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15d44cb740 10 client.0 _ll_register_callbacks cb 0x558f701848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f705105d0 0x558f70523530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 -- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 msgr2=0x558f70520830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f70520830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 -- 172.21.15.77:0/620826355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f705105d0 msgr2=0x558f70523530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f705105d0 0x558f70523530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.651+0000 7f15cbfff640 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f701e73c0 con 0x558f70523b80 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.653+0000 7f15cbfff640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f15c401ad60 tx=0x7f15c401ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad4c3b4a1a89baf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.653+0000 7f15aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c401ae30 con 0x558f70523b80 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f70483860 con 0x558f70523b80 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f15c400c7a0 con 0x558f70523b80 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f15c400c9a0 con 0x558f70523b80 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15d44cb740 10 client.18195 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.654+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f70532320 con 0x558f70523b80 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f70483860 con 0x558f70523b80 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f15c400b040 con 0x558f70523b80 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 10 client.18195 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 10 client.18195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 20 client.18195 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f70533730 0x558f70535b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 4 client.18195 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.655+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f7047dd70 con 0x558f70533730 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.656+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f70533730 0x558f70535b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.657+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f70533730 0x558f70535b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a59a9217f79720d server_cookie=9d25b222b70fd3db in_seq=0 out_seq=0 2024-09-15T22:23:21.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.657+0000 7f15aaffd640 10 client.18195 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.660+0000 7f15aaffd640 1 -- 172.21.15.77:0/620826355 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f7047dd70 con 0x558f70533730 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.660+0000 7f15aaffd640 10 client.18195 handle_command_reply: tid=0 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 client.18195 shutdown 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 msgr2=0x558f70520e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f15c401ad60 tx=0x7f15c401ad90 comp rx=0 tx=0).stop 2024-09-15T22:23:21.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 shutdown_connections 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f70533730 0x558f70535b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f70523b80 0x558f70520e40 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f70520830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 --2- 172.21.15.77:0/620826355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f705105d0 0x558f70523530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.661+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 >> 172.21.15.77:0/620826355 conn(0x558f704d9310 msgr2=0x558f70512960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.662+0000 7f15aaffd640 0 client.18195 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:23:21.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.662+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 shutdown_connections 2024-09-15T22:23:21.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.662+0000 7f15d44cb740 1 -- 172.21.15.77:0/620826355 wait complete. 2024-09-15T22:23:21.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.663+0000 7f15d44cb740 20 client.18195 trim_cache size 0 max 16384 2024-09-15T22:23:21.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 Processor -- start 2024-09-15T22:23:21.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 -- start start 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 0x558f7051f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70523b80 0x558f7053b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1acf10 con 0x558f705105d0 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1aea50 con 0x558f70511b40 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15d44cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ad5e0 con 0x558f70523b80 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60702/0 (socket says 172.21.15.77:60702) 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3046138065 learned_addr learned my addr 172.21.15.77:0/3046138065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15caffd640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 0x558f7051f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.684+0000 7f15cbfff640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70523b80 0x558f7053b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3046138065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70523b80 msgr2=0x558f7053b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70523b80 0x558f7053b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3046138065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 msgr2=0x558f7051f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 0x558f7051f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3046138065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f705262e0 con 0x558f705105d0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.685+0000 7f15caffd640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 0x558f7051f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.686+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f15c00986f0 tx=0x7f15c000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c98dc93dd4e1df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.687+0000 7f15b17fa640 1 -- 172.21.15.77:0/3046138065 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c009c290 con 0x558f705105d0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.687+0000 7f15b17fa640 1 -- 172.21.15.77:0/3046138065 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c00cde50 con 0x558f705105d0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.687+0000 7f15b17fa640 1 -- 172.21.15.77:0/3046138065 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c009a330 con 0x558f705105d0 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 -- 172.21.15.77:0/3046138065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 msgr2=0x558f7052fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f15c00986f0 tx=0x7f15c000cf20 comp rx=0 tx=0).stop 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 -- 172.21.15.77:0/3046138065 shutdown_connections 2024-09-15T22:23:21.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70523b80 0x558f7053b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70511b40 0x558f7051f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.688+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3046138065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052fb70 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.689+0000 7f15d44cb740 1 -- 172.21.15.77:0/3046138065 >> 172.21.15.77:0/3046138065 conn(0x558f704d9310 msgr2=0x558f7051eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.689+0000 7f15d44cb740 1 -- 172.21.15.77:0/3046138065 shutdown_connections 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.689+0000 7f15d44cb740 1 -- 172.21.15.77:0/3046138065 wait complete. 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.689+0000 7f15d44cb740 1 -- start start 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 Processor -- start 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f7052c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f7051cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ae380 con 0x558f705105d0 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1adcb0 con 0x558f70523b80 2024-09-15T22:23:21.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1aaf10 con 0x558f70511b40 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.690+0000 7f15d44cb740 10 client.0 _ll_register_callbacks cb 0x7f15c4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60714/0 (socket says 172.21.15.77:60714) 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3170556793 learned_addr learned my addr 172.21.15.77:0/3170556793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15caffd640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f7052c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 msgr2=0x558f7052c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f7052c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3170556793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 msgr2=0x558f7051cea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f7051cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 -- 172.21.15.77:0/3170556793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7051d460 con 0x558f705105d0 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15cb7fe640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c6a0 tx=0x7f15c009c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85984109874cfdd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:21.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.691+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00b0060 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f15c0098900 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c00cde50 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f15c00a0640 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 10 client.18201 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f7059d9c0 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c00cde50 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f15c00cd980 con 0x558f705105d0 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 10 client.18201 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 10 client.18201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 20 client.18201 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:21.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f7059dff0 0x558f705a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 4 client.18201 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.692+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f703fa170 con 0x558f7059dff0 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.693+0000 7f15caffd640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f7059dff0 0x558f705a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.693+0000 7f15caffd640 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f7059dff0 0x558f705a0410 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58c8acdfda79fa6b server_cookie=88e9265562052d32 in_seq=0 out_seq=0 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.693+0000 7f15aaffd640 10 client.18201 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.695+0000 7f15aaffd640 1 -- 172.21.15.77:0/3170556793 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f703fa170 con 0x558f7059dff0 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.695+0000 7f15aaffd640 10 client.18201 handle_command_reply: tid=0 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.695+0000 7f15d44cb740 1 client.18201 shutdown 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 msgr2=0x558f7052bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c6a0 tx=0x7f15c009c190 comp rx=0 tx=0).stop 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 shutdown_connections 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558f7059dff0 0x558f705a0410 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f70523b80 0x558f7051cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15aaffd640 0 client.18201 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f70511b40 0x558f7052c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 --2- 172.21.15.77:0/3170556793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f705105d0 0x558f7052bce0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 >> 172.21.15.77:0/3170556793 conn(0x558f704d9310 msgr2=0x558f70512110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 shutdown_connections 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 1 -- 172.21.15.77:0/3170556793 wait complete. 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.696+0000 7f15d44cb740 20 client.18201 trim_cache size 0 max 16384 2024-09-15T22:23:21.707 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 msgr2=0x7f15b40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 0x7f15b40a5a40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f15c401f000 tx=0x7f15c401c000 comp rx=0 tx=0).stop 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 msgr2=0x7f15cc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9f0 tx=0x7f15c000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 shutdown_connections 2024-09-15T22:23:21.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f15b40a3580 0x7f15b40a5a40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15cc10c160 0x7f15cc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15cc103650 0x7f15cc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 --2- 172.21.15.77:0/4272178555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15cc102d30 0x7f15cc100990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 >> 172.21.15.77:0/4272178555 conn(0x7f15cc0fc7d0 msgr2=0x7f15cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 shutdown_connections 2024-09-15T22:23:21.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:21.697+0000 7f15d20f2640 1 -- 172.21.15.77:0/4272178555 wait complete. 2024-09-15T22:23:22.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:21 smithi196 ceph-mon[35109]: pgmap v341: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-09-15T22:23:22.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:21 smithi117 ceph-mon[35021]: pgmap v341: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-09-15T22:23:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:21 smithi077 ceph-mon[30980]: pgmap v341: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 230 op/s 2024-09-15T22:23:22.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:23:22.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 855026 7.71 MB/sec execute 451 sec latency 16.252 ms 2024-09-15T22:23:22.641 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 -- 172.21.15.77:0/1374190888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f4106f20 msgr2=0x7fc8f410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1374190888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f4106f20 0x7fc8f410eaa0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 -- 172.21.15.77:0/1374190888 shutdown_connections 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1374190888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f4106f20 0x7fc8f410eaa0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1374190888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f41027a0 0x7fc8f41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1374190888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 -- 172.21.15.77:0/1374190888 >> 172.21.15.77:0/1374190888 conn(0x7fc8f40fd8f0 msgr2=0x7fc8f40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 -- 172.21.15.77:0/1374190888 shutdown_connections 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.137+0000 7fc8f9708640 1 -- 172.21.15.77:0/1374190888 wait complete. 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 Processor -- start 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 -- start start 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4106f20 0x7fc8f419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111f80 con 0x7fc8f41027a0 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4111e00 con 0x7fc8f4101df0 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f9708640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112100 con 0x7fc8f4106f20 2024-09-15T22:23:23.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60742/0 (socket says 172.21.15.77:60742) 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1995256057 learned_addr learned my addr 172.21.15.77:0/1995256057 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.138+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4106f20 0x7fc8f419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1995256057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4106f20 msgr2=0x7fc8f419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4106f20 0x7fc8f419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1995256057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 msgr2=0x7fc8f419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1995256057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f419f5a0 con 0x7fc8f41027a0 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:23.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800e9f0 tx=0x7fc8e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8dbfff640 1 -- 172.21.15.77:0/1995256057 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800cd80 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8dbfff640 1 -- 172.21.15.77:0/1995256057 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e8005500 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f419f890 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8dbfff640 1 -- 172.21.15.77:0/1995256057 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8010690 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.139+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a7170 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.141+0000 7fc8fbae1740 1 Processor -- start 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.141+0000 7fc8dbfff640 1 -- 172.21.15.77:0/1995256057 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc8e80046c0 con 0x7fc8f41027a0 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.141+0000 7fc8fbae1740 1 -- start start 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b447b172f0 con 0x55b447bb2b40 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b447abecf0 con 0x55b447bb15d0 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b447b132b0 con 0x55b447bbc370 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60744/0 (socket says 172.21.15.77:60744) 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34770/0 (socket says 172.21.15.77:34770) 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 -- 172.21.15.77:0/3238542928 learned_addr learned my addr 172.21.15.77:0/3238542928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 -- 172.21.15.77:0/3238542928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 msgr2=0x55b447bbe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bbe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 -- 172.21.15.77:0/3238542928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 msgr2=0x55b447bb1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 -- 172.21.15.77:0/3238542928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b447aa33a0 con 0x55b447bb2b40 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8dbfff640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 0x7fc8dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.142+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f494592f9c88e33e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 0x7fc8dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8dbfff640 1 -- 172.21.15.77:0/1995256057 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc8e80d0450 con 0x7fc8f41027a0 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8d97fa640 1 -- 172.21.15.77:0/3238542928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec031db0 con 0x55b447bb2b40 2024-09-15T22:23:23.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8d97fa640 1 -- 172.21.15.77:0/3238542928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec036040 con 0x55b447bb2b40 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8d97fa640 1 -- 172.21.15.77:0/3238542928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec004330 con 0x55b447bb2b40 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8fbae1740 1 -- 172.21.15.77:0/3238542928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 msgr2=0x55b447bb1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec005f10 comp rx=0 tx=0).stop 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.143+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 0x7fc8dc0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fd0 tx=0x7fc8e4005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- 172.21.15.77:0/3238542928 shutdown_connections 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bb1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/3238542928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb1090 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- 172.21.15.77:0/3238542928 >> 172.21.15.77:0/3238542928 conn(0x55b447b7a310 msgr2=0x55b447badc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- 172.21.15.77:0/3238542928 shutdown_connections 2024-09-15T22:23:23.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- 172.21.15.77:0/3238542928 wait complete. 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- start start 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 Processor -- start 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b447bbee90 con 0x55b447bb2b40 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b447ad2060 con 0x55b447bb15d0 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b447bbf010 con 0x55b447bbc370 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8fbae1740 10 client.0 _ll_register_callbacks cb 0x55b4478258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.144+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60748/0 (socket says 172.21.15.77:60748) 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f27fc640 1 -- 172.21.15.77:0/568329487 learned_addr learned my addr 172.21.15.77:0/568329487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 -- 172.21.15.77:0/568329487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 msgr2=0x55b447bb97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 -- 172.21.15.77:0/568329487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 msgr2=0x55b447bb9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 -- 172.21.15.77:0/568329487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4478883c0 con 0x55b447bb15d0 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bbbda0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4012d70 tx=0x7fc8e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2320d311f435f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4004050 con 0x55b447bb15d0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec036040 con 0x55b447bb15d0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc8e40041f0 con 0x55b447bb15d0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40043f0 con 0x55b447bb15d0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8fbae1740 10 client.27245 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.145+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b447bcaa20 con 0x55b447bb15d0 2024-09-15T22:23:23.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec036040 con 0x55b447bb15d0 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc8e4027020 con 0x55b447bb15d0 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 10 client.27245 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 10 client.27245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 20 client.27245 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447bcbe30 0x55b447bce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 4 client.27245 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b447b24860 con 0x55b447bcbe30 2024-09-15T22:23:23.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447bcbe30 0x55b447bce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8d6ff5640 10 client.27245 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.146+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447bcbe30 0x55b447bce220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=707bff1b77648c13 server_cookie=5939867dba74bc8b in_seq=0 out_seq=0 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/568329487 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b447b24860 con 0x55b447bcbe30 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8d6ff5640 10 client.27245 handle_command_reply: tid=0 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 client.27245 shutdown 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 msgr2=0x55b447bbbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bbbda0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4012d70 tx=0x7fc8e4004730 comp rx=0 tx=0).stop 2024-09-15T22:23:23.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 shutdown_connections 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447bcbe30 0x55b447bce220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bb9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8d6ff5640 0 client.27245 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/568329487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb15d0 0x55b447bbbda0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 >> 172.21.15.77:0/568329487 conn(0x55b447b7a310 msgr2=0x55b447bb3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.147+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 shutdown_connections 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.148+0000 7fc8fbae1740 1 -- 172.21.15.77:0/568329487 wait complete. 2024-09-15T22:23:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.148+0000 7fc8fbae1740 20 client.27245 trim_cache size 0 max 16384 2024-09-15T22:23:23.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 Processor -- start 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 -- start start 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 0x55b447bb2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b10f0 con 0x55b447bb15d0 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b2c30 con 0x55b447bb2b40 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.162+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b17c0 con 0x55b447bbc370 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 0x55b447bb2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34782/0 (socket says 172.21.15.77:34782) 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1869936388 learned_addr learned my addr 172.21.15.77:0/1869936388 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1869936388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 msgr2=0x55b447bb7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1869936388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 msgr2=0x55b447bb2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 0x55b447bb2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 -- 172.21.15.77:0/1869936388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b447bbead0 con 0x55b447bb2b40 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 0x55b447bb2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8004c00 tx=0x7fc8e80bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13740150fc352a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8d8ff9640 1 -- 172.21.15.77:0/1869936388 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e809bee0 con 0x55b447bb2b40 2024-09-15T22:23:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8d8ff9640 1 -- 172.21.15.77:0/1869936388 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e809caa0 con 0x55b447bb2b40 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.163+0000 7fc8d8ff9640 1 -- 172.21.15.77:0/1869936388 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e809bd00 con 0x55b447bb2b40 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- 172.21.15.77:0/1869936388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 msgr2=0x55b447bb7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8004c00 tx=0x7fc8e80bc040 comp rx=0 tx=0).stop 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- 172.21.15.77:0/1869936388 shutdown_connections 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bbc370 0x55b447bb7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bb2b40 0x55b447bb7920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/1869936388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb15d0 0x55b447bb2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- 172.21.15.77:0/1869936388 >> 172.21.15.77:0/1869936388 conn(0x55b447b7a310 msgr2=0x55b447ba8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- 172.21.15.77:0/1869936388 shutdown_connections 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- 172.21.15.77:0/1869936388 wait complete. 2024-09-15T22:23:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.164+0000 7fc8fbae1740 1 -- start start 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 Processor -- start 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bb15d0 0x55b447bbfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bbc370 0x55b447bc40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b2560 con 0x55b447bb2b40 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b1e90 con 0x55b447bbc370 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41af0f0 con 0x55b447bb15d0 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8fbae1740 10 client.0 _ll_register_callbacks cb 0x7fc8e4009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60776/0 (socket says 172.21.15.77:60776) 2024-09-15T22:23:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 -- 172.21.15.77:0/822278882 learned_addr learned my addr 172.21.15.77:0/822278882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bb15d0 0x55b447bbfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f37fe640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bbc370 0x55b447bc40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 -- 172.21.15.77:0/822278882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bb15d0 msgr2=0x55b447bbfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bb15d0 0x55b447bbfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 -- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bbc370 msgr2=0x55b447bc40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bbc370 0x55b447bc40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 -- 172.21.15.77:0/822278882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b447bc4730 con 0x55b447bb2b40 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8f27fc640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80049d0 tx=0x7fc8e809c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd416c472f55b915 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:23.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.165+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e809bee0 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc8e80cda90 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e809caa0 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8e80a0750 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 10 client.18219 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b447c36590 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e809caa0 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc8e80a4b80 con 0x55b447bb2b40 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 10 client.18219 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 10 client.18219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:23:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 20 client.18219 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447c36d00 0x55b447c390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 4 client.18219 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b447b1ed70 con 0x55b447c36d00 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.166+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447c36d00 0x55b447c390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8f2ffd640 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447c36d00 0x55b447c390f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6a0780967d35e5c server_cookie=310c3019ea23eb20 in_seq=0 out_seq=0 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8d6ff5640 10 client.18219 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8d6ff5640 1 -- 172.21.15.77:0/822278882 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b447b1ed70 con 0x55b447c36d00 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8d6ff5640 10 client.18219 handle_command_reply: tid=0 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 client.18219 shutdown 2024-09-15T22:23:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 msgr2=0x55b447bc0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80049d0 tx=0x7fc8e809c3a0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 shutdown_connections 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55b447c36d00 0x55b447c390f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b447bbc370 0x55b447bc40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b447bb2b40 0x55b447bc0020 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 --2- 172.21.15.77:0/822278882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b447bb15d0 0x55b447bbfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 >> 172.21.15.77:0/822278882 conn(0x55b447b7a310 msgr2=0x55b447bb78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8d6ff5640 0 client.18219 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 shutdown_connections 2024-09-15T22:23:23.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 1 -- 172.21.15.77:0/822278882 wait complete. 2024-09-15T22:23:23.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.167+0000 7fc8fbae1740 20 client.18219 trim_cache size 0 max 16384 2024-09-15T22:23:23.185 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 msgr2=0x7fc8dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 0x7fc8dc0a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009fd0 tx=0x7fc8e4005f90 comp rx=0 tx=0).stop 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 msgr2=0x7fc8f419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800e9f0 tx=0x7fc8e800edf0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 shutdown_connections 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8dc0a33b0 0x7fc8dc0a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4106f20 0x7fc8f419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8f41027a0 0x7fc8f419ab30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 --2- 172.21.15.77:0/1995256057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4101df0 0x7fc8f419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 >> 172.21.15.77:0/1995256057 conn(0x7fc8f40fd8f0 msgr2=0x7fc8f40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 shutdown_connections 2024-09-15T22:23:23.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:23.168+0000 7fc8f9708640 1 -- 172.21.15.77:0/1995256057 wait complete. 2024-09-15T22:23:23.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 856201 7.70 MB/sec execute 452 sec latency 101.938 ms 2024-09-15T22:23:23.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:23:24.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:23 smithi077 ceph-mon[30980]: pgmap v342: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:24.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:23 smithi196 ceph-mon[35109]: pgmap v342: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:24.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:23 smithi117 ceph-mon[35021]: pgmap v342: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:24.239 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:24.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 858413 7.71 MB/sec execute 453 sec latency 14.197 ms 2024-09-15T22:23:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 -- 172.21.15.77:0/4207738533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 msgr2=0x7fca2810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 --2- 172.21.15.77:0/4207738533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2810ec40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fca20009cd0 tx=0x7fca2002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 -- 172.21.15.77:0/4207738533 shutdown_connections 2024-09-15T22:23:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 --2- 172.21.15.77:0/4207738533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2810ec40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 --2- 172.21.15.77:0/4207738533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 --2- 172.21.15.77:0/4207738533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 0x7fca28103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 -- 172.21.15.77:0/4207738533 >> 172.21.15.77:0/4207738533 conn(0x7fca280fc7d0 msgr2=0x7fca280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 -- 172.21.15.77:0/4207738533 shutdown_connections 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.729+0000 7fca2e11d640 1 -- 172.21.15.77:0/4207738533 wait complete. 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 Processor -- start 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 -- start start 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 0x7fca28198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca281120a0 con 0x7fca281082a0 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca27fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca27fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60800/0 (socket says 172.21.15.77:60800) 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37954/0 (socket says 172.21.15.77:37954) 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca27fff640 1 -- 172.21.15.77:0/1539758090 learned_addr learned my addr 172.21.15.77:0/1539758090 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca28111f20 con 0x7fca28102d30 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca28112220 con 0x7fca281036e0 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.730+0000 7fca277fe640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 0x7fca28198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 -- 172.21.15.77:0/1539758090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 msgr2=0x7fca28198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 0x7fca28198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 -- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 msgr2=0x7fca2819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 -- 172.21.15.77:0/1539758090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2819d510 con 0x7fca281036e0 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca26ffd640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca280697d0 tx=0x7fca1000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca24ff9640 1 -- 172.21.15.77:0/1539758090 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1000cc40 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2819d800 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca24ff9640 1 -- 172.21.15.77:0/1539758090 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1000cde0 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca24ff9640 1 -- 172.21.15.77:0/1539758090 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10004a00 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.731+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281008c0 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca24ff9640 1 -- 172.21.15.77:0/1539758090 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fca10004ba0 con 0x7fca281036e0 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 Processor -- start 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 -- start start 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3721760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fe367a2f0 con 0x561fe371f370 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561fe3621cf0 con 0x561fe3715b40 2024-09-15T22:23:24.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca304f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fe36762b0 con 0x561fe37145d0 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca27fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca277fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3721760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca27fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34804/0 (socket says 172.21.15.77:34804) 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca27fff640 1 -- 172.21.15.77:0/3911664200 learned_addr learned my addr 172.21.15.77:0/3911664200 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.734+0000 7fca277fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37970/0 (socket says 172.21.15.77:37970) 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 -- 172.21.15.77:0/3911664200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 msgr2=0x561fe3714a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 -- 172.21.15.77:0/3911664200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 msgr2=0x561fe3721760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3721760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca24ff9640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 0x7fca140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 -- 172.21.15.77:0/3911664200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe36063a0 con 0x561fe3715b40 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca277fe640 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca26ffd640 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3721760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca277fe640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 0x7fca140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca24ff9640 1 -- 172.21.15.77:0/1539758090 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca100cff50 con 0x7fca281036e0 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca27fff640 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca20009a20 tx=0x7fca20005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfaa5890b292f0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca1dffb640 1 -- 172.21.15.77:0/3911664200 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20048070 con 0x561fe3715b40 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca277fe640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 0x7fca140a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x561fe3711ac0 tx=0x7fca18012040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca1dffb640 1 -- 172.21.15.77:0/3911664200 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20036070 con 0x561fe3715b40 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.735+0000 7fca1dffb640 1 -- 172.21.15.77:0/3911664200 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004390 con 0x561fe3715b40 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 -- 172.21.15.77:0/3911664200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 msgr2=0x561fe3714090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca20009a20 tx=0x7fca20005ef0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 -- 172.21.15.77:0/3911664200 shutdown_connections 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3721760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe37145d0 0x561fe3714a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 --2- 172.21.15.77:0/3911664200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe3715b40 0x561fe3714090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 -- 172.21.15.77:0/3911664200 >> 172.21.15.77:0/3911664200 conn(0x561fe36dd310 msgr2=0x561fe3710c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 -- 172.21.15.77:0/3911664200 shutdown_connections 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.736+0000 7fca304f6740 1 -- 172.21.15.77:0/3911664200 wait complete. 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 -- start start 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 Processor -- start 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe371c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561fe3722010 con 0x561fe371f370 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561fe3635060 con 0x561fe37145d0 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561fe3722190 con 0x561fe3715b40 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca304f6740 10 client.0 _ll_register_callbacks cb 0x561fe33888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca277fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34814/0 (socket says 172.21.15.77:34814) 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca277fe640 1 -- 172.21.15.77:0/2298004518 learned_addr learned my addr 172.21.15.77:0/2298004518 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca27fff640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe371c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca277fe640 1 -- 172.21.15.77:0/2298004518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 msgr2=0x561fe371c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.737+0000 7fca277fe640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca277fe640 1 -- 172.21.15.77:0/2298004518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 msgr2=0x561fe371c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca277fe640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe371c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca277fe640 1 -- 172.21.15.77:0/2298004518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe33eb3c0 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca26ffd640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca277fe640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca18012010 tx=0x7fca18004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c066c634380f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1802c020 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca1801db00 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20036070 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca18037050 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca304f6740 10 client.27263 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fe372daf0 con 0x561fe37145d0 2024-09-15T22:23:24.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20036070 con 0x561fe37145d0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.738+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca18028430 con 0x561fe37145d0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 10 client.27263 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 20 client.27263 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe372efb0 0x561fe3731410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 4 client.27263 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe3687860 con 0x561fe372efb0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca26ffd640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe372efb0 0x561fe3731410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca06ffd640 10 client.27263 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.739+0000 7fca26ffd640 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe372efb0 0x561fe3731410 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d29cdc67262c081 server_cookie=50df7e0fbe1dfa2e in_seq=0 out_seq=0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca06ffd640 1 -- 172.21.15.77:0/2298004518 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe3687860 con 0x561fe372efb0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca06ffd640 10 client.27263 handle_command_reply: tid=0 2024-09-15T22:23:24.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 client.27263 shutdown 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 msgr2=0x561fe371ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca18012010 tx=0x7fca18004730 comp rx=0 tx=0).stop 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 shutdown_connections 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe372efb0 0x561fe3731410 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe371c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 --2- 172.21.15.77:0/2298004518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe371ed50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca06ffd640 0 client.27263 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 >> 172.21.15.77:0/2298004518 conn(0x561fe36dd310 msgr2=0x561fe3716960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 shutdown_connections 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 1 -- 172.21.15.77:0/2298004518 wait complete. 2024-09-15T22:23:24.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.740+0000 7fca304f6740 20 client.27263 trim_cache size 0 max 16384 2024-09-15T22:23:24.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.755+0000 7fca304f6740 1 Processor -- start 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.755+0000 7fca304f6740 1 -- start start 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.755+0000 7fca304f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe372b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.755+0000 7fca304f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.755+0000 7fca304f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca304f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca281acd60 con 0x561fe371f370 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca304f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca281ae8a0 con 0x561fe37145d0 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca304f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca281ad430 con 0x561fe3715b40 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37984/0 (socket says 172.21.15.77:37984) 2024-09-15T22:23:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60822/0 (socket says 172.21.15.77:60822) 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca26ffd640 1 -- 172.21.15.77:0/1970044573 learned_addr learned my addr 172.21.15.77:0/1970044573 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca277fe640 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe372b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 -- 172.21.15.77:0/1970044573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 msgr2=0x561fe371df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 -- 172.21.15.77:0/1970044573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 msgr2=0x561fe372b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe372b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 -- 172.21.15.77:0/1970044573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe3710370 con 0x561fe371f370 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca27fff640 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fca20005e30 tx=0x7fca20003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd21f35d0b66845 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca1d7fa640 1 -- 172.21.15.77:0/1970044573 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20048040 con 0x561fe371f370 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.756+0000 7fca1d7fa640 1 -- 172.21.15.77:0/1970044573 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20036040 con 0x561fe371f370 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca1d7fa640 1 -- 172.21.15.77:0/1970044573 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20030ad0 con 0x561fe371f370 2024-09-15T22:23:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 -- 172.21.15.77:0/1970044573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 msgr2=0x561fe3736d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fca20005e30 tx=0x7fca20003ea0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 -- 172.21.15.77:0/1970044573 shutdown_connections 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe371f370 0x561fe3736d30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe371df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 --2- 172.21.15.77:0/1970044573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe37145d0 0x561fe372b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 -- 172.21.15.77:0/1970044573 >> 172.21.15.77:0/1970044573 conn(0x561fe36dd310 msgr2=0x561fe371d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 -- 172.21.15.77:0/1970044573 shutdown_connections 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.757+0000 7fca304f6740 1 -- 172.21.15.77:0/1970044573 wait complete. 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 -- start start 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 Processor -- start 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe3727b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe371f370 0x561fe37243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca281ae1d0 con 0x561fe37145d0 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca281adb00 con 0x561fe371f370 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca281aad60 con 0x561fe3715b40 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca304f6740 10 client.0 _ll_register_callbacks cb 0x7fca18010830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe3727b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60836/0 (socket says 172.21.15.77:60836) 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca277fe640 1 -- 172.21.15.77:0/2214065103 learned_addr learned my addr 172.21.15.77:0/2214065103 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.758+0000 7fca27fff640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe371f370 0x561fe37243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 -- 172.21.15.77:0/2214065103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 msgr2=0x561fe3727b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe3727b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 -- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe371f370 msgr2=0x561fe37243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:24.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe371f370 0x561fe37243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 -- 172.21.15.77:0/2214065103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe3724a60 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca277fe640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fca18006010 tx=0x7fca18028660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753a418d32641be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1802c040 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca1801da00 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20036040 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca1804d050 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca304f6740 10 client.18237 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fe379a670 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.759+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20036040 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca18044960 con 0x561fe37145d0 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 10 client.18237 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:24.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 20 client.18237 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe379ad30 0x561fe379d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 4 client.18237 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fe3681d70 con 0x561fe379ad30 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca26ffd640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe379ad30 0x561fe379d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca06ffd640 10 client.18237 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca26ffd640 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe379ad30 0x561fe379d140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749ab5645f8c9c67 server_cookie=521fa088ec4bdd70 in_seq=0 out_seq=0 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca06ffd640 1 -- 172.21.15.77:0/2214065103 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fe3681d70 con 0x561fe379ad30 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca06ffd640 10 client.18237 handle_command_reply: tid=0 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.760+0000 7fca304f6740 1 client.18237 shutdown 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 msgr2=0x561fe3727600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fca18006010 tx=0x7fca18028660 comp rx=0 tx=0).stop 2024-09-15T22:23:24.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 shutdown_connections 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561fe379ad30 0x561fe379d140 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561fe371f370 0x561fe37243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561fe3715b40 0x561fe3727b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 --2- 172.21.15.77:0/2214065103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561fe37145d0 0x561fe3727600 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca06ffd640 0 client.18237 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 >> 172.21.15.77:0/2214065103 conn(0x561fe36dd310 msgr2=0x561fe371d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 shutdown_connections 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 1 -- 172.21.15.77:0/2214065103 wait complete. 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca304f6740 20 client.18237 trim_cache size 0 max 16384 2024-09-15T22:23:24.770 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 msgr2=0x7fca140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 0x7fca140a5870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x561fe3711ac0 tx=0x7fca18012040 comp rx=0 tx=0).stop 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 msgr2=0x7fca28198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.761+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca280697d0 tx=0x7fca1000ee50 comp rx=0 tx=0).stop 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 shutdown_connections 2024-09-15T22:23:24.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca140a33b0 0x7fca140a5870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca281082a0 0x7fca2819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca281036e0 0x7fca28198aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 --2- 172.21.15.77:0/1539758090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28102d30 0x7fca28198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 >> 172.21.15.77:0/1539758090 conn(0x7fca280fc7d0 msgr2=0x7fca28105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 shutdown_connections 2024-09-15T22:23:24.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:24.762+0000 7fca2e11d640 1 -- 172.21.15.77:0/1539758090 wait complete. 2024-09-15T22:23:25.487 INFO:tasks.workunit.client.0.smithi077.stdout: 1 859884 7.71 MB/sec execute 454 sec latency 22.772 ms 2024-09-15T22:23:26.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:25 smithi077 ceph-mon[30980]: pgmap v343: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:26.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:25 smithi117 ceph-mon[35021]: pgmap v343: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:25 smithi196 ceph-mon[35109]: pgmap v343: 129 pgs: 129 active+clean; 547 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:23:26.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 862223 7.72 MB/sec execute 455 sec latency 17.196 ms 2024-09-15T22:23:26.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:23:26.695 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:23:26.959 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:27.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 863484 7.72 MB/sec execute 456 sec latency 34.998 ms 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/1796824876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf2410e4d0 msgr2=0x7fbf241108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/1796824876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf2410e4d0 0x7fbf241108e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ca0 tx=0x7fbf1c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/1796824876 shutdown_connections 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/1796824876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf2410e4d0 0x7fbf241108e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/1796824876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf24101360 0x7fbf2410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/1796824876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf241009b0 0x7fbf24100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/1796824876 >> 172.21.15.77:0/1796824876 conn(0x7fbf240fc7d0 msgr2=0x7fbf240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/1796824876 shutdown_connections 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.494+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/1796824876 wait complete. 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.495+0000 7fbf2a5fe640 1 Processor -- start 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.495+0000 7fbf2a5fe640 1 -- start start 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2410e4d0 0x7fbf2419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24114270 con 0x7fbf241009b0 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241140f0 con 0x7fbf2410e4d0 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf2a5fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241143f0 con 0x7fbf24101360 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60858/0 (socket says 172.21.15.77:60858) 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38016/0 (socket says 172.21.15.77:38016) 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf23fff640 1 -- 172.21.15.77:0/3339181975 learned_addr learned my addr 172.21.15.77:0/3339181975 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2410e4d0 0x7fbf2419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 -- 172.21.15.77:0/3339181975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2410e4d0 msgr2=0x7fbf2419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2410e4d0 0x7fbf2419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 -- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 msgr2=0x7fbf2419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 -- 172.21.15.77:0/3339181975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2419f6c0 con 0x7fbf24101360 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.496+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.497+0000 7fbf217fa640 1 -- 172.21.15.77:0/3339181975 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800ccc0 con 0x7fbf24101360 2024-09-15T22:23:27.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.497+0000 7fbf217fa640 1 -- 172.21.15.77:0/3339181975 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf18010910 con 0x7fbf24101360 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.497+0000 7fbf217fa640 1 -- 172.21.15.77:0/3339181975 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18018960 con 0x7fbf24101360 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.497+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2419f9b0 con 0x7fbf24101360 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.497+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241a7290 con 0x7fbf24101360 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.499+0000 7fbf217fa640 1 -- 172.21.15.77:0/3339181975 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbf18010ac0 con 0x7fbf24101360 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 Processor -- start 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 -- start start 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf217fa640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 0x55b921f3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f3ed40 0x55b921f41130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b921e99280 con 0x55b921f34480 2024-09-15T22:23:27.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b921e40b10 con 0x55b921f3ed40 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b921e95240 con 0x55b921f33b60 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.500+0000 7fbf217fa640 1 -- 172.21.15.77:0/3339181975 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fbf180d0430 con 0x7fbf24101360 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 0x55b921f3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38028/0 (socket says 172.21.15.77:38028) 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf23fff640 1 -- 172.21.15.77:0/278460936 learned_addr learned my addr 172.21.15.77:0/278460936 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f3ed40 0x55b921f41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 -- 172.21.15.77:0/278460936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 msgr2=0x55b921f33f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c009a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f33f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 -- 172.21.15.77:0/278460936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f3ed40 msgr2=0x55b921f41130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f3ed40 0x55b921f41130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 -- 172.21.15.77:0/278460936 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b921e4d1f0 con 0x55b921f34480 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf237fe640 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 0x55b921f3e800 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cd790 tx=0x7fbf1809bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91b68d48894dc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf0a7fc640 1 -- 172.21.15.77:0/278460936 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1809c3d0 con 0x55b921f34480 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf0a7fc640 1 -- 172.21.15.77:0/278460936 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf180a06d0 con 0x55b921f34480 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.501+0000 7fbf0a7fc640 1 -- 172.21.15.77:0/278460936 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180cda10 con 0x55b921f34480 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/278460936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 msgr2=0x55b921f3e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 0x55b921f3e800 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbf180cd790 tx=0x7fbf1809bdb0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/278460936 shutdown_connections 2024-09-15T22:23:27.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f3ed40 0x55b921f41130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f34480 0x55b921f3e800 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/278460936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/278460936 >> 172.21.15.77:0/278460936 conn(0x55b921f300f0 msgr2=0x55b921f304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/278460936 shutdown_connections 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.502+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/278460936 wait complete. 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 -- start start 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 Processor -- start 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f34480 0x55b921f3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b921f41910 con 0x55b921f3ed40 2024-09-15T22:23:27.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b921e53f10 con 0x55b921f33b60 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b921f41a90 con 0x55b921f34480 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf2c9d7740 10 client.0 _ll_register_callbacks cb 0x55b921e1ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf28b9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34876/0 (socket says 172.21.15.77:34876) 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf28b9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f3c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60872/0 (socket says 172.21.15.77:60872) 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013614142 learned_addr learned my addr 172.21.15.77:0/3013614142 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.503+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f34480 0x55b921f3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013614142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f34480 msgr2=0x55b921f3c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f34480 0x55b921f3c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013614142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 msgr2=0x55b921f3c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f3c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013614142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b921c0a6a0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf14012d40 tx=0x7fbf1400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e5688dd6d1a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf037fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140120d0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fbf14010430 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf180a06d0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf14012270 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf2c9d7740 10 client.27281 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.504+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b921f4d5f0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf180a06d0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbf1401cad0 con 0x55b921f33b60 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 10 client.27281 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:27.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 20 client.27281 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921f4eab0 0x55b921f50f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 4 client.27281 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b921ea67f0 con 0x55b921f4eab0 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.505+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921f4eab0 0x55b921f50f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf037fe640 10 client.27281 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921f4eab0 0x55b921f50f10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785ae44d4d1b6a9a server_cookie=fe170a513cf486d4 in_seq=0 out_seq=0 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf037fe640 1 -- 172.21.15.77:0/3013614142 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b921ea67f0 con 0x55b921f4eab0 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf037fe640 10 client.27281 handle_command_reply: tid=0 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 client.27281 shutdown 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 msgr2=0x55b921f3bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf14012d40 tx=0x7fbf1400f790 comp rx=0 tx=0).stop 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 shutdown_connections 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921f4eab0 0x55b921f50f10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f3c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f34480 0x55b921f3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013614142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f33b60 0x55b921f3bc50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf037fe640 0 client.27281 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.506+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 >> 172.21.15.77:0/3013614142 conn(0x55b921f300f0 msgr2=0x55b921f2f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.507+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 shutdown_connections 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.507+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013614142 wait complete. 2024-09-15T22:23:27.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.507+0000 7fbf2c9d7740 20 client.27281 trim_cache size 0 max 16384 2024-09-15T22:23:27.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 Processor -- start 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 -- start start 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f3c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f56830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241b11a0 con 0x55b921f3ed40 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241b2ce0 con 0x55b921f34480 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241b1870 con 0x55b921f33b60 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf28b9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f56830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38038/0 (socket says 172.21.15.77:38038) 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013647103 learned_addr learned my addr 172.21.15.77:0/3013647103 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf237fe640 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f3c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013647103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 msgr2=0x55b921f3c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f3c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013647103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 msgr2=0x55b921f56830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f56830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.525+0000 7fbf23fff640 1 -- 172.21.15.77:0/3013647103 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b921f31280 con 0x55b921f33b60 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.526+0000 7fbf23fff640 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf14012d10 tx=0x7fbf14009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd5b5f8f1e8da75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.526+0000 7fbf09ffb640 1 -- 172.21.15.77:0/3013647103 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14010430 con 0x55b921f33b60 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.526+0000 7fbf09ffb640 1 -- 172.21.15.77:0/3013647103 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf14010d20 con 0x55b921f33b60 2024-09-15T22:23:27.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.526+0000 7fbf09ffb640 1 -- 172.21.15.77:0/3013647103 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140056b0 con 0x55b921f33b60 2024-09-15T22:23:27.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013647103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 msgr2=0x55b921f365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf14012d10 tx=0x7fbf14009fd0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013647103 shutdown_connections 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f56830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f3c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/3013647103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f365c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013647103 >> 172.21.15.77:0/3013647103 conn(0x55b921f300f0 msgr2=0x55b921f3dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013647103 shutdown_connections 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/3013647103 wait complete. 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.527+0000 7fbf2c9d7740 1 -- start start 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 Processor -- start 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f42900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241b2610 con 0x55b921f3ed40 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241b1f40 con 0x55b921f34480 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241af1a0 con 0x55b921f33b60 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf2c9d7740 10 client.0 _ll_register_callbacks cb 0x7fbf14009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38046/0 (socket says 172.21.15.77:38046) 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60902/0 (socket says 172.21.15.77:60902) 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf23fff640 1 -- 172.21.15.77:0/2479824643 learned_addr learned my addr 172.21.15.77:0/2479824643 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf237fe640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f42900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 -- 172.21.15.77:0/2479824643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 msgr2=0x55b921f475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 -- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 msgr2=0x55b921f42900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:27.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f42900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf28b9b640 1 -- 172.21.15.77:0/2479824643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b921fb8800 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf23fff640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.528+0000 7fbf237fe640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f42900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf28b9b640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005e30 tx=0x7fbf1c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28541596af5b2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf037fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c042040 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf14010d20 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fbf1c031c90 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c058640 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf2c9d7740 10 client.18255 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b921fb9a30 con 0x55b921f3ed40 2024-09-15T22:23:27.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.529+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf14010d20 con 0x55b921f3ed40 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbf1c02f610 con 0x55b921f3ed40 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 10 client.18255 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 20 client.18255 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921fba130 0x55b921fbc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 4 client.18255 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b921ea0d00 con 0x55b921fba130 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf23fff640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921fba130 0x55b921fbc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:27.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf037fe640 10 client.18255 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.530+0000 7fbf23fff640 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921fba130 0x55b921fbc550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954431cb0b33aa55 server_cookie=a0a91b8b79fcc92a in_seq=0 out_seq=0 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.531+0000 7fbf037fe640 1 -- 172.21.15.77:0/2479824643 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b921ea0d00 con 0x55b921fba130 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.531+0000 7fbf037fe640 10 client.18255 handle_command_reply: tid=0 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.531+0000 7fbf2c9d7740 1 client.18255 shutdown 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 msgr2=0x55b921f43040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005e30 tx=0x7fbf1c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 shutdown_connections 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b921fba130 0x55b921fbc550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b921f3ed40 0x55b921f43040 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b921f34480 0x55b921f42900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 --2- 172.21.15.77:0/2479824643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b921f33b60 0x55b921f475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf037fe640 0 client.18255 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:27.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 >> 172.21.15.77:0/2479824643 conn(0x55b921f300f0 msgr2=0x55b921f3d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 shutdown_connections 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 1 -- 172.21.15.77:0/2479824643 wait complete. 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.532+0000 7fbf2c9d7740 20 client.18255 trim_cache size 0 max 16384 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "8028e6f1-bdb9-42fc-b43d-7c4ea9b9ea81", 2024-09-15T22:23:27.540 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:23:27.541 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:23:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 msgr2=0x7fbf0c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006010 tx=0x7fbf1c009a20 comp rx=0 tx=0).stop 2024-09-15T22:23:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 msgr2=0x7fbf2419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e9f0 tx=0x7fbf1800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 shutdown_connections 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbf0c0a33b0 0x7fbf0c0a5870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2410e4d0 0x7fbf2419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101360 0x7fbf2419ac50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 --2- 172.21.15.77:0/3339181975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf241009b0 0x7fbf2419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 >> 172.21.15.77:0/3339181975 conn(0x7fbf240fc7d0 msgr2=0x7fbf240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 shutdown_connections 2024-09-15T22:23:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:27.533+0000 7fbf2a5fe640 1 -- 172.21.15.77:0/3339181975 wait complete. 2024-09-15T22:23:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:27 smithi117 ceph-mon[35021]: pgmap v344: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:27 smithi077 ceph-mon[30980]: pgmap v344: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:27 smithi196 ceph-mon[35109]: pgmap v344: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:28.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:23:28.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 864835 7.72 MB/sec execute 457 sec latency 182.608 ms 2024-09-15T22:23:28.627 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:29.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:23:29.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:23:29.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:23:29.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.023 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:28 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:23:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:23:29.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:23:29.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:28 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.105+0000 7f4f1c848640 1 -- 172.21.15.77:0/297446051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18105560 msgr2=0x7f4f1810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.105+0000 7f4f1c848640 1 --2- 172.21.15.77:0/297446051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18105560 0x7f4f1810d0e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009ca0 tx=0x7f4f1002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 -- 172.21.15.77:0/297446051 shutdown_connections 2024-09-15T22:23:29.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 --2- 172.21.15.77:0/297446051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18105560 0x7f4f1810d0e0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 --2- 172.21.15.77:0/297446051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100de0 0x7f4f18104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 --2- 172.21.15.77:0/297446051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18100430 0x7f4f18100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 -- 172.21.15.77:0/297446051 >> 172.21.15.77:0/297446051 conn(0x7f4f18077230 msgr2=0x7f4f180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 -- 172.21.15.77:0/297446051 shutdown_connections 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 -- 172.21.15.77:0/297446051 wait complete. 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.106+0000 7f4f1c848640 1 Processor -- start 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 -- start start 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 0x7f4f181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 0x7f4f1819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110e90 con 0x7f4f18100de0 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110d10 con 0x7f4f18105560 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1c848640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18111010 con 0x7f4f18100430 2024-09-15T22:23:29.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 0x7f4f181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36918/0 (socket says 172.21.15.77:36918) 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 -- 172.21.15.77:0/1386076286 learned_addr learned my addr 172.21.15.77:0/1386076286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 0x7f4f1819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 -- 172.21.15.77:0/1386076286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 msgr2=0x7f4f181983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 0x7f4f181983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 -- 172.21.15.77:0/1386076286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 msgr2=0x7f4f1819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 0x7f4f1819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 -- 172.21.15.77:0/1386076286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1819d380 con 0x7f4f18100de0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 0x7f4f1819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f1659c640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 0x7f4f181983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.107+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9f0 tx=0x7f4f0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.108+0000 7f4f077fe640 1 -- 172.21.15.77:0/1386076286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00cd80 con 0x7f4f18100de0 2024-09-15T22:23:29.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.108+0000 7f4f077fe640 1 -- 172.21.15.77:0/1386076286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c005500 con 0x7f4f18100de0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.108+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1819d670 con 0x7f4f18100de0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.108+0000 7f4f077fe640 1 -- 172.21.15.77:0/1386076286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c010690 con 0x7f4f18100de0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.108+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a4f50 con 0x7f4f18100de0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.110+0000 7f4f077fe640 1 -- 172.21.15.77:0/1386076286 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f0c0046c0 con 0x7f4f18100de0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.110+0000 7f4f1ec21740 1 Processor -- start 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 -- start start 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fda140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fdca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d86f352e0 con 0x564d86fda680 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564d86edcce0 con 0x564d86fcf4a0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d86f312a0 con 0x564d86fcfdc0 2024-09-15T22:23:29.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fdca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43040/0 (socket says 172.21.15.77:43040) 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f16d9d640 1 -- 172.21.15.77:0/3861294571 learned_addr learned my addr 172.21.15.77:0/3861294571 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f077fe640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 0x7f4f000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f1659c640 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fda140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 0x7f4f000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.111+0000 7f4f15d9b640 1 -- 172.21.15.77:0/3861294571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 msgr2=0x564d86fda140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fda140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f15d9b640 1 -- 172.21.15.77:0/3861294571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 msgr2=0x564d86fcf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f15d9b640 1 -- 172.21.15.77:0/3861294571 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d86ec1440 con 0x564d86fda680 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f077fe640 1 -- 172.21.15.77:0/1386076286 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4f0c0d04b0 con 0x7f4f18100de0 2024-09-15T22:23:29.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fdca70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0cd810 tx=0x7f4f0c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8568749303db3fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f04ff9640 1 -- 172.21.15.77:0/3861294571 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c09bd00 con 0x564d86fda680 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f04ff9640 1 -- 172.21.15.77:0/3861294571 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c09caa0 con 0x564d86fda680 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f04ff9640 1 -- 172.21.15.77:0/3861294571 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01c600 con 0x564d86fda680 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.112+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 0x7f4f000a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009430 tx=0x7f4f1002a8c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- 172.21.15.77:0/3861294571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 msgr2=0x564d86fdca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fdca70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0cd810 tx=0x7f4f0c021000 comp rx=0 tx=0).stop 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- 172.21.15.77:0/3861294571 shutdown_connections 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fdca70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fda140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/3861294571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fcf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- 172.21.15.77:0/3861294571 >> 172.21.15.77:0/3861294571 conn(0x564d86fcba30 msgr2=0x564d86fcbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- 172.21.15.77:0/3861294571 shutdown_connections 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- 172.21.15.77:0/3861294571 wait complete. 2024-09-15T22:23:29.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 -- start start 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.113+0000 7f4f1ec21740 1 Processor -- start 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fd9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fd7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d86fdd250 con 0x564d86fda680 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564d86ef0050 con 0x564d86fcf4a0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564d86fdd3d0 con 0x564d86fcfdc0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1ec21740 10 client.0 _ll_register_callbacks cb 0x564d86f30160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36936/0 (socket says 172.21.15.77:36936) 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 -- 172.21.15.77:0/2407652046 learned_addr learned my addr 172.21.15.77:0/2407652046 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f1659c640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fd9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fd7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 -- 172.21.15.77:0/2407652046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 msgr2=0x564d86fd7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fd7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 -- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 msgr2=0x564d86fd9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fd9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 -- 172.21.15.77:0/2407652046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d86ca63c0 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.114+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4f1002ade0 tx=0x7f4f100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e810e7c9be12d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10050030 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4f1002fc70 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c09caa0 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f1002fe70 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 10 client.18267 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d86fe8c20 con 0x564d86fda680 2024-09-15T22:23:29.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c09caa0 con 0x564d86fda680 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f1004b040 con 0x564d86fda680 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 10 client.18267 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 20 client.18267 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d86fec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 4 client.18267 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d86f42850 con 0x564d86fea0d0 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.115+0000 7f4f1659c640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d86fec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.116+0000 7f4f1659c640 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d86fec500 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1de90d7ae7f4e7 server_cookie=9a7acf977fbcf24b in_seq=0 out_seq=0 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.116+0000 7f4efdffb640 10 client.18267 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4efdffb640 1 -- 172.21.15.77:0/2407652046 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d86f42850 con 0x564d86fea0d0 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4efdffb640 10 client.18267 handle_command_reply: tid=0 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 client.18267 shutdown 2024-09-15T22:23:29.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 msgr2=0x564d86fd77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4f1002ade0 tx=0x7f4f100095c0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 shutdown_connections 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d86fec500 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4efdffb640 0 client.18267 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fd77f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fd7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/2407652046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fd9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 >> 172.21.15.77:0/2407652046 conn(0x564d86fcba30 msgr2=0x564d86fd1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 shutdown_connections 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 1 -- 172.21.15.77:0/2407652046 wait complete. 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.117+0000 7f4f1ec21740 20 client.18267 trim_cache size 0 max 16384 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 Processor -- start 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 -- start start 2024-09-15T22:23:29.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcf4a0 0x564d86fe64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcfdc0 0x564d86fd27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181aefd0 con 0x564d86fda680 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181b0b10 con 0x564d86fcfdc0 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181af6a0 con 0x564d86fcf4a0 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f15d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcfdc0 0x564d86fd27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36938/0 (socket says 172.21.15.77:36938) 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 -- 172.21.15.77:0/1946733409 learned_addr learned my addr 172.21.15.77:0/1946733409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f1659c640 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcf4a0 0x564d86fe64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 -- 172.21.15.77:0/1946733409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcf4a0 msgr2=0x564d86fe64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcf4a0 0x564d86fe64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 -- 172.21.15.77:0/1946733409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcfdc0 msgr2=0x564d86fd27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcfdc0 0x564d86fd27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.134+0000 7f4f16d9d640 1 -- 172.21.15.77:0/1946733409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d86fccbc0 con 0x564d86fda680 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4f1003c900 tx=0x7f4f10005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe848d2d6713ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4efffff640 1 -- 172.21.15.77:0/1946733409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10050070 con 0x564d86fda680 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4efffff640 1 -- 172.21.15.77:0/1946733409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f10036040 con 0x564d86fda680 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4efffff640 1 -- 172.21.15.77:0/1946733409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10030e20 con 0x564d86fda680 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4f1ec21740 1 -- 172.21.15.77:0/1946733409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 msgr2=0x564d86ff1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.135+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4f1003c900 tx=0x7f4f10005fa0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- 172.21.15.77:0/1946733409 shutdown_connections 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86ff1dd0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcfdc0 0x564d86fd27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/1946733409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcf4a0 0x564d86fe64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- 172.21.15.77:0/1946733409 >> 172.21.15.77:0/1946733409 conn(0x564d86fcba30 msgr2=0x564d86fd2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- 172.21.15.77:0/1946733409 shutdown_connections 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- 172.21.15.77:0/1946733409 wait complete. 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- start start 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 Processor -- start 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fe2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fde160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fde700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181b0440 con 0x564d86fda680 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181afd70 con 0x564d86fcf4a0 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.136+0000 7f4f1ec21740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181acfd0 con 0x564d86fcfdc0 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f1ec21740 10 client.0 _ll_register_callbacks cb 0x7f4f10009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fe2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fe2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43072/0 (socket says 172.21.15.77:43072) 2024-09-15T22:23:29.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f1659c640 1 -- 172.21.15.77:0/433036780 learned_addr learned my addr 172.21.15.77:0/433036780 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f15d9b640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fde160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fde700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 -- 172.21.15.77:0/433036780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 msgr2=0x564d86fde160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fde160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 -- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 msgr2=0x564d86fe2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fe2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 -- 172.21.15.77:0/433036780 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d86fdef30 con 0x564d86fda680 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f16d9d640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fde700 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4f1003c880 tx=0x7f4f1003de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244d70df1f2565ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10050040 con 0x564d86fda680 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4f100305c0 con 0x564d86fda680 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.137+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f10036040 con 0x564d86fda680 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f10072050 con 0x564d86fda680 2024-09-15T22:23:29.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 10 client.18273 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d87054e80 con 0x564d86fda680 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f10036040 con 0x564d86fda680 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f1004b070 con 0x564d86fda680 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 10 client.18273 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 20 client.18273 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d87057450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 4 client.18273 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d86f3cd60 con 0x564d86fea0d0 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.138+0000 7f4f1659c640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d87057450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1659c640 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d87057450 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37fd453549a11eb2 server_cookie=721ce64446315a99 in_seq=0 out_seq=0 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4efdffb640 10 client.18273 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4efdffb640 1 -- 172.21.15.77:0/433036780 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564d86f3cd60 con 0x564d86fea0d0 2024-09-15T22:23:29.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4efdffb640 10 client.18273 handle_command_reply: tid=0 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 client.18273 shutdown 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 msgr2=0x564d86fde700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fde700 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4f1003c880 tx=0x7f4f1003de30 comp rx=0 tx=0).stop 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 shutdown_connections 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564d86fea0d0 0x564d87057450 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d86fda680 0x564d86fde700 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564d86fcfdc0 0x564d86fde160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 --2- 172.21.15.77:0/433036780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564d86fcf4a0 0x564d86fe2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 >> 172.21.15.77:0/433036780 conn(0x564d86fcba30 msgr2=0x564d86fd28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4efdffb640 0 client.18273 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 shutdown_connections 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 1 -- 172.21.15.77:0/433036780 wait complete. 2024-09-15T22:23:29.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.139+0000 7f4f1ec21740 20 client.18273 trim_cache size 0 max 16384 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "8028e6f1-bdb9-42fc-b43d-7c4ea9b9ea81": { 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "8028e6f1-bdb9-42fc-b43d-7c4ea9b9ea81", 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:29.157 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:29.158 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:23:29.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 msgr2=0x7f4f000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 0x7f4f000a5870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009430 tx=0x7f4f1002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 msgr2=0x7f4f18198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9f0 tx=0x7f4f0c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 shutdown_connections 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f000a33b0 0x7f4f000a5870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f18105560 0x7f4f1819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f18100de0 0x7f4f18198910 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 --2- 172.21.15.77:0/1386076286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f18100430 0x7f4f181983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 >> 172.21.15.77:0/1386076286 conn(0x7f4f18077230 msgr2=0x7f4f1810dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 shutdown_connections 2024-09-15T22:23:29.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:29.140+0000 7f4f1c848640 1 -- 172.21.15.77:0/1386076286 wait complete. 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:28 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:29.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 866501 7.73 MB/sec execute 458 sec latency 52.143 ms 2024-09-15T22:23:29.961 INFO:tasks.cephfs.filesystem:scrub status for tag:8028e6f1-bdb9-42fc-b43d-7c4ea9b9ea81 - {'path': '/', 'tag': '8028e6f1-bdb9-42fc-b43d-7c4ea9b9ea81', 'options': 'recursive,force'} 2024-09-15T22:23:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:29 smithi117 ceph-mon[35021]: pgmap v345: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:29 smithi077 ceph-mon[30980]: pgmap v345: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:29 smithi196 ceph-mon[35109]: pgmap v345: 129 pgs: 129 active+clean; 557 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-09-15T22:23:30.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 868275 7.72 MB/sec execute 459 sec latency 18.546 ms 2024-09-15T22:23:31.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 870220 7.74 MB/sec execute 460 sec latency 14.376 ms 2024-09-15T22:23:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:31 smithi077 ceph-mon[30980]: pgmap v346: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:31 smithi196 ceph-mon[35109]: pgmap v346: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:31 smithi117 ceph-mon[35021]: pgmap v346: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 273 op/s 2024-09-15T22:23:32.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 872246 7.74 MB/sec execute 461 sec latency 21.656 ms 2024-09-15T22:23:33.165 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:23:33.166 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-15T22:23:33.166 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:33.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:33.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 873685 7.74 MB/sec execute 462 sec latency 84.283 ms 2024-09-15T22:23:34.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 -- 172.21.15.77:0/3536601018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c100e20 msgr2=0x7fea9c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 --2- 172.21.15.77:0/3536601018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c100e20 0x7fea9c101200 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009900 tx=0x7fea8c02a480 comp rx=0 tx=0).stop 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 -- 172.21.15.77:0/3536601018 shutdown_connections 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 --2- 172.21.15.77:0/3536601018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c102ef0 0x7fea9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 --2- 172.21.15.77:0/3536601018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102570 0x7fea9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 --2- 172.21.15.77:0/3536601018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c100e20 0x7fea9c101200 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 -- 172.21.15.77:0/3536601018 >> 172.21.15.77:0/3536601018 conn(0x7fea9c0fc7d0 msgr2=0x7fea9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 -- 172.21.15.77:0/3536601018 shutdown_connections 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.068+0000 7feaa18af640 1 -- 172.21.15.77:0/3536601018 wait complete. 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 Processor -- start 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 -- start start 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c100e20 0x7fea9c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102ef0 0x7fea9c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110f80 con 0x7fea9c102570 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110e00 con 0x7fea9c102ef0 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7feaa18af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c111100 con 0x7fea9c100e20 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36962/0 (socket says 172.21.15.77:36962) 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 -- 172.21.15.77:0/2512473975 learned_addr learned my addr 172.21.15.77:0/2512473975 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9affd640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c100e20 0x7fea9c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9b7fe640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102ef0 0x7fea9c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 -- 172.21.15.77:0/2512473975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c100e20 msgr2=0x7fea9c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c100e20 0x7fea9c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 -- 172.21.15.77:0/2512473975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102ef0 msgr2=0x7fea9c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102ef0 0x7fea9c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 -- 172.21.15.77:0/2512473975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a48f0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.069+0000 7fea9a7fc640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c890 tx=0x7fea9000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.070+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90004220 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.070+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea900043c0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.070+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1a4ac0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.070+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea900049e0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.070+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a5000 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.071+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c101200 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.072+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fea900079b0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.072+0000 7feaa08ad640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 0x7fea840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.073+0000 7fea9affd640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 0x7fea840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.073+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fea900d00d0 con 0x7fea9c102570 2024-09-15T22:23:34.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.073+0000 7fea9affd640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 0x7fea840a58f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009230 tx=0x7fea8c0058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.076+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea90014030 con 0x7fea9c102570 2024-09-15T22:23:34.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.302+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea9c06a560 con 0x7fea9c102570 2024-09-15T22:23:34.304 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:33 smithi077 ceph-mon[30980]: pgmap v347: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:34.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.303+0000 7feaa08ad640 1 -- 172.21.15.77:0/2512473975 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fea9c06a560 con 0x7fea9c102570 2024-09-15T22:23:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:23:34.305 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:23:34.305 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:23:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 msgr2=0x7fea840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 0x7fea840a58f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009230 tx=0x7fea8c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 msgr2=0x7fea9c199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c890 tx=0x7fea9000cd50 comp rx=0 tx=0).stop 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 shutdown_connections 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fea840a3430 0x7fea840a58f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea9c102ef0 0x7fea9c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea9c102570 0x7fea9c199870 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 --2- 172.21.15.77:0/2512473975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea9c100e20 0x7fea9c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 >> 172.21.15.77:0/2512473975 conn(0x7fea9c0fc7d0 msgr2=0x7fea9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 shutdown_connections 2024-09-15T22:23:34.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:34.305+0000 7feaa18af640 1 -- 172.21.15.77:0/2512473975 wait complete. 2024-09-15T22:23:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:33 smithi196 ceph-mon[35109]: pgmap v347: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:33 smithi117 ceph-mon[35021]: pgmap v347: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:34.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 875079 7.74 MB/sec execute 463 sec latency 56.902 ms 2024-09-15T22:23:35.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:34 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2512473975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:35.126 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-15T22:23:35.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-15T22:23:35.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:34 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2512473975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:35.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:34 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2512473975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:35.389 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:35.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877175 7.75 MB/sec execute 464 sec latency 19.791 ms 2024-09-15T22:23:35.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- 172.21.15.77:0/2137938211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 msgr2=0x7fca3c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- 172.21.15.77:0/2137938211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c103110 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fca30009900 tx=0x7fca3002a480 comp rx=0 tx=0).stop 2024-09-15T22:23:35.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- 172.21.15.77:0/2137938211 shutdown_connections 2024-09-15T22:23:35.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- 172.21.15.77:0/2137938211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c10c160 0x7fca3c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- 172.21.15.77:0/2137938211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c103650 0x7fca3c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- 172.21.15.77:0/2137938211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c103110 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- 172.21.15.77:0/2137938211 >> 172.21.15.77:0/2137938211 conn(0x7fca3c0fc790 msgr2=0x7fca3c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- 172.21.15.77:0/2137938211 shutdown_connections 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- 172.21.15.77:0/2137938211 wait complete. 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 Processor -- start 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 -- start start 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 0x7fca3c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.920+0000 7fca445cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c10c160 0x7fca3c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca445cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c111f60 con 0x7fca3c102d30 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca445cf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c111de0 con 0x7fca3c103650 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca445cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1120e0 con 0x7fca3c10c160 2024-09-15T22:23:35.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36986/0 (socket says 172.21.15.77:36986) 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 -- 172.21.15.77:0/380648823 learned_addr learned my addr 172.21.15.77:0/380648823 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca41b6a640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 0x7fca3c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca42b6c640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c10c160 0x7fca3c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 -- 172.21.15.77:0/380648823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c10c160 msgr2=0x7fca3c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c10c160 0x7fca3c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 -- 172.21.15.77:0/380648823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 msgr2=0x7fca3c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 0x7fca3c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 -- 172.21.15.77:0/380648823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c19d4b0 con 0x7fca3c102d30 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca41b6a640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 0x7fca3c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca4236b640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fca30006010 tx=0x7fca30004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca2f7fe640 1 -- 172.21.15.77:0/380648823 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003f030 con 0x7fca3c102d30 2024-09-15T22:23:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca2f7fe640 1 -- 172.21.15.77:0/380648823 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca30004440 con 0x7fca3c102d30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c19d740 con 0x7fca3c102d30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca2f7fe640 1 -- 172.21.15.77:0/380648823 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30035070 con 0x7fca3c102d30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.921+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c1a5080 con 0x7fca3c102d30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.924+0000 7fca2f7fe640 1 -- 172.21.15.77:0/380648823 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fca30042030 con 0x7fca3c102d30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.924+0000 7fca469a8740 1 Processor -- start 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.924+0000 7fca469a8740 1 -- start start 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556f7e719160 con 0x556f7e7b4850 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f7e6c0720 con 0x556f7e7bfa30 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca469a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f7e715120 con 0x556f7e7b5170 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca2f7fe640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 0x7fca280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca42b6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43140/0 (socket says 172.21.15.77:43140) 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca42b6c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36992/0 (socket says 172.21.15.77:36992) 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 -- 172.21.15.77:0/2753461418 learned_addr learned my addr 172.21.15.77:0/2753461418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca42b6c640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 0x7fca280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca4236b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca2f7fe640 1 -- 172.21.15.77:0/380648823 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca300f0460 con 0x7fca3c102d30 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 -- 172.21.15.77:0/2753461418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 msgr2=0x556f7e7bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 -- 172.21.15.77:0/2753461418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 msgr2=0x556f7e7b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 -- 172.21.15.77:0/2753461418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7e48aba0 con 0x556f7e7bfa30 2024-09-15T22:23:35.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca4236b640 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca41b6a640 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca380114c0 tx=0x7fca38011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a841c14f48f0079d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.925+0000 7fca42b6c640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 0x7fca280a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fca34005fd0 tx=0x7fca34009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca2cff9640 1 -- 172.21.15.77:0/2753461418 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38004020 con 0x556f7e7bfa30 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca2cff9640 1 -- 172.21.15.77:0/2753461418 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca380041c0 con 0x556f7e7bfa30 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca2cff9640 1 -- 172.21.15.77:0/2753461418 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800f9d0 con 0x556f7e7bfa30 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 -- 172.21.15.77:0/2753461418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 msgr2=0x556f7e7c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca380114c0 tx=0x7fca38011980 comp rx=0 tx=0).stop 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 -- 172.21.15.77:0/2753461418 shutdown_connections 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c1e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 --2- 172.21.15.77:0/2753461418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 -- 172.21.15.77:0/2753461418 >> 172.21.15.77:0/2753461418 conn(0x556f7e7b0de0 msgr2=0x556f7e7b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 -- 172.21.15.77:0/2753461418 shutdown_connections 2024-09-15T22:23:35.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.926+0000 7fca469a8740 1 -- 172.21.15.77:0/2753461418 wait complete. 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 -- start start 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 Processor -- start 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b4850 0x556f7e7bc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556f7e7c2600 con 0x556f7e7b5170 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f7e6a1900 con 0x556f7e7b4850 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556f7e7c2780 con 0x556f7e7bfa30 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca469a8740 10 client.0 _ll_register_callbacks cb 0x556f7e69f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca41b6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca41b6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36994/0 (socket says 172.21.15.77:36994) 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca41b6a640 1 -- 172.21.15.77:0/557236481 learned_addr learned my addr 172.21.15.77:0/557236481 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:35.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca4236b640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b4850 0x556f7e7bc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.927+0000 7fca42b6c640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 -- 172.21.15.77:0/557236481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 msgr2=0x556f7e7bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 -- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b4850 msgr2=0x556f7e7bc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b4850 0x556f7e7bc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 -- 172.21.15.77:0/557236481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7e6e1c50 con 0x556f7e7b5170 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca41b6a640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fca38003d70 tx=0x7fca38003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c91bcb175ca7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d050 con 0x556f7e7b5170 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca38004370 con 0x556f7e7b5170 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca380041c0 con 0x556f7e7b5170 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca38032050 con 0x556f7e7b5170 2024-09-15T22:23:35.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca469a8740 10 client.18291 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556f7e7ce1f0 con 0x556f7e7b5170 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.928+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca380041c0 con 0x556f7e7b5170 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca38004aa0 con 0x556f7e7b5170 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 10 client.18291 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 10 client.18291 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 20 client.18291 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e7cf5a0 0x556f7e7d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 4 client.18291 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7e726890 con 0x556f7e7cf5a0 2024-09-15T22:23:35.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca4236b640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e7cf5a0 0x556f7e7d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca4236b640 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e7cf5a0 0x556f7e7d1990 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcbb1dfd8ea7ebf server_cookie=c7ac5e06ebf42054 in_seq=0 out_seq=0 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.929+0000 7fca25ffb640 10 client.18291 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca25ffb640 1 -- 172.21.15.77:0/557236481 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f7e726890 con 0x556f7e7cf5a0 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca25ffb640 10 client.18291 handle_command_reply: tid=0 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 client.18291 shutdown 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 msgr2=0x556f7e7bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fca38003d70 tx=0x7fca38003da0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 shutdown_connections 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e7cf5a0 0x556f7e7d1990 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca25ffb640 0 client.18291 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b5170 0x556f7e7bce30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 --2- 172.21.15.77:0/557236481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b4850 0x556f7e7bc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 >> 172.21.15.77:0/557236481 conn(0x556f7e7b0de0 msgr2=0x556f7e7b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:35.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 shutdown_connections 2024-09-15T22:23:35.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 1 -- 172.21.15.77:0/557236481 wait complete. 2024-09-15T22:23:35.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.930+0000 7fca469a8740 20 client.18291 trim_cache size 0 max 16384 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 Processor -- start 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 -- start start 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b5170 0x556f7e7d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7d7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.947+0000 7fca469a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1aefc0 con 0x556f7e7b4850 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca469a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b0b00 con 0x556f7e7b5170 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca469a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1af690 con 0x556f7e7bfa30 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36998/0 (socket says 172.21.15.77:36998) 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 -- 172.21.15.77:0/2381360516 learned_addr learned my addr 172.21.15.77:0/2381360516 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca42b6c640 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7d7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca41b6a640 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b5170 0x556f7e7d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 -- 172.21.15.77:0/2381360516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 msgr2=0x556f7e7d7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7d7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 -- 172.21.15.77:0/2381360516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b5170 msgr2=0x556f7e7d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b5170 0x556f7e7d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 -- 172.21.15.77:0/2381360516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7e7b0c10 con 0x556f7e7b4850 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca4236b640 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fca300b8860 tx=0x7fca300daf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648e6c74f2ace836 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca27fff640 1 -- 172.21.15.77:0/2381360516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30049040 con 0x556f7e7b4850 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca27fff640 1 -- 172.21.15.77:0/2381360516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca300c0700 con 0x556f7e7b4850 2024-09-15T22:23:35.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.948+0000 7fca27fff640 1 -- 172.21.15.77:0/2381360516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30003df0 con 0x556f7e7b4850 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 -- 172.21.15.77:0/2381360516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 msgr2=0x556f7e7be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fca300b8860 tx=0x7fca300daf90 comp rx=0 tx=0).stop 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 -- 172.21.15.77:0/2381360516 shutdown_connections 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7bfa30 0x556f7e7d7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7b5170 0x556f7e7d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 --2- 172.21.15.77:0/2381360516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7be860 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 -- 172.21.15.77:0/2381360516 >> 172.21.15.77:0/2381360516 conn(0x556f7e7b0de0 msgr2=0x556f7e7be150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 -- 172.21.15.77:0/2381360516 shutdown_connections 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.949+0000 7fca469a8740 1 -- 172.21.15.77:0/2381360516 wait complete. 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 -- start start 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 Processor -- start 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b0430 con 0x556f7e7b4850 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1afd60 con 0x556f7e7bfa30 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.950+0000 7fca469a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1acfc0 con 0x556f7e7b5170 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca469a8740 10 client.0 _ll_register_callbacks cb 0x7fca34009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37012/0 (socket says 172.21.15.77:37012) 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 -- 172.21.15.77:0/137579460 learned_addr learned my addr 172.21.15.77:0/137579460 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca41b6a640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 -- 172.21.15.77:0/137579460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 msgr2=0x556f7e7c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 -- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 msgr2=0x556f7e7c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca42b6c640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 -- 172.21.15.77:0/137579460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7e7ac3f0 con 0x556f7e7b4850 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca41b6a640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca42b6c640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.951+0000 7fca4236b640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fca30005ab0 tx=0x7fca30005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a740aa066fc86dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca300da060 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fca300ba700 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca300c0700 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca300d4030 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 10 client.18297 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556f7e83a0e0 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca300c0700 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca300eda40 con 0x556f7e7b4850 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 10 client.18297 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 10 client.18297 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 20 client.18297 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e83a850 0x556f7e83cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 4 client.18297 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-15T22:23:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.952+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x556f7e720be0 con 0x556f7e83a850 2024-09-15T22:23:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.953+0000 7fca41b6a640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e83a850 0x556f7e83cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.953+0000 7fca25ffb640 10 client.18297 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:35.953+0000 7fca41b6a640 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e83a850 0x556f7e83cc70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bddb38ee134a59 server_cookie=f76f76470fc25577 in_seq=0 out_seq=0 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca25ffb640 1 -- 172.21.15.77:0/137579460 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556f7e720be0 con 0x556f7e83a850 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca25ffb640 10 client.18297 handle_command_reply: tid=0 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 client.18297 shutdown 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 msgr2=0x556f7e7c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fca30005ab0 tx=0x7fca30005890 comp rx=0 tx=0).stop 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 shutdown_connections 2024-09-15T22:23:36.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556f7e83a850 0x556f7e83cc70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7e7bfa30 0x556f7e7c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556f7e7b5170 0x556f7e7c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca25ffb640 0 client.18297 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 --2- 172.21.15.77:0/137579460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f7e7b4850 0x556f7e7c36a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 >> 172.21.15.77:0/137579460 conn(0x556f7e7b0de0 msgr2=0x556f7e7b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 shutdown_connections 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 1 -- 172.21.15.77:0/137579460 wait complete. 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.023+0000 7fca469a8740 20 client.18297 trim_cache size 0 max 16384 2024-09-15T22:23:36.026 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 38, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "500ee9b1": { 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 38, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:23:36.027 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 141.8, 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:36.028 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 msgr2=0x7fca280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 0x7fca280a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fca34005fd0 tx=0x7fca34009500 comp rx=0 tx=0).stop 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 msgr2=0x7fca3c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fca30006010 tx=0x7fca30004040 comp rx=0 tx=0).stop 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 shutdown_connections 2024-09-15T22:23:36.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca280a33b0 0x7fca280a5870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca3c10c160 0x7fca3c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca3c103650 0x7fca3c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 --2- 172.21.15.77:0/380648823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca3c102d30 0x7fca3c198500 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 >> 172.21.15.77:0/380648823 conn(0x7fca3c0fc790 msgr2=0x7fca3c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 shutdown_connections 2024-09-15T22:23:36.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:36.024+0000 7fca445cf640 1 -- 172.21.15.77:0/380648823 wait complete. 2024-09-15T22:23:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:35 smithi077 ceph-mon[30980]: pgmap v348: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:35 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:23:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:35 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:23:36.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:35 smithi196 ceph-mon[35109]: pgmap v348: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:36.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:35 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:23:36.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:35 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:23:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:35 smithi117 ceph-mon[35021]: pgmap v348: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 175 op/s 2024-09-15T22:23:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:35 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:23:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:35 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:23:36.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.74 MB/sec execute 465 sec latency 518.596 ms 2024-09-15T22:23:36.925 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '500ee9b1', quiesce took 3.8 seconds. Will release after: 18.05 2024-09-15T22:23:37.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.72 MB/sec execute 466 sec latency 1518.685 ms 2024-09-15T22:23:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:37 smithi077 ceph-mon[30980]: pgmap v349: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-09-15T22:23:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:37 smithi196 ceph-mon[35109]: pgmap v349: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-09-15T22:23:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:37 smithi117 ceph-mon[35021]: pgmap v349: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 257 op/s 2024-09-15T22:23:38.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.70 MB/sec execute 467 sec latency 2518.751 ms 2024-09-15T22:23:39.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.69 MB/sec execute 468 sec latency 3518.859 ms 2024-09-15T22:23:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:39 smithi077 ceph-mon[30980]: pgmap v350: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-09-15T22:23:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:39 smithi196 ceph-mon[35109]: pgmap v350: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-09-15T22:23:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:39 smithi117 ceph-mon[35021]: pgmap v350: 129 pgs: 129 active+clean; 578 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 176 op/s 2024-09-15T22:23:40.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.67 MB/sec execute 469 sec latency 4518.928 ms 2024-09-15T22:23:41.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.65 MB/sec execute 470 sec latency 5518.973 ms 2024-09-15T22:23:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:41 smithi077 ceph-mon[30980]: pgmap v351: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 203 op/s 2024-09-15T22:23:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:41 smithi196 ceph-mon[35109]: pgmap v351: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 203 op/s 2024-09-15T22:23:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:41 smithi117 ceph-mon[35021]: pgmap v351: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.0 MiB/s wr, 203 op/s 2024-09-15T22:23:42.488 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.64 MB/sec execute 471 sec latency 6519.033 ms 2024-09-15T22:23:43.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.62 MB/sec execute 472 sec latency 7519.126 ms 2024-09-15T22:23:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:43 smithi077 ceph-mon[30980]: pgmap v352: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:43 smithi196 ceph-mon[35109]: pgmap v352: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:44.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:43 smithi117 ceph-mon[35021]: pgmap v352: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:44.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.61 MB/sec execute 473 sec latency 8519.205 ms 2024-09-15T22:23:45.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.59 MB/sec execute 474 sec latency 9519.235 ms 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: pgmap v353: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:45 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: pgmap v353: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:45 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: pgmap v353: 129 pgs: 129 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-09-15T22:23:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:45 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:46.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.57 MB/sec execute 475 sec latency 10519.320 ms 2024-09-15T22:23:47.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:46 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:46 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:47.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:46 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:47.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:46 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:46 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:46 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:47.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:46 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:47.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:46 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:46 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.365 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:46 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:47.365 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:46 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:47.365 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:46 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:23:47.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.56 MB/sec execute 476 sec latency 11519.420 ms 2024-09-15T22:23:48.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.54 MB/sec execute 477 sec latency 12519.505 ms 2024-09-15T22:23:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:48 smithi077 ceph-mon[30980]: pgmap v354: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 147 op/s 2024-09-15T22:23:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:48 smithi196 ceph-mon[35109]: pgmap v354: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 147 op/s 2024-09-15T22:23:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:48 smithi117 ceph-mon[35021]: pgmap v354: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 147 op/s 2024-09-15T22:23:49.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.53 MB/sec execute 478 sec latency 13519.639 ms 2024-09-15T22:23:50.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.51 MB/sec execute 479 sec latency 14519.690 ms 2024-09-15T22:23:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:50 smithi077 ceph-mon[30980]: pgmap v355: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-09-15T22:23:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:50 smithi196 ceph-mon[35109]: pgmap v355: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-09-15T22:23:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:50 smithi117 ceph-mon[35021]: pgmap v355: 129 pgs: 129 active+clean; 580 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-09-15T22:23:51.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.49 MB/sec execute 480 sec latency 15519.740 ms 2024-09-15T22:23:52.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.48 MB/sec execute 481 sec latency 16519.875 ms 2024-09-15T22:23:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:52 smithi077 ceph-mon[30980]: pgmap v356: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-09-15T22:23:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:52 smithi196 ceph-mon[35109]: pgmap v356: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-09-15T22:23:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:52 smithi117 ceph-mon[35021]: pgmap v356: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-09-15T22:23:53.489 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.46 MB/sec execute 482 sec latency 17519.971 ms 2024-09-15T22:23:54.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.45 MB/sec execute 483 sec latency 18520.085 ms 2024-09-15T22:23:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:54 smithi077 ceph-mon[30980]: pgmap v357: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:54 smithi196 ceph-mon[35109]: pgmap v357: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:54 smithi117 ceph-mon[35021]: pgmap v357: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:54.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:55.237 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:55.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.43 MB/sec execute 484 sec latency 19520.124 ms 2024-09-15T22:23:55.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 -- 172.21.15.77:0/3350916660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc10c170 msgr2=0x7fdebc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 --2- 172.21.15.77:0/3350916660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc10c170 0x7fdebc10e610 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009ca0 tx=0x7fdeb402a950 comp rx=0 tx=0).stop 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 -- 172.21.15.77:0/3350916660 shutdown_connections 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 --2- 172.21.15.77:0/3350916660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc10c170 0x7fdebc10e610 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 --2- 172.21.15.77:0/3350916660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc103660 0x7fdebc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 --2- 172.21.15.77:0/3350916660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc102d40 0x7fdebc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 -- 172.21.15.77:0/3350916660 >> 172.21.15.77:0/3350916660 conn(0x7fdebc0fc7a0 msgr2=0x7fdebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.741+0000 7fdec1da9640 1 -- 172.21.15.77:0/3350916660 shutdown_connections 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 -- 172.21.15.77:0/3350916660 wait complete. 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 Processor -- start 2024-09-15T22:23:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 -- start start 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 0x7fdebc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc10c170 0x7fdebc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc111f70 con 0x7fdebc102d40 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc111df0 con 0x7fdebc10c170 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.742+0000 7fdec1da9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc1120f0 con 0x7fdebc103660 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34350/0 (socket says 172.21.15.77:34350) 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 -- 172.21.15.77:0/2183548421 learned_addr learned my addr 172.21.15.77:0/2183548421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebbfff640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc10c170 0x7fdebc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebaffd640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 0x7fdebc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 -- 172.21.15.77:0/2183548421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 msgr2=0x7fdebc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 0x7fdebc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 -- 172.21.15.77:0/2183548421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc10c170 msgr2=0x7fdebc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc10c170 0x7fdebc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 -- 172.21.15.77:0/2183548421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1a68d0 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebaffd640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 0x7fdebc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdebb7fe640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00d770 tx=0x7fdeac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac017070 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1a6bc0 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeac005500 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004e00 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.743+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc1a70b0 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.744+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc103120 con 0x7fdebc102d40 2024-09-15T22:23:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.745+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdeac004fa0 con 0x7fdebc102d40 2024-09-15T22:23:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.746+0000 7fdeb8ff9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 0x7fdea40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.746+0000 7fdebaffd640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 0x7fdea40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.746+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fdeac0cfc40 con 0x7fdebc102d40 2024-09-15T22:23:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.747+0000 7fdebaffd640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 0x7fdea40a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005fd0 tx=0x7fdeb0005f00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.750+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdeac09c230 con 0x7fdebc102d40 2024-09-15T22:23:55.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.951+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdebc1a7390 con 0x7fdebc102d40 2024-09-15T22:23:55.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.951+0000 7fdeb8ff9640 1 -- 172.21.15.77:0/2183548421 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fdeac09bb30 con 0x7fdebc102d40 2024-09-15T22:23:55.953 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:23:55.953 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:23:55.954 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 msgr2=0x7fdea40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 0x7fdea40a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005fd0 tx=0x7fdeb0005f00 comp rx=0 tx=0).stop 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 msgr2=0x7fdebc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00d770 tx=0x7fdeac00dc30 comp rx=0 tx=0).stop 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 shutdown_connections 2024-09-15T22:23:55.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdea40a33b0 0x7fdea40a5870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdebc10c170 0x7fdebc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc103660 0x7fdebc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 --2- 172.21.15.77:0/2183548421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdebc102d40 0x7fdebc1009a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 >> 172.21.15.77:0/2183548421 conn(0x7fdebc0fc7a0 msgr2=0x7fdebc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 shutdown_connections 2024-09-15T22:23:55.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:55.954+0000 7fdec1da9640 1 -- 172.21.15.77:0/2183548421 wait complete. 2024-09-15T22:23:56.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.42 MB/sec execute 485 sec latency 20520.182 ms 2024-09-15T22:23:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:56 smithi077 ceph-mon[30980]: pgmap v358: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2183548421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:56 smithi196 ceph-mon[35109]: pgmap v358: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:56 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2183548421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:56 smithi117 ceph-mon[35021]: pgmap v358: 129 pgs: 129 active+clean; 586 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-15T22:23:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:56 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2183548421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:56.737 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 500ee9b1 --release --await' 2024-09-15T22:23:56.737 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 500ee9b1 --release --await 2024-09-15T22:23:57.001 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:57.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 877670 7.40 MB/sec execute 486 sec latency 21520.480 ms 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/2526171791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4102790 msgr2=0x7fa1e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/2526171791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4102790 0x7fa1e41067e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009900 tx=0x7fa1e002a410 comp rx=0 tx=0).stop 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/2526171791 shutdown_connections 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/2526171791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4106f10 0x7fa1e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/2526171791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4102790 0x7fa1e41067e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/2526171791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/2526171791 >> 172.21.15.77:0/2526171791 conn(0x7fa1e40fd8c0 msgr2=0x7fa1e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/2526171791 shutdown_connections 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.677+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/2526171791 wait complete. 2024-09-15T22:23:57.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 Processor -- start 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 -- start start 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 0x7fa1e4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4111fc0 con 0x7fa1e4106f10 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4111e40 con 0x7fa1e4101de0 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.678+0000 7fa1ec4a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4112140 con 0x7fa1e4102790 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34372/0 (socket says 172.21.15.77:34372) 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 -- 172.21.15.77:0/1652106178 learned_addr learned my addr 172.21.15.77:0/1652106178 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 0x7fa1e4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 -- 172.21.15.77:0/1652106178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 msgr2=0x7fa1e4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 0x7fa1e4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 -- 172.21.15.77:0/1652106178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 msgr2=0x7fa1e4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 -- 172.21.15.77:0/1652106178 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e419d3b0 con 0x7fa1e4106f10 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 0x7fa1e4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1eaa3e640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00eaa0 tx=0x7fa1dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1db7fe640 1 -- 172.21.15.77:0/1652106178 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00cdb0 con 0x7fa1e4106f10 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e419d6a0 con 0x7fa1e4106f10 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1db7fe640 1 -- 172.21.15.77:0/1652106178 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc005020 con 0x7fa1e4106f10 2024-09-15T22:23:57.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1db7fe640 1 -- 172.21.15.77:0/1652106178 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0116d0 con 0x7fa1e4106f10 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.679+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e41a4f80 con 0x7fa1e4106f10 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.681+0000 7fa1db7fe640 1 -- 172.21.15.77:0/1652106178 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa1dc0054d0 con 0x7fa1e4106f10 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 Processor -- start 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 -- start start 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dad8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30dae5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f30da3d620 con 0x55f30dad8980 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f30d9e4e50 con 0x55f30dae3240 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f30da395e0 con 0x55f30dad8060 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ea23d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ea23d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34376/0 (socket says 172.21.15.77:34376) 2024-09-15T22:23:57.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1ea23d640 1 -- 172.21.15.77:0/851803483 learned_addr learned my addr 172.21.15.77:0/851803483 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1db7fe640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 0x7fa1d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1eaa3e640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 0x7fa1d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.682+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30dae5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dad8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 -- 172.21.15.77:0/851803483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 msgr2=0x55f30dad8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dad8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 -- 172.21.15.77:0/851803483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 msgr2=0x55f30dae5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30dae5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 -- 172.21.15.77:0/851803483 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30d9d5d40 con 0x55f30dad8980 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1db7fe640 1 -- 172.21.15.77:0/1652106178 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa1dc0d0360 con 0x7fa1e4106f10 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dad8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:57.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400cca0 tx=0x7fa1d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbf707e05e8c6a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1eaa3e640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 0x7fa1d00a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cd7d0 tx=0x7fa1dc08e040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1d8ff9640 1 -- 172.21.15.77:0/851803483 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4005220 con 0x55f30dad8980 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1d8ff9640 1 -- 172.21.15.77:0/851803483 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d4004c80 con 0x55f30dad8980 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.683+0000 7fa1d8ff9640 1 -- 172.21.15.77:0/851803483 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4010890 con 0x55f30dad8980 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- 172.21.15.77:0/851803483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 msgr2=0x55f30dae2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400cca0 tx=0x7fa1d400e720 comp rx=0 tx=0).stop 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- 172.21.15.77:0/851803483 shutdown_connections 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30dae5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae2d00 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/851803483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dad8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- 172.21.15.77:0/851803483 >> 172.21.15.77:0/851803483 conn(0x55f30dad45f0 msgr2=0x55f30dad49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- 172.21.15.77:0/851803483 shutdown_connections 2024-09-15T22:23:57.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- 172.21.15.77:0/851803483 wait complete. 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- start start 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 Processor -- start 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dae0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f30dae5e10 con 0x55f30dae3240 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f30d9c5f10 con 0x55f30dad8980 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.684+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f30dae5f90 con 0x55f30dad8060 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1ee87a740 10 client.0 _ll_register_callbacks cb 0x55f30d9e9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34384/0 (socket says 172.21.15.77:34384) 2024-09-15T22:23:57.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/2423008539 learned_addr learned my addr 172.21.15.77:0/2423008539 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dae0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/2423008539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 msgr2=0x55f30dae00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 msgr2=0x55f30dae0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dae0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/2423008539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f30d74c580 con 0x55f30dae3240 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0033000 tx=0x7fa1e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ba4603a7ad60ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0035040 con 0x55f30dae3240 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa1e0003e40 con 0x55f30dae3240 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d4004c80 con 0x55f30dae3240 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1e0005020 con 0x55f30dae3240 2024-09-15T22:23:57.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1ee87a740 10 client.18315 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f30d816960 con 0x55f30dae3240 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.685+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d4004c80 con 0x55f30dae3240 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa1e003a040 con 0x55f30dae3240 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 10 client.18315 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 10 client.18315 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 20 client.18315 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30daf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 4 client.18315 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f30da4ab90 con 0x55f30daf2cd0 2024-09-15T22:23:57.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30daf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1cdffb640 10 client.18315 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.686+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30daf50c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb1a4a4109b2dcf server_cookie=7fccb78fb31d8af6 in_seq=0 out_seq=0 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1cdffb640 1 -- 172.21.15.77:0/2423008539 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f30da4ab90 con 0x55f30daf2cd0 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1cdffb640 10 client.18315 handle_command_reply: tid=0 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 client.18315 shutdown 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 msgr2=0x55f30dae0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0033000 tx=0x7fa1e0009260 comp rx=0 tx=0).stop 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 shutdown_connections 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30daf50c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1cdffb640 0 client.18315 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dae0b60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dae0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/2423008539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 >> 172.21.15.77:0/2423008539 conn(0x55f30dad45f0 msgr2=0x55f30dada2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 shutdown_connections 2024-09-15T22:23:57.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 1 -- 172.21.15.77:0/2423008539 wait complete. 2024-09-15T22:23:57.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.687+0000 7fa1ee87a740 20 client.18315 trim_cache size 0 max 16384 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 Processor -- start 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 -- start start 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dadaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dad3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41af000 con 0x55f30dae3240 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41b0b40 con 0x55f30dad8980 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41af6d0 con 0x55f30dad8060 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.704+0000 7fa1e9a3c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34394/0 (socket says 172.21.15.77:34394) 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/3276191976 learned_addr learned my addr 172.21.15.77:0/3276191976 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:57.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dad3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dadaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/3276191976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 msgr2=0x55f30dadaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dadaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/3276191976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 msgr2=0x55f30dad3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dad3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 -- 172.21.15.77:0/3276191976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30dad87b0 con 0x55f30dae3240 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00059d0 tx=0x7fa1e0043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f5754de1fde751 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1cffff640 1 -- 172.21.15.77:0/3276191976 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0035070 con 0x55f30dae3240 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1cffff640 1 -- 172.21.15.77:0/3276191976 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0003e20 con 0x55f30dae3240 2024-09-15T22:23:57.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.705+0000 7fa1cffff640 1 -- 172.21.15.77:0/3276191976 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0005420 con 0x55f30dae3240 2024-09-15T22:23:57.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3276191976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 msgr2=0x55f30dafa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00059d0 tx=0x7fa1e0043040 comp rx=0 tx=0).stop 2024-09-15T22:23:57.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3276191976 shutdown_connections 2024-09-15T22:23:57.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dae3240 0x55f30dafa920 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dad8980 0x55f30dad3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3276191976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dadaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3276191976 >> 172.21.15.77:0/3276191976 conn(0x55f30dad45f0 msgr2=0x55f30dad2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3276191976 shutdown_connections 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3276191976 wait complete. 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 -- start start 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.706+0000 7fa1ee87a740 1 Processor -- start 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30db5c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41b0470 con 0x55f30dad8980 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41afda0 con 0x55f30dae3240 2024-09-15T22:23:57.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41ad000 con 0x55f30dad8060 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ee87a740 10 client.0 _ll_register_callbacks cb 0x7fa1e0003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34408/0 (socket says 172.21.15.77:34408) 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 -- 172.21.15.77:0/3239645335 learned_addr learned my addr 172.21.15.77:0/3239645335 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30db5c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 -- 172.21.15.77:0/3239645335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 msgr2=0x55f30dae7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 -- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 msgr2=0x55f30db5c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30db5c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 -- 172.21.15.77:0/3239645335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f30db5ce30 con 0x55f30dad8980 2024-09-15T22:23:57.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30db5c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1ea23d640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400ec20 tx=0x7fa1d40104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c55240862482e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.707+0000 7fa1cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4004a70 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa1d4004c10 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e0003e20 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1d4036050 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 10 client.18321 fetch_fsmap learned FSMap version 34 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f30db5e150 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0003e20 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa1d4010aa0 con 0x55f30dad8980 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 10 client.18321 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:23:57.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 10 client.18321 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 20 client.18321 populate_metadata read hostname 'smithi077' 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30db60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 4 client.18321 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "500ee9b1", "release": true, "await": true}] 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.708+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "500ee9b1", "release": true, "await": true}) -- 0x55f30da450a0 con 0x55f30daf2cd0 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.709+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30db60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.709+0000 7fa1e9a3c640 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30db60600 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65f07d8187f38d1 server_cookie=aa287619e4242d3f in_seq=0 out_seq=0 2024-09-15T22:23:57.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.709+0000 7fa1cdffb640 10 client.18321 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:57.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.717+0000 7fa1cdffb640 1 -- 172.21.15.77:0/3239645335 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f30da450a0 con 0x55f30daf2cd0 2024-09-15T22:23:57.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.717+0000 7fa1cdffb640 10 client.18321 handle_command_reply: tid=0 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.717+0000 7fa1ee87a740 1 client.18321 shutdown 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 msgr2=0x55f30dae7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400ec20 tx=0x7fa1d40104d0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 shutdown_connections 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f30daf2cd0 0x55f30db60600 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f30dae3240 0x55f30db5c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f30dad8980 0x55f30dae7bc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 --2- 172.21.15.77:0/3239645335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f30dad8060 0x55f30dae7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 >> 172.21.15.77:0/3239645335 conn(0x55f30dad45f0 msgr2=0x55f30dae00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1cdffb640 0 client.18321 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:23:57.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 shutdown_connections 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 1 -- 172.21.15.77:0/3239645335 wait complete. 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ee87a740 20 client.18321 trim_cache size 0 max 16384 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 40, 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:23:57.721 INFO:teuthology.orchestra.run.smithi077.stdout: "500ee9b1": { 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 40, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 141.8, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:23:57.722 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:23:57.723 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 msgr2=0x7fa1d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 0x7fa1d00a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cd7d0 tx=0x7fa1dc08e040 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 msgr2=0x7fa1e419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00eaa0 tx=0x7fa1dc00ef60 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 shutdown_connections 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1d00a33b0 0x7fa1d00a5870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1e4106f10 0x7fa1e419ccd0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e4102790 0x7fa1e4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.718+0000 7fa1ec4a1640 1 --2- 172.21.15.77:0/1652106178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e4101de0 0x7fa1e4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.719+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 >> 172.21.15.77:0/1652106178 conn(0x7fa1e40fd8c0 msgr2=0x7fa1e40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:57.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.719+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 shutdown_connections 2024-09-15T22:23:57.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:57.719+0000 7fa1ec4a1640 1 -- 172.21.15.77:0/1652106178 wait complete. 2024-09-15T22:23:58.443 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '500ee9b1', total seconds elapsed: 25.3 2024-09-15T22:23:58.443 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-15T22:23:58.444 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:58.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 879356 7.40 MB/sec execute 487 sec latency 21747.657 ms 2024-09-15T22:23:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:23:58 smithi077 ceph-mon[30980]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.1 MiB/s wr, 83 op/s 2024-09-15T22:23:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:23:58 smithi196 ceph-mon[35109]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.1 MiB/s wr, 83 op/s 2024-09-15T22:23:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:23:58 smithi117 ceph-mon[35021]: pgmap v359: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 2.1 MiB/s wr, 83 op/s 2024-09-15T22:23:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:59.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.272+0000 7f616bebf640 1 -- 172.21.15.77:0/61962971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164103780 msgr2=0x7f6164107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.272+0000 7f616bebf640 1 --2- 172.21.15.77:0/61962971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164103780 0x7f6164107bc0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6154009930 tx=0x7f615402a420 comp rx=0 tx=0).stop 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 -- 172.21.15.77:0/61962971 shutdown_connections 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 --2- 172.21.15.77:0/61962971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164108300 0x7f616410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 --2- 172.21.15.77:0/61962971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164103780 0x7f6164107bc0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 --2- 172.21.15.77:0/61962971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164102dd0 0x7f61641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 -- 172.21.15.77:0/61962971 >> 172.21.15.77:0/61962971 conn(0x7f61640fc7d0 msgr2=0x7f61640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 -- 172.21.15.77:0/61962971 shutdown_connections 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 -- 172.21.15.77:0/61962971 wait complete. 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 Processor -- start 2024-09-15T22:23:59.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.273+0000 7f616bebf640 1 -- start start 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164103780 0x7f6164100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 0x7f61641014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61641120c0 con 0x7f6164108300 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6164111f40 con 0x7f6164102dd0 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6164112240 con 0x7f6164103780 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616a45c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 0x7f61641014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:44126/0 (socket says 172.21.15.77:44126) 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616a45c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 0x7f61641014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58914/0 (socket says 172.21.15.77:58914) 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 -- 172.21.15.77:0/4103865269 learned_addr learned my addr 172.21.15.77:0/4103865269 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:23:59.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616945a640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164103780 0x7f6164100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 -- 172.21.15.77:0/4103865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164103780 msgr2=0x7f6164100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164103780 0x7f6164100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 -- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 msgr2=0x7f61641014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 0x7f61641014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 -- 172.21.15.77:0/4103865269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61641a6ac0 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f6169c5b640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f616000e9f0 tx=0x7f616000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616000cd80 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641a6db0 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6160005500 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160010690 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.274+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641a7210 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.275+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61641031b0 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.277+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61600046c0 con 0x7f6164102dd0 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.277+0000 7f615affd640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 0x7f61500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:59.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.277+0000 7f616945a640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 0x7f61500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:59.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.278+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f61600d0660 con 0x7f6164102dd0 2024-09-15T22:23:59.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.278+0000 7f616945a640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 0x7f61500a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f615402a940 tx=0x7f6154005a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:59.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.281+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61600541b0 con 0x7f6164102dd0 2024-09-15T22:23:59.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 880783 7.41 MB/sec execute 488 sec latency 67.929 ms 2024-09-15T22:23:59.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.502+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6164101f60 con 0x7f6164102dd0 2024-09-15T22:23:59.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.503+0000 7f615affd640 1 -- 172.21.15.77:0/4103865269 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6164101f60 con 0x7f6164102dd0 2024-09-15T22:23:59.505 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:23:59.505 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:23:59.505 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 msgr2=0x7f61500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 0x7f61500a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f615402a940 tx=0x7f6154005a40 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 msgr2=0x7f6164100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f616000e9f0 tx=0x7f616000edf0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 shutdown_connections 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61500a33b0 0x7f61500a5870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6164108300 0x7f61641014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6164103780 0x7f6164100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 --2- 172.21.15.77:0/4103865269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102dd0 0x7f6164100a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:59.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.505+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 >> 172.21.15.77:0/4103865269 conn(0x7f61640fc7d0 msgr2=0x7f6164105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:59.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.506+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 shutdown_connections 2024-09-15T22:23:59.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:23:59.506+0000 7f616bebf640 1 -- 172.21.15.77:0/4103865269 wait complete. 2024-09-15T22:23:59.961 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:24:00.220 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-15T22:24:00.220 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-15T22:24:00.264 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:00.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 882456 7.41 MB/sec execute 489 sec latency 32.762 ms 2024-09-15T22:24:00.537 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:00 smithi077 ceph-mon[30980]: pgmap v360: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-09-15T22:24:00.538 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4103865269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:00 smithi196 ceph-mon[35109]: pgmap v360: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-09-15T22:24:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4103865269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:00 smithi117 ceph-mon[35021]: pgmap v360: 129 pgs: 129 active+clean; 584 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2024-09-15T22:24:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4103865269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:00.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/4195562119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 msgr2=0x7f4fc41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/4195562119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc41021c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009900 tx=0x7f4fb402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/4195562119 shutdown_connections 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/4195562119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 0x7f4fc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/4195562119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/4195562119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc41021c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/4195562119 >> 172.21.15.77:0/4195562119 conn(0x7f4fc40fd8c0 msgr2=0x7f4fc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/4195562119 shutdown_connections 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.812+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/4195562119 wait complete. 2024-09-15T22:24:00.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.813+0000 7f4fc9fd8640 1 Processor -- start 2024-09-15T22:24:00.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.813+0000 7f4fc9fd8640 1 -- start start 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 0x7f4fc4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4111f30 con 0x7f4fc4101de0 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4111db0 con 0x7f4fc4106f10 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc9fd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41120b0 con 0x7f4fc4102790 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58946/0 (socket says 172.21.15.77:58946) 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc37fe640 1 -- 172.21.15.77:0/1345583583 learned_addr learned my addr 172.21.15.77:0/1345583583 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48462/0 (socket says 172.21.15.77:48462) 2024-09-15T22:24:00.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 0x7f4fc4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/1345583583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 msgr2=0x7f4fc4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 0x7f4fc4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 msgr2=0x7f4fc40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.814+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/1345583583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc41a6930 con 0x7f4fc4102790 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc0ff9640 1 -- 172.21.15.77:0/1345583583 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800ccc0 con 0x7f4fc4102790 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a6c20 con 0x7f4fc4102790 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc0ff9640 1 -- 172.21.15.77:0/1345583583 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb8010910 con 0x7f4fc4102790 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a71b0 con 0x7f4fc4102790 2024-09-15T22:24:00.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.815+0000 7f4fc0ff9640 1 -- 172.21.15.77:0/1345583583 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018960 con 0x7f4fc4102790 2024-09-15T22:24:00.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.818+0000 7f4fc0ff9640 1 -- 172.21.15.77:0/1345583583 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4fb8018b00 con 0x7f4fc4102790 2024-09-15T22:24:00.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.818+0000 7f4fcc3b1740 1 Processor -- start 2024-09-15T22:24:00.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 -- start start 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 0x55d502018950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 0x55d50201b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d501f6b2e0 con 0x55d502018e90 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d501f12ce0 con 0x55d50200e5d0 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d501f672a0 con 0x55d50200dcb0 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48468/0 (socket says 172.21.15.77:48468) 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fc3fff640 1 -- 172.21.15.77:0/427664264 learned_addr learned my addr 172.21.15.77:0/427664264 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 0x55d50201b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.819+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 0x55d502018950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc0ff9640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 0x7f4fac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 -- 172.21.15.77:0/427664264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 msgr2=0x55d50200e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 -- 172.21.15.77:0/427664264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 msgr2=0x55d50201b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 0x55d50201b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 -- 172.21.15.77:0/427664264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d501ef7440 con 0x55d50200e5d0 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 0x7f4fac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 0x55d50201b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:00.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc0ff9640 1 -- 172.21.15.77:0/1345583583 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4fb80d0470 con 0x7f4fc4102790 2024-09-15T22:24:00.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.820+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 0x55d502018950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009230 tx=0x7f4fb4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac399e2f44a53c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.821+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 0x7f4fac0a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009fd0 tx=0x7f4fbc004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.821+0000 7f4fa9ffb640 1 -- 172.21.15.77:0/427664264 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4040070 con 0x55d50200e5d0 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.821+0000 7f4fa9ffb640 1 -- 172.21.15.77:0/427664264 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb40317b0 con 0x55d50200e5d0 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.821+0000 7f4fa9ffb640 1 -- 172.21.15.77:0/427664264 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035040 con 0x55d50200e5d0 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/427664264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 msgr2=0x55d502018950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 0x55d502018950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009230 tx=0x7f4fb4005e80 comp rx=0 tx=0).stop 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/427664264 shutdown_connections 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d502018e90 0x55d50201b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d50200e5d0 0x55d502018950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/427664264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d50200e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/427664264 >> 172.21.15.77:0/427664264 conn(0x55d50200a240 msgr2=0x55d50200a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.822+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/427664264 shutdown_connections 2024-09-15T22:24:00.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.823+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/427664264 wait complete. 2024-09-15T22:24:00.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.823+0000 7f4fcc3b1740 1 -- start start 2024-09-15T22:24:00.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.823+0000 7f4fcc3b1740 1 Processor -- start 2024-09-15T22:24:00.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502018780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d502015f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d50201ba60 con 0x55d50200e5d0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d501f26050 con 0x55d502018e90 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d50201bbe0 con 0x55d50200dcb0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58970/0 (socket says 172.21.15.77:58970) 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502018780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3702703735 learned_addr learned my addr 172.21.15.77:0/3702703735 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d502015f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fcc3b1740 10 client.0 _ll_register_callbacks cb 0x55d501f66160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3702703735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 msgr2=0x55d502018780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502018780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 msgr2=0x55d502015f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d502015f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.824+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3702703735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d501cdc3c0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c6a0 tx=0x7f4fb8098050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40511a0c17c7fab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb809c240 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4fb8010ac0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb40317b0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4fb809bdd0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fcc3b1740 10 client.18339 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5020273b0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.825+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb40317b0 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4fb80d0120 con 0x55d50200e5d0 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 10 client.18339 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:00.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 20 client.18339 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d50202ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 4 client.18339 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d501f78850 con 0x55d5020288c0 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.826+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d50202ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.827+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d50202ad10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5457eae55ec5f03 server_cookie=5baccba26c28691b in_seq=0 out_seq=0 2024-09-15T22:24:00.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.827+0000 7f4fa2ffd640 10 client.18339 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:00.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.827+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3702703735 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4fb404fe80 con 0x55d5020288c0 2024-09-15T22:24:00.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.827+0000 7f4fa2ffd640 10 client.18339 handle_command_reply: tid=0 2024-09-15T22:24:00.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 client.18339 shutdown 2024-09-15T22:24:00.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 msgr2=0x55d502015a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c6a0 tx=0x7f4fb8098050 comp rx=0 tx=0).stop 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 shutdown_connections 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d50202ad10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d502015f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fa2ffd640 0 client.18339 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502015a30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.828+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3702703735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502018780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.829+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 >> 172.21.15.77:0/3702703735 conn(0x55d50200a240 msgr2=0x55d502000b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.830+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 shutdown_connections 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.830+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3702703735 wait complete. 2024-09-15T22:24:00.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.830+0000 7f4fcc3b1740 20 client.18339 trim_cache size 0 max 16384 2024-09-15T22:24:00.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 Processor -- start 2024-09-15T22:24:00.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 -- start start 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502011000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d5020305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ace10 con 0x55d50200e5d0 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ae950 con 0x55d502018e90 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ad4e0 con 0x55d50200dcb0 2024-09-15T22:24:00.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48496/0 (socket says 172.21.15.77:48496) 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fc37fe640 1 -- 172.21.15.77:0/3457801007 learned_addr learned my addr 172.21.15.77:0/3457801007 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502011000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.855+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d5020305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 -- 172.21.15.77:0/3457801007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 msgr2=0x55d5020305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d5020305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 -- 172.21.15.77:0/3457801007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 msgr2=0x55d502011000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502011000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 -- 172.21.15.77:0/3457801007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50200a070 con 0x55d50200dcb0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d5020305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502011000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55d50200b0a0 tx=0x7f4fb40051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd462f00e098186 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fa97fa640 1 -- 172.21.15.77:0/3457801007 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4052020 con 0x55d50200dcb0 2024-09-15T22:24:00.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fa97fa640 1 -- 172.21.15.77:0/3457801007 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb404a040 con 0x55d50200dcb0 2024-09-15T22:24:00.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.856+0000 7f4fa97fa640 1 -- 172.21.15.77:0/3457801007 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4035040 con 0x55d50200dcb0 2024-09-15T22:24:00.860 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3457801007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 msgr2=0x55d502024cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55d50200b0a0 tx=0x7f4fb40051c0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3457801007 shutdown_connections 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d5020305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d502011000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3457801007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502024cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3457801007 >> 172.21.15.77:0/3457801007 conn(0x55d50200a240 msgr2=0x55d5020108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3457801007 shutdown_connections 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.857+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3457801007 wait complete. 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 -- start start 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 Processor -- start 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502021650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d50201cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ae280 con 0x55d50200e5d0 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41adbb0 con 0x55d502018e90 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41aae10 con 0x55d50200dcb0 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fcc3b1740 10 client.0 _ll_register_callbacks cb 0x7f4fb800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.858+0000 7f4fc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59000/0 (socket says 172.21.15.77:59000) 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3010924201 learned_addr learned my addr 172.21.15.77:0/3010924201 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:00.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502021650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d50201cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3010924201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 msgr2=0x55d502021650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502021650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 msgr2=0x55d50201cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d50201cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 -- 172.21.15.77:0/3010924201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50201d750 con 0x55d50200e5d0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc3fff640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d50201cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fc2ffd640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4865bff57ea56e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb809bfb0 con 0x55d50200e5d0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4fb8010ac0 con 0x55d50200e5d0 2024-09-15T22:24:00.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.859+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb404a040 con 0x55d50200e5d0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4fb809b8c0 con 0x55d50200e5d0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 10 client.18345 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d502093600 con 0x55d50200e5d0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb404a040 con 0x55d50200e5d0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4fb80981c0 con 0x55d50200e5d0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 10 client.18345 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 20 client.18345 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d502095c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 4 client.18345 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.860+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d502016300 con 0x55d5020288c0 2024-09-15T22:24:00.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d502095c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fa2ffd640 10 client.18345 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fc37fe640 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d502095c30 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592ab1261eff4ab1 server_cookie=1627ca0243e0049a in_seq=0 out_seq=0 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fa2ffd640 1 -- 172.21.15.77:0/3010924201 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d502016300 con 0x55d5020288c0 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fa2ffd640 10 client.18345 handle_command_reply: tid=0 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.861+0000 7f4fcc3b1740 1 client.18345 shutdown 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 msgr2=0x55d50201c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb800cea0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 shutdown_connections 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5020288c0 0x55d502095c30 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d502018e90 0x55d50201cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d50200e5d0 0x55d50201c980 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 --2- 172.21.15.77:0/3010924201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d50200dcb0 0x55d502021650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fa2ffd640 0 client.18345 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 >> 172.21.15.77:0/3010924201 conn(0x55d50200a240 msgr2=0x55d50200f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 shutdown_connections 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 1 -- 172.21.15.77:0/3010924201 wait complete. 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.862+0000 7f4fcc3b1740 20 client.18345 trim_cache size 0 max 16384 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:24:00.868 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 msgr2=0x7f4fac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 0x7f4fac0a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009fd0 tx=0x7f4fbc004a40 comp rx=0 tx=0).stop 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 msgr2=0x7f4fc4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 shutdown_connections 2024-09-15T22:24:00.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fac0a33b0 0x7f4fac0a5870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fc4106f10 0x7f4fc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fc4102790 0x7f4fc4074a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 --2- 172.21.15.77:0/1345583583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4101de0 0x7f4fc40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 >> 172.21.15.77:0/1345583583 conn(0x7f4fc40fd8c0 msgr2=0x7f4fc40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 shutdown_connections 2024-09-15T22:24:00.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:00.863+0000 7f4fc9fd8640 1 -- 172.21.15.77:0/1345583583 wait complete. 2024-09-15T22:24:01.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883708 7.41 MB/sec execute 490 sec latency 526.876 ms 2024-09-15T22:24:01.855 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:24:01.855 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.062+0000 7f350fa03640 1 -- 172.21.15.77:0/2661733853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102ef0 msgr2=0x7f350810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.062+0000 7f350fa03640 1 --2- 172.21.15.77:0/2661733853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102ef0 0x7f350810da40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3500009ca0 tx=0x7f350002a950 comp rx=0 tx=0).stop 2024-09-15T22:24:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 -- 172.21.15.77:0/2661733853 shutdown_connections 2024-09-15T22:24:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 --2- 172.21.15.77:0/2661733853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102ef0 0x7f350810da40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 --2- 172.21.15.77:0/2661733853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508102570 0x7f35081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 --2- 172.21.15.77:0/2661733853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508100e20 0x7f3508101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 -- 172.21.15.77:0/2661733853 >> 172.21.15.77:0/2661733853 conn(0x7f35080fc7d0 msgr2=0x7f35080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 -- 172.21.15.77:0/2661733853 shutdown_connections 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 -- 172.21.15.77:0/2661733853 wait complete. 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 Processor -- start 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.063+0000 7f350fa03640 1 -- start start 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110f80 con 0x7f3508102570 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110e00 con 0x7f3508100e20 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350fa03640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3508111100 con 0x7f3508102ef0 2024-09-15T22:24:02.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59012/0 (socket says 172.21.15.77:59012) 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350cf9e640 1 -- 172.21.15.77:0/2179733920 learned_addr learned my addr 172.21.15.77:0/2179733920 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:44218/0 (socket says 172.21.15.77:44218) 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 -- 172.21.15.77:0/2179733920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508102ef0 msgr2=0x7f35080fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 -- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 msgr2=0x7f35080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.064+0000 7f350d79f640 1 -- 172.21.15.77:0/2179733920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081a69c0 con 0x7f3508100e20 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f350d79f640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9f0 tx=0x7f350400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f34fe7fc640 1 -- 172.21.15.77:0/2179733920 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350400ccc0 con 0x7f3508100e20 2024-09-15T22:24:02.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081a6cb0 con 0x7f3508100e20 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f34fe7fc640 1 -- 172.21.15.77:0/2179733920 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3504005020 con 0x7f3508100e20 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f34fe7fc640 1 -- 172.21.15.77:0/2179733920 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040049e0 con 0x7f3508100e20 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.065+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a71a0 con 0x7f3508100e20 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 Processor -- start 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 -- start start 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fe94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774febe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556774f43160 con 0x556774fde850 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556774eea720 con 0x556774fe9a30 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.068+0000 7f3511ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556774f3f120 con 0x556774fdf170 2024-09-15T22:24:02.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f34fe7fc640 1 -- 172.21.15.77:0/2179733920 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3504004cc0 con 0x7f3508100e20 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350dfa0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350dfa0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59022/0 (socket says 172.21.15.77:59022) 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350dfa0640 1 -- 172.21.15.77:0/188358983 learned_addr learned my addr 172.21.15.77:0/188358983 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774febe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 -- 172.21.15.77:0/188358983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 msgr2=0x556774fe94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fe94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 -- 172.21.15.77:0/188358983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 msgr2=0x556774fdec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 -- 172.21.15.77:0/188358983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556774cb4ba0 con 0x556774fe9a30 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350dfa0640 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f350cf9e640 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774febe20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009c00 tx=0x7f34f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73d9b5d3e532ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.069+0000 7f34f37fe640 1 -- 172.21.15.77:0/188358983 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004230 con 0x556774fe9a30 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f34f37fe640 1 -- 172.21.15.77:0/188358983 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f80043d0 con 0x556774fe9a30 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f34f37fe640 1 -- 172.21.15.77:0/188358983 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004d90 con 0x556774fe9a30 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f34fe7fc640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 0x7f34f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f34fe7fc640 1 -- 172.21.15.77:0/2179733920 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f35040cffa0 con 0x7f3508100e20 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 0x7f34f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 -- 172.21.15.77:0/188358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 msgr2=0x556774febe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774febe20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009c00 tx=0x7f34f8011810 comp rx=0 tx=0).stop 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 -- 172.21.15.77:0/188358983 shutdown_connections 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774febe20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fe94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 --2- 172.21.15.77:0/188358983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fdec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.070+0000 7f3511ddc740 1 -- 172.21.15.77:0/188358983 >> 172.21.15.77:0/188358983 conn(0x556774fdade0 msgr2=0x556774fdb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.071+0000 7f3511ddc740 1 -- 172.21.15.77:0/188358983 shutdown_connections 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.071+0000 7f3511ddc740 1 -- 172.21.15.77:0/188358983 wait complete. 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.071+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 0x7f34f40a58f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3500009ca0 tx=0x7f3500005c50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.071+0000 7f3511ddc740 1 -- start start 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.071+0000 7f3511ddc740 1 Processor -- start 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 0x556774fe6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fe9a30 0x556774fe7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556774fec600 con 0x556774fdf170 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556774ecb900 con 0x556774fde850 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556774fec780 con 0x556774fe9a30 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f3511ddc740 10 client.0 _ll_register_callbacks cb 0x556774ec9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 0x556774fe6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59034/0 (socket says 172.21.15.77:59034) 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350cf9e640 1 -- 172.21.15.77:0/1083216208 learned_addr learned my addr 172.21.15.77:0/1083216208 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350dfa0640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fe9a30 0x556774fe7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 -- 172.21.15.77:0/1083216208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fe9a30 msgr2=0x556774fe7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fe9a30 0x556774fe7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 -- 172.21.15.77:0/1083216208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 msgr2=0x556774fe6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350d79f640 1 -- 172.21.15.77:0/1083216208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556774f0bc50 con 0x556774fde850 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.072+0000 7f350cf9e640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f350d79f640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 0x556774fe6900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35040cd360 tx=0x7f350409b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920476e37e232995 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350401c040 con 0x556774fde850 2024-09-15T22:24:02.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f35040d20f0 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f80043d0 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f350409bef0 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 10 client.27374 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556774ff82f0 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f80043d0 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f35040bab30 con 0x556774fde850 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 10 client.27374 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 10 client.27374 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 20 client.27374 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556774ff9700 0x556774ffbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 4 client.27374 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.073+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556774f50890 con 0x556774ff9700 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.074+0000 7f350cf9e640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556774ff9700 0x556774ffbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.074+0000 7f34f0ff9640 10 client.27374 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.074+0000 7f350cf9e640 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556774ff9700 0x556774ffbaf0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d1aba0917277aa server_cookie=dc56e2d42733c16e in_seq=0 out_seq=0 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f34f0ff9640 1 -- 172.21.15.77:0/1083216208 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556774f50890 con 0x556774ff9700 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f34f0ff9640 10 client.27374 handle_command_reply: tid=0 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 client.27374 shutdown 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 msgr2=0x556774fe6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 0x556774fe6900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35040cd360 tx=0x7f350409b910 comp rx=0 tx=0).stop 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 shutdown_connections 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556774ff9700 0x556774ffbaf0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fe9a30 0x556774fe7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fdf170 0x556774fe6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 --2- 172.21.15.77:0/1083216208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fde850 0x556774fe6900 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 >> 172.21.15.77:0/1083216208 conn(0x556774fdade0 msgr2=0x556774fe0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.075+0000 7f34f0ff9640 0 client.27374 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.076+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 shutdown_connections 2024-09-15T22:24:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.076+0000 7f3511ddc740 1 -- 172.21.15.77:0/1083216208 wait complete. 2024-09-15T22:24:02.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.077+0000 7f3511ddc740 20 client.27374 trim_cache size 0 max 16384 2024-09-15T22:24:02.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.096+0000 7f3511ddc740 1 Processor -- start 2024-09-15T22:24:02.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.096+0000 7f3511ddc740 1 -- start start 2024-09-15T22:24:02.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fe09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556775001890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35081acdb0 con 0x556774fde850 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35081ae8f0 con 0x556774fe9a30 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f3511ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35081ad480 con 0x556774fdf170 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48552/0 (socket says 172.21.15.77:48552) 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 -- 172.21.15.77:0/733390341 learned_addr learned my addr 172.21.15.77:0/733390341 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350d79f640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fe09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350dfa0640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556775001890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 -- 172.21.15.77:0/733390341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 msgr2=0x556775001890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556775001890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 -- 172.21.15.77:0/733390341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 msgr2=0x556774fe09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fe09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 -- 172.21.15.77:0/733390341 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556774fda880 con 0x556774fdf170 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350dfa0640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556775001890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.097+0000 7f350cf9e640 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95dc16c1f1463f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.098+0000 7f34f2ffd640 1 -- 172.21.15.77:0/733390341 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f801e040 con 0x556774fdf170 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.098+0000 7f34f2ffd640 1 -- 172.21.15.77:0/733390341 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f802a030 con 0x556774fdf170 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.098+0000 7f34f2ffd640 1 -- 172.21.15.77:0/733390341 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80035f0 con 0x556774fdf170 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- 172.21.15.77:0/733390341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 msgr2=0x556775001350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8009760 comp rx=0 tx=0).stop 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- 172.21.15.77:0/733390341 shutdown_connections 2024-09-15T22:24:02.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556775001890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556775001350 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 --2- 172.21.15.77:0/733390341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774fe09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- 172.21.15.77:0/733390341 >> 172.21.15.77:0/733390341 conn(0x556774fdade0 msgr2=0x556774fe02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- 172.21.15.77:0/733390341 shutdown_connections 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- 172.21.15.77:0/733390341 wait complete. 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 -- start start 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.099+0000 7f3511ddc740 1 Processor -- start 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774ff1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35081ae220 con 0x556774fde850 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35081adb50 con 0x556774fe9a30 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f350dfa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f350dfa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:44264/0 (socket says 172.21.15.77:44264) 2024-09-15T22:24:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 -- 172.21.15.77:0/2133234365 learned_addr learned my addr 172.21.15.77:0/2133234365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.100+0000 7f3511ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35081aadb0 con 0x556774fdf170 2024-09-15T22:24:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350d79f640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774ff1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f3511ddc740 10 client.0 _ll_register_callbacks cb 0x7f35040046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:02.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 -- 172.21.15.77:0/2133234365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 msgr2=0x556774fed200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 -- 172.21.15.77:0/2133234365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 msgr2=0x556774ff1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774ff1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 -- 172.21.15.77:0/2133234365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556774fedc80 con 0x556774fe9a30 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350d79f640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774ff1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f350dfa0640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3500006010 tx=0x7f35000095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87da0d4559d47bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f34f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500034040 con 0x556774fe9a30 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f802a030 con 0x556774fe9a30 2024-09-15T22:24:02.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.101+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f3500004450 con 0x556774fe9a30 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3500061050 con 0x556774fe9a30 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 10 client.27380 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556775064030 con 0x556774fe9a30 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f802a030 con 0x556774fe9a30 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3500053470 con 0x556774fe9a30 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 10 client.27380 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 10 client.27380 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 20 client.27380 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.102+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556775064880 0x556775066d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.103+0000 7f3511ddc740 4 client.27380 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.103+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x556774f4abe0 con 0x556775064880 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.103+0000 7f350d79f640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556775064880 0x556775066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.103+0000 7f34f0ff9640 10 client.27380 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.104+0000 7f350d79f640 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556775064880 0x556775066d00 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e206643a9b26a27 server_cookie=274dea9d3b94d3ff in_seq=0 out_seq=0 2024-09-15T22:24:02.192 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f34f0ff9640 1 -- 172.21.15.77:0/2133234365 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556774f4abe0 con 0x556775064880 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f34f0ff9640 10 client.27380 handle_command_reply: tid=0 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 client.27380 shutdown 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 msgr2=0x556774fed740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3500006010 tx=0x7f35000095c0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 shutdown_connections 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556775064880 0x556775066d00 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556774fe9a30 0x556774fed740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774fdf170 0x556774fed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f34f0ff9640 0 client.27380 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 --2- 172.21.15.77:0/2133234365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556774fde850 0x556774ff1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 >> 172.21.15.77:0/2133234365 conn(0x556774fdade0 msgr2=0x556774fe70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 shutdown_connections 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 1 -- 172.21.15.77:0/2133234365 wait complete. 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.203+0000 7f3511ddc740 20 client.27380 trim_cache size 0 max 16384 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:24:02.206 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 42, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "2ab04924": { 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 42, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 141.8, 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:24:02.207 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:02.208 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 msgr2=0x7f34f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 0x7f34f40a58f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3500009ca0 tx=0x7f3500005c50 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 msgr2=0x7f35080fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9f0 tx=0x7f350400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 shutdown_connections 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f34f40a3430 0x7f34f40a58f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 --2- 172.21.15.77:0/2179733920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:02.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 >> 172.21.15.77:0/2179733920 conn(0x7f35080fc7d0 msgr2=0x7f350806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:02.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 shutdown_connections 2024-09-15T22:24:02.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:02.204+0000 7f350fa03640 1 -- 172.21.15.77:0/2179733920 wait complete. 2024-09-15T22:24:02.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:02 smithi077 ceph-mon[30980]: pgmap v361: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-15T22:24:02.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.39 MB/sec execute 491 sec latency 341.475 ms 2024-09-15T22:24:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:02 smithi196 ceph-mon[35109]: pgmap v361: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-15T22:24:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:02 smithi117 ceph-mon[35021]: pgmap v361: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- 172.21.15.77:0/721842762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8102ef0 msgr2=0x7f70b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/721842762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8102ef0 0x7f70b810da40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009ca0 tx=0x7f70b002a950 comp rx=0 tx=0).stop 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- 172.21.15.77:0/721842762 shutdown_connections 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/721842762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8102ef0 0x7f70b810da40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/721842762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/721842762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8100e20 0x7f70b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- 172.21.15.77:0/721842762 >> 172.21.15.77:0/721842762 conn(0x7f70b80fc7d0 msgr2=0x7f70b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- 172.21.15.77:0/721842762 shutdown_connections 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- 172.21.15.77:0/721842762 wait complete. 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 Processor -- start 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.127+0000 7f70bf9d1640 1 -- start start 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8102ef0 0x7f70b80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8110f80 con 0x7f70b8100e20 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8110e00 con 0x7f70b8102570 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bf9d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8111100 con 0x7f70b8102ef0 2024-09-15T22:24:03.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59094/0 (socket says 172.21.15.77:59094) 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 -- 172.21.15.77:0/904268554 learned_addr learned my addr 172.21.15.77:0/904268554 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bdf6e640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8102ef0 0x7f70b80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bcf6c640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 -- 172.21.15.77:0/904268554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8102ef0 msgr2=0x7f70b80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8102ef0 0x7f70b80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 -- 172.21.15.77:0/904268554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 msgr2=0x7f70b80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 -- 172.21.15.77:0/904268554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b81a69c0 con 0x7f70b8100e20 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bcf6c640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.128+0000 7f70bd76d640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d770 tx=0x7f70b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.129+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4017070 con 0x7f70b8100e20 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.129+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b4005500 con 0x7f70b8100e20 2024-09-15T22:24:03.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.129+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4004e00 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.129+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b81a6cb0 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.129+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81a71a0 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.130+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b8101200 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.131+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f70b4004fa0 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.132+0000 7f70ae7fc640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 0x7f70a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.132+0000 7f70bcf6c640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 0x7f70a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.132+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f70b40cfac0 con 0x7f70b8100e20 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.132+0000 7f70bcf6c640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 0x7f70a40a58f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a8005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.136+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70b409c110 con 0x7f70b8100e20 2024-09-15T22:24:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.347+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70b810a0d0 con 0x7f70b8100e20 2024-09-15T22:24:03.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.348+0000 7f70ae7fc640 1 -- 172.21.15.77:0/904268554 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f70b409ba10 con 0x7f70b8100e20 2024-09-15T22:24:03.350 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:24:03.350 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:24:03.350 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 msgr2=0x7f70a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 0x7f70a40a58f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a8005850 comp rx=0 tx=0).stop 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 msgr2=0x7f70b80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d770 tx=0x7f70b400dc30 comp rx=0 tx=0).stop 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 shutdown_connections 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f70a40a3430 0x7f70a40a58f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b8102ef0 0x7f70b80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b8102570 0x7f70b80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 --2- 172.21.15.77:0/904268554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b8100e20 0x7f70b80fcc80 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:03.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 >> 172.21.15.77:0/904268554 conn(0x7f70b80fc7d0 msgr2=0x7f70b806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:03.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 shutdown_connections 2024-09-15T22:24:03.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:03.350+0000 7f70bf9d1640 1 -- 172.21.15.77:0/904268554 wait complete. 2024-09-15T22:24:03.490 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.38 MB/sec execute 492 sec latency 1341.556 ms 2024-09-15T22:24:04.084 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2ab04924', quiesce took 5.6 seconds. Will release after: 16.25 2024-09-15T22:24:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:04 smithi077 ceph-mon[30980]: pgmap v362: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:04 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/904268554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:04.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.36 MB/sec execute 493 sec latency 2341.663 ms 2024-09-15T22:24:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:04 smithi196 ceph-mon[35109]: pgmap v362: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:04.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:04 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/904268554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:04 smithi117 ceph-mon[35021]: pgmap v362: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:04.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:04 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/904268554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:04.776 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:24:05.024 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:05.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.35 MB/sec execute 494 sec latency 3341.699 ms 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- 172.21.15.77:0/2134311724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 msgr2=0x7f0c6410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 --2- 172.21.15.77:0/2134311724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6410d950 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009cd0 tx=0x7f0c5c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- 172.21.15.77:0/2134311724 shutdown_connections 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 --2- 172.21.15.77:0/2134311724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6410d950 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 --2- 172.21.15.77:0/2134311724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 --2- 172.21.15.77:0/2134311724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 0x7f0c64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- 172.21.15.77:0/2134311724 >> 172.21.15.77:0/2134311724 conn(0x7f0c640fc7a0 msgr2=0x7f0c640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- 172.21.15.77:0/2134311724 shutdown_connections 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- 172.21.15.77:0/2134311724 wait complete. 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 Processor -- start 2024-09-15T22:24:05.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.562+0000 7f0c69381640 1 -- start start 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c6419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 0x7f0c6419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64110e80 con 0x7f0c64101420 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64110d00 con 0x7f0c64106a80 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64111000 con 0x7f0c64100a80 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59108/0 (socket says 172.21.15.77:59108) 2024-09-15T22:24:05.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 -- 172.21.15.77:0/786937195 learned_addr learned my addr 172.21.15.77:0/786937195 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c6419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c637fe640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 0x7f0c6419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 -- 172.21.15.77:0/786937195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 msgr2=0x7f0c6419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c6419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 -- 172.21.15.77:0/786937195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 msgr2=0x7f0c6419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 0x7f0c6419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 -- 172.21.15.77:0/786937195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6419f480 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c6419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c627fc640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00c920 tx=0x7f0c4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c5bfff640 1 -- 172.21.15.77:0/786937195 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c007c70 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c5bfff640 1 -- 172.21.15.77:0/786937195 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c007e10 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6419f770 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.563+0000 7f0c5bfff640 1 -- 172.21.15.77:0/786937195 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c018890 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.564+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c641a7050 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.566+0000 7f0c5bfff640 1 -- 172.21.15.77:0/786937195 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0c4c018a70 con 0x7f0c64101420 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.566+0000 7f0c6b75a740 1 Processor -- start 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.566+0000 7f0c6b75a740 1 -- start start 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c5bfff640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 0x7f0c500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a77370 0x55af50a79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af509d22f0 con 0x55af50a6db40 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af50979cf0 con 0x55af50a77370 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af509ce2b0 con 0x55af50a6c5d0 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 0x7f0c500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59124/0 (socket says 172.21.15.77:59124) 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 -- 172.21.15.77:0/1857096099 learned_addr learned my addr 172.21.15.77:0/1857096099 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c5bfff640 1 -- 172.21.15.77:0/786937195 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0c4c0d02b0 con 0x7f0c64101420 2024-09-15T22:24:05.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c627fc640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a77370 0x55af50a79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 -- 172.21.15.77:0/1857096099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 msgr2=0x55af50a6ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a6ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 -- 172.21.15.77:0/1857096099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a77370 msgr2=0x55af50a79760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a77370 0x55af50a79760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 -- 172.21.15.77:0/1857096099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5095e3a0 con 0x55af50a6db40 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c627fc640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 0x7f0c500a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400baa0 tx=0x7f0c54005d20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.567+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c031860 tx=0x7f0c5c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f637bb33b10674d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.568+0000 7f0c597fa640 1 -- 172.21.15.77:0/1857096099 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c031a40 con 0x55af50a6db40 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.568+0000 7f0c597fa640 1 -- 172.21.15.77:0/1857096099 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c036040 con 0x55af50a6db40 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.568+0000 7f0c597fa640 1 -- 172.21.15.77:0/1857096099 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c004700 con 0x55af50a6db40 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.568+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1857096099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 msgr2=0x55af50a6c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.568+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c031860 tx=0x7f0c5c009380 comp rx=0 tx=0).stop 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1857096099 shutdown_connections 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a77370 0x55af50a79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a6ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1857096099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6db40 0x55af50a6c090 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1857096099 >> 172.21.15.77:0/1857096099 conn(0x55af50a35310 msgr2=0x55af50a68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1857096099 shutdown_connections 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1857096099 wait complete. 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- start start 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 Processor -- start 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a74020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a77370 0x55af50a74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af50a7a010 con 0x55af50a6c5d0 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af5098d060 con 0x55af50a6db40 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af50a7a190 con 0x55af50a77370 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.569+0000 7f0c6b75a740 10 client.0 _ll_register_callbacks cb 0x55af506e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59140/0 (socket says 172.21.15.77:59140) 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 -- 172.21.15.77:0/974974143 learned_addr learned my addr 172.21.15.77:0/974974143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c627fc640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a74020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c637fe640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a77370 0x55af50a74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 -- 172.21.15.77:0/974974143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a77370 msgr2=0x55af50a74610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a77370 0x55af50a74610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 -- 172.21.15.77:0/974974143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 msgr2=0x55af50a74020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a74020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 -- 172.21.15.77:0/974974143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af507433c0 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c627fc640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a74020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400f9d0 tx=0x7f0c54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58012395058ae8f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54020070 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0c5400ed80 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c036040 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c5402b050 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c6b75a740 10 client.18381 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af50a85b10 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.570+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c036040 con 0x55af50a6c5d0 2024-09-15T22:24:05.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0c5401c450 con 0x55af50a6c5d0 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 10 client.18381 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 10 client.18381 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 20 client.18381 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50a893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 4 client.18381 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af509df860 con 0x55af50a86f20 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c627fc640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50a893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c4a7fc640 10 client.18381 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.571+0000 7f0c627fc640 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50a893a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2c50242fd1ebb86 server_cookie=b36d9abef3e13d73 in_seq=0 out_seq=0 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/974974143 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af509df860 con 0x55af50a86f20 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c4a7fc640 10 client.18381 handle_command_reply: tid=0 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 client.18381 shutdown 2024-09-15T22:24:05.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 msgr2=0x55af50a76d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400f9d0 tx=0x7f0c54004730 comp rx=0 tx=0).stop 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 shutdown_connections 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50a893a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a77370 0x55af50a74610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a74020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c4a7fc640 0 client.18381 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/974974143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a6c5d0 0x55af50a76d70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 >> 172.21.15.77:0/974974143 conn(0x55af50a35310 msgr2=0x55af50a6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 shutdown_connections 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 1 -- 172.21.15.77:0/974974143 wait complete. 2024-09-15T22:24:05.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.572+0000 7f0c6b75a740 20 client.18381 trim_cache size 0 max 16384 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 Processor -- start 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 -- start start 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6c5d0 0x55af50a67c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6db40 0x55af50a6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641b1140 con 0x55af50a77370 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641b2c80 con 0x55af50a6c5d0 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641b1810 con 0x55af50a6db40 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59156/0 (socket says 172.21.15.77:59156) 2024-09-15T22:24:05.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.585+0000 7f0c637fe640 1 -- 172.21.15.77:0/1450227468 learned_addr learned my addr 172.21.15.77:0/1450227468 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6c5d0 0x55af50a67c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 -- 172.21.15.77:0/1450227468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6db40 msgr2=0x55af50a6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6db40 0x55af50a6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 -- 172.21.15.77:0/1450227468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6c5d0 msgr2=0x55af50a67c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6c5d0 0x55af50a67c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 -- 172.21.15.77:0/1450227468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af50a69960 con 0x55af50a77370 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c637fe640 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009a60 tx=0x7f0c5c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673a37a3214e303f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c58ff9640 1 -- 172.21.15.77:0/1450227468 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c031a40 con 0x55af50a77370 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c58ff9640 1 -- 172.21.15.77:0/1450227468 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c036040 con 0x55af50a77370 2024-09-15T22:24:05.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.586+0000 7f0c58ff9640 1 -- 172.21.15.77:0/1450227468 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c004740 con 0x55af50a77370 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1450227468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 msgr2=0x55af50a6f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009a60 tx=0x7f0c5c0093f0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1450227468 shutdown_connections 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a6f660 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6db40 0x55af50a6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/1450227468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6c5d0 0x55af50a67c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1450227468 >> 172.21.15.77:0/1450227468 conn(0x55af50a35310 msgr2=0x55af50a67500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1450227468 shutdown_connections 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- 172.21.15.77:0/1450227468 wait complete. 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- start start 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 Processor -- start 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a7fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641b25b0 con 0x55af50a77370 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641b1ee0 con 0x55af50a6db40 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c641af140 con 0x55af50a6c5d0 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c6b75a740 10 client.0 _ll_register_callbacks cb 0x7f0c54013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59160/0 (socket says 172.21.15.77:59160) 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c637fe640 1 -- 172.21.15.77:0/2034550217 learned_addr learned my addr 172.21.15.77:0/2034550217 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.587+0000 7f0c627fc640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a7fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 -- 172.21.15.77:0/2034550217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 msgr2=0x55af50a7fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a7fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 -- 172.21.15.77:0/2034550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 msgr2=0x55af50a7ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a7ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 -- 172.21.15.77:0/2034550217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af50a7bb60 con 0x55af50a77370 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c627fc640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c637fe640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c02ae00 tx=0x7f0c5c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b759ab9de2c03eef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c058060 con 0x55af50a77370 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c036040 con 0x55af50a77370 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0c5c045050 con 0x55af50a77370 2024-09-15T22:24:05.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c05f050 con 0x55af50a77370 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c6b75a740 10 client.18387 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af50af1c40 con 0x55af50a77370 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c036040 con 0x55af50a77370 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.588+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0c5c030070 con 0x55af50a77370 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 10 client.18387 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 10 client.18387 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 20 client.18387 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50af4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 4 client.18387 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af509d9d70 con 0x55af50a86f20 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50af4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:05.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c4a7fc640 10 client.18387 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c62ffd640 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50af4140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7050c224cab2c926 server_cookie=84dbe058eb5e76e7 in_seq=0 out_seq=0 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c4a7fc640 1 -- 172.21.15.77:0/2034550217 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af509d9d70 con 0x55af50a86f20 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c4a7fc640 10 client.18387 handle_command_reply: tid=0 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 client.18387 shutdown 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 msgr2=0x55af50a7b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c02ae00 tx=0x7f0c5c004770 comp rx=0 tx=0).stop 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 shutdown_connections 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55af50a86f20 0x55af50af4140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af50a77370 0x55af50a7b380 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af50a6db40 0x55af50a7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c4a7fc640 0 client.18387 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 --2- 172.21.15.77:0/2034550217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af50a6c5d0 0x55af50a7fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 >> 172.21.15.77:0/2034550217 conn(0x55af50a35310 msgr2=0x55af50a67d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 shutdown_connections 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.589+0000 7f0c6b75a740 1 -- 172.21.15.77:0/2034550217 wait complete. 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c6b75a740 20 client.18387 trim_cache size 0 max 16384 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 msgr2=0x7f0c500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 0x7f0c500a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400baa0 tx=0x7f0c54005d20 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 msgr2=0x7f0c6419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00c920 tx=0x7f0c4c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 shutdown_connections 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0c500a33b0 0x7f0c500a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c64106a80 0x7f0c6419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c64101420 0x7f0c6419aa10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 --2- 172.21.15.77:0/786937195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c64100a80 0x7f0c6419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 >> 172.21.15.77:0/786937195 conn(0x7f0c640fc7a0 msgr2=0x7f0c6410c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 shutdown_connections 2024-09-15T22:24:05.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:05.590+0000 7f0c69381640 1 -- 172.21.15.77:0/786937195 wait complete. 2024-09-15T22:24:06.301 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:24:06.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.33 MB/sec execute 495 sec latency 4341.722 ms 2024-09-15T22:24:06.549 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:06 smithi077 ceph-mon[30980]: pgmap v363: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:06 smithi196 ceph-mon[35109]: pgmap v363: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:06 smithi117 ceph-mon[35021]: pgmap v363: 129 pgs: 129 active+clean; 540 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-09-15T22:24:07.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 -- 172.21.15.77:0/3849113354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8100a90 msgr2=0x7efff8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 --2- 172.21.15.77:0/3849113354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8100a90 0x7efff8100ef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7effe0009930 tx=0x7effe002a420 comp rx=0 tx=0).stop 2024-09-15T22:24:07.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 -- 172.21.15.77:0/3849113354 shutdown_connections 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 --2- 172.21.15.77:0/3849113354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8101430 0x7efff810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 --2- 172.21.15.77:0/3849113354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8100a90 0x7efff8100ef0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 --2- 172.21.15.77:0/3849113354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8106a90 0x7efff8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 -- 172.21.15.77:0/3849113354 >> 172.21.15.77:0/3849113354 conn(0x7efff80fc7d0 msgr2=0x7efff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 -- 172.21.15.77:0/3849113354 shutdown_connections 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.284+0000 7efffe491640 1 -- 172.21.15.77:0/3849113354 wait complete. 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 Processor -- start 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 -- start start 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8100a90 0x7efff8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8106a90 0x7efff819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efff8110e90 con 0x7efff8101430 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff8110d10 con 0x7efff8106a90 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffe491640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efff8111010 con 0x7efff8100a90 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7effeffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7effeffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59180/0 (socket says 172.21.15.77:59180) 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7effeffff640 1 -- 172.21.15.77:0/4221523364 learned_addr learned my addr 172.21.15.77:0/4221523364 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efff7fff640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8100a90 0x7efff8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.285+0000 7efffca2e640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8106a90 0x7efff819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 -- 172.21.15.77:0/4221523364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8100a90 msgr2=0x7efff8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8100a90 0x7efff8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 -- 172.21.15.77:0/4221523364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8106a90 msgr2=0x7efff819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8106a90 0x7efff819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 -- 172.21.15.77:0/4221523364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff819d400 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7effeffff640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7effe0009230 tx=0x7effe0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7efff5ffb640 1 -- 172.21.15.77:0/4221523364 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe003f030 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7efff5ffb640 1 -- 172.21.15.77:0/4221523364 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe0003ad0 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff819d690 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7efff5ffb640 1 -- 172.21.15.77:0/4221523364 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0035070 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.286+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a4fd0 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.288+0000 7efff5ffb640 1 -- 172.21.15.77:0/4221523364 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7effe0042030 con 0x7efff8101430 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 Processor -- start 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 -- start start 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fa96472f0 con 0x560fa96e2b40 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fa95eecf0 con 0x560fa96f0780 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7f000086a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fa96432b0 con 0x560fa96e15d0 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59184/0 (socket says 172.21.15.77:59184) 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 -- 172.21.15.77:0/2413454661 learned_addr learned my addr 172.21.15.77:0/2413454661 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efff7fff640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7effeffff640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efff5ffb640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 -- 172.21.15.77:0/2413454661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 msgr2=0x560fa96e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 -- 172.21.15.77:0/2413454661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 msgr2=0x560fa96f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 -- 172.21.15.77:0/2413454661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa95d33a0 con 0x560fa96e2b40 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7effeffff640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efff5ffb640 1 -- 172.21.15.77:0/4221523364 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7effe00f0450 con 0x7efff8101430 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.289+0000 7efffca2e640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7efffca2e640 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7efff0010510 tx=0x7efff0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3beae3c10472598c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7effee7fc640 1 -- 172.21.15.77:0/2413454661 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0022070 con 0x560fa96e2b40 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7effee7fc640 1 -- 172.21.15.77:0/2413454661 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff000e370 con 0x560fa96e2b40 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7effee7fc640 1 -- 172.21.15.77:0/2413454661 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0025600 con 0x560fa96e2b40 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7f000086a740 1 -- 172.21.15.77:0/2413454661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 msgr2=0x560fa96e1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7f000086a740 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7efff0010510 tx=0x7efff0009f90 comp rx=0 tx=0).stop 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.290+0000 7efffca2e640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 0x7effe40a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efff001f000 tx=0x7efff001b000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- 172.21.15.77:0/2413454661 shutdown_connections 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- 172.21.15.77:0/2413454661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e2b40 0x560fa96e1090 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- 172.21.15.77:0/2413454661 >> 172.21.15.77:0/2413454661 conn(0x560fa96aa310 msgr2=0x560fa96ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- 172.21.15.77:0/2413454661 shutdown_connections 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- 172.21.15.77:0/2413454661 wait complete. 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- start start 2024-09-15T22:24:07.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 Processor -- start 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e2b40 0x560fa96ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560fa96f3480 con 0x560fa96f0780 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fa9602060 con 0x560fa96e2b40 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fa96f3600 con 0x560fa96e15d0 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.291+0000 7f000086a740 10 client.0 _ll_register_callbacks cb 0x560fa93558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59200/0 (socket says 172.21.15.77:59200) 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 -- 172.21.15.77:0/942256646 learned_addr learned my addr 172.21.15.77:0/942256646 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7effeffff640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e2b40 0x560fa96ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efff7fff640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 -- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 msgr2=0x560fa96f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 -- 172.21.15.77:0/942256646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e2b40 msgr2=0x560fa96ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e2b40 0x560fa96ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 -- 172.21.15.77:0/942256646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa93b83c0 con 0x560fa96f0780 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7efffca2e640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7efff0010e60 tx=0x7efff000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711fbacb6fc3dd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7effd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0033060 con 0x560fa96f0780 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7efff000e720 con 0x560fa96f0780 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff000e370 con 0x560fa96f0780 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efff003a050 con 0x560fa96f0780 2024-09-15T22:24:07.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7f000086a740 10 client.18399 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.292+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fa96fef50 con 0x560fa96f0780 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff000e370 con 0x560fa96f0780 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efff000f040 con 0x560fa96f0780 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 10 client.18399 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 10 client.18399 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 20 client.18399 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa97003b0 0x560fa97027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 4 client.18399 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa9654860 con 0x560fa97003b0 2024-09-15T22:24:07.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.293+0000 7efff7fff640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa97003b0 0x560fa97027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.294+0000 7efff7fff640 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa97003b0 0x560fa97027a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=498097f80cf127ae server_cookie=bee173968b547fa3 in_seq=0 out_seq=0 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.294+0000 7effd77fe640 10 client.18399 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.294+0000 7effd77fe640 1 -- 172.21.15.77:0/942256646 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fa9654860 con 0x560fa97003b0 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.294+0000 7effd77fe640 10 client.18399 handle_command_reply: tid=0 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.294+0000 7f000086a740 1 client.18399 shutdown 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 msgr2=0x560fa96eda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7efff0010e60 tx=0x7efff000edf0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 shutdown_connections 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa97003b0 0x560fa97027a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96eda50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7effd77fe640 0 client.18399 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e2b40 0x560fa96ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 --2- 172.21.15.77:0/942256646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e15d0 0x560fa96f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 >> 172.21.15.77:0/942256646 conn(0x560fa96aa310 msgr2=0x560fa96e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 shutdown_connections 2024-09-15T22:24:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 1 -- 172.21.15.77:0/942256646 wait complete. 2024-09-15T22:24:07.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.295+0000 7f000086a740 20 client.18399 trim_cache size 0 max 16384 2024-09-15T22:24:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 Processor -- start 2024-09-15T22:24:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 -- start start 2024-09-15T22:24:07.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efff81acd20 con 0x560fa96e15d0 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff81ae860 con 0x560fa96f0780 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7f000086a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efff81ad3f0 con 0x560fa96e2b40 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7efff7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7efff7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59206/0 (socket says 172.21.15.77:59206) 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.308+0000 7efff7fff640 1 -- 172.21.15.77:0/3799173755 learned_addr learned my addr 172.21.15.77:0/3799173755 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efffca2e640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7effeffff640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 -- 172.21.15.77:0/3799173755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 msgr2=0x560fa96ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96ecd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 -- 172.21.15.77:0/3799173755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 msgr2=0x560fa96ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 -- 172.21.15.77:0/3799173755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa96e1d50 con 0x560fa96e15d0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efffca2e640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7efff7fff640 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x560fa9702ea0 tx=0x7effe800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d5b032c8f17d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7effedffb640 1 -- 172.21.15.77:0/3799173755 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8005020 con 0x560fa96e15d0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7effedffb640 1 -- 172.21.15.77:0/3799173755 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe80047a0 con 0x560fa96e15d0 2024-09-15T22:24:07.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.309+0000 7effedffb640 1 -- 172.21.15.77:0/3799173755 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80116a0 con 0x560fa96e15d0 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- 172.21.15.77:0/3799173755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 msgr2=0x560fa96fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x560fa9702ea0 tx=0x7effe800b390 comp rx=0 tx=0).stop 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- 172.21.15.77:0/3799173755 shutdown_connections 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96f0780 0x560fa96ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 --2- 172.21.15.77:0/3799173755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96e15d0 0x560fa96fc720 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- 172.21.15.77:0/3799173755 >> 172.21.15.77:0/3799173755 conn(0x560fa96aa310 msgr2=0x560fa96eeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- 172.21.15.77:0/3799173755 shutdown_connections 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- 172.21.15.77:0/3799173755 wait complete. 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 -- start start 2024-09-15T22:24:07.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.310+0000 7f000086a740 1 Processor -- start 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e15d0 0x560fa96f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efff81ae190 con 0x560fa96f0780 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff81adac0 con 0x560fa96e15d0 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efff81aad20 con 0x560fa96e2b40 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7f000086a740 10 client.0 _ll_register_callbacks cb 0x7efff0020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7effeffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59216/0 (socket says 172.21.15.77:59216) 2024-09-15T22:24:07.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 -- 172.21.15.77:0/1906126676 learned_addr learned my addr 172.21.15.77:0/1906126676 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efff7fff640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e15d0 0x560fa96f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 -- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 msgr2=0x560fa96f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 -- 172.21.15.77:0/1906126676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e15d0 msgr2=0x560fa96f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e15d0 0x560fa96f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 -- 172.21.15.77:0/1906126676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa96f4ad0 con 0x560fa96f0780 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7effeffff640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7efffca2e640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efff0010510 tx=0x7efff0013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67765ca1799f1d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.311+0000 7effd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0022070 con 0x560fa96f0780 2024-09-15T22:24:07.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7efff001dbc0 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe80047a0 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efff0047050 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 10 client.18405 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fa976ac50 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe80047a0 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efff0005440 con 0x560fa96f0780 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 10 client.18405 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 10 client.18405 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 20 client.18405 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa976b300 0x560fa976d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 4 client.18405 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fa964ed70 con 0x560fa976b300 2024-09-15T22:24:07.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.312+0000 7efff7fff640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa976b300 0x560fa976d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7effd77fe640 10 client.18405 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7efff7fff640 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa976b300 0x560fa976d720 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=296b3975d7672f4 server_cookie=dacf21273dda3cda in_seq=0 out_seq=0 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7effd77fe640 1 -- 172.21.15.77:0/1906126676 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fa964ed70 con 0x560fa976b300 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7effd77fe640 10 client.18405 handle_command_reply: tid=0 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 client.18405 shutdown 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 msgr2=0x560fa96f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efff0010510 tx=0x7efff0013a50 comp rx=0 tx=0).stop 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 shutdown_connections 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560fa976b300 0x560fa976d720 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560fa96f0780 0x560fa96f4590 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fa96e2b40 0x560fa96f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 --2- 172.21.15.77:0/1906126676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fa96e15d0 0x560fa96f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 >> 172.21.15.77:0/1906126676 conn(0x560fa96aa310 msgr2=0x560fa96eeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7effd77fe640 0 client.18405 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:07.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 shutdown_connections 2024-09-15T22:24:07.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 1 -- 172.21.15.77:0/1906126676 wait complete. 2024-09-15T22:24:07.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.313+0000 7f000086a740 20 client.18405 trim_cache size 0 max 16384 2024-09-15T22:24:07.327 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 msgr2=0x7effe40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 0x7effe40a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efff001f000 tx=0x7efff001b000 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 msgr2=0x7efff8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7effe0009230 tx=0x7effe0004270 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 shutdown_connections 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7effe40a33b0 0x7effe40a5870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8106a90 0x7efff819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efff8101430 0x7efff8198990 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 --2- 172.21.15.77:0/4221523364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8100a90 0x7efff8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:07.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 >> 172.21.15.77:0/4221523364 conn(0x7efff80fc7d0 msgr2=0x7efff8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:07.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 shutdown_connections 2024-09-15T22:24:07.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:07.314+0000 7efffe491640 1 -- 172.21.15.77:0/4221523364 wait complete. 2024-09-15T22:24:07.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.32 MB/sec execute 496 sec latency 5341.813 ms 2024-09-15T22:24:08.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:24:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:08.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.30 MB/sec execute 497 sec latency 6341.873 ms 2024-09-15T22:24:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:08 smithi077 ceph-mon[30980]: pgmap v364: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-15T22:24:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:08 smithi196 ceph-mon[35109]: pgmap v364: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-15T22:24:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:08 smithi117 ceph-mon[35021]: pgmap v364: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2024-09-15T22:24:09.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- 172.21.15.77:0/1709870758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f525410c180 msgr2=0x7f525410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 --2- 172.21.15.77:0/1709870758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f525410c180 0x7f525410e610 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ca0 tx=0x7f524c02a950 comp rx=0 tx=0).stop 2024-09-15T22:24:09.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- 172.21.15.77:0/1709870758 shutdown_connections 2024-09-15T22:24:09.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 --2- 172.21.15.77:0/1709870758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f525410c180 0x7f525410e610 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 --2- 172.21.15.77:0/1709870758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5254103700 0x7f525410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 --2- 172.21.15.77:0/1709870758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254102de0 0x7f52541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- 172.21.15.77:0/1709870758 >> 172.21.15.77:0/1709870758 conn(0x7f52540fc7a0 msgr2=0x7f52540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- 172.21.15.77:0/1709870758 shutdown_connections 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- 172.21.15.77:0/1709870758 wait complete. 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 Processor -- start 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.032+0000 7f525a37f640 1 -- start start 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254103700 0x7f5254198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525410c180 0x7f525419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5254111f70 con 0x7f5254102de0 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5254111df0 con 0x7f525410c180 2024-09-15T22:24:09.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525a37f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52541120f0 con 0x7f5254103700 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50592/0 (socket says 172.21.15.77:50592) 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 -- 172.21.15.77:0/3123305527 learned_addr learned my addr 172.21.15.77:0/3123305527 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f525891c640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525410c180 0x7f525419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f52537fe640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254103700 0x7f5254198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 -- 172.21.15.77:0/3123305527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254103700 msgr2=0x7f5254198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254103700 0x7f5254198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 -- 172.21.15.77:0/3123305527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525410c180 msgr2=0x7f525419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525410c180 0x7f525419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 -- 172.21.15.77:0/3123305527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525419d3a0 con 0x7f5254102de0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.033+0000 7f5253fff640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f524400c970 tx=0x7f524400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.034+0000 7f52517fa640 1 -- 172.21.15.77:0/3123305527 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244007bb0 con 0x7f5254102de0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.034+0000 7f52517fa640 1 -- 172.21.15.77:0/3123305527 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5244007d50 con 0x7f5254102de0 2024-09-15T22:24:09.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.034+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525419d690 con 0x7f5254102de0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.034+0000 7f52517fa640 1 -- 172.21.15.77:0/3123305527 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400f450 con 0x7f5254102de0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.034+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541a4f70 con 0x7f5254102de0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f52517fa640 1 -- 172.21.15.77:0/3123305527 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5244005020 con 0x7f5254102de0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 Processor -- start 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 -- start start 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559eedaa22f0 con 0x559eedb3db40 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eeda49cf0 con 0x559eedb3c5d0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.036+0000 7f525c758740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eeda9e2b0 con 0x559eedb4b780 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50598/0 (socket says 172.21.15.77:50598) 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 -- 172.21.15.77:0/2162944714 learned_addr learned my addr 172.21.15.77:0/2162944714 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f52517fa640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 0x7f52400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f5253fff640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 0x7f52400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f52537fe640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 -- 172.21.15.77:0/2162944714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 msgr2=0x559eedb4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 -- 172.21.15.77:0/2162944714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 msgr2=0x559eedb3ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb3ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 -- 172.21.15.77:0/2162944714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eeda2e3a0 con 0x559eedb3db40 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f52517fa640 1 -- 172.21.15.77:0/3123305527 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f52440cfea0 con 0x7f5254102de0 2024-09-15T22:24:09.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f52537fe640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ca0 tx=0x7f524c02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b276521d223b57a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f525891c640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 0x7f52400a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f524c048000 tx=0x7f524c04a000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f524a7fc640 1 -- 172.21.15.77:0/2162944714 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c043050 con 0x559eedb3db40 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f524a7fc640 1 -- 172.21.15.77:0/2162944714 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524c04d020 con 0x559eedb3db40 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.037+0000 7f524a7fc640 1 -- 172.21.15.77:0/2162944714 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c034040 con 0x559eedb3db40 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- 172.21.15.77:0/2162944714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 msgr2=0x559eedb3c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f524c009ca0 tx=0x7f524c02f420 comp rx=0 tx=0).stop 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- 172.21.15.77:0/2162944714 shutdown_connections 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 --2- 172.21.15.77:0/2162944714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb3c090 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- 172.21.15.77:0/2162944714 >> 172.21.15.77:0/2162944714 conn(0x559eedb05310 msgr2=0x559eedb38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- 172.21.15.77:0/2162944714 shutdown_connections 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- 172.21.15.77:0/2162944714 wait complete. 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.038+0000 7f525c758740 1 -- start start 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 Processor -- start 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb4b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559eedb4e420 con 0x559eedb3db40 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eeda5d060 con 0x559eedb3c5d0 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eedb4e5a0 con 0x559eedb4b780 2024-09-15T22:24:09.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525c758740 10 client.0 _ll_register_callbacks cb 0x559eed7b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:09.071 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50612/0 (socket says 172.21.15.77:50612) 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 -- 172.21.15.77:0/4152244437 learned_addr learned my addr 172.21.15.77:0/4152244437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f5253fff640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525891c640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 -- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 msgr2=0x559eedb48bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb48bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 -- 172.21.15.77:0/4152244437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 msgr2=0x559eedb4b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb4b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f52537fe640 1 -- 172.21.15.77:0/4152244437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eed8133c0 con 0x559eedb3db40 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.039+0000 7f525891c640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb48bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52537fe640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f523c010670 tx=0x7f523c010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148255799f6a384f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c00ead0 con 0x559eedb3db40 2024-09-15T22:24:09.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f523c00ec70 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524c04d020 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f523c025050 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 10 client.18417 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559eedb59ed0 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524c04d020 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f523c00f040 con 0x559eedb3db40 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 10 client.18417 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 10 client.18417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 20 client.18417 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedb5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 4 client.18417 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.040+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eedaaf860 con 0x559eedb5b430 2024-09-15T22:24:09.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.041+0000 7f5253fff640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedb5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.041+0000 7f5253fff640 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedb5d870 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=141ecd8b9416231d server_cookie=e477a063c1ab03bf in_seq=0 out_seq=0 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.041+0000 7f52337fe640 10 client.18417 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f52337fe640 1 -- 172.21.15.77:0/4152244437 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eedaaf860 con 0x559eedb5b430 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f52337fe640 10 client.18417 handle_command_reply: tid=0 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 client.18417 shutdown 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 msgr2=0x559eedb48460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f523c010670 tx=0x7f523c010b30 comp rx=0 tx=0).stop 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 shutdown_connections 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedb5d870 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb48bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb48460 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f52337fe640 0 client.18417 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 --2- 172.21.15.77:0/4152244437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb4b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 >> 172.21.15.77:0/4152244437 conn(0x559eedb05310 msgr2=0x559eedb3e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 shutdown_connections 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 1 -- 172.21.15.77:0/4152244437 wait complete. 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.042+0000 7f525c758740 20 client.18417 trim_cache size 0 max 16384 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 Processor -- start 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 -- start start 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3db40 0x559eedb47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb63140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52541ace00 con 0x559eedb3c5d0 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52541ae940 con 0x559eedb3db40 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525c758740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52541ad4d0 con 0x559eedb4b780 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f5253fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f5253fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50626/0 (socket says 172.21.15.77:50626) 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f5253fff640 1 -- 172.21.15.77:0/3403021745 learned_addr learned my addr 172.21.15.77:0/3403021745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.055+0000 7f525891c640 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb63140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f52537fe640 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3db40 0x559eedb47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 -- 172.21.15.77:0/3403021745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 msgr2=0x559eedb63140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb63140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 -- 172.21.15.77:0/3403021745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3db40 msgr2=0x559eedb47440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3db40 0x559eedb47440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 -- 172.21.15.77:0/3403021745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eedb38510 con 0x559eedb3c5d0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5253fff640 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f52440cd2c0 tx=0x7f524409bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c99e295638562b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5249ffb640 1 -- 172.21.15.77:0/3403021745 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524401d040 con 0x559eedb3c5d0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5249ffb640 1 -- 172.21.15.77:0/3403021745 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524409bdf0 con 0x559eedb3c5d0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f5249ffb640 1 -- 172.21.15.77:0/3403021745 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244098460 con 0x559eedb3c5d0 2024-09-15T22:24:09.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f525c758740 1 -- 172.21.15.77:0/3403021745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 msgr2=0x559eedb47d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.056+0000 7f525c758740 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f52440cd2c0 tx=0x7f524409bc70 comp rx=0 tx=0).stop 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- 172.21.15.77:0/3403021745 shutdown_connections 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb63140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3db40 0x559eedb47440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- 172.21.15.77:0/3403021745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3c5d0 0x559eedb47d20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- 172.21.15.77:0/3403021745 >> 172.21.15.77:0/3403021745 conn(0x559eedb05310 msgr2=0x559eedb46d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- 172.21.15.77:0/3403021745 shutdown_connections 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- 172.21.15.77:0/3403021745 wait complete. 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- start start 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 Processor -- start 2024-09-15T22:24:09.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb53f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52541ae270 con 0x559eedb3db40 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52541adba0 con 0x559eedb3c5d0 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52541aae00 con 0x559eedb4b780 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f525c758740 10 client.0 _ll_register_callbacks cb 0x7f523c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.057+0000 7f52537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50640/0 (socket says 172.21.15.77:50640) 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 -- 172.21.15.77:0/1574855787 learned_addr learned my addr 172.21.15.77:0/1574855787 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f5253fff640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb53f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f525891c640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 -- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 msgr2=0x559eedb4f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 -- 172.21.15.77:0/1574855787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 msgr2=0x559eedb53f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb53f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 -- 172.21.15.77:0/1574855787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eedb50010 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52537fe640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x559eedb49aa0 tx=0x7f523c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88a4d559d90ec4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52337fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c004230 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f523c0043d0 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524409bdf0 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f523c03a050 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f525c758740 10 client.18423 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559eedbc6100 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.058+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524409bdf0 con 0x559eedb3db40 2024-09-15T22:24:09.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f523c004680 con 0x559eedb3db40 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 10 client.18423 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 10 client.18423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 20 client.18423 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedbc88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 4 client.18423 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eedaa9d70 con 0x559eedb5b430 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f5253fff640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedbc88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f5253fff640 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedbc88f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84ca4c8de7a8dd27 server_cookie=2b61dbef9dd9aa5 in_seq=0 out_seq=0 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f52337fe640 10 client.18423 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f52337fe640 1 -- 172.21.15.77:0/1574855787 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eedaa9d70 con 0x559eedb5b430 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f52337fe640 10 client.18423 handle_command_reply: tid=0 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.059+0000 7f525c758740 1 client.18423 shutdown 2024-09-15T22:24:09.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 msgr2=0x559eedb4f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x559eedb49aa0 tx=0x7f523c00e800 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 shutdown_connections 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eedb5b430 0x559eedbc88f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eedb4b780 0x559eedb4f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f52337fe640 0 client.18423 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eedb3db40 0x559eedb4f2a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 --2- 172.21.15.77:0/1574855787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eedb3c5d0 0x559eedb53f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 >> 172.21.15.77:0/1574855787 conn(0x559eedb05310 msgr2=0x559eedb47550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 shutdown_connections 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 1 -- 172.21.15.77:0/1574855787 wait complete. 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525c758740 20 client.18423 trim_cache size 0 max 16384 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.060+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 msgr2=0x7f52400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 0x7f52400a5870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f524c048000 tx=0x7f524c04a000 comp rx=0 tx=0).stop 2024-09-15T22:24:09.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 msgr2=0x7f52541983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f524400c970 tx=0x7f524400ce30 comp rx=0 tx=0).stop 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 shutdown_connections 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f52400a33b0 0x7f52400a5870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f525410c180 0x7f525419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5254103700 0x7f5254198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 --2- 172.21.15.77:0/3123305527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5254102de0 0x7f52541983f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 >> 172.21.15.77:0/3123305527 conn(0x7f52540fc7a0 msgr2=0x7f52540fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 shutdown_connections 2024-09-15T22:24:09.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:09.061+0000 7f525a37f640 1 -- 172.21.15.77:0/3123305527 wait complete. 2024-09-15T22:24:09.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.29 MB/sec execute 498 sec latency 7341.899 ms 2024-09-15T22:24:09.799 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:24:10.045 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:10.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.28 MB/sec execute 499 sec latency 8342.006 ms 2024-09-15T22:24:10.540 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:10 smithi077 ceph-mon[30980]: pgmap v365: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-15T22:24:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:10 smithi196 ceph-mon[35109]: pgmap v365: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-15T22:24:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:10 smithi117 ceph-mon[35021]: pgmap v365: 129 pgs: 129 active+clean; 549 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2024-09-15T22:24:10.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 -- 172.21.15.77:0/4131849431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0100a80 msgr2=0x7f84a0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 --2- 172.21.15.77:0/4131849431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0100a80 0x7f84a0100ee0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8488009930 tx=0x7f848802a420 comp rx=0 tx=0).stop 2024-09-15T22:24:10.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 -- 172.21.15.77:0/4131849431 shutdown_connections 2024-09-15T22:24:10.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 --2- 172.21.15.77:0/4131849431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 0x7f84a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 --2- 172.21.15.77:0/4131849431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0100a80 0x7f84a0100ee0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 --2- 172.21.15.77:0/4131849431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0106a80 0x7f84a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 -- 172.21.15.77:0/4131849431 >> 172.21.15.77:0/4131849431 conn(0x7f84a00fc7a0 msgr2=0x7f84a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 -- 172.21.15.77:0/4131849431 shutdown_connections 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.619+0000 7f84a50b9640 1 -- 172.21.15.77:0/4131849431 wait complete. 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.620+0000 7f84a50b9640 1 Processor -- start 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.620+0000 7f84a50b9640 1 -- start start 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0100a80 0x7f84a0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 0x7f84a0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0110e80 con 0x7f84a0106a80 2024-09-15T22:24:10.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0110d00 con 0x7f84a0100a80 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111000 con 0x7f84a0101420 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50664/0 (socket says 172.21.15.77:50664) 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 -- 172.21.15.77:0/2754969109 learned_addr learned my addr 172.21.15.77:0/2754969109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0100a80 0x7f84a0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849e59c640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 0x7f84a0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 -- 172.21.15.77:0/2754969109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 msgr2=0x7f84a0197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 0x7f84a0197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 -- 172.21.15.77:0/2754969109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0100a80 msgr2=0x7f84a0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0100a80 0x7f84a0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 -- 172.21.15.77:0/2754969109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019c2f0 con 0x7f84a0106a80 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f849f59e640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f849800eaa0 tx=0x7f849800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f8497fff640 1 -- 172.21.15.77:0/2754969109 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849800cdb0 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f8497fff640 1 -- 172.21.15.77:0/2754969109 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8498005020 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019c5e0 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.621+0000 7f8497fff640 1 -- 172.21.15.77:0/2754969109 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980116b0 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.622+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a3ec0 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.624+0000 7f8497fff640 1 -- 172.21.15.77:0/2754969109 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8498005560 con 0x7f84a0106a80 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.624+0000 7f84a7492740 1 Processor -- start 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.624+0000 7f84a7492740 1 -- start start 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57de0 0x564b7db58220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b7dab32f0 con 0x564b7db57070 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b7da5acf0 con 0x564b7db58760 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f84a7492740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b7daaf2b0 con 0x564b7db57de0 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50674/0 (socket says 172.21.15.77:50674) 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 -- 172.21.15.77:0/470725114 learned_addr learned my addr 172.21.15.77:0/470725114 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f8497fff640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 0x7f848c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849ed9d640 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57de0 0x564b7db58220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849f59e640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 0x7f848c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 -- 172.21.15.77:0/470725114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57de0 msgr2=0x564b7db58220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57de0 0x564b7db58220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 -- 172.21.15.77:0/470725114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 msgr2=0x564b7db5ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db5ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 -- 172.21.15.77:0/470725114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b7da3f3a0 con 0x564b7db57070 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f8497fff640 1 -- 172.21.15.77:0/2754969109 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f84980d0300 con 0x7f84a0106a80 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.625+0000 7f849e59c640 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f848802a940 tx=0x7f8488005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cabd9bb3a512fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f849f59e640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 0x7f848c0a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84980cdcc0 tx=0x7f849806f080 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84957fa640 1 -- 172.21.15.77:0/470725114 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488035050 con 0x564b7db57070 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84957fa640 1 -- 172.21.15.77:0/470725114 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8488031730 con 0x564b7db57070 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84957fa640 1 -- 172.21.15.77:0/470725114 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880046c0 con 0x564b7db57070 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 -- 172.21.15.77:0/470725114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 msgr2=0x564b7db57450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f848802a940 tx=0x7f8488005b50 comp rx=0 tx=0).stop 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 -- 172.21.15.77:0/470725114 shutdown_connections 2024-09-15T22:24:10.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57de0 0x564b7db58220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 --2- 172.21.15.77:0/470725114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57070 0x564b7db57450 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 -- 172.21.15.77:0/470725114 >> 172.21.15.77:0/470725114 conn(0x564b7db16310 msgr2=0x564b7db49720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 -- 172.21.15.77:0/470725114 shutdown_connections 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.626+0000 7f84a7492740 1 -- 172.21.15.77:0/470725114 wait complete. 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 -- start start 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 Processor -- start 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db5c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b7db5f4a0 con 0x564b7db57de0 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b7da6e060 con 0x564b7db57070 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b7db5f620 con 0x564b7db58760 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f84a7492740 10 client.0 _ll_register_callbacks cb 0x564b7d7c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50690/0 (socket says 172.21.15.77:50690) 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 -- 172.21.15.77:0/1350371756 learned_addr learned my addr 172.21.15.77:0/1350371756 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849e59c640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 -- 172.21.15.77:0/1350371756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 msgr2=0x564b7db5c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db5c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 -- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 msgr2=0x564b7db5eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db5eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849ed9d640 1 -- 172.21.15.77:0/1350371756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b7d8243c0 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.627+0000 7f849e59c640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f849ed9d640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f849000bf80 tx=0x7f84900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df502b2313431851 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490004020 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f84900041c0 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8488031730 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8490025050 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 10 client.18435 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b7db6add0 con 0x564b7db57de0 2024-09-15T22:24:10.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8488031730 con 0x564b7db57de0 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8490003950 con 0x564b7db57de0 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 10 client.18435 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 10 client.18435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 20 client.18435 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7db6c330 0x564b7db6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 4 client.18435 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.628+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b7dac0860 con 0x564b7db6c330 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.629+0000 7f849e59c640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7db6c330 0x564b7db6e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.629+0000 7f849e59c640 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7db6c330 0x564b7db6e790 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6ea7da0be1530e4 server_cookie=ac03709a7ba059cb in_seq=0 out_seq=0 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.629+0000 7f84867fc640 10 client.18435 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84867fc640 1 -- 172.21.15.77:0/1350371756 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b7dac0860 con 0x564b7db6c330 2024-09-15T22:24:10.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84867fc640 10 client.18435 handle_command_reply: tid=0 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 client.18435 shutdown 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 msgr2=0x564b7db5c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f849000bf80 tx=0x7f84900118d0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 shutdown_connections 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7db6c330 0x564b7db6e790 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db5c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84867fc640 0 client.18435 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db5c050 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 --2- 172.21.15.77:0/1350371756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 >> 172.21.15.77:0/1350371756 conn(0x564b7db16310 msgr2=0x564b7db48d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 shutdown_connections 2024-09-15T22:24:10.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 1 -- 172.21.15.77:0/1350371756 wait complete. 2024-09-15T22:24:10.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.630+0000 7f84a7492740 20 client.18435 trim_cache size 0 max 16384 2024-09-15T22:24:10.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.643+0000 7f84a7492740 1 Processor -- start 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 -- start start 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 0x564b7db68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01acd10 con 0x564b7db57de0 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ae850 con 0x564b7db58760 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f84a7492740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ad3e0 con 0x564b7db57070 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50702/0 (socket says 172.21.15.77:50702) 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 -- 172.21.15.77:0/2093807068 learned_addr learned my addr 172.21.15.77:0/2093807068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849e59c640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 0x564b7db68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849e59c640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 -- 172.21.15.77:0/2093807068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 msgr2=0x564b7db68700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 0x564b7db68700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 -- 172.21.15.77:0/2093807068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 msgr2=0x564b7db541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849ed9d640 1 -- 172.21.15.77:0/2093807068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b7db6ee10 con 0x564b7db57de0 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.644+0000 7f849e59c640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 0x564b7db68700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8490003f60 tx=0x7f8490005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ab381fcd6dcfe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f8494ff9640 1 -- 172.21.15.77:0/2093807068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849002d070 con 0x564b7db57de0 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f8494ff9640 1 -- 172.21.15.77:0/2093807068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84900035f0 con 0x564b7db57de0 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f8494ff9640 1 -- 172.21.15.77:0/2093807068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84900052a0 con 0x564b7db57de0 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f84a7492740 1 -- 172.21.15.77:0/2093807068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 msgr2=0x564b7db53c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.645+0000 7f84a7492740 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8490003f60 tx=0x7f8490005850 comp rx=0 tx=0).stop 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- 172.21.15.77:0/2093807068 shutdown_connections 2024-09-15T22:24:10.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db58760 0x564b7db541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db53c70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- 172.21.15.77:0/2093807068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db57070 0x564b7db68700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- 172.21.15.77:0/2093807068 >> 172.21.15.77:0/2093807068 conn(0x564b7db16310 msgr2=0x564b7db56190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- 172.21.15.77:0/2093807068 shutdown_connections 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- 172.21.15.77:0/2093807068 wait complete. 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- start start 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 Processor -- start 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db60850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01ae180 con 0x564b7db57de0 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01adab0 con 0x564b7db57070 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01aad10 con 0x564b7db58760 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f84a7492740 10 client.0 _ll_register_callbacks cb 0x7f849000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50716/0 (socket says 172.21.15.77:50716) 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.646+0000 7f849ed9d640 1 -- 172.21.15.77:0/2069233388 learned_addr learned my addr 172.21.15.77:0/2069233388 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849e59c640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 -- 172.21.15.77:0/2069233388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 msgr2=0x564b7db60850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849e59c640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db60850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db60850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 -- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 msgr2=0x564b7db64fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:10.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db64fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 -- 172.21.15.77:0/2069233388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b7db610a0 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849e59c640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db60850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f849ed9d640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8490011ed0 tx=0x7f84900039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8802bd52b7e5ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84867fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490039020 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f8490005040 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84900035f0 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8490044050 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84a7492740 10 client.18441 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b7dbd6fd0 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.647+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84900035f0 con 0x564b7db57de0 2024-09-15T22:24:10.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f849001ac90 con 0x564b7db57de0 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 10 client.18441 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 10 client.18441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 20 client.18441 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7dbd7680 0x564b7dbd9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 4 client.18441 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b7dabad70 con 0x564b7dbd7680 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f849e59c640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7dbd7680 0x564b7dbd9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84867fc640 10 client.18441 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f849e59c640 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7dbd7680 0x564b7dbd9a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9554305d3f17b50 server_cookie=91dbde012c2b1072 in_seq=0 out_seq=0 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84867fc640 1 -- 172.21.15.77:0/2069233388 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b7dabad70 con 0x564b7dbd7680 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84867fc640 10 client.18441 handle_command_reply: tid=0 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.648+0000 7f84a7492740 1 client.18441 shutdown 2024-09-15T22:24:10.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 msgr2=0x564b7db60310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8490011ed0 tx=0x7f84900039d0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 shutdown_connections 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x564b7dbd7680 0x564b7dbd9a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b7db58760 0x564b7db60850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b7db57de0 0x564b7db60310 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 --2- 172.21.15.77:0/2069233388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b7db57070 0x564b7db64fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 >> 172.21.15.77:0/2069233388 conn(0x564b7db16310 msgr2=0x564b7db484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84867fc640 0 client.18441 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:10.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 shutdown_connections 2024-09-15T22:24:10.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 1 -- 172.21.15.77:0/2069233388 wait complete. 2024-09-15T22:24:10.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a7492740 20 client.18441 trim_cache size 0 max 16384 2024-09-15T22:24:10.665 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 msgr2=0x7f848c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 0x7f848c0a5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84980cdcc0 tx=0x7f849806f080 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 msgr2=0x7f84a019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f849800eaa0 tx=0x7f849800ef60 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 shutdown_connections 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f848c0a33b0 0x7f848c0a5870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84a0106a80 0x7f84a019bc10 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84a0101420 0x7f84a0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 --2- 172.21.15.77:0/2754969109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84a0100a80 0x7f84a0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.649+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 >> 172.21.15.77:0/2754969109 conn(0x7f84a00fc7a0 msgr2=0x7f84a00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:10.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.650+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 shutdown_connections 2024-09-15T22:24:10.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:10.650+0000 7f84a50b9640 1 -- 172.21.15.77:0/2754969109 wait complete. 2024-09-15T22:24:11.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:24:11.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.26 MB/sec execute 500 sec latency 9342.073 ms 2024-09-15T22:24:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 -- 172.21.15.77:0/3151398388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140101450 msgr2=0x7f6140105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 --2- 172.21.15.77:0/3151398388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140101450 0x7f6140105920 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f613002a470 comp rx=0 tx=0).stop 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 -- 172.21.15.77:0/3151398388 shutdown_connections 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 --2- 172.21.15.77:0/3151398388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140105f40 0x7f61401063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 --2- 172.21.15.77:0/3151398388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140101450 0x7f6140105920 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 --2- 172.21.15.77:0/3151398388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140100b30 0x7f6140100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 -- 172.21.15.77:0/3151398388 >> 172.21.15.77:0/3151398388 conn(0x7f61400fc7a0 msgr2=0x7f61400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 -- 172.21.15.77:0/3151398388 shutdown_connections 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.130+0000 7f614813c640 1 -- 172.21.15.77:0/3151398388 wait complete. 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 Processor -- start 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 -- start start 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 0x7f61401a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140101450 0x7f61401a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6140118560 con 0x7f6140105f40 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61401183e0 con 0x7f6140100b30 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f614813c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61401186e0 con 0x7f6140101450 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f61466d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.131+0000 7f61466d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50722/0 (socket says 172.21.15.77:50722) 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 -- 172.21.15.77:0/1709432217 learned_addr learned my addr 172.21.15.77:0/1709432217 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:12.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61456d7640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140101450 0x7f61401a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f6145ed8640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 0x7f61401a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 -- 172.21.15.77:0/1709432217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140101450 msgr2=0x7f61401a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140101450 0x7f61401a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 -- 172.21.15.77:0/1709432217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 msgr2=0x7f61401a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 0x7f61401a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 -- 172.21.15.77:0/1709432217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614019efe0 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f6145ed8640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 0x7f61401a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f61466d9640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f613800cce0 tx=0x7f613800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f6136ffd640 1 -- 172.21.15.77:0/1709432217 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613800f450 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f6136ffd640 1 -- 172.21.15.77:0/1709432217 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613800fad0 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614019f2d0 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f6136ffd640 1 -- 172.21.15.77:0/1709432217 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138018c10 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.132+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61401ab5c0 con 0x7f6140105f40 2024-09-15T22:24:12.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f6136ffd640 1 -- 172.21.15.77:0/1709432217 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f613800f5f0 con 0x7f6140105f40 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 Processor -- start 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 -- start start 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e1e31c2f0 con 0x561e1e3c1370 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e1e2c3cf0 con 0x561e1e3b7b40 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f614a515740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e1e3182b0 con 0x561e1e3b65d0 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50738/0 (socket says 172.21.15.77:50738) 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.135+0000 7f61456d7640 1 -- 172.21.15.77:0/1582179187 learned_addr learned my addr 172.21.15.77:0/1582179187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f6136ffd640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 0x7f612c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f6145ed8640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 -- 172.21.15.77:0/1582179187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 msgr2=0x561e1e3b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 -- 172.21.15.77:0/1582179187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 msgr2=0x561e1e3b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61466d9640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 0x7f612c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 -- 172.21.15.77:0/1582179187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1e2a83a0 con 0x561e1e3c1370 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f6145ed8640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f6136ffd640 1 -- 172.21.15.77:0/1709432217 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f61380d06e0 con 0x7f6140105f40 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61456d7640 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f6130000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141e516c546e6527 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f612bfff640 1 -- 172.21.15.77:0/1582179187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130035070 con 0x561e1e3c1370 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f61466d9640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 0x7f612c0a58f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f613809c2d0 tx=0x7f613806e350 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f612bfff640 1 -- 172.21.15.77:0/1582179187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130004030 con 0x561e1e3c1370 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.136+0000 7f612bfff640 1 -- 172.21.15.77:0/1582179187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130005620 con 0x561e1e3c1370 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- 172.21.15.77:0/1582179187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 msgr2=0x561e1e3c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f6130000bc0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- 172.21.15.77:0/1582179187 shutdown_connections 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3c3760 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 --2- 172.21.15.77:0/1582179187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- 172.21.15.77:0/1582179187 >> 172.21.15.77:0/1582179187 conn(0x561e1e37f310 msgr2=0x561e1e3b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- 172.21.15.77:0/1582179187 shutdown_connections 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- 172.21.15.77:0/1582179187 wait complete. 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 -- start start 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.137+0000 7f614a515740 1 Processor -- start 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e1e3c4010 con 0x561e1e3c1370 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561e1e2d7060 con 0x561e1e3b7b40 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e1e3c4190 con 0x561e1e3b65d0 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f614a515740 10 client.0 _ll_register_callbacks cb 0x561e1e02a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50752/0 (socket says 172.21.15.77:50752) 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 -- 172.21.15.77:0/159286330 learned_addr learned my addr 172.21.15.77:0/159286330 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f6145ed8640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 -- 172.21.15.77:0/159286330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 msgr2=0x561e1e3c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 -- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 msgr2=0x561e1e3bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.138+0000 7f61456d7640 1 -- 172.21.15.77:0/159286330 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1e08d3c0 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61456d7640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6130046040 tx=0x7f61300052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91afda4719bddd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130035040 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f6130031630 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6130004030 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f613005b050 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 10 client.18453 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e1e3cfa10 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6130004030 con 0x561e1e3c1370 2024-09-15T22:24:12.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6130003680 con 0x561e1e3c1370 2024-09-15T22:24:12.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 10 client.18453 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:12.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:12.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 20 client.18453 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:12.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e3d0d60 0x561e1e3d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 4 client.18453 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.139+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e1e329860 con 0x561e1e3d0d60 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.140+0000 7f6145ed8640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e3d0d60 0x561e1e3d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.140+0000 7f61297fa640 10 client.18453 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.140+0000 7f6145ed8640 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e3d0d60 0x561e1e3d3150 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff76bcc01aece953 server_cookie=ff52de55b5be1ee5 in_seq=0 out_seq=0 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f61297fa640 1 -- 172.21.15.77:0/159286330 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e1e329860 con 0x561e1e3d0d60 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f61297fa640 10 client.18453 handle_command_reply: tid=0 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 client.18453 shutdown 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 msgr2=0x561e1e3be5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6130046040 tx=0x7f61300052e0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 shutdown_connections 2024-09-15T22:24:12.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e3d0d60 0x561e1e3d3150 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3be5d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f61297fa640 0 client.18453 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 --2- 172.21.15.77:0/159286330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b65d0 0x561e1e3c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 >> 172.21.15.77:0/159286330 conn(0x561e1e37f310 msgr2=0x561e1e3b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 shutdown_connections 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 1 -- 172.21.15.77:0/159286330 wait complete. 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.141+0000 7f614a515740 20 client.18453 trim_cache size 0 max 16384 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 Processor -- start 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 -- start start 2024-09-15T22:24:12.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b65d0 0x561e1e3b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b7b40 0x561e1e3d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b5550 con 0x561e1e3c1370 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b7090 con 0x561e1e3b65d0 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f614a515740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b5c20 con 0x561e1e3b7b40 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50768/0 (socket says 172.21.15.77:50768) 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 -- 172.21.15.77:0/4033071355 learned_addr learned my addr 172.21.15.77:0/4033071355 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b65d0 0x561e1e3b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f6145ed8640 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b7b40 0x561e1e3d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 -- 172.21.15.77:0/4033071355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b7b40 msgr2=0x561e1e3d6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b7b40 0x561e1e3d6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 -- 172.21.15.77:0/4033071355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b65d0 msgr2=0x561e1e3b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b65d0 0x561e1e3b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 -- 172.21.15.77:0/4033071355 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1e3bf450 con 0x561e1e3c1370 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.154+0000 7f61456d7640 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f6130000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11a2fe33816572e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f612b7fe640 1 -- 172.21.15.77:0/4033071355 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130035070 con 0x561e1e3c1370 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f612b7fe640 1 -- 172.21.15.77:0/4033071355 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61300043f0 con 0x561e1e3c1370 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f612b7fe640 1 -- 172.21.15.77:0/4033071355 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300046c0 con 0x561e1e3c1370 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- 172.21.15.77:0/4033071355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 msgr2=0x561e1e3d66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f6130000bc0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- 172.21.15.77:0/4033071355 shutdown_connections 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3c1370 0x561e1e3d66b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3b7b40 0x561e1e3d6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 --2- 172.21.15.77:0/4033071355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b65d0 0x561e1e3b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- 172.21.15.77:0/4033071355 >> 172.21.15.77:0/4033071355 conn(0x561e1e37f310 msgr2=0x561e1e3b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- 172.21.15.77:0/4033071355 shutdown_connections 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- 172.21.15.77:0/4033071355 wait complete. 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.155+0000 7f614a515740 1 -- start start 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 Processor -- start 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3d4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3c1370 0x561e1e3d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b69c0 con 0x561e1e3b65d0 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b62f0 con 0x561e1e3b7b40 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61401b3550 con 0x561e1e3c1370 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f614a515740 10 client.0 _ll_register_callbacks cb 0x7f6130032e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:12.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50772/0 (socket says 172.21.15.77:50772) 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 -- 172.21.15.77:0/2928133187 learned_addr learned my addr 172.21.15.77:0/2928133187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 -- 172.21.15.77:0/2928133187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3c1370 msgr2=0x561e1e3d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3c1370 0x561e1e3d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 -- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 msgr2=0x561e1e3d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f6145ed8640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3d4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.156+0000 7f61456d7640 1 -- 172.21.15.77:0/2928133187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1e43a6a0 con 0x561e1e3b65d0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61456d7640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f613002a990 tx=0x7f6130005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9149d048c4edf0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130035070 con 0x561e1e3b65d0 2024-09-15T22:24:12.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f6130003cc0 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61300043f0 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6130004e30 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 10 client.18459 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e1e43b980 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61300043f0 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6130031910 con 0x561e1e3b65d0 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 10 client.18459 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 20 client.18459 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e43be00 0x561e1e43e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 4 client.18459 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:12.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.157+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e1e323d70 con 0x561e1e43be00 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f6145ed8640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e43be00 0x561e1e43e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f61297fa640 10 client.18459 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f6145ed8640 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e43be00 0x561e1e43e1f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3db4890738aa3f server_cookie=9ef8fd92ddbdf41a in_seq=0 out_seq=0 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f61297fa640 1 -- 172.21.15.77:0/2928133187 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e1e323d70 con 0x561e1e43be00 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f61297fa640 10 client.18459 handle_command_reply: tid=0 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 client.18459 shutdown 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 msgr2=0x561e1e3d3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f613002a990 tx=0x7f6130005710 comp rx=0 tx=0).stop 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 shutdown_connections 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561e1e43be00 0x561e1e43e1f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e1e3c1370 0x561e1e3d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561e1e3b7b40 0x561e1e3d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 --2- 172.21.15.77:0/2928133187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e1e3b65d0 0x561e1e3d3f00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f61297fa640 0 client.18459 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 >> 172.21.15.77:0/2928133187 conn(0x561e1e37f310 msgr2=0x561e1e3b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.158+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 shutdown_connections 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614a515740 1 -- 172.21.15.77:0/2928133187 wait complete. 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614a515740 20 client.18459 trim_cache size 0 max 16384 2024-09-15T22:24:12.179 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:12.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 msgr2=0x7f612c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 0x7f612c0a58f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f613809c2d0 tx=0x7f613806e350 comp rx=0 tx=0).stop 2024-09-15T22:24:12.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 msgr2=0x7f614019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f613800cce0 tx=0x7f613800a620 comp rx=0 tx=0).stop 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 shutdown_connections 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f612c0a3430 0x7f612c0a58f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6140105f40 0x7f614019e8b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6140101450 0x7f61401a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 --2- 172.21.15.77:0/1709432217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6140100b30 0x7f61401a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 >> 172.21.15.77:0/1709432217 conn(0x7f61400fc7a0 msgr2=0x7f61401038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 shutdown_connections 2024-09-15T22:24:12.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:12.159+0000 7f614813c640 1 -- 172.21.15.77:0/1709432217 wait complete. 2024-09-15T22:24:12.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.25 MB/sec execute 501 sec latency 10342.158 ms 2024-09-15T22:24:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:12 smithi077 ceph-mon[30980]: pgmap v366: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:24:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:12 smithi077 ceph-mon[30980]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:24:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:12 smithi196 ceph-mon[35109]: pgmap v366: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:24:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:12 smithi196 ceph-mon[35109]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:24:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:12 smithi117 ceph-mon[35021]: pgmap v366: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:24:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:12 smithi117 ceph-mon[35021]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:24:13.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.23 MB/sec execute 502 sec latency 11342.190 ms 2024-09-15T22:24:13.944 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:24:13.944 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:24:14.203 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:14.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.22 MB/sec execute 503 sec latency 12342.217 ms 2024-09-15T22:24:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:14 smithi077 ceph-mon[30980]: pgmap v368: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:14 smithi196 ceph-mon[35109]: pgmap v368: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:14 smithi117 ceph-mon[35021]: pgmap v368: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:14.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- 172.21.15.77:0/789177806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc100e20 msgr2=0x7f2ccc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/789177806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc100e20 0x7f2ccc101200 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009900 tx=0x7f2cc802a480 comp rx=0 tx=0).stop 2024-09-15T22:24:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- 172.21.15.77:0/789177806 shutdown_connections 2024-09-15T22:24:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/789177806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/789177806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/789177806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc100e20 0x7f2ccc101200 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- 172.21.15.77:0/789177806 >> 172.21.15.77:0/789177806 conn(0x7f2ccc0fc7d0 msgr2=0x7f2ccc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- 172.21.15.77:0/789177806 shutdown_connections 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- 172.21.15.77:0/789177806 wait complete. 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 Processor -- start 2024-09-15T22:24:14.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.690+0000 7f2cd35c8640 1 -- start start 2024-09-15T22:24:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 0x7f2ccc197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111020 con 0x7f2ccc102ef0 2024-09-15T22:24:14.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc110ea0 con 0x7f2ccc100e20 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1111a0 con 0x7f2ccc102570 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50780/0 (socket says 172.21.15.77:50780) 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 -- 172.21.15.77:0/1275117418 learned_addr learned my addr 172.21.15.77:0/1275117418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1364640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 0x7f2ccc197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 -- 172.21.15.77:0/1275117418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 msgr2=0x7f2ccc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 -- 172.21.15.77:0/1275117418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 msgr2=0x7f2ccc197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 0x7f2ccc197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 -- 172.21.15.77:0/1275117418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19c4b0 con 0x7f2ccc102ef0 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1364640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 0x7f2ccc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd1b65640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400b970 tx=0x7f2cc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cbe7fc640 1 -- 172.21.15.77:0/1275117418 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4019070 con 0x7f2ccc102ef0 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19c7a0 con 0x7f2ccc102ef0 2024-09-15T22:24:14.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cbe7fc640 1 -- 172.21.15.77:0/1275117418 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc400cb40 con 0x7f2ccc102ef0 2024-09-15T22:24:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cbe7fc640 1 -- 172.21.15.77:0/1275117418 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4007650 con 0x7f2ccc102ef0 2024-09-15T22:24:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.691+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a4080 con 0x7f2ccc102ef0 2024-09-15T22:24:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.694+0000 7f2cbe7fc640 1 -- 172.21.15.77:0/1275117418 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2cc401c070 con 0x7f2ccc102ef0 2024-09-15T22:24:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.694+0000 7f2cd59a1740 1 Processor -- start 2024-09-15T22:24:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.694+0000 7f2cd59a1740 1 -- start start 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b23b60 0x55a3e1b23f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b31130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1a89280 con 0x55a3e1b24480 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1a30b10 con 0x55a3e1b23b60 2024-09-15T22:24:14.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1a85240 con 0x55a3e1b2ed40 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cbe7fc640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 0x7f2cb80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50786/0 (socket says 172.21.15.77:50786) 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 -- 172.21.15.77:0/3621455149 learned_addr learned my addr 172.21.15.77:0/3621455149 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b31130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b23b60 0x55a3e1b23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1b65640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 0x7f2cb80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cbe7fc640 1 -- 172.21.15.77:0/1275117418 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f2cc40cf700 con 0x7f2ccc102ef0 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 -- 172.21.15.77:0/3621455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b2ed40 msgr2=0x55a3e1b31130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b31130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 -- 172.21.15.77:0/3621455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b23b60 msgr2=0x55a3e1b23f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b23b60 0x55a3e1b23f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 -- 172.21.15.77:0/3621455149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e1a3d1f0 con 0x55a3e1b24480 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1364640 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc80041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e87cb6e105c237a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.695+0000 7f2cd1b65640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 0x7f2cb80a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4097b00 tx=0x7f2cc406e4d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cb77fe640 1 -- 172.21.15.77:0/3621455149 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc803f070 con 0x55a3e1b24480 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cb77fe640 1 -- 172.21.15.77:0/3621455149 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8005520 con 0x55a3e1b24480 2024-09-15T22:24:14.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cb77fe640 1 -- 172.21.15.77:0/3621455149 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8035030 con 0x55a3e1b24480 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3621455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 msgr2=0x55a3e1b2e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc80041f0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3621455149 shutdown_connections 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b31130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b2e800 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3621455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b23b60 0x55a3e1b23f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3621455149 >> 172.21.15.77:0/3621455149 conn(0x55a3e1b200f0 msgr2=0x55a3e1b204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3621455149 shutdown_connections 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.696+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3621455149 wait complete. 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 -- start start 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 Processor -- start 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b24480 0x55a3e1b2c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1b31910 con 0x55a3e1b23b60 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1a43f10 con 0x55a3e1b2ed40 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3e1b31a90 con 0x55a3e1b24480 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd59a1740 10 client.0 _ll_register_callbacks cb 0x55a3e1a0ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50802/0 (socket says 172.21.15.77:50802) 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd0b63640 1 -- 172.21.15.77:0/1332577733 learned_addr learned my addr 172.21.15.77:0/1332577733 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.697+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd1364640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b24480 0x55a3e1b2c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 -- 172.21.15.77:0/1332577733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b24480 msgr2=0x55a3e1b2c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b24480 0x55a3e1b2c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 -- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b2ed40 msgr2=0x55a3e1b2c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b2c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 -- 172.21.15.77:0/1332577733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e17fa6a0 con 0x55a3e1b23b60 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0019f00 tx=0x7f2cc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfcd2004a8cbf1f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000f040 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f2cc00175e0 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc8005520 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2cc002a050 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd59a1740 10 client.18471 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a3e1b3d4c0 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.698+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8005520 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2cc0015040 con 0x55a3e1b23b60 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 10 client.18471 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:14.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 20 client.18471 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b3e8d0 0x55a3e1b40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 4 client.18471 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3e1a967f0 con 0x55a3e1b3e8d0 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd1364640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b3e8d0 0x55a3e1b40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cd1364640 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b3e8d0 0x55a3e1b40ce0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2dc85ce8c53811 server_cookie=3a246f69738befef in_seq=0 out_seq=0 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.699+0000 7f2cb4ff9640 10 client.18471 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/1332577733 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3e1a967f0 con 0x55a3e1b3e8d0 2024-09-15T22:24:14.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cb4ff9640 10 client.18471 handle_command_reply: tid=0 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 client.18471 shutdown 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 msgr2=0x55a3e1b2bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0019f00 tx=0x7f2cc0008040 comp rx=0 tx=0).stop 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 shutdown_connections 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b3e8d0 0x55a3e1b40ce0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b2ed40 0x55a3e1b2c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cb4ff9640 0 client.18471 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b24480 0x55a3e1b2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/1332577733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b23b60 0x55a3e1b2bb70 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.700+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 >> 172.21.15.77:0/1332577733 conn(0x55a3e1b200f0 msgr2=0x55a3e1b1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.701+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 shutdown_connections 2024-09-15T22:24:14.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.701+0000 7f2cd59a1740 1 -- 172.21.15.77:0/1332577733 wait complete. 2024-09-15T22:24:14.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.701+0000 7f2cd59a1740 20 client.18471 trim_cache size 0 max 16384 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 Processor -- start 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 -- start start 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1b2aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1b370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1acf70 con 0x55a3e1b24480 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aeab0 con 0x55a3e1b200f0 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ad640 con 0x55a3e1b23b60 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50814/0 (socket says 172.21.15.77:50814) 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 -- 172.21.15.77:0/3254222320 learned_addr learned my addr 172.21.15.77:0/3254222320 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1b2aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd1364640 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1b370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 -- 172.21.15.77:0/3254222320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 msgr2=0x55a3e1b370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1b370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 -- 172.21.15.77:0/3254222320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 msgr2=0x55a3e1b2aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1b2aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.718+0000 7f2cd0b63640 1 -- 172.21.15.77:0/3254222320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e1b1ff20 con 0x55a3e1b24480 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0016bd0 tx=0x7f2cc00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecf097e59b5e1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cb6ffd640 1 -- 172.21.15.77:0/3254222320 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000c040 con 0x55a3e1b24480 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cb6ffd640 1 -- 172.21.15.77:0/3254222320 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0022360 con 0x55a3e1b24480 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cb6ffd640 1 -- 172.21.15.77:0/3254222320 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0006030 con 0x55a3e1b24480 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3254222320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 msgr2=0x55a3e1b37600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0016bd0 tx=0x7f2cc00047d0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3254222320 shutdown_connections 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1b37600 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1b370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/3254222320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1b2aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3254222320 >> 172.21.15.77:0/3254222320 conn(0x55a3e1b27560 msgr2=0x55a3e1b25dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3254222320 shutdown_connections 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.719+0000 7f2cd59a1740 1 -- 172.21.15.77:0/3254222320 wait complete. 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 -- start start 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 Processor -- start 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1ba5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1ba5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ae3e0 con 0x55a3e1b24480 2024-09-15T22:24:14.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1add10 con 0x55a3e1b200f0 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1aaf70 con 0x55a3e1b23b60 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd59a1740 10 client.0 _ll_register_callbacks cb 0x7f2cc0019b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd0b63640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50816/0 (socket says 172.21.15.77:50816) 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd0b63640 1 -- 172.21.15.77:0/2198481543 learned_addr learned my addr 172.21.15.77:0/2198481543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1ba5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.720+0000 7f2cd1364640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1ba5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 -- 172.21.15.77:0/2198481543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 msgr2=0x55a3e1ba5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1ba5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 -- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 msgr2=0x55a3e1ba5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1ba5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 -- 172.21.15.77:0/2198481543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e1ba1d50 con 0x55a3e1b24480 2024-09-15T22:24:14.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd0b63640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001eb0 tx=0x7f2cc0022800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ece91c87c6f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000c040 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f2cc0022d60 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0022360 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2cc002f410 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd59a1740 10 client.18477 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a3e1bae250 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.721+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0022360 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2cc0015070 con 0x55a3e1b24480 2024-09-15T22:24:14.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 10 client.18477 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 20 client.18477 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b2ed40 0x55a3e1bb0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 4 client.18477 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a3e1a90d00 con 0x55a3e1b2ed40 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd1364640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b2ed40 0x55a3e1bb0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cd1364640 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b2ed40 0x55a3e1bb0840 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741a9ec4aef231a3 server_cookie=44ce416eef1c0443 in_seq=0 out_seq=0 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.722+0000 7f2cb4ff9640 10 client.18477 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.723+0000 7f2cb4ff9640 1 -- 172.21.15.77:0/2198481543 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a3e1a90d00 con 0x55a3e1b2ed40 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.723+0000 7f2cb4ff9640 10 client.18477 handle_command_reply: tid=0 2024-09-15T22:24:14.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.723+0000 7f2cd59a1740 1 client.18477 shutdown 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.723+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 msgr2=0x55a3e1ba1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.723+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001eb0 tx=0x7f2cc0022800 comp rx=0 tx=0).stop 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 shutdown_connections 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3e1b2ed40 0x55a3e1bb0840 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3e1b24480 0x55a3e1ba1540 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3e1b23b60 0x55a3e1ba5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 --2- 172.21.15.77:0/2198481543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3e1b200f0 0x55a3e1ba5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 >> 172.21.15.77:0/2198481543 conn(0x55a3e1b27560 msgr2=0x55a3e1b1ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cb4ff9640 0 client.18477 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 shutdown_connections 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 1 -- 172.21.15.77:0/2198481543 wait complete. 2024-09-15T22:24:14.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd59a1740 20 client.18477 trim_cache size 0 max 16384 2024-09-15T22:24:14.743 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:14.743 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:24:14.743 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "e692846a-d0ae-4bdf-9ddf-e93f1f4317b1", 2024-09-15T22:24:14.743 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:24:14.743 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 msgr2=0x7f2cb80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 0x7f2cb80a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4097b00 tx=0x7f2cc406e4d0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.724+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 msgr2=0x7f2ccc19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400b970 tx=0x7f2cc400be30 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 shutdown_connections 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2cb80a33b0 0x7f2cb80a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ccc102ef0 0x7f2ccc19bdd0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc102570 0x7f2ccc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 --2- 172.21.15.77:0/1275117418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ccc100e20 0x7f2ccc197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 >> 172.21.15.77:0/1275117418 conn(0x7f2ccc0fc7d0 msgr2=0x7f2ccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:14.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 shutdown_connections 2024-09-15T22:24:14.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:14.725+0000 7f2cd35c8640 1 -- 172.21.15.77:0/1275117418 wait complete. 2024-09-15T22:24:15.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.20 MB/sec execute 504 sec latency 13342.248 ms 2024-09-15T22:24:15.607 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:24:15.854 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 -- 172.21.15.77:0/1798326959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec8102d30 msgr2=0x7f8ec8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 --2- 172.21.15.77:0/1798326959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec8102d30 0x7f8ec8103110 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009cd0 tx=0x7f8eb802a910 comp rx=0 tx=0).stop 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 -- 172.21.15.77:0/1798326959 shutdown_connections 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 --2- 172.21.15.77:0/1798326959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 --2- 172.21.15.77:0/1798326959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec81036e0 0x7f8ec8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 --2- 172.21.15.77:0/1798326959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec8102d30 0x7f8ec8103110 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 -- 172.21.15.77:0/1798326959 >> 172.21.15.77:0/1798326959 conn(0x7f8ec80fc7d0 msgr2=0x7f8ec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 -- 172.21.15.77:0/1798326959 shutdown_connections 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.343+0000 7f8ecd433640 1 -- 172.21.15.77:0/1798326959 wait complete. 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.344+0000 7f8ecd433640 1 Processor -- start 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.344+0000 7f8ecd433640 1 -- start start 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec8102d30 0x7f8ec8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8112100 con 0x7f8ec81036e0 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8111f80 con 0x7f8ec8102d30 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8112280 con 0x7f8ec81082a0 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50844/0 (socket says 172.21.15.77:50844) 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 -- 172.21.15.77:0/2003880303 learned_addr learned my addr 172.21.15.77:0/2003880303 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:16.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec8102d30 0x7f8ec8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 -- 172.21.15.77:0/2003880303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 msgr2=0x7f8ec819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 -- 172.21.15.77:0/2003880303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec8102d30 msgr2=0x7f8ec8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec8102d30 0x7f8ec8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 -- 172.21.15.77:0/2003880303 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec819d5d0 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc006fd0 tx=0x7f8ebc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8eaffff640 1 -- 172.21.15.77:0/2003880303 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011830 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8eaffff640 1 -- 172.21.15.77:0/2003880303 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc016070 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec819d8c0 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.345+0000 7f8eaffff640 1 -- 172.21.15.77:0/2003880303 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00f2e0 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.346+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81a51a0 con 0x7f8ec81036e0 2024-09-15T22:24:16.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8eaffff640 1 -- 172.21.15.77:0/2003880303 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8ebc01b460 con 0x7f8ec81036e0 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 Processor -- start 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 -- start start 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5b9c2e0 con 0x55d5b5c364a0 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5b43ce0 con 0x55d5b5c41680 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5b982a0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8eaffff640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 0x7f8eb00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50846/0 (socket says 172.21.15.77:50846) 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.348+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2036675381 learned_addr learned my addr 172.21.15.77:0/2036675381 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:16.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 0x7f8eb00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8eaffff640 1 -- 172.21.15.77:0/2003880303 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f8ebc0cf700 con 0x7f8ec81036e0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2036675381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 msgr2=0x55d5b5c41140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c41140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2036675381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 msgr2=0x55d5b5c43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c43a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2036675381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5b5b28440 con 0x55d5b5c364a0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c41140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00137c0 tx=0x7f8ec0013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d75a9d899ef3eba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ead7fa640 1 -- 172.21.15.77:0/2036675381 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec001d7c0 con 0x55d5b5c364a0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ead7fa640 1 -- 172.21.15.77:0/2036675381 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0022070 con 0x55d5b5c364a0 2024-09-15T22:24:16.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ead7fa640 1 -- 172.21.15.77:0/2036675381 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec001c980 con 0x55d5b5c364a0 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.349+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 0x7f8eb00a58f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0021000 tx=0x7f8ec001f000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2036675381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 msgr2=0x55d5b5c36880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00137c0 tx=0x7f8ec0013c80 comp rx=0 tx=0).stop 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2036675381 shutdown_connections 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c41140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2036675381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c364a0 0x55d5b5c36880 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2036675381 >> 172.21.15.77:0/2036675381 conn(0x55d5b5c32a30 msgr2=0x55d5b5c32e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2036675381 shutdown_connections 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2036675381 wait complete. 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 -- start start 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.350+0000 7f8ecf80c740 1 Processor -- start 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c41680 0x55d5b5c3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5c44250 con 0x55d5b5c36dc0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5b57050 con 0x55d5b5c364a0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5b5c443d0 con 0x55d5b5c41680 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 10 client.0 _ll_register_callbacks cb 0x55d5b5b97160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50854/0 (socket says 172.21.15.77:50854) 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 -- 172.21.15.77:0/4104150135 learned_addr learned my addr 172.21.15.77:0/4104150135 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c41680 0x55d5b5c3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 -- 172.21.15.77:0/4104150135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c41680 msgr2=0x55d5b5c3e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c41680 0x55d5b5c3e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 -- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 msgr2=0x55d5b5c40f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 -- 172.21.15.77:0/4104150135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5b590d3c0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00bde0 tx=0x7f8ebc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd9d672e5b7313d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8eaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc09b6a0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f8ebc0ccc80 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.351+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec0022070 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ebc0cce80 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 10 client.18489 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5b5c4fbc0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec0022070 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8ebc0cf1d0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 10 client.18489 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:16.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 20 client.18489 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5c51020 0x55d5b5c53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 4 client.18489 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5b5ba9850 con 0x55d5b5c51020 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.352+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5c51020 0x55d5b5c53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.353+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5c51020 0x55d5b5c53430 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4318f16128bbb476 server_cookie=4291d1a75c705a27 in_seq=0 out_seq=0 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.353+0000 7f8eaaff5640 10 client.18489 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.353+0000 7f8eaaff5640 1 -- 172.21.15.77:0/4104150135 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5b5ba9850 con 0x55d5b5c51020 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.353+0000 7f8eaaff5640 10 client.18489 handle_command_reply: tid=0 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.353+0000 7f8ecf80c740 1 client.18489 shutdown 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 msgr2=0x55d5b5c3e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00bde0 tx=0x7f8ebc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 shutdown_connections 2024-09-15T22:24:16.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5c51020 0x55d5b5c53430 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c41680 0x55d5b5c3e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c3e220 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8eaaff5640 0 client.18489 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/4104150135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 >> 172.21.15.77:0/4104150135 conn(0x55d5b5c32a30 msgr2=0x55d5b5c31b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 shutdown_connections 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 1 -- 172.21.15.77:0/4104150135 wait complete. 2024-09-15T22:24:16.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.354+0000 7f8ecf80c740 20 client.18489 trim_cache size 0 max 16384 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 Processor -- start 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 -- start start 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81af1b0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b0cf0 con 0x55d5b5c41680 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81af880 con 0x55d5b5c364a0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50856/0 (socket says 172.21.15.77:50856) 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 -- 172.21.15.77:0/104413963 learned_addr learned my addr 172.21.15.77:0/104413963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 -- 172.21.15.77:0/104413963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c364a0 msgr2=0x55d5b5c40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 -- 172.21.15.77:0/104413963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 msgr2=0x55d5b5c59210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 -- 172.21.15.77:0/104413963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5b5c36bf0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.371+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc097ab0 tx=0x7f8ebc099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3699baac74884b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8eacff9640 1 -- 172.21.15.77:0/104413963 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc09b470 con 0x55d5b5c36dc0 2024-09-15T22:24:16.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8eacff9640 1 -- 172.21.15.77:0/104413963 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc09fa50 con 0x55d5b5c36dc0 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8eacff9640 1 -- 172.21.15.77:0/104413963 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0cf1d0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- 172.21.15.77:0/104413963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 msgr2=0x55d5b5c58cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc097ab0 tx=0x7f8ebc099bb0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- 172.21.15.77:0/104413963 shutdown_connections 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c41680 0x55d5b5c59210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c58cd0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/104413963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c364a0 0x55d5b5c40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- 172.21.15.77:0/104413963 >> 172.21.15.77:0/104413963 conn(0x55d5b5c32a30 msgr2=0x55d5b5c3fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- 172.21.15.77:0/104413963 shutdown_connections 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- 172.21.15.77:0/104413963 wait complete. 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.372+0000 7f8ecf80c740 1 -- start start 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 Processor -- start 2024-09-15T22:24:16.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c49b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81b0620 con 0x55d5b5c41680 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81aff50 con 0x55d5b5c364a0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81ad1b0 con 0x55d5b5c36dc0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ecf80c740 10 client.0 _ll_register_callbacks cb 0x7f8ebc097280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50870/0 (socket says 172.21.15.77:50870) 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2042107201 learned_addr learned my addr 172.21.15.77:0/2042107201 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c49b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec67fc640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2042107201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 msgr2=0x55d5b5c46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 msgr2=0x55d5b5c49b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c49b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.373+0000 7f8ec77fe640 1 -- 172.21.15.77:0/2042107201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5b5c479b0 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ec77fe640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005fa0 tx=0x7f8ec0011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c2a8f4bcbba3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000b640 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f8ec000bd60 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc09fa50 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ec003c050 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 10 client.18495 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5b5cbba50 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc09fa50 con 0x55d5b5c41680 2024-09-15T22:24:16.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8ec0037030 con 0x55d5b5c41680 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 10 client.18495 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 20 client.18495 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5cbbf40 0x55d5b5cbe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 4 client.18495 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.374+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5b5ba3d60 con 0x55d5b5cbbf40 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5cbbf40 0x55d5b5cbe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ec6ffd640 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5cbbf40 0x55d5b5cbe330 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340defff2dd79567 server_cookie=589427dbeb3a7b0d in_seq=0 out_seq=0 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8eaaff5640 10 client.18495 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8eaaff5640 1 -- 172.21.15.77:0/2042107201 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d5b5ba3d60 con 0x55d5b5cbbf40 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8eaaff5640 10 client.18495 handle_command_reply: tid=0 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 client.18495 shutdown 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 msgr2=0x55d5b5c473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005fa0 tx=0x7f8ec0011eb0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 shutdown_connections 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5b5cbbf40 0x55d5b5cbe330 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5b5c41680 0x55d5b5c473f0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8eaaff5640 0 client.18495 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d5b5c36dc0 0x55d5b5c46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 --2- 172.21.15.77:0/2042107201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5b5c364a0 0x55d5b5c49b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 >> 172.21.15.77:0/2042107201 conn(0x55d5b5c32a30 msgr2=0x55d5b5c402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 shutdown_connections 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 1 -- 172.21.15.77:0/2042107201 wait complete. 2024-09-15T22:24:16.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.375+0000 7f8ecf80c740 20 client.18495 trim_cache size 0 max 16384 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "e692846a-d0ae-4bdf-9ddf-e93f1f4317b1": { 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "e692846a-d0ae-4bdf-9ddf-e93f1f4317b1", 2024-09-15T22:24:16.385 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:24:16.386 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:16.386 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:16.386 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:16.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 msgr2=0x7f8eb00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 0x7f8eb00a58f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0021000 tx=0x7f8ec001f000 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 msgr2=0x7f8ec8198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc006fd0 tx=0x7f8ebc00b8e0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 shutdown_connections 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8eb00a3430 0x7f8eb00a58f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ec81082a0 0x7f8ec819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ec81036e0 0x7f8ec8198b60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 --2- 172.21.15.77:0/2003880303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec8102d30 0x7f8ec8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 >> 172.21.15.77:0/2003880303 conn(0x7f8ec80fc7d0 msgr2=0x7f8ec8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 shutdown_connections 2024-09-15T22:24:16.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:16.376+0000 7f8ecd433640 1 -- 172.21.15.77:0/2003880303 wait complete. 2024-09-15T22:24:16.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.19 MB/sec execute 505 sec latency 14342.307 ms 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: pgmap v369: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: pgmap v369: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: pgmap v369: 129 pgs: 129 active+clean; 544 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-09-15T22:24:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:24:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:24:16.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:24:16.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:16.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:17.157 INFO:tasks.cephfs.filesystem:scrub status for tag:e692846a-d0ae-4bdf-9ddf-e93f1f4317b1 - {'path': '/', 'tag': 'e692846a-d0ae-4bdf-9ddf-e93f1f4317b1', 'options': 'recursive,force'} 2024-09-15T22:24:17.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.17 MB/sec execute 506 sec latency 15342.363 ms 2024-09-15T22:24:18.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.16 MB/sec execute 507 sec latency 16342.408 ms 2024-09-15T22:24:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:18 smithi077 ceph-mon[30980]: pgmap v370: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:18.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:18 smithi196 ceph-mon[35109]: pgmap v370: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:18 smithi117 ceph-mon[35021]: pgmap v370: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:19.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.15 MB/sec execute 508 sec latency 17342.505 ms 2024-09-15T22:24:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:19 smithi077 ceph-mon[30980]: pgmap v371: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:19 smithi077 ceph-mon[30980]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:24:20.335 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:19 smithi196 ceph-mon[35109]: pgmap v371: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:20.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:19 smithi196 ceph-mon[35109]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:24:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:19 smithi117 ceph-mon[35021]: pgmap v371: 129 pgs: 129 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 471 KiB/s wr, 38 op/s 2024-09-15T22:24:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:19 smithi117 ceph-mon[35021]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:24:20.491 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.13 MB/sec execute 509 sec latency 18342.548 ms 2024-09-15T22:24:20.593 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:20.981 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:24:20.982 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 -- 172.21.15.77:0/514398804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102570 msgr2=0x7f6a441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 --2- 172.21.15.77:0/514398804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102570 0x7f6a441029b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009900 tx=0x7f6a2c02a480 comp rx=0 tx=0).stop 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 -- 172.21.15.77:0/514398804 shutdown_connections 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 --2- 172.21.15.77:0/514398804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102ef0 0x7f6a4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 --2- 172.21.15.77:0/514398804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102570 0x7f6a441029b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 --2- 172.21.15.77:0/514398804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.058+0000 7f6a4912b640 1 -- 172.21.15.77:0/514398804 >> 172.21.15.77:0/514398804 conn(0x7f6a440fc7d0 msgr2=0x7f6a440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.059+0000 7f6a4912b640 1 -- 172.21.15.77:0/514398804 shutdown_connections 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.059+0000 7f6a4912b640 1 -- 172.21.15.77:0/514398804 wait complete. 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.059+0000 7f6a4912b640 1 Processor -- start 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.059+0000 7f6a4912b640 1 -- start start 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a441984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102570 0x7f6a441989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44110f80 con 0x7f6a44102ef0 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44110e00 con 0x7f6a44100e20 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4912b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111100 con 0x7f6a44102570 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:21.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55392/0 (socket says 172.21.15.77:55392) 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 -- 172.21.15.77:0/3172062113 learned_addr learned my addr 172.21.15.77:0/3172062113 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4259c640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102570 0x7f6a441989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a42d9d640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a441984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 -- 172.21.15.77:0/3172062113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102570 msgr2=0x7f6a441989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102570 0x7f6a441989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 -- 172.21.15.77:0/3172062113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 msgr2=0x7f6a441984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a441984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 -- 172.21.15.77:0/3172062113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4419d460 con 0x7f6a44102ef0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a42d9d640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a441984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a4359e640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400d750 tx=0x7f6a3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.060+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34017070 con 0x7f6a44102ef0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.061+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a34005500 con 0x7f6a44102ef0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.061+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4419d750 con 0x7f6a44102ef0 2024-09-15T22:24:21.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.061+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004d90 con 0x7f6a44102ef0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.061+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441a5030 con 0x7f6a44102ef0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.062+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44101200 con 0x7f6a44102ef0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.063+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6a34004f30 con 0x7f6a44102ef0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.064+0000 7f6a3bfff640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 0x7f6a300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.064+0000 7f6a42d9d640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 0x7f6a300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.064+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f6a340cfa00 con 0x7f6a44102ef0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.064+0000 7f6a42d9d640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 0x7f6a300a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c005fd0 tx=0x7f6a3c009600 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:21.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.068+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a3409c050 con 0x7f6a44102ef0 2024-09-15T22:24:21.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.269+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a4410a0a0 con 0x7f6a44102ef0 2024-09-15T22:24:21.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.270+0000 7f6a3bfff640 1 -- 172.21.15.77:0/3172062113 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f6a3409b950 con 0x7f6a44102ef0 2024-09-15T22:24:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:24:21.272 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:24:21.272 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:24:21.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 msgr2=0x7f6a300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:21.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 0x7f6a300a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c005fd0 tx=0x7f6a3c009600 comp rx=0 tx=0).stop 2024-09-15T22:24:21.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 msgr2=0x7f6a4419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:21.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400d750 tx=0x7f6a3400dc10 comp rx=0 tx=0).stop 2024-09-15T22:24:21.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 shutdown_connections 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6a300a33b0 0x7f6a300a5870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a44102ef0 0x7f6a4419cd80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a44102570 0x7f6a441989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 --2- 172.21.15.77:0/3172062113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a44100e20 0x7f6a441984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.272+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 >> 172.21.15.77:0/3172062113 conn(0x7f6a440fc7d0 msgr2=0x7f6a4406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.273+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 shutdown_connections 2024-09-15T22:24:21.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:21.273+0000 7f6a4912b640 1 -- 172.21.15.77:0/3172062113 wait complete. 2024-09-15T22:24:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:24:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:24:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:24:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:24:21.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.12 MB/sec execute 510 sec latency 19342.610 ms 2024-09-15T22:24:22.071 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:21 smithi077 ceph-mon[30980]: pgmap v373: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 308 KiB/s wr, 23 op/s 2024-09-15T22:24:22.072 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:21 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3172062113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:22.108 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 2ab04924 --release --await' 2024-09-15T22:24:22.108 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 2ab04924 --release --await 2024-09-15T22:24:22.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:21 smithi196 ceph-mon[35109]: pgmap v373: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 308 KiB/s wr, 23 op/s 2024-09-15T22:24:22.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:21 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3172062113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:22.358 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:21 smithi117 ceph-mon[35021]: pgmap v373: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 308 KiB/s wr, 23 op/s 2024-09-15T22:24:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:21 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3172062113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:22.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 883734 7.10 MB/sec execute 511 sec latency 20342.660 ms 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 -- 172.21.15.77:0/983931427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c106a80 msgr2=0x7fca6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 --2- 172.21.15.77:0/983931427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c106a80 0x7fca6c106e60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fca68009930 tx=0x7fca6802a420 comp rx=0 tx=0).stop 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 -- 172.21.15.77:0/983931427 shutdown_connections 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 --2- 172.21.15.77:0/983931427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca6c101420 0x7fca6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 --2- 172.21.15.77:0/983931427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c100a80 0x7fca6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 --2- 172.21.15.77:0/983931427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c106a80 0x7fca6c106e60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 -- 172.21.15.77:0/983931427 >> 172.21.15.77:0/983931427 conn(0x7fca6c0fc7a0 msgr2=0x7fca6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 -- 172.21.15.77:0/983931427 shutdown_connections 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.070+0000 7fca74313640 1 -- 172.21.15.77:0/983931427 wait complete. 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 Processor -- start 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 -- start start 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c101420 0x7fca6c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca6c106a80 0x7fca6c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110f20 con 0x7fca6c100a80 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110da0 con 0x7fca6c106a80 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca74313640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1110a0 con 0x7fca6c101420 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55420/0 (socket says 172.21.15.77:55420) 2024-09-15T22:24:23.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 -- 172.21.15.77:0/2444109574 learned_addr learned my addr 172.21.15.77:0/2444109574 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca718ae640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c101420 0x7fca6c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 -- 172.21.15.77:0/2444109574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c101420 msgr2=0x7fca6c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c101420 0x7fca6c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 -- 172.21.15.77:0/2444109574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca6c106a80 msgr2=0x7fca6c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca6c106a80 0x7fca6c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.071+0000 7fca720af640 1 -- 172.21.15.77:0/2444109574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c19c490 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca720af640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fca68009a50 tx=0x7fca68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca637fe640 1 -- 172.21.15.77:0/2444109574 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6803f030 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca637fe640 1 -- 172.21.15.77:0/2444109574 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca68003ce0 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c19c720 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca637fe640 1 -- 172.21.15.77:0/2444109574 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68035070 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.072+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c1a4060 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.074+0000 7fca637fe640 1 -- 172.21.15.77:0/2444109574 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fca68042030 con 0x7fca6c100a80 2024-09-15T22:24:23.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.074+0000 7fca766ec740 1 Processor -- start 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.074+0000 7fca766ec740 1 -- start start 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca637fe640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 0x7fca580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0603840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562df0559620 con 0x562df05f6b90 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562df0500e50 con 0x562df05f6270 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca766ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562df05555e0 con 0x562df0601450 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca728b0640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 0x7fca580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55432/0 (socket says 172.21.15.77:55432) 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca720af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 -- 172.21.15.77:0/2208157340 learned_addr learned my addr 172.21.15.77:0/2208157340 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca728b0640 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0603840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca637fe640 1 -- 172.21.15.77:0/2444109574 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fca680f0760 con 0x7fca6c100a80 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 -- 172.21.15.77:0/2208157340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 msgr2=0x562df0603840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0603840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 -- 172.21.15.77:0/2208157340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 msgr2=0x562df05f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 -- 172.21.15.77:0/2208157340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df04f1d40 con 0x562df05f6b90 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca728b0640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 0x7fca580a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca64008fd0 tx=0x7fca6400c970 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca718ae640 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b7f0 tx=0x7fca5c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de013c194e1fe015 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.075+0000 7fca60dea640 1 -- 172.21.15.77:0/2208157340 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00d950 con 0x562df05f6b90 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca60dea640 1 -- 172.21.15.77:0/2208157340 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c018070 con 0x562df05f6b90 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca60dea640 1 -- 172.21.15.77:0/2208157340 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0123e0 con 0x562df05f6b90 2024-09-15T22:24:23.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 -- 172.21.15.77:0/2208157340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 msgr2=0x562df0600f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b7f0 tx=0x7fca5c00bcb0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 -- 172.21.15.77:0/2208157340 shutdown_connections 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0603840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df0600f10 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 --2- 172.21.15.77:0/2208157340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 -- 172.21.15.77:0/2208157340 >> 172.21.15.77:0/2208157340 conn(0x562df05f2800 msgr2=0x562df05f2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 -- 172.21.15.77:0/2208157340 shutdown_connections 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.076+0000 7fca766ec740 1 -- 172.21.15.77:0/2208157340 wait complete. 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 -- start start 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 Processor -- start 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df060ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562df05f01c0 con 0x562df05f6b90 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562df04e1f10 con 0x562df05f6270 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562df05f0340 con 0x562df0601450 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca766ec740 10 client.0 _ll_register_callbacks cb 0x562df0505db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55436/0 (socket says 172.21.15.77:55436) 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 -- 172.21.15.77:0/1797050588 learned_addr learned my addr 172.21.15.77:0/1797050588 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca728b0640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df060ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca720af640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 -- 172.21.15.77:0/1797050588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 msgr2=0x562df060ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df060ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 -- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 msgr2=0x562df05fc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05fc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca718ae640 1 -- 172.21.15.77:0/1797050588 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df0268580 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.077+0000 7fca720af640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca718ae640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fca5c008010 tx=0x7fca5c013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7c1977a6eb48c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00dd60 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fca5c01c420 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c018070 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca5c02d050 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 10 client.18513 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562df0332960 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c018070 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca5c01b4d0 con 0x562df05f6b90 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 10 client.18513 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:23.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 10 client.18513 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 20 client.18513 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df060ec70 0x562df0611060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 4 client.18513 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.078+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df0566b90 con 0x562df060ec70 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.079+0000 7fca720af640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df060ec70 0x562df0611060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.079+0000 7fca720af640 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df060ec70 0x562df0611060 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468eaf3a68eae562 server_cookie=ac6b0d9a1eea7a90 in_seq=0 out_seq=0 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.079+0000 7fca55ffb640 10 client.18513 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:23.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca55ffb640 1 -- 172.21.15.77:0/1797050588 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562df0566b90 con 0x562df060ec70 2024-09-15T22:24:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca55ffb640 10 client.18513 handle_command_reply: tid=0 2024-09-15T22:24:23.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 client.18513 shutdown 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 msgr2=0x562df05fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fca5c008010 tx=0x7fca5c013730 comp rx=0 tx=0).stop 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 shutdown_connections 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df060ec70 0x562df0611060 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df060ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca55ffb640 0 client.18513 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05fce70 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 --2- 172.21.15.77:0/1797050588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df05fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 >> 172.21.15.77:0/1797050588 conn(0x562df05f2800 msgr2=0x562df05ef750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 shutdown_connections 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 1 -- 172.21.15.77:0/1797050588 wait complete. 2024-09-15T22:24:23.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.080+0000 7fca766ec740 20 client.18513 trim_cache size 0 max 16384 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 Processor -- start 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 -- start start 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df06007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df05fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ace30 con 0x562df05f6b90 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ae970 con 0x562df05f6270 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca766ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ad500 con 0x562df0601450 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55438/0 (socket says 172.21.15.77:55438) 2024-09-15T22:24:23.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 -- 172.21.15.77:0/27663843 learned_addr learned my addr 172.21.15.77:0/27663843 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 -- 172.21.15.77:0/27663843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 msgr2=0x562df05fa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df05fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 -- 172.21.15.77:0/27663843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 msgr2=0x562df06007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca720af640 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df06007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df06007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca718ae640 1 -- 172.21.15.77:0/27663843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df05f7100 con 0x562df05f6b90 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.097+0000 7fca720af640 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df06007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca718ae640 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fca5c004810 tx=0x7fca5c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9638f220930d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca57fff640 1 -- 172.21.15.77:0/27663843 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c02a030 con 0x562df05f6b90 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca57fff640 1 -- 172.21.15.77:0/27663843 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c018050 con 0x562df05f6b90 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca57fff640 1 -- 172.21.15.77:0/27663843 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c01b380 con 0x562df05f6b90 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 -- 172.21.15.77:0/27663843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 msgr2=0x562df05f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fca5c004810 tx=0x7fca5c001e70 comp rx=0 tx=0).stop 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 -- 172.21.15.77:0/27663843 shutdown_connections 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df05fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6b90 0x562df05f9e70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 --2- 172.21.15.77:0/27663843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6270 0x562df06007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.098+0000 7fca766ec740 1 -- 172.21.15.77:0/27663843 >> 172.21.15.77:0/27663843 conn(0x562df05f2800 msgr2=0x562df0601ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- 172.21.15.77:0/27663843 shutdown_connections 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- 172.21.15.77:0/27663843 wait complete. 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- start start 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 Processor -- start 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 0x562df06163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0609d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ae2a0 con 0x562df05f6270 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1adbd0 con 0x562df05f6b90 2024-09-15T22:24:23.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aae30 con 0x562df0601450 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca766ec740 10 client.0 _ll_register_callbacks cb 0x7fca5c00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca720af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca720af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55444/0 (socket says 172.21.15.77:55444) 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca720af640 1 -- 172.21.15.77:0/483487059 learned_addr learned my addr 172.21.15.77:0/483487059 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca718ae640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 0x562df06163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca720af640 1 -- 172.21.15.77:0/483487059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 msgr2=0x562df0609d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.099+0000 7fca728b0640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0609d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca720af640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0609d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca720af640 1 -- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 msgr2=0x562df06163c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca720af640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 0x562df06163c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca720af640 1 -- 172.21.15.77:0/483487059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df05fd1e0 con 0x562df05f6270 2024-09-15T22:24:23.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca718ae640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 0x562df06163c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca720af640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fca68031550 tx=0x7fca6802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699069df69edada2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68049040 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fca680ba950 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c018050 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca68042ed0 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 10 client.18519 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562df06798f0 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c018050 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fca680b8cc0 con 0x562df05f6270 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 10 client.18519 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 10 client.18519 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 20 client.18519 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:23.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df067a010 0x562df067c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 4 client.18519 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2ab04924", "release": true, "await": true}] 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.100+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2ab04924", "release": true, "await": true}) -- 0x562df05610a0 con 0x562df067a010 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.101+0000 7fca718ae640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df067a010 0x562df067c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.101+0000 7fca55ffb640 10 client.18519 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.101+0000 7fca718ae640 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df067a010 0x562df067c400 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c122311a5305d61c server_cookie=d21e67004b826faa in_seq=0 out_seq=0 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca55ffb640 1 -- 172.21.15.77:0/483487059 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562df05610a0 con 0x562df067a010 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca55ffb640 10 client.18519 handle_command_reply: tid=0 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 client.18519 shutdown 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 msgr2=0x562df0615e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fca68031550 tx=0x7fca6802f000 comp rx=0 tx=0).stop 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 shutdown_connections 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562df067a010 0x562df067c400 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562df0601450 0x562df0609d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562df05f6b90 0x562df06163c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 --2- 172.21.15.77:0/483487059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562df05f6270 0x562df0615e80 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 >> 172.21.15.77:0/483487059 conn(0x562df05f2800 msgr2=0x562df05ef750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca55ffb640 0 client.18519 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 shutdown_connections 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 1 -- 172.21.15.77:0/483487059 wait complete. 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.111+0000 7fca766ec740 20 client.18519 trim_cache size 0 max 16384 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:24:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 44, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "2ab04924": { 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 44, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 141.8, 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:24:23.180 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:24:23.181 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 msgr2=0x7fca580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 0x7fca580a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca64008fd0 tx=0x7fca6400c970 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 msgr2=0x7fca6c1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fca68009a50 tx=0x7fca68004270 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 shutdown_connections 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca580a33b0 0x7fca580a5870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca6c106a80 0x7fca6c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca6c101420 0x7fca6c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 --2- 172.21.15.77:0/2444109574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca6c100a80 0x7fca6c1974e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 >> 172.21.15.77:0/2444109574 conn(0x7fca6c0fc7a0 msgr2=0x7fca6c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:23.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 shutdown_connections 2024-09-15T22:24:23.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:23.112+0000 7fca74313640 1 -- 172.21.15.77:0/2444109574 wait complete. 2024-09-15T22:24:23.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 884151 7.10 MB/sec execute 512 sec latency 20959.851 ms 2024-09-15T22:24:23.919 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2ab04924', total seconds elapsed: 25.5 2024-09-15T22:24:24.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:23 smithi077 ceph-mon[30980]: pgmap v374: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:23 smithi196 ceph-mon[35109]: pgmap v374: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:23 smithi117 ceph-mon[35021]: pgmap v374: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:24.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 885626 7.10 MB/sec execute 513 sec latency 13.811 ms 2024-09-15T22:24:25.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 887682 7.11 MB/sec execute 514 sec latency 27.459 ms 2024-09-15T22:24:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:25 smithi077 ceph-mon[30980]: pgmap v375: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:25 smithi196 ceph-mon[35109]: pgmap v375: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:25 smithi117 ceph-mon[35021]: pgmap v375: 129 pgs: 129 active+clean; 523 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 286 KiB/s wr, 21 op/s 2024-09-15T22:24:26.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 889258 7.11 MB/sec execute 515 sec latency 16.747 ms 2024-09-15T22:24:27.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 891331 7.12 MB/sec execute 516 sec latency 28.961 ms 2024-09-15T22:24:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:27 smithi077 ceph-mon[30980]: pgmap v376: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:28.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:27 smithi196 ceph-mon[35109]: pgmap v376: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:27 smithi117 ceph-mon[35021]: pgmap v376: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:28.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 893025 7.12 MB/sec execute 517 sec latency 18.304 ms 2024-09-15T22:24:29.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 894982 7.13 MB/sec execute 518 sec latency 22.208 ms 2024-09-15T22:24:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:29 smithi077 ceph-mon[30980]: pgmap v377: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:29 smithi196 ceph-mon[35109]: pgmap v377: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:29 smithi117 ceph-mon[35021]: pgmap v377: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 KiB/s wr, 55 op/s 2024-09-15T22:24:30.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 897026 7.13 MB/sec execute 519 sec latency 16.283 ms 2024-09-15T22:24:31.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 898707 7.14 MB/sec execute 520 sec latency 17.183 ms 2024-09-15T22:24:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:31 smithi077 ceph-mon[30980]: pgmap v378: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-09-15T22:24:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:31 smithi196 ceph-mon[35109]: pgmap v378: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-09-15T22:24:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:31 smithi117 ceph-mon[35021]: pgmap v378: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-09-15T22:24:32.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 900974 7.15 MB/sec execute 521 sec latency 17.317 ms 2024-09-15T22:24:33.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 902488 7.15 MB/sec execute 522 sec latency 14.644 ms 2024-09-15T22:24:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:33 smithi077 ceph-mon[30980]: pgmap v379: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:33 smithi196 ceph-mon[35109]: pgmap v379: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:33 smithi117 ceph-mon[35021]: pgmap v379: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:34.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 905088 7.16 MB/sec execute 523 sec latency 28.693 ms 2024-09-15T22:24:35.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 906028 7.16 MB/sec execute 524 sec latency 54.840 ms 2024-09-15T22:24:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:35 smithi077 ceph-mon[30980]: pgmap v380: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:35 smithi196 ceph-mon[35109]: pgmap v380: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:35 smithi117 ceph-mon[35021]: pgmap v380: 129 pgs: 129 active+clean; 542 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 114 op/s 2024-09-15T22:24:36.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 908459 7.17 MB/sec execute 525 sec latency 38.495 ms 2024-09-15T22:24:37.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 909580 7.17 MB/sec execute 526 sec latency 79.312 ms 2024-09-15T22:24:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:37 smithi077 ceph-mon[30980]: pgmap v381: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-09-15T22:24:38.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:37 smithi196 ceph-mon[35109]: pgmap v381: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-09-15T22:24:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:37 smithi117 ceph-mon[35021]: pgmap v381: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-09-15T22:24:38.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 912132 7.18 MB/sec execute 527 sec latency 24.756 ms 2024-09-15T22:24:39.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 913453 7.18 MB/sec execute 528 sec latency 15.668 ms 2024-09-15T22:24:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:39 smithi077 ceph-mon[30980]: pgmap v382: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 149 op/s 2024-09-15T22:24:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:39 smithi196 ceph-mon[35109]: pgmap v382: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 149 op/s 2024-09-15T22:24:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:39 smithi117 ceph-mon[35021]: pgmap v382: 129 pgs: 129 active+clean; 555 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 149 op/s 2024-09-15T22:24:40.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 915934 7.19 MB/sec execute 529 sec latency 27.124 ms 2024-09-15T22:24:41.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 916689 7.18 MB/sec execute 530 sec latency 22.993 ms 2024-09-15T22:24:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:41 smithi077 ceph-mon[30980]: pgmap v383: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-15T22:24:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:41 smithi196 ceph-mon[35109]: pgmap v383: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-15T22:24:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:41 smithi117 ceph-mon[35021]: pgmap v383: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-15T22:24:42.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 918377 7.22 MB/sec execute 531 sec latency 42.065 ms 2024-09-15T22:24:43.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 920695 7.23 MB/sec execute 532 sec latency 14.000 ms 2024-09-15T22:24:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:43 smithi077 ceph-mon[30980]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:43 smithi196 ceph-mon[35109]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:43 smithi117 ceph-mon[35021]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:44.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 923005 7.24 MB/sec execute 533 sec latency 16.803 ms 2024-09-15T22:24:45.492 INFO:tasks.workunit.client.0.smithi077.stdout: 1 924544 7.24 MB/sec execute 534 sec latency 91.823 ms 2024-09-15T22:24:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:45 smithi077 ceph-mon[30980]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:45 smithi196 ceph-mon[35109]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:45 smithi117 ceph-mon[35021]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2024-09-15T22:24:46.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 927299 7.26 MB/sec execute 535 sec latency 13.385 ms 2024-09-15T22:24:47.158 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:24:47.182 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:46 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:47.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:46 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:47.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:46 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:47.432 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:47.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 928762 7.26 MB/sec execute 536 sec latency 43.225 ms 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.921+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2716182305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102ef0 msgr2=0x7f2f6c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.921+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2716182305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c10da40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009ca0 tx=0x7f2f6402a950 comp rx=0 tx=0).stop 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.921+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2716182305 shutdown_connections 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.921+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2716182305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c10da40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.921+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2716182305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c102570 0x7f2f6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2716182305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c100e20 0x7f2f6c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2716182305 >> 172.21.15.77:0/2716182305 conn(0x7f2f6c0fc7d0 msgr2=0x7f2f6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2716182305 shutdown_connections 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2716182305 wait complete. 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 Processor -- start 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.922+0000 7f2f70b6f640 1 -- start start 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c100e20 0x7f2f6c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c110fd0 con 0x7f2f6c102570 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c110e50 con 0x7f2f6c102ef0 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c111150 con 0x7f2f6c100e20 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43728/0 (socket says 172.21.15.77:43728) 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 -- 172.21.15.77:0/2574322581 learned_addr learned my addr 172.21.15.77:0/2574322581 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c100e20 0x7f2f6c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 -- 172.21.15.77:0/2574322581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c100e20 msgr2=0x7f2f6c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c100e20 0x7f2f6c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 -- 172.21.15.77:0/2574322581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c102ef0 msgr2=0x7f2f6c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 -- 172.21.15.77:0/2574322581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c19d4e0 con 0x7f2f6c102570 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9f0 tx=0x7f2f6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f5b7fe640 1 -- 172.21.15.77:0/2574322581 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000cd80 con 0x7f2f6c102570 2024-09-15T22:24:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c19d7d0 con 0x7f2f6c102570 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f5b7fe640 1 -- 172.21.15.77:0/2574322581 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f60005500 con 0x7f2f6c102570 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f5b7fe640 1 -- 172.21.15.77:0/2574322581 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60010690 con 0x7f2f6c102570 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.923+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a50b0 con 0x7f2f6c102570 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.926+0000 7f2f5b7fe640 1 -- 172.21.15.77:0/2574322581 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2f600046c0 con 0x7f2f6c102570 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 Processor -- start 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 -- start start 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 0x55b19446c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 0x55b194477140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b194479a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b1943d22e0 con 0x55b194477680 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b194379ce0 con 0x55b19446cdc0 2024-09-15T22:24:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f72f48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b1943ce2a0 con 0x55b19446c4a0 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b194479a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 0x55b19446c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 0x55b19446c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37000/0 (socket says 172.21.15.77:37000) 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f6ad9d640 1 -- 172.21.15.77:0/320059074 learned_addr learned my addr 172.21.15.77:0/320059074 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.927+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 0x55b194477140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f5b7fe640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 0x7f2f540a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 0x7f2f540a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 -- 172.21.15.77:0/320059074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 msgr2=0x55b19446c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 0x55b19446c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f5b7fe640 1 -- 172.21.15.77:0/2574322581 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f2f600d04b0 con 0x7f2f6c102570 2024-09-15T22:24:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 -- 172.21.15.77:0/320059074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 msgr2=0x55b194479a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b194479a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 -- 172.21.15.77:0/320059074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19435e440 con 0x55b19446cdc0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b194479a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 0x55b194477140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bbf0 tx=0x7f2f5c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba6ee8e8a5490c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 0x7f2f540a9c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009a60 tx=0x7f2f6402a950 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f58ff9640 1 -- 172.21.15.77:0/320059074 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c004030 con 0x55b19446cdc0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f58ff9640 1 -- 172.21.15.77:0/320059074 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f5c0041d0 con 0x55b19446cdc0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.928+0000 7f2f58ff9640 1 -- 172.21.15.77:0/320059074 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0116a0 con 0x55b19446cdc0 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 -- 172.21.15.77:0/320059074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 msgr2=0x55b194477140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 0x55b194477140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bbf0 tx=0x7f2f5c0107a0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 -- 172.21.15.77:0/320059074 shutdown_connections 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b194479a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446cdc0 0x55b194477140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 --2- 172.21.15.77:0/320059074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446c4a0 0x55b19446c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 -- 172.21.15.77:0/320059074 >> 172.21.15.77:0/320059074 conn(0x55b194468a30 msgr2=0x55b194468e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 -- 172.21.15.77:0/320059074 shutdown_connections 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.929+0000 7f2f72f48740 1 -- 172.21.15.77:0/320059074 wait complete. 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 -- start start 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 Processor -- start 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b194474280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 0x55b1944747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b19447a250 con 0x55b19446c4a0 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b19438d050 con 0x55b194477680 2024-09-15T22:24:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b19447a3d0 con 0x55b19446cdc0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f72f48740 10 client.0 _ll_register_callbacks cb 0x55b1943cd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43748/0 (socket says 172.21.15.77:43748) 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 -- 172.21.15.77:0/2046954293 learned_addr learned my addr 172.21.15.77:0/2046954293 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b194474280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 0x55b1944747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 -- 172.21.15.77:0/2046954293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 msgr2=0x55b194474280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b194474280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 -- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 msgr2=0x55b1944747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 0x55b1944747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6a59c640 1 -- 172.21.15.77:0/2046954293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1941433c0 con 0x55b19446c4a0 2024-09-15T22:24:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.930+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 0x55b1944747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000c00 tx=0x7f2f5c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947c6db6981e7e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c02a060 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f2f5c0035f0 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c0041d0 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c0037f0 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f72f48740 10 client.18531 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b194485c20 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.931+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f5c0041d0 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f5c0047e0 con 0x55b19446c4a0 2024-09-15T22:24:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 10 client.18531 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 20 client.18531 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b194489420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 4 client.18531 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1943df850 con 0x55b194487030 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.932+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b194489420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.933+0000 7f2f51ffb640 10 client.18531 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.933+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b194489420 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf8a0b5301e7508 server_cookie=1cafd6224567ec8d in_seq=0 out_seq=0 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.933+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2046954293 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1943df850 con 0x55b194487030 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.933+0000 7f2f51ffb640 10 client.18531 handle_command_reply: tid=0 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.933+0000 7f2f72f48740 1 client.18531 shutdown 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 msgr2=0x55b194476fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000c00 tx=0x7f2f5c00bfb0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 shutdown_connections 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b194489420 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f51ffb640 0 client.18531 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b194477680 0x55b1944747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b194474280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2046954293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446c4a0 0x55b194476fd0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 >> 172.21.15.77:0/2046954293 conn(0x55b194468a30 msgr2=0x55b19446e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 shutdown_connections 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 1 -- 172.21.15.77:0/2046954293 wait complete. 2024-09-15T22:24:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.934+0000 7f2f72f48740 20 client.18531 trim_cache size 0 max 16384 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 Processor -- start 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 -- start start 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b1944834b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b194477680 0x55b19448ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1ace10 con 0x55b19446cdc0 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1ae950 con 0x55b19446c4a0 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f72f48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1ad4e0 con 0x55b194477680 2024-09-15T22:24:47.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43756/0 (socket says 172.21.15.77:43756) 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 -- 172.21.15.77:0/698288943 learned_addr learned my addr 172.21.15.77:0/698288943 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b194477680 0x55b19448ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b1944834b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 -- 172.21.15.77:0/698288943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b194477680 msgr2=0x55b19448ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b194477680 0x55b19448ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 -- 172.21.15.77:0/698288943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 msgr2=0x55b1944834b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b1944834b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f69d9b640 1 -- 172.21.15.77:0/698288943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b194489790 con 0x55b19446cdc0 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.953+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b1944834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.954+0000 7f2f69d9b640 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2f60098d70 tx=0x7f2f600bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2144bba7cd84a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.954+0000 7f2f53fff640 1 -- 172.21.15.77:0/698288943 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6009bd00 con 0x55b19446cdc0 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.954+0000 7f2f53fff640 1 -- 172.21.15.77:0/698288943 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6009caa0 con 0x55b19446cdc0 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.954+0000 7f2f53fff640 1 -- 172.21.15.77:0/698288943 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60098a10 con 0x55b19446cdc0 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- 172.21.15.77:0/698288943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 msgr2=0x55b19448e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2f60098d70 tx=0x7f2f600bc040 comp rx=0 tx=0).stop 2024-09-15T22:24:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- 172.21.15.77:0/698288943 shutdown_connections 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b194477680 0x55b19448ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b19446cdc0 0x55b19448e9c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 --2- 172.21.15.77:0/698288943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b1944834b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- 172.21.15.77:0/698288943 >> 172.21.15.77:0/698288943 conn(0x55b194468a30 msgr2=0x55b19446f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- 172.21.15.77:0/698288943 shutdown_connections 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- 172.21.15.77:0/698288943 wait complete. 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 -- start start 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.955+0000 7f2f72f48740 1 Processor -- start 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b19447fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b19447ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1ae280 con 0x55b194477680 2024-09-15T22:24:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1adbb0 con 0x55b19446c4a0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c1aae10 con 0x55b19446cdc0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f72f48740 10 client.0 _ll_register_callbacks cb 0x7f2f600986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f69d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b19447fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b19447ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53154/0 (socket says 172.21.15.77:53154) 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6a59c640 1 -- 172.21.15.77:0/2635203519 learned_addr learned my addr 172.21.15.77:0/2635203519 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 -- 172.21.15.77:0/2635203519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 msgr2=0x55b19447fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b19447fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 -- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 msgr2=0x55b19447f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 -- 172.21.15.77:0/2635203519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19447b5a0 con 0x55b194477680 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f6ad9d640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b19447ad70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f64006010 tx=0x7f2f640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121c6aefc6f00d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.956+0000 7f2f51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64043050 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f2f64004240 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6009caa0 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f64034040 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 10 client.18537 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b1944f1580 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6009caa0 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f64053020 con 0x55b194477680 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 10 client.18537 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 10 client.18537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 20 client.18537 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b1944f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 4 client.18537 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1943d9d60 con 0x55b194487030 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.957+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b1944f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f51ffb640 10 client.18537 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f6a59c640 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b1944f3cb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf56ed20caae067 server_cookie=7d9e1262b31bc2b7 in_seq=0 out_seq=0 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f51ffb640 1 -- 172.21.15.77:0/2635203519 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b1943d9d60 con 0x55b194487030 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f51ffb640 10 client.18537 handle_command_reply: tid=0 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 client.18537 shutdown 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 msgr2=0x55b19447ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b19447ad70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f64006010 tx=0x7f2f640095c0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 shutdown_connections 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b194487030 0x55b1944f3cb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f51ffb640 0 client.18537 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b194477680 0x55b19447ad70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b19446cdc0 0x55b19447fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 --2- 172.21.15.77:0/2635203519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b19446c4a0 0x55b19447f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 >> 172.21.15.77:0/2635203519 conn(0x55b194468a30 msgr2=0x55b19446def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 shutdown_connections 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 1 -- 172.21.15.77:0/2635203519 wait complete. 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.958+0000 7f2f72f48740 20 client.18537 trim_cache size 0 max 16384 2024-09-15T22:24:47.965 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:24:47.966 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:24:47.966 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:24:47.966 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:24:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 msgr2=0x7f2f540a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 0x7f2f540a9c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009a60 tx=0x7f2f6402a950 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 msgr2=0x7f2f6c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9f0 tx=0x7f2f6000edf0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 shutdown_connections 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2f540a77c0 0x7f2f540a9c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f6c102ef0 0x7f2f6c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f6c102570 0x7f2f6c198a70 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 --2- 172.21.15.77:0/2574322581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f6c100e20 0x7f2f6c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 >> 172.21.15.77:0/2574322581 conn(0x7f2f6c0fc7d0 msgr2=0x7f2f6c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 shutdown_connections 2024-09-15T22:24:47.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:47.959+0000 7f2f70b6f640 1 -- 172.21.15.77:0/2574322581 wait complete. 2024-09-15T22:24:48.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:47 smithi077 ceph-mon[30980]: pgmap v386: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-15T22:24:48.254 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:47 smithi196 ceph-mon[35109]: pgmap v386: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-15T22:24:48.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:47 smithi117 ceph-mon[35021]: pgmap v386: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-09-15T22:24:48.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 931043 7.27 MB/sec execute 537 sec latency 68.947 ms 2024-09-15T22:24:48.863 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:24:48.864 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:49.178 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:49.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 932775 7.27 MB/sec execute 538 sec latency 38.011 ms 2024-09-15T22:24:49.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 -- 172.21.15.77:0/928502430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4106f00 msgr2=0x7f77b410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 --2- 172.21.15.77:0/928502430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4106f00 0x7f77b410ea80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009ca0 tx=0x7f77ac02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 -- 172.21.15.77:0/928502430 shutdown_connections 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 --2- 172.21.15.77:0/928502430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4106f00 0x7f77b410ea80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 --2- 172.21.15.77:0/928502430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4102780 0x7f77b41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 --2- 172.21.15.77:0/928502430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4101dd0 0x7f77b41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 -- 172.21.15.77:0/928502430 >> 172.21.15.77:0/928502430 conn(0x7f77b40fd8b0 msgr2=0x7f77b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 -- 172.21.15.77:0/928502430 shutdown_connections 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.666+0000 7f77bc519640 1 -- 172.21.15.77:0/928502430 wait complete. 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 Processor -- start 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 -- start start 2024-09-15T22:24:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 0x7f77b41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4106f00 0x7f77b419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4111f60 con 0x7f77b4102780 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4111de0 con 0x7f77b4101dd0 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77bc519640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41120e0 con 0x7f77b4106f00 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42610/0 (socket says 172.21.15.77:42610) 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 -- 172.21.15.77:0/2674453275 learned_addr learned my addr 172.21.15.77:0/2674453275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77baab6640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4106f00 0x7f77b419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77ba2b5640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 0x7f77b41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 -- 172.21.15.77:0/2674453275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4106f00 msgr2=0x7f77b419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4106f00 0x7f77b419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 -- 172.21.15.77:0/2674453275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 msgr2=0x7f77b41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 0x7f77b41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77b9ab4640 1 -- 172.21.15.77:0/2674453275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b419d370 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.667+0000 7f77ba2b5640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 0x7f77b41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77b9ab4640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ea60 tx=0x7f77a400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a400cde0 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b419d660 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a40035f0 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4010670 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.668+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b41a4f40 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.669+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b41021b0 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.670+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77a40037a0 con 0x7f77b4102780 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.671+0000 7f77ab7fe640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 0x7f77a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.671+0000 7f77ba2b5640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 0x7f77a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.671+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f77a40d0490 con 0x7f77b4102780 2024-09-15T22:24:49.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.672+0000 7f77ba2b5640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 0x7f77a00a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009c20 tx=0x7f77b0005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:49.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.675+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77a409cb70 con 0x7f77b4102780 2024-09-15T22:24:49.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.896+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b402d7c0 con 0x7f77b4102780 2024-09-15T22:24:49.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.896+0000 7f77ab7fe640 1 -- 172.21.15.77:0/2674453275 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f77a409c470 con 0x7f77b4102780 2024-09-15T22:24:49.898 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:24:49.898 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:24:49.899 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 msgr2=0x7f77a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 0x7f77a00a5870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009c20 tx=0x7f77b0005f90 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 msgr2=0x7f77b4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ea60 tx=0x7f77a400ef20 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 shutdown_connections 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77a00a33b0 0x7f77a00a5870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b4106f00 0x7f77b419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b4102780 0x7f77b4198900 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 --2- 172.21.15.77:0/2674453275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b4101dd0 0x7f77b41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 >> 172.21.15.77:0/2674453275 conn(0x7f77b40fd8b0 msgr2=0x7f77b40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:49.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 shutdown_connections 2024-09-15T22:24:49.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:49.899+0000 7f77bc519640 1 -- 172.21.15.77:0/2674453275 wait complete. 2024-09-15T22:24:50.243 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:49 smithi077 ceph-mon[30980]: pgmap v387: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-09-15T22:24:50.244 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:49 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2674453275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:50.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:49 smithi196 ceph-mon[35109]: pgmap v387: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-09-15T22:24:50.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:49 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2674453275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:49 smithi117 ceph-mon[35021]: pgmap v387: 129 pgs: 129 active+clean; 580 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-09-15T22:24:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:49 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2674453275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:50.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 934984 7.28 MB/sec execute 539 sec latency 38.948 ms 2024-09-15T22:24:50.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:24:51.040 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:51.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:50 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:51.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:50 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:51.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:50 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:24:51.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:50 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:51.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:50 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:51.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:50 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:24:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:50 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:50 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:50 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:24:51.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 937369 7.29 MB/sec execute 540 sec latency 39.566 ms 2024-09-15T22:24:51.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 -- 172.21.15.77:0/236107095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc104010 msgr2=0x7f84dc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 --2- 172.21.15.77:0/236107095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc104010 0x7f84dc1043f0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009900 tx=0x7f84c402a410 comp rx=0 tx=0).stop 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 -- 172.21.15.77:0/236107095 shutdown_connections 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 --2- 172.21.15.77:0/236107095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 0x7f84dc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 --2- 172.21.15.77:0/236107095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc1049c0 0x7f84dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 --2- 172.21.15.77:0/236107095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc104010 0x7f84dc1043f0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 -- 172.21.15.77:0/236107095 >> 172.21.15.77:0/236107095 conn(0x7f84dc0ffa50 msgr2=0x7f84dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 -- 172.21.15.77:0/236107095 shutdown_connections 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 -- 172.21.15.77:0/236107095 wait complete. 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.680+0000 7f84e1e55640 1 Processor -- start 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 -- start start 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc104010 0x7f84dc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 0x7f84dc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc114170 con 0x7f84dc1049c0 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc113ff0 con 0x7f84dc104010 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84e1e55640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1142f0 con 0x7f84dc109140 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42628/0 (socket says 172.21.15.77:42628) 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 -- 172.21.15.77:0/2745235239 learned_addr learned my addr 172.21.15.77:0/2745235239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84db7fe640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc104010 0x7f84dc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84dbfff640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 0x7f84dc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 -- 172.21.15.77:0/2745235239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 msgr2=0x7f84dc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 0x7f84dc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 -- 172.21.15.77:0/2745235239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc104010 msgr2=0x7f84dc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc104010 0x7f84dc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 -- 172.21.15.77:0/2745235239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc19f5e0 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.681+0000 7f84daffd640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9f0 tx=0x7f84cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.682+0000 7f84d8ff9640 1 -- 172.21.15.77:0/2745235239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc00cd80 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.682+0000 7f84d8ff9640 1 -- 172.21.15.77:0/2745235239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc010910 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.682+0000 7f84d8ff9640 1 -- 172.21.15.77:0/2745235239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc018990 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.682+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc19f8d0 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.682+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a71b0 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.684+0000 7f84d8ff9640 1 -- 172.21.15.77:0/2745235239 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f84cc010ac0 con 0x7f84dc1049c0 2024-09-15T22:24:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.684+0000 7f84e422e740 1 Processor -- start 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.684+0000 7f84e422e740 1 -- start start 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77d0370 0x5627f77d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84d8ff9640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 0x7f84c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5627f772b2f0 con 0x5627f77d0370 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5627f76d2cf0 con 0x5627f77c6b40 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84e422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5627f77272b0 con 0x5627f77c55d0 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77d0370 0x5627f77d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:50948/0 (socket says 172.21.15.77:50948) 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 -- 172.21.15.77:0/4035048110 learned_addr learned my addr 172.21.15.77:0/4035048110 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 0x7f84c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84d8ff9640 1 -- 172.21.15.77:0/2745235239 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f84cc0d03c0 con 0x7f84dc1049c0 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 -- 172.21.15.77:0/4035048110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 msgr2=0x5627f77c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 -- 172.21.15.77:0/4035048110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77d0370 msgr2=0x5627f77d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77d0370 0x5627f77d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 -- 172.21.15.77:0/4035048110 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f76b73a0 con 0x5627f77c6b40 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.685+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005f40 tx=0x7f84d400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8805ec7a59eda52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84dbfff640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 0x7f84c80a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f84d400fc90 tx=0x7f84d4018000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84d1ffb640 1 -- 172.21.15.77:0/4035048110 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4022070 con 0x5627f77c6b40 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84d1ffb640 1 -- 172.21.15.77:0/4035048110 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4003e50 con 0x5627f77c6b40 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84d1ffb640 1 -- 172.21.15.77:0/4035048110 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4004590 con 0x5627f77c6b40 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 -- 172.21.15.77:0/4035048110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 msgr2=0x5627f77c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005f40 tx=0x7f84d400f790 comp rx=0 tx=0).stop 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 -- 172.21.15.77:0/4035048110 shutdown_connections 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77d0370 0x5627f77d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.686+0000 7f84e422e740 1 --2- 172.21.15.77:0/4035048110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c6b40 0x5627f77c5090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- 172.21.15.77:0/4035048110 >> 172.21.15.77:0/4035048110 conn(0x5627f778e310 msgr2=0x5627f77c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- 172.21.15.77:0/4035048110 shutdown_connections 2024-09-15T22:24:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- 172.21.15.77:0/4035048110 wait complete. 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- start start 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 Processor -- start 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5627f77d3010 con 0x5627f77c6b40 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5627f76e6060 con 0x5627f77d0370 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5627f77d3190 con 0x5627f77c55d0 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84e422e740 10 client.0 _ll_register_callbacks cb 0x5627f74398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.687+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42644/0 (socket says 172.21.15.77:42644) 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47382/0 (socket says 172.21.15.77:47382) 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84daffd640 1 -- 172.21.15.77:0/4165629991 learned_addr learned my addr 172.21.15.77:0/4165629991 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 -- 172.21.15.77:0/4165629991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 msgr2=0x5627f77cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 -- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 msgr2=0x5627f77cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 -- 172.21.15.77:0/4165629991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f749c3c0 con 0x5627f77d0370 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77cd630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84d40041f0 tx=0x7f84d4019160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abec46619a6ff48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84baffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:24:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4022040 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d4003e50 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f84d4003b80 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84d4038050 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84e422e740 10 client.27533 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.688+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5627f77deaa0 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4003e50 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84d4027820 con 0x5627f77d0370 2024-09-15T22:24:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 10 client.27533 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 10 client.27533 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 20 client.27533 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f77dfdf0 0x5627f77e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 4 client.27533 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f7738860 con 0x5627f77dfdf0 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f77dfdf0 0x5627f77e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84baffd640 10 client.27533 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.689+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f77dfdf0 0x5627f77e21e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bdb9fba30389f5eb server_cookie=1f468c04240b2889 in_seq=0 out_seq=0 2024-09-15T22:24:51.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84baffd640 1 -- 172.21.15.77:0/4165629991 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627f7738860 con 0x5627f77dfdf0 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84baffd640 10 client.27533 handle_command_reply: tid=0 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 client.27533 shutdown 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 msgr2=0x5627f77cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77cd630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84d40041f0 tx=0x7f84d4019160 comp rx=0 tx=0).stop 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 shutdown_connections 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f77dfdf0 0x5627f77e21e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77cd630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 --2- 172.21.15.77:0/4165629991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c55d0 0x5627f77cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 >> 172.21.15.77:0/4165629991 conn(0x5627f778e310 msgr2=0x5627f77c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84baffd640 0 client.27533 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 shutdown_connections 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 1 -- 172.21.15.77:0/4165629991 wait complete. 2024-09-15T22:24:51.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.690+0000 7f84e422e740 20 client.27533 trim_cache size 0 max 16384 2024-09-15T22:24:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.703+0000 7f84e422e740 1 Processor -- start 2024-09-15T22:24:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.703+0000 7f84e422e740 1 -- start start 2024-09-15T22:24:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 0x5627f77dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77d0370 0x5627f77c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1acd70 con 0x5627f77c6b40 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ae8b0 con 0x5627f77c55d0 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84e422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ad440 con 0x5627f77d0370 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42646/0 (socket says 172.21.15.77:42646) 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84daffd640 1 -- 172.21.15.77:0/4280240908 learned_addr learned my addr 172.21.15.77:0/4280240908 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 0x5627f77dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84dbfff640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77d0370 0x5627f77c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 -- 172.21.15.77:0/4280240908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77d0370 msgr2=0x5627f77c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77d0370 0x5627f77c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 -- 172.21.15.77:0/4280240908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 msgr2=0x5627f77c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 -- 172.21.15.77:0/4280240908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f77d0090 con 0x5627f77c55d0 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84daffd640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84db7fe640 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 0x5627f77dc2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005940 tx=0x7f84c40058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf90014d104dda97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.704+0000 7f84d17fa640 1 -- 172.21.15.77:0/4280240908 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4035040 con 0x5627f77c55d0 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84d17fa640 1 -- 172.21.15.77:0/4280240908 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4004240 con 0x5627f77c55d0 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84d17fa640 1 -- 172.21.15.77:0/4280240908 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4005650 con 0x5627f77c55d0 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 -- 172.21.15.77:0/4280240908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 msgr2=0x5627f77dc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 0x5627f77dc2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005940 tx=0x7f84c40058d0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 -- 172.21.15.77:0/4280240908 shutdown_connections 2024-09-15T22:24:51.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77d0370 0x5627f77c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c6b40 0x5627f77c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 --2- 172.21.15.77:0/4280240908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77c55d0 0x5627f77dc2b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 -- 172.21.15.77:0/4280240908 >> 172.21.15.77:0/4280240908 conn(0x5627f778e310 msgr2=0x5627f77cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 -- 172.21.15.77:0/4280240908 shutdown_connections 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.705+0000 7f84e422e740 1 -- 172.21.15.77:0/4280240908 wait complete. 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 -- start start 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 Processor -- start 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 0x5627f77d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1ae1e0 con 0x5627f77c55d0 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1adb10 con 0x5627f77d0370 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1aad70 con 0x5627f77c6b40 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84e422e740 10 client.0 _ll_register_callbacks cb 0x7f84cc0baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84db7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42654/0 (socket says 172.21.15.77:42654) 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84daffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 0x5627f77d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84db7fe640 1 -- 172.21.15.77:0/2731634463 learned_addr learned my addr 172.21.15.77:0/2731634463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:51.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.706+0000 7f84dbfff640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 -- 172.21.15.77:0/2731634463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 msgr2=0x5627f77c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 -- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 msgr2=0x5627f77d8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 -- 172.21.15.77:0/2731634463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f77ca0e0 con 0x5627f77c6b40 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84db7fe640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84dbfff640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84daffd640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 0x5627f77d8b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9c0 tx=0x7f84cc098060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f421fd2376602351 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84baffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0cdc00 con 0x5627f77c6b40 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c4004240 con 0x5627f77c6b40 2024-09-15T22:24:51.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f84cc018b30 con 0x5627f77c6b40 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84cc0b5240 con 0x5627f77c6b40 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84e422e740 10 client.27454 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5627f784a5e0 con 0x5627f77c6b40 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.707+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4004240 con 0x5627f77c6b40 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84cc0a0170 con 0x5627f77c6b40 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 10 client.27454 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 10 client.27454 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 20 client.27454 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f784ad80 0x5627f784d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 4 client.27454 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627f7732d70 con 0x5627f784ad80 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84db7fe640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f784ad80 0x5627f784d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:51.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84db7fe640 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f784ad80 0x5627f784d1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=257b3121dfc101ed server_cookie=7814d6555fb578a2 in_seq=0 out_seq=0 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84baffd640 10 client.27454 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84baffd640 1 -- 172.21.15.77:0/2731634463 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627f7732d70 con 0x5627f784ad80 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84baffd640 10 client.27454 handle_command_reply: tid=0 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 1 client.27454 shutdown 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.708+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 msgr2=0x5627f77d8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 0x5627f77d8b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9c0 tx=0x7f84cc098060 comp rx=0 tx=0).stop 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 shutdown_connections 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5627f784ad80 0x5627f784d1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627f77d0370 0x5627f77c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627f77c6b40 0x5627f77d8b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 --2- 172.21.15.77:0/2731634463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627f77c55d0 0x5627f77d8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 >> 172.21.15.77:0/2731634463 conn(0x5627f778e310 msgr2=0x5627f77ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84baffd640 0 client.27454 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:24:51.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 shutdown_connections 2024-09-15T22:24:51.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 1 -- 172.21.15.77:0/2731634463 wait complete. 2024-09-15T22:24:51.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e422e740 20 client.27454 trim_cache size 0 max 16384 2024-09-15T22:24:51.723 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 msgr2=0x7f84c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 0x7f84c80a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f84d400fc90 tx=0x7f84d4018000 comp rx=0 tx=0).stop 2024-09-15T22:24:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 msgr2=0x7f84dc19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:51.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9f0 tx=0x7f84cc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 shutdown_connections 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f84c80a33b0 0x7f84c80a5870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84dc109140 0x7f84dc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84dc1049c0 0x7f84dc19ab70 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 --2- 172.21.15.77:0/2745235239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84dc104010 0x7f84dc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.709+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 >> 172.21.15.77:0/2745235239 conn(0x7f84dc0ffa50 msgr2=0x7f84dc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.710+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 shutdown_connections 2024-09-15T22:24:51.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:51.710+0000 7f84e1e55640 1 -- 172.21.15.77:0/2745235239 wait complete. 2024-09-15T22:24:52.476 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:52 smithi077 ceph-mon[30980]: pgmap v388: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2024-09-15T22:24:52.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 938925 7.29 MB/sec execute 541 sec latency 31.935 ms 2024-09-15T22:24:52.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:24:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:52 smithi196 ceph-mon[35109]: pgmap v388: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2024-09-15T22:24:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:52 smithi117 ceph-mon[35021]: pgmap v388: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 199 op/s 2024-09-15T22:24:52.770 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 -- 172.21.15.77:0/2220373255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2264101df0 msgr2=0x7f22641021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 --2- 172.21.15.77:0/2220373255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2264101df0 0x7f22641021d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 -- 172.21.15.77:0/2220373255 shutdown_connections 2024-09-15T22:24:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 --2- 172.21.15.77:0/2220373255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 0x7f226410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 --2- 172.21.15.77:0/2220373255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22641027a0 0x7f22641067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 --2- 172.21.15.77:0/2220373255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2264101df0 0x7f22641021d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 -- 172.21.15.77:0/2220373255 >> 172.21.15.77:0/2220373255 conn(0x7f22640fd8b0 msgr2=0x7f22640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 -- 172.21.15.77:0/2220373255 shutdown_connections 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.259+0000 7f226abcd640 1 -- 172.21.15.77:0/2220373255 wait complete. 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 Processor -- start 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 -- start start 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264101df0 0x7f22640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 0x7f2264074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2264111f80 con 0x7f22641027a0 2024-09-15T22:24:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2264111e00 con 0x7f2264106f20 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226abcd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2264112100 con 0x7f2264101df0 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f2263fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f2268969640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264101df0 0x7f22640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f2263fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42682/0 (socket says 172.21.15.77:42682) 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f2263fff640 1 -- 172.21.15.77:0/866358995 learned_addr learned my addr 172.21.15.77:0/866358995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.260+0000 7f226916a640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 0x7f2264074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 -- 172.21.15.77:0/866358995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264101df0 msgr2=0x7f22640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264101df0 0x7f22640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 -- 172.21.15.77:0/866358995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 msgr2=0x7f2264074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 0x7f2264074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 -- 172.21.15.77:0/866358995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641a6950 con 0x7f22641027a0 2024-09-15T22:24:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2263fff640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f225000e9f0 tx=0x7f225000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2261ffb640 1 -- 172.21.15.77:0/866358995 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225000cd80 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2261ffb640 1 -- 172.21.15.77:0/866358995 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250010910 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641a6c40 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f2261ffb640 1 -- 172.21.15.77:0/866358995 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250018990 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.261+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641a7180 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2261ffb640 1 -- 172.21.15.77:0/866358995 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2250010ac0 con 0x7f22641027a0 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 Processor -- start 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 -- start start 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56554337ab40 0x565543379090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565543384370 0x565543386760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5655432df2f0 con 0x565543384370 2024-09-15T22:24:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565543286cf0 con 0x56554337ab40 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226cfa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5655432db2b0 con 0x5655433795d0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2261ffb640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 0x7f22540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226916a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56554337ab40 0x565543379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2268969640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2268969640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47440/0 (socket says 172.21.15.77:47440) 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2268969640 1 -- 172.21.15.77:0/1049970300 learned_addr learned my addr 172.21.15.77:0/1049970300 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f2263fff640 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565543384370 0x565543386760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.264+0000 7f226916a640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 0x7f22540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2261ffb640 1 -- 172.21.15.77:0/866358995 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f22500d0320 con 0x7f22641027a0 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 -- 172.21.15.77:0/1049970300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56554337ab40 msgr2=0x565543379090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56554337ab40 0x565543379090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 -- 172.21.15.77:0/1049970300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565543384370 msgr2=0x565543386760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565543384370 0x565543386760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 -- 172.21.15.77:0/1049970300 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554326b3a0 con 0x5655433795d0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f2268969640 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e26c7ff0f8deef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f226916a640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 0x7f22540a58f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f224effd640 1 -- 172.21.15.77:0/1049970300 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258049070 con 0x5655433795d0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f224effd640 1 -- 172.21.15.77:0/1049970300 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258036040 con 0x5655433795d0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.265+0000 7f224effd640 1 -- 172.21.15.77:0/1049970300 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225802f990 con 0x5655433795d0 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- 172.21.15.77:0/1049970300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 msgr2=0x565543379a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2258009bf0 tx=0x7f2258005ef0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- 172.21.15.77:0/1049970300 shutdown_connections 2024-09-15T22:24:53.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565543384370 0x565543386760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x565543379a10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 --2- 172.21.15.77:0/1049970300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56554337ab40 0x565543379090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- 172.21.15.77:0/1049970300 >> 172.21.15.77:0/1049970300 conn(0x565543342310 msgr2=0x565543375c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- 172.21.15.77:0/1049970300 shutdown_connections 2024-09-15T22:24:53.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- 172.21.15.77:0/1049970300 wait complete. 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 -- start start 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.266+0000 7f226cfa6740 1 Processor -- start 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x565543380fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x565543381530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x565543386e90 con 0x5655433795d0 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56554329a060 con 0x565543384370 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565543387010 con 0x56554337ab40 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226cfa6740 10 client.0 _ll_register_callbacks cb 0x565542fed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42710/0 (socket says 172.21.15.77:42710) 2024-09-15T22:24:53.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 -- 172.21.15.77:0/2203893324 learned_addr learned my addr 172.21.15.77:0/2203893324 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2263fff640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x565543380fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f226916a640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x565543381530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 -- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 msgr2=0x565543380fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x565543380fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 -- 172.21.15.77:0/2203893324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 msgr2=0x565543381530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x565543381530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 -- 172.21.15.77:0/2203893324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655430503c0 con 0x5655433795d0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2263fff640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x565543380fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2268969640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2258047040 tx=0x7f2258031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3659160d4e5cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.267+0000 7f2243fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258054060 con 0x5655433795d0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f225802f6f0 con 0x5655433795d0 2024-09-15T22:24:53.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2258036040 con 0x5655433795d0 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2258049070 con 0x5655433795d0 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 10 client.18573 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565543386e90 con 0x5655433795d0 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258036040 con 0x5655433795d0 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f22580309c0 con 0x5655433795d0 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 10 client.18573 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 10 client.18573 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 20 client.18573 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565543393a40 0x565543395e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 4 client.18573 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:53.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655432ec860 con 0x565543393a40 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.268+0000 7f2263fff640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565543393a40 0x565543395e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.269+0000 7f2263fff640 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565543393a40 0x565543395e30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=efaa5a6273824ff2 server_cookie=3001d93452af5b38 in_seq=0 out_seq=0 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.269+0000 7f2243fff640 10 client.18573 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.269+0000 7f2243fff640 1 -- 172.21.15.77:0/2203893324 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5655432ec860 con 0x565543393a40 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.269+0000 7f2243fff640 10 client.18573 handle_command_reply: tid=0 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.269+0000 7f226cfa6740 1 client.18573 shutdown 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 msgr2=0x565543383d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2258047040 tx=0x7f2258031700 comp rx=0 tx=0).stop 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 shutdown_connections 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x565543393a40 0x565543395e30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x565543381530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x565543380fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 --2- 172.21.15.77:0/2203893324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543383d00 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f2243fff640 0 client.18573 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 >> 172.21.15.77:0/2203893324 conn(0x565543342310 msgr2=0x56554337b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 shutdown_connections 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 1 -- 172.21.15.77:0/2203893324 wait complete. 2024-09-15T22:24:53.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.270+0000 7f226cfa6740 20 client.18573 trim_cache size 0 max 16384 2024-09-15T22:24:53.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 Processor -- start 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 -- start start 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x56554337c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b1170 con 0x5655433795d0 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b2cb0 con 0x565543384370 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f226cfa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b1840 con 0x56554337ab40 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.285+0000 7f2268969640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2263fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x56554337c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42714/0 (socket says 172.21.15.77:42714) 2024-09-15T22:24:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 -- 172.21.15.77:0/3405969722 learned_addr learned my addr 172.21.15.77:0/3405969722 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f226916a640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 -- 172.21.15.77:0/3405969722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 msgr2=0x56554337c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x56554337c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 -- 172.21.15.77:0/3405969722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 msgr2=0x56554337c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 -- 172.21.15.77:0/3405969722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565543376960 con 0x5655433795d0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f226916a640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2263fff640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x56554337c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f2268969640 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2258005cc0 tx=0x7f2258031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c969aae1cb0028c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f224e7fc640 1 -- 172.21.15.77:0/3405969722 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258054070 con 0x5655433795d0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f224e7fc640 1 -- 172.21.15.77:0/3405969722 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258040070 con 0x5655433795d0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.286+0000 7f224e7fc640 1 -- 172.21.15.77:0/3405969722 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258044a90 con 0x5655433795d0 2024-09-15T22:24:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 -- 172.21.15.77:0/3405969722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 msgr2=0x565543380260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2258005cc0 tx=0x7f2258031ec0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 -- 172.21.15.77:0/3405969722 shutdown_connections 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56554337ab40 0x56554337c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 --2- 172.21.15.77:0/3405969722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5655433795d0 0x565543380260 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 -- 172.21.15.77:0/3405969722 >> 172.21.15.77:0/3405969722 conn(0x565543342310 msgr2=0x565543374d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.287+0000 7f226cfa6740 1 -- 172.21.15.77:0/3405969722 shutdown_connections 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 -- 172.21.15.77:0/3405969722 wait complete. 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 -- start start 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 Processor -- start 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x56554338c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b25e0 con 0x56554337ab40 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b1f10 con 0x565543384370 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22641af170 con 0x5655433795d0 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.288+0000 7f226cfa6740 10 client.0 _ll_register_callbacks cb 0x7f2258047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226916a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2268969640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x56554338c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42730/0 (socket says 172.21.15.77:42730) 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 -- 172.21.15.77:0/4104961776 learned_addr learned my addr 172.21.15.77:0/4104961776 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226916a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51006/0 (socket says 172.21.15.77:51006) 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 -- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 msgr2=0x56554338c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x56554338c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 -- 172.21.15.77:0/4104961776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 msgr2=0x56554337e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:53.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 -- 172.21.15.77:0/4104961776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554337e6f0 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226916a640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2263fff640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f225009c020 tx=0x7f22500985c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbacf3554085e4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2243fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250026040 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f22500d0120 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2258040070 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f22500a4ee0 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226cfa6740 10 client.18579 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.289+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5655433fe540 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2258040070 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f22500a0170 con 0x56554337ab40 2024-09-15T22:24:53.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 10 client.18579 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 10 client.18579 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 20 client.18579 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5655433fec20 0x565543401040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 4 client.18579 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5655432e6d70 con 0x5655433fec20 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f2268969640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5655433fec20 0x565543401040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f2243fff640 10 client.18579 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.290+0000 7f2268969640 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5655433fec20 0x565543401040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f996b3a5c381b223 server_cookie=6905d6b7589169de in_seq=0 out_seq=0 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f2243fff640 1 -- 172.21.15.77:0/4104961776 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5655432e6d70 con 0x5655433fec20 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f2243fff640 10 client.18579 handle_command_reply: tid=0 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 client.18579 shutdown 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 msgr2=0x56554337db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f225009c020 tx=0x7f22500985c0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 shutdown_connections 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5655433fec20 0x565543401040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565543384370 0x56554337e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f2243fff640 0 client.18579 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56554337ab40 0x56554337db70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 --2- 172.21.15.77:0/4104961776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5655433795d0 0x56554338c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 >> 172.21.15.77:0/4104961776 conn(0x565543342310 msgr2=0x565543382810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 shutdown_connections 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 1 -- 172.21.15.77:0/4104961776 wait complete. 2024-09-15T22:24:53.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.291+0000 7f226cfa6740 20 client.18579 trim_cache size 0 max 16384 2024-09-15T22:24:53.307 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 msgr2=0x7f22540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 0x7f22540a58f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c005f90 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 msgr2=0x7f2264074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f225000e9f0 tx=0x7f225000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 shutdown_connections 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22540a3430 0x7f22540a58f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264106f20 0x7f2264074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22641027a0 0x7f2264074a30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 --2- 172.21.15.77:0/866358995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2264101df0 0x7f22640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 >> 172.21.15.77:0/866358995 conn(0x7f22640fd8b0 msgr2=0x7f22640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 shutdown_connections 2024-09-15T22:24:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:53.292+0000 7f226abcd640 1 -- 172.21.15.77:0/866358995 wait complete. 2024-09-15T22:24:53.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 941557 7.31 MB/sec execute 542 sec latency 38.395 ms 2024-09-15T22:24:54.349 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:24:54.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 943231 7.31 MB/sec execute 543 sec latency 13.821 ms 2024-09-15T22:24:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:54 smithi077 ceph-mon[30980]: pgmap v389: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:54 smithi196 ceph-mon[35109]: pgmap v389: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:54.608 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:54 smithi117 ceph-mon[35021]: pgmap v389: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:55.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 -- 172.21.15.77:0/219823423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 msgr2=0x7fcea4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 --2- 172.21.15.77:0/219823423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4100ee0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fce98009930 tx=0x7fce9802a420 comp rx=0 tx=0).stop 2024-09-15T22:24:55.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 -- 172.21.15.77:0/219823423 shutdown_connections 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 --2- 172.21.15.77:0/219823423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 --2- 172.21.15.77:0/219823423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4100ee0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 --2- 172.21.15.77:0/219823423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 0x7fcea4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.112+0000 7fceaa189640 1 -- 172.21.15.77:0/219823423 >> 172.21.15.77:0/219823423 conn(0x7fcea40fc7a0 msgr2=0x7fcea40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.113+0000 7fceaa189640 1 -- 172.21.15.77:0/219823423 shutdown_connections 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.113+0000 7fceaa189640 1 -- 172.21.15.77:0/219823423 wait complete. 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.113+0000 7fceaa189640 1 Processor -- start 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.113+0000 7fceaa189640 1 -- start start 2024-09-15T22:24:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 0x7fcea41a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4110e80 con 0x7fcea4100a80 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4110d00 con 0x7fcea4101420 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fceaa189640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4111000 con 0x7fcea4106a80 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42756/0 (socket says 172.21.15.77:42756) 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 -- 172.21.15.77:0/3878091030 learned_addr learned my addr 172.21.15.77:0/3878091030 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea3fff640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 0x7fcea41a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 -- 172.21.15.77:0/3878091030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 msgr2=0x7fcea41a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 0x7fcea41a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 -- 172.21.15.77:0/3878091030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 msgr2=0x7fcea4199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea4199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 -- 172.21.15.77:0/3878091030 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41a48f0 con 0x7fcea4100a80 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.114+0000 7fcea37fe640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9f0 tx=0x7fce9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.115+0000 7fcea0ff9640 1 -- 172.21.15.77:0/3878091030 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9400cd80 con 0x7fcea4100a80 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.115+0000 7fcea0ff9640 1 -- 172.21.15.77:0/3878091030 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce94005500 con 0x7fcea4100a80 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.115+0000 7fcea0ff9640 1 -- 172.21.15.77:0/3878091030 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94010690 con 0x7fcea4100a80 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.115+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4197120 con 0x7fcea4100a80 2024-09-15T22:24:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.115+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41975e0 con 0x7fcea4100a80 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.117+0000 7fcea0ff9640 1 -- 172.21.15.77:0/3878091030 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fce940046c0 con 0x7fcea4100a80 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.117+0000 7fceac562740 1 Processor -- start 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.117+0000 7fceac562740 1 -- start start 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259044a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 0x55c259051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c258faa2f0 con 0x55c2590445d0 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c258f51cf0 con 0x55c25904f370 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fceac562740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c258fa62b0 con 0x55c259045b40 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47502/0 (socket says 172.21.15.77:47502) 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259044a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 -- 172.21.15.77:0/1422144236 learned_addr learned my addr 172.21.15.77:0/1422144236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 0x55c259051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea0ff9640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 0x7fce8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 -- 172.21.15.77:0/1422144236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 msgr2=0x55c259051760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 0x55c259051760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 -- 172.21.15.77:0/1422144236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 msgr2=0x55c259044a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259044a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 -- 172.21.15.77:0/1422144236 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c258f363a0 con 0x55c259045b40 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 0x55c259051760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea0ff9640 1 -- 172.21.15.77:0/3878091030 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fce940d0460 con 0x7fcea4100a80 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 0x7fce8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.118+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce9c010510 tx=0x7fce9c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd2feb683556272 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fce89ffb640 1 -- 172.21.15.77:0/1422144236 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c01d400 con 0x55c259045b40 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fce89ffb640 1 -- 172.21.15.77:0/1422144236 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c0134c0 con 0x55c259045b40 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fce89ffb640 1 -- 172.21.15.77:0/1422144236 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c011900 con 0x55c259045b40 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fcea3fff640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 0x7fce8c0a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce9c021000 tx=0x7fce9c01e000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 -- 172.21.15.77:0/1422144236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 msgr2=0x55c259044090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce9c010510 tx=0x7fce9c009f90 comp rx=0 tx=0).stop 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 -- 172.21.15.77:0/1422144236 shutdown_connections 2024-09-15T22:24:55.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c25904f370 0x55c259051760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259044a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.119+0000 7fceac562740 1 --2- 172.21.15.77:0/1422144236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c259045b40 0x55c259044090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- 172.21.15.77:0/1422144236 >> 172.21.15.77:0/1422144236 conn(0x55c25900d310 msgr2=0x55c259040c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- 172.21.15.77:0/1422144236 shutdown_connections 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- 172.21.15.77:0/1422144236 wait complete. 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- start start 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 Processor -- start 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c25904bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c25904c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c259052070 con 0x55c2590445d0 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c258f65060 con 0x55c259045b40 2024-09-15T22:24:55.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.120+0000 7fceac562740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2590521f0 con 0x55c25904f370 2024-09-15T22:24:55.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fceac562740 10 client.0 _ll_register_callbacks cb 0x55c258cb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:55.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c25904bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42772/0 (socket says 172.21.15.77:42772) 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 -- 172.21.15.77:0/466129161 learned_addr learned my addr 172.21.15.77:0/466129161 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 -- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 msgr2=0x55c25904c5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c25904c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 -- 172.21.15.77:0/466129161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 msgr2=0x55c25904bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c25904bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 -- 172.21.15.77:0/466129161 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c258d1b3c0 con 0x55c2590445d0 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c25904bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fcea37fe640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fce94098e20 tx=0x7fce940d3c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffcb5c9d015bc83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fce82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9409bf90 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.121+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fce940a4c30 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c0134c0 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce940b2e60 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 10 client.18591 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c25905db40 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c0134c0 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fce9409b8a0 con 0x55c2590445d0 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 10 client.18591 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 10 client.18591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 20 client.18591 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c25905f000 0x55c259061410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 4 client.18591 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:55.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c258fb7860 con 0x55c25905f000 2024-09-15T22:24:55.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.122+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c25905f000 0x55c259061410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.123+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c25905f000 0x55c259061410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98220a46651b7379 server_cookie=fe4cdba3569ffebf in_seq=0 out_seq=0 2024-09-15T22:24:55.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.123+0000 7fce82ffd640 10 client.18591 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fce82ffd640 1 -- 172.21.15.77:0/466129161 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c258fb7860 con 0x55c25905f000 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fce82ffd640 10 client.18591 handle_command_reply: tid=0 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 client.18591 shutdown 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 msgr2=0x55c25904ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fce94098e20 tx=0x7fce940d3c30 comp rx=0 tx=0).stop 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 shutdown_connections 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c25905f000 0x55c259061410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c25904c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c25904bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 --2- 172.21.15.77:0/466129161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c25904ed40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 >> 172.21.15.77:0/466129161 conn(0x55c25900d310 msgr2=0x55c259046960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.124+0000 7fce82ffd640 0 client.18591 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.125+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 shutdown_connections 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.125+0000 7fceac562740 1 -- 172.21.15.77:0/466129161 wait complete. 2024-09-15T22:24:55.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.125+0000 7fceac562740 20 client.18591 trim_cache size 0 max 16384 2024-09-15T22:24:55.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 Processor -- start 2024-09-15T22:24:55.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 -- start start 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2590445d0 0x55c25905b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b1260 con 0x55c259045b40 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b2da0 con 0x55c2590445d0 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fceac562740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b1930 con 0x55c25904f370 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42774/0 (socket says 172.21.15.77:42774) 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fcea2ffd640 1 -- 172.21.15.77:0/1263401390 learned_addr learned my addr 172.21.15.77:0/1263401390 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.139+0000 7fcea37fe640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2590445d0 0x55c25905b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 -- 172.21.15.77:0/1263401390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2590445d0 msgr2=0x55c25905b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2590445d0 0x55c25905b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 -- 172.21.15.77:0/1263401390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 msgr2=0x55c259066c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 -- 172.21.15.77:0/1263401390 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25904c950 con 0x55c25904f370 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fcea3fff640 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590671c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce9c010510 tx=0x7fce9c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac207a1d5fe322c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fce897fa640 1 -- 172.21.15.77:0/1263401390 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0134c0 con 0x55c25904f370 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fce897fa640 1 -- 172.21.15.77:0/1263401390 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c003e50 con 0x55c25904f370 2024-09-15T22:24:55.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.140+0000 7fce897fa640 1 -- 172.21.15.77:0/1263401390 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0117e0 con 0x55c25904f370 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- 172.21.15.77:0/1263401390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 msgr2=0x55c2590671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590671c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce9c010510 tx=0x7fce9c009a20 comp rx=0 tx=0).stop 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- 172.21.15.77:0/1263401390 shutdown_connections 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590671c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c259045b40 0x55c259066c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 --2- 172.21.15.77:0/1263401390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2590445d0 0x55c25905b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- 172.21.15.77:0/1263401390 >> 172.21.15.77:0/1263401390 conn(0x55c25900d310 msgr2=0x55c25904a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- 172.21.15.77:0/1263401390 shutdown_connections 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- 172.21.15.77:0/1263401390 wait complete. 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.141+0000 7fceac562740 1 -- start start 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 Processor -- start 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c259048c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b26d0 con 0x55c2590445d0 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b2000 con 0x55c259045b40 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41af260 con 0x55c25904f370 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fceac562740 10 client.0 _ll_register_callbacks cb 0x7fce9400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42790/0 (socket says 172.21.15.77:42790) 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 -- 172.21.15.77:0/4220497349 learned_addr learned my addr 172.21.15.77:0/4220497349 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea3fff640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c259048c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 -- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 msgr2=0x55c2590491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 -- 172.21.15.77:0/4220497349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 msgr2=0x55c259048c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.142+0000 7fcea37fe640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c259048c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fcea37fe640 1 -- 172.21.15.77:0/4220497349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2590499a0 con 0x55c2590445d0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c259048c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fcea37fe640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fce940d3ca0 tx=0x7fce940d03e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a152e4338cbf040e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9409bdb0 con 0x55c2590445d0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fce940d3cd0 con 0x55c2590445d0 2024-09-15T22:24:55.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c003e50 con 0x55c2590445d0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce9409c450 con 0x55c2590445d0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fceac562740 10 client.18597 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2590c9c00 con 0x55c2590445d0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c003e50 con 0x55c2590445d0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fce9409b310 con 0x55c2590445d0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fceac562740 10 client.18597 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.143+0000 7fceac562740 10 client.18597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fceac562740 20 client.18597 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c2590ca320 0x55c2590cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fceac562740 4 client.18597 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c258fb1d70 con 0x55c2590ca320 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c2590ca320 0x55c2590cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fce82ffd640 10 client.18597 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:55.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.144+0000 7fcea2ffd640 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c2590ca320 0x55c2590cc710 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5600d68aca0883ff server_cookie=98b14c44dd49c31d in_seq=0 out_seq=0 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.145+0000 7fce82ffd640 1 -- 172.21.15.77:0/4220497349 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c258fb1d70 con 0x55c2590ca320 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.145+0000 7fce82ffd640 10 client.18597 handle_command_reply: tid=0 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.145+0000 7fceac562740 1 client.18597 shutdown 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 msgr2=0x55c259057b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fce940d3ca0 tx=0x7fce940d03e0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 shutdown_connections 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c2590ca320 0x55c2590cc710 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c25904f370 0x55c2590491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c259045b40 0x55c259048c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 --2- 172.21.15.77:0/4220497349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2590445d0 0x55c259057b70 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 >> 172.21.15.77:0/4220497349 conn(0x55c25900d310 msgr2=0x55c259046110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fce82ffd640 0 client.18597 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 shutdown_connections 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 1 -- 172.21.15.77:0/4220497349 wait complete. 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.146+0000 7fceac562740 20 client.18597 trim_cache size 0 max 16384 2024-09-15T22:24:55.153 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 msgr2=0x7fce8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 0x7fce8c0a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce9c021000 tx=0x7fce9c01e000 comp rx=0 tx=0).stop 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 msgr2=0x7fcea4199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9f0 tx=0x7fce9400edf0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 shutdown_connections 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fce8c0a33b0 0x7fce8c0a5870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcea4106a80 0x7fcea41a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcea4101420 0x7fcea4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 --2- 172.21.15.77:0/3878091030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcea4100a80 0x7fcea4199330 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 >> 172.21.15.77:0/3878091030 conn(0x7fcea40fc7a0 msgr2=0x7fcea40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 shutdown_connections 2024-09-15T22:24:55.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:55.147+0000 7fceaa189640 1 -- 172.21.15.77:0/3878091030 wait complete. 2024-09-15T22:24:55.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 945478 7.32 MB/sec execute 544 sec latency 28.446 ms 2024-09-15T22:24:56.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:56 smithi077 ceph-mon[30980]: pgmap v390: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:56.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:24:56.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 947514 7.32 MB/sec execute 545 sec latency 17.700 ms 2024-09-15T22:24:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:56 smithi196 ceph-mon[35109]: pgmap v390: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:56 smithi117 ceph-mon[35021]: pgmap v390: 129 pgs: 129 active+clean; 533 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-09-15T22:24:56.747 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.409+0000 7f31ea74d640 1 -- 172.21.15.77:0/687029119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 msgr2=0x7f31e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.409+0000 7f31ea74d640 1 --2- 172.21.15.77:0/687029119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e4101200 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009900 tx=0x7f31d402a480 comp rx=0 tx=0).stop 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 -- 172.21.15.77:0/687029119 shutdown_connections 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 --2- 172.21.15.77:0/687029119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102ef0 0x7f31e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 --2- 172.21.15.77:0/687029119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102570 0x7f31e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 --2- 172.21.15.77:0/687029119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e4101200 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 -- 172.21.15.77:0/687029119 >> 172.21.15.77:0/687029119 conn(0x7f31e40fc7d0 msgr2=0x7f31e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 -- 172.21.15.77:0/687029119 shutdown_connections 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 -- 172.21.15.77:0/687029119 wait complete. 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 Processor -- start 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.410+0000 7f31ea74d640 1 -- start start 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102570 0x7f31e41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102ef0 0x7f31e419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410fe40 con 0x7f31e4100e20 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410fcc0 con 0x7f31e4102ef0 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410ffc0 con 0x7f31e4102570 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42810/0 (socket says 172.21.15.77:42810) 2024-09-15T22:24:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 -- 172.21.15.77:0/105282112 learned_addr learned my addr 172.21.15.77:0/105282112 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e37fe640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102570 0x7f31e41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e8cea640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102ef0 0x7f31e419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 -- 172.21.15.77:0/105282112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102570 msgr2=0x7f31e41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102570 0x7f31e41a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 -- 172.21.15.77:0/105282112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102ef0 msgr2=0x7f31e419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102ef0 0x7f31e419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 -- 172.21.15.77:0/105282112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e419acf0 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e3fff640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f31d4006010 tx=0x7f31d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e17fa640 1 -- 172.21.15.77:0/105282112 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d403f070 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e419af80 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e17fa640 1 -- 172.21.15.77:0/105282112 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d4031c30 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.411+0000 7f31e17fa640 1 -- 172.21.15.77:0/105282112 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4035030 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.412+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a72d0 con 0x7f31e4100e20 2024-09-15T22:24:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31e17fa640 1 -- 172.21.15.77:0/105282112 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f31d4042070 con 0x7f31e4100e20 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 Processor -- start 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 -- start start 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64addf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64addf5d0 0x55d64addfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d64ad452f0 con 0x55d64adea370 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d64aceccf0 con 0x55d64ade0b40 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31ecb26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d64ad412b0 con 0x55d64addf5d0 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.414+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64addf5d0 0x55d64addfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42824/0 (socket says 172.21.15.77:42824) 2024-09-15T22:24:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 -- 172.21.15.77:0/2403750556 learned_addr learned my addr 172.21.15.77:0/2403750556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e8cea640 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64addf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e17fa640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 0x7f31cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 -- 172.21.15.77:0/2403750556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64addf5d0 msgr2=0x55d64addfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64addf5d0 0x55d64addfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 -- 172.21.15.77:0/2403750556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 msgr2=0x55d64addf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64addf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 -- 172.21.15.77:0/2403750556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64acd13a0 con 0x55d64adea370 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e8cea640 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64addf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e8cea640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 0x7f31cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e17fa640 1 -- 172.21.15.77:0/105282112 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f31d40f0400 con 0x7f31e4100e20 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31e37fe640 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009c60 tx=0x7f31d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8bb829ffa17b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31ca7fc640 1 -- 172.21.15.77:0/2403750556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8004230 con 0x55d64adea370 2024-09-15T22:24:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31ca7fc640 1 -- 172.21.15.77:0/2403750556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d80043d0 con 0x55d64adea370 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.415+0000 7f31ca7fc640 1 -- 172.21.15.77:0/2403750556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8004de0 con 0x55d64adea370 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- 172.21.15.77:0/2403750556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 msgr2=0x55d64adec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009c60 tx=0x7f31d8011810 comp rx=0 tx=0).stop 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31e8cea640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 0x7f31cc0a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b3c0 tx=0x7f31dc004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- 172.21.15.77:0/2403750556 shutdown_connections 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64adec760 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64addf5d0 0x55d64addfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2403750556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64addf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- 172.21.15.77:0/2403750556 >> 172.21.15.77:0/2403750556 conn(0x55d64ada8310 msgr2=0x55d64addbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- 172.21.15.77:0/2403750556 shutdown_connections 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- 172.21.15.77:0/2403750556 wait complete. 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.416+0000 7f31ecb26740 1 -- start start 2024-09-15T22:24:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 Processor -- start 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64ade9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64ade0b40 0x55d64ade6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d64aded070 con 0x55d64adea370 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d64ad00060 con 0x55d64addf5d0 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d64aded1f0 con 0x55d64ade0b40 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31ecb26740 10 client.0 _ll_register_callbacks cb 0x55d64aa538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42834/0 (socket says 172.21.15.77:42834) 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 -- 172.21.15.77:0/18877304 learned_addr learned my addr 172.21.15.77:0/18877304 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e37fe640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64ade0b40 0x55d64ade6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e3fff640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64ade9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 -- 172.21.15.77:0/18877304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64ade0b40 msgr2=0x55d64ade6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64ade0b40 0x55d64ade6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 -- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 msgr2=0x55d64ade9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64ade9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 -- 172.21.15.77:0/18877304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64aab63c0 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31e8cea640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f31dc017df0 tx=0x7f31dc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7682f1ebc3a77407 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.417+0000 7f31c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc02a060 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f31dc022320 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31d80043d0 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31dc031050 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 10 client.18609 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d64adf8ae0 con 0x55d64adea370 2024-09-15T22:24:57.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d80043d0 con 0x55d64adea370 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31dc023380 con 0x55d64adea370 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 10 client.18609 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 10 client.18609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 20 client.18609 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64adf9fa0 0x55d64adfc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 4 client.18609 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64ad52860 con 0x55d64adf9fa0 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.418+0000 7f31e3fff640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64adf9fa0 0x55d64adfc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.419+0000 7f31e3fff640 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64adf9fa0 0x55d64adfc3b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71b70c520a394297 server_cookie=6ff7eb9fbb3e75b3 in_seq=0 out_seq=0 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.419+0000 7f31c37fe640 10 client.18609 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.419+0000 7f31c37fe640 1 -- 172.21.15.77:0/18877304 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d64ad52860 con 0x55d64adf9fa0 2024-09-15T22:24:57.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.419+0000 7f31c37fe640 10 client.18609 handle_command_reply: tid=0 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.419+0000 7f31ecb26740 1 client.18609 shutdown 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 msgr2=0x55d64ade74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f31dc017df0 tx=0x7f31dc00d040 comp rx=0 tx=0).stop 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 shutdown_connections 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64adf9fa0 0x55d64adfc3b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64adea370 0x55d64ade74d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64ade0b40 0x55d64ade6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 --2- 172.21.15.77:0/18877304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64ade9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 >> 172.21.15.77:0/18877304 conn(0x55d64ada8310 msgr2=0x55d64ade1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31c37fe640 0 client.18609 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 shutdown_connections 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 1 -- 172.21.15.77:0/18877304 wait complete. 2024-09-15T22:24:57.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.420+0000 7f31ecb26740 20 client.18609 trim_cache size 0 max 16384 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 Processor -- start 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 -- start start 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64ade64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64ae01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41b12b0 con 0x55d64addf5d0 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41b2df0 con 0x55d64ade0b40 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31ecb26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41b1980 con 0x55d64adea370 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42848/0 (socket says 172.21.15.77:42848) 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31e3fff640 1 -- 172.21.15.77:0/1273601645 learned_addr learned my addr 172.21.15.77:0/1273601645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.433+0000 7f31e37fe640 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64ade64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e8cea640 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64ae01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 -- 172.21.15.77:0/1273601645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 msgr2=0x55d64ae01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64ae01c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 -- 172.21.15.77:0/1273601645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 msgr2=0x55d64ade64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64ade64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 -- 172.21.15.77:0/1273601645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64ade54c0 con 0x55d64addf5d0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31e3fff640 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f31d40ed870 tx=0x7f31d40b87d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9862b8a41107b342 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31c9ffb640 1 -- 172.21.15.77:0/1273601645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4049040 con 0x55d64addf5d0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31c9ffb640 1 -- 172.21.15.77:0/1273601645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d40c4fc0 con 0x55d64addf5d0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.434+0000 7f31c9ffb640 1 -- 172.21.15.77:0/1273601645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b8a70 con 0x55d64addf5d0 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- 172.21.15.77:0/1273601645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 msgr2=0x55d64adf6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f31d40ed870 tx=0x7f31d40b87d0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- 172.21.15.77:0/1273601645 shutdown_connections 2024-09-15T22:24:57.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64ae01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64ade0b40 0x55d64ade64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 --2- 172.21.15.77:0/1273601645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64addf5d0 0x55d64adf6310 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- 172.21.15.77:0/1273601645 >> 172.21.15.77:0/1273601645 conn(0x55d64ada8310 msgr2=0x55d64ade5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- 172.21.15.77:0/1273601645 shutdown_connections 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- 172.21.15.77:0/1273601645 wait complete. 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.435+0000 7f31ecb26740 1 -- start start 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 Processor -- start 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64adf2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 0x55d64adee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64adee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41b2720 con 0x55d64ade0b40 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41b2050 con 0x55d64addf5d0 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41af2b0 con 0x55d64adea370 2024-09-15T22:24:57.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31ecb26740 10 client.0 _ll_register_callbacks cb 0x7f31dc003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 0x55d64adee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64adf2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 0x55d64adee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42850/0 (socket says 172.21.15.77:42850) 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e37fe640 1 -- 172.21.15.77:0/2664755772 learned_addr learned my addr 172.21.15.77:0/2664755772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e8cea640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64adee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 -- 172.21.15.77:0/2664755772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 msgr2=0x55d64adee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64adee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 -- 172.21.15.77:0/2664755772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 msgr2=0x55d64adee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 0x55d64adee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.436+0000 7f31e3fff640 1 -- 172.21.15.77:0/2664755772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64ae63c50 con 0x55d64addf5d0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31e3fff640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64adf2da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31d40bc100 tx=0x7f31d4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b99a41d7986f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:24:57.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4049070 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f31d403b460 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31d40c4fc0 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31d40da050 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 10 client.27593 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d64ae64e30 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d40c4fc0 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31d40eda40 con 0x55d64addf5d0 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 10 client.27593 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 10 client.27593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 20 client.27593 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64ae65450 0x55d64ae67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 4 client.27593 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:57.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d64ad4cd70 con 0x55d64ae65450 2024-09-15T22:24:57.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.437+0000 7f31e37fe640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64ae65450 0x55d64ae67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31c37fe640 10 client.27593 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31e37fe640 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64ae65450 0x55d64ae67840 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43d5091f1f40cc23 server_cookie=e9512857791e3141 in_seq=0 out_seq=0 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31c37fe640 1 -- 172.21.15.77:0/2664755772 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d64ad4cd70 con 0x55d64ae65450 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31c37fe640 10 client.27593 handle_command_reply: tid=0 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 client.27593 shutdown 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 msgr2=0x55d64adf2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64adf2da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31d40bc100 tx=0x7f31d4009650 comp rx=0 tx=0).stop 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 shutdown_connections 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55d64ae65450 0x55d64ae67840 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d64adea370 0x55d64adee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31c37fe640 0 client.27593 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d64ade0b40 0x55d64adee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 --2- 172.21.15.77:0/2664755772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d64addf5d0 0x55d64adf2da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 >> 172.21.15.77:0/2664755772 conn(0x55d64ada8310 msgr2=0x55d64ade1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 shutdown_connections 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.438+0000 7f31ecb26740 1 -- 172.21.15.77:0/2664755772 wait complete. 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ecb26740 20 client.27593 trim_cache size 0 max 16384 2024-09-15T22:24:57.457 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 msgr2=0x7f31cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 0x7f31cc0a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00b3c0 tx=0x7f31dc004a40 comp rx=0 tx=0).stop 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 msgr2=0x7f31e41a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f31d4006010 tx=0x7f31d4003c10 comp rx=0 tx=0).stop 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 shutdown_connections 2024-09-15T22:24:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f31cc0a33b0 0x7f31cc0a5870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e4102ef0 0x7f31e419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31e4102570 0x7f31e41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 --2- 172.21.15.77:0/105282112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31e4100e20 0x7f31e41a0400 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 >> 172.21.15.77:0/105282112 conn(0x7f31e40fc7d0 msgr2=0x7f31e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 shutdown_connections 2024-09-15T22:24:57.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:57.439+0000 7f31ea74d640 1 -- 172.21.15.77:0/105282112 wait complete. 2024-09-15T22:24:57.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 949338 7.33 MB/sec execute 546 sec latency 39.794 ms 2024-09-15T22:24:58.139 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:24:58.442 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:58.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 951993 7.34 MB/sec execute 547 sec latency 16.643 ms 2024-09-15T22:24:58.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:24:58 smithi077 ceph-mon[30980]: pgmap v391: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-09-15T22:24:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:24:58 smithi196 ceph-mon[35109]: pgmap v391: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-09-15T22:24:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:24:58 smithi117 ceph-mon[35021]: pgmap v391: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 203 op/s 2024-09-15T22:24:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 -- 172.21.15.77:0/3138381980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360100e20 msgr2=0x7f0360101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 --2- 172.21.15.77:0/3138381980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360100e20 0x7f0360101200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0350009930 tx=0x7f035002a4d0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 -- 172.21.15.77:0/3138381980 shutdown_connections 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 --2- 172.21.15.77:0/3138381980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 --2- 172.21.15.77:0/3138381980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360102570 0x7f03601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 --2- 172.21.15.77:0/3138381980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360100e20 0x7f0360101200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 -- 172.21.15.77:0/3138381980 >> 172.21.15.77:0/3138381980 conn(0x7f03600fc7d0 msgr2=0x7f03600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 -- 172.21.15.77:0/3138381980 shutdown_connections 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.922+0000 7f03660eb640 1 -- 172.21.15.77:0/3138381980 wait complete. 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 Processor -- start 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 -- start start 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 0x7f036019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360102570 0x7f036019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0360110f80 con 0x7f0360102ef0 2024-09-15T22:24:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0360110e00 con 0x7f0360102570 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f03660eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0360111100 con 0x7f0360100e20 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f035ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 0x7f036019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 0x7f036019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47584/0 (socket says 172.21.15.77:47584) 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.923+0000 7f035f7fe640 1 -- 172.21.15.77:0/2414080323 learned_addr learned my addr 172.21.15.77:0/2414080323 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f0357fff640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360102570 0x7f036019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 -- 172.21.15.77:0/2414080323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 msgr2=0x7f036019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 0x7f036019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 -- 172.21.15.77:0/2414080323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360102570 msgr2=0x7f036019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360102570 0x7f036019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 -- 172.21.15.77:0/2414080323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036019f6f0 con 0x7f0360102ef0 2024-09-15T22:24:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035ffff640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036019f010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f035800ec50 tx=0x7f035800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035d7fa640 1 -- 172.21.15.77:0/2414080323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358003b70 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035d7fa640 1 -- 172.21.15.77:0/2414080323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0358003d10 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036019f9e0 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f035d7fa640 1 -- 172.21.15.77:0/2414080323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358010890 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.924+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601a72c0 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.926+0000 7f035d7fa640 1 -- 172.21.15.77:0/2414080323 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0358003fc0 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f035d7fa640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 0x7f034c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 Processor -- start 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 -- start start 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f035f7fe640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 0x7f034c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f035d7fa640 1 -- 172.21.15.77:0/2414080323 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f03580d0740 con 0x7f0360102ef0 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d471970b40 0x55d47196f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4718d52f0 con 0x55d47197a370 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d47187ccf0 con 0x55d471970b40 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.927+0000 7f03684c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4718d12b0 con 0x55d47196f5d0 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42878/0 (socket says 172.21.15.77:42878) 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47588/0 (socket says 172.21.15.77:47588) 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035ffff640 1 -- 172.21.15.77:0/449651982 learned_addr learned my addr 172.21.15.77:0/449651982 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035f7fe640 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d471970b40 0x55d47196f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 -- 172.21.15.77:0/449651982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d471970b40 msgr2=0x55d47196f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d471970b40 0x55d47196f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035f7fe640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 0x7f034c0a58f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0350000c00 tx=0x7f03500315b0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 -- 172.21.15.77:0/449651982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 msgr2=0x55d47197c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 -- 172.21.15.77:0/449651982 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4718613a0 con 0x55d47196f5d0 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f035ffff640 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0357fff640 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0348009b40 tx=0x7f03480113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bd19fa8cf81da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0355ffb640 1 -- 172.21.15.77:0/449651982 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348011e10 con 0x55d47196f5d0 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0355ffb640 1 -- 172.21.15.77:0/449651982 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0348012930 con 0x55d47196f5d0 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.928+0000 7f0355ffb640 1 -- 172.21.15.77:0/449651982 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034800fac0 con 0x55d47196f5d0 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- 172.21.15.77:0/449651982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 msgr2=0x55d47196fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0348009b40 tx=0x7f03480113e0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- 172.21.15.77:0/449651982 shutdown_connections 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47197a370 0x55d47197c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d47196f5d0 0x55d47196fa10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 --2- 172.21.15.77:0/449651982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d471970b40 0x55d47196f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- 172.21.15.77:0/449651982 >> 172.21.15.77:0/449651982 conn(0x55d471938310 msgr2=0x55d47196bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- 172.21.15.77:0/449651982 shutdown_connections 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- 172.21.15.77:0/449651982 wait complete. 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.929+0000 7f03684c4740 1 -- start start 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 Processor -- start 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471976fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471977570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d47197d070 con 0x55d47196f5d0 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d471890060 con 0x55d47197a370 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d47197d1f0 con 0x55d471970b40 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f03684c4740 10 client.0 _ll_register_callbacks cb 0x55d4715e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f0357fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471976fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42886/0 (socket says 172.21.15.77:42886) 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471977570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 -- 172.21.15.77:0/3104204857 learned_addr learned my addr 172.21.15.77:0/3104204857 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 -- 172.21.15.77:0/3104204857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 msgr2=0x55d471976fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471976fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 -- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 msgr2=0x55d471977570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471977570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 -- 172.21.15.77:0/3104204857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4716463c0 con 0x55d47196f5d0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f0357fff640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471976fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035ffff640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471977570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0350005ec0 tx=0x7f035002a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d8d44f877ed01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.930+0000 7f033effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350035040 con 0x55d47196f5d0 2024-09-15T22:24:58.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f0350003ac0 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0348012930 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0350003cc0 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 10 client.18627 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d471988ac0 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0348012930 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f035003d570 con 0x55d47196f5d0 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 10 client.18627 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 20 client.18627 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d47198c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 4 client.18627 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4718e2860 con 0x55d471989f80 2024-09-15T22:24:58.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.931+0000 7f0357fff640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d47198c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.932+0000 7f0357fff640 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d47198c3b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b27d0763d177 server_cookie=84d31971eb0cc318 in_seq=0 out_seq=0 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.932+0000 7f033effd640 10 client.18627 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.932+0000 7f033effd640 1 -- 172.21.15.77:0/3104204857 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4718e2860 con 0x55d471989f80 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.932+0000 7f033effd640 10 client.18627 handle_command_reply: tid=0 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.932+0000 7f03684c4740 1 client.18627 shutdown 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 msgr2=0x55d471979d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0350005ec0 tx=0x7f035002a4d0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 shutdown_connections 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d47198c3b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f033effd640 0 client.18627 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471977570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471976fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 --2- 172.21.15.77:0/3104204857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471979d20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 >> 172.21.15.77:0/3104204857 conn(0x55d471938310 msgr2=0x55d471971960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 shutdown_connections 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 1 -- 172.21.15.77:0/3104204857 wait complete. 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.933+0000 7f03684c4740 20 client.18627 trim_cache size 0 max 16384 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.945+0000 7f03684c4740 1 Processor -- start 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.945+0000 7f03684c4740 1 -- start start 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471972120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471972660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f03601b1110 con 0x55d47196f5d0 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03601b2c50 con 0x55d47197a370 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03684c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03601b17e0 con 0x55d471970b40 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42892/0 (socket says 172.21.15.77:42892) 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 -- 172.21.15.77:0/3645687284 learned_addr learned my addr 172.21.15.77:0/3645687284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f0357fff640 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471972120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035ffff640 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471972660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 -- 172.21.15.77:0/3645687284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 msgr2=0x55d471972120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471972120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 -- 172.21.15.77:0/3645687284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 msgr2=0x55d471972660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471972660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 -- 172.21.15.77:0/3645687284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d471972ba0 con 0x55d47196f5d0 2024-09-15T22:24:58.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f035f7fe640 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f035003d940 tx=0x7f0350005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91804370b19870cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03557fa640 1 -- 172.21.15.77:0/3645687284 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350064070 con 0x55d47196f5d0 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03557fa640 1 -- 172.21.15.77:0/3645687284 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f035004d070 con 0x55d47196f5d0 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.946+0000 7f03557fa640 1 -- 172.21.15.77:0/3645687284 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035003d6b0 con 0x55d47196f5d0 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- 172.21.15.77:0/3645687284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 msgr2=0x55d47196abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f035003d940 tx=0x7f0350005b70 comp rx=0 tx=0).stop 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- 172.21.15.77:0/3645687284 shutdown_connections 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d471972660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d471972120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 --2- 172.21.15.77:0/3645687284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d47196abf0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- 172.21.15.77:0/3645687284 >> 172.21.15.77:0/3645687284 conn(0x55d471938310 msgr2=0x55d47196a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- 172.21.15.77:0/3645687284 shutdown_connections 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- 172.21.15.77:0/3645687284 wait complete. 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.947+0000 7f03684c4740 1 -- start start 2024-09-15T22:24:58.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 Processor -- start 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d47197de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d47197e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f03601b2580 con 0x55d47196f5d0 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03601b1eb0 con 0x55d47197a370 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03601af110 con 0x55d471970b40 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f03684c4740 10 client.0 _ll_register_callbacks cb 0x7f0350009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42896/0 (socket says 172.21.15.77:42896) 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 -- 172.21.15.77:0/1202415501 learned_addr learned my addr 172.21.15.77:0/1202415501 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f0357fff640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d47197de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035ffff640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d47197e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 -- 172.21.15.77:0/1202415501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 msgr2=0x55d47197de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d47197de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 -- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 msgr2=0x55d47197e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d47197e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 -- 172.21.15.77:0/1202415501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47197eb60 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f035f7fe640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0350062040 tx=0x7f0350005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef1c9d34c2c4517 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.948+0000 7f033effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350064040 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f035003d450 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f035004d070 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f035003d650 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 10 client.18633 fetch_fsmap learned FSMap version 34 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d4719f4ba0 con 0x55d47196f5d0 2024-09-15T22:24:58.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f035004d070 con 0x55d47196f5d0 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0350067030 con 0x55d47196f5d0 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 10 client.18633 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 10 client.18633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 20 client.18633 populate_metadata read hostname 'smithi077' 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d4719f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 4 client.18633 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4718dcd70 con 0x55d471989f80 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.949+0000 7f0357fff640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d4719f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f0357fff640 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d4719f7090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34692cc8f8559a0 server_cookie=230b1c6fd0c3edf9 in_seq=0 out_seq=0 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f033effd640 10 client.18633 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f033effd640 1 -- 172.21.15.77:0/1202415501 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4718dcd70 con 0x55d471989f80 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f033effd640 10 client.18633 handle_command_reply: tid=0 2024-09-15T22:24:58.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 client.18633 shutdown 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 msgr2=0x55d471982b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0350062040 tx=0x7f0350005970 comp rx=0 tx=0).stop 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 shutdown_connections 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d471989f80 0x55d4719f7090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d47197a370 0x55d47197e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d471970b40 0x55d47197de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 --2- 172.21.15.77:0/1202415501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d47196f5d0 0x55d471982b10 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 >> 172.21.15.77:0/1202415501 conn(0x55d471938310 msgr2=0x55d47196ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f033effd640 0 client.18633 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 shutdown_connections 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 1 -- 172.21.15.77:0/1202415501 wait complete. 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.950+0000 7f03684c4740 20 client.18633 trim_cache size 0 max 16384 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 msgr2=0x7f034c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 0x7f034c0a58f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0350000c00 tx=0x7f03500315b0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 msgr2=0x7f036019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036019f010 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f035800ec50 tx=0x7f035800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 shutdown_connections 2024-09-15T22:24:58.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f034c0a3430 0x7f034c0a58f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0360102ef0 0x7f036019f010 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0360102570 0x7f036019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 --2- 172.21.15.77:0/2414080323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360100e20 0x7f036019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 >> 172.21.15.77:0/2414080323 conn(0x7f03600fc7d0 msgr2=0x7f036010c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 shutdown_connections 2024-09-15T22:24:58.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:24:58.951+0000 7f03660eb640 1 -- 172.21.15.77:0/2414080323 wait complete. 2024-09-15T22:24:59.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 953702 7.34 MB/sec execute 548 sec latency 46.833 ms 2024-09-15T22:25:00.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 955812 7.35 MB/sec execute 549 sec latency 43.930 ms 2024-09-15T22:25:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:00 smithi077 ceph-mon[30980]: pgmap v392: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-15T22:25:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:00 smithi196 ceph-mon[35109]: pgmap v392: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-15T22:25:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:00 smithi117 ceph-mon[35021]: pgmap v392: 129 pgs: 129 active+clean; 548 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-15T22:25:00.898 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:25:00.899 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:25:01.181 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:01.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 956760 7.35 MB/sec execute 550 sec latency 59.616 ms 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 -- 172.21.15.77:0/177866124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944100e20 msgr2=0x7fa944101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 --2- 172.21.15.77:0/177866124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944100e20 0x7fa944101200 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa934009900 tx=0x7fa93402a480 comp rx=0 tx=0).stop 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 -- 172.21.15.77:0/177866124 shutdown_connections 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 --2- 172.21.15.77:0/177866124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944102ef0 0x7fa94410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 --2- 172.21.15.77:0/177866124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102570 0x7fa9441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 --2- 172.21.15.77:0/177866124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944100e20 0x7fa944101200 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.889+0000 7fa94973a640 1 -- 172.21.15.77:0/177866124 >> 172.21.15.77:0/177866124 conn(0x7fa9440fc7d0 msgr2=0x7fa9440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.890+0000 7fa94973a640 1 -- 172.21.15.77:0/177866124 shutdown_connections 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.890+0000 7fa94973a640 1 -- 172.21.15.77:0/177866124 wait complete. 2024-09-15T22:25:01.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.890+0000 7fa94973a640 1 Processor -- start 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 -- start start 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 0x7fa9441993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 0x7fa944199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 0x7fa9441a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa944110f80 con 0x7fa944102570 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa944110e00 con 0x7fa944100e20 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa94973a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa944111100 con 0x7fa944102ef0 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 0x7fa944199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 0x7fa9441993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 0x7fa944199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50882/0 (socket says 172.21.15.77:50882) 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa9427fc640 1 -- 172.21.15.77:0/856781129 learned_addr learned my addr 172.21.15.77:0/856781129 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa9437fe640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 0x7fa9441a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 -- 172.21.15.77:0/856781129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 msgr2=0x7fa9441a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 0x7fa9441a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 -- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 msgr2=0x7fa944199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 0x7fa944199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 -- 172.21.15.77:0/856781129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9441a4990 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa9437fe640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 0x7fa9441a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.891+0000 7fa942ffd640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 0x7fa9441993d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa934005940 tx=0x7fa934003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.892+0000 7fa92bfff640 1 -- 172.21.15.77:0/856781129 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93403f070 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.892+0000 7fa92bfff640 1 -- 172.21.15.77:0/856781129 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa934031c30 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.892+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9441a4b60 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.892+0000 7fa92bfff640 1 -- 172.21.15.77:0/856781129 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934035030 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.892+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9441a4fb0 con 0x7fa944100e20 2024-09-15T22:25:01.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.894+0000 7fa92bfff640 1 -- 172.21.15.77:0/856781129 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa934042070 con 0x7fa944100e20 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 Processor -- start 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa92bfff640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 0x7fa92c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 -- start start 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa9437fe640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 0x7fa92c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa92bfff640 1 -- 172.21.15.77:0/856781129 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fa9340f0700 con 0x7fa944100e20 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448bb480 0x564b448c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448c5d40 0x564b448c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b44820280 con 0x564b448bab60 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b447c7b10 con 0x564b448c5d40 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa94bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b4481c240 con 0x564b448bb480 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50888/0 (socket says 172.21.15.77:50888) 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 -- 172.21.15.77:0/3765608710 learned_addr learned my addr 172.21.15.77:0/3765608710 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa9427fc640 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448bb480 0x564b448c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa9437fe640 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448c5d40 0x564b448c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 -- 172.21.15.77:0/3765608710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448bb480 msgr2=0x564b448c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448bb480 0x564b448c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 -- 172.21.15.77:0/3765608710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448c5d40 msgr2=0x564b448c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448c5d40 0x564b448c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa942ffd640 1 -- 172.21.15.77:0/3765608710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b447d41f0 con 0x564b448bab60 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.895+0000 7fa9437fe640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 0x7fa92c0a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c0107a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa942ffd640 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa9340b8fc0 tx=0x7fa934000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e631f8a81cb5e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa9297fa640 1 -- 172.21.15.77:0/3765608710 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9340d5070 con 0x564b448bab60 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa9297fa640 1 -- 172.21.15.77:0/3765608710 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9340bc5f0 con 0x564b448bab60 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa9297fa640 1 -- 172.21.15.77:0/3765608710 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934049070 con 0x564b448bab60 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 -- 172.21.15.77:0/3765608710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 msgr2=0x564b448baf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa9340b8fc0 tx=0x7fa934000bc0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 -- 172.21.15.77:0/3765608710 shutdown_connections 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448c5d40 0x564b448c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448bb480 0x564b448c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 --2- 172.21.15.77:0/3765608710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448baf40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.896+0000 7fa94bb13740 1 -- 172.21.15.77:0/3765608710 >> 172.21.15.77:0/3765608710 conn(0x564b448b70f0 msgr2=0x564b448b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- 172.21.15.77:0/3765608710 shutdown_connections 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- 172.21.15.77:0/3765608710 wait complete. 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- start start 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 Processor -- start 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b448c86d0 con 0x564b448bb480 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b447daf10 con 0x564b448bab60 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b448c8850 con 0x564b448c5d40 2024-09-15T22:25:01.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa94bb13740 10 client.0 _ll_register_callbacks cb 0x564b447a3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50894/0 (socket says 172.21.15.77:50894) 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa9427fc640 1 -- 172.21.15.77:0/2150702585 learned_addr learned my addr 172.21.15.77:0/2150702585 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.897+0000 7fa9437fe640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 -- 172.21.15.77:0/2150702585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 msgr2=0x564b448c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 -- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 msgr2=0x564b448c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 -- 172.21.15.77:0/2150702585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b445916a0 con 0x564b448bb480 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa938012840 tx=0x7fa938012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156eda6cebd14225 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93800f920 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fa93800fac0 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9340bc5f0 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa93800fd00 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa94bb13740 10 client.18645 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b448d4320 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.898+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9340bc5f0 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa9380104c0 con 0x564b448bb480 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 10 client.18645 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 20 client.18645 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b448d57e0 0x564b448d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 4 client.18645 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b4482d7f0 con 0x564b448d57e0 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa942ffd640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b448d57e0 0x564b448d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa926ff5640 10 client.18645 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.899+0000 7fa942ffd640 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b448d57e0 0x564b448d7c30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e36521107b414cc server_cookie=4a6ccfe3b1d6d533 in_seq=0 out_seq=0 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa926ff5640 1 -- 172.21.15.77:0/2150702585 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b4482d7f0 con 0x564b448d57e0 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa926ff5640 10 client.18645 handle_command_reply: tid=0 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 client.18645 shutdown 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 msgr2=0x564b448c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa938012840 tx=0x7fa938012d00 comp rx=0 tx=0).stop 2024-09-15T22:25:01.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 shutdown_connections 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b448d57e0 0x564b448d7c30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c3190 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2150702585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 >> 172.21.15.77:0/2150702585 conn(0x564b448b70f0 msgr2=0x564b448b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa926ff5640 0 client.18645 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 shutdown_connections 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.900+0000 7fa94bb13740 1 -- 172.21.15.77:0/2150702585 wait complete. 2024-09-15T22:25:01.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.901+0000 7fa94bb13740 20 client.18645 trim_cache size 0 max 16384 2024-09-15T22:25:01.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 Processor -- start 2024-09-15T22:25:01.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 -- start start 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441aeff0 con 0x564b448bab60 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441b0b30 con 0x564b448bb480 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa94bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441af6c0 con 0x564b448c5d40 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:46706/0 (socket says 172.21.15.77:46706) 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448bd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50900/0 (socket says 172.21.15.77:50900) 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa9427fc640 1 -- 172.21.15.77:0/2375420409 learned_addr learned my addr 172.21.15.77:0/2375420409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.918+0000 7fa9437fe640 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 -- 172.21.15.77:0/2375420409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 msgr2=0x564b448dd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448dd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 -- 172.21.15.77:0/2375420409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 msgr2=0x564b448bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 -- 172.21.15.77:0/2375420409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b448c1c00 con 0x564b448bb480 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9437fe640 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa938001eb0 tx=0x7fa9380097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966feebcc3881f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa928ff9640 1 -- 172.21.15.77:0/2375420409 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380122e0 con 0x564b448bb480 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa928ff9640 1 -- 172.21.15.77:0/2375420409 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938010b50 con 0x564b448bb480 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.919+0000 7fa928ff9640 1 -- 172.21.15.77:0/2375420409 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93801c950 con 0x564b448bb480 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- 172.21.15.77:0/2375420409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 msgr2=0x564b448c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa938001eb0 tx=0x7fa9380097a0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- 172.21.15.77:0/2375420409 shutdown_connections 2024-09-15T22:25:01.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bb480 0x564b448c4ab0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2375420409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bab60 0x564b448bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- 172.21.15.77:0/2375420409 >> 172.21.15.77:0/2375420409 conn(0x564b448b70f0 msgr2=0x564b448c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- 172.21.15.77:0/2375420409 shutdown_connections 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- 172.21.15.77:0/2375420409 wait complete. 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- start start 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 Processor -- start 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441b0460 con 0x564b448bb480 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441afd90 con 0x564b448bab60 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441acff0 con 0x564b448c5d40 2024-09-15T22:25:01.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.920+0000 7fa94bb13740 10 client.0 _ll_register_callbacks cb 0x7fa9380098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:46710/0 (socket says 172.21.15.77:46710) 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 -- 172.21.15.77:0/2104301974 learned_addr learned my addr 172.21.15.77:0/2104301974 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50902/0 (socket says 172.21.15.77:50902) 2024-09-15T22:25:01.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa9437fe640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 -- 172.21.15.77:0/2104301974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 msgr2=0x564b448ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 -- 172.21.15.77:0/2104301974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 msgr2=0x564b448c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 -- 172.21.15.77:0/2104301974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b448b2830 con 0x564b448bab60 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa942ffd640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa934005ec0 tx=0x7fa9340b8260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cb0ca47e2c3b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934049040 con 0x564b448bab60 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa938010b50 con 0x564b448bab60 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.921+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fa934031de0 con 0x564b448bab60 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa9340d5040 con 0x564b448bab60 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 10 client.27629 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b449408b0 con 0x564b448bab60 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938010b50 con 0x564b448bab60 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa9340edce0 con 0x564b448bab60 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 10 client.27629 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 10 client.27629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 20 client.27629 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b44940e90 0x564b44943280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 4 client.27629 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b44827d00 con 0x564b44940e90 2024-09-15T22:25:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.922+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b44940e90 0x564b44943280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:01.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.923+0000 7fa9427fc640 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b44940e90 0x564b44943280 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6035fdfc709509c6 server_cookie=97355382ff289383 in_seq=0 out_seq=0 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.923+0000 7fa926ff5640 10 client.27629 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.923+0000 7fa926ff5640 1 -- 172.21.15.77:0/2104301974 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b44827d00 con 0x564b44940e90 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa926ff5640 10 client.27629 handle_command_reply: tid=0 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 client.27629 shutdown 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 msgr2=0x564b448c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa934005ec0 tx=0x7fa9340b8260 comp rx=0 tx=0).stop 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 shutdown_connections 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b44940e90 0x564b44943280 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b448c5d40 0x564b448ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b448bb480 0x564b448c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa926ff5640 0 client.27629 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 --2- 172.21.15.77:0/2104301974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b448bab60 0x564b448c9970 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 >> 172.21.15.77:0/2104301974 conn(0x564b448b70f0 msgr2=0x564b448b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 shutdown_connections 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 1 -- 172.21.15.77:0/2104301974 wait complete. 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.924+0000 7fa94bb13740 20 client.27629 trim_cache size 0 max 16384 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:25:01.934 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "d4b2b2bf-b3e8-4f2e-b67b-827c4807cf00", 2024-09-15T22:25:01.935 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:25:01.935 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:01.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 msgr2=0x7fa92c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 0x7fa92c0a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c0107a0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 msgr2=0x7fa9441993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 0x7fa9441993d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa934005940 tx=0x7fa934003c10 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 shutdown_connections 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa92c0a33b0 0x7fa92c0a5870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa944102ef0 0x7fa9441a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa944102570 0x7fa944199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 --2- 172.21.15.77:0/856781129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa944100e20 0x7fa9441993d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 >> 172.21.15.77:0/856781129 conn(0x7fa9440fc7d0 msgr2=0x7fa9440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 shutdown_connections 2024-09-15T22:25:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:01.925+0000 7fa94973a640 1 -- 172.21.15.77:0/856781129 wait complete. 2024-09-15T22:25:02.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 957836 7.34 MB/sec execute 551 sec latency 96.624 ms 2024-09-15T22:25:02.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:02 smithi077 ceph-mon[30980]: pgmap v393: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 220 op/s 2024-09-15T22:25:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:02 smithi196 ceph-mon[35109]: pgmap v393: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 220 op/s 2024-09-15T22:25:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:02 smithi117 ceph-mon[35021]: pgmap v393: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 220 op/s 2024-09-15T22:25:02.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:25:03.088 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:03.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 959879 7.35 MB/sec execute 552 sec latency 18.529 ms 2024-09-15T22:25:03.506 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:25:03.506 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.506 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:25:03.506 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:25:03.507 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.507 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:03 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:25:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:25:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:25:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:03 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:03 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:03.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.592+0000 7ff8f35c0640 1 -- 172.21.15.77:0/3751305478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec102570 msgr2=0x7ff8ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.592+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/3751305478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec102570 0x7ff8ec1029b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009900 tx=0x7ff8dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:25:03.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 -- 172.21.15.77:0/3751305478 shutdown_connections 2024-09-15T22:25:03.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/3751305478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 0x7ff8ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/3751305478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec102570 0x7ff8ec1029b0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/3751305478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec100e20 0x7ff8ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 -- 172.21.15.77:0/3751305478 >> 172.21.15.77:0/3751305478 conn(0x7ff8ec0fc7d0 msgr2=0x7ff8ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 -- 172.21.15.77:0/3751305478 shutdown_connections 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.593+0000 7ff8f35c0640 1 -- 172.21.15.77:0/3751305478 wait complete. 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 Processor -- start 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 -- start start 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 0x7ff8ec1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 0x7ff8ec197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 0x7ff8ec19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec110f80 con 0x7ff8ec100e20 2024-09-15T22:25:03.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec110e00 con 0x7ff8ec102570 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f35c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111100 con 0x7ff8ec102ef0 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 0x7ff8ec1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 0x7ff8ec1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50924/0 (socket says 172.21.15.77:50924) 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 0x7ff8ec197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f135c640 1 -- 172.21.15.77:0/2194950130 learned_addr learned my addr 172.21.15.77:0/2194950130 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 0x7ff8ec19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/2194950130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 msgr2=0x7ff8ec19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 0x7ff8ec19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 msgr2=0x7ff8ec1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 0x7ff8ec1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.594+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/2194950130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec19c3b0 con 0x7ff8ec102570 2024-09-15T22:25:03.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 0x7ff8ec197920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005940 tx=0x7ff8dc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8e27fc640 1 -- 172.21.15.77:0/2194950130 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc04d070 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec19c640 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8e27fc640 1 -- 172.21.15.77:0/2194950130 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc031540 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8e27fc640 1 -- 172.21.15.77:0/2194950130 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc035070 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.595+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec069f70 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.598+0000 7ff8e27fc640 1 -- 172.21.15.77:0/2194950130 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff8dc050070 con 0x7ff8ec102570 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.598+0000 7ff8f5999740 1 Processor -- start 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 -- start start 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359036880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359041680 0x55d359043a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d358f9c2e0 con 0x55d359041680 2024-09-15T22:25:03.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d358f43ce0 con 0x55d359036dc0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f5999740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d358f982a0 con 0x55d3590364a0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f0b5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359041680 0x55d359043a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:46728/0 (socket says 172.21.15.77:46728) 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 -- 172.21.15.77:0/3066578364 learned_addr learned my addr 172.21.15.77:0/3066578364 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359036880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 -- 172.21.15.77:0/3066578364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 msgr2=0x55d359036880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359036880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 -- 172.21.15.77:0/3066578364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359041680 msgr2=0x55d359043a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359041680 0x55d359043a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.599+0000 7ff8f135c640 1 -- 172.21.15.77:0/3066578364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d358f28440 con 0x55d359036dc0 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.600+0000 7ff8e27fc640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 0x7ff8d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.600+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009dc0 tx=0x7ff8e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928dc47be888a1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.600+0000 7ff8d77fe640 1 -- 172.21.15.77:0/3066578364 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8005220 con 0x55d359036dc0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.600+0000 7ff8d77fe640 1 -- 172.21.15.77:0/3066578364 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8004c80 con 0x55d359036dc0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8e27fc640 1 -- 172.21.15.77:0/2194950130 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7ff8dc0f8310 con 0x7ff8ec102570 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8d77fe640 1 -- 172.21.15.77:0/3066578364 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8019910 con 0x55d359036dc0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 0x7ff8d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 -- 172.21.15.77:0/3066578364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 msgr2=0x55d359041140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 0x7ff8d80a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fd0 tx=0x7ff8e4009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009dc0 tx=0x7ff8e8011810 comp rx=0 tx=0).stop 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 -- 172.21.15.77:0/3066578364 shutdown_connections 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359041680 0x55d359043a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359041140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3066578364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359036880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.601+0000 7ff8f5999740 1 -- 172.21.15.77:0/3066578364 >> 172.21.15.77:0/3066578364 conn(0x55d359032a30 msgr2=0x55d359032e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.602+0000 7ff8f5999740 1 -- 172.21.15.77:0/3066578364 shutdown_connections 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.602+0000 7ff8f5999740 1 -- 172.21.15.77:0/3066578364 wait complete. 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.602+0000 7ff8f5999740 1 -- start start 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.602+0000 7ff8f5999740 1 Processor -- start 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 0x55d35903e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d35903e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f0b5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 0x55d35903e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42066/0 (socket says 172.21.15.77:42066) 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f0b5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 0x55d35903e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50954/0 (socket says 172.21.15.77:50954) 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 -- 172.21.15.77:0/3076180245 learned_addr learned my addr 172.21.15.77:0/3076180245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:03.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3590442a0 con 0x55d359036dc0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d358f57050 con 0x55d359041680 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d359044420 con 0x55d3590364a0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f5999740 10 client.0 _ll_register_callbacks cb 0x55d358f97160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d35903e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 -- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 msgr2=0x55d35903e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d35903e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 -- 172.21.15.77:0/3076180245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 msgr2=0x55d35903e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 0x55d35903e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.603+0000 7ff8f135c640 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d358d0d3c0 con 0x55d3590364a0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8019470 tx=0x7ff8e80052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2529f2bab55642d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:03.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8027070 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e8004c80 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7ff8e8004450 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8e8031050 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8f5999740 10 client.27553 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.604+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d35904fc20 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.605+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8004c80 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff8e8019c70 con 0x55d3590364a0 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 10 client.27553 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 10 client.27553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:03.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 20 client.27553 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d359051080 0x55d359053470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 4 client.27553 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d358fa9850 con 0x55d359051080 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d359051080 0x55d359053470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d359051080 0x55d359053470 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8986e78bc9b590d server_cookie=af171f5971bc9d29 in_seq=0 out_seq=0 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.606+0000 7ff8d4ff9640 10 client.27553 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3076180245 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d358fa9850 con 0x55d359051080 2024-09-15T22:25:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8d4ff9640 10 client.27553 handle_command_reply: tid=0 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 client.27553 shutdown 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 msgr2=0x55d359040fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8019470 tx=0x7ff8e80052b0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 shutdown_connections 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d359051080 0x55d359053470 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d35903e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d359036dc0 0x55d35903e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8d4ff9640 0 client.27553 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3076180245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d3590364a0 0x55d359040fa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 >> 172.21.15.77:0/3076180245 conn(0x55d359032a30 msgr2=0x55d359038710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.607+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 shutdown_connections 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.608+0000 7ff8f5999740 1 -- 172.21.15.77:0/3076180245 wait complete. 2024-09-15T22:25:03.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.608+0000 7ff8f5999740 20 client.27553 trim_cache size 0 max 16384 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.627+0000 7ff8f5999740 1 Processor -- start 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.627+0000 7ff8f5999740 1 -- start start 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.627+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.627+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359036dc0 0x55d359058c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d359059170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f5999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1acda0 con 0x55d3590364a0 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f5999740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ae8e0 con 0x55d359041680 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f5999740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ad470 con 0x55d359036dc0 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50958/0 (socket says 172.21.15.77:50958) 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 -- 172.21.15.77:0/2380225085 learned_addr learned my addr 172.21.15.77:0/2380225085 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359036dc0 0x55d359058c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d359059170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 -- 172.21.15.77:0/2380225085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359036dc0 msgr2=0x55d359058c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359036dc0 0x55d359058c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 -- 172.21.15.77:0/2380225085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 msgr2=0x55d359059170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d359059170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 -- 172.21.15.77:0/2380225085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d359036bf0 con 0x55d3590364a0 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8f135c640 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011850 tx=0x7ff8e8009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111b9c91931a09c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8d6ffd640 1 -- 172.21.15.77:0/2380225085 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8027070 con 0x55d3590364a0 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8d6ffd640 1 -- 172.21.15.77:0/2380225085 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e803b070 con 0x55d3590364a0 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.628+0000 7ff8d6ffd640 1 -- 172.21.15.77:0/2380225085 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004250 con 0x55d3590364a0 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- 172.21.15.77:0/2380225085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 msgr2=0x55d35904d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8011850 tx=0x7ff8e8009c10 comp rx=0 tx=0).stop 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- 172.21.15.77:0/2380225085 shutdown_connections 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359041680 0x55d359059170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359036dc0 0x55d359058c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 --2- 172.21.15.77:0/2380225085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d35904d500 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- 172.21.15.77:0/2380225085 >> 172.21.15.77:0/2380225085 conn(0x55d359032a30 msgr2=0x55d35903d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- 172.21.15.77:0/2380225085 shutdown_connections 2024-09-15T22:25:03.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- 172.21.15.77:0/2380225085 wait complete. 2024-09-15T22:25:03.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.629+0000 7ff8f5999740 1 -- start start 2024-09-15T22:25:03.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 Processor -- start 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359046db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359041680 0x55d3590b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1ae210 con 0x55d3590364a0 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1adb40 con 0x55d359036dc0 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1aada0 con 0x55d359041680 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f5999740 10 client.0 _ll_register_callbacks cb 0x7ff8dc0fbd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f135c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50964/0 (socket says 172.21.15.77:50964) 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359046db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f135c640 1 -- 172.21.15.77:0/3633697231 learned_addr learned my addr 172.21.15.77:0/3633697231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.630+0000 7ff8f1b5d640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359041680 0x55d3590b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/3633697231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359041680 msgr2=0x55d3590b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359041680 0x55d3590b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/3633697231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 msgr2=0x55d359046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 -- 172.21.15.77:0/3633697231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3590ba140 con 0x55d359036dc0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f0b5b640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359046db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0f56d0 tx=0x7ff8dc030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc85e73d84471a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0c40e0 con 0x55d359036dc0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7ff8dc0316f0 con 0x55d359036dc0 2024-09-15T22:25:03.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e803b070 con 0x55d359036dc0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc0c3c70 con 0x55d359036dc0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f5999740 10 client.27647 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3590bb350 con 0x55d359036dc0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.631+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc0f8150 con 0x55d359036dc0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff8dc047030 con 0x55d359036dc0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 10 client.27647 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 10 client.27647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 20 client.27647 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d3590bba80 0x55d3590bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 4 client.27647 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff8e803b070 con 0x55d3590bba80 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d3590bba80 0x55d3590bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:03.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8d4ff9640 10 client.27647 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.632+0000 7ff8f135c640 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d3590bba80 0x55d3590bdea0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45f76d37d69dd3 server_cookie=64f883a5db356e27 in_seq=0 out_seq=0 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8d4ff9640 1 -- 172.21.15.77:0/3633697231 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ff8e803b070 con 0x55d3590bba80 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8d4ff9640 10 client.27647 handle_command_reply: tid=0 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 client.27647 shutdown 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 msgr2=0x55d359046db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359046db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0f56d0 tx=0x7ff8dc030000 comp rx=0 tx=0).stop 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 shutdown_connections 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d3590bba80 0x55d3590bdea0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d359041680 0x55d3590b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d359036dc0 0x55d359046db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 --2- 172.21.15.77:0/3633697231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3590364a0 0x55d359046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 >> 172.21.15.77:0/3633697231 conn(0x55d359032a30 msgr2=0x55d359031350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8d4ff9640 0 client.27647 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 shutdown_connections 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 1 -- 172.21.15.77:0/3633697231 wait complete. 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.633+0000 7ff8f5999740 20 client.27647 trim_cache size 0 max 16384 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout: "d4b2b2bf-b3e8-4f2e-b67b-827c4807cf00": { 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:25:03.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "d4b2b2bf-b3e8-4f2e-b67b-827c4807cf00", 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 msgr2=0x7ff8d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 0x7ff8d80a5870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fd0 tx=0x7ff8e4009500 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 msgr2=0x7ff8ec197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 0x7ff8ec197920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005940 tx=0x7ff8dc005dc0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 shutdown_connections 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff8d80a33b0 0x7ff8d80a5870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8ec102ef0 0x7ff8ec19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8ec102570 0x7ff8ec197920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 --2- 172.21.15.77:0/2194950130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8ec100e20 0x7ff8ec1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 >> 172.21.15.77:0/2194950130 conn(0x7ff8ec0fc7d0 msgr2=0x7ff8ec1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 shutdown_connections 2024-09-15T22:25:03.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:03.634+0000 7ff8f35c0640 1 -- 172.21.15.77:0/2194950130 wait complete. 2024-09-15T22:25:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:04 smithi077 ceph-mon[30980]: pgmap v394: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:04.458 INFO:tasks.cephfs.filesystem:scrub status for tag:d4b2b2bf-b3e8-4f2e-b67b-827c4807cf00 - {'path': '/', 'tag': 'd4b2b2bf-b3e8-4f2e-b67b-827c4807cf00', 'options': 'recursive,force'} 2024-09-15T22:25:04.493 INFO:tasks.workunit.client.0.smithi077.stdout: 1 962295 7.36 MB/sec execute 553 sec latency 19.244 ms 2024-09-15T22:25:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:04 smithi196 ceph-mon[35109]: pgmap v394: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:04 smithi117 ceph-mon[35021]: pgmap v394: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:05.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 963552 7.36 MB/sec execute 554 sec latency 134.668 ms 2024-09-15T22:25:06.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 965748 7.37 MB/sec execute 555 sec latency 33.806 ms 2024-09-15T22:25:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:06 smithi077 ceph-mon[30980]: pgmap v395: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:06 smithi196 ceph-mon[35109]: pgmap v395: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:06 smithi117 ceph-mon[35021]: pgmap v395: 129 pgs: 129 active+clean; 568 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-09-15T22:25:07.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 967599 7.37 MB/sec execute 556 sec latency 12.719 ms 2024-09-15T22:25:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:07 smithi077 ceph-mon[30980]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:25:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:07 smithi196 ceph-mon[35109]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:25:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:07 smithi117 ceph-mon[35021]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:25:08.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 970139 7.39 MB/sec execute 557 sec latency 18.157 ms 2024-09-15T22:25:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:08 smithi077 ceph-mon[30980]: pgmap v397: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:08 smithi196 ceph-mon[35109]: pgmap v397: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:08 smithi117 ceph-mon[35021]: pgmap v397: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:09.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 971903 7.39 MB/sec execute 558 sec latency 18.266 ms 2024-09-15T22:25:10.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 973853 7.40 MB/sec execute 559 sec latency 48.135 ms 2024-09-15T22:25:10.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:10 smithi077 ceph-mon[30980]: pgmap v398: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:10.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:10 smithi196 ceph-mon[35109]: pgmap v398: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:10 smithi117 ceph-mon[35021]: pgmap v398: 129 pgs: 129 active+clean; 580 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-09-15T22:25:11.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 975668 7.39 MB/sec execute 560 sec latency 39.496 ms 2024-09-15T22:25:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:25:11.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:25:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:25:11.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:25:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:25:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:25:12.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 977700 7.40 MB/sec execute 561 sec latency 12.954 ms 2024-09-15T22:25:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:12 smithi077 ceph-mon[30980]: pgmap v399: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:12 smithi196 ceph-mon[35109]: pgmap v399: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:12 smithi117 ceph-mon[35021]: pgmap v399: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:13.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 980342 7.42 MB/sec execute 562 sec latency 14.036 ms 2024-09-15T22:25:14.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 981775 7.42 MB/sec execute 563 sec latency 20.707 ms 2024-09-15T22:25:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:14 smithi077 ceph-mon[30980]: pgmap v400: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:14 smithi077 ceph-mon[30980]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:25:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:14 smithi196 ceph-mon[35109]: pgmap v400: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:14 smithi196 ceph-mon[35109]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:25:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:14 smithi117 ceph-mon[35021]: pgmap v400: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 242 op/s 2024-09-15T22:25:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:14 smithi117 ceph-mon[35021]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:25:15.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 984245 7.43 MB/sec execute 564 sec latency 70.709 ms 2024-09-15T22:25:16.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 985596 7.43 MB/sec execute 565 sec latency 145.590 ms 2024-09-15T22:25:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:16 smithi077 ceph-mon[30980]: pgmap v402: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 127 op/s 2024-09-15T22:25:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:16 smithi196 ceph-mon[35109]: pgmap v402: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 127 op/s 2024-09-15T22:25:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:16 smithi117 ceph-mon[35021]: pgmap v402: 129 pgs: 129 active+clean; 592 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 127 op/s 2024-09-15T22:25:17.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 987791 7.44 MB/sec execute 566 sec latency 24.510 ms 2024-09-15T22:25:18.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 988912 7.43 MB/sec execute 567 sec latency 238.904 ms 2024-09-15T22:25:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:18 smithi077 ceph-mon[30980]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:18 smithi196 ceph-mon[35109]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:18 smithi117 ceph-mon[35021]: pgmap v403: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:19.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 991307 7.44 MB/sec execute 568 sec latency 14.595 ms 2024-09-15T22:25:20.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 992761 7.44 MB/sec execute 569 sec latency 20.137 ms 2024-09-15T22:25:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:20 smithi077 ceph-mon[30980]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:20 smithi196 ceph-mon[35109]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:20 smithi117 ceph-mon[35021]: pgmap v404: 129 pgs: 129 active+clean; 622 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-15T22:25:21.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 994775 7.45 MB/sec execute 570 sec latency 147.292 ms 2024-09-15T22:25:22.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 995958 7.45 MB/sec execute 571 sec latency 13.815 ms 2024-09-15T22:25:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:22 smithi077 ceph-mon[30980]: pgmap v405: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:22 smithi196 ceph-mon[35109]: pgmap v405: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:22 smithi117 ceph-mon[35021]: pgmap v405: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:23.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 998381 7.46 MB/sec execute 572 sec latency 30.223 ms 2024-09-15T22:25:24.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 999748 7.46 MB/sec execute 573 sec latency 14.526 ms 2024-09-15T22:25:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:24 smithi077 ceph-mon[30980]: pgmap v406: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:24 smithi196 ceph-mon[35109]: pgmap v406: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:24 smithi117 ceph-mon[35021]: pgmap v406: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.1 MiB/s wr, 204 op/s 2024-09-15T22:25:25.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1001945 7.47 MB/sec execute 574 sec latency 14.169 ms 2024-09-15T22:25:26.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1002932 7.46 MB/sec execute 575 sec latency 46.245 ms 2024-09-15T22:25:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:26 smithi077 ceph-mon[30980]: pgmap v407: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-09-15T22:25:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:26 smithi196 ceph-mon[35109]: pgmap v407: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-09-15T22:25:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:26 smithi117 ceph-mon[35021]: pgmap v407: 129 pgs: 129 active+clean; 570 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.7 MiB/s wr, 190 op/s 2024-09-15T22:25:27.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1005423 7.47 MB/sec execute 576 sec latency 21.749 ms 2024-09-15T22:25:28.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1006824 7.47 MB/sec execute 577 sec latency 15.897 ms 2024-09-15T22:25:28.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:28 smithi077 ceph-mon[30980]: pgmap v408: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 260 op/s 2024-09-15T22:25:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:28 smithi196 ceph-mon[35109]: pgmap v408: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 260 op/s 2024-09-15T22:25:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:28 smithi117 ceph-mon[35021]: pgmap v408: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.0 MiB/s wr, 260 op/s 2024-09-15T22:25:29.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1009075 7.48 MB/sec execute 578 sec latency 20.076 ms 2024-09-15T22:25:30.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1010566 7.48 MB/sec execute 579 sec latency 27.549 ms 2024-09-15T22:25:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:30 smithi077 ceph-mon[30980]: pgmap v409: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 165 op/s 2024-09-15T22:25:30.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:30 smithi196 ceph-mon[35109]: pgmap v409: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 165 op/s 2024-09-15T22:25:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:30 smithi117 ceph-mon[35021]: pgmap v409: 129 pgs: 129 active+clean; 582 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 165 op/s 2024-09-15T22:25:31.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1012620 7.49 MB/sec execute 580 sec latency 84.917 ms 2024-09-15T22:25:32.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1014013 7.49 MB/sec execute 581 sec latency 36.069 ms 2024-09-15T22:25:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:32 smithi077 ceph-mon[30980]: pgmap v410: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T22:25:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:32 smithi196 ceph-mon[35109]: pgmap v410: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T22:25:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:32 smithi117 ceph-mon[35021]: pgmap v410: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T22:25:33.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1016183 7.50 MB/sec execute 582 sec latency 16.943 ms 2024-09-15T22:25:34.459 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:25:34.494 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1017737 7.49 MB/sec execute 583 sec latency 24.160 ms 2024-09-15T22:25:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:34 smithi077 ceph-mon[30980]: pgmap v411: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:34 smithi196 ceph-mon[35109]: pgmap v411: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:34 smithi117 ceph-mon[35021]: pgmap v411: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:34.726 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:35.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 -- 172.21.15.77:0/1855650294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40100e20 msgr2=0x7fed40101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 --2- 172.21.15.77:0/1855650294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40100e20 0x7fed40101200 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed3002a480 comp rx=0 tx=0).stop 2024-09-15T22:25:35.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 -- 172.21.15.77:0/1855650294 shutdown_connections 2024-09-15T22:25:35.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 --2- 172.21.15.77:0/1855650294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40102ef0 0x7fed4010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 --2- 172.21.15.77:0/1855650294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102570 0x7fed401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 --2- 172.21.15.77:0/1855650294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40100e20 0x7fed40101200 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 -- 172.21.15.77:0/1855650294 >> 172.21.15.77:0/1855650294 conn(0x7fed400fc7d0 msgr2=0x7fed400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 -- 172.21.15.77:0/1855650294 shutdown_connections 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.253+0000 7fed45ae5640 1 -- 172.21.15.77:0/1855650294 wait complete. 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 Processor -- start 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 -- start start 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40100e20 0x7fed401973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102ef0 0x7fed4019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed40110f80 con 0x7fed40102570 2024-09-15T22:25:35.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed40110e00 con 0x7fed40102ef0 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.254+0000 7fed45ae5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed40111100 con 0x7fed40100e20 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41076/0 (socket says 172.21.15.77:41076) 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40100e20 0x7fed401973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 -- 172.21.15.77:0/2174031204 learned_addr learned my addr 172.21.15.77:0/2174031204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed3ffff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102ef0 0x7fed4019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 -- 172.21.15.77:0/2174031204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40100e20 msgr2=0x7fed401973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40100e20 0x7fed401973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 -- 172.21.15.77:0/2174031204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102ef0 msgr2=0x7fed4019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102ef0 0x7fed4019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 -- 172.21.15.77:0/2174031204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4019c370 con 0x7fed40102570 2024-09-15T22:25:35.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed37fff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eb00 tx=0x7fed28009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed3d7fa640 1 -- 172.21.15.77:0/2174031204 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2801c070 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4019c660 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed3d7fa640 1 -- 172.21.15.77:0/2174031204 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2800bc30 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed3d7fa640 1 -- 172.21.15.77:0/2174031204 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280166a0 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.255+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed40069f70 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed3d7fa640 1 -- 172.21.15.77:0/2174031204 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fed2801f070 con 0x7fed40102570 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 Processor -- start 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 -- start start 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed3d7fa640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f2058be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f204e02e0 con 0x555f2057a380 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f20487ce0 con 0x555f2057aca0 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.258+0000 7fed47ebe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f204dc2a0 con 0x555f20589a90 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3ffff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3d7fa640 1 -- 172.21.15.77:0/2174031204 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fed280d1670 con 0x7fed40102570 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed37fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed37fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38618/0 (socket says 172.21.15.77:38618) 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed37fff640 1 -- 172.21.15.77:0/722187896 learned_addr learned my addr 172.21.15.77:0/722187896 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41080/0 (socket says 172.21.15.77:41080) 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3ffff640 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f2058be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 -- 172.21.15.77:0/722187896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 msgr2=0x555f2058be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f2058be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 -- 172.21.15.77:0/722187896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 msgr2=0x555f20589550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 -- 172.21.15.77:0/722187896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2046c440 con 0x555f2057a380 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed37fff640 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3ffff640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 0x7fed2c0a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed38009fd0 tx=0x7fed3801dbf0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fed30005940 tx=0x7fed30005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c088c0e4223fc3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed35ffb640 1 -- 172.21.15.77:0/722187896 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30035030 con 0x555f2057a380 2024-09-15T22:25:35.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed35ffb640 1 -- 172.21.15.77:0/722187896 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30040070 con 0x555f2057a380 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.259+0000 7fed35ffb640 1 -- 172.21.15.77:0/722187896 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004cb0 con 0x555f2057a380 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 -- 172.21.15.77:0/722187896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 msgr2=0x555f2057a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fed30005940 tx=0x7fed30005dd0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 -- 172.21.15.77:0/722187896 shutdown_connections 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f2058be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20589550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 --2- 172.21.15.77:0/722187896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f2057a760 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 -- 172.21.15.77:0/722187896 >> 172.21.15.77:0/722187896 conn(0x555f20576a30 msgr2=0x555f20576e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 -- 172.21.15.77:0/722187896 shutdown_connections 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.260+0000 7fed47ebe740 1 -- 172.21.15.77:0/722187896 wait complete. 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 -- start start 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 Processor -- start 2024-09-15T22:25:35.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20582e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f205833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f2057d320 con 0x555f2057a380 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555f2049b050 con 0x555f2057aca0 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f2057d4a0 con 0x555f20589a90 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed47ebe740 10 client.0 _ll_register_callbacks cb 0x555f204db160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41090/0 (socket says 172.21.15.77:41090) 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed3f7fe640 1 -- 172.21.15.77:0/2762479865 learned_addr learned my addr 172.21.15.77:0/2762479865 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed37fff640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20582e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.261+0000 7fed3ffff640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f205833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 -- 172.21.15.77:0/2762479865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 msgr2=0x555f205833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f205833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 -- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 msgr2=0x555f20582e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20582e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 -- 172.21.15.77:0/2762479865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f202513c0 con 0x555f2057a380 2024-09-15T22:25:35.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed37fff640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20582e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed30009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f98fd790befeab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30052060 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fed30004c30 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30040070 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed30059050 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed47ebe740 10 client.18681 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f20593c70 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30040070 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.262+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fed30003a30 con 0x555f2057a380 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 10 client.18681 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:35.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 20 client.18681 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f205973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 4 client.18681 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f204ed850 con 0x555f20594fc0 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed37fff640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f205973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed37fff640 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f205973b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832fff657b9315f3 server_cookie=3543312447929bfc in_seq=0 out_seq=0 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.263+0000 7fed1effd640 10 client.18681 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed1effd640 1 -- 172.21.15.77:0/2762479865 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f204ed850 con 0x555f20594fc0 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed1effd640 10 client.18681 handle_command_reply: tid=0 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 client.18681 shutdown 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 msgr2=0x555f20582950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed30009610 comp rx=0 tx=0).stop 2024-09-15T22:25:35.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 shutdown_connections 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f205973b0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed1effd640 0 client.18681 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f205833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f20582e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2762479865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20582950 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 >> 172.21.15.77:0/2762479865 conn(0x555f20576a30 msgr2=0x555f2057c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 shutdown_connections 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 1 -- 172.21.15.77:0/2762479865 wait complete. 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.264+0000 7fed47ebe740 20 client.18681 trim_cache size 0 max 16384 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.281+0000 7fed47ebe740 1 Processor -- start 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.281+0000 7fed47ebe740 1 -- start start 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f2057fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f20580110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed401acd60 con 0x555f2057a380 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ae8a0 con 0x555f2057aca0 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed47ebe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ad430 con 0x555f20589a90 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41098/0 (socket says 172.21.15.77:41098) 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed37fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f2057fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 -- 172.21.15.77:0/4068544505 learned_addr learned my addr 172.21.15.77:0/4068544505 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3ffff640 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f20580110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 -- 172.21.15.77:0/4068544505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 msgr2=0x555f20580110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f20580110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 -- 172.21.15.77:0/4068544505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 msgr2=0x555f2057fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f2057fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 -- 172.21.15.77:0/4068544505 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2057e540 con 0x555f2057a380 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fed30006010 tx=0x7fed30009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5a4d7bbdd37d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed357fa640 1 -- 172.21.15.77:0/4068544505 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30052040 con 0x555f2057a380 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed357fa640 1 -- 172.21.15.77:0/4068544505 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30040070 con 0x555f2057a380 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.282+0000 7fed357fa640 1 -- 172.21.15.77:0/4068544505 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30035070 con 0x555f2057a380 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 -- 172.21.15.77:0/4068544505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 msgr2=0x555f20583220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fed30006010 tx=0x7fed30009680 comp rx=0 tx=0).stop 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 -- 172.21.15.77:0/4068544505 shutdown_connections 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f20589a90 0x555f20580110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057aca0 0x555f2057fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 --2- 172.21.15.77:0/4068544505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2057a380 0x555f20583220 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 -- 172.21.15.77:0/4068544505 >> 172.21.15.77:0/4068544505 conn(0x555f20576a30 msgr2=0x555f20581910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 -- 172.21.15.77:0/4068544505 shutdown_connections 2024-09-15T22:25:35.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.283+0000 7fed47ebe740 1 -- 172.21.15.77:0/4068544505 wait complete. 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 -- start start 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 Processor -- start 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 0x555f2058f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 0x555f2058f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ae1d0 con 0x555f20589a90 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed401adb00 con 0x555f2057a380 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed401aad60 con 0x555f2057aca0 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed47ebe740 10 client.0 _ll_register_callbacks cb 0x7fed28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.284+0000 7fed3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 0x555f2058f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 0x555f2058f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38632/0 (socket says 172.21.15.77:38632) 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed3f7fe640 1 -- 172.21.15.77:0/2788595591 learned_addr learned my addr 172.21.15.77:0/2788595591 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed3ffff640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 0x555f2058f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41102/0 (socket says 172.21.15.77:41102) 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 -- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 msgr2=0x555f20591cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 -- 172.21.15.77:0/2788595591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 msgr2=0x555f2058f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 0x555f2058f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 -- 172.21.15.77:0/2788595591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2058fed0 con 0x555f2057aca0 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed37fff640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 0x555f2058f160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fed2809a040 tx=0x7fed28009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a27b86be2daae0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280a6230 con 0x555f2057aca0 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fed28099ce0 con 0x555f2057aca0 2024-09-15T22:25:35.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30040070 con 0x555f2057aca0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed280a5da0 con 0x555f2057aca0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed47ebe740 10 client.27577 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.285+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f205ff8b0 con 0x555f2057aca0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed280d0040 con 0x555f2057aca0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fed280993c0 con 0x555f2057aca0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 10 client.27577 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 20 client.27577 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f206020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 4 client.27577 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fed30040070 con 0x555f20594fc0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f206020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed1effd640 10 client.27577 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:35.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.286+0000 7fed3f7fe640 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f206020c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fe4f7eb6340d8b server_cookie=65072d9fe06d7e7a in_seq=0 out_seq=0 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed1effd640 1 -- 172.21.15.77:0/2788595591 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fed30040070 con 0x555f20594fc0 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed1effd640 10 client.27577 handle_command_reply: tid=0 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 client.27577 shutdown 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 msgr2=0x555f2058f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 0x555f2058f160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fed2809a040 tx=0x7fed28009780 comp rx=0 tx=0).stop 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 shutdown_connections 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555f20594fc0 0x555f206020c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f20589a90 0x555f2058f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f2057aca0 0x555f2058f160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 --2- 172.21.15.77:0/2788595591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f2057a380 0x555f20591cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 >> 172.21.15.77:0/2788595591 conn(0x555f20576a30 msgr2=0x555f20575350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed1effd640 0 client.27577 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:35.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 shutdown_connections 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 1 -- 172.21.15.77:0/2788595591 wait complete. 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.287+0000 7fed47ebe740 20 client.27577 trim_cache size 0 max 16384 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:25:35.308 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 msgr2=0x7fed2c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 0x7fed2c0a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed38009fd0 tx=0x7fed3801dbf0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 msgr2=0x7fed40197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eb00 tx=0x7fed28009f20 comp rx=0 tx=0).stop 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 shutdown_connections 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed2c0a33b0 0x7fed2c0a5870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed40102ef0 0x7fed4019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed40102570 0x7fed40197900 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 --2- 172.21.15.77:0/2174031204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed40100e20 0x7fed401973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 >> 172.21.15.77:0/2174031204 conn(0x7fed400fc7d0 msgr2=0x7fed400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 shutdown_connections 2024-09-15T22:25:35.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:35.288+0000 7fed45ae5640 1 -- 172.21.15.77:0/2174031204 wait complete. 2024-09-15T22:25:35.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1019751 7.50 MB/sec execute 584 sec latency 35.284 ms 2024-09-15T22:25:36.206 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:25:36.206 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:36.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1021302 7.50 MB/sec execute 585 sec latency 55.066 ms 2024-09-15T22:25:36.508 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:36 smithi077 ceph-mon[30980]: pgmap v412: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:36 smithi196 ceph-mon[35109]: pgmap v412: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:36 smithi117 ceph-mon[35021]: pgmap v412: 129 pgs: 129 active+clean; 597 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.1 MiB/s wr, 189 op/s 2024-09-15T22:25:37.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 -- 172.21.15.77:0/783318895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490109120 msgr2=0x7fb490110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:37.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 --2- 172.21.15.77:0/783318895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490109120 0x7fb490110ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb480009ca0 tx=0x7fb48002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 -- 172.21.15.77:0/783318895 shutdown_connections 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 --2- 172.21.15.77:0/783318895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490109120 0x7fb490110ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 --2- 172.21.15.77:0/783318895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 0x7fb4901089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 --2- 172.21.15.77:0/783318895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb490103ff0 0x7fb4901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 -- 172.21.15.77:0/783318895 >> 172.21.15.77:0/783318895 conn(0x7fb4900ffab0 msgr2=0x7fb490101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 -- 172.21.15.77:0/783318895 shutdown_connections 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.036+0000 7fb497597640 1 -- 172.21.15.77:0/783318895 wait complete. 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 Processor -- start 2024-09-15T22:25:37.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 -- start start 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 0x7fb4901a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb490109120 0x7fb49010fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb490077cc0 con 0x7fb4901049a0 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb490077b40 con 0x7fb490103ff0 2024-09-15T22:25:37.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.037+0000 7fb497597640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb490077e40 con 0x7fb490109120 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38644/0 (socket says 172.21.15.77:38644) 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 -- 172.21.15.77:0/2753776448 learned_addr learned my addr 172.21.15.77:0/2753776448 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb494b32640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 0x7fb4901a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 -- 172.21.15.77:0/2753776448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb490109120 msgr2=0x7fb49010fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb490109120 0x7fb49010fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 -- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 msgr2=0x7fb4901a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 0x7fb4901a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 -- 172.21.15.77:0/2753776448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb490110260 con 0x7fb490103ff0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.038+0000 7fb495333640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c920 tx=0x7fb48800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.039+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488007c70 con 0x7fb490103ff0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.039+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb488007e10 con 0x7fb490103ff0 2024-09-15T22:25:37.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.039+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb490110550 con 0x7fb490103ff0 2024-09-15T22:25:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.039+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901ab6f0 con 0x7fb490103ff0 2024-09-15T22:25:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.039+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488018890 con 0x7fb490103ff0 2024-09-15T22:25:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.040+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c005290 con 0x7fb490103ff0 2024-09-15T22:25:37.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.042+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb48800fb30 con 0x7fb490103ff0 2024-09-15T22:25:37.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.043+0000 7fb48e7fc640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 0x7fb4780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.043+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fb4880d01c0 con 0x7fb490103ff0 2024-09-15T22:25:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.043+0000 7fb494b32640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 0x7fb4780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:37.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.044+0000 7fb494b32640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 0x7fb4780a58f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4901111b0 tx=0x7fb484005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:37.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.049+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb488014030 con 0x7fb490103ff0 2024-09-15T22:25:37.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.281+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb47c005850 con 0x7fb490103ff0 2024-09-15T22:25:37.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.282+0000 7fb48e7fc640 1 -- 172.21.15.77:0/2753776448 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7fb48809c7b0 con 0x7fb490103ff0 2024-09-15T22:25:37.284 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:25:37.284 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:25:37.284 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 msgr2=0x7fb4780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 0x7fb4780a58f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4901111b0 tx=0x7fb484005f90 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 msgr2=0x7fb4901a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c920 tx=0x7fb48800cde0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 shutdown_connections 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4780a3430 0x7fb4780a58f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb490109120 0x7fb49010fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4901049a0 0x7fb4901a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 --2- 172.21.15.77:0/2753776448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb490103ff0 0x7fb4901a4700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 >> 172.21.15.77:0/2753776448 conn(0x7fb4900ffab0 msgr2=0x7fb490101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 shutdown_connections 2024-09-15T22:25:37.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:37.284+0000 7fb497597640 1 -- 172.21.15.77:0/2753776448 wait complete. 2024-09-15T22:25:37.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1023279 7.51 MB/sec execute 586 sec latency 36.910 ms 2024-09-15T22:25:37.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:37 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2753776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:37 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2753776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:37.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:37 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2753776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:38.002 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:25:38.278 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:38.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1024830 7.51 MB/sec execute 587 sec latency 22.066 ms 2024-09-15T22:25:38.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:38 smithi077 ceph-mon[30980]: pgmap v413: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.8 MiB/s wr, 272 op/s 2024-09-15T22:25:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:38 smithi196 ceph-mon[35109]: pgmap v413: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.8 MiB/s wr, 272 op/s 2024-09-15T22:25:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:38 smithi117 ceph-mon[35021]: pgmap v413: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.8 MiB/s wr, 272 op/s 2024-09-15T22:25:38.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 -- 172.21.15.77:0/2282968975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01082a0 msgr2=0x7f12b010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 --2- 172.21.15.77:0/2282968975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01082a0 0x7f12b010ec40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009cd0 tx=0x7f12a802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 -- 172.21.15.77:0/2282968975 shutdown_connections 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 --2- 172.21.15.77:0/2282968975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01082a0 0x7f12b010ec40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 --2- 172.21.15.77:0/2282968975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01036e0 0x7f12b0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 --2- 172.21.15.77:0/2282968975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 -- 172.21.15.77:0/2282968975 >> 172.21.15.77:0/2282968975 conn(0x7f12b00fc7d0 msgr2=0x7f12b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 -- 172.21.15.77:0/2282968975 shutdown_connections 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.789+0000 7f12b520a640 1 -- 172.21.15.77:0/2282968975 wait complete. 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 Processor -- start 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 -- start start 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 0x7f12b0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01082a0 0x7f12b019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01120a0 con 0x7f12b01036e0 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0111f20 con 0x7f12b0102d30 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12b520a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0112220 con 0x7f12b01082a0 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 0x7f12b0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12aed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12aed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38650/0 (socket says 172.21.15.77:38650) 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12aed9d640 1 -- 172.21.15.77:0/821429306 learned_addr learned my addr 172.21.15.77:0/821429306 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.790+0000 7f12af59e640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01082a0 0x7f12b019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 -- 172.21.15.77:0/821429306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01082a0 msgr2=0x7f12b019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01082a0 0x7f12b019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 -- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 msgr2=0x7f12b0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 0x7f12b0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 -- 172.21.15.77:0/821429306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b019d4f0 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12a7fff640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 0x7f12b0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12aed9d640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12a000d950 tx=0x7f12a000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12acd99640 1 -- 172.21.15.77:0/821429306 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0017070 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12acd99640 1 -- 172.21.15.77:0/821429306 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a0009c30 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b019d7e0 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12acd99640 1 -- 172.21.15.77:0/821429306 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a000f6b0 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.791+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01008c0 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.793+0000 7f12acd99640 1 -- 172.21.15.77:0/821429306 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f12a00109d0 con 0x7f12b0102d30 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 Processor -- start 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 -- start start 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.805 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7ef7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7f04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5585b7e5d2f0 con 0x5585b7f02370 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5585b7e04cf0 con 0x5585b7ef8b40 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.794+0000 7f12b75e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5585b7e592b0 con 0x5585b7ef75d0 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12acd99640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 0x7f129c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38652/0 (socket says 172.21.15.77:38652) 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 -- 172.21.15.77:0/3082956028 learned_addr learned my addr 172.21.15.77:0/3082956028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12aed9d640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7ef7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12a7fff640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7f04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12acd99640 1 -- 172.21.15.77:0/821429306 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f12a00cfc90 con 0x7f12b0102d30 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 0x7f129c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 -- 172.21.15.77:0/3082956028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 msgr2=0x5585b7ef7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7ef7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 -- 172.21.15.77:0/3082956028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 msgr2=0x5585b7f04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7f04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12af59e640 1 -- 172.21.15.77:0/3082956028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b7de93a0 con 0x5585b7ef8b40 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.795+0000 7f12a7fff640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7f04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.796+0000 7f12af59e640 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009430 tx=0x7f12a8031b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78aa3d21feb2ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.796+0000 7f12af59e640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 0x7f129c0a5870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12a804f000 tx=0x7f12a8053000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.796+0000 7f12a57fa640 1 -- 172.21.15.77:0/3082956028 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8059070 con 0x5585b7ef8b40 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.796+0000 7f12a57fa640 1 -- 172.21.15.77:0/3082956028 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8036040 con 0x5585b7ef8b40 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.796+0000 7f12a57fa640 1 -- 172.21.15.77:0/3082956028 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8030c40 con 0x5585b7ef8b40 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- 172.21.15.77:0/3082956028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 msgr2=0x5585b7ef7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009430 tx=0x7f12a8031b00 comp rx=0 tx=0).stop 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- 172.21.15.77:0/3082956028 shutdown_connections 2024-09-15T22:25:38.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7f04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7ef7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3082956028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7ef7090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- 172.21.15.77:0/3082956028 >> 172.21.15.77:0/3082956028 conn(0x5585b7ec0310 msgr2=0x5585b7ef3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- 172.21.15.77:0/3082956028 shutdown_connections 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- 172.21.15.77:0/3082956028 wait complete. 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 -- start start 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.797+0000 7f12b75e3740 1 Processor -- start 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7eff010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 0x5585b7eff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5585b7f05010 con 0x5585b7ef8b40 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5585b7e18060 con 0x5585b7f02370 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5585b7f05190 con 0x5585b7ef75d0 2024-09-15T22:25:38.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12b75e3740 10 client.0 _ll_register_callbacks cb 0x5585b7b6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7eff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35646/0 (socket says 172.21.15.77:35646) 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 -- 172.21.15.77:0/3351841065 learned_addr learned my addr 172.21.15.77:0/3351841065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12af59e640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 0x5585b7eff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 -- 172.21.15.77:0/3351841065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 msgr2=0x5585b7eff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 0x5585b7eff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 -- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 msgr2=0x5585b7eff010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7eff010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 -- 172.21.15.77:0/3351841065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b7bce3c0 con 0x5585b7ef75d0 2024-09-15T22:25:38.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12af59e640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 0x5585b7eff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12a7fff640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7eff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12aed9d640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12a00980f0 tx=0x7f12a006a980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8c20440107e261 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.798+0000 7f12967fc640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a00cd4d0 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a8036040 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f12a0007ad0 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f12a0007cd0 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 10 client.27589 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585b7f10b00 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8036040 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f12a009b0e0 con 0x5585b7ef75d0 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 10 client.27589 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:38.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 10 client.27589 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 20 client.27589 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f12280 0x5585b7f14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 4 client.27589 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b7e6a860 con 0x5585b7f12280 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.799+0000 7f12a7fff640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f12280 0x5585b7f14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12a7fff640 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f12280 0x5585b7f14670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14b52fe2b3e15783 server_cookie=6db2bed52d53e156 in_seq=0 out_seq=0 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12967fc640 10 client.27589 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12967fc640 1 -- 172.21.15.77:0/3351841065 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585b7e6a860 con 0x5585b7f12280 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12967fc640 10 client.27589 handle_command_reply: tid=0 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12b75e3740 1 client.27589 shutdown 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 msgr2=0x5585b7f01d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.800+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12a00980f0 tx=0x7f12a006a980 comp rx=0 tx=0).stop 2024-09-15T22:25:38.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 shutdown_connections 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f12280 0x5585b7f14670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7f02370 0x5585b7eff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7eff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 --2- 172.21.15.77:0/3351841065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f01d60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 >> 172.21.15.77:0/3351841065 conn(0x5585b7ec0310 msgr2=0x5585b7ef9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12967fc640 0 client.27589 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 shutdown_connections 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 1 -- 172.21.15.77:0/3351841065 wait complete. 2024-09-15T22:25:38.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.801+0000 7f12b75e3740 20 client.27589 trim_cache size 0 max 16384 2024-09-15T22:25:38.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 Processor -- start 2024-09-15T22:25:38.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 -- start start 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f00f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7efa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ace30 con 0x5585b7f02370 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ae970 con 0x5585b7ef8b40 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.815+0000 7f12b75e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ad500 con 0x5585b7ef75d0 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12af59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7efa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12aed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f00f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38682/0 (socket says 172.21.15.77:38682) 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 -- 172.21.15.77:0/1811288700 learned_addr learned my addr 172.21.15.77:0/1811288700 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 -- 172.21.15.77:0/1811288700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 msgr2=0x5585b7f00f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f00f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 -- 172.21.15.77:0/1811288700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 msgr2=0x5585b7efa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7efa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 -- 172.21.15.77:0/1811288700 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b7f020d0 con 0x5585b7ef8b40 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a7fff640 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1298005fd0 tx=0x7f129800bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe32e31184eb6e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a4ff9640 1 -- 172.21.15.77:0/1811288700 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298004020 con 0x5585b7ef8b40 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a4ff9640 1 -- 172.21.15.77:0/1811288700 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298005500 con 0x5585b7ef8b40 2024-09-15T22:25:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.816+0000 7f12a4ff9640 1 -- 172.21.15.77:0/1811288700 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298004e00 con 0x5585b7ef8b40 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 -- 172.21.15.77:0/1811288700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 msgr2=0x5585b7efa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1298005fd0 tx=0x7f129800bb90 comp rx=0 tx=0).stop 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 -- 172.21.15.77:0/1811288700 shutdown_connections 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7f02370 0x5585b7efa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef8b40 0x5585b7efa120 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 --2- 172.21.15.77:0/1811288700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7ef75d0 0x5585b7f00f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 -- 172.21.15.77:0/1811288700 >> 172.21.15.77:0/1811288700 conn(0x5585b7ec0310 msgr2=0x5585b7f00870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 -- 172.21.15.77:0/1811288700 shutdown_connections 2024-09-15T22:25:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.817+0000 7f12b75e3740 1 -- 172.21.15.77:0/1811288700 wait complete. 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 -- start start 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 Processor -- start 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 0x5585b7f14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7f02370 0x5585b7f7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01ae2a0 con 0x5585b7ef8b40 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01adbd0 con 0x5585b7ef75d0 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01aae30 con 0x5585b7f02370 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12b75e3740 10 client.0 _ll_register_callbacks cb 0x7f12a009b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41182/0 (socket says 172.21.15.77:41182) 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12a7fff640 1 -- 172.21.15.77:0/896881624 learned_addr learned my addr 172.21.15.77:0/896881624 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:38.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12af59e640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7f02370 0x5585b7f7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.818+0000 7f12aed9d640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 0x5585b7f14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 -- 172.21.15.77:0/896881624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7f02370 msgr2=0x5585b7f7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7f02370 0x5585b7f7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 -- 172.21.15.77:0/896881624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 msgr2=0x5585b7f14710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 0x5585b7f14710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 -- 172.21.15.77:0/896881624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b7f7b1a0 con 0x5585b7ef8b40 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12aed9d640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 0x5585b7f14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12a7fff640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1298005940 tx=0x7f1298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2fb2c6fa46e5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12967fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12980052b0 con 0x5585b7ef8b40 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1298005500 con 0x5585b7ef8b40 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f1298004a80 con 0x5585b7ef8b40 2024-09-15T22:25:38.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1298039050 con 0x5585b7ef8b40 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12b75e3740 10 client.18711 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.819+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585b7f7c480 con 0x5585b7ef8b40 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298005500 con 0x5585b7ef8b40 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f129801e470 con 0x5585b7ef8b40 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 10 client.18711 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 10 client.18711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 20 client.18711 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f7cb20 0x5585b7f7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 4 client.18711 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585b7e64d70 con 0x5585b7f7cb20 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12aed9d640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f7cb20 0x5585b7f7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.820+0000 7f12967fc640 10 client.18711 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:25:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12aed9d640 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f7cb20 0x5585b7f7ef40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1623f51fccd4939d server_cookie=63ac87799934041b in_seq=0 out_seq=0 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12967fc640 1 -- 172.21.15.77:0/896881624 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585b7e64d70 con 0x5585b7f7cb20 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12967fc640 10 client.18711 handle_command_reply: tid=0 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12b75e3740 1 client.18711 shutdown 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 msgr2=0x5585b7f14c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.821+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1298005940 tx=0x7f1298004730 comp rx=0 tx=0).stop 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 shutdown_connections 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5585b7f7cb20 0x5585b7f7ef40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12967fc640 0 client.18711 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5585b7f02370 0x5585b7f7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5585b7ef8b40 0x5585b7f14c50 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 --2- 172.21.15.77:0/896881624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5585b7ef75d0 0x5585b7f14710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 >> 172.21.15.77:0/896881624 conn(0x5585b7ec0310 msgr2=0x5585b7f01090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 shutdown_connections 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 1 -- 172.21.15.77:0/896881624 wait complete. 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.822+0000 7f12b75e3740 20 client.18711 trim_cache size 0 max 16384 2024-09-15T22:25:38.829 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 msgr2=0x7f129c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 0x7f129c0a5870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12a804f000 tx=0x7f12a8053000 comp rx=0 tx=0).stop 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 msgr2=0x7f12b0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12a000d950 tx=0x7f12a000de10 comp rx=0 tx=0).stop 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 shutdown_connections 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f129c0a33b0 0x7f129c0a5870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12b01082a0 0x7f12b019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b01036e0 0x7f12b0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 --2- 172.21.15.77:0/821429306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b0102d30 0x7f12b0198540 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 >> 172.21.15.77:0/821429306 conn(0x7f12b00fc7d0 msgr2=0x7f12b0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 shutdown_connections 2024-09-15T22:25:38.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:38.823+0000 7f12b520a640 1 -- 172.21.15.77:0/821429306 wait complete. 2024-09-15T22:25:39.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1026845 7.52 MB/sec execute 588 sec latency 34.669 ms 2024-09-15T22:25:39.548 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:25:39.806 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:40.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.292+0000 7f229414c640 1 -- 172.21.15.77:0/1919150106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 msgr2=0x7f228c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.292+0000 7f229414c640 1 --2- 172.21.15.77:0/1919150106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c1067d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f227c009900 tx=0x7f227c02a410 comp rx=0 tx=0).stop 2024-09-15T22:25:40.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 -- 172.21.15.77:0/1919150106 shutdown_connections 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 --2- 172.21.15.77:0/1919150106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c106f00 0x7f228c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 --2- 172.21.15.77:0/1919150106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c1067d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 --2- 172.21.15.77:0/1919150106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c101dd0 0x7f228c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 -- 172.21.15.77:0/1919150106 >> 172.21.15.77:0/1919150106 conn(0x7f228c0fd890 msgr2=0x7f228c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 -- 172.21.15.77:0/1919150106 shutdown_connections 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 -- 172.21.15.77:0/1919150106 wait complete. 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 Processor -- start 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.293+0000 7f229414c640 1 -- start start 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c101dd0 0x7f228c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111f60 con 0x7f228c106f00 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111de0 con 0x7f228c102780 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1120e0 con 0x7f228c101dd0 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22916e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c101dd0 0x7f228c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53258/0 (socket says 172.21.15.77:53258) 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22916e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41068/0 (socket says 172.21.15.77:41068) 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 -- 172.21.15.77:0/4022459409 learned_addr learned my addr 172.21.15.77:0/4022459409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 -- 172.21.15.77:0/4022459409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c101dd0 msgr2=0x7f228c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c101dd0 0x7f228c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 -- 172.21.15.77:0/4022459409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 msgr2=0x7f228c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 -- 172.21.15.77:0/4022459409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c19d520 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22916e7640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f22926e9640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f228400eb10 tx=0x7f228400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f2282ffd640 1 -- 172.21.15.77:0/4022459409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284004b20 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c19d810 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f2282ffd640 1 -- 172.21.15.77:0/4022459409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2284004cc0 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.294+0000 7f2282ffd640 1 -- 172.21.15.77:0/4022459409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284010760 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.295+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1a50f0 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2282ffd640 1 -- 172.21.15.77:0/4022459409 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2284005020 con 0x7f228c106f00 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 Processor -- start 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 -- start start 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637997292f0 con 0x5637997c35d0 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637996d0cf0 con 0x5637997c4b40 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2296525740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637997252b0 con 0x5637997ce370 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53260/0 (socket says 172.21.15.77:53260) 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f2291ee8640 1 -- 172.21.15.77:0/3226001361 learned_addr learned my addr 172.21.15.77:0/3226001361 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f22916e7640 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.298+0000 7f22916e7640 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2282ffd640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 0x7f22780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 -- 172.21.15.77:0/3226001361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 msgr2=0x5637997d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 -- 172.21.15.77:0/3226001361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 msgr2=0x5637997c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 -- 172.21.15.77:0/3226001361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637996b53a0 con 0x5637997c35d0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2282ffd640 1 -- 172.21.15.77:0/4022459409 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f22840d06f0 con 0x7f228c106f00 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f22926e9640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 0x7f22780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2291ee8640 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2288009cf0 tx=0x7f2288011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f755718837f1547 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2277fff640 1 -- 172.21.15.77:0/3226001361 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228801e070 con 0x5637997c35d0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2277fff640 1 -- 172.21.15.77:0/3226001361 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288009de0 con 0x5637997c35d0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.299+0000 7f2277fff640 1 -- 172.21.15.77:0/3226001361 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288018800 con 0x5637997c35d0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- 172.21.15.77:0/3226001361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 msgr2=0x5637997c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f22926e9640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 0x7f22780a5870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22840ce0b0 tx=0x7f2284005b40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2288009cf0 tx=0x7f2288011810 comp rx=0 tx=0).stop 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- 172.21.15.77:0/3226001361 shutdown_connections 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997c3a10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 --2- 172.21.15.77:0/3226001361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- 172.21.15.77:0/3226001361 >> 172.21.15.77:0/3226001361 conn(0x56379978c310 msgr2=0x5637997bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- 172.21.15.77:0/3226001361 shutdown_connections 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- 172.21.15.77:0/3226001361 wait complete. 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.300+0000 7f2296525740 1 -- start start 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 Processor -- start 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c4b40 0x5637997cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997ce370 0x5637997cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5637997d1070 con 0x5637997ce370 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637996e4060 con 0x5637997c35d0 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637997d11f0 con 0x5637997c4b40 2024-09-15T22:25:40.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2296525740 10 client.0 _ll_register_callbacks cb 0x5637994378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41074/0 (socket says 172.21.15.77:41074) 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 -- 172.21.15.77:0/1417759375 learned_addr learned my addr 172.21.15.77:0/1417759375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c4b40 0x5637997cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997ce370 0x5637997cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 -- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c4b40 msgr2=0x5637997cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c4b40 0x5637997cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 -- 172.21.15.77:0/1417759375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997ce370 msgr2=0x5637997cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997ce370 0x5637997cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.301+0000 7f22916e7640 1 -- 172.21.15.77:0/1417759375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379949a3c0 con 0x5637997c35d0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22916e7640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f227c000c00 tx=0x7f227c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe4e924db79b610 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22757fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:40.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c035040 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f227c005020 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2288009de0 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f227c055050 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f2296525740 10 client.27701 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5637997dcb60 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.302+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288009de0 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f227c0036b0 con 0x5637997c35d0 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 10 client.27701 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 10 client.27701 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 20 client.27701 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5637997de020 0x5637997e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 4 client.27701 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563799736860 con 0x5637997de020 2024-09-15T22:25:40.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5637997de020 0x5637997e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f22757fa640 10 client.27701 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.303+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5637997de020 0x5637997e0410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9eb3f9c9f908648 server_cookie=502d3a7151fc5226 in_seq=0 out_seq=0 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f22757fa640 1 -- 172.21.15.77:0/1417759375 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563799736860 con 0x5637997de020 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f22757fa640 10 client.27701 handle_command_reply: tid=0 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 client.27701 shutdown 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 msgr2=0x5637997cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f227c000c00 tx=0x7f227c009260 comp rx=0 tx=0).stop 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 shutdown_connections 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5637997de020 0x5637997e0410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997ce370 0x5637997cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c4b40 0x5637997cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f22757fa640 0 client.27701 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:25:40.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 --2- 172.21.15.77:0/1417759375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c35d0 0x5637997cddb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 >> 172.21.15.77:0/1417759375 conn(0x56379978c310 msgr2=0x5637997c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.304+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 shutdown_connections 2024-09-15T22:25:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.305+0000 7f2296525740 1 -- 172.21.15.77:0/1417759375 wait complete. 2024-09-15T22:25:40.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.305+0000 7f2296525740 20 client.27701 trim_cache size 0 max 16384 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 Processor -- start 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 -- start start 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997ce370 0x5637997e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1af040 con 0x5637997c4b40 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1b0b80 con 0x5637997ce370 2024-09-15T22:25:40.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2296525740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1af710 con 0x5637997c35d0 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f22916e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53280/0 (socket says 172.21.15.77:53280) 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f2291ee8640 1 -- 172.21.15.77:0/1639452214 learned_addr learned my addr 172.21.15.77:0/1639452214 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.318+0000 7f22916e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55162/0 (socket says 172.21.15.77:55162) 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f22916e7640 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997ce370 0x5637997e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 -- 172.21.15.77:0/1639452214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 msgr2=0x5637997c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 -- 172.21.15.77:0/1639452214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997ce370 msgr2=0x5637997e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997ce370 0x5637997e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 -- 172.21.15.77:0/1639452214 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637997e0a40 con 0x5637997c4b40 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f22916e7640 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:40.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f2291ee8640 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2288018450 tx=0x7f2288009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6743a7af7894505 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f22777fe640 1 -- 172.21.15.77:0/1639452214 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228801e040 con 0x5637997c4b40 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f22777fe640 1 -- 172.21.15.77:0/1639452214 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288009de0 con 0x5637997c4b40 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.319+0000 7f22777fe640 1 -- 172.21.15.77:0/1639452214 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288019bf0 con 0x5637997c4b40 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- 172.21.15.77:0/1639452214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 msgr2=0x5637997c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2288018450 tx=0x7f2288009770 comp rx=0 tx=0).stop 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- 172.21.15.77:0/1639452214 shutdown_connections 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997ce370 0x5637997e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c4b40 0x5637997c6660 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 --2- 172.21.15.77:0/1639452214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997c35d0 0x5637997c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- 172.21.15.77:0/1639452214 >> 172.21.15.77:0/1639452214 conn(0x56379978c310 msgr2=0x5637997baa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- 172.21.15.77:0/1639452214 shutdown_connections 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- 172.21.15.77:0/1639452214 wait complete. 2024-09-15T22:25:40.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 -- start start 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.320+0000 7f2296525740 1 Processor -- start 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1b04b0 con 0x5637997c35d0 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1afde0 con 0x5637997c4b40 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1ad040 con 0x5637997ce370 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2296525740 10 client.0 _ll_register_callbacks cb 0x7f227c031820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f22916e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f22916e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53286/0 (socket says 172.21.15.77:53286) 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f22916e7640 1 -- 172.21.15.77:0/2348473749 learned_addr learned my addr 172.21.15.77:0/2348473749 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:40.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f22916e7640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 -- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 msgr2=0x5637997d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 -- 172.21.15.77:0/2348473749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 msgr2=0x5637997d6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997d6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.321+0000 7f2291ee8640 1 -- 172.21.15.77:0/2348473749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563799847c40 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f2291ee8640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997d2060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5637997db050 tx=0x7f2288009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43765efb12d3315a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228801e040 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2288009de0 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f2288005250 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2288005490 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f2296525740 10 client.27707 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563799848da0 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288009de0 con 0x5637997c4b40 2024-09-15T22:25:40.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.322+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2288003820 con 0x5637997c4b40 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 10 client.27707 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 10 client.27707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 20 client.27707 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563799849270 0x56379984b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 4 client.27707 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563799730d70 con 0x563799849270 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f22916e7640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563799849270 0x56379984b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f22757fa640 10 client.27707 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f22916e7640 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563799849270 0x56379984b660 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae752ecd64d5700b server_cookie=6ec19ea9982a5359 in_seq=0 out_seq=0 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f22757fa640 1 -- 172.21.15.77:0/2348473749 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563799730d70 con 0x563799849270 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f22757fa640 10 client.27707 handle_command_reply: tid=0 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 1 client.27707 shutdown 2024-09-15T22:25:40.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 msgr2=0x5637997d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.323+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997d2060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5637997db050 tx=0x7f2288009390 comp rx=0 tx=0).stop 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 shutdown_connections 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563799849270 0x56379984b660 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637997ce370 0x5637997d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637997c4b40 0x5637997d2060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f22757fa640 0 client.27707 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 --2- 172.21.15.77:0/2348473749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5637997c35d0 0x5637997d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 >> 172.21.15.77:0/2348473749 conn(0x56379978c310 msgr2=0x5637997be4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 shutdown_connections 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 1 -- 172.21.15.77:0/2348473749 wait complete. 2024-09-15T22:25:40.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f2296525740 20 client.27707 trim_cache size 0 max 16384 2024-09-15T22:25:40.336 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:25:40.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 msgr2=0x7f22780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 0x7f22780a5870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f22840ce0b0 tx=0x7f2284005b40 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 msgr2=0x7f228c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.324+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f228400eb10 tx=0x7f228400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 shutdown_connections 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22780a33b0 0x7f22780a5870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f228c106f00 0x7f228c19ce40 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c102780 0x7f228c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 --2- 172.21.15.77:0/4022459409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f228c101dd0 0x7f228c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 >> 172.21.15.77:0/4022459409 conn(0x7f228c0fd890 msgr2=0x7f228c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 shutdown_connections 2024-09-15T22:25:40.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:40.325+0000 7f229414c640 1 -- 172.21.15.77:0/4022459409 wait complete. 2024-09-15T22:25:40.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1028383 7.52 MB/sec execute 589 sec latency 28.621 ms 2024-09-15T22:25:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:40 smithi077 ceph-mon[30980]: pgmap v414: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-09-15T22:25:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:40 smithi196 ceph-mon[35109]: pgmap v414: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-09-15T22:25:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:40 smithi117 ceph-mon[35021]: pgmap v414: 129 pgs: 129 active+clean; 610 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-09-15T22:25:41.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:25:41.367 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:41.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1030339 7.52 MB/sec execute 590 sec latency 85.185 ms 2024-09-15T22:25:41.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 -- 172.21.15.77:0/1617599913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90100e20 msgr2=0x7f3a90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1617599913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90100e20 0x7f3a90101200 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009900 tx=0x7f3a8c02a480 comp rx=0 tx=0).stop 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 -- 172.21.15.77:0/1617599913 shutdown_connections 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1617599913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90102ef0 0x7f3a9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1617599913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102570 0x7f3a901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1617599913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90100e20 0x7f3a90101200 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 -- 172.21.15.77:0/1617599913 >> 172.21.15.77:0/1617599913 conn(0x7f3a900fc7d0 msgr2=0x7f3a900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 -- 172.21.15.77:0/1617599913 shutdown_connections 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 -- 172.21.15.77:0/1617599913 wait complete. 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.880+0000 7f3a982e8640 1 Processor -- start 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 -- start start 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 0x7f3a901993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 0x7f3a901a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90110f80 con 0x7f3a90102570 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90110e00 con 0x7f3a90102ef0 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a982e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90111100 con 0x7f3a90100e20 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a96084640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 0x7f3a901993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a96084640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 0x7f3a901993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55190/0 (socket says 172.21.15.77:55190) 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53306/0 (socket says 172.21.15.77:53306) 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a96084640 1 -- 172.21.15.77:0/1957196819 learned_addr learned my addr 172.21.15.77:0/1957196819 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a96885640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 0x7f3a901a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 -- 172.21.15.77:0/1957196819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 msgr2=0x7f3a901993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 0x7f3a901993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 -- 172.21.15.77:0/1957196819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 msgr2=0x7f3a901a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 0x7f3a901a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 -- 172.21.15.77:0/1957196819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a901a4990 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a96885640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 0x7f3a901a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.881+0000 7f3a95883640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e9f0 tx=0x7f3a8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.882+0000 7f3a877fe640 1 -- 172.21.15.77:0/1957196819 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000ccc0 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.882+0000 7f3a877fe640 1 -- 172.21.15.77:0/1957196819 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a80010910 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.882+0000 7f3a877fe640 1 -- 172.21.15.77:0/1957196819 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80018960 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.882+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a901a4b60 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.882+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901a50a0 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.884+0000 7f3a877fe640 1 -- 172.21.15.77:0/1957196819 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3a80010ac0 con 0x7f3a90102570 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.884+0000 7f3a9a6c1740 1 Processor -- start 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 -- start start 2024-09-15T22:25:41.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 0x562e379a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e3790f2f0 con 0x562e379aab40 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e378b6cf0 con 0x562e379a95d0 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e3790b2b0 con 0x562e379b4370 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96885640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96885640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53314/0 (socket says 172.21.15.77:53314) 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96885640 1 -- 172.21.15.77:0/1109811620 learned_addr learned my addr 172.21.15.77:0/1109811620 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 0x562e379a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a95883640 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a877fe640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 0x7f3a7c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 -- 172.21.15.77:0/1109811620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 msgr2=0x562e379b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 -- 172.21.15.77:0/1109811620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 msgr2=0x562e379a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96084640 1 -- 172.21.15.77:0/1109811620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3789b3a0 con 0x562e379a95d0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.885+0000 7f3a96885640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 0x7f3a7c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.886+0000 7f3a877fe640 1 -- 172.21.15.77:0/1957196819 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f3a800d03c0 con 0x7f3a90102570 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.886+0000 7f3a96084640 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 0x562e379a9a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009230 tx=0x7f3a8c005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4a73841ca29fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.886+0000 7f3a84dea640 1 -- 172.21.15.77:0/1109811620 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c040070 con 0x562e379a95d0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.886+0000 7f3a84dea640 1 -- 172.21.15.77:0/1109811620 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c004480 con 0x562e379a95d0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.886+0000 7f3a84dea640 1 -- 172.21.15.77:0/1109811620 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c035040 con 0x562e379a95d0 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/1109811620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 msgr2=0x562e379a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 0x562e379a9a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009230 tx=0x7f3a8c005e10 comp rx=0 tx=0).stop 2024-09-15T22:25:41.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a96885640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 0x7f3a7c0a58f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009fd0 tx=0x7f3a88004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/1109811620 shutdown_connections 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379a95d0 0x562e379a9a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/1109811620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/1109811620 >> 172.21.15.77:0/1109811620 conn(0x562e37972310 msgr2=0x562e379a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/1109811620 shutdown_connections 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/1109811620 wait complete. 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 -- start start 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.887+0000 7f3a9a6c1740 1 Processor -- start 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379a95d0 0x562e379b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e379b7010 con 0x562e379a95d0 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e378ca060 con 0x562e379aab40 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e379b7190 con 0x562e379b4370 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a9a6c1740 10 client.0 _ll_register_callbacks cb 0x562e3761d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a96084640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379a95d0 0x562e379b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a96885640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41128/0 (socket says 172.21.15.77:41128) 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 -- 172.21.15.77:0/2421110165 learned_addr learned my addr 172.21.15.77:0/2421110165 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 -- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 msgr2=0x562e379b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 -- 172.21.15.77:0/2421110165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379a95d0 msgr2=0x562e379b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379a95d0 0x562e379b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 -- 172.21.15.77:0/2421110165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e376803c0 con 0x562e379aab40 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a95883640 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c350 tx=0x7f3a8000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d79f2836c59c466 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.888+0000 7f3a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009bf00 con 0x562e379aab40 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f3a800bc2e0 con 0x562e379aab40 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8c004480 con 0x562e379aab40 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a8009bd20 con 0x562e379aab40 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 10 client.27719 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:41.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e379c2a70 con 0x562e379aab40 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c004480 con 0x562e379aab40 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3a8009b810 con 0x562e379aab40 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 10 client.27719 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 10 client.27719 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 20 client.27719 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e379c3e80 0x562e379c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 4 client.27719 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.889+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e3791c860 con 0x562e379c3e80 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.890+0000 7f3a96084640 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e379c3e80 0x562e379c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.890+0000 7f3a96084640 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e379c3e80 0x562e379c6300 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebdd833808315a43 server_cookie=38bdf4569af1a201 in_seq=0 out_seq=0 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.890+0000 7f3a79ffb640 10 client.27719 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:25:41.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2421110165 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3a8c050e80 con 0x562e379c3e80 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a79ffb640 10 client.27719 handle_command_reply: tid=0 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 client.27719 shutdown 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 msgr2=0x562e379b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c350 tx=0x7f3a8000cf60 comp rx=0 tx=0).stop 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 shutdown_connections 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e379c3e80 0x562e379c6300 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379b4370 0x562e379b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379b0fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2421110165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379a95d0 0x562e379b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a79ffb640 0 client.27719 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 >> 172.21.15.77:0/2421110165 conn(0x562e37972310 msgr2=0x562e379ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 shutdown_connections 2024-09-15T22:25:41.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2421110165 wait complete. 2024-09-15T22:25:41.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.891+0000 7f3a9a6c1740 20 client.27719 trim_cache size 0 max 16384 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 Processor -- start 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 -- start start 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379b4370 0x562e379b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ace50 con 0x562e379aab40 2024-09-15T22:25:41.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ae990 con 0x562e379b4370 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ad520 con 0x562e379a95d0 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a95883640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a95883640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53340/0 (socket says 172.21.15.77:53340) 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.912+0000 7f3a95883640 1 -- 172.21.15.77:0/487975284 learned_addr learned my addr 172.21.15.77:0/487975284 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a96084640 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a96885640 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379b4370 0x562e379b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 -- 172.21.15.77:0/487975284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 msgr2=0x562e379c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 -- 172.21.15.77:0/487975284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379b4370 msgr2=0x562e379b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379b4370 0x562e379b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 -- 172.21.15.77:0/487975284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e379b3950 con 0x562e379aab40 2024-09-15T22:25:41.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.913+0000 7f3a95883640 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3a80098820 tx=0x7f3a80010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c556203f9d692bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.914+0000 7f3a7bfff640 1 -- 172.21.15.77:0/487975284 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009c3c0 con 0x562e379aab40 2024-09-15T22:25:41.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.914+0000 7f3a7bfff640 1 -- 172.21.15.77:0/487975284 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800a06c0 con 0x562e379aab40 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.914+0000 7f3a7bfff640 1 -- 172.21.15.77:0/487975284 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800bc0c0 con 0x562e379aab40 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.915+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/487975284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 msgr2=0x562e379aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.915+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3a80098820 tx=0x7f3a80010ef0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/487975284 shutdown_connections 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379b4370 0x562e379b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379aab40 0x562e379aa300 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/487975284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/487975284 >> 172.21.15.77:0/487975284 conn(0x562e37972310 msgr2=0x562e379a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/487975284 shutdown_connections 2024-09-15T22:25:41.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.916+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/487975284 wait complete. 2024-09-15T22:25:41.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.917+0000 7f3a9a6c1740 1 -- start start 2024-09-15T22:25:41.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.917+0000 7f3a9a6c1740 1 Processor -- start 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 0x562e379b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901ae2c0 con 0x562e379b4370 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901adbf0 con 0x562e379aab40 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901aae50 con 0x562e379a95d0 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a9a6c1740 10 client.0 _ll_register_callbacks cb 0x7f3a800d3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a95883640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a95883640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379bcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41142/0 (socket says 172.21.15.77:41142) 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a95883640 1 -- 172.21.15.77:0/2387053356 learned_addr learned my addr 172.21.15.77:0/2387053356 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:41.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a96885640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 0x562e379b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.918+0000 7f3a96084640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 -- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 msgr2=0x562e379bc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379bc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 -- 172.21.15.77:0/2387053356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 msgr2=0x562e379bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379bcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 -- 172.21.15.77:0/2387053356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e379b82f0 con 0x562e379b4370 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96084640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379bc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a96885640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 0x562e379b7ac0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x562e379b7240 tx=0x7f3a88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11b9342ae5751c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88032060 con 0x562e379b4370 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f3a8800e880 con 0x562e379b4370 2024-09-15T22:25:41.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a800a06c0 con 0x562e379b4370 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.919+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3a88039050 con 0x562e379b4370 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 10 client.18747 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e37a36fc0 con 0x562e379b4370 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800a06c0 con 0x562e379b4370 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3a8802c070 con 0x562e379b4370 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 10 client.18747 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 10 client.18747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 20 client.18747 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e37a376e0 0x562e37a39b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 4 client.18747 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e37916d70 con 0x562e37a376e0 2024-09-15T22:25:41.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.920+0000 7f3a96084640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e37a376e0 0x562e37a39b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:41.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.921+0000 7f3a79ffb640 10 client.18747 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:25:41.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.921+0000 7f3a96084640 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e37a376e0 0x562e37a39b00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67e6fd46bc61fcb3 server_cookie=bb1b14c8f179aa37 in_seq=0 out_seq=0 2024-09-15T22:25:41.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.925+0000 7f3a79ffb640 1 -- 172.21.15.77:0/2387053356 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e37916d70 con 0x562e37a376e0 2024-09-15T22:25:41.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.925+0000 7f3a79ffb640 10 client.18747 handle_command_reply: tid=0 2024-09-15T22:25:41.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.925+0000 7f3a9a6c1740 1 client.18747 shutdown 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 msgr2=0x562e379b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 0x562e379b7ac0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x562e379b7240 tx=0x7f3a88004ab0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 shutdown_connections 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x562e37a376e0 0x562e37a39b00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e379b4370 0x562e379b7ac0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e379aab40 0x562e379bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 --2- 172.21.15.77:0/2387053356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e379a95d0 0x562e379bc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 >> 172.21.15.77:0/2387053356 conn(0x562e37972310 msgr2=0x562e379b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a79ffb640 0 client.18747 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:25:41.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 shutdown_connections 2024-09-15T22:25:41.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.929+0000 7f3a9a6c1740 1 -- 172.21.15.77:0/2387053356 wait complete. 2024-09-15T22:25:41.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.930+0000 7f3a9a6c1740 20 client.18747 trim_cache size 0 max 16384 2024-09-15T22:25:41.932 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:25:41.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.931+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 msgr2=0x7f3a7c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.931+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 0x7f3a7c0a58f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009fd0 tx=0x7f3a88004a40 comp rx=0 tx=0).stop 2024-09-15T22:25:41.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.931+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 msgr2=0x7f3a90199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.931+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e9f0 tx=0x7f3a8000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 shutdown_connections 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3a7c0a3430 0x7f3a7c0a58f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a90102ef0 0x7f3a901a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a90102570 0x7f3a90199910 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 --2- 172.21.15.77:0/1957196819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a90100e20 0x7f3a901993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 >> 172.21.15.77:0/1957196819 conn(0x7f3a900fc7d0 msgr2=0x7f3a900fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 shutdown_connections 2024-09-15T22:25:41.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:41.932+0000 7f3a982e8640 1 -- 172.21.15.77:0/1957196819 wait complete. 2024-09-15T22:25:42.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1031656 7.52 MB/sec execute 591 sec latency 36.810 ms 2024-09-15T22:25:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:42 smithi077 ceph-mon[30980]: pgmap v415: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 266 op/s 2024-09-15T22:25:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:42 smithi196 ceph-mon[35109]: pgmap v415: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 266 op/s 2024-09-15T22:25:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:42 smithi117 ceph-mon[35021]: pgmap v415: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 266 op/s 2024-09-15T22:25:42.651 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:25:42.915 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:43.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 -- 172.21.15.77:0/3556618688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310102790 msgr2=0x7fd3101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 --2- 172.21.15.77:0/3556618688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310102790 0x7fd3101067e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd30002a410 comp rx=0 tx=0).stop 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 -- 172.21.15.77:0/3556618688 shutdown_connections 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 --2- 172.21.15.77:0/3556618688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310106f10 0x7fd31010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 --2- 172.21.15.77:0/3556618688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310102790 0x7fd3101067e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 --2- 172.21.15.77:0/3556618688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310101de0 0x7fd3101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 -- 172.21.15.77:0/3556618688 >> 172.21.15.77:0/3556618688 conn(0x7fd3100fd8c0 msgr2=0x7fd3100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 -- 172.21.15.77:0/3556618688 shutdown_connections 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.411+0000 7fd317bea640 1 -- 172.21.15.77:0/3556618688 wait complete. 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 Processor -- start 2024-09-15T22:25:43.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 -- start start 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310102790 0x7fd310198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310106f10 0x7fd31019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd310111f70 con 0x7fd310101de0 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd310111df0 con 0x7fd310106f10 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd317bea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101120f0 con 0x7fd310102790 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd315986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd315986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53360/0 (socket says 172.21.15.77:53360) 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd315986640 1 -- 172.21.15.77:0/295245553 learned_addr learned my addr 172.21.15.77:0/295245553 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.412+0000 7fd315185640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310102790 0x7fd310198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd316187640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310106f10 0x7fd31019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 -- 172.21.15.77:0/295245553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310102790 msgr2=0x7fd310198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310102790 0x7fd310198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 -- 172.21.15.77:0/295245553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310106f10 msgr2=0x7fd31019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310106f10 0x7fd31019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 -- 172.21.15.77:0/295245553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31019d400 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd315986640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00d950 tx=0x7fd30c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd306ffd640 1 -- 172.21.15.77:0/295245553 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c017030 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd306ffd640 1 -- 172.21.15.77:0/295245553 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c004020 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31019d6f0 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd306ffd640 1 -- 172.21.15.77:0/295245553 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0047a0 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.413+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a4fd0 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd306ffd640 1 -- 172.21.15.77:0/295245553 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd30c0041d0 con 0x7fd310101de0 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 Processor -- start 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 -- start start 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 0x5635ddb6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5635ddad32f0 con 0x5635ddb6d5d0 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5635dda7acf0 con 0x5635ddb78370 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd319fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635ddacf2b0 con 0x5635ddb6eb40 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd315986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 0x5635ddb6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd316187640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd306ffd640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 0x7fd2fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41166/0 (socket says 172.21.15.77:41166) 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd315185640 1 -- 172.21.15.77:0/1325123102 learned_addr learned my addr 172.21.15.77:0/1325123102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd316187640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb6d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55278/0 (socket says 172.21.15.77:55278) 2024-09-15T22:25:43.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.416+0000 7fd315986640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 0x5635ddb6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53372/0 (socket says 172.21.15.77:53372) 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd316187640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 0x7fd2fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 -- 172.21.15.77:0/1325123102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 msgr2=0x5635ddb6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 -- 172.21.15.77:0/1325123102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 msgr2=0x5635ddb6da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 0x5635ddb6da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 -- 172.21.15.77:0/1325123102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635dda5f3a0 con 0x5635ddb78370 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd306ffd640 1 -- 172.21.15.77:0/295245553 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fd30c0cf9c0 con 0x7fd310101de0 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd315185640 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd300005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3fc40f4bc2f99c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd316187640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 0x7fd2fc0a5870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308004ab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd2fbfff640 1 -- 172.21.15.77:0/1325123102 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300040070 con 0x5635ddb78370 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd2fbfff640 1 -- 172.21.15.77:0/1325123102 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3000318f0 con 0x5635ddb78370 2024-09-15T22:25:43.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.417+0000 7fd2fbfff640 1 -- 172.21.15.77:0/1325123102 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300035040 con 0x5635ddb78370 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.418+0000 7fd319fc3740 1 -- 172.21.15.77:0/1325123102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 msgr2=0x5635ddb7a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.418+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd300005e00 comp rx=0 tx=0).stop 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 -- 172.21.15.77:0/1325123102 shutdown_connections 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb78370 0x5635ddb7a760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb6d5d0 0x5635ddb6da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1325123102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 -- 172.21.15.77:0/1325123102 >> 172.21.15.77:0/1325123102 conn(0x5635ddb36310 msgr2=0x5635ddb69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 -- 172.21.15.77:0/1325123102 shutdown_connections 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 -- 172.21.15.77:0/1325123102 wait complete. 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 -- start start 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.419+0000 7fd319fc3740 1 Processor -- start 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5635ddb7b010 con 0x5635ddb75c50 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5635dda8e060 con 0x5635ddb6eb40 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635ddb7b190 con 0x5635ddb6d5d0 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd319fc3740 10 client.0 _ll_register_callbacks cb 0x5635dd7e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41170/0 (socket says 172.21.15.77:41170) 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 -- 172.21.15.77:0/2363761126 learned_addr learned my addr 172.21.15.77:0/2363761126 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd316187640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 -- 172.21.15.77:0/2363761126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 msgr2=0x5635ddb751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 -- 172.21.15.77:0/2363761126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 msgr2=0x5635ddb844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 -- 172.21.15.77:0/2363761126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635dd8443c0 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd315185640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd30002a930 tx=0x7fd300005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa67bdd6585974c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.420+0000 7fd2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30004b070 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3000318f0 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fd300004250 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd300057050 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 10 client.27737 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5635ddb87040 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3000318f0 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd3000035f0 con 0x5635ddb6eb40 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 10 client.27737 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 10 client.27737 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 20 client.27737 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:43.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddb88390 0x5635ddb8a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 4 client.27737 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:43.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ddae0860 con 0x5635ddb88390 2024-09-15T22:25:43.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.421+0000 7fd315986640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddb88390 0x5635ddb8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.422+0000 7fd2f97fa640 10 client.27737 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:25:43.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.422+0000 7fd315986640 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddb88390 0x5635ddb8a780 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d89d7daa34d80178 server_cookie=a92bb737c5671742 in_seq=0 out_seq=0 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2363761126 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635ddae0860 con 0x5635ddb88390 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd2f97fa640 10 client.27737 handle_command_reply: tid=0 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 client.27737 shutdown 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 msgr2=0x5635ddb75710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd30002a930 tx=0x7fd300005230 comp rx=0 tx=0).stop 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 shutdown_connections 2024-09-15T22:25:43.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddb88390 0x5635ddb8a780 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb75710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2363761126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 >> 172.21.15.77:0/2363761126 conn(0x5635ddb36310 msgr2=0x5635ddb6f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd2f97fa640 0 client.27737 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 shutdown_connections 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 1 -- 172.21.15.77:0/2363761126 wait complete. 2024-09-15T22:25:43.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.423+0000 7fd319fc3740 20 client.27737 trim_cache size 0 max 16384 2024-09-15T22:25:43.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 Processor -- start 2024-09-15T22:25:43.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 -- start start 2024-09-15T22:25:43.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 0x5635ddb76a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb70120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb70660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101af000 con 0x5635ddb75c50 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101b0b40 con 0x5635ddb6d5d0 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd319fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101af6d0 con 0x5635ddb6eb40 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd315986640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 0x5635ddb76a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd316187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb70660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd315185640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb70120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd316187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb70660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53384/0 (socket says 172.21.15.77:53384) 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.438+0000 7fd316187640 1 -- 172.21.15.77:0/1962987482 learned_addr learned my addr 172.21.15.77:0/1962987482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 -- 172.21.15.77:0/1962987482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 msgr2=0x5635ddb70120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb70120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 -- 172.21.15.77:0/1962987482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 msgr2=0x5635ddb70660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb70660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 -- 172.21.15.77:0/1962987482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ddb778e0 con 0x5635ddb6d5d0 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd315986640 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 0x5635ddb76a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0ccd80 tx=0x7fd30c0d1ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bb87ae2ca6987b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd2fb7fe640 1 -- 172.21.15.77:0/1962987482 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0860b0 con 0x5635ddb6d5d0 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd2fb7fe640 1 -- 172.21.15.77:0/1962987482 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c09c010 con 0x5635ddb6d5d0 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.439+0000 7fd2fb7fe640 1 -- 172.21.15.77:0/1962987482 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c097f80 con 0x5635ddb6d5d0 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- 172.21.15.77:0/1962987482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 msgr2=0x5635ddb76a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 0x5635ddb76a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0ccd80 tx=0x7fd30c0d1ed0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- 172.21.15.77:0/1962987482 shutdown_connections 2024-09-15T22:25:43.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb70660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6eb40 0x5635ddb70120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 --2- 172.21.15.77:0/1962987482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6d5d0 0x5635ddb76a30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- 172.21.15.77:0/1962987482 >> 172.21.15.77:0/1962987482 conn(0x5635ddb36310 msgr2=0x5635ddb74190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- 172.21.15.77:0/1962987482 shutdown_connections 2024-09-15T22:25:43.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- 172.21.15.77:0/1962987482 wait complete. 2024-09-15T22:25:43.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 -- start start 2024-09-15T22:25:43.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.440+0000 7fd319fc3740 1 Processor -- start 2024-09-15T22:25:43.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb7c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101b0470 con 0x5635ddb75c50 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101afda0 con 0x5635ddb6eb40 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101ad000 con 0x5635ddb6d5d0 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd319fc3740 10 client.0 _ll_register_callbacks cb 0x7fd30c09b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd316187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb7c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55310/0 (socket says 172.21.15.77:55310) 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315185640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41188/0 (socket says 172.21.15.77:41188) 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 -- 172.21.15.77:0/2770718508 learned_addr learned my addr 172.21.15.77:0/2770718508 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 -- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 msgr2=0x5635ddb7bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 -- 172.21.15.77:0/2770718508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 msgr2=0x5635ddb7c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb7c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 -- 172.21.15.77:0/2770718508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ddb7cc50 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315185640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd315986640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd30c004770 tx=0x7fd30c005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7075c9bac848bca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.441+0000 7fd2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0b3070 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c09c010 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fd30c097f80 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd30c09b910 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 10 client.27649 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5635ddbf2d80 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c09c010 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd30c0b6be0 con 0x5635ddb6d5d0 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 10 client.27649 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:43.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 10 client.27649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 20 client.27649 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddbf34a0 0x5635ddbf58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 4 client.27649 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635ddadad70 con 0x5635ddbf34a0 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.442+0000 7fd315185640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddbf34a0 0x5635ddbf58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd2f97fa640 10 client.27649 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd315185640 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddbf34a0 0x5635ddbf58c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd7c5ceea986ab0 server_cookie=908655ca04bd8a2e in_seq=0 out_seq=0 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd2f97fa640 1 -- 172.21.15.77:0/2770718508 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635ddadad70 con 0x5635ddbf34a0 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd2f97fa640 10 client.27649 handle_command_reply: tid=0 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 client.27649 shutdown 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 msgr2=0x5635ddb80b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd30c004770 tx=0x7fd30c005ab0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 shutdown_connections 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5635ddbf34a0 0x5635ddbf58c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635ddb75c50 0x5635ddb7c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635ddb6eb40 0x5635ddb7bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 --2- 172.21.15.77:0/2770718508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635ddb6d5d0 0x5635ddb80b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 >> 172.21.15.77:0/2770718508 conn(0x5635ddb36310 msgr2=0x5635ddb684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd2f97fa640 0 client.27649 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 shutdown_connections 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 1 -- 172.21.15.77:0/2770718508 wait complete. 2024-09-15T22:25:43.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.443+0000 7fd319fc3740 20 client.27649 trim_cache size 0 max 16384 2024-09-15T22:25:43.453 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 msgr2=0x7fd2fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 0x7fd2fc0a5870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308004ab0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 msgr2=0x7fd310198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00d950 tx=0x7fd30c00de10 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 shutdown_connections 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd2fc0a33b0 0x7fd2fc0a5870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd310106f10 0x7fd31019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd310102790 0x7fd310198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 --2- 172.21.15.77:0/295245553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd310101de0 0x7fd310198450 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:43.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 >> 172.21.15.77:0/295245553 conn(0x7fd3100fd8c0 msgr2=0x7fd3100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:43.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 shutdown_connections 2024-09-15T22:25:43.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:43.444+0000 7fd317bea640 1 -- 172.21.15.77:0/295245553 wait complete. 2024-09-15T22:25:43.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1033864 7.53 MB/sec execute 592 sec latency 19.304 ms 2024-09-15T22:25:44.160 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:25:44.422 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:44.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1035418 7.53 MB/sec execute 593 sec latency 14.346 ms 2024-09-15T22:25:44.544 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:44 smithi077 ceph-mon[30980]: pgmap v416: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:44 smithi196 ceph-mon[35109]: pgmap v416: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:44 smithi117 ceph-mon[35021]: pgmap v416: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:44.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.901+0000 7efd16ba0640 1 -- 172.21.15.77:0/2352501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10102570 msgr2=0x7efd101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.902+0000 7efd16ba0640 1 --2- 172.21.15.77:0/2352501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10102570 0x7efd101029b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009900 tx=0x7efcfc02a480 comp rx=0 tx=0).stop 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.903+0000 7efd16ba0640 1 -- 172.21.15.77:0/2352501 shutdown_connections 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.903+0000 7efd16ba0640 1 --2- 172.21.15.77:0/2352501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 0x7efd1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.903+0000 7efd16ba0640 1 --2- 172.21.15.77:0/2352501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10102570 0x7efd101029b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.903+0000 7efd16ba0640 1 --2- 172.21.15.77:0/2352501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10100e20 0x7efd10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.903+0000 7efd16ba0640 1 -- 172.21.15.77:0/2352501 >> 172.21.15.77:0/2352501 conn(0x7efd100fc7d0 msgr2=0x7efd100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.904+0000 7efd16ba0640 1 -- 172.21.15.77:0/2352501 shutdown_connections 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.905+0000 7efd16ba0640 1 -- 172.21.15.77:0/2352501 wait complete. 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.905+0000 7efd16ba0640 1 Processor -- start 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.905+0000 7efd16ba0640 1 -- start start 2024-09-15T22:25:44.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10100e20 0x7efd10107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 0x7efd101084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd10110f50 con 0x7efd10100e20 2024-09-15T22:25:44.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd10110dd0 con 0x7efd10102ef0 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd16ba0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd101110d0 con 0x7efd10102570 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55318/0 (socket says 172.21.15.77:55318) 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd0ffff640 1 -- 172.21.15.77:0/973239494 learned_addr learned my addr 172.21.15.77:0/973239494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.906+0000 7efd1513d640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 0x7efd101084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 -- 172.21.15.77:0/973239494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 msgr2=0x7efd101084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 0x7efd101084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 -- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10100e20 msgr2=0x7efd10107a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd1493c640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10100e20 0x7efd10107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10100e20 0x7efd10107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 -- 172.21.15.77:0/973239494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd101aad80 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.907+0000 7efd0ffff640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000c00 tx=0x7efcfc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.908+0000 7efd0dffb640 1 -- 172.21.15.77:0/973239494 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc03f070 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.908+0000 7efd0dffb640 1 -- 172.21.15.77:0/973239494 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcfc031c20 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.908+0000 7efd0dffb640 1 -- 172.21.15.77:0/973239494 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc035030 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.909+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd101ab010 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.909+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd101ab4a0 con 0x7efd10102570 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.912+0000 7efd18f79740 1 Processor -- start 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.912+0000 7efd18f79740 1 -- start start 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b98683a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b986a1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b985e92f0 con 0x555b986835d0 2024-09-15T22:25:44.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b98590cf0 con 0x555b98684b40 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd18f79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b985e52b0 con 0x555b9869f380 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1493c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b98683a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41210/0 (socket says 172.21.15.77:41210) 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 -- 172.21.15.77:0/2331145829 learned_addr learned my addr 172.21.15.77:0/2331145829 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 -- 172.21.15.77:0/2331145829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 msgr2=0x555b986a1770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b986a1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 -- 172.21.15.77:0/2331145829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 msgr2=0x555b98683a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b98683a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1513d640 1 -- 172.21.15.77:0/2331145829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b985753a0 con 0x555b98684b40 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.913+0000 7efd1493c640 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b98683a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.914+0000 7efd1513d640 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd08010370 tx=0x7efd08010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc1f81e900a089a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.914+0000 7efd0dffb640 1 -- 172.21.15.77:0/973239494 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efcfc042070 con 0x7efd10102570 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.914+0000 7efcfaffd640 1 -- 172.21.15.77:0/2331145829 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08016070 con 0x555b98684b40 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.914+0000 7efcfaffd640 1 -- 172.21.15.77:0/2331145829 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd08004d20 con 0x555b98684b40 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.914+0000 7efcfaffd640 1 -- 172.21.15.77:0/2331145829 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080054e0 con 0x555b98684b40 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd0dffb640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 0x7efd000a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd0dffb640 1 -- 172.21.15.77:0/973239494 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7efcfc0f0860 con 0x7efd10102570 2024-09-15T22:25:44.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd1493c640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 0x7efd000a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd18f79740 1 -- 172.21.15.77:0/2331145829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 msgr2=0x555b98683090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd18f79740 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd08010370 tx=0x7efd08010830 comp rx=0 tx=0).stop 2024-09-15T22:25:44.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.915+0000 7efd18f79740 1 -- 172.21.15.77:0/2331145829 shutdown_connections 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b986a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b98683a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 --2- 172.21.15.77:0/2331145829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b98684b40 0x555b98683090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 -- 172.21.15.77:0/2331145829 >> 172.21.15.77:0/2331145829 conn(0x555b9864c310 msgr2=0x555b9867fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 -- 172.21.15.77:0/2331145829 shutdown_connections 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 -- 172.21.15.77:0/2331145829 wait complete. 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 -- start start 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd1493c640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 0x7efd000a5a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555b98680ac0 tx=0x7efd04005ed0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.916+0000 7efd18f79740 1 Processor -- start 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b9869ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b98684b40 0x555b9869c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b9869c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b986a2120 con 0x555b98684b40 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b985a4060 con 0x555b986835d0 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1493c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b9869ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b986a22a0 con 0x555b9869f380 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1493c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b9869ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41218/0 (socket says 172.21.15.77:41218) 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1493c640 1 -- 172.21.15.77:0/3493265100 learned_addr learned my addr 172.21.15.77:0/3493265100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b9869c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd18f79740 10 client.0 _ll_register_callbacks cb 0x555b982f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd0ffff640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b98684b40 0x555b9869c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 -- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 msgr2=0x555b9869ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b9869ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 -- 172.21.15.77:0/3493265100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b98684b40 msgr2=0x555b9869c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b98684b40 0x555b9869c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.917+0000 7efd1513d640 1 -- 172.21.15.77:0/3493265100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b9835a3c0 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.918+0000 7efd1513d640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b9869c650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd08010370 tx=0x7efd08003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a06a8974e3459c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.918+0000 7efceffff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.918+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08016070 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.918+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7efd08004040 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.918+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd08004d20 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.919+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efd08004240 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.919+0000 7efd18f79740 10 client.27658 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.919+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b986adba0 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd08004d20 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efd0800f040 con 0x555b9869f380 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 10 client.27658 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 10 client.27658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:44.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 20 client.27658 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b986b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 4 client.27658 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b985f6860 con 0x555b986aeef0 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.920+0000 7efd1493c640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b986b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.921+0000 7efceffff640 10 client.27658 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.921+0000 7efd1493c640 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b986b12e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5da68da54cce663 server_cookie=60e2e1391ad6f11a in_seq=0 out_seq=0 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.922+0000 7efceffff640 1 -- 172.21.15.77:0/3493265100 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b985f6860 con 0x555b986aeef0 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.922+0000 7efceffff640 10 client.27658 handle_command_reply: tid=0 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.922+0000 7efd18f79740 1 client.27658 shutdown 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.923+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 msgr2=0x555b9869c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.923+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b9869c650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd08010370 tx=0x7efd08003ba0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 shutdown_connections 2024-09-15T22:25:44.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b986b12e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b9869f380 0x555b9869c650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efceffff640 0 client.27658 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b98684b40 0x555b9869c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 --2- 172.21.15.77:0/3493265100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b9869ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 >> 172.21.15.77:0/3493265100 conn(0x555b9864c310 msgr2=0x555b98685930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 shutdown_connections 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.924+0000 7efd18f79740 1 -- 172.21.15.77:0/3493265100 wait complete. 2024-09-15T22:25:44.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.925+0000 7efd18f79740 20 client.27658 trim_cache size 0 max 16384 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.946+0000 7efd18f79740 1 Processor -- start 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.946+0000 7efd18f79740 1 -- start start 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.946+0000 7efd18f79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.946+0000 7efd18f79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b9867fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd18f79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9869f380 0x555b986b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd18f79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd101acf50 con 0x555b9869f380 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd18f79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd101aea90 con 0x555b986835d0 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd18f79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd101ad620 con 0x555b98684b40 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1513d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9869f380 0x555b986b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41220/0 (socket says 172.21.15.77:41220) 2024-09-15T22:25:44.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 -- 172.21.15.77:0/1457586558 learned_addr learned my addr 172.21.15.77:0/1457586558 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd0ffff640 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b9867fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 -- 172.21.15.77:0/1457586558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 msgr2=0x555b9867fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b9867fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 -- 172.21.15.77:0/1457586558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9869f380 msgr2=0x555b986b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9869f380 0x555b986b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.947+0000 7efd1493c640 1 -- 172.21.15.77:0/1457586558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b9869f100 con 0x555b986835d0 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.948+0000 7efd1493c640 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd0400f7f0 tx=0x7efd04015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7ead5b8d566e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.948+0000 7efcfa7fc640 1 -- 172.21.15.77:0/1457586558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04005220 con 0x555b986835d0 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.948+0000 7efcfa7fc640 1 -- 172.21.15.77:0/1457586558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd04004c80 con 0x555b986835d0 2024-09-15T22:25:44.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.948+0000 7efcfa7fc640 1 -- 172.21.15.77:0/1457586558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04028810 con 0x555b986835d0 2024-09-15T22:25:44.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 -- 172.21.15.77:0/1457586558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 msgr2=0x555b986ab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd0400f7f0 tx=0x7efd04015000 comp rx=0 tx=0).stop 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 -- 172.21.15.77:0/1457586558 shutdown_connections 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b9869f380 0x555b986b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b9867fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 --2- 172.21.15.77:0/1457586558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b986835d0 0x555b986ab5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 -- 172.21.15.77:0/1457586558 >> 172.21.15.77:0/1457586558 conn(0x555b9864c310 msgr2=0x555b9867f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 -- 172.21.15.77:0/1457586558 shutdown_connections 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.949+0000 7efd18f79740 1 -- 172.21.15.77:0/1457586558 wait complete. 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.950+0000 7efd18f79740 1 -- start start 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.950+0000 7efd18f79740 1 Processor -- start 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b986b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b986b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 0x555b986b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd101ae3c0 con 0x555b986835d0 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd101adcf0 con 0x555b9869f380 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd10108f80 con 0x555b98684b40 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd18f79740 10 client.0 _ll_register_callbacks cb 0x7efd08009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b986b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b986b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55350/0 (socket says 172.21.15.77:55350) 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd0ffff640 1 -- 172.21.15.77:0/4261233745 learned_addr learned my addr 172.21.15.77:0/4261233745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1513d640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 0x555b986b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1493c640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b986b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1513d640 1 -- 172.21.15.77:0/4261233745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 msgr2=0x555b986b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1513d640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b986b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1513d640 1 -- 172.21.15.77:0/4261233745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 msgr2=0x555b986b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.951+0000 7efd1513d640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b986b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efd1513d640 1 -- 172.21.15.77:0/4261233745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b986b2510 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efd1513d640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 0x555b986b1fd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd08010d30 tx=0x7efd08005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281cb2d1c7cd9180 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efceffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08012e60 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7efd080045f0 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.952+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd04004c80 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efd08016070 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 10 client.27761 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b98719520 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd04004c80 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efd08031030 con 0x555b9869f380 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 10 client.27761 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:44.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 10 client.27761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 20 client.27761 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b9871bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 4 client.27761 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b985f0d70 con 0x555b986aeef0 2024-09-15T22:25:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.953+0000 7efd1493c640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b9871bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:44.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efceffff640 10 client.27761 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:44.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd1493c640 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b9871bc80 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1954173fda34d422 server_cookie=ea629588b307d583 in_seq=0 out_seq=0 2024-09-15T22:25:44.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efceffff640 1 -- 172.21.15.77:0/4261233745 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b985f0d70 con 0x555b986aeef0 2024-09-15T22:25:44.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efceffff640 10 client.27761 handle_command_reply: tid=0 2024-09-15T22:25:44.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 client.27761 shutdown 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 msgr2=0x555b986b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 0x555b986b1fd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd08010d30 tx=0x7efd08005fd0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 shutdown_connections 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b986aeef0 0x555b9871bc80 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b9869f380 0x555b986b1fd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b98684b40 0x555b986b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 --2- 172.21.15.77:0/4261233745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b986835d0 0x555b986b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 >> 172.21.15.77:0/4261233745 conn(0x555b9864c310 msgr2=0x555b98685930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.954+0000 7efceffff640 0 client.27761 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.955+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 shutdown_connections 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.955+0000 7efd18f79740 1 -- 172.21.15.77:0/4261233745 wait complete. 2024-09-15T22:25:44.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.955+0000 7efd18f79740 20 client.27761 trim_cache size 0 max 16384 2024-09-15T22:25:44.962 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 msgr2=0x7efd000a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 0x7efd000a5a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555b98680ac0 tx=0x7efd04005ed0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 msgr2=0x7efd10107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000c00 tx=0x7efcfc003c10 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 shutdown_connections 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7efd000a3580 0x7efd000a5a40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd10102ef0 0x7efd101084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd10102570 0x7efd10107f90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 --2- 172.21.15.77:0/973239494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd10100e20 0x7efd10107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 >> 172.21.15.77:0/973239494 conn(0x7efd100fc7d0 msgr2=0x7efd1006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:44.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 shutdown_connections 2024-09-15T22:25:44.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:44.956+0000 7efd16ba0640 1 -- 172.21.15.77:0/973239494 wait complete. 2024-09-15T22:25:45.077 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.4 2024-09-15T22:25:45.077 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:45.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1037468 7.54 MB/sec execute 594 sec latency 34.521 ms 2024-09-15T22:25:45.779 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 -- 172.21.15.77:0/307414138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954105560 msgr2=0x7f695410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 --2- 172.21.15.77:0/307414138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954105560 0x7f695410d0e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f694c00adb0 tx=0x7f694c02b7a0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 -- 172.21.15.77:0/307414138 shutdown_connections 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 --2- 172.21.15.77:0/307414138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954105560 0x7f695410d0e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 --2- 172.21.15.77:0/307414138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f6954104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 --2- 172.21.15.77:0/307414138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954100430 0x7f6954100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 -- 172.21.15.77:0/307414138 >> 172.21.15.77:0/307414138 conn(0x7f6954077230 msgr2=0x7f69540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:46.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 -- 172.21.15.77:0/307414138 shutdown_connections 2024-09-15T22:25:46.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.293+0000 7f6959892640 1 -- 172.21.15.77:0/307414138 wait complete. 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.294+0000 7f6959892640 1 Processor -- start 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.294+0000 7f6959892640 1 -- start start 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954100430 0x7f695419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954105560 0x7f695419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:46.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f695410fd40 con 0x7f6954100430 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f695410fbc0 con 0x7f6954100de0 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f695410fec0 con 0x7f6954105560 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6952ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954100430 0x7f695419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41246/0 (socket says 172.21.15.77:41246) 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 -- 172.21.15.77:0/4274245183 learned_addr learned my addr 172.21.15.77:0/4274245183 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69537fe640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954105560 0x7f695419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 -- 172.21.15.77:0/4274245183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954105560 msgr2=0x7f695419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954105560 0x7f695419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 -- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954100430 msgr2=0x7f695419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954100430 0x7f695419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 -- 172.21.15.77:0/4274245183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695419f620 con 0x7f6954100de0 2024-09-15T22:25:46.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f69527fc640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f694800e9f0 tx=0x7f694800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694800cd80 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948010910 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.295+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695419f910 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.296+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948018990 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.296+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a71f0 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.297+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6954100810 con 0x7f6954100de0 2024-09-15T22:25:46.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.298+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6948010ac0 con 0x7f6954100de0 2024-09-15T22:25:46.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.298+0000 7f6958890640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 0x7f693c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:46.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.298+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f69480d0690 con 0x7f6954100de0 2024-09-15T22:25:46.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.298+0000 7f6952ffd640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 0x7f693c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:46.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.299+0000 7f6952ffd640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 0x7f693c0a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6944009bd0 tx=0x7f6944005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:46.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.301+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f694809c630 con 0x7f6954100de0 2024-09-15T22:25:46.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1038774 7.54 MB/sec execute 595 sec latency 75.421 ms 2024-09-15T22:25:46.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.529+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69541a74e0 con 0x7f6954100de0 2024-09-15T22:25:46.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:46 smithi077 ceph-mon[30980]: pgmap v417: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:46.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.530+0000 7f6958890640 1 -- 172.21.15.77:0/4274245183 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f69480a0930 con 0x7f6954100de0 2024-09-15T22:25:46.533 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:25:46.533 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:25:46.533 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 msgr2=0x7f693c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 0x7f693c0a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6944009bd0 tx=0x7f6944005f90 comp rx=0 tx=0).stop 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 msgr2=0x7f695419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f694800e9f0 tx=0x7f694800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 shutdown_connections 2024-09-15T22:25:46.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f693c0a33b0 0x7f693c0a5870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6954105560 0x7f695419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6954100de0 0x7f695419abb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 --2- 172.21.15.77:0/4274245183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6954100430 0x7f695419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.533+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 >> 172.21.15.77:0/4274245183 conn(0x7f6954077230 msgr2=0x7f6954103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.534+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 shutdown_connections 2024-09-15T22:25:46.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:46.534+0000 7f6959892640 1 -- 172.21.15.77:0/4274245183 wait complete. 2024-09-15T22:25:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:46 smithi196 ceph-mon[35109]: pgmap v417: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:46 smithi117 ceph-mon[35021]: pgmap v417: 129 pgs: 129 active+clean; 623 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 166 op/s 2024-09-15T22:25:46.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:25:46.769 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:25:47.275 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0' 2024-09-15T22:25:47.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0 2024-09-15T22:25:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:47.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1040873 7.54 MB/sec execute 596 sec latency 77.636 ms 2024-09-15T22:25:47.546 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:47 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4274245183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:47.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:47 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4274245183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:47 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4274245183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 -- 172.21.15.77:0/4026281566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c1008e0 msgr2=0x7f234c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 --2- 172.21.15.77:0/4026281566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c1008e0 0x7f234c100cc0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2340009930 tx=0x7f234002a420 comp rx=0 tx=0).stop 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 -- 172.21.15.77:0/4026281566 shutdown_connections 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 --2- 172.21.15.77:0/4026281566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c108160 0x7f234c10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 --2- 172.21.15.77:0/4026281566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c101290 0x7f234c107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 --2- 172.21.15.77:0/4026281566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c1008e0 0x7f234c100cc0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 -- 172.21.15.77:0/4026281566 >> 172.21.15.77:0/4026281566 conn(0x7f234c077210 msgr2=0x7f234c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 -- 172.21.15.77:0/4026281566 shutdown_connections 2024-09-15T22:25:47.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.919+0000 7f2352c16640 1 -- 172.21.15.77:0/4026281566 wait complete. 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 Processor -- start 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 -- start start 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c1008e0 0x7f234c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c108160 0x7f234c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f234c111ff0 con 0x7f234c101290 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f234c111e70 con 0x7f234c1008e0 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f2352c16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f234c112170 con 0x7f234c108160 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53492/0 (socket says 172.21.15.77:53492) 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 -- 172.21.15.77:0/1269749342 learned_addr learned my addr 172.21.15.77:0/1269749342 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:47.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f23511b3640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c108160 0x7f234c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f23509b2640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c1008e0 0x7f234c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 -- 172.21.15.77:0/1269749342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c108160 msgr2=0x7f234c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c108160 0x7f234c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 -- 172.21.15.77:0/1269749342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c1008e0 msgr2=0x7f234c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c1008e0 0x7f234c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.920+0000 7f234bfff640 1 -- 172.21.15.77:0/1269749342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c19d4d0 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f234bfff640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9f0 tx=0x7f233800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f2349ffb640 1 -- 172.21.15.77:0/1269749342 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233800ccc0 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f2349ffb640 1 -- 172.21.15.77:0/1269749342 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2338005500 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c19d7c0 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f2349ffb640 1 -- 172.21.15.77:0/1269749342 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338010690 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.921+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1a50a0 con 0x7f234c101290 2024-09-15T22:25:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.923+0000 7f2349ffb640 1 -- 172.21.15.77:0/1269749342 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f23380046c0 con 0x7f234c101290 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 Processor -- start 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 -- start start 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea39ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607ea305280 con 0x5607ea3aad40 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607ea2acb10 con 0x5607ea3a0480 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2354fef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607ea301240 con 0x5607ea39fb60 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2349ffb640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 0x7f233c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23511b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea39ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41264/0 (socket says 172.21.15.77:41264) 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 -- 172.21.15.77:0/805431790 learned_addr learned my addr 172.21.15.77:0/805431790 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f234bfff640 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23511b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea39ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55406/0 (socket says 172.21.15.77:55406) 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23511b3640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 0x7f233c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f2349ffb640 1 -- 172.21.15.77:0/1269749342 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f23380d04b0 con 0x7f234c101290 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 -- 172.21.15.77:0/805431790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 msgr2=0x5607ea39ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea39ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 -- 172.21.15.77:0/805431790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 msgr2=0x5607ea3ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.924+0000 7f23509b2640 1 -- 172.21.15.77:0/805431790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ea2b91f0 con 0x5607ea3a0480 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.925+0000 7f23509b2640 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f234002a940 tx=0x7f2340005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189c9e3dc23fd2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.925+0000 7f23511b3640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 0x7f233c0a58f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2344009fd0 tx=0x7f2344009600 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.925+0000 7f2336ffd640 1 -- 172.21.15.77:0/805431790 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234003f070 con 0x5607ea3a0480 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.925+0000 7f2336ffd640 1 -- 172.21.15.77:0/805431790 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340004030 con 0x5607ea3a0480 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.925+0000 7f2336ffd640 1 -- 172.21.15.77:0/805431790 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340035070 con 0x5607ea3a0480 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- 172.21.15.77:0/805431790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 msgr2=0x5607ea3aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f234002a940 tx=0x7f2340005dd0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- 172.21.15.77:0/805431790 shutdown_connections 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3a0480 0x5607ea3aa800 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- 172.21.15.77:0/805431790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea39ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- 172.21.15.77:0/805431790 >> 172.21.15.77:0/805431790 conn(0x5607ea39c0f0 msgr2=0x5607ea39c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- 172.21.15.77:0/805431790 shutdown_connections 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- 172.21.15.77:0/805431790 wait complete. 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- start start 2024-09-15T22:25:47.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 Processor -- start 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5607ea3b1d20 con 0x5607ea39fb60 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607ea2bff10 con 0x5607ea3aad40 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5607ea3b1ea0 con 0x5607ea3a0480 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.926+0000 7f2354fef740 10 client.0 _ll_register_callbacks cb 0x5607ea288a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53502/0 (socket says 172.21.15.77:53502) 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 -- 172.21.15.77:0/3007447710 learned_addr learned my addr 172.21.15.77:0/3007447710 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23511b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f234bfff640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 -- 172.21.15.77:0/3007447710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 msgr2=0x5607ea3a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 -- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 msgr2=0x5607ea3a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 -- 172.21.15.77:0/3007447710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ea0766a0 con 0x5607ea39fb60 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23511b3640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f23509b2640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2340032f60 tx=0x7f2340032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f76dd53ebc7a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f232bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340035070 con 0x5607ea39fb60 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2340004030 con 0x5607ea39fb60 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f23400315b0 con 0x5607ea39fb60 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2340055050 con 0x5607ea39fb60 2024-09-15T22:25:47.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f2354fef740 10 client.18795 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.927+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607ea3bd920 con 0x5607ea39fb60 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340004030 con 0x5607ea39fb60 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f23400046c0 con 0x5607ea39fb60 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 10 client.18795 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 10 client.18795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 20 client.18795 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea3c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 4 client.18795 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607ea3127f0 con 0x5607ea3bec70 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f234bfff640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea3c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f234bfff640 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea3c1060 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ebe2f9e356e4e5 server_cookie=3a872fd4188f989 in_seq=0 out_seq=0 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.928+0000 7f232bfff640 10 client.18795 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f232bfff640 1 -- 172.21.15.77:0/3007447710 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607ea3127f0 con 0x5607ea3bec70 2024-09-15T22:25:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f232bfff640 10 client.18795 handle_command_reply: tid=0 2024-09-15T22:25:47.967 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:47.967 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:25:47.967 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "69f032d3-0ef4-4849-b29c-90b976257a86", 2024-09-15T22:25:47.967 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:25:47.967 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:47.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 client.18795 shutdown 2024-09-15T22:25:47.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 msgr2=0x5607ea3a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2340032f60 tx=0x7f2340032f90 comp rx=0 tx=0).stop 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 shutdown_connections 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea3c1060 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f232bfff640 0 client.18795 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 --2- 172.21.15.77:0/3007447710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea39fb60 0x5607ea3a7c10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 >> 172.21.15.77:0/3007447710 conn(0x5607ea39c0f0 msgr2=0x5607ea3a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 shutdown_connections 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 1 -- 172.21.15.77:0/3007447710 wait complete. 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.929+0000 7f2354fef740 20 client.18795 trim_cache size 0 max 16384 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 Processor -- start 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 -- start start 2024-09-15T22:25:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea39fb60 0x5607ea3a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ace10 con 0x5607ea3aad40 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ae950 con 0x5607ea39fb60 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f2354fef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ad4e0 con 0x5607ea3a0480 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.947+0000 7f23511b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53506/0 (socket says 172.21.15.77:53506) 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 -- 172.21.15.77:0/2884525019 learned_addr learned my addr 172.21.15.77:0/2884525019 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f234bfff640 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23509b2640 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea39fb60 0x5607ea3a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 -- 172.21.15.77:0/2884525019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 msgr2=0x5607ea3a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 -- 172.21.15.77:0/2884525019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea39fb60 msgr2=0x5607ea3a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea39fb60 0x5607ea3a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 -- 172.21.15.77:0/2884525019 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ea39bb90 con 0x5607ea3aad40 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23511b3640 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2344010b00 tx=0x7f234400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d26624cf16726fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23367fc640 1 -- 172.21.15.77:0/2884525019 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344004d20 con 0x5607ea3aad40 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23367fc640 1 -- 172.21.15.77:0/2884525019 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2344005020 con 0x5607ea3aad40 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.948+0000 7f23367fc640 1 -- 172.21.15.77:0/2884525019 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440296c0 con 0x5607ea3aad40 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- 172.21.15.77:0/2884525019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 msgr2=0x5607ea3a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2344010b00 tx=0x7f234400e000 comp rx=0 tx=0).stop 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- 172.21.15.77:0/2884525019 shutdown_connections 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3aad40 0x5607ea3a9a70 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea3a0480 0x5607ea3a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 --2- 172.21.15.77:0/2884525019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea39fb60 0x5607ea3a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- 172.21.15.77:0/2884525019 >> 172.21.15.77:0/2884525019 conn(0x5607ea39c0f0 msgr2=0x5607ea3a9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- 172.21.15.77:0/2884525019 shutdown_connections 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- 172.21.15.77:0/2884525019 wait complete. 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.949+0000 7f2354fef740 1 -- start start 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 Processor -- start 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1ae280 con 0x5607ea3a0480 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1adbb0 con 0x5607ea3aad40 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f234c1aae10 con 0x5607ea39fb60 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f2354fef740 10 client.0 _ll_register_callbacks cb 0x7f2344009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f23509b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f23509b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55440/0 (socket says 172.21.15.77:55440) 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53518/0 (socket says 172.21.15.77:53518) 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f23509b2640 1 -- 172.21.15.77:0/2826704307 learned_addr learned my addr 172.21.15.77:0/2826704307 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f23511b3640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 -- 172.21.15.77:0/2826704307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 msgr2=0x5607ea3c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 -- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 msgr2=0x5607ea3c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f234bfff640 1 -- 172.21.15.77:0/2826704307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ea3c2640 con 0x5607ea3a0480 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.950+0000 7f23509b2640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f234bfff640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f23380cd810 tx=0x7f233806bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe728b262a880d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380a18e0 con 0x5607ea3a0480 2024-09-15T22:25:47.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f2338098a10 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2344005020 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f233809caa0 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 10 client.18801 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607ea429530 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2344005020 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f23380cda90 con 0x5607ea3a0480 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 10 client.18801 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 20 client.18801 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea42bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 4 client.18801 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:25:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.951+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5607ea30cd00 con 0x5607ea3bec70 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.952+0000 7f23509b2640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea42bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.952+0000 7f23509b2640 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea42bd40 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef6ca079cba9adb server_cookie=4d05e0e245c7fcee in_seq=0 out_seq=0 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.952+0000 7f232bfff640 10 client.18801 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f232bfff640 1 -- 172.21.15.77:0/2826704307 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5607ea30cd00 con 0x5607ea3bec70 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f232bfff640 10 client.18801 handle_command_reply: tid=0 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 client.18801 shutdown 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 msgr2=0x5607ea3c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f23380cd810 tx=0x7f233806bfc0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 shutdown_connections 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5607ea3bec70 0x5607ea42bd40 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f232bfff640 0 client.18801 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ea3aad40 0x5607ea3c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5607ea3a0480 0x5607ea3c1a10 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 --2- 172.21.15.77:0/2826704307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607ea39fb60 0x5607ea3c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 >> 172.21.15.77:0/2826704307 conn(0x5607ea39c0f0 msgr2=0x5607ea3a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 shutdown_connections 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 1 -- 172.21.15.77:0/2826704307 wait complete. 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.953+0000 7f2354fef740 20 client.18801 trim_cache size 0 max 16384 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 msgr2=0x7f233c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 0x7f233c0a58f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2344009fd0 tx=0x7f2344009600 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 msgr2=0x7f234c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9f0 tx=0x7f233800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 shutdown_connections 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f233c0a3430 0x7f233c0a58f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f234c108160 0x7f234c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c101290 0x7f234c198a60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 --2- 172.21.15.77:0/1269749342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f234c1008e0 0x7f234c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 >> 172.21.15.77:0/1269749342 conn(0x7f234c077210 msgr2=0x7f234c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 shutdown_connections 2024-09-15T22:25:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:47.954+0000 7f2352c16640 1 -- 172.21.15.77:0/1269749342 wait complete. 2024-09-15T22:25:48.336 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:48.495 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1041930 7.54 MB/sec execute 597 sec latency 65.729 ms 2024-09-15T22:25:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:48 smithi077 ceph-mon[30980]: pgmap v418: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-09-15T22:25:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:48 smithi196 ceph-mon[35109]: pgmap v418: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-09-15T22:25:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:48 smithi117 ceph-mon[35021]: pgmap v418: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 252 op/s 2024-09-15T22:25:49.060 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:25:49.512 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1044415 7.55 MB/sec execute 598 sec latency 30.052 ms 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:49 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:25:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:25:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:25:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:49 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:49 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 -- 172.21.15.77:0/1595250918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102ef0 msgr2=0x7fb58010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1595250918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102ef0 0x7fb58010da40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ca0 tx=0x7fb57802a950 comp rx=0 tx=0).stop 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 -- 172.21.15.77:0/1595250918 shutdown_connections 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1595250918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102ef0 0x7fb58010da40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1595250918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580102570 0x7fb5801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1595250918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580100e20 0x7fb580101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 -- 172.21.15.77:0/1595250918 >> 172.21.15.77:0/1595250918 conn(0x7fb5800fc7d0 msgr2=0x7fb5800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 -- 172.21.15.77:0/1595250918 shutdown_connections 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.711+0000 7fb5863f7640 1 -- 172.21.15.77:0/1595250918 wait complete. 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 Processor -- start 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 -- start start 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 0x7fb5801993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 0x7fb580199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb580110f80 con 0x7fb580102570 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb580110e00 con 0x7fb580100e20 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb5863f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb580111100 con 0x7fb580102ef0 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 0x7fb580199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb584994640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb584994640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52688/0 (socket says 172.21.15.77:52688) 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb584994640 1 -- 172.21.15.77:0/1166583337 learned_addr learned my addr 172.21.15.77:0/1166583337 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.712+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 0x7fb5801993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 -- 172.21.15.77:0/1166583337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 msgr2=0x7fb5801993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 0x7fb5801993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 -- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 msgr2=0x7fb580199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 0x7fb580199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 -- 172.21.15.77:0/1166583337 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5801a47d0 con 0x7fb580102ef0 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 0x7fb580199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 0x7fb5801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb584994640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb578004af0 tx=0x7fb57802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb57d7fa640 1 -- 172.21.15.77:0/1166583337 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578034040 con 0x7fb580102ef0 2024-09-15T22:25:49.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb57d7fa640 1 -- 172.21.15.77:0/1166583337 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578030920 con 0x7fb580102ef0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5801a4a60 con 0x7fb580102ef0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb57d7fa640 1 -- 172.21.15.77:0/1166583337 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004630 con 0x7fb580102ef0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.713+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5801a4f90 con 0x7fb580102ef0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb57d7fa640 1 -- 172.21.15.77:0/1166583337 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb578037020 con 0x7fb580102ef0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 Processor -- start 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 -- start start 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdada760 0x5649bdadab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 0x5649bdae5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5649bda402a0 con 0x5649bdadb080 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5649bd9e7b10 con 0x5649bdae5940 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb5887d0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649bda3c260 con 0x5649bdada760 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 0x5649bdae5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 0x5649bdae5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35292/0 (socket says 172.21.15.77:35292) 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.716+0000 7fb57ffff640 1 -- 172.21.15.77:0/526434809 learned_addr learned my addr 172.21.15.77:0/526434809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57d7fa640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdada760 0x5649bdadab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb584994640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 -- 172.21.15.77:0/526434809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdada760 msgr2=0x5649bdadab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdada760 0x5649bdadab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 -- 172.21.15.77:0/526434809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 msgr2=0x5649bdae5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 0x5649bdae5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 -- 172.21.15.77:0/526434809 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649bd7b1b90 con 0x5649bdae5940 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57d7fa640 1 -- 172.21.15.77:0/1166583337 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fb5780f0cd0 con 0x7fb580102ef0 2024-09-15T22:25:49.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae7d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb574009730 tx=0x7fb5740096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1564ef52d5a4fe97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.717+0000 7fb584994640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 0x7fb5680a58f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5780b9190 tx=0x7fb57808eab0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.718+0000 7fb5667fc640 1 -- 172.21.15.77:0/526434809 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574004030 con 0x5649bdae5940 2024-09-15T22:25:49.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.718+0000 7fb5667fc640 1 -- 172.21.15.77:0/526434809 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400fdf0 con 0x5649bdae5940 2024-09-15T22:25:49.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.718+0000 7fb5667fc640 1 -- 172.21.15.77:0/526434809 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5740047a0 con 0x5649bdae5940 2024-09-15T22:25:49.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- 172.21.15.77:0/526434809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 msgr2=0x5649bdae7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae7d30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb574009730 tx=0x7fb5740096c0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- 172.21.15.77:0/526434809 shutdown_connections 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae7d30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdadb080 0x5649bdae5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 --2- 172.21.15.77:0/526434809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdada760 0x5649bdadab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- 172.21.15.77:0/526434809 >> 172.21.15.77:0/526434809 conn(0x5649bdad6cf0 msgr2=0x5649bdad70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- 172.21.15.77:0/526434809 shutdown_connections 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- 172.21.15.77:0/526434809 wait complete. 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.719+0000 7fb5887d0740 1 -- start start 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 Processor -- start 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdae28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5649bdae8510 con 0x5649bdada760 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5649bd9c86e0 con 0x5649bdae5940 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5649bdae8690 con 0x5649bdadb080 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb5887d0740 10 client.0 _ll_register_callbacks cb 0x5649bd9eca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdae28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52702/0 (socket says 172.21.15.77:52702) 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb57ffff640 1 -- 172.21.15.77:0/3679757168 learned_addr learned my addr 172.21.15.77:0/3679757168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.720+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 -- 172.21.15.77:0/3679757168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 msgr2=0x5649bdae2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 -- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 msgr2=0x5649bdae3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 -- 172.21.15.77:0/3679757168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649bdaf1b10 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57ffff640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdae28c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb574009730 tx=0x7fb574005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222365497b108a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57400fa80 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fb5740046c0 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57400fdf0 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb574035050 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb5887d0740 10 client.18813 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5649bdaf4440 con 0x5649bdada760 2024-09-15T22:25:49.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400fdf0 con 0x5649bdada760 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.721+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb57401f850 con 0x5649bdada760 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 10 client.18813 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 10 client.18813 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 20 client.18813 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdaf5790 0x5649bdaf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 4 client.18813 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649bda4d9d0 con 0x5649bdaf5790 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb57ffff640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdaf5790 0x5649bdaf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb55f7fe640 10 client.18813 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.722+0000 7fb57ffff640 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdaf5790 0x5649bdaf7b80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8047154b785278 server_cookie=ba43c05403036088 in_seq=0 out_seq=0 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb55f7fe640 1 -- 172.21.15.77:0/3679757168 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649bda4d9d0 con 0x5649bdaf5790 2024-09-15T22:25:49.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb55f7fe640 10 client.18813 handle_command_reply: tid=0 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 client.18813 shutdown 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 msgr2=0x5649bdae28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdae28c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb574009730 tx=0x7fb574005260 comp rx=0 tx=0).stop 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 shutdown_connections 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdaf5790 0x5649bdaf7b80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdae3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdae2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 --2- 172.21.15.77:0/3679757168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdae28c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 >> 172.21.15.77:0/3679757168 conn(0x5649bdad6cf0 msgr2=0x5649bdadca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb55f7fe640 0 client.18813 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 shutdown_connections 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 1 -- 172.21.15.77:0/3679757168 wait complete. 2024-09-15T22:25:49.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.723+0000 7fb5887d0740 20 client.18813 trim_cache size 0 max 16384 2024-09-15T22:25:49.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 Processor -- start 2024-09-15T22:25:49.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 -- start start 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdafd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdafd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801aef20 con 0x5649bdada760 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801b0a60 con 0x5649bdae5940 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb5887d0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801af5f0 con 0x5649bdadb080 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35324/0 (socket says 172.21.15.77:35324) 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 -- 172.21.15.77:0/2608042592 learned_addr learned my addr 172.21.15.77:0/2608042592 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57ffff640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdafd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdafd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 -- 172.21.15.77:0/2608042592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 msgr2=0x5649bdafd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdafd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 -- 172.21.15.77:0/2608042592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 msgr2=0x5649bdafd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdafd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57f7fe640 1 -- 172.21.15.77:0/2608042592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649bdaf7ef0 con 0x5649bdada760 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.744+0000 7fb57ffff640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdafd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.745+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb5740059f0 tx=0x7fb574000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d212559e1b446bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.745+0000 7fb565ffb640 1 -- 172.21.15.77:0/2608042592 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574009c80 con 0x5649bdada760 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.745+0000 7fb565ffb640 1 -- 172.21.15.77:0/2608042592 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400fa80 con 0x5649bdada760 2024-09-15T22:25:49.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.745+0000 7fb565ffb640 1 -- 172.21.15.77:0/2608042592 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57401f900 con 0x5649bdada760 2024-09-15T22:25:49.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- 172.21.15.77:0/2608042592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 msgr2=0x5649bdad5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb5740059f0 tx=0x7fb574000bc0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- 172.21.15.77:0/2608042592 shutdown_connections 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdae5940 0x5649bdafd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdadb080 0x5649bdafd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 --2- 172.21.15.77:0/2608042592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdad5d60 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- 172.21.15.77:0/2608042592 >> 172.21.15.77:0/2608042592 conn(0x5649bdad6cf0 msgr2=0x5649bdad5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- 172.21.15.77:0/2608042592 shutdown_connections 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- 172.21.15.77:0/2608042592 wait complete. 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 -- start start 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.746+0000 7fb5887d0740 1 Processor -- start 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdaedca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 0x5649bdaee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801b0390 con 0x5649bdada760 2024-09-15T22:25:49.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801afcc0 con 0x5649bdadb080 2024-09-15T22:25:49.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801acf20 con 0x5649bdae5940 2024-09-15T22:25:49.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb5887d0740 10 client.0 _ll_register_callbacks cb 0x7fb5740119d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52718/0 (socket says 172.21.15.77:52718) 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 -- 172.21.15.77:0/1140260753 learned_addr learned my addr 172.21.15.77:0/1140260753 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 0x5649bdaee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 -- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 msgr2=0x5649bdaee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 0x5649bdaee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 -- 172.21.15.77:0/1140260753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 msgr2=0x5649bdaedca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdaedca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57f7fe640 1 -- 172.21.15.77:0/1140260753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649bdaea960 con 0x5649bdae5940 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.747+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 0x5649bdaee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb57f7fe640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb574009560 tx=0x7fb574005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784f7a54f2949f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:25:49.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574046060 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57400fa80 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7fb57402f0c0 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb57404d050 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb5887d0740 10 client.27706 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5649bdb65370 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.748+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400fa80 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb574003a00 con 0x5649bdae5940 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 10 client.27706 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 10 client.27706 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 20 client.27706 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdb65a00 0x5649bdb67df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 4 client.27706 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}] 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}) -- 0x5649bda47d20 con 0x5649bdb65a00 2024-09-15T22:25:49.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdb65a00 0x5649bdb67df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:49.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb55f7fe640 10 client.27706 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:49.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.749+0000 7fb57ffff640 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdb65a00 0x5649bdb67df0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6752984b234df0 server_cookie=744f433bae52a13f in_seq=0 out_seq=0 2024-09-15T22:25:49.789 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:49.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb55f7fe640 1 -- 172.21.15.77:0/1140260753 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649bda47d20 con 0x5649bdb65a00 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb55f7fe640 10 client.27706 handle_command_reply: tid=0 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 client.27706 shutdown 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 msgr2=0x5649bdaea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb574009560 tx=0x7fb574005020 comp rx=0 tx=0).stop 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 shutdown_connections 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5649bdb65a00 0x5649bdb67df0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5649bdae5940 0x5649bdaea230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5649bdadb080 0x5649bdaee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 --2- 172.21.15.77:0/1140260753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5649bdada760 0x5649bdaedca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb55f7fe640 0 client.27706 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 >> 172.21.15.77:0/1140260753 conn(0x5649bdad6cf0 msgr2=0x5649bdad5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 shutdown_connections 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 1 -- 172.21.15.77:0/1140260753 wait complete. 2024-09-15T22:25:49.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.859+0000 7fb5887d0740 20 client.27706 trim_cache size 0 max 16384 2024-09-15T22:25:49.865 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:49.865 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:25:49.865 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:25:49.865 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 46, 2024-09-15T22:25:49.865 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "c8e6e028": { 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 46, 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:25:49.866 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 131.4, 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:25:49.867 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:49.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:49.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:49.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:49.868 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:49.868 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:49.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 msgr2=0x7fb5680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 0x7fb5680a58f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb5780b9190 tx=0x7fb57808eab0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 msgr2=0x7fb580199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:49.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb578004af0 tx=0x7fb57802f3a0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 shutdown_connections 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5680a3430 0x7fb5680a58f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb580102ef0 0x7fb580199e90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb580102570 0x7fb580199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 --2- 172.21.15.77:0/1166583337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb580100e20 0x7fb5801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 >> 172.21.15.77:0/1166583337 conn(0x7fb5800fc7d0 msgr2=0x7fb5801068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 shutdown_connections 2024-09-15T22:25:49.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:49.860+0000 7fb5863f7640 1 -- 172.21.15.77:0/1166583337 wait complete. 2024-09-15T22:25:50.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 -- 172.21.15.77:0/211600076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 msgr2=0x7f8bd810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 --2- 172.21.15.77:0/211600076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd810ec70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009cd0 tx=0x7f8bc802a8d0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 -- 172.21.15.77:0/211600076 shutdown_connections 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 --2- 172.21.15.77:0/211600076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd810ec70 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 --2- 172.21.15.77:0/211600076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 0x7f8bd8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 --2- 172.21.15.77:0/211600076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 0x7f8bd81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 -- 172.21.15.77:0/211600076 >> 172.21.15.77:0/211600076 conn(0x7f8bd80fc7d0 msgr2=0x7f8bd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 -- 172.21.15.77:0/211600076 shutdown_connections 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.339+0000 7f8bdf972640 1 -- 172.21.15.77:0/211600076 wait complete. 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 Processor -- start 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 -- start start 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 0x7f8bd81a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 0x7f8bd81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8112120 con 0x7f8bd8108300 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8111fa0 con 0x7f8bd8102dd0 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdf972640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81122a0 con 0x7f8bd8103780 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35368/0 (socket says 172.21.15.77:35368) 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 -- 172.21.15.77:0/1004987402 learned_addr learned my addr 172.21.15.77:0/1004987402 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 0x7f8bd81a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 0x7f8bd81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 -- 172.21.15.77:0/1004987402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 msgr2=0x7f8bd81a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 0x7f8bd81a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 -- 172.21.15.77:0/1004987402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 msgr2=0x7f8bd81a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 0x7f8bd81a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.340+0000 7f8bddf0f640 1 -- 172.21.15.77:0/1004987402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd819ace0 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bddf0f640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8031040 tx=0x7f8bc8031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bd67fc640 1 -- 172.21.15.77:0/1004987402 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8031790 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd819af70 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bd67fc640 1 -- 172.21.15.77:0/1004987402 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc8003e50 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bd67fc640 1 -- 172.21.15.77:0/1004987402 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80307f0 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.341+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a72c0 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.344+0000 7f8be1d4b740 1 Processor -- start 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.344+0000 7f8bd67fc640 1 -- 172.21.15.77:0/1004987402 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8bc8003970 con 0x7f8bd8108300 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.344+0000 7f8be1d4b740 1 -- start start 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55974b31f2e0 con 0x55974b3c4680 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55974b2c6ce0 con 0x55974b3b94a0 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55974b31b2a0 con 0x55974b3b9dc0 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52746/0 (socket says 172.21.15.77:52746) 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3941911378 learned_addr learned my addr 172.21.15.77:0/3941911378 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:50.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48120/0 (socket says 172.21.15.77:48120) 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bd67fc640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 0x7f8bc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3941911378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 msgr2=0x55974b3b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3941911378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 msgr2=0x55974b3c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3941911378 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974b2ab440 con 0x55974b3b9dc0 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bddf0f640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 0x7f8bc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.345+0000 7f8bd67fc640 1 -- 172.21.15.77:0/1004987402 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8bc80f0de0 con 0x7f8bd8108300 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ac20 tx=0x7f8bd000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f91a48c6770979 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8bbf7fe640 1 -- 172.21.15.77:0/3941911378 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd001d070 con 0x55974b3b9dc0 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8bbf7fe640 1 -- 172.21.15.77:0/3941911378 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0018b20 con 0x55974b3b9dc0 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8bbf7fe640 1 -- 172.21.15.77:0/3941911378 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0010b70 con 0x55974b3b9dc0 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3941911378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 msgr2=0x55974b3c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.346+0000 7f8bddf0f640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 0x7f8bc00a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80b92a0 tx=0x7f8bc808ebc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ac20 tx=0x7f8bd000f790 comp rx=0 tx=0).stop 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3941911378 shutdown_connections 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3c4140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3941911378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3941911378 >> 172.21.15.77:0/3941911378 conn(0x55974b3b5a30 msgr2=0x55974b3b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3941911378 shutdown_connections 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3941911378 wait complete. 2024-09-15T22:25:50.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- start start 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 Processor -- start 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b94a0 0x55974b3c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3c4680 0x55974b3c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.347+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55974b3c7250 con 0x55974b3b9dc0 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55974b2da050 con 0x55974b3c4680 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55974b3c73d0 con 0x55974b3b94a0 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8be1d4b740 10 client.0 _ll_register_callbacks cb 0x55974b31a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35376/0 (socket says 172.21.15.77:35376) 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3229552515 learned_addr learned my addr 172.21.15.77:0/3229552515 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:50.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b94a0 0x55974b3c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3c4680 0x55974b3c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3229552515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b94a0 msgr2=0x55974b3c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b94a0 0x55974b3c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3c4680 msgr2=0x55974b3c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3c4680 0x55974b3c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 -- 172.21.15.77:0/3229552515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974b0903c0 con 0x55974b3b9dc0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0010470 tx=0x7f8bd00186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61deb0373ec881a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd001d050 con 0x55974b3b9dc0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8bd0010890 con 0x55974b3b9dc0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd0018b20 con 0x55974b3b9dc0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8bd0031050 con 0x55974b3b9dc0 2024-09-15T22:25:50.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.348+0000 7f8be1d4b740 10 client.18828 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55974b3d2bc0 con 0x55974b3b9dc0 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd0018b20 con 0x55974b3b9dc0 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8bd0005480 con 0x55974b3b9dc0 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 10 client.18828 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 20 client.18828 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b3d4020 0x55974b3d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 4 client.18828 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55974b32c850 con 0x55974b3d4020 2024-09-15T22:25:50.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.349+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b3d4020 0x55974b3d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b3d4020 0x55974b3d6410 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd135dc610b222e server_cookie=4192005caa64ba18 in_seq=0 out_seq=0 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8bbcff9640 10 client.18828 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8bbcff9640 1 -- 172.21.15.77:0/3229552515 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55974b32c850 con 0x55974b3d4020 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8bbcff9640 10 client.18828 handle_command_reply: tid=0 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8be1d4b740 1 client.18828 shutdown 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 msgr2=0x55974b3c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0010470 tx=0x7f8bd00186f0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 shutdown_connections 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.350+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b3d4020 0x55974b3d6410 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3c4680 0x55974b3c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3b9dc0 0x55974b3c1220 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/3229552515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b94a0 0x55974b3c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 >> 172.21.15.77:0/3229552515 conn(0x55974b3b5a30 msgr2=0x55974b3bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8bbcff9640 0 client.18828 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:50.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 shutdown_connections 2024-09-15T22:25:50.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 1 -- 172.21.15.77:0/3229552515 wait complete. 2024-09-15T22:25:50.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.351+0000 7f8be1d4b740 20 client.18828 trim_cache size 0 max 16384 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.369+0000 7f8be1d4b740 1 Processor -- start 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.369+0000 7f8be1d4b740 1 -- start start 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b1250 con 0x55974b3c4680 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b2d90 con 0x55974b3b94a0 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b1920 con 0x55974b3b9dc0 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35392/0 (socket says 172.21.15.77:35392) 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52776/0 (socket says 172.21.15.77:52776) 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdcf0d640 1 -- 172.21.15.77:0/1331297259 learned_addr learned my addr 172.21.15.77:0/1331297259 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 -- 172.21.15.77:0/1331297259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 msgr2=0x55974b3d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 -- 172.21.15.77:0/1331297259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 msgr2=0x55974b3dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdd70e640 1 -- 172.21.15.77:0/1331297259 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974b3b92d0 con 0x55974b3b9dc0 2024-09-15T22:25:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.370+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.371+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ac20 tx=0x7f8bd0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359f02835e40bed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.371+0000 7f8bbeffd640 1 -- 172.21.15.77:0/1331297259 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd001d070 con 0x55974b3b9dc0 2024-09-15T22:25:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.371+0000 7f8bbeffd640 1 -- 172.21.15.77:0/1331297259 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd001e040 con 0x55974b3b9dc0 2024-09-15T22:25:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.371+0000 7f8bbeffd640 1 -- 172.21.15.77:0/1331297259 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00035f0 con 0x55974b3b9dc0 2024-09-15T22:25:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- 172.21.15.77:0/1331297259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 msgr2=0x55974b3db9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ac20 tx=0x7f8bd0005dc0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- 172.21.15.77:0/1331297259 shutdown_connections 2024-09-15T22:25:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3db9e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/1331297259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- 172.21.15.77:0/1331297259 >> 172.21.15.77:0/1331297259 conn(0x55974b3b5a30 msgr2=0x55974b3c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- 172.21.15.77:0/1331297259 shutdown_connections 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- 172.21.15.77:0/1331297259 wait complete. 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 -- start start 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.372+0000 7f8be1d4b740 1 Processor -- start 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b26c0 con 0x55974b3c4680 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81b1ff0 con 0x55974b3b94a0 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81af250 con 0x55974b3b9dc0 2024-09-15T22:25:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8be1d4b740 10 client.0 _ll_register_callbacks cb 0x7f8bd001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48142/0 (socket says 172.21.15.77:48142) 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 -- 172.21.15.77:0/4276463559 learned_addr learned my addr 172.21.15.77:0/4276463559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 -- 172.21.15.77:0/4276463559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 msgr2=0x55974b3ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 -- 172.21.15.77:0/4276463559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 msgr2=0x55974b3c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 -- 172.21.15.77:0/4276463559 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974b3c8780 con 0x55974b3b94a0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bdcf0d640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bd806a2d0 tx=0x7f8bcc00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee28b0a3aa2a8030 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:50.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.373+0000 7f8bbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc003b40 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd001e040 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f8bcc003ce0 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc039050 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 10 client.27818 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55974b43e6a0 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd001e040 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8bcc02b070 con 0x55974b3b94a0 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 10 client.27818 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 10 client.27818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:50.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 20 client.27818 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:50.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b43eda0 0x55974b4411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:50.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 4 client.27818 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55974b326d60 con 0x55974b43eda0 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.374+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b43eda0 0x55974b4411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8bbcff9640 10 client.27818 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8bdd70e640 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b43eda0 0x55974b4411c0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb5dbeb101e99ec server_cookie=521ae5a6ff4c3217 in_seq=0 out_seq=0 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8bbcff9640 1 -- 172.21.15.77:0/4276463559 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55974b326d60 con 0x55974b43eda0 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8bbcff9640 10 client.27818 handle_command_reply: tid=0 2024-09-15T22:25:50.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 client.27818 shutdown 2024-09-15T22:25:50.383 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:50 smithi077 ceph-mon[30980]: pgmap v419: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 msgr2=0x55974b3cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bd806a2d0 tx=0x7f8bcc00b390 comp rx=0 tx=0).stop 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 shutdown_connections 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55974b43eda0 0x55974b4411c0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55974b3c4680 0x55974b3c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8bbcff9640 0 client.27818 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974b3b9dc0 0x55974b3ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 --2- 172.21.15.77:0/4276463559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55974b3b94a0 0x55974b3cc9f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.375+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 >> 172.21.15.77:0/4276463559 conn(0x55974b3b5a30 msgr2=0x55974b3b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 shutdown_connections 2024-09-15T22:25:50.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8be1d4b740 1 -- 172.21.15.77:0/4276463559 wait complete. 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8be1d4b740 20 client.27818 trim_cache size 0 max 16384 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "69f032d3-0ef4-4849-b29c-90b976257a86": { 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "69f032d3-0ef4-4849-b29c-90b976257a86", 2024-09-15T22:25:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:25:50.387 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:50.387 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:50.387 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:50.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 msgr2=0x7f8bc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 0x7f8bc00a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80b92a0 tx=0x7f8bc808ebc0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 msgr2=0x7f8bd819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.376+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8031040 tx=0x7f8bc8031420 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 shutdown_connections 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8bc00a33b0 0x7f8bc00a5870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bd8108300 0x7f8bd819a5b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bd8103780 0x7f8bd81a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 --2- 172.21.15.77:0/1004987402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd8102dd0 0x7f8bd81a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 >> 172.21.15.77:0/1004987402 conn(0x7f8bd80fc7d0 msgr2=0x7f8bd8101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 shutdown_connections 2024-09-15T22:25:50.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:50.377+0000 7f8bdf972640 1 -- 172.21.15.77:0/1004987402 wait complete. 2024-09-15T22:25:50.512 INFO:tasks.workunit.client.0.smithi077.stdout: 1 1044527 7.54 MB/sec execute 599 sec latency 769.285 ms 2024-09-15T22:25:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:50 smithi196 ceph-mon[35109]: pgmap v419: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-15T22:25:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:50 smithi117 ceph-mon[35021]: pgmap v419: 129 pgs: 129 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 169 op/s 2024-09-15T22:25:51.076 INFO:tasks.cephfs.filesystem:scrub status for tag:69f032d3-0ef4-4849-b29c-90b976257a86 - {'path': '/', 'tag': '69f032d3-0ef4-4849-b29c-90b976257a86', 'options': 'recursive,force'} 2024-09-15T22:25:51.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 600 sec 2024-09-15T22:25:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:25:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:25:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:25:52.076 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c8e6e028', quiesce took 7.0 seconds. Will release after: 4.4 2024-09-15T22:25:52.512 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 601 sec 2024-09-15T22:25:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:52 smithi196 ceph-mon[35109]: pgmap v420: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-15T22:25:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:52 smithi117 ceph-mon[35021]: pgmap v420: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-15T22:25:52.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:52 smithi077 ceph-mon[30980]: pgmap v420: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-15T22:25:53.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 602 sec 2024-09-15T22:25:54.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 603 sec 2024-09-15T22:25:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:54 smithi196 ceph-mon[35109]: pgmap v421: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:54 smithi117 ceph-mon[35021]: pgmap v421: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:54 smithi077 ceph-mon[30980]: pgmap v421: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:55.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 604 sec 2024-09-15T22:25:56.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:56.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 605 sec 2024-09-15T22:25:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:56 smithi196 ceph-mon[35109]: pgmap v422: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:56 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:25:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:56 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:25:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:56 smithi117 ceph-mon[35021]: pgmap v422: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:56 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:25:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:56 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:25:56.724 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:56.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:56 smithi077 ceph-mon[30980]: pgmap v422: 129 pgs: 129 active+clean; 581 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-15T22:25:56.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:56 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:25:56.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:56 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 -- 172.21.15.77:0/3400759107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102d40 msgr2=0x7f1f00103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/3400759107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102d40 0x7f1f00103120 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002a480 comp rx=0 tx=0).stop 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 -- 172.21.15.77:0/3400759107 shutdown_connections 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/3400759107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f0010c170 0x7f1f0010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/3400759107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00103660 0x7f1f0010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/3400759107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f00102d40 0x7f1f00103120 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 -- 172.21.15.77:0/3400759107 >> 172.21.15.77:0/3400759107 conn(0x7f1f000fc7a0 msgr2=0x7f1f000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 -- 172.21.15.77:0/3400759107 shutdown_connections 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.221+0000 7f1f05ce5640 1 -- 172.21.15.77:0/3400759107 wait complete. 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 Processor -- start 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 -- start start 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102d40 0x7f1f001009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f00103660 0x7f1f00100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00111f70 con 0x7f1f0010c170 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00111df0 con 0x7f1f00103660 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.222+0000 7f1f05ce5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001120f0 con 0x7f1f00102d40 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35422/0 (socket says 172.21.15.77:35422) 2024-09-15T22:25:57.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 -- 172.21.15.77:0/898396887 learned_addr learned my addr 172.21.15.77:0/898396887 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efeffd640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f00103660 0x7f1f00100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102d40 0x7f1f001009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 -- 172.21.15.77:0/898396887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102d40 msgr2=0x7f1f001009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102d40 0x7f1f001009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 -- 172.21.15.77:0/898396887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f00103660 msgr2=0x7f1f00100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f00103660 0x7f1f00100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 -- 172.21.15.77:0/898396887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f001a6910 con 0x7f1f0010c170 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efffff640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800eaa0 tx=0x7f1ef800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800cdb0 con 0x7f1f0010c170 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef8005020 con 0x7f1f0010c170 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f001a6c00 con 0x7f1f0010c170 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80116b0 con 0x7f1f0010c170 2024-09-15T22:25:57.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.223+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f001a7140 con 0x7f1f0010c170 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.224+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f00103120 con 0x7f1f0010c170 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.225+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1ef8005560 con 0x7f1f0010c170 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.226+0000 7f1efcff9640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.226+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.226+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f1ef80d01e0 con 0x7f1f0010c170 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.227+0000 7f1eff7fe640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009230 tx=0x7f1ef00058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:57.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.230+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ef8016030 con 0x7f1f0010c170 2024-09-15T22:25:57.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.429+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f001071e0 con 0x7f1f0010c170 2024-09-15T22:25:57.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.429+0000 7f1efcff9640 1 -- 172.21.15.77:0/898396887 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11989 (secure 0 0 0) 0x7f1ef809c1e0 con 0x7f1f0010c170 2024-09-15T22:25:57.432 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 34 2024-09-15T22:25:57.432 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:25:57.432 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":34,"btime":"2024-09-15T22:16:36:190507+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:16:35.405172+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:25:57.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 msgr2=0x7f1ee80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009230 tx=0x7f1ef00058d0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 msgr2=0x7f1f00101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800eaa0 tx=0x7f1ef800ef60 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 shutdown_connections 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1ee80a33b0 0x7f1ee80a5870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1f0010c170 0x7f1f00101420 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f00103660 0x7f1f00100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 --2- 172.21.15.77:0/898396887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f00102d40 0x7f1f001009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 >> 172.21.15.77:0/898396887 conn(0x7f1f000fc7a0 msgr2=0x7f1f000fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 shutdown_connections 2024-09-15T22:25:57.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:57.432+0000 7f1f05ce5640 1 -- 172.21.15.77:0/898396887 wait complete. 2024-09-15T22:25:57.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 606 sec 2024-09-15T22:25:58.427 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id c8e6e028 --release --await' 2024-09-15T22:25:58.428 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id c8e6e028 --release --await 2024-09-15T22:25:58.513 INFO:tasks.workunit.client.0.smithi077.stdout: 1 cleanup 607 sec 2024-09-15T22:25:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:58 smithi196 ceph-mon[35109]: pgmap v423: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-09-15T22:25:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:25:58 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/898396887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:58 smithi117 ceph-mon[35021]: pgmap v423: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-09-15T22:25:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:25:58 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/898396887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:58.713 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:58.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:58 smithi077 ceph-mon[30980]: pgmap v423: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.4 MiB/s wr, 197 op/s 2024-09-15T22:25:58.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:25:58 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/898396887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 -- 172.21.15.77:0/3492683676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c106a80 msgr2=0x7f239c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/3492683676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c106a80 0x7f239c106e60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2398009930 tx=0x7f239802a420 comp rx=0 tx=0).stop 2024-09-15T22:25:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 -- 172.21.15.77:0/3492683676 shutdown_connections 2024-09-15T22:25:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/3492683676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c101420 0x7f239c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/3492683676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/3492683676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c106a80 0x7f239c106e60 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 -- 172.21.15.77:0/3492683676 >> 172.21.15.77:0/3492683676 conn(0x7f239c0fc7a0 msgr2=0x7f239c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 -- 172.21.15.77:0/3492683676 shutdown_connections 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.196+0000 7f23a3d3e640 1 -- 172.21.15.77:0/3492683676 wait complete. 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 Processor -- start 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 -- start start 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c0fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 0x7f239c0fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f239c110ec0 con 0x7f239c101420 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f239c110d40 con 0x7f239c100a80 2024-09-15T22:25:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a3d3e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f239c111040 con 0x7f239c106a80 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43166/0 (socket says 172.21.15.77:43166) 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 -- 172.21.15.77:0/610895674 learned_addr learned my addr 172.21.15.77:0/610895674 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a22db640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 0x7f239c0fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a1ada640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c0fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 -- 172.21.15.77:0/610895674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 msgr2=0x7f239c0fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 0x7f239c0fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 -- 172.21.15.77:0/610895674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 msgr2=0x7f239c0fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c0fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a12d9640 1 -- 172.21.15.77:0/610895674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c1a69b0 con 0x7f239c101420 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a22db640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 0x7f239c0fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.197+0000 7f23a1ada640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c0fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:59.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f23a12d9640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f238c00b4a0 tx=0x7f238c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f2392ffd640 1 -- 172.21.15.77:0/610895674 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c014070 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f2392ffd640 1 -- 172.21.15.77:0/610895674 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238c010af0 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c1a6ca0 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f2392ffd640 1 -- 172.21.15.77:0/610895674 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c017d10 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.198+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c1a7190 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.200+0000 7f2392ffd640 1 -- 172.21.15.77:0/610895674 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f238c017070 con 0x7f239c101420 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.200+0000 7f23a6117740 1 Processor -- start 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 -- start start 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f1f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f2c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561323e84620 con 0x561323f1f980 2024-09-15T22:25:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561323e2be50 con 0x561323f2a240 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a6117740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561323e805e0 con 0x561323f1f060 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43178/0 (socket says 172.21.15.77:43178) 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 -- 172.21.15.77:0/3608711333 learned_addr learned my addr 172.21.15.77:0/3608711333 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f2392ffd640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 0x7f23880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a22db640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f1f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a22db640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 0x7f23880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f2392ffd640 1 -- 172.21.15.77:0/610895674 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f238c0cf3c0 con 0x7f239c101420 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a12d9640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f2c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 -- 172.21.15.77:0/3608711333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 msgr2=0x561323f1f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f1f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 -- 172.21.15.77:0/3608711333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 msgr2=0x561323f2c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f2c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 -- 172.21.15.77:0/3608711333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561323e1cd40 con 0x561323f1f980 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a12d9640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f2c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.201+0000 7f23a1ada640 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2398006010 tx=0x7f2398009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a186c1e2a644fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a22db640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 0x7f23880a5870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2394009fd0 tx=0x7f23940094a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f2387fff640 1 -- 172.21.15.77:0/3608711333 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239803f070 con 0x561323f1f980 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f2387fff640 1 -- 172.21.15.77:0/3608711333 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23980317b0 con 0x561323f1f980 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f2387fff640 1 -- 172.21.15.77:0/3608711333 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398035070 con 0x561323f1f980 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 -- 172.21.15.77:0/3608711333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 msgr2=0x561323f29d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2398006010 tx=0x7f2398009680 comp rx=0 tx=0).stop 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 -- 172.21.15.77:0/3608711333 shutdown_connections 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f2c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f29d00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 --2- 172.21.15.77:0/3608711333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f1f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.202+0000 7f23a6117740 1 -- 172.21.15.77:0/3608711333 >> 172.21.15.77:0/3608711333 conn(0x561323f1b5f0 msgr2=0x561323f1b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- 172.21.15.77:0/3608711333 shutdown_connections 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- 172.21.15.77:0/3608711333 wait complete. 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- start start 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 Processor -- start 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f27b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561323f2ce10 con 0x561323f1f980 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561323e0cf10 con 0x561323f2a240 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561323f2cf90 con 0x561323f1f060 2024-09-15T22:25:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a6117740 10 client.0 _ll_register_callbacks cb 0x561323e30db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.203+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43190/0 (socket says 172.21.15.77:43190) 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 -- 172.21.15.77:0/4084209683 learned_addr learned my addr 172.21.15.77:0/4084209683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a22db640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f27b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 -- 172.21.15.77:0/4084209683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 msgr2=0x561323f270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 -- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 msgr2=0x561323f27b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f27b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 -- 172.21.15.77:0/4084209683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561323b93580 con 0x561323f1f980 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a22db640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f27b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f238c097df0 tx=0x7f238c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e055cedcc1d0405 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23857fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c01e070 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f238c010ca0 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23980317b0 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f238c09f7d0 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a6117740 10 client.18852 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561323c5d960 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.204+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23980317b0 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f238c0b4150 con 0x561323f1f980 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 10 client.18852 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 10 client.18852 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 20 client.18852 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323f39bb0 0x561323f3c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 4 client.18852 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:59.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561323e91b90 con 0x561323f39bb0 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323f39bb0 0x561323f3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23857fa640 10 client.18852 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.205+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323f39bb0 0x561323f3c000 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2bf8ccf906f969c server_cookie=30eb9cd4b55cc867 in_seq=0 out_seq=0 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23857fa640 1 -- 172.21.15.77:0/4084209683 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561323e91b90 con 0x561323f39bb0 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23857fa640 10 client.18852 handle_command_reply: tid=0 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 client.18852 shutdown 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 msgr2=0x561323f275e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f238c097df0 tx=0x7f238c0046c0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 shutdown_connections 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323f39bb0 0x561323f3c000 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f2a240 0x561323f27b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f275e0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23857fa640 0 client.18852 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:59.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 --2- 172.21.15.77:0/4084209683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 >> 172.21.15.77:0/4084209683 conn(0x561323f1b5f0 msgr2=0x561323f212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 shutdown_connections 2024-09-15T22:25:59.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 1 -- 172.21.15.77:0/4084209683 wait complete. 2024-09-15T22:25:59.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.206+0000 7f23a6117740 20 client.18852 trim_cache size 0 max 16384 2024-09-15T22:25:59.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 Processor -- start 2024-09-15T22:25:59.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 -- start start 2024-09-15T22:25:59.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f418d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f980 0x561323f41e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1b1260 con 0x561323f2a240 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1b2da0 con 0x561323f1f980 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.223+0000 7f23a6117740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1b1930 con 0x561323f1f060 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43206/0 (socket says 172.21.15.77:43206) 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 -- 172.21.15.77:0/217346163 learned_addr learned my addr 172.21.15.77:0/217346163 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 -- 172.21.15.77:0/217346163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 msgr2=0x561323f418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 -- 172.21.15.77:0/217346163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f980 msgr2=0x561323f41e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f980 0x561323f41e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 -- 172.21.15.77:0/217346163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561323f1b420 con 0x561323f2a240 2024-09-15T22:25:59.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23a22db640 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f239400daf0 tx=0x7f2394009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22feb62074ff44b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23877fe640 1 -- 172.21.15.77:0/217346163 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400fb70 con 0x561323f2a240 2024-09-15T22:25:59.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23877fe640 1 -- 172.21.15.77:0/217346163 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239400fd10 con 0x561323f2a240 2024-09-15T22:25:59.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.224+0000 7f23877fe640 1 -- 172.21.15.77:0/217346163 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394029820 con 0x561323f2a240 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- 172.21.15.77:0/217346163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 msgr2=0x561323f24300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f239400daf0 tx=0x7f2394009770 comp rx=0 tx=0).stop 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- 172.21.15.77:0/217346163 shutdown_connections 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f2a240 0x561323f24300 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f980 0x561323f41e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 --2- 172.21.15.77:0/217346163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f1f060 0x561323f418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- 172.21.15.77:0/217346163 >> 172.21.15.77:0/217346163 conn(0x561323f1b5f0 msgr2=0x561323f25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- 172.21.15.77:0/217346163 shutdown_connections 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- 172.21.15.77:0/217346163 wait complete. 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 -- start start 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.225+0000 7f23a6117740 1 Processor -- start 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f060 0x561323f32520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f2a240 0x561323f2d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1b26d0 con 0x561323f1f980 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1b2000 con 0x561323f1f060 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f239c1af260 con 0x561323f2a240 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 10 client.0 _ll_register_callbacks cb 0x7f238c09b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:59.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43208/0 (socket says 172.21.15.77:43208) 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 -- 172.21.15.77:0/4075217322 learned_addr learned my addr 172.21.15.77:0/4075217322 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f060 0x561323f32520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a22db640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f2a240 0x561323f2d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 -- 172.21.15.77:0/4075217322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f2a240 msgr2=0x561323f2d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f2a240 0x561323f2d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 -- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f060 msgr2=0x561323f32520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f060 0x561323f32520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 -- 172.21.15.77:0/4075217322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561323f2db00 con 0x561323f1f980 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a12d9640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f238c00be60 tx=0x7f238c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc180af8ac946bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23857fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c01e070 con 0x561323f1f980 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9438+0+0 (secure 0 0 0) 0x7f238c0b4150 con 0x561323f1f980 2024-09-15T22:25:59.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.226+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f239400fd10 con 0x561323f1f980 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f238c009230 con 0x561323f1f980 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 10 client.18858 fetch_fsmap learned FSMap version 34 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561323fa5500 con 0x561323f1f980 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239400fd10 con 0x561323f1f980 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f238c097170 con 0x561323f1f980 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 10 client.18858 fetch_fsmap finished waiting for FSMap version 34 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 10 client.18858 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 20 client.18858 populate_metadata read hostname 'smithi077' 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323fa5a50 0x561323fa7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 4 client.18858 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c8e6e028", "release": true, "await": true}] 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c8e6e028", "release": true, "await": true}) -- 0x561323e8c0a0 con 0x561323fa5a50 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.227+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323fa5a50 0x561323fa7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.228+0000 7f23a1ada640 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323fa5a50 0x561323fa7e70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1629be30c464658d server_cookie=d6accb7cfa1735d3 in_seq=0 out_seq=0 2024-09-15T22:25:59.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.228+0000 7f23857fa640 10 client.18858 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:59.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.247+0000 7f23857fa640 1 -- 172.21.15.77:0/4075217322 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561323e8c0a0 con 0x561323fa5a50 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.247+0000 7f23857fa640 10 client.18858 handle_command_reply: tid=0 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.247+0000 7f23a6117740 1 client.18858 shutdown 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 msgr2=0x561323f32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f238c00be60 tx=0x7f238c0047d0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 shutdown_connections 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561323fa5a50 0x561323fa7e70 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561323f2a240 0x561323f2d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561323f1f980 0x561323f32a60 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 --2- 172.21.15.77:0/4075217322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561323f1f060 0x561323f32520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23857fa640 0 client.18858 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 >> 172.21.15.77:0/4075217322 conn(0x561323f1b5f0 msgr2=0x561323f19f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 shutdown_connections 2024-09-15T22:25:59.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 1 -- 172.21.15.77:0/4075217322 wait complete. 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a6117740 20 client.18858 trim_cache size 0 max 16384 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 34, 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 48, 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "c8e6e028": { 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 48, 2024-09-15T22:25:59.251 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 131.4, 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:25:59.252 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:59.253 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:59.253 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:59.253 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:59.253 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:25:59.253 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 msgr2=0x7f23880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 0x7f23880a5870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2394009fd0 tx=0x7f23940094a0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 msgr2=0x7f239c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.248+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f238c00b4a0 tx=0x7f238c00b960 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 shutdown_connections 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f23880a33b0 0x7f23880a5870 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f239c106a80 0x7f239c0fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f239c101420 0x7f239c0fd300 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 --2- 172.21.15.77:0/610895674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c100a80 0x7f239c0fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 >> 172.21.15.77:0/610895674 conn(0x7f239c0fc7a0 msgr2=0x7f239c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 shutdown_connections 2024-09-15T22:25:59.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:25:59.249+0000 7f23a3d3e640 1 -- 172.21.15.77:0/610895674 wait complete. 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: 0 cleanup 608 sec 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: Operation Count AvgLat MaxLat 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: ---------------------------------------- 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: NTCreateX 143597 1.449 27166.963 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: Close 105535 0.002 0.088 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: Rename 6084 4.616 127.385 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: Unlink 28986 2.919 22322.924 2024-09-15T22:25:59.461 INFO:tasks.workunit.client.0.smithi077.stdout: Deltree 4 87.985 134.469 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Mkdir 2 0.004 0.004 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Qpathinfo 130200 0.895 526.869 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Qfileinfo 22839 0.001 0.038 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Qfsinfo 23876 0.199 28.859 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Sfileinfo 11708 5.420 25133.952 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Find 50355 0.421 14749.186 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: WriteX 71700 0.419 20959.846 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: ReadX 225286 0.032 182.599 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: LockX 468 0.519 10.713 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: UnlockX 468 0.548 1.983 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: Flush 10074 4.079 336.372 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:25:59.462 INFO:tasks.workunit.client.0.smithi077.stdout:Throughput 7.53838 MB/sec 1 clients 1 procs max_latency=27166.968 ms 2024-09-15T22:25:59.463 INFO:tasks.workunit.client.0.smithi077.stderr:+ dbench 10 2024-09-15T22:25:59.463 INFO:tasks.workunit.client.0.smithi077.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-15T22:25:59.463 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:25:59.463 INFO:tasks.workunit.client.0.smithi077.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-15T22:25:59.463 INFO:tasks.workunit.client.0.smithi077.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-15T22:25:59.563 INFO:tasks.workunit.client.0.smithi077.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-15T22:25:59.563 INFO:tasks.workunit.client.0.smithi077.stdout:releasing clients 2024-09-15T22:26:00.360 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c8e6e028', total seconds elapsed: 15.3 2024-09-15T22:26:00.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 373 133.32 MB/sec warmup 1 sec latency 80.751 ms 2024-09-15T22:26:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:00 smithi077 ceph-mon[30980]: pgmap v424: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-15T22:26:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:00 smithi196 ceph-mon[35109]: pgmap v424: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-15T22:26:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:00 smithi117 ceph-mon[35021]: pgmap v424: 129 pgs: 129 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 111 op/s 2024-09-15T22:26:01.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 787 107.14 MB/sec warmup 2 sec latency 184.792 ms 2024-09-15T22:26:02.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 1075 75.67 MB/sec warmup 3 sec latency 507.207 ms 2024-09-15T22:26:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:02 smithi117 ceph-mon[35021]: pgmap v425: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 164 op/s 2024-09-15T22:26:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:02 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:02 smithi117 ceph-mon[35021]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:26:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:02 smithi077 ceph-mon[30980]: pgmap v425: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 164 op/s 2024-09-15T22:26:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:02 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:02 smithi077 ceph-mon[30980]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:26:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:02 smithi196 ceph-mon[35109]: pgmap v425: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 164 op/s 2024-09-15T22:26:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:02 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:02 smithi196 ceph-mon[35109]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:26:03.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 1870 68.02 MB/sec warmup 4 sec latency 1282.371 ms 2024-09-15T22:26:04.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 2721 62.27 MB/sec warmup 5 sec latency 2282.422 ms 2024-09-15T22:26:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:04 smithi117 ceph-mon[35021]: pgmap v427: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:04.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:04 smithi077 ceph-mon[30980]: pgmap v427: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:04 smithi196 ceph-mon[35109]: pgmap v427: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:05.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 3316 60.01 MB/sec warmup 6 sec latency 3219.910 ms 2024-09-15T22:26:06.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 3832 55.94 MB/sec warmup 7 sec latency 54.415 ms 2024-09-15T22:26:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:06 smithi117 ceph-mon[35021]: pgmap v428: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:06.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:06 smithi077 ceph-mon[30980]: pgmap v428: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:06 smithi196 ceph-mon[35109]: pgmap v428: 129 pgs: 129 active+clean; 582 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 100 op/s 2024-09-15T22:26:07.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 4382 52.20 MB/sec warmup 8 sec latency 54.864 ms 2024-09-15T22:26:07.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:07 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:07.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:07 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:07.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:07 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:26:08.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 4869 48.47 MB/sec warmup 9 sec latency 69.580 ms 2024-09-15T22:26:08.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:08 smithi077 ceph-mon[30980]: pgmap v429: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:08 smithi196 ceph-mon[35109]: pgmap v429: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:08 smithi117 ceph-mon[35021]: pgmap v429: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:09.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 5516 46.55 MB/sec warmup 10 sec latency 60.278 ms 2024-09-15T22:26:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:09 smithi077 ceph-mon[30980]: pgmap v430: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:10.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:09 smithi077 ceph-mon[30980]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:26:10.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:09 smithi196 ceph-mon[35109]: pgmap v430: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:10.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:09 smithi196 ceph-mon[35109]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:26:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:09 smithi117 ceph-mon[35021]: pgmap v430: 129 pgs: 129 active+clean; 742 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 22 MiB/s wr, 395 op/s 2024-09-15T22:26:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:09 smithi117 ceph-mon[35021]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:26:10.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 6210 46.47 MB/sec warmup 11 sec latency 92.919 ms 2024-09-15T22:26:11.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 7012 45.09 MB/sec warmup 12 sec latency 62.650 ms 2024-09-15T22:26:12.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:11 smithi077 ceph-mon[30980]: pgmap v432: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 593 op/s 2024-09-15T22:26:12.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:11 smithi196 ceph-mon[35109]: pgmap v432: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 593 op/s 2024-09-15T22:26:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:11 smithi117 ceph-mon[35021]: pgmap v432: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 593 op/s 2024-09-15T22:26:12.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 7817 45.58 MB/sec warmup 13 sec latency 135.753 ms 2024-09-15T22:26:13.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 8527 45.76 MB/sec warmup 14 sec latency 69.158 ms 2024-09-15T22:26:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:13 smithi077 ceph-mon[30980]: pgmap v433: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:14.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:13 smithi196 ceph-mon[35109]: pgmap v433: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:14.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:13 smithi117 ceph-mon[35021]: pgmap v433: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:14.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 9056 44.92 MB/sec warmup 15 sec latency 66.205 ms 2024-09-15T22:26:15.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 9533 43.20 MB/sec warmup 16 sec latency 127.092 ms 2024-09-15T22:26:16.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:15 smithi077 ceph-mon[30980]: pgmap v434: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:16.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:15 smithi196 ceph-mon[35109]: pgmap v434: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:15 smithi117 ceph-mon[35021]: pgmap v434: 129 pgs: 129 active+clean; 763 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 549 op/s 2024-09-15T22:26:16.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 9892 41.36 MB/sec warmup 17 sec latency 192.429 ms 2024-09-15T22:26:17.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 10530 40.99 MB/sec warmup 18 sec latency 212.630 ms 2024-09-15T22:26:18.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:17 smithi077 ceph-mon[30980]: pgmap v435: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:18.163 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:17 smithi117 ceph-mon[35021]: pgmap v435: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:18.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:17 smithi196 ceph-mon[35109]: pgmap v435: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:18.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 11469 41.00 MB/sec warmup 19 sec latency 57.916 ms 2024-09-15T22:26:19.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 12381 42.22 MB/sec warmup 20 sec latency 35.127 ms 2024-09-15T22:26:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:19 smithi077 ceph-mon[30980]: pgmap v436: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:19 smithi196 ceph-mon[35109]: pgmap v436: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:19 smithi117 ceph-mon[35021]: pgmap v436: 129 pgs: 129 active+clean; 784 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.9 MiB/s wr, 444 op/s 2024-09-15T22:26:20.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 12977 42.03 MB/sec warmup 21 sec latency 57.371 ms 2024-09-15T22:26:21.078 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:26:21.397 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:21.532 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-09-15T22:26:21.532 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:26:21.563 INFO:tasks.workunit.client.0.smithi077.stdout: 10 13566 41.43 MB/sec warmup 22 sec latency 83.747 ms 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 -- 172.21.15.77:0/2494845072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 msgr2=0x7fa4c810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/2494845072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c810ec40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009cd0 tx=0x7fa4c002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 -- 172.21.15.77:0/2494845072 shutdown_connections 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/2494845072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c810ec40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/2494845072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/2494845072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.954+0000 7fa4ce52d640 1 -- 172.21.15.77:0/2494845072 >> 172.21.15.77:0/2494845072 conn(0x7fa4c80fc7d0 msgr2=0x7fa4c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 -- 172.21.15.77:0/2494845072 shutdown_connections 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 -- 172.21.15.77:0/2494845072 wait complete. 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 Processor -- start 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 -- start start 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.955+0000 7fa4ce52d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4ce52d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4ce52d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81120a0 con 0x7fa4c81082a0 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4ce52d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8111f20 con 0x7fa4c8102d30 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4ce52d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8112220 con 0x7fa4c81036e0 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4ccaca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36750/0 (socket says 172.21.15.77:36750) 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 -- 172.21.15.77:0/1931012254 learned_addr learned my addr 172.21.15.77:0/1931012254 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43078/0 (socket says 172.21.15.77:43078) 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 -- 172.21.15.77:0/1931012254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 msgr2=0x7fa4c8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 -- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 msgr2=0x7fa4c819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 -- 172.21.15.77:0/1931012254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c819d510 con 0x7fa4c81036e0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000e9c0 tx=0x7fa4b000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.956+0000 7fa4c5ffb640 1 -- 172.21.15.77:0/1931012254 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000ccc0 con 0x7fa4c81036e0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.957+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c819d800 con 0x7fa4c81036e0 2024-09-15T22:26:21.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.957+0000 7fa4c5ffb640 1 -- 172.21.15.77:0/1931012254 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0005500 con 0x7fa4c81036e0 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.957+0000 7fa4c5ffb640 1 -- 172.21.15.77:0/1931012254 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0010690 con 0x7fa4c81036e0 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.957+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81a50e0 con 0x7fa4c81036e0 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.959+0000 7fa4c5ffb640 1 -- 172.21.15.77:0/1931012254 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa4b0004e80 con 0x7fa4c81036e0 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.959+0000 7fa4d0906740 1 Processor -- start 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 -- start start 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c5ffb640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 0x7fa4b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 0x7fa4b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36764/0 (socket says 172.21.15.77:36764) 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c5ffb640 1 -- 172.21.15.77:0/1931012254 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa4b00d0470 con 0x7fa4c81036e0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b7a72462e0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7a71edce0 con 0x55b7a72eb680 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4d0906740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7a72422a0 con 0x55b7a72e04a0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1396637754 learned_addr learned my addr 172.21.15.77:0/1396637754 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1396637754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 msgr2=0x55b7a72eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:21.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1396637754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 msgr2=0x55b7a72eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72eb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.960+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1396637754 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a71d2440 con 0x55b7a72e04a0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.961+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009cf0 tx=0x7fa4b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec551fc7dffe41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.961+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 0x7fa4b40a58f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0004ab0 tx=0x7fa4c0031430 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.961+0000 7fa4be7fc640 1 -- 172.21.15.77:0/1396637754 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8004030 con 0x55b7a72e04a0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.961+0000 7fa4be7fc640 1 -- 172.21.15.77:0/1396637754 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b80041d0 con 0x55b7a72e04a0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.961+0000 7fa4be7fc640 1 -- 172.21.15.77:0/1396637754 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8016810 con 0x55b7a72e04a0 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- 172.21.15.77:0/1396637754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 msgr2=0x55b7a72e0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009cf0 tx=0x7fa4b8011810 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- 172.21.15.77:0/1396637754 shutdown_connections 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1396637754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72e0880 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- 172.21.15.77:0/1396637754 >> 172.21.15.77:0/1396637754 conn(0x55b7a72dca30 msgr2=0x55b7a72dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- 172.21.15.77:0/1396637754 shutdown_connections 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- 172.21.15.77:0/1396637754 wait complete. 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 -- start start 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.962+0000 7fa4d0906740 1 Processor -- start 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72e8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b7a72ee250 con 0x55b7a72e0dc0 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7a7201050 con 0x55b7a72eb680 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36774/0 (socket says 172.21.15.77:36774) 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4c7fff640 1 -- 172.21.15.77:0/3938703187 learned_addr learned my addr 172.21.15.77:0/3938703187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7a72ee3d0 con 0x55b7a72e04a0 2024-09-15T22:26:21.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4d0906740 10 client.0 _ll_register_callbacks cb 0x55b7a7241160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 -- 172.21.15.77:0/3938703187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 msgr2=0x55b7a72eaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 -- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 msgr2=0x55b7a72e8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72e8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4bffff640 1 -- 172.21.15.77:0/3938703187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a6fb73c0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.963+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4bffff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e81e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00cd7d0 tx=0x7fa4b000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5f6f639f97712c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:21.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00b5060 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa4b00b5200 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b80041d0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4b00a4fd0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.964+0000 7fa4d0906740 10 client.18870 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b7a72f9bd0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b80041d0 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa4b00a4b40 con 0x55b7a72e0dc0 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 10 client.18870 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 10 client.18870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 20 client.18870 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72fb090 0x55b7a72fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 4 client.18870 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:21.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a7253850 con 0x55b7a72fb090 2024-09-15T22:26:21.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.965+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72fb090 0x55b7a72fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:21.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.966+0000 7fa4a77fe640 10 client.18870 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:21.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.966+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72fb090 0x55b7a72fd480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197075d6e6cf70e5 server_cookie=ad577ebe6128dd90 in_seq=0 out_seq=0 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.970+0000 7fa4a77fe640 1 -- 172.21.15.77:0/3938703187 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7a7253850 con 0x55b7a72fb090 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.970+0000 7fa4a77fe640 10 client.18870 handle_command_reply: tid=0 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.970+0000 7fa4d0906740 1 client.18870 shutdown 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.970+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 msgr2=0x55b7a72e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.970+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e81e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00cd7d0 tx=0x7fa4b000cf20 comp rx=0 tx=0).stop 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 shutdown_connections 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72fb090 0x55b7a72fd480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72eb680 0x55b7a72e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e81e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 --2- 172.21.15.77:0/3938703187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e04a0 0x55b7a72eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 >> 172.21.15.77:0/3938703187 conn(0x55b7a72dca30 msgr2=0x55b7a72e2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4a77fe640 0 client.18870 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 shutdown_connections 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 1 -- 172.21.15.77:0/3938703187 wait complete. 2024-09-15T22:26:21.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:21.971+0000 7fa4d0906740 20 client.18870 trim_cache size 0 max 16384 2024-09-15T22:26:22.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.004+0000 7fa4d0906740 1 Processor -- start 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.004+0000 7fa4d0906740 1 -- start start 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72dca30 0x55b7a72f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81aceb0 con 0x55b7a72e04a0 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81aeab0 con 0x55b7a72dca30 2024-09-15T22:26:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4d0906740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81ad580 con 0x55b7a72e0dc0 2024-09-15T22:26:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.005+0000 7fa4ccaca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.006+0000 7fa4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.006+0000 7fa4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52272/0 (socket says 172.21.15.77:52272) 2024-09-15T22:26:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.006+0000 7fa4bffff640 1 -- 172.21.15.77:0/1189822019 learned_addr learned my addr 172.21.15.77:0/1189822019 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:22.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 -- 172.21.15.77:0/1189822019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 msgr2=0x55b7a72e37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:22.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 -- 172.21.15.77:0/1189822019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72dca30 msgr2=0x55b7a72f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72dca30 0x55b7a72f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 -- 172.21.15.77:0/1189822019 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a72fd7f0 con 0x55b7a72e04a0 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.007+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa4b009c060 tx=0x7fa4b009a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4d5ff07a799398 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.008+0000 7fa4bdffb640 1 -- 172.21.15.77:0/1189822019 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b009bea0 con 0x55b7a72e04a0 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.008+0000 7fa4bdffb640 1 -- 172.21.15.77:0/1189822019 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b009bad0 con 0x55b7a72e04a0 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.008+0000 7fa4bdffb640 1 -- 172.21.15.77:0/1189822019 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00a4b40 con 0x55b7a72e04a0 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.009+0000 7fa4d0906740 1 -- 172.21.15.77:0/1189822019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 msgr2=0x55b7a72e81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.009+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa4b009c060 tx=0x7fa4b009a870 comp rx=0 tx=0).stop 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.011+0000 7fa4d0906740 1 -- 172.21.15.77:0/1189822019 shutdown_connections 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.012+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72e37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.012+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72e04a0 0x55b7a72e81a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.012+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1189822019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72dca30 0x55b7a72f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.012+0000 7fa4d0906740 1 -- 172.21.15.77:0/1189822019 >> 172.21.15.77:0/1189822019 conn(0x55b7a72e8880 msgr2=0x55b7a72e30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.016+0000 7fa4d0906740 1 -- 172.21.15.77:0/1189822019 shutdown_connections 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.017+0000 7fa4d0906740 1 -- 172.21.15.77:0/1189822019 wait complete. 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.018+0000 7fa4d0906740 1 -- start start 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.018+0000 7fa4d0906740 1 Processor -- start 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.021+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.021+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72e04a0 0x55b7a72ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.022+0000 7fa4d0906740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72ef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.022+0000 7fa4d0906740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81ae320 con 0x55b7a72dca30 2024-09-15T22:26:22.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.022+0000 7fa4d0906740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81adc50 con 0x55b7a72e04a0 2024-09-15T22:26:22.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.022+0000 7fa4d0906740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81aaeb0 con 0x55b7a72e0dc0 2024-09-15T22:26:22.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.022+0000 7fa4d0906740 10 client.0 _ll_register_callbacks cb 0x7fa4b009c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:22.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.026+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:22.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.026+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52276/0 (socket says 172.21.15.77:52276) 2024-09-15T22:26:22.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.026+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1095279592 learned_addr learned my addr 172.21.15.77:0/1095279592 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.026+0000 7fa4ccaca640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.032+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1095279592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 msgr2=0x55b7a72ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.032+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72ef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.032+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72e04a0 msgr2=0x55b7a72ef130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:26:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.032+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72e04a0 0x55b7a72ef130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.032+0000 7fa4c7fff640 1 -- 172.21.15.77:0/1095279592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a72eb2a0 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.036+0000 7fa4c7fff640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009910 tx=0x7fa4b8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed489dc6bff86c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.036+0000 7fa4a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.037+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8003f30 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.037+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b009bad0 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.038+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa4b8024040 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.038+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4b803b050 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.039+0000 7fa4d0906740 10 client.18876 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.039+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b7a7365bb0 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.039+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b009bad0 con 0x55b7a72dca30 2024-09-15T22:26:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.039+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa4b8029630 con 0x55b7a72dca30 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 10 client.18876 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 10 client.18876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 20 client.18876 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72eb680 0x55b7a73682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 4 client.18876 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:26:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.040+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7a724dd60 con 0x55b7a72eb680 2024-09-15T22:26:22.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.046+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72eb680 0x55b7a73682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:22.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.060+0000 7fa4bffff640 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72eb680 0x55b7a73682d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f99be8d2f05c28 server_cookie=1b9fb79db7214ba9 in_seq=0 out_seq=0 2024-09-15T22:26:22.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.060+0000 7fa4a77fe640 10 client.18876 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.065+0000 7fa4a77fe640 1 -- 172.21.15.77:0/1095279592 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7a724dd60 con 0x55b7a72eb680 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.065+0000 7fa4a77fe640 10 client.18876 handle_command_reply: tid=0 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.065+0000 7fa4d0906740 1 client.18876 shutdown 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 msgr2=0x55b7a72f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009910 tx=0x7fa4b8011e50 comp rx=0 tx=0).stop 2024-09-15T22:26:22.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 shutdown_connections 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b7a72eb680 0x55b7a73682d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7a72e0dc0 0x55b7a72ef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7a72e04a0 0x55b7a72ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 --2- 172.21.15.77:0/1095279592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b7a72dca30 0x55b7a72f3e00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 >> 172.21.15.77:0/1095279592 conn(0x55b7a72e8880 msgr2=0x55b7a72ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4a77fe640 0 client.18876 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 shutdown_connections 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 1 -- 172.21.15.77:0/1095279592 wait complete. 2024-09-15T22:26:22.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.067+0000 7fa4d0906740 20 client.18876 trim_cache size 0 max 16384 2024-09-15T22:26:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.071+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 msgr2=0x7fa4b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.071+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 0x7fa4b40a58f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0004ab0 tx=0x7fa4c0031430 comp rx=0 tx=0).stop 2024-09-15T22:26:22.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.071+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 msgr2=0x7fa4c8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.071+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000e9c0 tx=0x7fa4b000ee80 comp rx=0 tx=0).stop 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 shutdown_connections 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa4b40a3430 0x7fa4b40a58f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4c81082a0 0x7fa4c819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c81036e0 0x7fa4c8198aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 --2- 172.21.15.77:0/1931012254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c8102d30 0x7fa4c8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:22.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 >> 172.21.15.77:0/1931012254 conn(0x7fa4c80fc7d0 msgr2=0x7fa4c8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:22.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.074+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 shutdown_connections 2024-09-15T22:26:22.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:22.077+0000 7fa4ce52d640 1 -- 172.21.15.77:0/1931012254 wait complete. 2024-09-15T22:26:22.105 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:21 smithi077 ceph-mon[30980]: pgmap v437: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-09-15T22:26:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:21 smithi196 ceph-mon[35109]: pgmap v437: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-09-15T22:26:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:21 smithi117 ceph-mon[35021]: pgmap v437: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 573 op/s 2024-09-15T22:26:22.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 13769 40.14 MB/sec warmup 23 sec latency 370.944 ms 2024-09-15T22:26:23.121 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:23.162 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:26:23.162 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:26:23.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 14056 39.19 MB/sec warmup 24 sec latency 339.767 ms 2024-09-15T22:26:23.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.693+0000 7f77b60ef640 1 -- 172.21.15.77:0/1921560727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b01008c0 msgr2=0x7f77b0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:23.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.693+0000 7f77b60ef640 1 --2- 172.21.15.77:0/1921560727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b01008c0 0x7f77b0100ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009890 tx=0x7f77a002a380 comp rx=0 tx=0).stop 2024-09-15T22:26:23.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 -- 172.21.15.77:0/1921560727 shutdown_connections 2024-09-15T22:26:23.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 --2- 172.21.15.77:0/1921560727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b0105ec0 0x7f77b0106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 --2- 172.21.15.77:0/1921560727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 --2- 172.21.15.77:0/1921560727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b01008c0 0x7f77b0100ca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 -- 172.21.15.77:0/1921560727 >> 172.21.15.77:0/1921560727 conn(0x7f77b00fc7d0 msgr2=0x7f77b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 -- 172.21.15.77:0/1921560727 shutdown_connections 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 -- 172.21.15.77:0/1921560727 wait complete. 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.694+0000 7f77b60ef640 1 Processor -- start 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 -- start start 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 0x7f77b01a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b0105ec0 0x7f77b019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0118630 con 0x7f77b01008c0 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f77b01184b0 con 0x7f77b0105ec0 2024-09-15T22:26:23.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77b60ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77b01187b0 con 0x7f77b01011e0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 0x7f77b01a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 0x7f77b01a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52306/0 (socket says 172.21.15.77:52306) 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77affff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b0105ec0 0x7f77b019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77af7fe640 1 -- 172.21.15.77:0/2094529535 learned_addr learned my addr 172.21.15.77:0/2094529535 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.695+0000 7f77aeffd640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 -- 172.21.15.77:0/2094529535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b0105ec0 msgr2=0x7f77b019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b0105ec0 0x7f77b019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 -- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 msgr2=0x7f77b01a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 0x7f77b01a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 -- 172.21.15.77:0/2094529535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b019f110 con 0x7f77b01011e0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77aeffd640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b01a4d60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009fd0 tx=0x7f77a400eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004220 con 0x7f77b01011e0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a400cd90 con 0x7f77b01011e0 2024-09-15T22:26:23.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b019f400 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.696+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004a00 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.697+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b01ab6f0 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.699+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b006a850 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.699+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77a4004ba0 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.700+0000 7f77acff9640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 0x7f77980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.700+0000 7f77af7fe640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 0x7f77980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.700+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f77a40d00b0 con 0x7f77b01011e0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.701+0000 7f77af7fe640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 0x7f77980a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f77a0035040 tx=0x7f77a00094d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:23.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:23.705+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77a409c6a0 con 0x7f77b01011e0 2024-09-15T22:26:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:23.952 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:23 smithi077 ceph-mon[30980]: pgmap v438: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:24.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.039+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b01ab8c0 con 0x7f77b01011e0 2024-09-15T22:26:24.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.041+0000 7f77acff9640 1 -- 172.21.15.77:0/2094529535 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7f77a409bfa0 con 0x7f77b01011e0 2024-09-15T22:26:24.042 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:26:24.043 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:26:24.043 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:26:24.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 msgr2=0x7f77980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:24.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 0x7f77980a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f77a0035040 tx=0x7f77a00094d0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 msgr2=0x7f77b01a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:24.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b01a4d60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009fd0 tx=0x7f77a400eb60 comp rx=0 tx=0).stop 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 shutdown_connections 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77980a33b0 0x7f77980a5870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b0105ec0 0x7f77b019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77b01011e0 0x7f77b01a4d60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 --2- 172.21.15.77:0/2094529535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77b01008c0 0x7f77b01a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 >> 172.21.15.77:0/2094529535 conn(0x7f77b00fc7d0 msgr2=0x7f77b0103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 shutdown_connections 2024-09-15T22:26:24.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.044+0000 7f77b60ef640 1 -- 172.21.15.77:0/2094529535 wait complete. 2024-09-15T22:26:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:23 smithi196 ceph-mon[35109]: pgmap v438: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:23 smithi117 ceph-mon[35021]: pgmap v438: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:24.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 14435 38.19 MB/sec warmup 25 sec latency 72.511 ms 2024-09-15T22:26:24.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-09-15T22:26:24.765 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 -- 172.21.15.77:0/2367541184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40106b70 msgr2=0x7fee40106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 --2- 172.21.15.77:0/2367541184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40106b70 0x7fee40106f50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fee34009900 tx=0x7fee3402a410 comp rx=0 tx=0).stop 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 -- 172.21.15.77:0/2367541184 shutdown_connections 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 --2- 172.21.15.77:0/2367541184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee401057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 --2- 172.21.15.77:0/2367541184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40100090 0x7fee40100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 --2- 172.21.15.77:0/2367541184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40106b70 0x7fee40106f50 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 -- 172.21.15.77:0/2367541184 >> 172.21.15.77:0/2367541184 conn(0x7fee40077210 msgr2=0x7fee400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:24.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 -- 172.21.15.77:0/2367541184 shutdown_connections 2024-09-15T22:26:24.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.990+0000 7fee46dc0640 1 -- 172.21.15.77:0/2367541184 wait complete. 2024-09-15T22:26:24.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.991+0000 7fee46dc0640 1 Processor -- start 2024-09-15T22:26:24.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 -- start start 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40100090 0x7fee400738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40106b70 0x7fee4006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee40118610 con 0x7fee40100090 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee40118490 con 0x7fee40106b70 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee46dc0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee40118790 con 0x7fee40100a50 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:24.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee3ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36830/0 (socket says 172.21.15.77:36830) 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.992+0000 7fee3ffff640 1 -- 172.21.15.77:0/3891778567 learned_addr learned my addr 172.21.15.77:0/3891778567 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee4535d640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40106b70 0x7fee4006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee3ffff640 1 -- 172.21.15.77:0/3891778567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40106b70 msgr2=0x7fee4006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee3ffff640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40106b70 0x7fee4006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee3ffff640 1 -- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40100090 msgr2=0x7fee400738d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee3ffff640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40100090 0x7fee400738d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.993+0000 7fee3ffff640 1 -- 172.21.15.77:0/3891778567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4006d340 con 0x7fee40100a50 2024-09-15T22:26:24.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.994+0000 7fee3ffff640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c920 tx=0x7fee2c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.994+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007c70 con 0x7fee40100a50 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.994+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4006d630 con 0x7fee40100a50 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.994+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee40071680 con 0x7fee40100a50 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.995+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c007e10 con 0x7fee40100a50 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.995+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0188c0 con 0x7fee40100a50 2024-09-15T22:26:24.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.996+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24005290 con 0x7fee40100a50 2024-09-15T22:26:24.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.998+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fee2c018b20 con 0x7fee40100a50 2024-09-15T22:26:25.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:24.999+0000 7fee3dffb640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 0x7fee300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:25.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.000+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fee2c0d0470 con 0x7fee40100a50 2024-09-15T22:26:25.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.000+0000 7fee44b5c640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 0x7fee300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:25.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.000+0000 7fee44b5c640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 0x7fee300a58f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee3402a930 tx=0x7fee34005fd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:25.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.005+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee2c014030 con 0x7fee40100a50 2024-09-15T22:26:25.157 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:25.287 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2094529535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:25.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.318+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee24005580 con 0x7fee40100a50 2024-09-15T22:26:25.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.320+0000 7fee3dffb640 1 -- 172.21.15.77:0/3891778567 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7fee2c09cb40 con 0x7fee40100a50 2024-09-15T22:26:25.322 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:26:25.322 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:26:25.322 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 msgr2=0x7fee300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 0x7fee300a58f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee3402a930 tx=0x7fee34005fd0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 msgr2=0x7fee40073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c920 tx=0x7fee2c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 shutdown_connections 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fee300a3430 0x7fee300a58f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee40106b70 0x7fee4006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee40100a50 0x7fee40073e10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 --2- 172.21.15.77:0/3891778567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee40100090 0x7fee400738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 >> 172.21.15.77:0/3891778567 conn(0x7fee40077210 msgr2=0x7fee401037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:25.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 shutdown_connections 2024-09-15T22:26:25.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:25.323+0000 7fee46dc0640 1 -- 172.21.15.77:0/3891778567 wait complete. 2024-09-15T22:26:25.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2094529535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2094529535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:25.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 14934 37.44 MB/sec warmup 26 sec latency 87.799 ms 2024-09-15T22:26:26.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:25 smithi077 ceph-mon[30980]: pgmap v439: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:26.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3891778567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:26.121 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:26:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:25 smithi196 ceph-mon[35109]: pgmap v439: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3891778567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:26.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:25 smithi117 ceph-mon[35021]: pgmap v439: 129 pgs: 129 active+clean; 805 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.2 MiB/s wr, 377 op/s 2024-09-15T22:26:26.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3891778567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 -- 172.21.15.77:0/612635654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c101df0 msgr2=0x7fca8c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 --2- 172.21.15.77:0/612635654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c101df0 0x7fca8c1021d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fca80009900 tx=0x7fca8002a410 comp rx=0 tx=0).stop 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 -- 172.21.15.77:0/612635654 shutdown_connections 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 --2- 172.21.15.77:0/612635654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c106f20 0x7fca8c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 --2- 172.21.15.77:0/612635654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c1027a0 0x7fca8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 --2- 172.21.15.77:0/612635654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c101df0 0x7fca8c1021d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 -- 172.21.15.77:0/612635654 >> 172.21.15.77:0/612635654 conn(0x7fca8c0fd8b0 msgr2=0x7fca8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 -- 172.21.15.77:0/612635654 shutdown_connections 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.372+0000 7fca93281640 1 -- 172.21.15.77:0/612635654 wait complete. 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.373+0000 7fca93281640 1 Processor -- start 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.373+0000 7fca93281640 1 -- start start 2024-09-15T22:26:26.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c101df0 0x7fca8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c106f20 0x7fca8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c111f40 con 0x7fca8c1027a0 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52336/0 (socket says 172.21.15.77:52336) 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c111dc0 con 0x7fca8c101df0 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 -- 172.21.15.77:0/3423314817 learned_addr learned my addr 172.21.15.77:0/3423314817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1120c0 con 0x7fca8c106f20 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9101d640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c101df0 0x7fca8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9181e640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c106f20 0x7fca8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 -- 172.21.15.77:0/3423314817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c106f20 msgr2=0x7fca8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c106f20 0x7fca8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 -- 172.21.15.77:0/3423314817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c101df0 msgr2=0x7fca8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c101df0 0x7fca8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 -- 172.21.15.77:0/3423314817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c1a68c0 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.374+0000 7fca9081c640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fca8800d950 tx=0x7fca8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.375+0000 7fca7e7fc640 1 -- 172.21.15.77:0/3423314817 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88017030 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.375+0000 7fca7e7fc640 1 -- 172.21.15.77:0/3423314817 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca88004020 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.375+0000 7fca7e7fc640 1 -- 172.21.15.77:0/3423314817 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca880047a0 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.375+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c1a6bb0 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.375+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1a7140 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.377+0000 7fca7e7fc640 1 -- 172.21.15.77:0/3423314817 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fca880041d0 con 0x7fca8c1027a0 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.378+0000 7fca7e7fc640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 0x7fca780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.378+0000 7fca9565a740 1 Processor -- start 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.378+0000 7fca9565a740 1 -- start start 2024-09-15T22:26:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.378+0000 7fca7e7fc640 1 -- 172.21.15.77:0/3423314817 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fca880cfa10 con 0x7fca8c1027a0 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.378+0000 7fca9101d640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 0x7fca780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b01400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d958a5c2a0 con 0x55d958af6760 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d958a03b10 con 0x55d958af7080 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9565a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d958a58260 con 0x55d958b01940 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9181e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9081c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b01400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9181e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36870/0 (socket says 172.21.15.77:36870) 2024-09-15T22:26:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52338/0 (socket says 172.21.15.77:52338) 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9181e640 1 -- 172.21.15.77:0/1482963925 learned_addr learned my addr 172.21.15.77:0/1482963925 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 0x7fca780a58f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca80005ab0 tx=0x7fca80005e70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 -- 172.21.15.77:0/1482963925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 msgr2=0x55d958b03d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 -- 172.21.15.77:0/1482963925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 msgr2=0x55d958b01400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b01400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 -- 172.21.15.77:0/1482963925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9587cdb90 con 0x55d958af6760 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9181e640 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.379+0000 7fca9101d640 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fca80009230 tx=0x7fca80044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f0a401657cd82d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.380+0000 7fca777fe640 1 -- 172.21.15.77:0/1482963925 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8004d070 con 0x55d958af6760 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.380+0000 7fca777fe640 1 -- 172.21.15.77:0/1482963925 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca80005520 con 0x55d958af6760 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.380+0000 7fca777fe640 1 -- 172.21.15.77:0/1482963925 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80035050 con 0x55d958af6760 2024-09-15T22:26:26.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 -- 172.21.15.77:0/1482963925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 msgr2=0x55d958af6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fca80009230 tx=0x7fca80044000 comp rx=0 tx=0).stop 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 -- 172.21.15.77:0/1482963925 shutdown_connections 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958b01940 0x55d958b03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b01400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 --2- 172.21.15.77:0/1482963925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af6760 0x55d958af6b40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 -- 172.21.15.77:0/1482963925 >> 172.21.15.77:0/1482963925 conn(0x55d958af2cf0 msgr2=0x55d958af30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 -- 172.21.15.77:0/1482963925 shutdown_connections 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.381+0000 7fca9565a740 1 -- 172.21.15.77:0/1482963925 wait complete. 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.382+0000 7fca9565a740 1 -- start start 2024-09-15T22:26:26.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.382+0000 7fca9565a740 1 Processor -- start 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af7080 0x55d958afecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958b01940 0x55d958aff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d958b045b0 con 0x55d958af7080 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9589e46e0 con 0x55d958b01940 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36886/0 (socket says 172.21.15.77:36886) 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 -- 172.21.15.77:0/3089419020 learned_addr learned my addr 172.21.15.77:0/3089419020 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9081c640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af7080 0x55d958afecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d958b04730 con 0x55d958af6760 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9565a740 10 client.0 _ll_register_callbacks cb 0x55d958a08a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 -- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958b01940 msgr2=0x55d958aff230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958b01940 0x55d958aff230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 -- 172.21.15.77:0/3089419020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af7080 msgr2=0x55d958afecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.383+0000 7fca9101d640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af7080 0x55d958afecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.384+0000 7fca9101d640 1 -- 172.21.15.77:0/3089419020 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d958b0da20 con 0x55d958af6760 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca9101d640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca80004620 tx=0x7fca800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8032acb87d0a643a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80057060 con 0x55d958af6760 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca80005520 con 0x55d958af6760 2024-09-15T22:26:26.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fca80003a00 con 0x55d958af6760 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fca8005e050 con 0x55d958af6760 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca9565a740 10 client.27778 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.385+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d958b045b0 con 0x55d958af6760 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8004e040 con 0x55d958af6760 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fca80048b20 con 0x55d958af6760 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 10 client.27778 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 10 client.27778 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 20 client.27778 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b13900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 4 client.27778 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.386+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca80005520 con 0x55d958b11510 2024-09-15T22:26:26.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.387+0000 7fca9081c640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b13900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.388+0000 7fca9081c640 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b13900 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9893decdc7a27c server_cookie=80ee10b3175ebc89 in_seq=0 out_seq=0 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.388+0000 7fca74ff9640 10 client.27778 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca74ff9640 1 -- 172.21.15.77:0/3089419020 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fca80005520 con 0x55d958b11510 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca74ff9640 10 client.27778 handle_command_reply: tid=0 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 client.27778 shutdown 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 msgr2=0x55d958afe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca80004620 tx=0x7fca800046c0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 shutdown_connections 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b13900 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.393+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958b01940 0x55d958aff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958af7080 0x55d958afecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca74ff9640 0 client.27778 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:26.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 1 --2- 172.21.15.77:0/3089419020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958afe7b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 >> 172.21.15.77:0/3089419020 conn(0x55d958af2cf0 msgr2=0x55d958af8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:26.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 shutdown_connections 2024-09-15T22:26:26.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 1 -- 172.21.15.77:0/3089419020 wait complete. 2024-09-15T22:26:26.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.394+0000 7fca9565a740 20 client.27778 trim_cache size 0 max 16384 2024-09-15T22:26:26.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 Processor -- start 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 -- start start 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 0x55d958af91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b1180 con 0x55d958b01940 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b2d80 con 0x55d958af6760 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.417+0000 7fca9565a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b1850 con 0x55d958af7080 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9181e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9081c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9081c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36902/0 (socket says 172.21.15.77:36902) 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9081c640 1 -- 172.21.15.77:0/3789536648 learned_addr learned my addr 172.21.15.77:0/3789536648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 0x55d958af91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 -- 172.21.15.77:0/3789536648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 msgr2=0x55d958b00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 -- 172.21.15.77:0/3789536648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 msgr2=0x55d958b190e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b190e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 -- 172.21.15.77:0/3789536648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d958af2b20 con 0x55d958af6760 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9081c640 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca9101d640 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 0x55d958af91c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca80048ef0 tx=0x7fca8002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521aea4001a7d151 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.418+0000 7fca76ffd640 1 -- 172.21.15.77:0/3789536648 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80057040 con 0x55d958af6760 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca76ffd640 1 -- 172.21.15.77:0/3789536648 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca80003ce0 con 0x55d958af6760 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca76ffd640 1 -- 172.21.15.77:0/3789536648 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8004d070 con 0x55d958af6760 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 -- 172.21.15.77:0/3789536648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 msgr2=0x55d958af91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 0x55d958af91c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca80048ef0 tx=0x7fca8002a410 comp rx=0 tx=0).stop 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 -- 172.21.15.77:0/3789536648 shutdown_connections 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b190e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af7080 0x55d958b00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 --2- 172.21.15.77:0/3789536648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af6760 0x55d958af91c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.419+0000 7fca9565a740 1 -- 172.21.15.77:0/3789536648 >> 172.21.15.77:0/3789536648 conn(0x55d958af2cf0 msgr2=0x55d958af1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.420+0000 7fca9565a740 1 -- 172.21.15.77:0/3789536648 shutdown_connections 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.420+0000 7fca9565a740 1 -- 172.21.15.77:0/3789536648 wait complete. 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.420+0000 7fca9565a740 1 -- start start 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.420+0000 7fca9565a740 1 Processor -- start 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b06390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b7b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b25f0 con 0x55d958b01940 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1b1f20 con 0x55d958af7080 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c1af180 con 0x55d958af6760 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9565a740 10 client.0 _ll_register_callbacks cb 0x7fca840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9181e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b7b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36918/0 (socket says 172.21.15.77:36918) 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 -- 172.21.15.77:0/76500631 learned_addr learned my addr 172.21.15.77:0/76500631 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9081c640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b06390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 -- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 msgr2=0x55d958b06390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b06390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 -- 172.21.15.77:0/76500631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 msgr2=0x55d958b7b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b7b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 -- 172.21.15.77:0/76500631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d958aee260 con 0x55d958af6760 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9081c640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b06390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9181e640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b7b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca9101d640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca80009900 tx=0x7fca80000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b7bf6df614026c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.421+0000 7fca74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80057070 con 0x55d958af6760 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fca80048480 con 0x55d958af6760 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca80003ce0 con 0x55d958af6760 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fca80071050 con 0x55d958af6760 2024-09-15T22:26:26.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 10 client.27784 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d958af6590 con 0x55d958af6760 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca80003ce0 con 0x55d958af6760 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fca80052440 con 0x55d958af6760 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 10 client.27784 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 10 client.27784 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 20 client.27784 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b7ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 4 client.27784 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.422+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x55d958a699d0 con 0x55d958b11510 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.423+0000 7fca9081c640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b7ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.423+0000 7fca9081c640 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b7ec80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9016ef276304b743 server_cookie=e7d7b26664b208 in_seq=0 out_seq=0 2024-09-15T22:26:26.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:26.423+0000 7fca74ff9640 10 client.27784 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:26.518 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:26.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15663 37.71 MB/sec warmup 27 sec latency 118.301 ms 2024-09-15T22:26:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca74ff9640 1 -- 172.21.15.77:0/76500631 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d958a699d0 con 0x55d958b11510 2024-09-15T22:26:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca74ff9640 10 client.27784 handle_command_reply: tid=0 2024-09-15T22:26:27.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca9565a740 1 client.27784 shutdown 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 msgr2=0x55d958b05e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca80009900 tx=0x7fca80000bc0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 shutdown_connections 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.423+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d958b11510 0x55d958b7ec80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d958b01940 0x55d958b7b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca74ff9640 0 client.27784 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d958af7080 0x55d958b06390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 --2- 172.21.15.77:0/76500631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d958af6760 0x55d958b05e50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 >> 172.21.15.77:0/76500631 conn(0x55d958af2cf0 msgr2=0x55d958b0c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 shutdown_connections 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 1 -- 172.21.15.77:0/76500631 wait complete. 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.424+0000 7fca9565a740 20 client.27784 trim_cache size 0 max 16384 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 36, 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 50, 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "1fc55bd9": { 2024-09-15T22:26:27.427 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 50, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 146.5, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:26:27.428 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.427+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 msgr2=0x7fca780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:27.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.427+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 0x7fca780a58f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fca80005ab0 tx=0x7fca80005e70 comp rx=0 tx=0).stop 2024-09-15T22:26:27.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.427+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 msgr2=0x7fca8c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:27.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.427+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fca8800d950 tx=0x7fca8800de10 comp rx=0 tx=0).stop 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 shutdown_connections 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fca780a3430 0x7fca780a58f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca8c106f20 0x7fca8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca8c1027a0 0x7fca8c074a30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 --2- 172.21.15.77:0/3423314817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca8c101df0 0x7fca8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 >> 172.21.15.77:0/3423314817 conn(0x7fca8c0fd8b0 msgr2=0x7fca8c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 shutdown_connections 2024-09-15T22:26:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:27.430+0000 7fca93281640 1 -- 172.21.15.77:0/3423314817 wait complete. 2024-09-15T22:26:27.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 36.38 MB/sec warmup 28 sec latency 1118.335 ms 2024-09-15T22:26:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:27 smithi077 ceph-mon[30980]: pgmap v440: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-09-15T22:26:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:27 smithi196 ceph-mon[35109]: pgmap v440: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-09-15T22:26:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:27 smithi117 ceph-mon[35021]: pgmap v440: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-09-15T22:26:28.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 35.12 MB/sec warmup 29 sec latency 2118.403 ms 2024-09-15T22:26:28.708 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1fc55bd9', quiesce took 7.2 seconds. Will release after: 19.3 2024-09-15T22:26:28.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.978+0000 7fa3e1454640 1 -- 172.21.15.77:0/596634712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc100430 msgr2=0x7fa3dc100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.978+0000 7fa3e1454640 1 --2- 172.21.15.77:0/596634712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc100430 0x7fa3dc100810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009900 tx=0x7fa3cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 -- 172.21.15.77:0/596634712 shutdown_connections 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 --2- 172.21.15.77:0/596634712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc105560 0x7fa3dc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 --2- 172.21.15.77:0/596634712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 --2- 172.21.15.77:0/596634712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc100430 0x7fa3dc100810 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009900 tx=0x7fa3cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 -- 172.21.15.77:0/596634712 >> 172.21.15.77:0/596634712 conn(0x7fa3dc077230 msgr2=0x7fa3dc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 -- 172.21.15.77:0/596634712 shutdown_connections 2024-09-15T22:26:28.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 -- 172.21.15.77:0/596634712 wait complete. 2024-09-15T22:26:28.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.979+0000 7fa3e1454640 1 Processor -- start 2024-09-15T22:26:28.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 -- start start 2024-09-15T22:26:28.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc198590 0x7fa3dc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10fd40 con 0x7fa3dc100de0 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10fbc0 con 0x7fa3dc105560 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.980+0000 7fa3e1454640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10fec0 con 0x7fa3dc198590 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52398/0 (socket says 172.21.15.77:52398) 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3daffd640 1 -- 172.21.15.77:0/3279367631 learned_addr learned my addr 172.21.15.77:0/3279367631 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:28.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43226/0 (socket says 172.21.15.77:43226) 2024-09-15T22:26:28.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.981+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc198590 0x7fa3dc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3279367631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc198590 msgr2=0x7fa3dc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc198590 0x7fa3dc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 msgr2=0x7fa3dc19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3279367631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc1990b0 con 0x7fa3dc105560 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400e9f0 tx=0x7fa3c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d8ff9640 1 -- 172.21.15.77:0/3279367631 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c400ccc0 con 0x7fa3dc105560 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc1993a0 con 0x7fa3dc105560 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d8ff9640 1 -- 172.21.15.77:0/3279367631 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c4010910 con 0x7fa3dc105560 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc1a5630 con 0x7fa3dc105560 2024-09-15T22:26:28.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.982+0000 7fa3d8ff9640 1 -- 172.21.15.77:0/3279367631 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40188e0 con 0x7fa3dc105560 2024-09-15T22:26:28.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.985+0000 7fa3d8ff9640 1 -- 172.21.15.77:0/3279367631 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa3c4010ac0 con 0x7fa3dc105560 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.986+0000 7fa3e382d740 1 Processor -- start 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.986+0000 7fa3e382d740 1 -- start start 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.986+0000 7fa3d8ff9640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 0x7fa3c80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.986+0000 7fa3d8ff9640 1 -- 172.21.15.77:0/3279367631 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa3c4018680 con 0x7fa3dc105560 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb8ff9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb900df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561bb8f5e2f0 con 0x561bb8ff8120 2024-09-15T22:26:28.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561bb8f05cf0 con 0x561bb8ff9840 2024-09-15T22:26:28.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.987+0000 7fa3e382d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bb8f5a2b0 con 0x561bb8ff8b80 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.989+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.989+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55786/0 (socket says 172.21.15.77:55786) 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.989+0000 7fa3d3fff640 1 -- 172.21.15.77:0/174517286 learned_addr learned my addr 172.21.15.77:0/174517286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 0x7fa3c80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 -- 172.21.15.77:0/174517286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 msgr2=0x561bb900df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb900df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 -- 172.21.15.77:0/174517286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 msgr2=0x561bb8ff9c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb8ff9c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 -- 172.21.15.77:0/174517286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb8eea3a0 con 0x561bb8ff8120 2024-09-15T22:26:28.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.990+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40a4970 tx=0x7fa3c409bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1f402b54e17b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:28.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.991+0000 7fa3d17fa640 1 -- 172.21.15.77:0/174517286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40a05e0 con 0x561bb8ff8120 2024-09-15T22:26:28.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.991+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 0x7fa3c80a5aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400a380 tx=0x7fa3d4010040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:28.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.991+0000 7fa3d17fa640 1 -- 172.21.15.77:0/174517286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c40a4ea0 con 0x561bb8ff8120 2024-09-15T22:26:28.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.991+0000 7fa3d17fa640 1 -- 172.21.15.77:0/174517286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40bc0b0 con 0x561bb8ff8120 2024-09-15T22:26:28.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.992+0000 7fa3e382d740 1 -- 172.21.15.77:0/174517286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 msgr2=0x561bb8ff8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:28.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.992+0000 7fa3e382d740 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40a4970 tx=0x7fa3c409bb90 comp rx=0 tx=0).stop 2024-09-15T22:26:29.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 -- 172.21.15.77:0/174517286 shutdown_connections 2024-09-15T22:26:29.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb900df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8120 0x561bb8ff8560 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 --2- 172.21.15.77:0/174517286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb8ff9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 -- 172.21.15.77:0/174517286 >> 172.21.15.77:0/174517286 conn(0x561bb8fc1310 msgr2=0x561bb8ff4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 -- 172.21.15.77:0/174517286 shutdown_connections 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.994+0000 7fa3e382d740 1 -- 172.21.15.77:0/174517286 wait complete. 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.995+0000 7fa3e382d740 1 -- start start 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.995+0000 7fa3e382d740 1 Processor -- start 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.995+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 0x561bb900df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.995+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 0x561bb900df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb900b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55790/0 (socket says 172.21.15.77:55790) 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 0x561bb900df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47818/0 (socket says 172.21.15.77:47818) 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3303169692 learned_addr learned my addr 172.21.15.77:0/3303169692 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561bb900e8d0 con 0x561bb8ff8b80 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561bb8f19060 con 0x561bb8ff9840 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bb900ea50 con 0x561bb8ff8120 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb900b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3303169692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 msgr2=0x561bb900df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 0x561bb900df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3303169692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 msgr2=0x561bb900b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb900b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb8ccf3c0 con 0x561bb8ff8b80 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 10 client.0 _ll_register_callbacks cb 0x561bb8c6c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c40a4ea0 con 0x561bb8ff8b80 2024-09-15T22:26:29.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.996+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40d0400 tx=0x7fa3c409bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a226bc78b1c9767 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3c27fc640 10 client.18915 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3303169692 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40b5220 con 0x561bb8ff8b80 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3303169692 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa3c40bc0e0 con 0x561bb8ff8b80 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3303169692 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa3c40bc320 con 0x561bb8ff8b80 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3e382d740 10 client.18915 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.997+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=36}) -- 0x561bb9007980 con 0x561bb8ff8b80 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3303169692 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa3c40a0120 con 0x561bb8ff8b80 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 10 client.18915 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 10 client.18915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 20 client.18915 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb9024090 0x561bb9026480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 4 client.18915 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bb8f6b860 con 0x561bb9024090 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.998+0000 7fa3daffd640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb9024090 0x561bb9026480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.999+0000 7fa3daffd640 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb9024090 0x561bb9026480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=673f2bc12ba81af0 server_cookie=93880e7d44c0cba7 in_seq=0 out_seq=0 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:28.999+0000 7fa3c27fc640 10 client.18915 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3303169692 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bb8f6b860 con 0x561bb9024090 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3c27fc640 10 client.18915 handle_command_reply: tid=0 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 client.18915 shutdown 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 msgr2=0x561bb900b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:29.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40d0400 tx=0x7fa3c409bcc0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 shutdown_connections 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb9024090 0x561bb9026480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff9840 0x561bb900b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900b1d0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3303169692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8120 0x561bb900df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.000+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 >> 172.21.15.77:0/3303169692 conn(0x561bb8fc1310 msgr2=0x561bb8ffa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.001+0000 7fa3c27fc640 0 client.18915 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.002+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 shutdown_connections 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.002+0000 7fa3e382d740 1 -- 172.21.15.77:0/3303169692 wait complete. 2024-09-15T22:26:29.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.002+0000 7fa3e382d740 20 client.18915 trim_cache size 0 max 16384 2024-09-15T22:26:29.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 Processor -- start 2024-09-15T22:26:29.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 -- start start 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb8ff4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.023+0000 7fa3e382d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc0739b0 con 0x561bb8ff8b80 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3e382d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a6ae0 con 0x561bb8ff8120 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3e382d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc074080 con 0x561bb8ff9840 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3db7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47834/0 (socket says 172.21.15.77:47834) 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3db7fe640 1 -- 172.21.15.77:0/4031509761 learned_addr learned my addr 172.21.15.77:0/4031509761 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3daffd640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb8ff4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.024+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 -- 172.21.15.77:0/4031509761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 msgr2=0x561bb8ff4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb8ff4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 -- 172.21.15.77:0/4031509761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 msgr2=0x561bb900a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 -- 172.21.15.77:0/4031509761 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb900e290 con 0x561bb8ff9840 2024-09-15T22:26:29.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3d401b820 tx=0x7fa3d4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5110b3c6fad42646 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3d0ff9640 1 -- 172.21.15.77:0/4031509761 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d402b070 con 0x561bb8ff9840 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3d0ff9640 1 -- 172.21.15.77:0/4031509761 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d4003e50 con 0x561bb8ff9840 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.025+0000 7fa3d0ff9640 1 -- 172.21.15.77:0/4031509761 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4004050 con 0x561bb8ff9840 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- 172.21.15.77:0/4031509761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 msgr2=0x561bb902bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3d401b820 tx=0x7fa3d4017000 comp rx=0 tx=0).stop 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- 172.21.15.77:0/4031509761 shutdown_connections 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff9840 0x561bb902bbf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff8b80 0x561bb900a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 --2- 172.21.15.77:0/4031509761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb8ff4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- 172.21.15.77:0/4031509761 >> 172.21.15.77:0/4031509761 conn(0x561bb8fc1310 msgr2=0x561bb9009ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- 172.21.15.77:0/4031509761 shutdown_connections 2024-09-15T22:26:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- 172.21.15.77:0/4031509761 wait complete. 2024-09-15T22:26:29.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 -- start start 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.027+0000 7fa3e382d740 1 Processor -- start 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb90030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff9840 0x561bb908cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb90030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3daffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb90030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43792/0 (socket says 172.21.15.77:43792) 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3d3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47842/0 (socket says 172.21.15.77:47842) 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3daffd640 1 -- 172.21.15.77:0/3491830822 learned_addr learned my addr 172.21.15.77:0/3491830822 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a6350 con 0x561bb8ff9840 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1a5c80 con 0x561bb8ff8120 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc0719b0 con 0x561bb8ff8b80 2024-09-15T22:26:29.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.028+0000 7fa3e382d740 10 client.0 _ll_register_callbacks cb 0x7fa3c40a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:29.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3db7fe640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff9840 0x561bb908cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3491830822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 msgr2=0x561bb90030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb90030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff9840 msgr2=0x561bb908cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff9840 0x561bb908cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.029+0000 7fa3d3fff640 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb9003ee0 con 0x561bb8ff8b80 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.030+0000 7fa3d3fff640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c6a0 tx=0x7fa3c409bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fef2b4e1e4c129a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:29.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.030+0000 7fa3c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.030+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40bc2f0 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.030+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa3c4010430 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.031+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d4003e50 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.031+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa3c4010630 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.031+0000 7fa3e382d740 10 client.27799 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.031+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561bb908e200 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c407e070 con 0x561bb8ff8b80 2024-09-15T22:26:29.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa3c40a1770 con 0x561bb8ff8b80 2024-09-15T22:26:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 10 client.27799 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 10 client.27799 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:26:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 20 client.27799 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb908e6d0 0x561bb9090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:29.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 4 client.27799 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bb8f65d70 con 0x561bb908e6d0 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.032+0000 7fa3daffd640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb908e6d0 0x561bb9090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.033+0000 7fa3c27fc640 10 client.27799 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.033+0000 7fa3daffd640 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb908e6d0 0x561bb9090ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c46721db8c2b554 server_cookie=46c029ccc8f109c8 in_seq=0 out_seq=0 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3c27fc640 1 -- 172.21.15.77:0/3491830822 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bb8f65d70 con 0x561bb908e6d0 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3c27fc640 10 client.27799 handle_command_reply: tid=0 2024-09-15T22:26:29.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 client.27799 shutdown 2024-09-15T22:26:29.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 msgr2=0x561bb9003620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:29.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c6a0 tx=0x7fa3c409bb90 comp rx=0 tx=0).stop 2024-09-15T22:26:29.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 shutdown_connections 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x561bb908e6d0 0x561bb9090ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561bb8ff9840 0x561bb908cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bb8ff8b80 0x561bb9003620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 --2- 172.21.15.77:0/3491830822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bb8ff8120 0x561bb90030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 >> 172.21.15.77:0/3491830822 conn(0x561bb8fc1310 msgr2=0x561bb900a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3c27fc640 0 client.27799 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 shutdown_connections 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.034+0000 7fa3e382d740 1 -- 172.21.15.77:0/3491830822 wait complete. 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.035+0000 7fa3e382d740 20 client.27799 trim_cache size 0 max 16384 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 msgr2=0x7fa3c80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 0x7fa3c80a5aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400a380 tx=0x7fa3d4010040 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 msgr2=0x7fa3dc19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400e9f0 tx=0x7fa3c400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 shutdown_connections 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3c80a35e0 0x7fa3c80a5aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3dc198590 0x7fa3dc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3dc105560 0x7fa3dc19e910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 --2- 172.21.15.77:0/3279367631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3dc100de0 0x7fa3dc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.036+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 >> 172.21.15.77:0/3279367631 conn(0x7fa3dc077230 msgr2=0x7fa3dc10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.037+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 shutdown_connections 2024-09-15T22:26:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:29.037+0000 7fa3e1454640 1 -- 172.21.15.77:0/3279367631 wait complete. 2024-09-15T22:26:29.564 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 33.95 MB/sec warmup 30 sec latency 3118.508 ms 2024-09-15T22:26:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:29 smithi077 ceph-mon[30980]: pgmap v441: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 516 op/s 2024-09-15T22:26:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:29 smithi196 ceph-mon[35109]: pgmap v441: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 516 op/s 2024-09-15T22:26:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:29 smithi117 ceph-mon[35021]: pgmap v441: 129 pgs: 129 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 12 MiB/s wr, 516 op/s 2024-09-15T22:26:30.464 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:26:30.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 32.86 MB/sec warmup 31 sec latency 4118.577 ms 2024-09-15T22:26:30.715 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 -- 172.21.15.77:0/2060736030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8102790 msgr2=0x7f41a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 --2- 172.21.15.77:0/2060736030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8102790 0x7f41a81067e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f419c009900 tx=0x7f419c02a410 comp rx=0 tx=0).stop 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 -- 172.21.15.77:0/2060736030 shutdown_connections 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 --2- 172.21.15.77:0/2060736030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8106f10 0x7f41a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 --2- 172.21.15.77:0/2060736030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8102790 0x7f41a81067e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 --2- 172.21.15.77:0/2060736030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8101de0 0x7f41a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 -- 172.21.15.77:0/2060736030 >> 172.21.15.77:0/2060736030 conn(0x7f41a80fd8c0 msgr2=0x7f41a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 -- 172.21.15.77:0/2060736030 shutdown_connections 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.194+0000 7f41afaa3640 1 -- 172.21.15.77:0/2060736030 wait complete. 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 Processor -- start 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 -- start start 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8102790 0x7f41a81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f10 0x7f41a819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111f90 con 0x7f41a8101de0 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111e10 con 0x7f41a8102790 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41afaa3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8112110 con 0x7f41a8106f10 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41ad83f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41ad83f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55834/0 (socket says 172.21.15.77:55834) 2024-09-15T22:26:31.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41ad83f640 1 -- 172.21.15.77:0/801938525 learned_addr learned my addr 172.21.15.77:0/801938525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.195+0000 7f41ad03e640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8102790 0x7f41a81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 -- 172.21.15.77:0/801938525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f10 msgr2=0x7f41a819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f10 0x7f41a819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 -- 172.21.15.77:0/801938525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8102790 msgr2=0x7f41a81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8102790 0x7f41a81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 -- 172.21.15.77:0/801938525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a819d440 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41ad83f640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f41a000b4a0 tx=0x7f41a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41a6ffd640 1 -- 172.21.15.77:0/801938525 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0014070 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41a6ffd640 1 -- 172.21.15.77:0/801938525 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0010af0 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a819d730 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41a6ffd640 1 -- 172.21.15.77:0/801938525 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017d10 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.196+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a81a5010 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.198+0000 7f41a6ffd640 1 -- 172.21.15.77:0/801938525 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41a8101de0 2024-09-15T22:26:31.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.198+0000 7f41b1e7c740 1 Processor -- start 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 -- start start 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b980675d0 0x564b98067a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b98074760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b97fcd2f0 con 0x564b98068b40 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b97f74cf0 con 0x564b980675d0 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b97fc92b0 con 0x564b98072370 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55850/0 (socket says 172.21.15.77:55850) 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 -- 172.21.15.77:0/4077688594 learned_addr learned my addr 172.21.15.77:0/4077688594 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ad83f640 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b980675d0 0x564b98067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41a6ffd640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 0x7f41900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ad03e640 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b98074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 -- 172.21.15.77:0/4077688594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 msgr2=0x564b98074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b98074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 -- 172.21.15.77:0/4077688594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b980675d0 msgr2=0x564b98067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b980675d0 0x564b98067a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41ae040640 1 -- 172.21.15.77:0/4077688594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b97f593a0 con 0x564b98068b40 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.199+0000 7f41a6ffd640 1 -- 172.21.15.77:0/801938525 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f41a00cf360 con 0x7f41a8101de0 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f41ae040640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 0x7f41900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f41ae040640 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4198009f00 tx=0x7f419800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752655c87afc25de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f418ffff640 1 -- 172.21.15.77:0/4077688594 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419801b550 con 0x564b98068b40 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f418ffff640 1 -- 172.21.15.77:0/4077688594 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419801bbd0 con 0x564b98068b40 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f418ffff640 1 -- 172.21.15.77:0/4077688594 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198011ad0 con 0x564b98068b40 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.200+0000 7f41ae040640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 0x7f41900a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f419801f000 tx=0x7f4198020000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- 172.21.15.77:0/4077688594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 msgr2=0x564b98067090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4198009f00 tx=0x7f419800e6c0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- 172.21.15.77:0/4077688594 shutdown_connections 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b98074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b980675d0 0x564b98067a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/4077688594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98067090 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- 172.21.15.77:0/4077688594 >> 172.21.15.77:0/4077688594 conn(0x564b98030310 msgr2=0x564b98063c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- 172.21.15.77:0/4077688594 shutdown_connections 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- 172.21.15.77:0/4077688594 wait complete. 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 -- start start 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.201+0000 7f41b1e7c740 1 Processor -- start 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b9806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b98075070 con 0x564b980675d0 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b97f88060 con 0x564b98068b40 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b980751f0 con 0x564b98072370 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 10 client.0 _ll_register_callbacks cb 0x564b97cdb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55862/0 (socket says 172.21.15.77:55862) 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 -- 172.21.15.77:0/841927188 learned_addr learned my addr 172.21.15.77:0/841927188 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad03e640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ae040640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b9806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 -- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 msgr2=0x564b9806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b9806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 -- 172.21.15.77:0/841927188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 msgr2=0x564b9806eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 -- 172.21.15.77:0/841927188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b97d3e3c0 con 0x564b980675d0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41ad83f640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f41a0097820 tx=0x7f41a009af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fedc29085859d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f418d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a001e040 con 0x564b980675d0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f41a00cc9a0 con 0x564b980675d0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.202+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419801bbd0 con 0x564b980675d0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41a009b3c0 con 0x564b980675d0 2024-09-15T22:26:31.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 10 client.18933 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b98080b40 con 0x564b980675d0 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419801bbd0 con 0x564b980675d0 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f41a0097a30 con 0x564b980675d0 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 10 client.18933 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 10 client.18933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 20 client.18933 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b98082000 0x564b980843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 4 client.18933 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b97fda860 con 0x564b98082000 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.203+0000 7f41ad03e640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b98082000 0x564b980843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f418d7fa640 10 client.18933 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:26:31.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f41ad03e640 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b98082000 0x564b980843f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c4a64385eca9290 server_cookie=e826fcd40ea9518c in_seq=0 out_seq=0 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f418d7fa640 1 -- 172.21.15.77:0/841927188 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b97fda860 con 0x564b98082000 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f418d7fa640 10 client.18933 handle_command_reply: tid=0 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f41b1e7c740 1 client.18933 shutdown 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 msgr2=0x564b98071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.204+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f41a0097820 tx=0x7f41a009af00 comp rx=0 tx=0).stop 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 shutdown_connections 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b98082000 0x564b980843f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b98072370 0x564b9806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/841927188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b980675d0 0x564b98071d40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f418d7fa640 0 client.18933 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:26:31.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 >> 172.21.15.77:0/841927188 conn(0x564b98030310 msgr2=0x564b98069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 shutdown_connections 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 1 -- 172.21.15.77:0/841927188 wait complete. 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.205+0000 7f41b1e7c740 20 client.18933 trim_cache size 0 max 16384 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.217+0000 7f41b1e7c740 1 Processor -- start 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.217+0000 7f41b1e7c740 1 -- start start 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b9805ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81aefb0 con 0x564b98072370 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81b0bb0 con 0x564b98068b40 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81af680 con 0x564b980675d0 2024-09-15T22:26:31.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55876/0 (socket says 172.21.15.77:55876) 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 -- 172.21.15.77:0/806319341 learned_addr learned my addr 172.21.15.77:0/806319341 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ad83f640 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b9805ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ad03e640 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 -- 172.21.15.77:0/806319341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 msgr2=0x564b9805ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b9805ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 -- 172.21.15.77:0/806319341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 msgr2=0x564b9806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 -- 172.21.15.77:0/806319341 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9806dfa0 con 0x564b98072370 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f41ae040640 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4198009550 tx=0x7f4198009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552ba3c6f36d8a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f418f7fe640 1 -- 172.21.15.77:0/806319341 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198034070 con 0x564b98072370 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f418f7fe640 1 -- 172.21.15.77:0/806319341 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198011e60 con 0x564b98072370 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.218+0000 7f418f7fe640 1 -- 172.21.15.77:0/806319341 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419801b920 con 0x564b98072370 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- 172.21.15.77:0/806319341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 msgr2=0x564b9806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4198009550 tx=0x7f4198009580 comp rx=0 tx=0).stop 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- 172.21.15.77:0/806319341 shutdown_connections 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98072370 0x564b9806a120 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98068b40 0x564b9806d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/806319341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b9805ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- 172.21.15.77:0/806319341 >> 172.21.15.77:0/806319341 conn(0x564b98030310 msgr2=0x564b98071070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- 172.21.15.77:0/806319341 shutdown_connections 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- 172.21.15.77:0/806319341 wait complete. 2024-09-15T22:26:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 -- start start 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.219+0000 7f41b1e7c740 1 Processor -- start 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b98076a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98072370 0x564b980e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81b0420 con 0x564b98068b40 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81afd50 con 0x564b98072370 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81acfb0 con 0x564b980675d0 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 10 client.0 _ll_register_callbacks cb 0x7f41a0017ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55882/0 (socket says 172.21.15.77:55882) 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 -- 172.21.15.77:0/2012581349 learned_addr learned my addr 172.21.15.77:0/2012581349 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad83f640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b98076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ae040640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98072370 0x564b980e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 -- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 msgr2=0x564b98076a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b98076a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 -- 172.21.15.77:0/2012581349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98072370 msgr2=0x564b980e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98072370 0x564b980e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 -- 172.21.15.77:0/2012581349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9806f950 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41ad03e640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f419c02a930 tx=0x7f419c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d632690bba0b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f418d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c04d030 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f419c004250 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.220+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4198011e60 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f419c057050 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 10 client.18939 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b980ed470 con 0x564b98068b40 2024-09-15T22:26:31.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198011e60 con 0x564b98068b40 2024-09-15T22:26:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f419c048650 con 0x564b98068b40 2024-09-15T22:26:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 10 client.18939 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 10 client.18939 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:26:31.240 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 20 client.18939 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b980edc70 0x564b980f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 4 client.18939 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b97fd4d70 con 0x564b980edc70 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.221+0000 7f41ad83f640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b980edc70 0x564b980f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f418d7fa640 10 client.18939 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41ad83f640 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b980edc70 0x564b980f0090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf668566e9e9aa01 server_cookie=a9dc4302cff861a5 in_seq=0 out_seq=0 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f418d7fa640 1 -- 172.21.15.77:0/2012581349 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b97fd4d70 con 0x564b980edc70 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f418d7fa640 10 client.18939 handle_command_reply: tid=0 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 client.18939 shutdown 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 msgr2=0x564b98076fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f419c02a930 tx=0x7f419c031a50 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 shutdown_connections 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x564b980edc70 0x564b980f0090 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b98072370 0x564b980e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b98068b40 0x564b98076fd0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f418d7fa640 0 client.18939 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 --2- 172.21.15.77:0/2012581349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b980675d0 0x564b98076a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 >> 172.21.15.77:0/2012581349 conn(0x564b98030310 msgr2=0x564b98070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 shutdown_connections 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 1 -- 172.21.15.77:0/2012581349 wait complete. 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.222+0000 7f41b1e7c740 20 client.18939 trim_cache size 0 max 16384 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 msgr2=0x7f41900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 0x7f41900a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f419801f000 tx=0x7f4198020000 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 msgr2=0x7f41a8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f41a000b4a0 tx=0x7f41a000b960 comp rx=0 tx=0).stop 2024-09-15T22:26:31.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 shutdown_connections 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41900a3430 0x7f41900a58f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f10 0x7f41a819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8102790 0x7f41a81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 --2- 172.21.15.77:0/801938525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8101de0 0x7f41a8198490 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 >> 172.21.15.77:0/801938525 conn(0x7f41a80fd8c0 msgr2=0x7f41a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 shutdown_connections 2024-09-15T22:26:31.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:31.223+0000 7f41afaa3640 1 -- 172.21.15.77:0/801938525 wait complete. 2024-09-15T22:26:31.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 31.83 MB/sec warmup 32 sec latency 5118.660 ms 2024-09-15T22:26:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:31 smithi077 ceph-mon[30980]: pgmap v442: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 749 op/s 2024-09-15T22:26:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:31 smithi196 ceph-mon[35109]: pgmap v442: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 749 op/s 2024-09-15T22:26:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:31 smithi117 ceph-mon[35021]: pgmap v442: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 749 op/s 2024-09-15T22:26:32.530 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:26:32.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 30.87 MB/sec warmup 33 sec latency 6118.959 ms 2024-09-15T22:26:32.783 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 -- 172.21.15.77:0/1924677060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4810e480 msgr2=0x7f6b48110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 --2- 172.21.15.77:0/1924677060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4810e480 0x7f6b48110870 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009ca0 tx=0x7f6b4002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 -- 172.21.15.77:0/1924677060 shutdown_connections 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 --2- 172.21.15.77:0/1924677060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4810e480 0x7f6b48110870 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 --2- 172.21.15.77:0/1924677060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b48101280 0x7f6b4810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 --2- 172.21.15.77:0/1924677060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b481008d0 0x7f6b48100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 -- 172.21.15.77:0/1924677060 >> 172.21.15.77:0/1924677060 conn(0x7f6b480fc7a0 msgr2=0x7f6b480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 -- 172.21.15.77:0/1924677060 shutdown_connections 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.255+0000 7f6b50353640 1 -- 172.21.15.77:0/1924677060 wait complete. 2024-09-15T22:26:33.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 Processor -- start 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 -- start start 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 0x7f6b4819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4810e480 0x7f6b4819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481141c0 con 0x7f6b48101280 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48114040 con 0x7f6b4810e480 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b50353640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48114340 con 0x7f6b481008d0 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b4d8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b4d8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55900/0 (socket says 172.21.15.77:55900) 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/1045043025 learned_addr learned my addr 172.21.15.77:0/1045043025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:33.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.256+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4810e480 0x7f6b4819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 0x7f6b4819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/1045043025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 msgr2=0x7f6b4819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 0x7f6b4819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/1045043025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4810e480 msgr2=0x7f6b4819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4810e480 0x7f6b4819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/1045043025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4819f670 con 0x7f6b48101280 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 0x7f6b4819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800b4a0 tx=0x7f6b3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b3f7fe640 1 -- 172.21.15.77:0/1045043025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38014070 con 0x7f6b48101280 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b3f7fe640 1 -- 172.21.15.77:0/1045043025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b38010af0 con 0x7f6b48101280 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4819f960 con 0x7f6b48101280 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b3f7fe640 1 -- 172.21.15.77:0/1045043025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38017d10 con 0x7f6b48101280 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.257+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b481a7240 con 0x7f6b48101280 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.259+0000 7f6b3f7fe640 1 -- 172.21.15.77:0/1045043025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6b38017070 con 0x7f6b48101280 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 Processor -- start 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 -- start start 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589900102f0 con 0x5589900abb40 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55898ffb7cf0 con 0x5589900aa5d0 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b5272c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55899000c2b0 con 0x5589900b5370 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b4e8f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b4e8f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55914/0 (socket says 172.21.15.77:55914) 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/3207776374 learned_addr learned my addr 172.21.15.77:0/3207776374 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b3f7fe640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 0x7f6b340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.260+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/3207776374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 msgr2=0x5589900b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/3207776374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 msgr2=0x5589900aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/3207776374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898ff9c3a0 con 0x5589900abb40 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b3f7fe640 1 -- 172.21.15.77:0/1045043025 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f6b380cf290 con 0x7f6b48101280 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 0x7f6b340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009ca0 tx=0x7f6b400099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d708602771c60d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b3cdea640 1 -- 172.21.15.77:0/3207776374 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40042040 con 0x5589900abb40 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b3cdea640 1 -- 172.21.15.77:0/3207776374 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b40036040 con 0x5589900abb40 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.261+0000 7f6b3cdea640 1 -- 172.21.15.77:0/3207776374 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4002fa30 con 0x5589900abb40 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 0x7f6b340a58f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b4004d000 tx=0x7f6b4004e000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- 172.21.15.77:0/3207776374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 msgr2=0x5589900aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009ca0 tx=0x7f6b400099b0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- 172.21.15.77:0/3207776374 shutdown_connections 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3207776374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900aa090 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- 172.21.15.77:0/3207776374 >> 172.21.15.77:0/3207776374 conn(0x558990073310 msgr2=0x5589900a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- 172.21.15.77:0/3207776374 shutdown_connections 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- 172.21.15.77:0/3207776374 wait complete. 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 -- start start 2024-09-15T22:26:33.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.262+0000 7f6b5272c740 1 Processor -- start 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900aa5d0 0x5589900b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900abb40 0x5589900b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589900b8070 con 0x5589900b5370 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55898ffcb060 con 0x5589900abb40 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5589900b81f0 con 0x5589900aa5d0 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b5272c740 10 client.0 _ll_register_callbacks cb 0x55898fd1e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55928/0 (socket says 172.21.15.77:55928) 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/2731181686 learned_addr learned my addr 172.21.15.77:0/2731181686 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900aa5d0 0x5589900b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900abb40 0x5589900b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900aa5d0 msgr2=0x5589900b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900aa5d0 0x5589900b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/2731181686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900abb40 msgr2=0x5589900b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900abb40 0x5589900b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 -- 172.21.15.77:0/2731181686 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898fd813c0 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6b4002f630 tx=0x7f6b40033030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2601049f83a310 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.263+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40004000 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f6b400041a0 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b40036040 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b40064050 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 10 client.18951 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589900b8070 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b40036040 con 0x5589900b5370 2024-09-15T22:26:33.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f6b400309e0 con 0x5589900b5370 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 10 client.18951 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 10 client.18951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 20 client.18951 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5589900c4bc0 0x5589900c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 4 client.18951 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.264+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899001d860 con 0x5589900c4bc0 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.265+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5589900c4bc0 0x5589900c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.265+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5589900c4bc0 0x5589900c6fb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=facdea495e6d2925 server_cookie=57da2abe88f13747 in_seq=0 out_seq=0 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.265+0000 7f6b31ffb640 10 client.18951 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b31ffb640 1 -- 172.21.15.77:0/2731181686 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55899001d860 con 0x5589900c4bc0 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b31ffb640 10 client.18951 handle_command_reply: tid=0 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 client.18951 shutdown 2024-09-15T22:26:33.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 msgr2=0x5589900b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6b4002f630 tx=0x7f6b40033030 comp rx=0 tx=0).stop 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 shutdown_connections 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5589900c4bc0 0x5589900c6fb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900b5370 0x5589900b25d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b31ffb640 0 client.18951 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900abb40 0x5589900b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 --2- 172.21.15.77:0/2731181686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900aa5d0 0x5589900b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 >> 172.21.15.77:0/2731181686 conn(0x558990073310 msgr2=0x5589900ac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 shutdown_connections 2024-09-15T22:26:33.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 1 -- 172.21.15.77:0/2731181686 wait complete. 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.266+0000 7f6b5272c740 20 client.18951 trim_cache size 0 max 16384 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 Processor -- start 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 -- start start 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 0x5589900ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900b5370 0x5589900ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481b1100 con 0x5589900aa5d0 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481b2d00 con 0x5589900b5370 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b5272c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481b17d0 con 0x5589900abb40 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55934/0 (socket says 172.21.15.77:55934) 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 -- 172.21.15.77:0/3197869955 learned_addr learned my addr 172.21.15.77:0/3197869955 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:33.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 0x5589900ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 -- 172.21.15.77:0/3197869955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 msgr2=0x5589900ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 0x5589900ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 -- 172.21.15.77:0/3197869955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900b5370 msgr2=0x5589900ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900b5370 0x5589900ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 -- 172.21.15.77:0/3197869955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589900a7960 con 0x5589900aa5d0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 0x5589900ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x5589900c76e0 tx=0x7f6b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32acd48116ee9f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.279+0000 7f6b33fff640 1 -- 172.21.15.77:0/3197869955 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b440130d0 con 0x5589900aa5d0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b33fff640 1 -- 172.21.15.77:0/3197869955 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44013270 con 0x5589900aa5d0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b33fff640 1 -- 172.21.15.77:0/3197869955 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44003830 con 0x5589900aa5d0 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 -- 172.21.15.77:0/3197869955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 msgr2=0x5589900c1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x5589900c76e0 tx=0x7f6b440046c0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 -- 172.21.15.77:0/3197869955 shutdown_connections 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900b5370 0x5589900ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900abb40 0x5589900ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 --2- 172.21.15.77:0/3197869955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900aa5d0 0x5589900c1310 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 -- 172.21.15.77:0/3197869955 >> 172.21.15.77:0/3197869955 conn(0x558990073310 msgr2=0x5589900b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 -- 172.21.15.77:0/3197869955 shutdown_connections 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.280+0000 7f6b5272c740 1 -- 172.21.15.77:0/3197869955 wait complete. 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 -- start start 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 Processor -- start 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481b2570 con 0x5589900abb40 2024-09-15T22:26:33.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481b1ea0 con 0x5589900aa5d0 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b481af100 con 0x5589900b5370 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b5272c740 10 client.0 _ll_register_callbacks cb 0x7f6b4400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55940/0 (socket says 172.21.15.77:55940) 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/851164070 learned_addr learned my addr 172.21.15.77:0/851164070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 msgr2=0x5589900b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/851164070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 msgr2=0x5589900bddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900bddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 -- 172.21.15.77:0/851164070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899012e740 con 0x5589900abb40 2024-09-15T22:26:33.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4e8f0640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b4d8ee640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6b380cc7c0 tx=0x7f6b3806af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2427f69ee6e849f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.281+0000 7f6b31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3801e040 con 0x5589900abb40 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f6b380cc8d0 con 0x5589900abb40 2024-09-15T22:26:33.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b44013270 con 0x5589900abb40 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b38095e60 con 0x5589900abb40 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 10 client.18957 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55899012f8f0 con 0x5589900abb40 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44013270 con 0x5589900abb40 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f6b38097960 con 0x5589900abb40 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 10 client.18957 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 10 client.18957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 20 client.18957 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558990130010 0x558990132400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 4 client.18957 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558990017d70 con 0x558990130010 2024-09-15T22:26:33.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.282+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558990130010 0x558990132400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b31ffb640 10 client.18957 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b4e0ef640 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558990130010 0x558990132400 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f44ac9c8e30593d server_cookie=cea4c3c20bd04088 in_seq=0 out_seq=0 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b31ffb640 1 -- 172.21.15.77:0/851164070 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558990017d70 con 0x558990130010 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b31ffb640 10 client.18957 handle_command_reply: tid=0 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 client.18957 shutdown 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 msgr2=0x5589900b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6b380cc7c0 tx=0x7f6b3806af10 comp rx=0 tx=0).stop 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 shutdown_connections 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x558990130010 0x558990132400 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5589900b5370 0x5589900b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b31ffb640 0 client.18957 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5589900abb40 0x5589900b90e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 --2- 172.21.15.77:0/851164070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5589900aa5d0 0x5589900bddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 >> 172.21.15.77:0/851164070 conn(0x558990073310 msgr2=0x5589900a5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 shutdown_connections 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 1 -- 172.21.15.77:0/851164070 wait complete. 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.283+0000 7f6b5272c740 20 client.18957 trim_cache size 0 max 16384 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 msgr2=0x7f6b340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 0x7f6b340a58f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b4004d000 tx=0x7f6b4004e000 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 msgr2=0x7f6b4819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800b4a0 tx=0x7f6b3800b960 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 shutdown_connections 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6b340a3430 0x7f6b340a58f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b4810e480 0x7f6b4819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b48101280 0x7f6b4819ac00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 --2- 172.21.15.77:0/1045043025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b481008d0 0x7f6b4819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 >> 172.21.15.77:0/1045043025 conn(0x7f6b480fc7a0 msgr2=0x7f6b480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:33.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 shutdown_connections 2024-09-15T22:26:33.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:33.284+0000 7f6b50353640 1 -- 172.21.15.77:0/1045043025 wait complete. 2024-09-15T22:26:33.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 29.96 MB/sec warmup 34 sec latency 7119.034 ms 2024-09-15T22:26:34.249 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:33 smithi077 ceph-mon[30980]: pgmap v443: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:34.281 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:26:34.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:33 smithi196 ceph-mon[35109]: pgmap v443: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:33 smithi117 ceph-mon[35021]: pgmap v443: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:34.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 29.10 MB/sec warmup 35 sec latency 8119.059 ms 2024-09-15T22:26:34.566 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:35.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 28.29 MB/sec warmup 36 sec latency 9119.142 ms 2024-09-15T22:26:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- 172.21.15.77:0/2865243273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8102790 msgr2=0x7fa1f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/2865243273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8102790 0x7fa1f81067e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009900 tx=0x7fa1e002a410 comp rx=0 tx=0).stop 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- 172.21.15.77:0/2865243273 shutdown_connections 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/2865243273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8106f10 0x7fa1f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/2865243273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8102790 0x7fa1f81067e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/2865243273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8101de0 0x7fa1f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- 172.21.15.77:0/2865243273 >> 172.21.15.77:0/2865243273 conn(0x7fa1f80fd8c0 msgr2=0x7fa1f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- 172.21.15.77:0/2865243273 shutdown_connections 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- 172.21.15.77:0/2865243273 wait complete. 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 Processor -- start 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.003+0000 7fa1fdc79640 1 -- start start 2024-09-15T22:26:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8101de0 0x7fa1f8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8102790 0x7fa1f81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8111f70 con 0x7fa1f8106f10 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8111df0 con 0x7fa1f8102790 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1fdc79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81120f0 con 0x7fa1f8101de0 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55964/0 (socket says 172.21.15.77:55964) 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 -- 172.21.15.77:0/4045519984 learned_addr learned my addr 172.21.15.77:0/4045519984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1effff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8102790 0x7fa1f81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8101de0 0x7fa1f8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 -- 172.21.15.77:0/4045519984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8101de0 msgr2=0x7fa1f8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8101de0 0x7fa1f8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 -- 172.21.15.77:0/4045519984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8102790 msgr2=0x7fa1f81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8102790 0x7fa1f81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 -- 172.21.15.77:0/4045519984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f819d440 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f7fff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000eaa0 tx=0x7fa1f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.004+0000 7fa1f57fa640 1 -- 172.21.15.77:0/4045519984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000cdb0 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.005+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f819d730 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.005+0000 7fa1f57fa640 1 -- 172.21.15.77:0/4045519984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0005020 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.005+0000 7fa1f57fa640 1 -- 172.21.15.77:0/4045519984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00116b0 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.005+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81a5010 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa1f57fa640 1 -- 172.21.15.77:0/4045519984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa1f0005560 con 0x7fa1f8106f10 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 Processor -- start 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 -- start start 2024-09-15T22:26:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548eeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558548e512f0 con 0x558548eeb5d0 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558548df8cf0 con 0x558548eecb40 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa200052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558548e4d2b0 con 0x558548ef6370 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa1f57fa640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 0x7fa1e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55976/0 (socket says 172.21.15.77:55976) 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.007+0000 7fa1f77fe640 1 -- 172.21.15.77:0/1743153625 learned_addr learned my addr 172.21.15.77:0/1743153625 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f7fff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 0x7fa1e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1effff640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548eeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1effff640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 -- 172.21.15.77:0/1743153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 msgr2=0x558548ef8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 -- 172.21.15.77:0/1743153625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 msgr2=0x558548eeb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548eeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 -- 172.21.15.77:0/1743153625 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558548ddd3a0 con 0x558548eeb5d0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f57fa640 1 -- 172.21.15.77:0/4045519984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa1f00d01e0 con 0x7fa1f8106f10 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1effff640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800bd10 tx=0x7fa1e80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcff678713d1b6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1edffb640 1 -- 172.21.15.77:0/1743153625 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004030 con 0x558548eeb5d0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1f7fff640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 0x7fa1e40a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00cdba0 tx=0x7fa1f00cb040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1edffb640 1 -- 172.21.15.77:0/1743153625 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8015d30 con 0x558548eeb5d0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.008+0000 7fa1edffb640 1 -- 172.21.15.77:0/1743153625 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003c20 con 0x558548eeb5d0 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- 172.21.15.77:0/1743153625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 msgr2=0x558548eeba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800bd10 tx=0x7fa1e80107a0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- 172.21.15.77:0/1743153625 shutdown_connections 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548eeba10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 --2- 172.21.15.77:0/1743153625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548eeb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- 172.21.15.77:0/1743153625 >> 172.21.15.77:0/1743153625 conn(0x558548eb4310 msgr2=0x558548ee7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- 172.21.15.77:0/1743153625 shutdown_connections 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- 172.21.15.77:0/1743153625 wait complete. 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.009+0000 7fa200052740 1 -- start start 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 Processor -- start 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558548ef9070 con 0x558548eeb5d0 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558548e0c060 con 0x558548eecb40 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558548ef91f0 con 0x558548ef6370 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa200052740 10 client.0 _ll_register_callbacks cb 0x558548b5f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55990/0 (socket says 172.21.15.77:55990) 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 -- 172.21.15.77:0/1540856182 learned_addr learned my addr 172.21.15.77:0/1540856182 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 -- 172.21.15.77:0/1540856182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 msgr2=0x558548ef35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 -- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 msgr2=0x558548ef3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 -- 172.21.15.77:0/1540856182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558548bc23c0 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1effff640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009900 tx=0x7fa1e00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14dc8b56e345c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.010+0000 7fa1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0035040 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa1e0004240 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e8015d30 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa1e0004440 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 10 client.18969 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558548f04a60 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8015d30 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa1e003b550 con 0x558548eeb5d0 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 10 client.18969 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 10 client.18969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:26:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 20 client.18969 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f05f20 0x558548f08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 4 client.18969 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.011+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558548e5e860 con 0x558548f05f20 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.012+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f05f20 0x558548f08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.012+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f05f20 0x558548f08350 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a7d5952acf362f2 server_cookie=d6c71fc9fad6fb30 in_seq=0 out_seq=0 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.012+0000 7fa1d6ffd640 10 client.18969 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1540856182 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558548e5e860 con 0x558548f05f20 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa1d6ffd640 10 client.18969 handle_command_reply: tid=0 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 client.18969 shutdown 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 msgr2=0x558548ef5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009900 tx=0x7fa1e00315b0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 shutdown_connections 2024-09-15T22:26:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f05f20 0x558548f08350 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548ef6370 0x558548ef35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa1d6ffd640 0 client.18969 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 --2- 172.21.15.77:0/1540856182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548eeb5d0 0x558548ef5d50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 >> 172.21.15.77:0/1540856182 conn(0x558548eb4310 msgr2=0x558548eed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 shutdown_connections 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 1 -- 172.21.15.77:0/1540856182 wait complete. 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.013+0000 7fa200052740 20 client.18969 trim_cache size 0 max 16384 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 Processor -- start 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 -- start start 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548f02ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ee6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81ace30 con 0x558548ef6370 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81aea30 con 0x558548eecb40 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa200052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81ad500 con 0x558548eeb5d0 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56004/0 (socket says 172.21.15.77:56004) 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 -- 172.21.15.77:0/1265947618 learned_addr learned my addr 172.21.15.77:0/1265947618 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ee6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 -- 172.21.15.77:0/1265947618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 msgr2=0x558548f02ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548f02ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 -- 172.21.15.77:0/1265947618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 msgr2=0x558548ee6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ee6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 -- 172.21.15.77:0/1265947618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558548ee6310 con 0x558548ef6370 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ee6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.026+0000 7fa1effff640 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa1e002a930 tx=0x7fa1e0031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67b75472ce9f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa1ed7fa640 1 -- 172.21.15.77:0/1265947618 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0035070 con 0x558548ef6370 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa1ed7fa640 1 -- 172.21.15.77:0/1265947618 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0005550 con 0x558548ef6370 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa1ed7fa640 1 -- 172.21.15.77:0/1265947618 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00046c0 con 0x558548ef6370 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 -- 172.21.15.77:0/1265947618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 msgr2=0x558548eee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa1e002a930 tx=0x7fa1e0031a70 comp rx=0 tx=0).stop 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 -- 172.21.15.77:0/1265947618 shutdown_connections 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548eee120 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ee6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 --2- 172.21.15.77:0/1265947618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548f02ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 -- 172.21.15.77:0/1265947618 >> 172.21.15.77:0/1265947618 conn(0x558548eb4310 msgr2=0x558548ee6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 -- 172.21.15.77:0/1265947618 shutdown_connections 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.027+0000 7fa200052740 1 -- 172.21.15.77:0/1265947618 wait complete. 2024-09-15T22:26:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 -- start start 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 Processor -- start 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548eeffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81ae2a0 con 0x558548ef6370 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81adbd0 con 0x558548eecb40 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81aae30 con 0x558548eeb5d0 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa200052740 10 client.0 _ll_register_callbacks cb 0x7fa1e0032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56016/0 (socket says 172.21.15.77:56016) 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 -- 172.21.15.77:0/1485274394 learned_addr learned my addr 172.21.15.77:0/1485274394 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548eeffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 -- 172.21.15.77:0/1485274394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 msgr2=0x558548eeffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548eeffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 -- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 msgr2=0x558548ef0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.028+0000 7fa1effff640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1effff640 1 -- 172.21.15.77:0/1485274394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558548f6fc30 con 0x558548ef6370 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1effff640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0031740 tx=0x7fa1e003bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dacb129f2a8568 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0035070 con 0x558548ef6370 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fa1e0054460 con 0x558548ef6370 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e0005550 con 0x558548ef6370 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa1e0054660 con 0x558548ef6370 2024-09-15T22:26:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa200052740 10 client.18975 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558548f70e00 con 0x558548ef6370 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0054ce0 con 0x558548ef6370 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.029+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fa1e00325b0 con 0x558548ef6370 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 10 client.18975 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 10 client.18975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 20 client.18975 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f71270 0x558548f73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 4 client.18975 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa1e0005550 con 0x558548f71270 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f71270 0x558548f73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1f77fe640 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f71270 0x558548f73660 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce57dd4a9ba67b2e server_cookie=d76b9503d25d8c9d in_seq=0 out_seq=0 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1d6ffd640 10 client.18975 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1d6ffd640 1 -- 172.21.15.77:0/1485274394 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa1e0005550 con 0x558548f71270 2024-09-15T22:26:36.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1d6ffd640 10 client.18975 handle_command_reply: tid=0 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 client.18975 shutdown 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 msgr2=0x558548ef0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0031740 tx=0x7fa1e003bea0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 shutdown_connections 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x558548f71270 0x558548f73660 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558548ef6370 0x558548ef0a50 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa1d6ffd640 0 client.18975 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.030+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558548eecb40 0x558548ef0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa200052740 1 --2- 172.21.15.77:0/1485274394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558548eeb5d0 0x558548eeffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 >> 172.21.15.77:0/1485274394 conn(0x558548eb4310 msgr2=0x558548ee6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 shutdown_connections 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa200052740 1 -- 172.21.15.77:0/1485274394 wait complete. 2024-09-15T22:26:36.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa200052740 20 client.18975 trim_cache size 0 max 16384 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 msgr2=0x7fa1e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 0x7fa1e40a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00cdba0 tx=0x7fa1f00cb040 comp rx=0 tx=0).stop 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 msgr2=0x7fa1f819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000eaa0 tx=0x7fa1f000ef60 comp rx=0 tx=0).stop 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 shutdown_connections 2024-09-15T22:26:36.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa1e40a33b0 0x7fa1e40a5870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f8106f10 0x7fa1f819cd60 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f8102790 0x7fa1f81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 --2- 172.21.15.77:0/4045519984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f8101de0 0x7fa1f8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 >> 172.21.15.77:0/4045519984 conn(0x7fa1f80fd8c0 msgr2=0x7fa1f80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 shutdown_connections 2024-09-15T22:26:36.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:36.031+0000 7fa1fdc79640 1 -- 172.21.15.77:0/4045519984 wait complete. 2024-09-15T22:26:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:35 smithi077 ceph-mon[30980]: pgmap v444: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:36.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:35 smithi196 ceph-mon[35109]: pgmap v444: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:35 smithi117 ceph-mon[35021]: pgmap v444: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 560 op/s 2024-09-15T22:26:36.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 27.53 MB/sec warmup 37 sec latency 10119.261 ms 2024-09-15T22:26:36.765 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:26:37.010 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 -- 172.21.15.77:0/555834846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58106f10 msgr2=0x7f9a5810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/555834846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58106f10 0x7f9a5810ea90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 -- 172.21.15.77:0/555834846 shutdown_connections 2024-09-15T22:26:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/555834846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58106f10 0x7f9a5810ea90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/555834846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58102790 0x7f9a581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/555834846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58101de0 0x7f9a581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 -- 172.21.15.77:0/555834846 >> 172.21.15.77:0/555834846 conn(0x7f9a580fd8c0 msgr2=0x7f9a580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 -- 172.21.15.77:0/555834846 shutdown_connections 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.468+0000 7f9a5f20e640 1 -- 172.21.15.77:0/555834846 wait complete. 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 Processor -- start 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 -- start start 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 0x7f9a5819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58106f10 0x7f9a5819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58111f70 con 0x7f9a58101de0 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58111df0 con 0x7f9a58102790 2024-09-15T22:26:37.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5f20e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581120f0 con 0x7f9a58106f10 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56046/0 (socket says 172.21.15.77:56046) 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2980376672 learned_addr learned my addr 172.21.15.77:0/2980376672 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58106f10 0x7f9a5819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 0x7f9a5819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2980376672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58106f10 msgr2=0x7f9a5819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58106f10 0x7f9a5819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2980376672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 msgr2=0x7f9a5819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 0x7f9a5819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2980376672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5819f5c0 con 0x7f9a58101de0 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 0x7f9a5819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.469+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9f0 tx=0x7f9a4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.470+0000 7f9a55ffb640 1 -- 172.21.15.77:0/2980376672 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00cd80 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.470+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5819f8b0 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.470+0000 7f9a55ffb640 1 -- 172.21.15.77:0/2980376672 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4c010910 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.470+0000 7f9a55ffb640 1 -- 172.21.15.77:0/2980376672 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0188f0 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.470+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581a7190 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.472+0000 7f9a55ffb640 1 -- 172.21.15.77:0/2980376672 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9a4c010ac0 con 0x7f9a58101de0 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.472+0000 7f9a615e7740 1 Processor -- start 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.472+0000 7f9a615e7740 1 -- start start 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a55ffb640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 0x7f9a480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a1522b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15238760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a151912f0 con 0x555a1522cb40 2024-09-15T22:26:37.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a15138cf0 con 0x555a15236370 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a615e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a1518d2b0 con 0x555a1522b5d0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a55ffb640 1 -- 172.21.15.77:0/2980376672 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f9a4c0d05a0 con 0x7f9a58101de0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 0x7f9a480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a5d7ab640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15238760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:48052/0 (socket says 172.21.15.77:48052) 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 -- 172.21.15.77:0/2871525447 learned_addr learned my addr 172.21.15.77:0/2871525447 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a1522b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 0x7f9a480a5870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fe0 tx=0x7f9a50031bb0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 -- 172.21.15.77:0/2871525447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 msgr2=0x555a15238760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15238760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 -- 172.21.15.77:0/2871525447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 msgr2=0x555a1522b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a1522b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 -- 172.21.15.77:0/2871525447 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1511d3a0 con 0x555a1522b5d0 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.473+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ba0 tx=0x7f9a4400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546d8cbd76b1047a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a42ffd640 1 -- 172.21.15.77:0/2871525447 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44016070 con 0x555a1522b5d0 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a42ffd640 1 -- 172.21.15.77:0/2871525447 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4400ee70 con 0x555a1522b5d0 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a42ffd640 1 -- 172.21.15.77:0/2871525447 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44010890 con 0x555a1522b5d0 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 -- 172.21.15.77:0/2871525447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 msgr2=0x555a1522ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ba0 tx=0x7f9a4400e6c0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 -- 172.21.15.77:0/2871525447 shutdown_connections 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15238760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522ba10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2871525447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a1522b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 -- 172.21.15.77:0/2871525447 >> 172.21.15.77:0/2871525447 conn(0x555a151f4310 msgr2=0x555a15227c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 -- 172.21.15.77:0/2871525447 shutdown_connections 2024-09-15T22:26:37.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.474+0000 7f9a615e7740 1 -- 172.21.15.77:0/2871525447 wait complete. 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 -- start start 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 Processor -- start 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522cb40 0x555a15232f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a152334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a15239070 con 0x555a1522b5d0 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a1514c060 con 0x555a15236370 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555a152391f0 con 0x555a1522cb40 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a615e7740 10 client.0 _ll_register_callbacks cb 0x555a14e9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a5cfaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a5cfaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56072/0 (socket says 172.21.15.77:56072) 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2754007239 learned_addr learned my addr 172.21.15.77:0/2754007239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:37.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.475+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522cb40 0x555a15232f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2754007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522cb40 msgr2=0x555a15232f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522cb40 0x555a15232f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 msgr2=0x555a152334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a152334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 -- 172.21.15.77:0/2754007239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a14f023c0 con 0x555a1522b5d0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c6a0 tx=0x7f9a4c09cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6da6f31a20d418f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0d0330 con 0x555a1522b5d0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f9a4c09b940 con 0x555a1522b5d0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4400ee70 con 0x555a1522b5d0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c0a5160 con 0x555a1522b5d0 2024-09-15T22:26:37.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a615e7740 10 client.18987 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555a15244a80 con 0x555a1522b5d0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4400ee70 con 0x555a1522b5d0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.476+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f9a4c098240 con 0x555a1522b5d0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 10 client.18987 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 20 client.18987 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a15248370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 4 client.18987 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1519e860 con 0x555a15245ee0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a15248370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a37fff640 10 client.18987 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.477+0000 7f9a57fff640 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a15248370 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14428ea66a3cc610 server_cookie=6aba72c0c29f62dd in_seq=0 out_seq=0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a37fff640 1 -- 172.21.15.77:0/2754007239 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a1519e860 con 0x555a15245ee0 2024-09-15T22:26:37.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a37fff640 10 client.18987 handle_command_reply: tid=0 2024-09-15T22:26:37.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 client.18987 shutdown 2024-09-15T22:26:37.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 msgr2=0x555a15235ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c6a0 tx=0x7f9a4c09cd20 comp rx=0 tx=0).stop 2024-09-15T22:26:37.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 shutdown_connections 2024-09-15T22:26:37.516 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a15248370 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a37fff640 0 client.18987 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a152334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522cb40 0x555a15232f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 --2- 172.21.15.77:0/2754007239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522b5d0 0x555a15235ce0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 >> 172.21.15.77:0/2754007239 conn(0x555a151f4310 msgr2=0x555a1522d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 shutdown_connections 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 1 -- 172.21.15.77:0/2754007239 wait complete. 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.478+0000 7f9a615e7740 20 client.18987 trim_cache size 0 max 16384 2024-09-15T22:26:37.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 Processor -- start 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 -- start start 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a15227e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15222ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b1210 con 0x555a1522cb40 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b2e10 con 0x555a15236370 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.491+0000 7f9a615e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b18e0 con 0x555a1522b5d0 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56088/0 (socket says 172.21.15.77:56088) 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 -- 172.21.15.77:0/1730388634 learned_addr learned my addr 172.21.15.77:0/1730388634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15222ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 -- 172.21.15.77:0/1730388634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 msgr2=0x555a15227e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a15227e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 -- 172.21.15.77:0/1730388634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 msgr2=0x555a15222ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15222ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 -- 172.21.15.77:0/1730388634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a152314e0 con 0x555a1522cb40 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15222ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a57fff640 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ba0 tx=0x7f9a440097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d34d6967b2c7f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a427fc640 1 -- 172.21.15.77:0/1730388634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44016040 con 0x555a1522cb40 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a427fc640 1 -- 172.21.15.77:0/1730388634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a44009e90 con 0x555a1522cb40 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.492+0000 7f9a427fc640 1 -- 172.21.15.77:0/1730388634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44004b80 con 0x555a1522cb40 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- 172.21.15.77:0/1730388634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 msgr2=0x555a15242310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ba0 tx=0x7f9a440097a0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- 172.21.15.77:0/1730388634 shutdown_connections 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a15236370 0x555a15222ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a1522cb40 0x555a15242310 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1730388634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a15227e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- 172.21.15.77:0/1730388634 >> 172.21.15.77:0/1730388634 conn(0x555a151f4310 msgr2=0x555a15227700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- 172.21.15.77:0/1730388634 shutdown_connections 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- 172.21.15.77:0/1730388634 wait complete. 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 -- start start 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.493+0000 7f9a615e7740 1 Processor -- start 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a1522cb40 0x555a152303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b2680 con 0x555a15236370 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b1fb0 con 0x555a1522cb40 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581af210 con 0x555a1522b5d0 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 10 client.0 _ll_register_callbacks cb 0x7f9a4c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56090/0 (socket says 172.21.15.77:56090) 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 -- 172.21.15.77:0/1728687370 learned_addr learned my addr 172.21.15.77:0/1728687370 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a57fff640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a1522cb40 0x555a152303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 -- 172.21.15.77:0/1728687370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 msgr2=0x555a1522fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 -- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a1522cb40 msgr2=0x555a152303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a1522cb40 0x555a152303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 -- 172.21.15.77:0/1728687370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a152afc10 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a5d7ab640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9a5002ade0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377a2c738e334eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50052060 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f9a50004450 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44009e90 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9a500319e0 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 10 client.18993 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.494+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555a152b0d60 con 0x555a15236370 2024-09-15T22:26:37.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a44009e90 con 0x555a15236370 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f9a500305c0 con 0x555a15236370 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 10 client.18993 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 20 client.18993 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a152b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 4 client.18993 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a15198d70 con 0x555a15245ee0 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a152b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a37fff640 10 client.18993 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.495+0000 7f9a5cfaa640 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a152b35b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b48ecf8345806f9 server_cookie=b7304807a8d3b369 in_seq=0 out_seq=0 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a37fff640 1 -- 172.21.15.77:0/1728687370 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a15198d70 con 0x555a15245ee0 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a37fff640 10 client.18993 handle_command_reply: tid=0 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 client.18993 shutdown 2024-09-15T22:26:37.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 msgr2=0x555a15230920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9a5002ade0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 shutdown_connections 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555a15245ee0 0x555a152b35b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a15236370 0x555a15230920 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a1522cb40 0x555a152303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a37fff640 0 client.18993 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 --2- 172.21.15.77:0/1728687370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555a1522b5d0 0x555a1522fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 >> 172.21.15.77:0/1728687370 conn(0x555a151f4310 msgr2=0x555a15227700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 shutdown_connections 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 1 -- 172.21.15.77:0/1728687370 wait complete. 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.496+0000 7f9a615e7740 20 client.18993 trim_cache size 0 max 16384 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 msgr2=0x7f9a480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 0x7f9a480a5870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fe0 tx=0x7f9a50031bb0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 msgr2=0x7f9a5819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9f0 tx=0x7f9a4c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 shutdown_connections 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9a480a33b0 0x7f9a480a5870 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a58106f10 0x7f9a5819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a58102790 0x7f9a5819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 --2- 172.21.15.77:0/2980376672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a58101de0 0x7f9a5819a610 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 >> 172.21.15.77:0/2980376672 conn(0x7f9a580fd8c0 msgr2=0x7f9a580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 shutdown_connections 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:37.497+0000 7f9a5f20e640 1 -- 172.21.15.77:0/2980376672 wait complete. 2024-09-15T22:26:37.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 26.80 MB/sec warmup 38 sec latency 11119.350 ms 2024-09-15T22:26:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:37 smithi077 ceph-mon[30980]: pgmap v445: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-09-15T22:26:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:37 smithi196 ceph-mon[35109]: pgmap v445: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-09-15T22:26:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:37 smithi117 ceph-mon[35021]: pgmap v445: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 583 op/s 2024-09-15T22:26:38.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 26.12 MB/sec warmup 39 sec latency 12119.429 ms 2024-09-15T22:26:39.386 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:26:39.386 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:26:39.566 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 25.46 MB/sec warmup 40 sec latency 13119.458 ms 2024-09-15T22:26:39.642 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:40.045 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:39 smithi077 ceph-mon[30980]: pgmap v446: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 256 op/s 2024-09-15T22:26:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 -- 172.21.15.77:0/2864413437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c100a80 msgr2=0x7f773c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 --2- 172.21.15.77:0/2864413437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c100a80 0x7f773c100ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7730009930 tx=0x7f773002a420 comp rx=0 tx=0).stop 2024-09-15T22:26:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 -- 172.21.15.77:0/2864413437 shutdown_connections 2024-09-15T22:26:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 --2- 172.21.15.77:0/2864413437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c101420 0x7f773c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 --2- 172.21.15.77:0/2864413437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c100a80 0x7f773c100ee0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 --2- 172.21.15.77:0/2864413437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 0x7f773c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 -- 172.21.15.77:0/2864413437 >> 172.21.15.77:0/2864413437 conn(0x7f773c0fc7a0 msgr2=0x7f773c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 -- 172.21.15.77:0/2864413437 shutdown_connections 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.124+0000 7f7741e14640 1 -- 172.21.15.77:0/2864413437 wait complete. 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 Processor -- start 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 -- start start 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 0x7f773c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 0x7f773c1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f773c110e80 con 0x7f773c101420 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f773c110d00 con 0x7f773c100a80 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f7741e14640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f773c111000 con 0x7f773c106a80 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f773affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f773affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59900/0 (socket says 172.21.15.77:59900) 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.125+0000 7f773affd640 1 -- 172.21.15.77:0/3023902424 learned_addr learned my addr 172.21.15.77:0/3023902424 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773b7fe640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 0x7f773c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 -- 172.21.15.77:0/3023902424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 msgr2=0x7f773c1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773bfff640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 0x7f773c1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 0x7f773c1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 -- 172.21.15.77:0/3023902424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 msgr2=0x7f773c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 0x7f773c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 -- 172.21.15.77:0/3023902424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c1a48d0 con 0x7f773c101420 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773b7fe640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 0x7f773c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f773affd640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7730009230 tx=0x7f7730004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f7738ff9640 1 -- 172.21.15.77:0/3023902424 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773003f030 con 0x7f773c101420 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f7738ff9640 1 -- 172.21.15.77:0/3023902424 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7730003ad0 con 0x7f773c101420 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c1a4aa0 con 0x7f773c101420 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f7738ff9640 1 -- 172.21.15.77:0/3023902424 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730035070 con 0x7f773c101420 2024-09-15T22:26:40.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.126+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c1a4ef0 con 0x7f773c101420 2024-09-15T22:26:40.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.128+0000 7f7738ff9640 1 -- 172.21.15.77:0/3023902424 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7730042030 con 0x7f773c101420 2024-09-15T22:26:40.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 Processor -- start 2024-09-15T22:26:40.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 -- start start 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559009dd7280 con 0x559009e71b60 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559009d7eb10 con 0x559009e72480 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f77441ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559009dd3240 con 0x559009e7cd40 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59902/0 (socket says 172.21.15.77:59902) 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773bfff640 1 -- 172.21.15.77:0/195475988 learned_addr learned my addr 172.21.15.77:0/195475988 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773b7fe640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f7738ff9640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 0x7f77240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773affd640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.129+0000 7f773bfff640 1 -- 172.21.15.77:0/195475988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 msgr2=0x559009e7f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 -- 172.21.15.77:0/195475988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 msgr2=0x559009e7c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 -- 172.21.15.77:0/195475988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559009d8b1f0 con 0x559009e71b60 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773b7fe640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 0x7f77240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f7738ff9640 1 -- 172.21.15.77:0/3023902424 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f77300f0650 con 0x7f773c101420 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7734009f30 tx=0x7f773400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130e93b4f7cca685 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f7721ffb640 1 -- 172.21.15.77:0/195475988 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734025640 con 0x559009e71b60 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f773bfff640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 0x7f77240a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7734020000 tx=0x7f773401d000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f7721ffb640 1 -- 172.21.15.77:0/195475988 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773401a8f0 con 0x559009e71b60 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.130+0000 7f7721ffb640 1 -- 172.21.15.77:0/195475988 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340107b0 con 0x559009e71b60 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- 172.21.15.77:0/195475988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 msgr2=0x559009e71f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7734009f30 tx=0x7f773400f7f0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- 172.21.15.77:0/195475988 shutdown_connections 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 --2- 172.21.15.77:0/195475988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e71f40 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- 172.21.15.77:0/195475988 >> 172.21.15.77:0/195475988 conn(0x559009e6e0f0 msgr2=0x559009e6e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- 172.21.15.77:0/195475988 shutdown_connections 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- 172.21.15.77:0/195475988 wait complete. 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 -- start start 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.131+0000 7f77441ed740 1 Processor -- start 2024-09-15T22:26:40.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559009e83d20 con 0x559009e71b60 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559009d91f10 con 0x559009e72480 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559009e83ea0 con 0x559009e7cd40 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f77441ed740 10 client.0 _ll_register_callbacks cb 0x559009d5aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59916/0 (socket says 172.21.15.77:59916) 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 -- 172.21.15.77:0/774168964 learned_addr learned my addr 172.21.15.77:0/774168964 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773affd640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773bfff640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 -- 172.21.15.77:0/774168964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 msgr2=0x559009e7a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 -- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 msgr2=0x559009e7a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 -- 172.21.15.77:0/774168964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559009b486a0 con 0x559009e71b60 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773affd640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f773b7fe640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f772c00bf50 tx=0x7f772c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd3d63219d92afe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.132+0000 7f771affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c01f060 con 0x559009e71b60 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f772c010960 con 0x559009e71b60 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f773401a8f0 con 0x559009e71b60 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f772c026050 con 0x559009e71b60 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 10 client.19005 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:40.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559009e8fa80 con 0x559009e71b60 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773401a8f0 con 0x559009e71b60 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f772c019030 con 0x559009e71b60 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 10 client.19005 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 20 client.19005 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009e90f40 0x559009e93390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 4 client.19005 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.133+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559009de47f0 con 0x559009e90f40 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.134+0000 7f773affd640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009e90f40 0x559009e93390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.134+0000 7f771affd640 10 client.19005 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.134+0000 7f773affd640 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009e90f40 0x559009e93390 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba0c0afbf1220de server_cookie=65e9f778af45e31d in_seq=0 out_seq=0 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f771affd640 1 -- 172.21.15.77:0/774168964 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f77300a1e80 con 0x559009e90f40 2024-09-15T22:26:40.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f771affd640 10 client.19005 handle_command_reply: tid=0 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 client.19005 shutdown 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 msgr2=0x559009e79cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f772c00bf50 tx=0x7f772c0128e0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 shutdown_connections 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009e90f40 0x559009e93390 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e7cd40 0x559009e7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e7a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 --2- 172.21.15.77:0/774168964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e71b60 0x559009e79cd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 >> 172.21.15.77:0/774168964 conn(0x559009e6e0f0 msgr2=0x559009e73e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f771affd640 0 client.19005 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:40.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 shutdown_connections 2024-09-15T22:26:40.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 1 -- 172.21.15.77:0/774168964 wait complete. 2024-09-15T22:26:40.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.135+0000 7f77441ed740 20 client.19005 trim_cache size 0 max 16384 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 Processor -- start 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 -- start start 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e98ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1acd60 con 0x559009e7cd40 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1ae960 con 0x559009e72480 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f77441ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1ad430 con 0x559009e71b60 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59930/0 (socket says 172.21.15.77:59930) 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 -- 172.21.15.77:0/2073190244 learned_addr learned my addr 172.21.15.77:0/2073190244 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773affd640 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e98ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773b7fe640 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 -- 172.21.15.77:0/2073190244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 msgr2=0x559009e7bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e7bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 -- 172.21.15.77:0/2073190244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 msgr2=0x559009e98ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.152+0000 7f773bfff640 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e98ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f773bfff640 1 -- 172.21.15.77:0/2073190244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559009e6f1a0 con 0x559009e7cd40 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f773bfff640 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7734009e90 tx=0x7f77340103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65297af0f7e88cad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77217fa640 1 -- 172.21.15.77:0/2073190244 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734034070 con 0x559009e7cd40 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77217fa640 1 -- 172.21.15.77:0/2073190244 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734029040 con 0x559009e7cd40 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77217fa640 1 -- 172.21.15.77:0/2073190244 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773402f480 con 0x559009e7cd40 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 -- 172.21.15.77:0/2073190244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 msgr2=0x559009e991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7734009e90 tx=0x7f77340103d0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 -- 172.21.15.77:0/2073190244 shutdown_connections 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e991e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e98ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 --2- 172.21.15.77:0/2073190244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e7bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 -- 172.21.15.77:0/2073190244 >> 172.21.15.77:0/2073190244 conn(0x559009e6e0f0 msgr2=0x559009e7b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.153+0000 7f77441ed740 1 -- 172.21.15.77:0/2073190244 shutdown_connections 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 -- 172.21.15.77:0/2073190244 wait complete. 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 -- start start 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 Processor -- start 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e89770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1ae1d0 con 0x559009e7cd40 2024-09-15T22:26:40.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1adb00 con 0x559009e72480 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f773c1aad60 con 0x559009e71b60 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f77441ed740 10 client.0 _ll_register_callbacks cb 0x7f772c00b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f773bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59942/0 (socket says 172.21.15.77:59942) 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.154+0000 7f773bfff640 1 -- 172.21.15.77:0/4294929279 learned_addr learned my addr 172.21.15.77:0/4294929279 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773b7fe640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e89770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773affd640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 -- 172.21.15.77:0/4294929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 msgr2=0x559009e89770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e89770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 -- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 msgr2=0x559009e768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 -- 172.21.15.77:0/4294929279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559009e77630 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f773bfff640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7734032040 tx=0x7f77340056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f6e6f4a791ad95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f771affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734034040 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f7734004c00 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7734029040 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7734043050 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f77441ed740 10 client.19011 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559009efbbf0 con 0x559009e7cd40 2024-09-15T22:26:40.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.155+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734029040 con 0x559009e7cd40 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f773402f9e0 con 0x559009e7cd40 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 10 client.19011 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 20 client.19011 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009efc1c0 0x559009efe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 4 client.19011 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559009dded00 con 0x559009efc1c0 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f773b7fe640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009efc1c0 0x559009efe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f771affd640 10 client.19011 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:40.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.156+0000 7f773b7fe640 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009efc1c0 0x559009efe5e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666bbb68bc0c5702 server_cookie=6b8822fbbf4285be in_seq=0 out_seq=0 2024-09-15T22:26:40.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.157+0000 7f771affd640 1 -- 172.21.15.77:0/4294929279 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559009dded00 con 0x559009efc1c0 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.157+0000 7f771affd640 10 client.19011 handle_command_reply: tid=0 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.157+0000 7f77441ed740 1 client.19011 shutdown 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 msgr2=0x559009e76e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7734032040 tx=0x7f77340056b0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 shutdown_connections 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559009efc1c0 0x559009efe5e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559009e7cd40 0x559009e76e00 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559009e72480 0x559009e768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 --2- 172.21.15.77:0/4294929279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559009e71b60 0x559009e89770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 >> 172.21.15.77:0/4294929279 conn(0x559009e6e0f0 msgr2=0x559009e6d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f771affd640 0 client.19011 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 shutdown_connections 2024-09-15T22:26:40.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 1 -- 172.21.15.77:0/4294929279 wait complete. 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.158+0000 7f77441ed740 20 client.19011 trim_cache size 0 max 16384 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "12ad12a8-d1bd-4462-a520-cc370fd873c7", 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:26:40.168 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 msgr2=0x7f77240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 0x7f77240a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7734020000 tx=0x7f773401d000 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 msgr2=0x7f773c199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7730009230 tx=0x7f7730004270 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 shutdown_connections 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f77240a33b0 0x7f77240a5870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f773c106a80 0x7f773c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f773c101420 0x7f773c199850 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 --2- 172.21.15.77:0/3023902424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f773c100a80 0x7f773c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:40.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 >> 172.21.15.77:0/3023902424 conn(0x7f773c0fc7a0 msgr2=0x7f773c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:40.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 shutdown_connections 2024-09-15T22:26:40.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:40.159+0000 7f7741e14640 1 -- 172.21.15.77:0/3023902424 wait complete. 2024-09-15T22:26:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:39 smithi196 ceph-mon[35109]: pgmap v446: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 256 op/s 2024-09-15T22:26:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:39 smithi117 ceph-mon[35021]: pgmap v446: 129 pgs: 129 active+clean; 866 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 256 op/s 2024-09-15T22:26:40.565 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 24.84 MB/sec warmup 41 sec latency 14119.488 ms 2024-09-15T22:26:41.003 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:26:41.247 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:26:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:26:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:26:41.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:26:41.577 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 24.25 MB/sec warmup 42 sec latency 15131.581 ms 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 -- 172.21.15.77:0/2102574034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0102790 msgr2=0x7feaf01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 --2- 172.21.15.77:0/2102574034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0102790 0x7feaf01067e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7feae4009900 tx=0x7feae402a410 comp rx=0 tx=0).stop 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 -- 172.21.15.77:0/2102574034 shutdown_connections 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 --2- 172.21.15.77:0/2102574034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0106f10 0x7feaf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 --2- 172.21.15.77:0/2102574034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0102790 0x7feaf01067e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 --2- 172.21.15.77:0/2102574034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaf0101de0 0x7feaf01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 -- 172.21.15.77:0/2102574034 >> 172.21.15.77:0/2102574034 conn(0x7feaf00fd8c0 msgr2=0x7feaf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 -- 172.21.15.77:0/2102574034 shutdown_connections 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.716+0000 7feaf612e640 1 -- 172.21.15.77:0/2102574034 wait complete. 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 Processor -- start 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 -- start start 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 0x7feaf0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaf0106f10 0x7feaf019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0112010 con 0x7feaf0101de0 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0111e90 con 0x7feaf0102790 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaf612e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0112190 con 0x7feaf0106f10 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59974/0 (socket says 172.21.15.77:59974) 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.717+0000 7feaef7fe640 1 -- 172.21.15.77:0/236746102 learned_addr learned my addr 172.21.15.77:0/236746102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 -- 172.21.15.77:0/236746102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaf0106f10 msgr2=0x7feaf019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaf0106f10 0x7feaf019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaeeffd640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 0x7feaf0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 -- 172.21.15.77:0/236746102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 msgr2=0x7feaf0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 0x7feaf0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 -- 172.21.15.77:0/236746102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf019d560 con 0x7feaf0101de0 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaeeffd640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 0x7feaf0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaef7fe640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7feae000ba50 tx=0x7feae000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaecff9640 1 -- 172.21.15.77:0/236746102 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0019070 con 0x7feaf0101de0 2024-09-15T22:26:41.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaecff9640 1 -- 172.21.15.77:0/236746102 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae000cbe0 con 0x7feaf0101de0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf019d850 con 0x7feaf0101de0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaecff9640 1 -- 172.21.15.77:0/236746102 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00076e0 con 0x7feaf0101de0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.718+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf01a5130 con 0x7feaf0101de0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.720+0000 7feaecff9640 1 -- 172.21.15.77:0/236746102 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7feae001c070 con 0x7feaf0101de0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 Processor -- start 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 -- start start 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaecff9640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 0x7fead80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a516140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a518a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55978a4712e0 con 0x55978a50b4a0 2024-09-15T22:26:41.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55978a418ce0 con 0x55978a50bdc0 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaf8507740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55978a46d2a0 con 0x55978a516680 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaeffff640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 0x7fead80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaecff9640 1 -- 172.21.15.77:0/236746102 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7feae00cf6b0 con 0x7feaf0101de0 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59990/0 (socket says 172.21.15.77:59990) 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 -- 172.21.15.77:0/4107450143 learned_addr learned my addr 172.21.15.77:0/4107450143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaeeffd640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a516140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaeffff640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a518a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 -- 172.21.15.77:0/4107450143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 msgr2=0x55978a518a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a518a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 -- 172.21.15.77:0/4107450143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 msgr2=0x55978a516140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a516140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaef7fe640 1 -- 172.21.15.77:0/4107450143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978a3fd440 con 0x55978a50b4a0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.721+0000 7feaeffff640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a518a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaef7fe640 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feae00a3df0 tx=0x7feae009b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91f764ffca6e463 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaeffff640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 0x7fead80a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55978a508db0 tx=0x7feae80091c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7fead5ffb640 1 -- 172.21.15.77:0/4107450143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae009b6b0 con 0x55978a50b4a0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7fead5ffb640 1 -- 172.21.15.77:0/4107450143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae009fa60 con 0x55978a50b4a0 2024-09-15T22:26:41.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7fead5ffb640 1 -- 172.21.15.77:0/4107450143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae007f330 con 0x55978a50b4a0 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 -- 172.21.15.77:0/4107450143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 msgr2=0x55978a50b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7feae00a3df0 tx=0x7feae009b320 comp rx=0 tx=0).stop 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 -- 172.21.15.77:0/4107450143 shutdown_connections 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a518a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a516140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 --2- 172.21.15.77:0/4107450143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a50b880 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 -- 172.21.15.77:0/4107450143 >> 172.21.15.77:0/4107450143 conn(0x55978a507a30 msgr2=0x55978a507e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.722+0000 7feaf8507740 1 -- 172.21.15.77:0/4107450143 shutdown_connections 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 -- 172.21.15.77:0/4107450143 wait complete. 2024-09-15T22:26:41.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 -- start start 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 Processor -- start 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a5132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a513830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55978a519250 con 0x55978a50b4a0 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55978a42c050 con 0x55978a50bdc0 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55978a5193d0 con 0x55978a516680 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaf8507740 10 client.0 _ll_register_callbacks cb 0x55978a46c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59992/0 (socket says 172.21.15.77:59992) 2024-09-15T22:26:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.723+0000 7feaef7fe640 1 -- 172.21.15.77:0/3605902115 learned_addr learned my addr 172.21.15.77:0/3605902115 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaeeffd640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a5132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 -- 172.21.15.77:0/3605902115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 msgr2=0x55978a513830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a513830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 -- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 msgr2=0x55978a5132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a5132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 -- 172.21.15.77:0/3605902115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978a1e23c0 con 0x55978a50b4a0 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaef7fe640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feae0000f80 tx=0x7feae00999b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1e3253807ba314 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00ccea0 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7feae00a3e90 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae009fa60 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feae009b6b0 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaf8507740 10 client.19023 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55978a524bb0 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.724+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae009fa60 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7feae00bb040 con 0x55978a50b4a0 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 10 client.19023 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:41.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 20 client.19023 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a525f00 0x55978a5282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 4 client.19023 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55978a47e850 con 0x55978a525f00 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaeeffd640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a525f00 0x55978a5282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaceffd640 10 client.19023 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.725+0000 7feaeeffd640 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a525f00 0x55978a5282f0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c759ce00946a9a7 server_cookie=32bab962b6d7f6f4 in_seq=0 out_seq=0 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaceffd640 1 -- 172.21.15.77:0/3605902115 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55978a47e850 con 0x55978a525f00 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaceffd640 10 client.19023 handle_command_reply: tid=0 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 client.19023 shutdown 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 msgr2=0x55978a516000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7feae0000f80 tx=0x7feae00999b0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 shutdown_connections 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a525f00 0x55978a5282f0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a516680 0x55978a513830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a50bdc0 0x55978a5132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaceffd640 0 client.19023 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 --2- 172.21.15.77:0/3605902115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a516000 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 >> 172.21.15.77:0/3605902115 conn(0x55978a507a30 msgr2=0x55978a50d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.726+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 shutdown_connections 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.727+0000 7feaf8507740 1 -- 172.21.15.77:0/3605902115 wait complete. 2024-09-15T22:26:41.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.727+0000 7feaf8507740 20 client.19023 trim_cache size 0 max 16384 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.743+0000 7feaf8507740 1 Processor -- start 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.743+0000 7feaf8507740 1 -- start start 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50bdc0 0x55978a51ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a51ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01af110 con 0x55978a50b4a0 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b0d10 con 0x55978a516680 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaf8507740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01af7e0 con 0x55978a50bdc0 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60008/0 (socket says 172.21.15.77:60008) 2024-09-15T22:26:41.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 -- 172.21.15.77:0/684277638 learned_addr learned my addr 172.21.15.77:0/684277638 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 -- 172.21.15.77:0/684277638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50bdc0 msgr2=0x55978a51ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50bdc0 0x55978a51ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 -- 172.21.15.77:0/684277638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 msgr2=0x55978a51ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a51ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 -- 172.21.15.77:0/684277638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978a516370 con 0x55978a50b4a0 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7feaef7fe640 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feae0099960 tx=0x7feae009b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4222daaef3fbcc60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7fead57fa640 1 -- 172.21.15.77:0/684277638 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00ccea0 con 0x55978a50b4a0 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7fead57fa640 1 -- 172.21.15.77:0/684277638 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae009fa60 con 0x55978a50b4a0 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.744+0000 7fead57fa640 1 -- 172.21.15.77:0/684277638 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00afbd0 con 0x55978a50b4a0 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- 172.21.15.77:0/684277638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 msgr2=0x55978a5224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7feae0099960 tx=0x7feae009b320 comp rx=0 tx=0).stop 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- 172.21.15.77:0/684277638 shutdown_connections 2024-09-15T22:26:41.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a51ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50bdc0 0x55978a51ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 --2- 172.21.15.77:0/684277638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50b4a0 0x55978a5224b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- 172.21.15.77:0/684277638 >> 172.21.15.77:0/684277638 conn(0x55978a507a30 msgr2=0x55978a50e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- 172.21.15.77:0/684277638 shutdown_connections 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- 172.21.15.77:0/684277638 wait complete. 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 -- start start 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.745+0000 7feaf8507740 1 Processor -- start 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50b4a0 0x55978a58d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a50dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b0580 con 0x55978a50bdc0 2024-09-15T22:26:41.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01afeb0 con 0x55978a516680 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01ad110 con 0x55978a50b4a0 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 10 client.0 _ll_register_callbacks cb 0x7feae009b1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60010/0 (socket says 172.21.15.77:60010) 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 -- 172.21.15.77:0/1236506248 learned_addr learned my addr 172.21.15.77:0/1236506248 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaef7fe640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50b4a0 0x55978a58d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeffff640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a50dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 -- 172.21.15.77:0/1236506248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50b4a0 msgr2=0x55978a58d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50b4a0 0x55978a58d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 -- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 msgr2=0x55978a50dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:41.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a50dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 -- 172.21.15.77:0/1236506248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978a50e450 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeffff640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a50dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaeeffd640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7feae4005ab0 tx=0x7feae40327a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a923860e1327dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaceffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4035040 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7feae4004040 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.746+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae009fa60 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feae4051050 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 10 client.19029 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55978a594970 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae009fa60 con 0x55978a50bdc0 2024-09-15T22:26:41.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7feae40047e0 con 0x55978a50bdc0 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 10 client.19029 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 20 client.19029 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a595230 0x55978a597650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 4 client.19029 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55978a478d60 con 0x55978a595230 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.747+0000 7feaef7fe640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a595230 0x55978a597650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaef7fe640 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a595230 0x55978a597650 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96135162c43d951a server_cookie=61a72c4245fa3d7d in_seq=0 out_seq=0 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaceffd640 10 client.19029 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaceffd640 1 -- 172.21.15.77:0/1236506248 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55978a478d60 con 0x55978a595230 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaceffd640 10 client.19029 handle_command_reply: tid=0 2024-09-15T22:26:41.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 client.19029 shutdown 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 msgr2=0x55978a58d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7feae4005ab0 tx=0x7feae40327a0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 shutdown_connections 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55978a595230 0x55978a597650 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55978a516680 0x55978a50dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55978a50bdc0 0x55978a58d620 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 --2- 172.21.15.77:0/1236506248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55978a50b4a0 0x55978a58d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 >> 172.21.15.77:0/1236506248 conn(0x55978a507a30 msgr2=0x55978a514b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaceffd640 0 client.19029 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 shutdown_connections 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 1 -- 172.21.15.77:0/1236506248 wait complete. 2024-09-15T22:26:41.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.748+0000 7feaf8507740 20 client.19029 trim_cache size 0 max 16384 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "12ad12a8-d1bd-4462-a520-cc370fd873c7": { 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "12ad12a8-d1bd-4462-a520-cc370fd873c7", 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:41.763 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:26:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 msgr2=0x7fead80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 0x7fead80a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55978a508db0 tx=0x7feae80091c0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 msgr2=0x7feaf01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7feae000ba50 tx=0x7feae000bf10 comp rx=0 tx=0).stop 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 shutdown_connections 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fead80a33b0 0x7fead80a5870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaf0106f10 0x7feaf019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0102790 0x7feaf0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 --2- 172.21.15.77:0/236746102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feaf0101de0 0x7feaf01985b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 >> 172.21.15.77:0/236746102 conn(0x7feaf00fd8c0 msgr2=0x7feaf00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 shutdown_connections 2024-09-15T22:26:41.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:41.749+0000 7feaf612e640 1 -- 172.21.15.77:0/236746102 wait complete. 2024-09-15T22:26:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:41 smithi077 ceph-mon[30980]: pgmap v447: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2024-09-15T22:26:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:41 smithi196 ceph-mon[35109]: pgmap v447: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2024-09-15T22:26:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:41 smithi117 ceph-mon[35021]: pgmap v447: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.7 MiB/s wr, 263 op/s 2024-09-15T22:26:42.577 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 23.68 MB/sec warmup 43 sec latency 16131.635 ms 2024-09-15T22:26:42.589 INFO:tasks.cephfs.filesystem:scrub status for tag:12ad12a8-d1bd-4462-a520-cc370fd873c7 - {'path': '/', 'tag': '12ad12a8-d1bd-4462-a520-cc370fd873c7', 'options': 'recursive,force'} 2024-09-15T22:26:43.578 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 23.14 MB/sec warmup 44 sec latency 17131.723 ms 2024-09-15T22:26:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:43 smithi077 ceph-mon[30980]: pgmap v448: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:43 smithi196 ceph-mon[35109]: pgmap v448: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:43 smithi117 ceph-mon[35021]: pgmap v448: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:44.580 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 22.63 MB/sec warmup 45 sec latency 18131.845 ms 2024-09-15T22:26:45.578 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 22.14 MB/sec warmup 46 sec latency 19131.966 ms 2024-09-15T22:26:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:45 smithi077 ceph-mon[30980]: pgmap v449: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:45 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:26:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:45 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:26:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:45 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:26:46.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:45 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:26:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:45 smithi196 ceph-mon[35109]: pgmap v449: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:45 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:26:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:45 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:26:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:45 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:26:46.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:45 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:26:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:45 smithi117 ceph-mon[35021]: pgmap v449: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 30 op/s 2024-09-15T22:26:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:45 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:26:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:45 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:26:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:45 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:26:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:45 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:26:46.578 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 21.67 MB/sec warmup 47 sec latency 20132.062 ms 2024-09-15T22:26:47.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:26:47.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:26:47.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:26:47.578 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 21.22 MB/sec warmup 48 sec latency 21132.092 ms 2024-09-15T22:26:48.009 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:26:48.255 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:47 smithi196 ceph-mon[35109]: pgmap v450: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 156 KiB/s wr, 35 op/s 2024-09-15T22:26:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:48.263 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:47 smithi117 ceph-mon[35021]: pgmap v450: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 156 KiB/s wr, 35 op/s 2024-09-15T22:26:48.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:47 smithi077 ceph-mon[30980]: pgmap v450: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 156 KiB/s wr, 35 op/s 2024-09-15T22:26:48.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 20.78 MB/sec warmup 49 sec latency 22145.660 ms 2024-09-15T22:26:48.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/2370682141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 msgr2=0x7f6cd4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:48.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/2370682141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd4114f80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009ca0 tx=0x7f6ccc02a950 comp rx=0 tx=0).stop 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/2370682141 shutdown_connections 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/2370682141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd4114f80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/2370682141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4074eb0 0x7f6cd4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/2370682141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 0x7f6cd4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.745+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/2370682141 >> 172.21.15.77:0/2370682141 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/2370682141 shutdown_connections 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/2370682141 wait complete. 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 Processor -- start 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- start start 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 0x7f6cd41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4074eb0 0x7f6cd41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:48.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4118550 con 0x7f6cd4112b90 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41183d0 con 0x7f6cd4074590 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd9e4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41186d0 con 0x7f6cd4074eb0 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60030/0 (socket says 172.21.15.77:60030) 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd3fff640 1 -- 172.21.15.77:0/942067279 learned_addr learned my addr 172.21.15.77:0/942067279 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.746+0000 7f6cd37fe640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 0x7f6cd41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 -- 172.21.15.77:0/942067279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4074eb0 msgr2=0x7f6cd41a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4074eb0 0x7f6cd41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 -- 172.21.15.77:0/942067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 msgr2=0x7f6cd41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 0x7f6cd41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 -- 172.21.15.77:0/942067279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd419efa0 con 0x7f6cd4112b90 2024-09-15T22:26:48.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd3fff640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004af0 tx=0x7f6ccc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc034040 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc030920 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd419f230 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004780 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.747+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41ab580 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.748+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd4074970 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.749+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6ccc037020 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.750+0000 7f6cd17fa640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 0x7f6cc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.750+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7f6ccc0f0ff0 con 0x7f6cd4112b90 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.750+0000 7f6cd37fe640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 0x7f6cc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.750+0000 7f6cd37fe640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 0x7f6cc00a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6cd419f9b0 tx=0x7f6cc4005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.753+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ccc0811b0 con 0x7f6cd4112b90 2024-09-15T22:26:48.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.955+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cd41ab870 con 0x7f6cd4112b90 2024-09-15T22:26:48.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.955+0000 7f6cd17fa640 1 -- 172.21.15.77:0/942067279 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7f6ccc0bd6f0 con 0x7f6cd4112b90 2024-09-15T22:26:48.957 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:26:48.957 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:26:48.958 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:26:48.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 msgr2=0x7f6cc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:48.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 0x7f6cc00a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6cd419f9b0 tx=0x7f6cc4005f90 comp rx=0 tx=0).stop 2024-09-15T22:26:48.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 msgr2=0x7f6cd419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004af0 tx=0x7f6ccc02f5b0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 shutdown_connections 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6cc00a33b0 0x7f6cc00a5870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4112b90 0x7f6cd419e870 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4074eb0 0x7f6cd41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 --2- 172.21.15.77:0/942067279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd4074590 0x7f6cd41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 >> 172.21.15.77:0/942067279 conn(0x7f6cd40fd8c0 msgr2=0x7f6cd40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 shutdown_connections 2024-09-15T22:26:48.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:48.958+0000 7f6cd9e4b640 1 -- 172.21.15.77:0/942067279 wait complete. 2024-09-15T22:26:49.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:48 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/942067279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:49.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:48 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/942067279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:49.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:48 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/942067279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:26:49.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 20.36 MB/sec warmup 50 sec latency 23145.954 ms 2024-09-15T22:26:49.747 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 1fc55bd9 --release --await' 2024-09-15T22:26:49.747 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 1fc55bd9 --release --await 2024-09-15T22:26:49.997 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:50.266 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:49 smithi077 ceph-mon[30980]: pgmap v451: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-15T22:26:50.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:49 smithi196 ceph-mon[35109]: pgmap v451: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-15T22:26:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:49 smithi117 ceph-mon[35021]: pgmap v451: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-15T22:26:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 -- 172.21.15.77:0/1187054333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c101de0 msgr2=0x7fb50c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 --2- 172.21.15.77:0/1187054333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c101de0 0x7fb50c1021c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb508009900 tx=0x7fb50802a410 comp rx=0 tx=0).stop 2024-09-15T22:26:50.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 -- 172.21.15.77:0/1187054333 shutdown_connections 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 --2- 172.21.15.77:0/1187054333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c106f10 0x7fb50c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 --2- 172.21.15.77:0/1187054333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 0x7fb50c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 --2- 172.21.15.77:0/1187054333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c101de0 0x7fb50c1021c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 -- 172.21.15.77:0/1187054333 >> 172.21.15.77:0/1187054333 conn(0x7fb50c0fd8c0 msgr2=0x7fb50c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 -- 172.21.15.77:0/1187054333 shutdown_connections 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.495+0000 7fb5143be640 1 -- 172.21.15.77:0/1187054333 wait complete. 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 Processor -- start 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 -- start start 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 0x7fb50c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 0x7fb50c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c111f70 con 0x7fb50c106f10 2024-09-15T22:26:50.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c111df0 con 0x7fb50c101de0 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb5143be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1120f0 con 0x7fb50c102790 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56576/0 (socket says 172.21.15.77:56576) 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 -- 172.21.15.77:0/2659153035 learned_addr learned my addr 172.21.15.77:0/2659153035 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51215a640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 0x7fb50c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 -- 172.21.15.77:0/2659153035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 msgr2=0x7fb50c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb511959640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 0x7fb50c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 0x7fb50c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 -- 172.21.15.77:0/2659153035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 msgr2=0x7fb50c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 0x7fb50c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51295b640 1 -- 172.21.15.77:0/2659153035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c19d3f0 con 0x7fb50c106f10 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.496+0000 7fb51215a640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 0x7fb50c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:50.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb51295b640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb50400eaa0 tx=0x7fb50400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb5037fe640 1 -- 172.21.15.77:0/2659153035 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50400cdb0 con 0x7fb50c106f10 2024-09-15T22:26:50.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb5037fe640 1 -- 172.21.15.77:0/2659153035 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504005020 con 0x7fb50c106f10 2024-09-15T22:26:50.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c19d6e0 con 0x7fb50c106f10 2024-09-15T22:26:50.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb5037fe640 1 -- 172.21.15.77:0/2659153035 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5040116b0 con 0x7fb50c106f10 2024-09-15T22:26:50.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.497+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a4fc0 con 0x7fb50c106f10 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.499+0000 7fb5037fe640 1 -- 172.21.15.77:0/2659153035 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb504005560 con 0x7fb50c106f10 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.499+0000 7fb516797740 1 Processor -- start 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 -- start start 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561d4b41a620 con 0x561d4b4c0240 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb5037fe640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 0x7fb4f80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d4b3c1e50 con 0x561d4b4b5060 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb516797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4b4165e0 con 0x561d4b4b5980 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56586/0 (socket says 172.21.15.77:56586) 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb511959640 1 -- 172.21.15.77:0/742472274 learned_addr learned my addr 172.21.15.77:0/742472274 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb51295b640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 0x7fb4f80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb51215a640 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb511959640 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.500+0000 7fb5037fe640 1 -- 172.21.15.77:0/2659153035 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fb5040d02f0 con 0x7fb50c106f10 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 -- 172.21.15.77:0/742472274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 msgr2=0x561d4b4bfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 -- 172.21.15.77:0/742472274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 msgr2=0x561d4b4b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 -- 172.21.15.77:0/742472274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4b3b2d40 con 0x561d4b4c0240 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb511959640 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc00e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75424a64513caf13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb51295b640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 0x7fb4f80a58f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5040cd760 tx=0x7fb5040058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb500dea640 1 -- 172.21.15.77:0/742472274 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc018030 con 0x561d4b4c0240 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb500dea640 1 -- 172.21.15.77:0/742472274 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc005500 con 0x561d4b4c0240 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.501+0000 7fb500dea640 1 -- 172.21.15.77:0/742472274 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc010640 con 0x561d4b4c0240 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- 172.21.15.77:0/742472274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 msgr2=0x561d4b4c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005fd0 tx=0x7fb4fc00e5b0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- 172.21.15.77:0/742472274 shutdown_connections 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c2630 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- 172.21.15.77:0/742472274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- 172.21.15.77:0/742472274 >> 172.21.15.77:0/742472274 conn(0x561d4b4b15f0 msgr2=0x561d4b4b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- 172.21.15.77:0/742472274 shutdown_connections 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- 172.21.15.77:0/742472274 wait complete. 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- start start 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 Processor -- start 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4c0240 0x561d4b4bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561d4b4c2e10 con 0x561d4b4b5060 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d4b3a2f10 con 0x561d4b4c0240 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.502+0000 7fb516797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4b4c2f90 con 0x561d4b4b5980 2024-09-15T22:26:50.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb516797740 10 client.0 _ll_register_callbacks cb 0x561d4b3c6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56590/0 (socket says 172.21.15.77:56590) 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 -- 172.21.15.77:0/2230424592 learned_addr learned my addr 172.21.15.77:0/2230424592 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb51215a640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 -- 172.21.15.77:0/2230424592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 msgr2=0x561d4b4bd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 -- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4c0240 msgr2=0x561d4b4bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4c0240 0x561d4b4bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 -- 172.21.15.77:0/2230424592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4b129580 con 0x561d4b4b5060 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb511959640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc004fc0 tx=0x7fb4fc00e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3c9e450483be98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc018050 con 0x561d4b4b5060 2024-09-15T22:26:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fb4fc026470 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc005500 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc02f050 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb516797740 10 client.19047 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.503+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d4b1f3960 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc005500 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb4fc00f040 con 0x561d4b4b5060 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 10 client.19047 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 10 client.19047 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 20 client.19047 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b4d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 4 client.19047 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:26:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d4b427b90 con 0x561d4b4cfde0 2024-09-15T22:26:50.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.504+0000 7fb51215a640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b4d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb51215a640 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b4d21d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a6f7a3e0fdeb25 server_cookie=b7338e1e56a3796d in_seq=0 out_seq=0 2024-09-15T22:26:50.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb4f5ffb640 10 client.19047 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/2230424592 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d4b427b90 con 0x561d4b4cfde0 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb4f5ffb640 10 client.19047 handle_command_reply: tid=0 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb516797740 1 client.19047 shutdown 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 msgr2=0x561d4b4bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.505+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc004fc0 tx=0x7fb4fc00e6e0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 shutdown_connections 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b4d21d0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4c0240 0x561d4b4bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb4f5ffb640 0 client.19047 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4bd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 --2- 172.21.15.77:0/2230424592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5060 0x561d4b4bd100 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 >> 172.21.15.77:0/2230424592 conn(0x561d4b4b15f0 msgr2=0x561d4b4b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 shutdown_connections 2024-09-15T22:26:50.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 1 -- 172.21.15.77:0/2230424592 wait complete. 2024-09-15T22:26:50.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.506+0000 7fb516797740 20 client.19047 trim_cache size 0 max 16384 2024-09-15T22:26:50.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 Processor -- start 2024-09-15T22:26:50.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 -- start start 2024-09-15T22:26:50.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 0x561d4b4d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1af090 con 0x561d4b4b5980 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b0c90 con 0x561d4b4b5060 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb516797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1af760 con 0x561d4b4c0240 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56592/0 (socket says 172.21.15.77:56592) 2024-09-15T22:26:50.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 -- 172.21.15.77:0/4061774246 learned_addr learned my addr 172.21.15.77:0/4061774246 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:50.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 -- 172.21.15.77:0/4061774246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 msgr2=0x561d4b4d7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb511959640 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 0x561d4b4d7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 0x561d4b4d7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 -- 172.21.15.77:0/4061774246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 msgr2=0x561d4b4b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 -- 172.21.15.77:0/4061774246 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4b4b1420 con 0x561d4b4b5980 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb511959640 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 0x561d4b4d7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.523+0000 7fb51215a640 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb5080096c0 tx=0x7fb508005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6288dde087f899 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.524+0000 7fb4f7fff640 1 -- 172.21.15.77:0/4061774246 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508035040 con 0x561d4b4b5980 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.524+0000 7fb4f7fff640 1 -- 172.21.15.77:0/4061774246 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb508049070 con 0x561d4b4b5980 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.524+0000 7fb4f7fff640 1 -- 172.21.15.77:0/4061774246 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508004bf0 con 0x561d4b4b5980 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- 172.21.15.77:0/4061774246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 msgr2=0x561d4b4d7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb5080096c0 tx=0x7fb508005b50 comp rx=0 tx=0).stop 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- 172.21.15.77:0/4061774246 shutdown_connections 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4c0240 0x561d4b4d7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4b5980 0x561d4b4d7800 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- 172.21.15.77:0/4061774246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- 172.21.15.77:0/4061774246 >> 172.21.15.77:0/4061774246 conn(0x561d4b4b15f0 msgr2=0x561d4b4b6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- 172.21.15.77:0/4061774246 shutdown_connections 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- 172.21.15.77:0/4061774246 wait complete. 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- start start 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 Processor -- start 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1b0500 con 0x561d4b4c0240 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1afe30 con 0x561d4b4b5060 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ad090 con 0x561d4b4b5980 2024-09-15T22:26:50.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.525+0000 7fb516797740 10 client.0 _ll_register_callbacks cb 0x7fb4fc005450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56600/0 (socket says 172.21.15.77:56600) 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 -- 172.21.15.77:0/1374625283 learned_addr learned my addr 172.21.15.77:0/1374625283 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb51215a640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 -- 172.21.15.77:0/1374625283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 msgr2=0x561d4b4c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 -- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 msgr2=0x561d4b4c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 -- 172.21.15.77:0/1374625283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4b4b2280 con 0x561d4b4c0240 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb511959640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00ead0 tx=0x7fb4fc0053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ed60cfbb2d5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:50.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc03a030 con 0x561d4b4c0240 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9438+0+0 (secure 0 0 0) 0x7fb4fc02c070 con 0x561d4b4c0240 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.526+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb508049070 con 0x561d4b4c0240 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc044050 con 0x561d4b4c0240 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 10 client.19053 fetch_fsmap learned FSMap version 36 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d4b53adf0 con 0x561d4b4c0240 2024-09-15T22:26:50.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb508049070 con 0x561d4b4c0240 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb4fc026470 con 0x561d4b4c0240 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 10 client.19053 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 10 client.19053 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 20 client.19053 populate_metadata read hostname 'smithi077' 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b53d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 4 client.19053 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1fc55bd9", "release": true, "await": true}] 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1fc55bd9", "release": true, "await": true}) -- 0x561d4b4220a0 con 0x561d4b4cfde0 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.527+0000 7fb51215a640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b53d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.528+0000 7fb4f5ffb640 10 client.19053 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:50.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.528+0000 7fb51215a640 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b53d4f0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea10d2d044ea6bc server_cookie=a2fde2e508817993 in_seq=0 out_seq=0 2024-09-15T22:26:50.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 15679 19.96 MB/sec warmup 51 sec latency 24145.986 ms 2024-09-15T22:26:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.597+0000 7fb4f5ffb640 1 -- 172.21.15.77:0/1374625283 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561d4b4220a0 con 0x561d4b4cfde0 2024-09-15T22:26:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.597+0000 7fb4f5ffb640 10 client.19053 handle_command_reply: tid=0 2024-09-15T22:26:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.597+0000 7fb516797740 1 client.19053 shutdown 2024-09-15T22:26:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 msgr2=0x561d4b4c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00ead0 tx=0x7fb4fc0053a0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 shutdown_connections 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x561d4b4cfde0 0x561d4b53d4f0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561d4b4c0240 0x561d4b4c51b0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b4b5980 0x561d4b4c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb4f5ffb640 0 client.19053 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 --2- 172.21.15.77:0/1374625283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d4b4b5060 0x561d4b4c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 >> 172.21.15.77:0/1374625283 conn(0x561d4b4b15f0 msgr2=0x561d4b4bd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 shutdown_connections 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 1 -- 172.21.15.77:0/1374625283 wait complete. 2024-09-15T22:26:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.598+0000 7fb516797740 20 client.19053 trim_cache size 0 max 16384 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 36, 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 52, 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "1fc55bd9": { 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 52, 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:26:50.601 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 146.5, 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:50.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:50.603 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:26:50.603 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:26:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.599+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 msgr2=0x7fb4f80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.599+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 0x7fb4f80a58f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5040cd760 tx=0x7fb5040058d0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.599+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 msgr2=0x7fb50c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.599+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb50400eaa0 tx=0x7fb50400ef60 comp rx=0 tx=0).stop 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 shutdown_connections 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb4f80a3430 0x7fb4f80a58f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c106f10 0x7fb50c19cd10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb50c102790 0x7fb50c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 --2- 172.21.15.77:0/2659153035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c101de0 0x7fb50c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 >> 172.21.15.77:0/2659153035 conn(0x7fb50c0fd8c0 msgr2=0x7fb50c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 shutdown_connections 2024-09-15T22:26:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:26:50.600+0000 7fb5143be640 1 -- 172.21.15.77:0/2659153035 wait complete. 2024-09-15T22:26:51.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:50 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:26:51.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:50 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:26:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:50 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:26:51.397 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1fc55bd9', total seconds elapsed: 29.9 2024-09-15T22:26:51.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 16457 20.22 MB/sec warmup 52 sec latency 24180.224 ms 2024-09-15T22:26:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:51 smithi077 ceph-mon[30980]: pgmap v452: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-09-15T22:26:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:26:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:26:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:51 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:26:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:51 smithi196 ceph-mon[35109]: pgmap v452: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-09-15T22:26:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:26:52.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:26:52.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:51 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:26:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:51 smithi117 ceph-mon[35021]: pgmap v452: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-09-15T22:26:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:26:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:26:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:51 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:26:52.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 17191 20.73 MB/sec warmup 53 sec latency 46.685 ms 2024-09-15T22:26:53.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 18043 21.43 MB/sec warmup 54 sec latency 56.369 ms 2024-09-15T22:26:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:53 smithi077 ceph-mon[30980]: pgmap v453: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:53 smithi196 ceph-mon[35109]: pgmap v453: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:53 smithi117 ceph-mon[35021]: pgmap v453: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:54.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 18482 21.47 MB/sec warmup 55 sec latency 50.899 ms 2024-09-15T22:26:55.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 19016 21.45 MB/sec warmup 56 sec latency 61.458 ms 2024-09-15T22:26:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:55 smithi077 ceph-mon[30980]: pgmap v454: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:56.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:55 smithi196 ceph-mon[35109]: pgmap v454: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:56 smithi117 ceph-mon[35021]: pgmap v454: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-09-15T22:26:56.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 19543 21.54 MB/sec warmup 57 sec latency 141.227 ms 2024-09-15T22:26:57.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 20056 21.53 MB/sec warmup 58 sec latency 60.967 ms 2024-09-15T22:26:58.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:26:58 smithi077 ceph-mon[30980]: pgmap v455: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-15T22:26:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:26:58 smithi196 ceph-mon[35109]: pgmap v455: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-15T22:26:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:26:58 smithi117 ceph-mon[35021]: pgmap v455: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-09-15T22:26:58.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 20721 21.76 MB/sec warmup 59 sec latency 213.129 ms 2024-09-15T22:26:59.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 21616 22.24 MB/sec warmup 60 sec latency 78.322 ms 2024-09-15T22:27:00.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:00 smithi077 ceph-mon[30980]: pgmap v456: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-15T22:27:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:00 smithi196 ceph-mon[35109]: pgmap v456: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-15T22:27:00.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:00 smithi117 ceph-mon[35021]: pgmap v456: 129 pgs: 129 active+clean; 887 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-15T22:27:00.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 22029 22.27 MB/sec warmup 61 sec latency 329.179 ms 2024-09-15T22:27:01.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 22398 22.26 MB/sec warmup 62 sec latency 140.654 ms 2024-09-15T22:27:02.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:02 smithi077 ceph-mon[30980]: pgmap v457: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:27:02.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:02 smithi196 ceph-mon[35109]: pgmap v457: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:27:02.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:02 smithi117 ceph-mon[35021]: pgmap v457: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:27:02.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 22523 22.01 MB/sec warmup 63 sec latency 197.036 ms 2024-09-15T22:27:03.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 22732 21.90 MB/sec warmup 64 sec latency 188.453 ms 2024-09-15T22:27:04.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:04 smithi077 ceph-mon[30980]: pgmap v458: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:04.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:04 smithi196 ceph-mon[35109]: pgmap v458: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:04.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:04 smithi117 ceph-mon[35021]: pgmap v458: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:04.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 23325 22.10 MB/sec warmup 65 sec latency 218.876 ms 2024-09-15T22:27:05.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 23808 22.17 MB/sec warmup 66 sec latency 72.018 ms 2024-09-15T22:27:06.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:06 smithi077 ceph-mon[30980]: pgmap v459: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:06 smithi196 ceph-mon[35109]: pgmap v459: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:06 smithi117 ceph-mon[35021]: pgmap v459: 129 pgs: 129 active+clean; 904 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 5.9 MiB/s wr, 429 op/s 2024-09-15T22:27:06.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 24203 22.07 MB/sec warmup 67 sec latency 154.071 ms 2024-09-15T22:27:07.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:07 smithi117 ceph-mon[35021]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:27:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:07 smithi077 ceph-mon[30980]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:27:07.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 24616 21.96 MB/sec warmup 68 sec latency 170.436 ms 2024-09-15T22:27:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:07 smithi196 ceph-mon[35109]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:27:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:08 smithi117 ceph-mon[35021]: pgmap v461: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:08 smithi077 ceph-mon[30980]: pgmap v461: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:08.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 25221 22.19 MB/sec warmup 69 sec latency 164.503 ms 2024-09-15T22:27:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:08 smithi196 ceph-mon[35109]: pgmap v461: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:09.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 25912 22.38 MB/sec warmup 70 sec latency 98.286 ms 2024-09-15T22:27:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:10 smithi117 ceph-mon[35021]: pgmap v462: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:10 smithi077 ceph-mon[30980]: pgmap v462: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:10.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 26583 22.55 MB/sec warmup 71 sec latency 66.624 ms 2024-09-15T22:27:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:10 smithi196 ceph-mon[35109]: pgmap v462: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2024-09-15T22:27:11.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 27232 22.79 MB/sec warmup 72 sec latency 84.623 ms 2024-09-15T22:27:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:12 smithi117 ceph-mon[35021]: pgmap v463: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:12 smithi077 ceph-mon[30980]: pgmap v463: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:12.590 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:27:12.592 INFO:tasks.workunit.client.0.smithi077.stdout: 10 27805 22.99 MB/sec warmup 73 sec latency 80.800 ms 2024-09-15T22:27:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:12 smithi196 ceph-mon[35109]: pgmap v463: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:12.930 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:13.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 -- 172.21.15.77:0/4165081314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec103780 msgr2=0x7f98ec107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 --2- 172.21.15.77:0/4165081314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec103780 0x7f98ec107bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009900 tx=0x7f98dc02a400 comp rx=0 tx=0).stop 2024-09-15T22:27:13.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 -- 172.21.15.77:0/4165081314 shutdown_connections 2024-09-15T22:27:13.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 --2- 172.21.15.77:0/4165081314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 --2- 172.21.15.77:0/4165081314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec103780 0x7f98ec107bc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 --2- 172.21.15.77:0/4165081314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec102dd0 0x7f98ec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 -- 172.21.15.77:0/4165081314 >> 172.21.15.77:0/4165081314 conn(0x7f98ec0fc7d0 msgr2=0x7f98ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 -- 172.21.15.77:0/4165081314 shutdown_connections 2024-09-15T22:27:13.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.463+0000 7f98f4137640 1 -- 172.21.15.77:0/4165081314 wait complete. 2024-09-15T22:27:13.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.464+0000 7f98f4137640 1 Processor -- start 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.464+0000 7f98f4137640 1 -- start start 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec102dd0 0x7f98ec100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec103780 0x7f98ec100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec112070 con 0x7f98ec103780 2024-09-15T22:27:13.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec111ef0 con 0x7f98ec108300 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f4137640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1121f0 con 0x7f98ec102dd0 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43336/0 (socket says 172.21.15.77:43336) 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 -- 172.21.15.77:0/356951503 learned_addr learned my addr 172.21.15.77:0/356951503 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f16d2640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec103780 0x7f98ec100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 -- 172.21.15.77:0/356951503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec102dd0 msgr2=0x7f98ec100a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec102dd0 0x7f98ec100a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 -- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec103780 msgr2=0x7f98ec100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec103780 0x7f98ec100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 -- 172.21.15.77:0/356951503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec1aafe0 con 0x7f98ec108300 2024-09-15T22:27:13.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.465+0000 7f98f26d4640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98e400eb10 tx=0x7f98e400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.466+0000 7f98e2ffd640 1 -- 172.21.15.77:0/356951503 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4004b20 con 0x7f98ec108300 2024-09-15T22:27:13.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.466+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec1ab210 con 0x7f98ec108300 2024-09-15T22:27:13.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.466+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec1ab770 con 0x7f98ec108300 2024-09-15T22:27:13.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.466+0000 7f98e2ffd640 1 -- 172.21.15.77:0/356951503 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98e4004cc0 con 0x7f98ec108300 2024-09-15T22:27:13.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.466+0000 7f98e2ffd640 1 -- 172.21.15.77:0/356951503 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4010760 con 0x7f98ec108300 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.469+0000 7f98e2ffd640 1 -- 172.21.15.77:0/356951503 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f98e4005020 con 0x7f98ec108300 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.469+0000 7f98f6510740 1 Processor -- start 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.469+0000 7f98f6510740 1 -- start start 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc5524a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc5524ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc551a52e0 con 0x55fc5524a680 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc5514cce0 con 0x55fc5523f4a0 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f6510740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc551a12a0 con 0x55fc5523fdc0 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc5524a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43352/0 (socket says 172.21.15.77:43352) 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 -- 172.21.15.77:0/928564604 learned_addr learned my addr 172.21.15.77:0/928564604 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc5524ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 -- 172.21.15.77:0/928564604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 msgr2=0x55fc5524a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc5524a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 -- 172.21.15.77:0/928564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 msgr2=0x55fc5524ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc5524ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.470+0000 7f98f16d2640 1 -- 172.21.15.77:0/928564604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc55131440 con 0x55fc5523f4a0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98f16d2640 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000c00 tx=0x7f98dc031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955cd374ad0c6393 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98e2ffd640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 0x7f98d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98d7fff640 1 -- 172.21.15.77:0/928564604 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc035030 con 0x55fc5523f4a0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98d7fff640 1 -- 172.21.15.77:0/928564604 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc004750 con 0x55fc5523f4a0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98d7fff640 1 -- 172.21.15.77:0/928564604 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0317a0 con 0x55fc5523f4a0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 0x7f98d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.471+0000 7f98e2ffd640 1 -- 172.21.15.77:0/356951503 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f98e40d07d0 con 0x7f98ec108300 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 -- 172.21.15.77:0/928564604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 msgr2=0x55fc5523f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98dc000c00 tx=0x7f98dc031400 comp rx=0 tx=0).stop 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 0x7f98d80a58f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fc5523c890 tx=0x7f98e8005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 -- 172.21.15.77:0/928564604 shutdown_connections 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc5524ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc5524a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 --2- 172.21.15.77:0/928564604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc5523f880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 -- 172.21.15.77:0/928564604 >> 172.21.15.77:0/928564604 conn(0x55fc5523ba30 msgr2=0x55fc5523be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.472+0000 7f98f6510740 1 -- 172.21.15.77:0/928564604 shutdown_connections 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.473+0000 7f98f6510740 1 -- 172.21.15.77:0/928564604 wait complete. 2024-09-15T22:27:13.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.473+0000 7f98f6510740 1 -- start start 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.473+0000 7f98f6510740 1 Processor -- start 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55247230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55247770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fc5524d2a0 con 0x55fc5524a680 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc55160050 con 0x55fc5523f4a0 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc5524d420 con 0x55fc5523fdc0 2024-09-15T22:27:13.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43354/0 (socket says 172.21.15.77:43354) 2024-09-15T22:27:13.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 -- 172.21.15.77:0/586505267 learned_addr learned my addr 172.21.15.77:0/586505267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:13.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f6510740 10 client.0 _ll_register_callbacks cb 0x55fc551a0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:13.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f16d2640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55247230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f26d4640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55247770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 -- 172.21.15.77:0/586505267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 msgr2=0x55fc55247230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55247230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 -- 172.21.15.77:0/586505267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 msgr2=0x55fc55247770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55247770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.474+0000 7f98f1ed3640 1 -- 172.21.15.77:0/586505267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc54f163c0 con 0x55fc5523f4a0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.475+0000 7f98f16d2640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55247230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.475+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98e80128a0 tx=0x7f98e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6862c56cb7bcc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.475+0000 7f98d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800fc60 con 0x55fc5523f4a0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f98e800fe00 con 0x55fc5523f4a0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc004750 con 0x55fc5523f4a0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98e8030050 con 0x55fc5523f4a0 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 10 client.27977 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:13.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc55258cb0 con 0x55fc5523f4a0 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc004750 con 0x55fc5523f4a0 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f98e8027520 con 0x55fc5523f4a0 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 10 client.27977 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 10 client.27977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 20 client.27977 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc5525a1c0 0x55fc5525c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 4 client.27977 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:13.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.476+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc551b2850 con 0x55fc5525a1c0 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.477+0000 7f98f16d2640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc5525a1c0 0x55fc5525c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.477+0000 7f98d57fa640 10 client.27977 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.477+0000 7f98f16d2640 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc5525a1c0 0x55fc5525c660 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3aa27caa1ad7b server_cookie=d6130c41a0cb408 in_seq=0 out_seq=0 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.478+0000 7f98d57fa640 1 -- 172.21.15.77:0/586505267 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc551b2850 con 0x55fc5525a1c0 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.478+0000 7f98d57fa640 10 client.27977 handle_command_reply: tid=0 2024-09-15T22:27:13.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.478+0000 7f98f6510740 1 client.27977 shutdown 2024-09-15T22:27:13.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 msgr2=0x55fc55249f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98e80128a0 tx=0x7f98e8004730 comp rx=0 tx=0).stop 2024-09-15T22:27:13.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 shutdown_connections 2024-09-15T22:27:13.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc5525a1c0 0x55fc5525c660 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55247770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55247230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 --2- 172.21.15.77:0/586505267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55249f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 >> 172.21.15.77:0/586505267 conn(0x55fc5523ba30 msgr2=0x55fc5523ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98d57fa640 0 client.27977 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:13.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 shutdown_connections 2024-09-15T22:27:13.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 1 -- 172.21.15.77:0/586505267 wait complete. 2024-09-15T22:27:13.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.479+0000 7f98f6510740 20 client.27977 trim_cache size 0 max 16384 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.517+0000 7f98f6510740 1 Processor -- start 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.517+0000 7f98f6510740 1 -- start start 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 0x55fc55256560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 0x55fc55241600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5524a680 0x55fc55261f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98d000ffb0 con 0x55fc5523f4a0 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0011bb0 con 0x55fc5523fdc0 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f6510740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0010680 con 0x55fc5524a680 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 0x55fc55256560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 0x55fc55256560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48186/0 (socket says 172.21.15.77:48186) 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f1ed3640 1 -- 172.21.15.77:0/3501844204 learned_addr learned my addr 172.21.15.77:0/3501844204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 0x55fc55241600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f26d4640 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5524a680 0x55fc55261f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 -- 172.21.15.77:0/3501844204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5524a680 msgr2=0x55fc55261f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5524a680 0x55fc55261f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 -- 172.21.15.77:0/3501844204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 msgr2=0x55fc55256560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 0x55fc55256560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.518+0000 7f98f16d2640 1 -- 172.21.15.77:0/3501844204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5524d130 con 0x55fc5523fdc0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.519+0000 7f98f16d2640 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 0x55fc55241600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55fc55248870 tx=0x7f98dc0042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84375b448bb6c277 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.519+0000 7f98d77fe640 1 -- 172.21.15.77:0/3501844204 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc035070 con 0x55fc5523fdc0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.519+0000 7f98d77fe640 1 -- 172.21.15.77:0/3501844204 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc054030 con 0x55fc5523fdc0 2024-09-15T22:27:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.519+0000 7f98d77fe640 1 -- 172.21.15.77:0/3501844204 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03d6b0 con 0x55fc5523fdc0 2024-09-15T22:27:13.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 -- 172.21.15.77:0/3501844204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 msgr2=0x55fc55241600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 0x55fc55241600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55fc55248870 tx=0x7f98dc0042c0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 -- 172.21.15.77:0/3501844204 shutdown_connections 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5524a680 0x55fc55261f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523fdc0 0x55fc55241600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 --2- 172.21.15.77:0/3501844204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5523f4a0 0x55fc55256560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 -- 172.21.15.77:0/3501844204 >> 172.21.15.77:0/3501844204 conn(0x55fc5523ba30 msgr2=0x55fc55240ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 -- 172.21.15.77:0/3501844204 shutdown_connections 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.520+0000 7f98f6510740 1 -- 172.21.15.77:0/3501844204 wait complete. 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 -- start start 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 Processor -- start 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55244880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55244dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0011420 con 0x55fc5524a680 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0010d50 con 0x55fc5523f4a0 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f6510740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98d000dfb0 con 0x55fc5523fdc0 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.521+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f16d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55244880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f6510740 10 client.0 _ll_register_callbacks cb 0x7f98e80093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43364/0 (socket says 172.21.15.77:43364) 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 -- 172.21.15.77:0/2041278803 learned_addr learned my addr 172.21.15.77:0/2041278803 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f26d4640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55244dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 -- 172.21.15.77:0/2041278803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 msgr2=0x55fc55244880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55244880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 -- 172.21.15.77:0/2041278803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 msgr2=0x55fc55244dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55244dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 -- 172.21.15.77:0/2041278803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc552c3ff0 con 0x55fc5523f4a0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f26d4640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55244dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98f1ed3640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98e8027ab0 tx=0x7f98e8027de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c23d44a97fe2d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8012e20 con 0x55fc5523f4a0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.522+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9526+0+0 (secure 0 0 0) 0x7f98e8027610 con 0x55fc5523f4a0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc054030 con 0x55fc5523f4a0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98e8046050 con 0x55fc5523f4a0 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 10 client.27983 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc552c52e0 con 0x55fc5523f4a0 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc054030 con 0x55fc5523f4a0 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f98e801faf0 con 0x55fc5523f4a0 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 10 client.27983 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 10 client.27983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 20 client.27983 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.523+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc552c5a60 0x55fc552c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.524+0000 7f98f6510740 4 client.27983 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.524+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc551acd60 con 0x55fc552c5a60 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.524+0000 7f98f16d2640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc552c5a60 0x55fc552c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.524+0000 7f98f16d2640 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc552c5a60 0x55fc552c7e80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9dd0ca4577524d server_cookie=fab9f42121b18981 in_seq=0 out_seq=0 2024-09-15T22:27:13.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.524+0000 7f98d57fa640 10 client.27983 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:13.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98d57fa640 1 -- 172.21.15.77:0/2041278803 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc551acd60 con 0x55fc552c5a60 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98d57fa640 10 client.27983 handle_command_reply: tid=0 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 client.27983 shutdown 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 msgr2=0x55fc55244340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98e8027ab0 tx=0x7f98e8027de0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 shutdown_connections 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55fc552c5a60 0x55fc552c7e80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fc5524a680 0x55fc55244dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98d57fa640 0 client.27983 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc5523fdc0 0x55fc55244880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 --2- 172.21.15.77:0/2041278803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5523f4a0 0x55fc55244340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.526+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 >> 172.21.15.77:0/2041278803 conn(0x55fc5523ba30 msgr2=0x55fc55241710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.527+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 shutdown_connections 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.527+0000 7f98f6510740 1 -- 172.21.15.77:0/2041278803 wait complete. 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.527+0000 7f98f6510740 20 client.27983 trim_cache size 0 max 16384 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:27:13.534 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.527+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 msgr2=0x7f98d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.527+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 0x7f98d80a58f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fc5523c890 tx=0x7f98e8005f90 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 msgr2=0x7f98ec1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98e400eb10 tx=0x7f98e400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 shutdown_connections 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98d80a3430 0x7f98d80a58f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ec108300 0x7f98ec1014b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98ec103780 0x7f98ec100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 --2- 172.21.15.77:0/356951503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98ec102dd0 0x7f98ec100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:13.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 >> 172.21.15.77:0/356951503 conn(0x7f98ec0fc7d0 msgr2=0x7f98ec105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:13.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 shutdown_connections 2024-09-15T22:27:13.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:13.528+0000 7f98f4137640 1 -- 172.21.15.77:0/356951503 wait complete. 2024-09-15T22:27:13.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 28355 23.10 MB/sec warmup 74 sec latency 78.800 ms 2024-09-15T22:27:14.330 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:14 smithi077 ceph-mon[30980]: pgmap v464: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:14.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:14 smithi077 ceph-mon[30980]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:27:14.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:14 smithi117 ceph-mon[35021]: pgmap v464: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:14.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:14 smithi117 ceph-mon[35021]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:27:14.367 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:27:14.367 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:27:14.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 28802 23.06 MB/sec warmup 75 sec latency 79.994 ms 2024-09-15T22:27:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:14 smithi196 ceph-mon[35109]: pgmap v464: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.3 MiB/s wr, 613 op/s 2024-09-15T22:27:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:14 smithi196 ceph-mon[35109]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:27:14.718 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3588148202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 msgr2=0x7f7cf810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3588148202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf810ec40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009cd0 tx=0x7f7cf002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3588148202 shutdown_connections 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3588148202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf810ec40 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3588148202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf81036e0 0x7f7cf8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3588148202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf8102d30 0x7f7cf8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3588148202 >> 172.21.15.77:0/3588148202 conn(0x7f7cf80fc7d0 msgr2=0x7f7cf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3588148202 shutdown_connections 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3588148202 wait complete. 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.261+0000 7f7cffdbe640 1 Processor -- start 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 -- start start 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 0x7f7cf8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 0x7f7cf8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:15.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81120a0 con 0x7f7cf81082a0 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8111f20 con 0x7f7cf8102d30 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cffdbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8112220 con 0x7f7cf81036e0 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfd359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 0x7f7cf8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfdb5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 0x7f7cf8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48214/0 (socket says 172.21.15.77:48214) 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfdb5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 0x7f7cf8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43382/0 (socket says 172.21.15.77:43382) 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 -- 172.21.15.77:0/3464626168 learned_addr learned my addr 172.21.15.77:0/3464626168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfd359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 0x7f7cf8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47924/0 (socket says 172.21.15.77:47924) 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 -- 172.21.15.77:0/3464626168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 msgr2=0x7f7cf8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:15.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 0x7f7cf8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 -- 172.21.15.77:0/3464626168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 msgr2=0x7f7cf8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 0x7f7cf8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 -- 172.21.15.77:0/3464626168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf819d520 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.262+0000 7f7cfe35b640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0005cc0 tx=0x7f7cf00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.263+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003e00 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.263+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0004520 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.263+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf819d7b0 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.263+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0030db0 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.263+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81a50f0 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.264+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf8103110 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.265+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7cf00046d0 con 0x7f7cf81082a0 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.266+0000 7f7ceeffd640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 0x7f7ce40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:15.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.266+0000 7f7cfdb5a640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 0x7f7ce40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.266+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f7cf002e030 con 0x7f7cf81082a0 2024-09-15T22:27:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.266+0000 7f7cfdb5a640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 0x7f7ce40a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009bd0 tx=0x7f7cf4005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:15.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.269+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cf004b060 con 0x7f7cf81082a0 2024-09-15T22:27:15.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.541+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cf810a2b0 con 0x7f7cf81082a0 2024-09-15T22:27:15.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.542+0000 7f7ceeffd640 1 -- 172.21.15.77:0/3464626168 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7f7cf00bdec0 con 0x7f7cf81082a0 2024-09-15T22:27:15.543 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:27:15.544 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:27:15.544 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:27:15.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 msgr2=0x7f7ce40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 0x7f7ce40a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009bd0 tx=0x7f7cf4005f90 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 msgr2=0x7f7cf819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0005cc0 tx=0x7f7cf00093d0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 shutdown_connections 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7ce40a33b0 0x7f7ce40a5870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cf81082a0 0x7f7cf819ce40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cf81036e0 0x7f7cf8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 --2- 172.21.15.77:0/3464626168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf8102d30 0x7f7cf8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 >> 172.21.15.77:0/3464626168 conn(0x7f7cf80fc7d0 msgr2=0x7f7cf8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 shutdown_connections 2024-09-15T22:27:15.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:15.545+0000 7f7cffdbe640 1 -- 172.21.15.77:0/3464626168 wait complete. 2024-09-15T22:27:15.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 29205 22.90 MB/sec warmup 76 sec latency 118.177 ms 2024-09-15T22:27:16.250 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:27:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:16 smithi117 ceph-mon[35021]: pgmap v466: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-09-15T22:27:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:16 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3464626168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:16 smithi077 ceph-mon[30980]: pgmap v466: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-09-15T22:27:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:16 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3464626168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:16.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 29875 23.10 MB/sec warmup 77 sec latency 71.921 ms 2024-09-15T22:27:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:16 smithi196 ceph-mon[35109]: pgmap v466: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-09-15T22:27:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:16 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3464626168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:16.609 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:17.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 -- 172.21.15.77:0/481133411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4100e20 msgr2=0x7f36a4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/481133411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4100e20 0x7f36a4101200 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009900 tx=0x7f36a002a480 comp rx=0 tx=0).stop 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 -- 172.21.15.77:0/481133411 shutdown_connections 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/481133411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 0x7f36a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/481133411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4102570 0x7f36a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/481133411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4100e20 0x7f36a4101200 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.193+0000 7f36ac6cf640 1 -- 172.21.15.77:0/481133411 >> 172.21.15.77:0/481133411 conn(0x7f36a40fc7d0 msgr2=0x7f36a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.194+0000 7f36ac6cf640 1 -- 172.21.15.77:0/481133411 shutdown_connections 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.194+0000 7f36ac6cf640 1 -- 172.21.15.77:0/481133411 wait complete. 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.195+0000 7f36ac6cf640 1 Processor -- start 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.195+0000 7f36ac6cf640 1 -- start start 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 0x7f36a41a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 0x7f36a41a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36a9c6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36a9c6a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48242/0 (socket says 172.21.15.77:48242) 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 0x7f36a41a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43402/0 (socket says 172.21.15.77:43402) 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36a9c6a640 1 -- 172.21.15.77:0/508958883 learned_addr learned my addr 172.21.15.77:0/508958883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 0x7f36a419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4110f50 con 0x7f36a4102570 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4110dd0 con 0x7f36a4100e20 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41110d0 con 0x7f36a4102ef0 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.196+0000 7f36aac6c640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 0x7f36a419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 -- 172.21.15.77:0/508958883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 msgr2=0x7f36a419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 0x7f36a419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 -- 172.21.15.77:0/508958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 msgr2=0x7f36a41a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 0x7f36a41a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a419f4d0 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.197+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f369400e9f0 tx=0x7f369400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.198+0000 7f369b7fe640 1 -- 172.21.15.77:0/508958883 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400cd80 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.198+0000 7f369b7fe640 1 -- 172.21.15.77:0/508958883 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3694010910 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.198+0000 7f369b7fe640 1 -- 172.21.15.77:0/508958883 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694018990 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.198+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a419f7c0 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.198+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a41ab630 con 0x7f36a4102570 2024-09-15T22:27:17.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.202+0000 7f36aeaa8740 1 Processor -- start 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 -- start start 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f369b7fe640 1 -- 172.21.15.77:0/508958883 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3694010ac0 con 0x7f36a4102570 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56516ed692f0 con 0x56516ee0e370 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56516ed10cf0 con 0x56516ee04b40 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56516ed652b0 con 0x56516ee035d0 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aac6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aac6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43410/0 (socket says 172.21.15.77:43410) 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.203+0000 7f36aac6c640 1 -- 172.21.15.77:0/513332846 learned_addr learned my addr 172.21.15.77:0/513332846 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.204+0000 7f36aa46b640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.205+0000 7f369b7fe640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 0x7f36900a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f369b7fe640 1 -- 172.21.15.77:0/508958883 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f36940d0310 con 0x7f36a4102570 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36aac6c640 1 -- 172.21.15.77:0/513332846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 msgr2=0x56516ee03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36aac6c640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36aac6c640 1 -- 172.21.15.77:0/513332846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 msgr2=0x56516ee10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36aac6c640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36aac6c640 1 -- 172.21.15.77:0/513332846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516ecf53a0 con 0x56516ee04b40 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.206+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.207+0000 7f36aac6c640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 0x7f36900a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.208+0000 7f36aac6c640 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f369c009f50 tx=0x7f369c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f6bef9710faa83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.209+0000 7f3698ff9640 1 -- 172.21.15.77:0/513332846 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c004040 con 0x56516ee04b40 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.209+0000 7f3698ff9640 1 -- 172.21.15.77:0/513332846 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c0046c0 con 0x56516ee04b40 2024-09-15T22:27:17.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.209+0000 7f3698ff9640 1 -- 172.21.15.77:0/513332846 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c00f480 con 0x56516ee04b40 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.209+0000 7f36aac6c640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 0x7f36900a59c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f369c021000 tx=0x7f369c00fb30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.210+0000 7f36aeaa8740 1 -- 172.21.15.77:0/513332846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 msgr2=0x56516ee03090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.210+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f369c009f50 tx=0x7f369c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 -- 172.21.15.77:0/513332846 shutdown_connections 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/513332846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee03090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 -- 172.21.15.77:0/513332846 >> 172.21.15.77:0/513332846 conn(0x56516edcc310 msgr2=0x56516edffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 -- 172.21.15.77:0/513332846 shutdown_connections 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.212+0000 7f36aeaa8740 1 -- 172.21.15.77:0/513332846 wait complete. 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.213+0000 7f36aeaa8740 1 -- start start 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.213+0000 7f36aeaa8740 1 Processor -- start 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.214+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.214+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 0x56516ee0b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.214+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56516ee11070 con 0x56516ee04b40 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56516ed24060 con 0x56516ee0e370 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56516ee111f0 con 0x56516ee035d0 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aeaa8740 10 client.0 _ll_register_callbacks cb 0x56516ea778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aac6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aac6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43422/0 (socket says 172.21.15.77:43422) 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aac6c640 1 -- 172.21.15.77:0/3376940187 learned_addr learned my addr 172.21.15.77:0/3376940187 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 0x56516ee0b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aa46b640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 -- 172.21.15.77:0/3376940187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 msgr2=0x56516ee0dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 -- 172.21.15.77:0/3376940187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 msgr2=0x56516ee0b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36a9c6a640 1 -- 172.21.15.77:0/3376940187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516eada3c0 con 0x56516ee04b40 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aac6c640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.215+0000 7f36aa46b640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.216+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 0x56516ee0b000 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f36940cd670 tx=0x7f369409c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7be0edebea21cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.216+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.216+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940cdb00 con 0x56516ee04b40 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.216+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f36940a0160 con 0x56516ee04b40 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.217+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c0046c0 con 0x56516ee04b40 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.218+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36940bc2c0 con 0x56516ee04b40 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.218+0000 7f36aeaa8740 10 client.19083 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:17.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.218+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56516ee1ca60 con 0x56516ee04b40 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.218+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c0046c0 con 0x56516ee04b40 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f369409b700 con 0x56516ee04b40 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 10 client.19083 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 10 client.19083 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 20 client.19083 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee1df20 0x56516ee20310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 4 client.19083 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.219+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56516ed76860 con 0x56516ee1df20 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.220+0000 7f36aa46b640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee1df20 0x56516ee20310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.220+0000 7f36aa46b640 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee1df20 0x56516ee20310 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c10fe4aef0a0bc76 server_cookie=1d0d53aab7ab0fe2 in_seq=0 out_seq=0 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.220+0000 7f368dffb640 10 client.19083 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f368dffb640 1 -- 172.21.15.77:0/3376940187 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56516ed76860 con 0x56516ee1df20 2024-09-15T22:27:17.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f368dffb640 10 client.19083 handle_command_reply: tid=0 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 client.19083 shutdown 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 msgr2=0x56516ee0b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 0x56516ee0b000 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f36940cd670 tx=0x7f369409c2b0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 shutdown_connections 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee1df20 0x56516ee20310 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee0e370 0x56516ee0b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee04b40 0x56516ee0b000 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3376940187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 >> 172.21.15.77:0/3376940187 conn(0x56516edcc310 msgr2=0x56516ee05960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f368dffb640 0 client.19083 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 shutdown_connections 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3376940187 wait complete. 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.221+0000 7f36aeaa8740 20 client.19083 trim_cache size 0 max 16384 2024-09-15T22:27:17.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.236+0000 7f36aeaa8740 1 Processor -- start 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.236+0000 7f36aeaa8740 1 -- start start 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.236+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee1a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee25b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41ad010 con 0x56516ee0e370 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41aec10 con 0x56516ee04b40 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aeaa8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41ad6e0 con 0x56516ee035d0 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47974/0 (socket says 172.21.15.77:47974) 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aa46b640 1 -- 172.21.15.77:0/3789610268 learned_addr learned my addr 172.21.15.77:0/3789610268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee25b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee1a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 -- 172.21.15.77:0/3789610268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 msgr2=0x56516ee0afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 -- 172.21.15.77:0/3789610268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 msgr2=0x56516ee1a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee1a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 -- 172.21.15.77:0/3789610268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516ee03be0 con 0x56516ee0e370 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee1a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aa46b640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.237+0000 7f36aac6c640 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee25b20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f369c01aa90 tx=0x7f369c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a1e8d0c6f1fe45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.238+0000 7f368ffff640 1 -- 172.21.15.77:0/3789610268 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c003bf0 con 0x56516ee0e370 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.238+0000 7f368ffff640 1 -- 172.21.15.77:0/3789610268 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c00f480 con 0x56516ee0e370 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.238+0000 7f368ffff640 1 -- 172.21.15.77:0/3789610268 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c0198d0 con 0x56516ee0e370 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3789610268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 msgr2=0x56516ee25b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee25b20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f369c01aa90 tx=0x7f369c01c000 comp rx=0 tx=0).stop 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3789610268 shutdown_connections 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee0e370 0x56516ee25b20 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee1a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/3789610268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee035d0 0x56516ee0afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3789610268 >> 172.21.15.77:0/3789610268 conn(0x56516edcc310 msgr2=0x56516ee0a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3789610268 shutdown_connections 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- 172.21.15.77:0/3789610268 wait complete. 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 -- start start 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.239+0000 7f36aeaa8740 1 Processor -- start 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee0e370 0x56516ee12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48288/0 (socket says 172.21.15.77:48288) 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36a9c6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36a9c6a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43440/0 (socket says 172.21.15.77:43440) 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 -- 172.21.15.77:0/2077200181 learned_addr learned my addr 172.21.15.77:0/2077200181 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:17.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41ae480 con 0x56516ee035d0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41addb0 con 0x56516ee04b40 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4101240 con 0x56516ee0e370 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aac6c640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee0e370 0x56516ee12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aeaa8740 10 client.0 _ll_register_callbacks cb 0x7f3694098550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 -- 172.21.15.77:0/2077200181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee0e370 msgr2=0x56516ee12be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee0e370 0x56516ee12be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 -- 172.21.15.77:0/2077200181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 msgr2=0x56516ee126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.240+0000 7f36aa46b640 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56516ee87af0 con 0x56516ee035d0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.241+0000 7f36aa46b640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005940 tx=0x7f36a00043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18adc8988fe2608c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.241+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.241+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0035040 con 0x56516ee035d0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.241+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f36a0003a00 con 0x56516ee035d0 2024-09-15T22:27:17.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.241+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c00f480 con 0x56516ee035d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36a0065050 con 0x56516ee035d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 10 client.19089 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56516ee88d50 con 0x56516ee035d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a0003c00 con 0x56516ee035d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f36a00563a0 con 0x56516ee035d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 10 client.19089 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 10 client.19089 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 20 client.19089 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee894d0 0x56516ee8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 4 client.19089 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.242+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f369c00f480 con 0x56516ee894d0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.243+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee894d0 0x56516ee8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:17.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.243+0000 7f368dffb640 10 client.19089 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.243+0000 7f36a9c6a640 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee894d0 0x56516ee8b8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=444331d01deb22be server_cookie=a0f5568c12375ad in_seq=0 out_seq=0 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f368dffb640 1 -- 172.21.15.77:0/2077200181 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f369c00f480 con 0x56516ee894d0 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f368dffb640 10 client.19089 handle_command_reply: tid=0 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 client.19089 shutdown 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 msgr2=0x56516ee12160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005940 tx=0x7f36a00043d0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 shutdown_connections 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56516ee894d0 0x56516ee8b8f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56516ee0e370 0x56516ee12be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56516ee04b40 0x56516ee126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f368dffb640 0 client.19089 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 --2- 172.21.15.77:0/2077200181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56516ee035d0 0x56516ee12160 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 >> 172.21.15.77:0/2077200181 conn(0x56516edcc310 msgr2=0x56516edfe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 shutdown_connections 2024-09-15T22:27:17.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 1 -- 172.21.15.77:0/2077200181 wait complete. 2024-09-15T22:27:17.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.244+0000 7f36aeaa8740 20 client.19089 trim_cache size 0 max 16384 2024-09-15T22:27:17.294 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 msgr2=0x7f36900a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 0x7f36900a59c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f369c021000 tx=0x7f369c00fb30 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 msgr2=0x7f36a41a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f369400e9f0 tx=0x7f369400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 shutdown_connections 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36900a3500 0x7f36900a59c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a4102ef0 0x7f36a419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36a4102570 0x7f36a41a4d60 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 --2- 172.21.15.77:0/508958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a4100e20 0x7f36a41a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:17.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 >> 172.21.15.77:0/508958883 conn(0x7f36a40fc7d0 msgr2=0x7f36a406a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:17.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 shutdown_connections 2024-09-15T22:27:17.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:17.245+0000 7f36ac6cf640 1 -- 172.21.15.77:0/508958883 wait complete. 2024-09-15T22:27:17.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 30676 23.35 MB/sec warmup 78 sec latency 60.529 ms 2024-09-15T22:27:18.079 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:27:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:18 smithi077 ceph-mon[30980]: pgmap v467: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:18.379 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:18.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:18 smithi196 ceph-mon[35109]: pgmap v467: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:18.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 31355 23.62 MB/sec warmup 79 sec latency 46.563 ms 2024-09-15T22:27:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:18 smithi117 ceph-mon[35021]: pgmap v467: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:19.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 -- 172.21.15.77:0/3201202269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 msgr2=0x7ff5e811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 --2- 172.21.15.77:0/3201202269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e811d530 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc01a910 tx=0x7ff5dc03b430 comp rx=0 tx=0).stop 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 -- 172.21.15.77:0/3201202269 shutdown_connections 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 --2- 172.21.15.77:0/3201202269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e811d530 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 --2- 172.21.15.77:0/3201202269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109830 0x7ff5e8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 --2- 172.21.15.77:0/3201202269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109450 0x7ff5e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 -- 172.21.15.77:0/3201202269 >> 172.21.15.77:0/3201202269 conn(0x7ff5e806e140 msgr2=0x7ff5e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 -- 172.21.15.77:0/3201202269 shutdown_connections 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.910+0000 7ff5ed654640 1 -- 172.21.15.77:0/3201202269 wait complete. 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 Processor -- start 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 -- start start 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109450 0x7ff5e811a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109830 0x7ff5e81167f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8113320 con 0x7ff5e811b140 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81131a0 con 0x7ff5e8109450 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.911+0000 7ff5ed654640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81134a0 con 0x7ff5e8109830 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48310/0 (socket says 172.21.15.77:48310) 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1170671928 learned_addr learned my addr 172.21.15.77:0/1170671928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109450 0x7ff5e811a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1170671928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109830 msgr2=0x7ff5e81167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109830 0x7ff5e81167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1170671928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109450 msgr2=0x7ff5e811a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109450 0x7ff5e811a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.912+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1170671928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e81172f0 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc03b9a0 tx=0x7ff5dc040cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5cffff640 1 -- 172.21.15.77:0/1170671928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc046070 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5ed654640 1 -- 172.21.15.77:0/1170671928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e8117580 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5ed654640 1 -- 172.21.15.77:0/1170671928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81cd460 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5cffff640 1 -- 172.21.15.77:0/1170671928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc004ba0 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.913+0000 7ff5cffff640 1 -- 172.21.15.77:0/1170671928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc003b30 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.918+0000 7ff5efa2d740 1 Processor -- start 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.919+0000 7ff5cffff640 1 -- 172.21.15.77:0/1170671928 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff5dc049070 con 0x7ff5e811b140 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.919+0000 7ff5efa2d740 1 -- start start 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b68d3462f0 con 0x55b68d3e0780 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b68d2edcf0 con 0x55b68d3e1b10 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b68d3422b0 con 0x55b68d3eb520 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48324/0 (socket says 172.21.15.77:48324) 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.920+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/1486904766 learned_addr learned my addr 172.21.15.77:0/1486904766 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/1486904766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 msgr2=0x55b68d3ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/1486904766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 msgr2=0x55b68d3e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/1486904766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d2d23a0 con 0x55b68d3e0780 2024-09-15T22:27:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5cffff640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 0x7ff5d00a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5e67fc640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 0x7ff5d00a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.921+0000 7ff5cffff640 1 -- 172.21.15.77:0/1170671928 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7ff5dc0f1010 con 0x7ff5e811b140 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.922+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800bcb0 tx=0x7ff5d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e10c0f9c3d28ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.922+0000 7ff5ccff9640 1 -- 172.21.15.77:0/1486904766 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8015070 con 0x55b68d3e0780 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.922+0000 7ff5ccff9640 1 -- 172.21.15.77:0/1486904766 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d801a940 con 0x55b68d3e0780 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.923+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1486904766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 msgr2=0x55b68d3e0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.923+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800bcb0 tx=0x7ff5d800f730 comp rx=0 tx=0).stop 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.923+0000 7ff5e67fc640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 0x7ff5d00a59b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55b68d3e1220 tx=0x7ff5e00018b0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1486904766 shutdown_connections 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3e0bc0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1486904766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1486904766 >> 172.21.15.77:0/1486904766 conn(0x55b68d3a9310 msgr2=0x55b68d3dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1486904766 shutdown_connections 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.924+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1486904766 wait complete. 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 -- start start 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 Processor -- start 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3e0780 0x55b68d3eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b68d3ee090 con 0x55b68d3eb520 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b68d301060 con 0x55b68d3e1b10 2024-09-15T22:27:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b68d3ee210 con 0x55b68d3e0780 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.925+0000 7ff5efa2d740 10 client.0 _ll_register_callbacks cb 0x55b68d0548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.927+0000 7ff5e6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3e0780 0x55b68d3eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.927+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.927+0000 7ff5e77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.927+0000 7ff5e77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48336/0 (socket says 172.21.15.77:48336) 2024-09-15T22:27:19.276 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.927+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1116572360 learned_addr learned my addr 172.21.15.77:0/1116572360 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.928+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3e0780 msgr2=0x55b68d3eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.928+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3e0780 0x55b68d3eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.928+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1116572360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 msgr2=0x55b68d3e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.928+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.928+0000 7ff5e77fe640 1 -- 172.21.15.77:0/1116572360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d0b73c0 con 0x55b68d3eb520 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.932+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005ab0 tx=0x7ff5dc0bcb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00ce9429f1bca35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.933+0000 7ff5caff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.933+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01a100 con 0x55b68d3eb520 2024-09-15T22:27:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.933+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7ff5dc003cd0 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.933+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d801a940 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.934+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff5dc0bd600 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.934+0000 7ff5efa2d740 10 client.19101 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.934+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b68d3f9af0 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d801a940 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7ff5dc0ee5f0 con 0x55b68d3eb520 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 10 client.19101 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 10 client.19101 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 20 client.19101 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d3faf50 0x55b68d3fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 4 client.19101 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b68d353860 con 0x55b68d3faf50 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.935+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d3faf50 0x55b68d3fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.936+0000 7ff5caff5640 10 client.19101 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.938+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d3faf50 0x55b68d3fd3b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0970103712e29e7 server_cookie=52bb9f0d6ca13ddd in_seq=0 out_seq=0 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.939+0000 7ff5caff5640 1 -- 172.21.15.77:0/1116572360 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b68d353860 con 0x55b68d3faf50 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.939+0000 7ff5caff5640 10 client.19101 handle_command_reply: tid=0 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.939+0000 7ff5efa2d740 1 client.19101 shutdown 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 msgr2=0x55b68d3e86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005ab0 tx=0x7ff5dc0bcb00 comp rx=0 tx=0).stop 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 shutdown_connections 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d3faf50 0x55b68d3fd3b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3eb520 0x55b68d3e86a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d3e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/1116572360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3e0780 0x55b68d3eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 >> 172.21.15.77:0/1116572360 conn(0x55b68d3a9310 msgr2=0x55b68d3e2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5caff5640 0 client.19101 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 shutdown_connections 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 1 -- 172.21.15.77:0/1116572360 wait complete. 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.940+0000 7ff5efa2d740 20 client.19101 trim_cache size 0 max 16384 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.960+0000 7ff5efa2d740 1 Processor -- start 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.960+0000 7ff5efa2d740 1 -- start start 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d402cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d403200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d76b0 con 0x55b68d3e0780 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d92b0 con 0x55b68d3e1b10 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d7d80 con 0x55b68d3eb520 2024-09-15T22:27:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d402cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48348/0 (socket says 172.21.15.77:48348) 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d402cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:43468/0 (socket says 172.21.15.77:43468) 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/2263108518 learned_addr learned my addr 172.21.15.77:0/2263108518 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d403200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/2263108518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 msgr2=0x55b68d403200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d403200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/2263108518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 msgr2=0x55b68d402cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d402cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 -- 172.21.15.77:0/2263108518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68d3b6800 con 0x55b68d3e0780 2024-09-15T22:27:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.961+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800bbd0 tx=0x7ff5d800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c25fd28d38afdb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5cc7f8640 1 -- 172.21.15.77:0/2263108518 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8015050 con 0x55b68d3e0780 2024-09-15T22:27:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5cc7f8640 1 -- 172.21.15.77:0/2263108518 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d8005520 con 0x55b68d3e0780 2024-09-15T22:27:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5cc7f8640 1 -- 172.21.15.77:0/2263108518 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80048e0 con 0x55b68d3e0780 2024-09-15T22:27:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2263108518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 msgr2=0x55b68d3f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800bbd0 tx=0x7ff5d800b5a0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2263108518 shutdown_connections 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d403200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e1b10 0x55b68d402cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2263108518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e0780 0x55b68d3f7380 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.962+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2263108518 >> 172.21.15.77:0/2263108518 conn(0x55b68d3a9310 msgr2=0x55b68d3dc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2263108518 shutdown_connections 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2263108518 wait complete. 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- start start 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 Processor -- start 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e0780 0x55b68d3f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d8b20 con 0x55b68d3e1b10 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d8450 con 0x55b68d3e0780 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81d56b0 con 0x55b68d3eb520 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5efa2d740 10 client.0 _ll_register_callbacks cb 0x7ff5dc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48350/0 (socket says 172.21.15.77:48350) 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5e67fc640 1 -- 172.21.15.77:0/2850225443 learned_addr learned my addr 172.21.15.77:0/2850225443 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e0780 0x55b68d3f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.963+0000 7ff5e77fe640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 -- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 msgr2=0x55b68d3f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 -- 172.21.15.77:0/2850225443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e0780 msgr2=0x55b68d3f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e0780 0x55b68d3f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 -- 172.21.15.77:0/2850225443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68d3f4170 con 0x55b68d3e1b10 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5e67fc640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8117fb0 tx=0x7ff5e00012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e0bf1216c02577 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5ca7f4640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0027070 con 0x55b68d3e1b10 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7ff5e0023440 con 0x55b68d3e1b10 2024-09-15T22:27:19.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d8005520 con 0x55b68d3e1b10 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff5e0013e60 con 0x55b68d3e1b10 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5efa2d740 10 client.19107 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.964+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b68d466330 con 0x55b68d3e1b10 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d8005520 con 0x55b68d3e1b10 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7ff5e002b7e0 con 0x55b68d3e1b10 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 10 client.19107 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:19.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 10 client.19107 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 20 client.19107 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d466b10 0x55b68d468f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 4 client.19107 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b68d34dd70 con 0x55b68d466b10 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d466b10 0x55b68d468f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5ca7f4640 10 client.19107 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.965+0000 7ff5e6ffd640 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d466b10 0x55b68d468f30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9394356f8034743 server_cookie=b2802d6030480209 in_seq=0 out_seq=0 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5ca7f4640 1 -- 172.21.15.77:0/2850225443 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b68d34dd70 con 0x55b68d466b10 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5ca7f4640 10 client.19107 handle_command_reply: tid=0 2024-09-15T22:27:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 client.19107 shutdown 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 msgr2=0x55b68d3f3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8117fb0 tx=0x7ff5e00012e0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 shutdown_connections 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b68d466b10 0x55b68d468f30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5ca7f4640 0 client.19107 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b68d3eb520 0x55b68d3f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b68d3e1b10 0x55b68d3f3a30 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 --2- 172.21.15.77:0/2850225443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68d3e0780 0x55b68d3f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 >> 172.21.15.77:0/2850225443 conn(0x55b68d3a9310 msgr2=0x55b68d3db980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 shutdown_connections 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 1 -- 172.21.15.77:0/2850225443 wait complete. 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.966+0000 7ff5efa2d740 20 client.19107 trim_cache size 0 max 16384 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 msgr2=0x7ff5d00a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 0x7ff5d00a59b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55b68d3e1220 tx=0x7ff5e00018b0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 msgr2=0x7ff5e8116d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc03b9a0 tx=0x7ff5dc040cd0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 shutdown_connections 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ff5d00a34f0 0x7ff5d00a59b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5e811b140 0x7ff5e8116d30 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5e8109830 0x7ff5e81167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 --2- 172.21.15.77:0/1170671928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5e8109450 0x7ff5e811a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 >> 172.21.15.77:0/1170671928 conn(0x7ff5e806e140 msgr2=0x7ff5e8070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 shutdown_connections 2024-09-15T22:27:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:18.967+0000 7ff5cdffb640 1 -- 172.21.15.77:0/1170671928 wait complete. 2024-09-15T22:27:19.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 31851 23.66 MB/sec warmup 80 sec latency 84.752 ms 2024-09-15T22:27:20.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:20 smithi077 ceph-mon[30980]: pgmap v468: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:20.306 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:27:20.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 32466 23.76 MB/sec warmup 81 sec latency 88.011 ms 2024-09-15T22:27:20.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:20 smithi196 ceph-mon[35109]: pgmap v468: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:20 smithi117 ceph-mon[35021]: pgmap v468: 129 pgs: 129 active+clean; 898 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:27:20.643 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:21.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 32954 23.86 MB/sec warmup 82 sec latency 66.233 ms 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.979+0000 7f4508d67640 1 -- 172.21.15.77:0/754593070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 msgr2=0x7f450410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.979+0000 7f4508d67640 1 --2- 172.21.15.77:0/754593070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450410ea90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009ca0 tx=0x7f44fc02a8d0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 -- 172.21.15.77:0/754593070 shutdown_connections 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 --2- 172.21.15.77:0/754593070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450410ea90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 --2- 172.21.15.77:0/754593070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4504102790 0x7f45041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 --2- 172.21.15.77:0/754593070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 -- 172.21.15.77:0/754593070 >> 172.21.15.77:0/754593070 conn(0x7f45040fd8e0 msgr2=0x7f45040ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 -- 172.21.15.77:0/754593070 shutdown_connections 2024-09-15T22:27:21.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.980+0000 7f4508d67640 1 -- 172.21.15.77:0/754593070 wait complete. 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 Processor -- start 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 -- start start 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4504102790 0x7f45041a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45041185e0 con 0x7f4504106f10 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4504118460 con 0x7f4504102790 2024-09-15T22:27:21.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f4508d67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4504118760 con 0x7f4504101de0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f450259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f450259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57960/0 (socket says 172.21.15.77:57960) 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.981+0000 7f450259c640 1 -- 172.21.15.77:0/2159214606 learned_addr learned my addr 172.21.15.77:0/2159214606 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f4502d9d640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 -- 172.21.15.77:0/2159214606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4504102790 msgr2=0x7f45041a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4504102790 0x7f45041a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 -- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 msgr2=0x7f450419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 -- 172.21.15.77:0/2159214606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450419efc0 con 0x7f4504101de0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.982+0000 7f450259c640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44f400daf0 tx=0x7f44f400af60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.983+0000 7f44f37fe640 1 -- 172.21.15.77:0/2159214606 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f401a070 con 0x7f4504101de0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.983+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450419f2b0 con 0x7f4504101de0 2024-09-15T22:27:21.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.983+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45041ab5a0 con 0x7f4504101de0 2024-09-15T22:27:21.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.984+0000 7f44f37fe640 1 -- 172.21.15.77:0/2159214606 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f400cc70 con 0x7f4504101de0 2024-09-15T22:27:21.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.984+0000 7f44f37fe640 1 -- 172.21.15.77:0/2159214606 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40146f0 con 0x7f4504101de0 2024-09-15T22:27:21.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.986+0000 7f450b140740 1 Processor -- start 2024-09-15T22:27:21.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 -- start start 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab58b262f0 con 0x55ab58bc05d0 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab58acdcf0 con 0x55ab58bcb370 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f450b140740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab58b222b0 con 0x55ab58bc1b40 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f44f37fe640 1 -- 172.21.15.77:0/2159214606 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f44f401d070 con 0x7f4504101de0 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57966/0 (socket says 172.21.15.77:57966) 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.987+0000 7f4502d9d640 1 -- 172.21.15.77:0/1936474365 learned_addr learned my addr 172.21.15.77:0/1936474365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f450259c640 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f4502d9d640 1 -- 172.21.15.77:0/1936474365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 msgr2=0x55ab58bcd760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:21.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f4502d9d640 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bcd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f4502d9d640 1 -- 172.21.15.77:0/1936474365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 msgr2=0x55ab58bc0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:21.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f4502d9d640 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.988+0000 7f4502d9d640 1 -- 172.21.15.77:0/1936474365 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab58ab23a0 con 0x55ab58bc1b40 2024-09-15T22:27:21.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.989+0000 7f4502d9d640 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009ca0 tx=0x7f44fc02f320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4874c83db43aab32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:21.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.989+0000 7f44f37fe640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 0x7f44ec0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.989+0000 7f44f37fe640 1 -- 172.21.15.77:0/2159214606 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f44f40cf640 con 0x7f4504101de0 2024-09-15T22:27:21.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.989+0000 7f44f0ff9640 1 -- 172.21.15.77:0/1936474365 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc031e40 con 0x55ab58bc1b40 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.989+0000 7f44f0ff9640 1 -- 172.21.15.77:0/1936474365 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc036030 con 0x55ab58bc1b40 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.990+0000 7f44f0ff9640 1 -- 172.21.15.77:0/1936474365 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004330 con 0x55ab58bc1b40 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 -- 172.21.15.77:0/1936474365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 msgr2=0x55ab58bc0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009ca0 tx=0x7f44fc02f320 comp rx=0 tx=0).stop 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 -- 172.21.15.77:0/1936474365 shutdown_connections 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 --2- 172.21.15.77:0/1936474365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc0090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.991+0000 7f450b140740 1 -- 172.21.15.77:0/1936474365 >> 172.21.15.77:0/1936474365 conn(0x55ab58b89310 msgr2=0x55ab58bbcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:21.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.992+0000 7f450259c640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 0x7f44ec0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.993+0000 7f450b140740 1 -- 172.21.15.77:0/1936474365 shutdown_connections 2024-09-15T22:27:21.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.993+0000 7f450b140740 1 -- 172.21.15.77:0/1936474365 wait complete. 2024-09-15T22:27:21.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.993+0000 7f450b140740 1 -- start start 2024-09-15T22:27:21.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.993+0000 7f450b140740 1 Processor -- start 2024-09-15T22:27:21.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.993+0000 7f450259c640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 0x7f44ec0a5970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55ab58bbdac0 tx=0x7f44f4005880 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:21.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc1b40 0x55ab58bc7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:21.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ab58bce010 con 0x55ab58bc1b40 2024-09-15T22:27:21.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab58ae1060 con 0x55ab58bcb370 2024-09-15T22:27:21.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ab58bce190 con 0x55ab58bc05d0 2024-09-15T22:27:21.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f450b140740 10 client.0 _ll_register_callbacks cb 0x55ab588348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:21.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f4501d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc1b40 0x55ab58bc7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57974/0 (socket says 172.21.15.77:57974) 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f4502d9d640 1 -- 172.21.15.77:0/475520330 learned_addr learned my addr 172.21.15.77:0/475520330 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.994+0000 7f4502d9d640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 -- 172.21.15.77:0/475520330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 msgr2=0x55ab58bc8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 -- 172.21.15.77:0/475520330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc1b40 msgr2=0x55ab58bc7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc1b40 0x55ab58bc7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 -- 172.21.15.77:0/475520330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab588973c0 con 0x55ab58bc05d0 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f4502d9d640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009430 tx=0x7f44fc031a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811cf1b72210d3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f44e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:27:21.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0044c0 con 0x55ab58bc05d0 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44fc036030 con 0x55ab58bc05d0 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.995+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f44fc004660 con 0x55ab58bc05d0 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.996+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44fc004860 con 0x55ab58bc05d0 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.996+0000 7f450b140740 10 client.27931 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.996+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab58bce010 con 0x55ab58bc05d0 2024-09-15T22:27:22.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.996+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc036030 con 0x55ab58bc05d0 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.997+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f44fc043070 con 0x55ab58bc05d0 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.997+0000 7f450b140740 10 client.27931 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.998+0000 7f450b140740 10 client.27931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.998+0000 7f450b140740 20 client.27931 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.998+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58bdd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.998+0000 7f450b140740 4 client.27931 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:22.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:21.998+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab58b33860 con 0x55ab58bdae00 2024-09-15T22:27:22.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.000+0000 7f4501d9b640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58bdd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.001+0000 7f44e9ffb640 10 client.27931 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:27:22.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.001+0000 7f4501d9b640 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58bdd260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef23c03c6bb84e6c server_cookie=4c98c11e4efbc37c in_seq=0 out_seq=0 2024-09-15T22:27:22.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.002+0000 7f44e9ffb640 1 -- 172.21.15.77:0/475520330 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab58b33860 con 0x55ab58bdae00 2024-09-15T22:27:22.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.003+0000 7f44e9ffb640 10 client.27931 handle_command_reply: tid=0 2024-09-15T22:27:22.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.003+0000 7f450b140740 1 client.27931 shutdown 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 msgr2=0x55ab58bcacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009430 tx=0x7f44fc031a30 comp rx=0 tx=0).stop 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 shutdown_connections 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58bdd260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f44e9ffb640 0 client.27931 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc1b40 0x55ab58bc7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 --2- 172.21.15.77:0/475520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc05d0 0x55ab58bcacc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.004+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 >> 172.21.15.77:0/475520330 conn(0x55ab58b89310 msgr2=0x55ab58bc2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.005+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 shutdown_connections 2024-09-15T22:27:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.005+0000 7f450b140740 1 -- 172.21.15.77:0/475520330 wait complete. 2024-09-15T22:27:22.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.005+0000 7f450b140740 20 client.27931 trim_cache size 0 max 16384 2024-09-15T22:27:22.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 Processor -- start 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 -- start start 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58be2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58be30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400fe90 con 0x55ab58bc05d0 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4011a90 con 0x55ab58bcb370 2024-09-15T22:27:22.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f450b140740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4010560 con 0x55ab58bc1b40 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38354/0 (socket says 172.21.15.77:38354) 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f4502d9d640 1 -- 172.21.15.77:0/3182331500 learned_addr learned my addr 172.21.15.77:0/3182331500 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f4501d9b640 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58be2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.026+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58be30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 -- 172.21.15.77:0/3182331500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 msgr2=0x55ab58be2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:22.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58be2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 -- 172.21.15.77:0/3182331500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 msgr2=0x55ab58be30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58be30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 -- 172.21.15.77:0/3182331500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab58bc3120 con 0x55ab58bc05d0 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f44fc02adf0 tx=0x7f44fc02a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7119e99bf40a175a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f44ebfff640 1 -- 172.21.15.77:0/3182331500 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc030b10 con 0x55ab58bc05d0 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f44ebfff640 1 -- 172.21.15.77:0/3182331500 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc036040 con 0x55ab58bc05d0 2024-09-15T22:27:22.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.027+0000 7f44ebfff640 1 -- 172.21.15.77:0/3182331500 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc02f880 con 0x55ab58bc05d0 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 -- 172.21.15.77:0/3182331500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 msgr2=0x55ab58bd7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f44fc02adf0 tx=0x7f44fc02a8d0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 -- 172.21.15.77:0/3182331500 shutdown_connections 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58be30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58be2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 --2- 172.21.15.77:0/3182331500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bd7340 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 -- 172.21.15.77:0/3182331500 >> 172.21.15.77:0/3182331500 conn(0x55ab58b89310 msgr2=0x55ab58bc3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 -- 172.21.15.77:0/3182331500 shutdown_connections 2024-09-15T22:27:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.028+0000 7f450b140740 1 -- 172.21.15.77:0/3182331500 wait complete. 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 -- start start 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 Processor -- start 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4011300 con 0x55ab58bc05d0 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44e4010c30 con 0x55ab58bcb370 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400de90 con 0x55ab58bc1b40 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f450b140740 10 client.0 _ll_register_callbacks cb 0x7f44fc0099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.029+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4501d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4501d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57990/0 (socket says 172.21.15.77:57990) 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39126/0 (socket says 172.21.15.77:39126) 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4501d9b640 1 -- 172.21.15.77:0/3936015763 learned_addr learned my addr 172.21.15.77:0/3936015763 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 -- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 msgr2=0x55ab58bc55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 -- 172.21.15.77:0/3936015763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 msgr2=0x55ab58bc5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 -- 172.21.15.77:0/3936015763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab58c44bf0 con 0x55ab58bcb370 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4501d9b640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f4502d9d640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097f0 tx=0x7f44fc000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f407dfb6956631 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f44e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc02f620 con 0x55ab58bcb370 2024-09-15T22:27:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.030+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44fc036040 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f44fc02f7c0 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44fc02f9c0 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 10 client.28031 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab58c45eb0 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc036040 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f44fc043070 con 0x55ab58bcb370 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 10 client.28031 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 10 client.28031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 20 client.28031 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58c48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 4 client.28031 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.031+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab58b2dd70 con 0x55ab58bdae00 2024-09-15T22:27:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f4501d9b640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58c48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f4501d9b640 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58c48530 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73bfca9b9a1fe26c server_cookie=19a7b4e3bc0b51e0 in_seq=0 out_seq=0 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f44e9ffb640 10 client.28031 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f44e9ffb640 1 -- 172.21.15.77:0/3936015763 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab58b2dd70 con 0x55ab58bdae00 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f44e9ffb640 10 client.28031 handle_command_reply: tid=0 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 client.28031 shutdown 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 msgr2=0x55ab58bc5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097f0 tx=0x7f44fc000f40 comp rx=0 tx=0).stop 2024-09-15T22:27:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 shutdown_connections 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55ab58bdae00 0x55ab58c48530 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab58bcb370 0x55ab58bc5b10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ab58bc1b40 0x55ab58bc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 --2- 172.21.15.77:0/3936015763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ab58bc05d0 0x55ab58bc5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 >> 172.21.15.77:0/3936015763 conn(0x55ab58b89310 msgr2=0x55ab58bc2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.032+0000 7f44e9ffb640 0 client.28031 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 shutdown_connections 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f450b140740 1 -- 172.21.15.77:0/3936015763 wait complete. 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f450b140740 20 client.28031 trim_cache size 0 max 16384 2024-09-15T22:27:22.042 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:27:22.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 msgr2=0x7f44ec0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 0x7f44ec0a5970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55ab58bbdac0 tx=0x7f44f4005880 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 msgr2=0x7f45041a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44f400daf0 tx=0x7f44f400af60 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.033+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 shutdown_connections 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44ec0a34b0 0x7f44ec0a5970 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4504106f10 0x7f450419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4504102790 0x7f45041a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 --2- 172.21.15.77:0/2159214606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4504101de0 0x7f45041a46d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 >> 172.21.15.77:0/2159214606 conn(0x7f45040fd8e0 msgr2=0x7f45040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 shutdown_connections 2024-09-15T22:27:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:22.034+0000 7f4508d67640 1 -- 172.21.15.77:0/2159214606 wait complete. 2024-09-15T22:27:22.545 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:22 smithi077 ceph-mon[30980]: pgmap v469: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:22.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 33409 23.74 MB/sec warmup 83 sec latency 71.673 ms 2024-09-15T22:27:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:22 smithi196 ceph-mon[35109]: pgmap v469: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:22 smithi117 ceph-mon[35021]: pgmap v469: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:22.850 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:27:23.189 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:23.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 34003 23.83 MB/sec warmup 84 sec latency 136.931 ms 2024-09-15T22:27:23.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 -- 172.21.15.77:0/3178287080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 msgr2=0x7f65141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 --2- 172.21.15.77:0/3178287080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65141021c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6504009900 tx=0x7f650402a410 comp rx=0 tx=0).stop 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 -- 172.21.15.77:0/3178287080 shutdown_connections 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 --2- 172.21.15.77:0/3178287080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514106f10 0x7f651410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 --2- 172.21.15.77:0/3178287080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514102790 0x7f65141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 --2- 172.21.15.77:0/3178287080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65141021c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 -- 172.21.15.77:0/3178287080 >> 172.21.15.77:0/3178287080 conn(0x7f65140fd880 msgr2=0x7f65140ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 -- 172.21.15.77:0/3178287080 shutdown_connections 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.788+0000 7f6519124640 1 -- 172.21.15.77:0/3178287080 wait complete. 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 Processor -- start 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 -- start start 2024-09-15T22:27:23.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514106f10 0x7f6514074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6514111f30 con 0x7f6514101de0 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6514111db0 con 0x7f6514106f10 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6519124640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65141120b0 con 0x7f6514102790 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f651259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38368/0 (socket says 172.21.15.77:38368) 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f651259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:58010/0 (socket says 172.21.15.77:58010) 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f6512d9d640 1 -- 172.21.15.77:0/2840866410 learned_addr learned my addr 172.21.15.77:0/2840866410 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.789+0000 7f651359e640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514106f10 0x7f6514074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 -- 172.21.15.77:0/2840866410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514106f10 msgr2=0x7f6514074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514106f10 0x7f6514074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 -- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 msgr2=0x7f65140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 -- 172.21.15.77:0/2840866410 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65141a6890 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f651259c640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6508009e60 tx=0x7f650800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f64fbfff640 1 -- 172.21.15.77:0/2840866410 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650800ccc0 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f64fbfff640 1 -- 172.21.15.77:0/2840866410 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6508021920 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6514198190 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f64fbfff640 1 -- 172.21.15.77:0/2840866410 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508029940 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.790+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141986d0 con 0x7f6514102790 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 Processor -- start 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 -- start start 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560df4adb2f0 con 0x560df4b76b40 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560df4a82cf0 con 0x560df4b80370 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651b4fd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560df4ad72b0 con 0x560df4b755d0 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38370/0 (socket says 172.21.15.77:38370) 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f651359e640 1 -- 172.21.15.77:0/1004354485 learned_addr learned my addr 172.21.15.77:0/1004354485 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.793+0000 7f64fbfff640 1 -- 172.21.15.77:0/2840866410 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6508021ad0 con 0x7f6514102790 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 -- 172.21.15.77:0/1004354485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 msgr2=0x560df4b75a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b75a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 -- 172.21.15.77:0/1004354485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 msgr2=0x560df4b82760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 -- 172.21.15.77:0/1004354485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df4a673a0 con 0x560df4b76b40 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f6512d9d640 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b75a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f651359e640 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f650c009f50 tx=0x7f650c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d163e9cadc15c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f64f97fa640 1 -- 172.21.15.77:0/1004354485 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c004d20 con 0x560df4b76b40 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f64f97fa640 1 -- 172.21.15.77:0/1004354485 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c004050 con 0x560df4b76b40 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f64fbfff640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 0x7f64fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f64f97fa640 1 -- 172.21.15.77:0/1004354485 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c005220 con 0x560df4b76b40 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.794+0000 7f6512d9d640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 0x7f64fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f64fbfff640 1 -- 172.21.15.77:0/2840866410 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f65080e1310 con 0x7f6514102790 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f6512d9d640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 0x7f64fc0a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x560df4b72ac0 tx=0x7f6504005a20 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 -- 172.21.15.77:0/1004354485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 msgr2=0x560df4b75090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f650c009f50 tx=0x7f650c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 -- 172.21.15.77:0/1004354485 shutdown_connections 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b75a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 --2- 172.21.15.77:0/1004354485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b76b40 0x560df4b75090 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.795+0000 7f651b4fd740 1 -- 172.21.15.77:0/1004354485 >> 172.21.15.77:0/1004354485 conn(0x560df4b3e310 msgr2=0x560df4b71c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- 172.21.15.77:0/1004354485 shutdown_connections 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- 172.21.15.77:0/1004354485 wait complete. 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- start start 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 Processor -- start 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b7d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560df4b830c0 con 0x560df4b755d0 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560df4a96060 con 0x560df4b80370 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560df4b83240 con 0x560df4b76b40 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.796+0000 7f651b4fd740 10 client.0 _ll_register_callbacks cb 0x560df47e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:58014/0 (socket says 172.21.15.77:58014) 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38378/0 (socket says 172.21.15.77:38378) 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 -- 172.21.15.77:0/3608628190 learned_addr learned my addr 172.21.15.77:0/3608628190 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651359e640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b7d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 -- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 msgr2=0x560df4b7d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b7d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 -- 172.21.15.77:0/3608628190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 msgr2=0x560df4b7fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 -- 172.21.15.77:0/3608628190 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df484c3c0 con 0x560df4b76b40 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651259c640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f650800c6a0 tx=0x7f65080ad450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d055ae0310d33c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f64f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080e1170 con 0x560df4b76b40 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f65080cd2b0 con 0x560df4b76b40 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.797+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f650c004050 con 0x560df4b76b40 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f65080ac760 con 0x560df4b76b40 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 10 client.27949 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560df4b8eb90 con 0x560df4b76b40 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c004050 con 0x560df4b76b40 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f6508029ae0 con 0x560df4b76b40 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 10 client.27949 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 10 client.27949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 20 client.27949 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4b90050 0x560df4b92440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 4 client.27949 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df4ae8860 con 0x560df4b90050 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.798+0000 7f6512d9d640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4b90050 0x560df4b92440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.799+0000 7f64f6ff5640 10 client.27949 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.799+0000 7f6512d9d640 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4b90050 0x560df4b92440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b5d3e0b1fb53624 server_cookie=ee882f16a379e6d0 in_seq=0 out_seq=0 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f64f6ff5640 1 -- 172.21.15.77:0/3608628190 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560df4ae8860 con 0x560df4b90050 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f64f6ff5640 10 client.27949 handle_command_reply: tid=0 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 client.27949 shutdown 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 msgr2=0x560df4b7d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f650800c6a0 tx=0x7f65080ad450 comp rx=0 tx=0).stop 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 shutdown_connections 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4b90050 0x560df4b92440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b80370 0x560df4b7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b76b40 0x560df4b7d040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3608628190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f64f6ff5640 0 client.27949 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 >> 172.21.15.77:0/3608628190 conn(0x560df4b3e310 msgr2=0x560df4b77960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 shutdown_connections 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 1 -- 172.21.15.77:0/3608628190 wait complete. 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.800+0000 7f651b4fd740 20 client.27949 trim_cache size 0 max 16384 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.816+0000 7f651b4fd740 1 Processor -- start 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.816+0000 7f651b4fd740 1 -- start start 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b8c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ace70 con 0x560df4b755d0 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65141aea70 con 0x560df4b76b40 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651b4fd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ad540 con 0x560df4b80370 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:58024/0 (socket says 172.21.15.77:58024) 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 -- 172.21.15.77:0/3502495064 learned_addr learned my addr 172.21.15.77:0/3502495064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651259c640 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b8c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f6512d9d640 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 -- 172.21.15.77:0/3502495064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 msgr2=0x560df4b8c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b8c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 -- 172.21.15.77:0/3502495064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 msgr2=0x560df4b7c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.817+0000 7f651359e640 1 -- 172.21.15.77:0/3502495064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df4b92a70 con 0x560df4b80370 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.818+0000 7f651359e640 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f650c006010 tx=0x7f650c005a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5452a934ac7c23c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.818+0000 7f64f8ff9640 1 -- 172.21.15.77:0/3502495064 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c003c30 con 0x560df4b80370 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.818+0000 7f64f8ff9640 1 -- 172.21.15.77:0/3502495064 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c003dd0 con 0x560df4b80370 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.818+0000 7f64f8ff9640 1 -- 172.21.15.77:0/3502495064 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00fc90 con 0x560df4b80370 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- 172.21.15.77:0/3502495064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 msgr2=0x560df4b97ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f650c006010 tx=0x7f650c005a70 comp rx=0 tx=0).stop 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- 172.21.15.77:0/3502495064 shutdown_connections 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b80370 0x560df4b97ca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b8c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 --2- 172.21.15.77:0/3502495064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b755d0 0x560df4b7c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- 172.21.15.77:0/3502495064 >> 172.21.15.77:0/3502495064 conn(0x560df4b3e310 msgr2=0x560df4b7b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- 172.21.15.77:0/3502495064 shutdown_connections 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- 172.21.15.77:0/3502495064 wait complete. 2024-09-15T22:27:23.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.819+0000 7f651b4fd740 1 -- start start 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 Processor -- start 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b80370 0x560df4b84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ae2e0 con 0x560df4b80370 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:58034/0 (socket says 172.21.15.77:58034) 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f6512d9d640 1 -- 172.21.15.77:0/2262196979 learned_addr learned my addr 172.21.15.77:0/2262196979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651259c640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651359e640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b80370 0x560df4b84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65141adc10 con 0x560df4b76b40 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65141aae70 con 0x560df4b755d0 2024-09-15T22:27:23.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.820+0000 7f651b4fd740 10 client.0 _ll_register_callbacks cb 0x7f650800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 -- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 msgr2=0x560df4b840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 -- 172.21.15.77:0/2262196979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b80370 msgr2=0x560df4b84820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b80370 0x560df4b84820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 -- 172.21.15.77:0/2262196979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df4b78120 con 0x560df4b755d0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f6512d9d640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6504005ec0 tx=0x7f6504009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e7aeb62b6050e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f64f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504035040 con 0x560df4b755d0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f650c003dd0 con 0x560df4b755d0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f6504005040 con 0x560df4b755d0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6504064050 con 0x560df4b755d0 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f651b4fd740 10 client.27955 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:23.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560df4b7dfe0 con 0x560df4b755d0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.821+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650c003dd0 con 0x560df4b755d0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f65040035f0 con 0x560df4b755d0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 10 client.27955 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 10 client.27955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 20 client.27955 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4bfb1a0 0x560df4bfd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 4 client.27955 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560df4ae2d70 con 0x560df4bfb1a0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651259c640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4bfb1a0 0x560df4bfd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f64f6ff5640 10 client.27955 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.822+0000 7f651259c640 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4bfb1a0 0x560df4bfd590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b07d64962eba9795 server_cookie=d2572ada68239ae8 in_seq=0 out_seq=0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f64f6ff5640 1 -- 172.21.15.77:0/2262196979 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560df4ae2d70 con 0x560df4bfb1a0 2024-09-15T22:27:23.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f64f6ff5640 10 client.27955 handle_command_reply: tid=0 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 client.27955 shutdown 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 msgr2=0x560df4b88db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6504005ec0 tx=0x7f6504009260 comp rx=0 tx=0).stop 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 shutdown_connections 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x560df4bfb1a0 0x560df4bfd590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560df4b80370 0x560df4b84820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f64f6ff5640 0 client.27955 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560df4b76b40 0x560df4b840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 --2- 172.21.15.77:0/2262196979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560df4b755d0 0x560df4b88db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.823+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 >> 172.21.15.77:0/2262196979 conn(0x560df4b3e310 msgr2=0x560df4b7e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 shutdown_connections 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f651b4fd740 1 -- 172.21.15.77:0/2262196979 wait complete. 2024-09-15T22:27:23.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f651b4fd740 20 client.27955 trim_cache size 0 max 16384 2024-09-15T22:27:23.832 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:27:23.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 msgr2=0x7f64fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 0x7f64fc0a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x560df4b72ac0 tx=0x7f6504005a20 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 msgr2=0x7f6514074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6508009e60 tx=0x7f650800edc0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 shutdown_connections 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f64fc0a33b0 0x7f64fc0a5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6514106f10 0x7f6514074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6514102790 0x7f6514074a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 --2- 172.21.15.77:0/2840866410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6514101de0 0x7f65140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 >> 172.21.15.77:0/2840866410 conn(0x7f65140fd880 msgr2=0x7f65140fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 shutdown_connections 2024-09-15T22:27:23.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:23.824+0000 7f6519124640 1 -- 172.21.15.77:0/2840866410 wait complete. 2024-09-15T22:27:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:24 smithi077 ceph-mon[30980]: pgmap v470: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:24.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 34564 23.93 MB/sec warmup 85 sec latency 166.808 ms 2024-09-15T22:27:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:24 smithi196 ceph-mon[35109]: pgmap v470: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:24 smithi117 ceph-mon[35021]: pgmap v470: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 443 op/s 2024-09-15T22:27:25.442 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:27:25.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 35152 24.03 MB/sec warmup 86 sec latency 115.462 ms 2024-09-15T22:27:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:26.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:26 smithi077 ceph-mon[30980]: pgmap v471: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 413 op/s 2024-09-15T22:27:26.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 35699 24.09 MB/sec warmup 87 sec latency 146.488 ms 2024-09-15T22:27:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:26 smithi196 ceph-mon[35109]: pgmap v471: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 413 op/s 2024-09-15T22:27:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:26 smithi117 ceph-mon[35021]: pgmap v471: 129 pgs: 129 active+clean; 927 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 413 op/s 2024-09-15T22:27:27.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 36381 24.29 MB/sec warmup 88 sec latency 62.625 ms 2024-09-15T22:27:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:28 smithi077 ceph-mon[30980]: pgmap v472: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2024-09-15T22:27:28.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 36826 24.29 MB/sec warmup 89 sec latency 104.970 ms 2024-09-15T22:27:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:28 smithi196 ceph-mon[35109]: pgmap v472: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2024-09-15T22:27:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:28 smithi117 ceph-mon[35021]: pgmap v472: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2024-09-15T22:27:29.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 36866 24.04 MB/sec warmup 90 sec latency 924.835 ms 2024-09-15T22:27:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:30 smithi077 ceph-mon[30980]: pgmap v473: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.7 MiB/s wr, 426 op/s 2024-09-15T22:27:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 -- 172.21.15.77:0/1233399009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 msgr2=0x7fb1181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 --2- 172.21.15.77:0/1233399009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 0x7fb1181029b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c02a480 comp rx=0 tx=0).stop 2024-09-15T22:27:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 -- 172.21.15.77:0/1233399009 shutdown_connections 2024-09-15T22:27:30.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 --2- 172.21.15.77:0/1233399009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118102ef0 0x7fb11810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 --2- 172.21.15.77:0/1233399009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 0x7fb1181029b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 --2- 172.21.15.77:0/1233399009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118100e20 0x7fb118101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 -- 172.21.15.77:0/1233399009 >> 172.21.15.77:0/1233399009 conn(0x7fb1180fc7d0 msgr2=0x7fb1180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 -- 172.21.15.77:0/1233399009 shutdown_connections 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.497+0000 7fb11db36640 1 -- 172.21.15.77:0/1233399009 wait complete. 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.498+0000 7fb11db36640 1 Processor -- start 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.498+0000 7fb11db36640 1 -- start start 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.499+0000 7fb11db36640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb11db36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 0x7fb118198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb11db36640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118102ef0 0x7fb11819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51568/0 (socket says 172.21.15.77:51568) 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb1177fe640 1 -- 172.21.15.77:0/2698098147 learned_addr learned my addr 172.21.15.77:0/2698098147 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb118110fa0 con 0x7fb118102570 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb118110e20 con 0x7fb118102ef0 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.500+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb118111120 con 0x7fb118100e20 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb117fff640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118102ef0 0x7fb11819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb1177fe640 1 -- 172.21.15.77:0/2698098147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118102ef0 msgr2=0x7fb11819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118102ef0 0x7fb11819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb1177fe640 1 -- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 msgr2=0x7fb118198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 0x7fb118198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.501+0000 7fb1177fe640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819d510 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.502+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb108009f20 tx=0x7fb10800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.503+0000 7fb114ff9640 1 -- 172.21.15.77:0/2698098147 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108020070 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.503+0000 7fb114ff9640 1 -- 172.21.15.77:0/2698098147 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10800cd80 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.503+0000 7fb114ff9640 1 -- 172.21.15.77:0/2698098147 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10801bbb0 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.503+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819d800 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.505+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1180fcbb0 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.510+0000 7fb114ff9640 1 -- 172.21.15.77:0/2698098147 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb108023070 con 0x7fb118100e20 2024-09-15T22:27:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.511+0000 7fb114ff9640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 0x7fb1000a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.511+0000 7fb114ff9640 1 -- 172.21.15.77:0/2698098147 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb1080d4160 con 0x7fb118100e20 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.514+0000 7fb11ff0f740 1 Processor -- start 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.515+0000 7fb116ffd640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 0x7fb1000a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.516+0000 7fb116ffd640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 0x7fb1000a5970 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0096c0 tx=0x7fb10c000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 -- start start 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7335495d0 0x55c733549a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73355cb80 0x55c73355ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c7334af2f0 con 0x55c73355cb80 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c733456cf0 con 0x55c7335495d0 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.517+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c7334ab2b0 con 0x55c73354ab40 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.518+0000 7fb116ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7335495d0 0x55c733549a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.518+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.518+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51584/0 (socket says 172.21.15.77:51584) 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.518+0000 7fb1177fe640 1 -- 172.21.15.77:0/4099904140 learned_addr learned my addr 172.21.15.77:0/4099904140 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.519+0000 7fb1177fe640 1 -- 172.21.15.77:0/4099904140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7335495d0 msgr2=0x55c733549a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:30.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.519+0000 7fb1177fe640 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7335495d0 0x55c733549a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.519+0000 7fb1177fe640 1 -- 172.21.15.77:0/4099904140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73355cb80 msgr2=0x55c73355ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.519+0000 7fb1177fe640 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73355cb80 0x55c73355ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.519+0000 7fb1177fe640 1 -- 172.21.15.77:0/4099904140 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73343b3a0 con 0x55c73354ab40 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.520+0000 7fb1177fe640 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb10809cbe0 tx=0x7fb10809fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de34ed7f50fbfef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.521+0000 7fb0fdffb640 1 -- 172.21.15.77:0/4099904140 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080a4510 con 0x55c73354ab40 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.521+0000 7fb0fdffb640 1 -- 172.21.15.77:0/4099904140 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1080a8dd0 con 0x55c73354ab40 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.522+0000 7fb11ff0f740 1 -- 172.21.15.77:0/4099904140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 msgr2=0x55c733549090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.522+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb10809cbe0 tx=0x7fb10809fe10 comp rx=0 tx=0).stop 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.527+0000 7fb11ff0f740 1 -- 172.21.15.77:0/4099904140 shutdown_connections 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.527+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73355cb80 0x55c73355ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.527+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c7335495d0 0x55c733549a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.527+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/4099904140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c733549090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.527+0000 7fb11ff0f740 1 -- 172.21.15.77:0/4099904140 >> 172.21.15.77:0/4099904140 conn(0x55c733512310 msgr2=0x55c733545c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.533+0000 7fb11ff0f740 1 -- 172.21.15.77:0/4099904140 shutdown_connections 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.536+0000 7fb11ff0f740 1 -- 172.21.15.77:0/4099904140 wait complete. 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.536+0000 7fb11ff0f740 1 -- start start 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 Processor -- start 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 0x55c733559760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73355cb80 0x55c733559ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c73355f8d0 con 0x55c7335495d0 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c73346a060 con 0x55c73354ab40 2024-09-15T22:27:30.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c73355fa50 con 0x55c73355cb80 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.537+0000 7fb11ff0f740 10 client.0 _ll_register_callbacks cb 0x55c7331bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.538+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.538+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56146/0 (socket says 172.21.15.77:56146) 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb1177fe640 1 -- 172.21.15.77:0/1555237220 learned_addr learned my addr 172.21.15.77:0/1555237220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 0x55c733559760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb117fff640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73355cb80 0x55c733559ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 -- 172.21.15.77:0/1555237220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73355cb80 msgr2=0x55c733559ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73355cb80 0x55c733559ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 -- 172.21.15.77:0/1555237220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 msgr2=0x55c73355c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.539+0000 7fb116ffd640 1 -- 172.21.15.77:0/1555237220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7332203c0 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.540+0000 7fb1177fe640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.544+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 0x55c733559760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009230 tx=0x7fb10c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3dc7ef1b2b4fb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.544+0000 7fb0f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.545+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c04c070 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.545+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb10c031950 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.546+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1080a8dd0 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.547+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb10c040070 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.551+0000 7fb11ff0f740 10 client.28058 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.551+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c73356b360 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.555+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1080a8dd0 con 0x55c73354ab40 2024-09-15T22:27:30.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.555+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb10c03a3f0 con 0x55c73354ab40 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 10 client.28058 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 20 client.28058 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c73356cc10 0x55c73356f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 4 client.28058 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.556+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7334bc860 con 0x55c73356cc10 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.559+0000 7fb1177fe640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c73356cc10 0x55c73356f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.572+0000 7fb0f6ffd640 10 client.28058 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.572+0000 7fb1177fe640 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c73356cc10 0x55c73356f090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4dedd5f53b6a2d3 server_cookie=5cfe68989111ed37 in_seq=0 out_seq=0 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.575+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/1555237220 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7334bc860 con 0x55c73356cc10 2024-09-15T22:27:30.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.575+0000 7fb0f6ffd640 10 client.28058 handle_command_reply: tid=0 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.575+0000 7fb11ff0f740 1 client.28058 shutdown 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 msgr2=0x55c733559760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 0x55c733559760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009230 tx=0x7fb10c009260 comp rx=0 tx=0).stop 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 shutdown_connections 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c73356cc10 0x55c73356f090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73355cb80 0x55c733559ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73354ab40 0x55c733559760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1555237220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c73355c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 >> 172.21.15.77:0/1555237220 conn(0x55c733512310 msgr2=0x55c73354b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.576+0000 7fb0f6ffd640 0 client.28058 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.579+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 shutdown_connections 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.579+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1555237220 wait complete. 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.579+0000 7fb11ff0f740 20 client.28058 trim_cache size 0 max 16384 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.611+0000 7fb11ff0f740 1 Processor -- start 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 -- start start 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7335495d0 0x55c733568bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c73355e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181acf80 con 0x55c73354ab40 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181aeb80 con 0x55c73355cb80 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb11ff0f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ad650 con 0x55c7335495d0 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.613+0000 7fb116ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56154/0 (socket says 172.21.15.77:56154) 2024-09-15T22:27:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 -- 172.21.15.77:0/1916220449 learned_addr learned my addr 172.21.15.77:0/1916220449 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 -- 172.21.15.77:0/1916220449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7335495d0 msgr2=0x55c733568bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7335495d0 0x55c733568bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 -- 172.21.15.77:0/1916220449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 msgr2=0x55c73355e4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c73355e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.614+0000 7fb116ffd640 1 -- 172.21.15.77:0/1916220449 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c733545ed0 con 0x55c73354ab40 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.615+0000 7fb116ffd640 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00240f527e77b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.617+0000 7fb0fd7fa640 1 -- 172.21.15.77:0/1916220449 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c04c040 con 0x55c73354ab40 2024-09-15T22:27:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.617+0000 7fb0fd7fa640 1 -- 172.21.15.77:0/1916220449 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10c03a3f0 con 0x55c73354ab40 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.617+0000 7fb0fd7fa640 1 -- 172.21.15.77:0/1916220449 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c035040 con 0x55c73354ab40 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.618+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1916220449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 msgr2=0x55c73355df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.618+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009900 tx=0x7fb10c004530 comp rx=0 tx=0).stop 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.621+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1916220449 shutdown_connections 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.621+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c73355e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.621+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73354ab40 0x55c73355df70 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.621+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/1916220449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7335495d0 0x55c733568bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.621+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1916220449 >> 172.21.15.77:0/1916220449 conn(0x55c733512310 msgr2=0x55c733554b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.625+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1916220449 shutdown_connections 2024-09-15T22:27:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.627+0000 7fb11ff0f740 1 -- 172.21.15.77:0/1916220449 wait complete. 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.627+0000 7fb11ff0f740 1 -- start start 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb11ff0f740 1 Processor -- start 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb11ff0f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c7335d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56162/0 (socket says 172.21.15.77:56162) 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb1177fe640 1 -- 172.21.15.77:0/2640812834 learned_addr learned my addr 172.21.15.77:0/2640812834 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.628+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c7335d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ae3f0 con 0x55c7335495d0 2024-09-15T22:27:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181add20 con 0x55c73355cb80 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181aaf80 con 0x55c73354ab40 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb11ff0f740 10 client.0 _ll_register_callbacks cb 0x7fb10c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb1177fe640 1 -- 172.21.15.77:0/2640812834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 msgr2=0x55c7335d4960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c7335d4960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb1177fe640 1 -- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 msgr2=0x55c7335d50a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c7335d50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.629+0000 7fb1177fe640 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73354c120 con 0x55c7335495d0 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.634+0000 7fb1177fe640 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c6a0 tx=0x7fb108013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509320c0144b10bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.634+0000 7fb0f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.636+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080a8dd0 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.636+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fb1080c00d0 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.636+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb10c03a3f0 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.641+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb1080a4510 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.642+0000 7fb11ff0f740 10 client.19155 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.642+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c733563d20 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.643+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10c03a3f0 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.643+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb10809f660 con 0x55c7335495d0 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 10 client.19155 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 10 client.19155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 20 client.19155 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c733564060 0x55c733564470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 4 client.19155 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.646+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7334b6d70 con 0x55c733564060 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.655+0000 7fb116ffd640 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c733564060 0x55c733564470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.664+0000 7fb116ffd640 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c733564060 0x55c733564470 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495c43256b08eb30 server_cookie=af000f1f9dc43e4d in_seq=0 out_seq=0 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.666+0000 7fb0f6ffd640 10 client.19155 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.671+0000 7fb0f6ffd640 1 -- 172.21.15.77:0/2640812834 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7334b6d70 con 0x55c733564060 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.671+0000 7fb0f6ffd640 10 client.19155 handle_command_reply: tid=0 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.671+0000 7fb11ff0f740 1 client.19155 shutdown 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.675+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 msgr2=0x55c733557450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.675+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c6a0 tx=0x7fb108013dd0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 shutdown_connections 2024-09-15T22:27:30.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c733564060 0x55c733564470 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c73355cb80 0x55c7335d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c73354ab40 0x55c7335d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 --2- 172.21.15.77:0/2640812834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7335495d0 0x55c733557450 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb0f6ffd640 0 client.19155 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 >> 172.21.15.77:0/2640812834 conn(0x55c733512310 msgr2=0x55c73354b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 shutdown_connections 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 1 -- 172.21.15.77:0/2640812834 wait complete. 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.676+0000 7fb11ff0f740 20 client.19155 trim_cache size 0 max 16384 2024-09-15T22:27:30.609 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 msgr2=0x7fb1000a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 0x7fb1000a5970 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0096c0 tx=0x7fb10c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 msgr2=0x7fb118198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb108009f20 tx=0x7fb10800ee80 comp rx=0 tx=0).stop 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 shutdown_connections 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb1000a34b0 0x7fb1000a5970 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb118102ef0 0x7fb11819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb118102570 0x7fb118198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 --2- 172.21.15.77:0/2698098147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb118100e20 0x7fb118198560 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:30.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.677+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 >> 172.21.15.77:0/2698098147 conn(0x7fb1180fc7d0 msgr2=0x7fb11806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:30.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.678+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 shutdown_connections 2024-09-15T22:27:30.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:29.678+0000 7fb11db36640 1 -- 172.21.15.77:0/2698098147 wait complete. 2024-09-15T22:27:30.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 36946 23.84 MB/sec warmup 91 sec latency 1676.589 ms 2024-09-15T22:27:30.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:30 smithi196 ceph-mon[35109]: pgmap v473: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.7 MiB/s wr, 426 op/s 2024-09-15T22:27:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:30 smithi117 ceph-mon[35021]: pgmap v473: 129 pgs: 129 active+clean; 945 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.7 MiB/s wr, 426 op/s 2024-09-15T22:27:31.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 37198 23.70 MB/sec warmup 92 sec latency 265.459 ms 2024-09-15T22:27:32.528 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:27:32.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:27:32.560 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:32 smithi077 ceph-mon[30980]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 772 op/s 2024-09-15T22:27:32.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 37581 23.66 MB/sec warmup 93 sec latency 86.644 ms 2024-09-15T22:27:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:32 smithi196 ceph-mon[35109]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 772 op/s 2024-09-15T22:27:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:32 smithi117 ceph-mon[35021]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 772 op/s 2024-09-15T22:27:32.917 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:33.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 37999 23.69 MB/sec warmup 94 sec latency 89.100 ms 2024-09-15T22:27:34.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.065+0000 7fad0159a640 1 -- 172.21.15.77:0/1381194778 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc030e30 con 0x7fad0411b140 2024-09-15T22:27:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.065+0000 7fad08baa640 1 -- 172.21.15.77:0/1381194778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad0411b140 msgr2=0x7fad0411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.065+0000 7fad08baa640 1 --2- 172.21.15.77:0/1381194778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad0411b140 0x7fad0411d530 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7facfc009be0 tx=0x7facfc02a6c0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.068+0000 7fad08baa640 1 -- 172.21.15.77:0/1381194778 shutdown_connections 2024-09-15T22:27:34.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.068+0000 7fad08baa640 1 --2- 172.21.15.77:0/1381194778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad0411b140 0x7fad0411d530 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.068+0000 7fad08baa640 1 --2- 172.21.15.77:0/1381194778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad04109830 0x7fad04109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.068+0000 7fad08baa640 1 --2- 172.21.15.77:0/1381194778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad04110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.068+0000 7fad08baa640 1 -- 172.21.15.77:0/1381194778 >> 172.21.15.77:0/1381194778 conn(0x7fad0406e140 msgr2=0x7fad0406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.069+0000 7fad08baa640 1 -- 172.21.15.77:0/1381194778 shutdown_connections 2024-09-15T22:27:34.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.070+0000 7fad08baa640 1 -- 172.21.15.77:0/1381194778 wait complete. 2024-09-15T22:27:34.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.071+0000 7fad08baa640 1 Processor -- start 2024-09-15T22:27:34.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.072+0000 7fad08baa640 1 -- start start 2024-09-15T22:27:34.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad041186c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 0x7fad04111750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad0411b140 0x7fad04111c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad04120df0 con 0x7fad04109450 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad04120c70 con 0x7fad0411b140 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad08baa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad04120f70 con 0x7fad04109830 2024-09-15T22:27:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 0x7fad04111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 0x7fad04111750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51628/0 (socket says 172.21.15.77:51628) 2024-09-15T22:27:34.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad01d9b640 1 -- 172.21.15.77:0/4199511948 learned_addr learned my addr 172.21.15.77:0/4199511948 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:34.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad041186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 -- 172.21.15.77:0/4199511948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 msgr2=0x7fad04111750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 0x7fad04111750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 -- 172.21.15.77:0/4199511948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad0411b140 msgr2=0x7fad04111c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:27:34.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad0411b140 0x7fad04111c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.073+0000 7fad0259c640 1 -- 172.21.15.77:0/4199511948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad04112200 con 0x7fad04109450 2024-09-15T22:27:34.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.074+0000 7fad0259c640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad041186c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7facec00e790 tx=0x7facec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.074+0000 7facfb7fe640 1 -- 172.21.15.77:0/4199511948 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec00eea0 con 0x7fad04109450 2024-09-15T22:27:34.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.074+0000 7facfb7fe640 1 -- 172.21.15.77:0/4199511948 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec00ccc0 con 0x7fad04109450 2024-09-15T22:27:34.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.075+0000 7facfb7fe640 1 -- 172.21.15.77:0/4199511948 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec018780 con 0x7fad04109450 2024-09-15T22:27:34.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.075+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad041124f0 con 0x7fad04109450 2024-09-15T22:27:34.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.075+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad04116480 con 0x7fad04109450 2024-09-15T22:27:34.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.078+0000 7fad0af83740 1 Processor -- start 2024-09-15T22:27:34.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.080+0000 7facfb7fe640 1 -- 172.21.15.77:0/4199511948 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7facec01f020 con 0x7fad04109450 2024-09-15T22:27:34.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.081+0000 7facfb7fe640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 0x7facf00a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.081+0000 7facfb7fe640 1 -- 172.21.15.77:0/4199511948 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7facec0d0120 con 0x7fad04109450 2024-09-15T22:27:34.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.081+0000 7fad0af83740 1 -- start start 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916577b60 0x558916577f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916582d40 0x558916585130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589164dd280 con 0x558916577b60 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558916484b10 con 0x558916582d40 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad0af83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5589164d9240 con 0x558916578480 2024-09-15T22:27:34.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51630/0 (socket says 172.21.15.77:51630) 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 -- 172.21.15.77:0/3739313817 learned_addr learned my addr 172.21.15.77:0/3739313817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 -- 172.21.15.77:0/3739313817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916582d40 msgr2=0x558916585130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916582d40 0x558916585130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 -- 172.21.15.77:0/3739313817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916577b60 msgr2=0x558916577f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916577b60 0x558916577f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.082+0000 7fad01d9b640 1 -- 172.21.15.77:0/3739313817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589164911f0 con 0x558916578480 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.083+0000 7fad01d9b640 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7facf4009c80 tx=0x7facf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c30b97e3538eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.083+0000 7facf8ff9640 1 -- 172.21.15.77:0/3739313817 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4004230 con 0x558916578480 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.083+0000 7facf8ff9640 1 -- 172.21.15.77:0/3739313817 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf40043d0 con 0x558916578480 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.083+0000 7facf8ff9640 1 -- 172.21.15.77:0/3739313817 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4004e60 con 0x558916578480 2024-09-15T22:27:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.084+0000 7fad02d9d640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 0x7facf00a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 -- 172.21.15.77:0/3739313817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 msgr2=0x558916582800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7facf4009c80 tx=0x7facf4011810 comp rx=0 tx=0).stop 2024-09-15T22:27:34.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 -- 172.21.15.77:0/3739313817 shutdown_connections 2024-09-15T22:27:34.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916582d40 0x558916585130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916582800 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 --2- 172.21.15.77:0/3739313817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916577b60 0x558916577f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 -- 172.21.15.77:0/3739313817 >> 172.21.15.77:0/3739313817 conn(0x5589165740f0 msgr2=0x5589165744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad02d9d640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 0x7facf00a5940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facfc004af0 tx=0x7facfc005fd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.085+0000 7fad0af83740 1 -- 172.21.15.77:0/3739313817 shutdown_connections 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.086+0000 7fad0af83740 1 -- 172.21.15.77:0/3739313817 wait complete. 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.086+0000 7fad0af83740 1 -- start start 2024-09-15T22:27:34.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.086+0000 7fad0af83740 1 Processor -- start 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad0af83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916577b60 0x55891657fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad0af83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 0x5589165801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad0af83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916580720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 0x5589165801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 0x5589165801e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36404/0 (socket says 172.21.15.77:36404) 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad01d9b640 1 -- 172.21.15.77:0/891980203 learned_addr learned my addr 172.21.15.77:0/891980203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.087+0000 7fad02d9d640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916580720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558916585910 con 0x558916582d40 2024-09-15T22:27:34.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558916497f10 con 0x558916578480 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558916585a90 con 0x558916577b60 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad0af83740 10 client.0 _ll_register_callbacks cb 0x558916460a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad02d9d640 1 -- 172.21.15.77:0/891980203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916577b60 msgr2=0x55891657fca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad02d9d640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916577b60 0x55891657fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad02d9d640 1 -- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 msgr2=0x5589165801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad02d9d640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 0x5589165801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.088+0000 7fad02d9d640 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891624e6a0 con 0x558916582d40 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.089+0000 7fad02d9d640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916580720 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x558916575470 tx=0x7facfc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39b2a9ab1aa152b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.089+0000 7face9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.089+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc043050 con 0x558916582d40 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.089+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7facfc003d80 con 0x558916582d40 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.089+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf40043d0 con 0x558916582d40 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.091+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7facfc036040 con 0x558916582d40 2024-09-15T22:27:34.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.092+0000 7fad0af83740 10 client.19164 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.092+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558916591570 con 0x558916582d40 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.092+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf40043d0 con 0x558916582d40 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.092+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7facfc0044d0 con 0x558916582d40 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 10 client.19164 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 10 client.19164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 20 client.19164 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558916592f70 0x558916595430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 4 client.19164 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:34.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.093+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589164ea7f0 con 0x558916592f70 2024-09-15T22:27:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.094+0000 7fad0259c640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558916592f70 0x558916595430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.095+0000 7fad0259c640 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558916592f70 0x558916595430 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f33ba4c5d4a5774 server_cookie=19e1917b68a69c66 in_seq=0 out_seq=0 2024-09-15T22:27:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.095+0000 7face9ffb640 10 client.19164 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.100+0000 7face9ffb640 1 -- 172.21.15.77:0/891980203 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589164ea7f0 con 0x558916592f70 2024-09-15T22:27:34.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.100+0000 7face9ffb640 10 client.19164 handle_command_reply: tid=0 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.100+0000 7fad0af83740 1 client.19164 shutdown 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.102+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 msgr2=0x558916580720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.102+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916580720 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x558916575470 tx=0x7facfc02a6c0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 shutdown_connections 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558916592f70 0x558916595430 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916580720 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7face9ffb640 0 client.19164 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916578480 0x5589165801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 --2- 172.21.15.77:0/891980203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916577b60 0x55891657fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 >> 172.21.15.77:0/891980203 conn(0x5589165740f0 msgr2=0x558916579e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 shutdown_connections 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.103+0000 7fad0af83740 1 -- 172.21.15.77:0/891980203 wait complete. 2024-09-15T22:27:34.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.104+0000 7fad0af83740 20 client.19164 trim_cache size 0 max 16384 2024-09-15T22:27:34.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.136+0000 7fad0af83740 1 Processor -- start 2024-09-15T22:27:34.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.136+0000 7fad0af83740 1 -- start start 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.136+0000 7fad0af83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 0x558916581b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.136+0000 7fad0af83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916578480 0x55891659ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.136+0000 7fad0af83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 0x55891659b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad0af83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c6610 con 0x558916578480 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad0af83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c8210 con 0x558916577b60 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad0af83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c6ce0 con 0x558916582d40 2024-09-15T22:27:34.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 0x55891659b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 0x55891659b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51662/0 (socket says 172.21.15.77:51662) 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad02d9d640 1 -- 172.21.15.77:0/361197094 learned_addr learned my addr 172.21.15.77:0/361197094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.137+0000 7fad0259c640 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 0x558916581b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 -- 172.21.15.77:0/361197094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 msgr2=0x55891659b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 0x55891659b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 -- 172.21.15.77:0/361197094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916578480 msgr2=0x55891659ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916578480 0x55891659ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 -- 172.21.15.77:0/361197094 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589165782b0 con 0x558916577b60 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7fad0259c640 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 0x558916581b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7facec0cd480 tx=0x7facec00cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23dc96493eab742f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.138+0000 7facebfff640 1 -- 172.21.15.77:0/361197094 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec0a03c0 con 0x558916577b60 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.139+0000 7facebfff640 1 -- 172.21.15.77:0/361197094 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec0a4c80 con 0x558916577b60 2024-09-15T22:27:34.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.139+0000 7facebfff640 1 -- 172.21.15.77:0/361197094 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec098810 con 0x558916577b60 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 -- 172.21.15.77:0/361197094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 msgr2=0x558916581b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 0x558916581b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7facec0cd480 tx=0x7facec00cf00 comp rx=0 tx=0).stop 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 -- 172.21.15.77:0/361197094 shutdown_connections 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916582d40 0x55891659b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916578480 0x55891659ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 --2- 172.21.15.77:0/361197094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916577b60 0x558916581b00 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7facec0cd480 tx=0x7facec00cf00 comp rx=0 tx=0).stop 2024-09-15T22:27:34.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.141+0000 7fad0af83740 1 -- 172.21.15.77:0/361197094 >> 172.21.15.77:0/361197094 conn(0x5589165740f0 msgr2=0x5589165813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.142+0000 7fad0af83740 1 -- 172.21.15.77:0/361197094 shutdown_connections 2024-09-15T22:27:34.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.142+0000 7fad0af83740 1 -- 172.21.15.77:0/361197094 wait complete. 2024-09-15T22:27:34.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.143+0000 7fad0af83740 1 -- start start 2024-09-15T22:27:34.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.143+0000 7fad0af83740 1 Processor -- start 2024-09-15T22:27:34.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916587230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916587770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c7a80 con 0x558916582d40 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c73b0 con 0x558916592f70 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad041c4610 con 0x558916578480 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad0af83740 10 client.0 _ll_register_callbacks cb 0x7fad04110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad02d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36422/0 (socket says 172.21.15.77:36422) 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad02d9d640 1 -- 172.21.15.77:0/68132109 learned_addr learned my addr 172.21.15.77:0/68132109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:34.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.144+0000 7fad01d9b640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916587770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad0259c640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916587230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 -- 172.21.15.77:0/68132109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 msgr2=0x558916587230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916587230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 -- 172.21.15.77:0/68132109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 msgr2=0x558916587770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916587770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 -- 172.21.15.77:0/68132109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589165f7290 con 0x558916592f70 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.147+0000 7fad02d9d640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facfc004940 tx=0x7facfc031e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7105d697288108d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7face9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc05f020 con 0x558916592f70 2024-09-15T22:27:34.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7facfc03c9a0 con 0x558916592f70 2024-09-15T22:27:34.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facec0a4c80 con 0x558916592f70 2024-09-15T22:27:34.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7facfc043040 con 0x558916592f70 2024-09-15T22:27:34.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7fad0af83740 10 client.28070 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:34.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.148+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589165fe650 con 0x558916592f70 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.149+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc04d070 con 0x558916592f70 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.149+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7facfc048a30 con 0x558916592f70 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.149+0000 7fad0af83740 10 client.28070 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.149+0000 7fad0af83740 10 client.28070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.150+0000 7fad0af83740 20 client.28070 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.150+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5589165fede0 0x558916601200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.150+0000 7fad0af83740 4 client.28070 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.150+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7facec0a4c80 con 0x5589165fede0 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.150+0000 7fad0259c640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5589165fede0 0x558916601200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:34.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.151+0000 7face9ffb640 10 client.28070 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:34.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.151+0000 7fad0259c640 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5589165fede0 0x558916601200 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f4033c8c318e31 server_cookie=5125f2a7602534c1 in_seq=0 out_seq=0 2024-09-15T22:27:34.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.157+0000 7face9ffb640 1 -- 172.21.15.77:0/68132109 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7facec0a4c80 con 0x5589165fede0 2024-09-15T22:27:34.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.157+0000 7face9ffb640 10 client.28070 handle_command_reply: tid=0 2024-09-15T22:27:34.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.157+0000 7fad0af83740 1 client.28070 shutdown 2024-09-15T22:27:34.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.158+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 msgr2=0x5589165f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.158+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7facfc004940 tx=0x7facfc031e10 comp rx=0 tx=0).stop 2024-09-15T22:27:34.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 shutdown_connections 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5589165fede0 0x558916601200 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558916592f70 0x5589165f6ca0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558916582d40 0x558916587770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 --2- 172.21.15.77:0/68132109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558916578480 0x558916587230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 >> 172.21.15.77:0/68132109 conn(0x5589165740f0 msgr2=0x558916573270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7face9ffb640 0 client.28070 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:34.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.160+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 shutdown_connections 2024-09-15T22:27:34.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.161+0000 7fad0af83740 1 -- 172.21.15.77:0/68132109 wait complete. 2024-09-15T22:27:34.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.162+0000 7fad0af83740 20 client.28070 trim_cache size 0 max 16384 2024-09-15T22:27:34.165 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:27:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:27:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "f6e3d023-b312-4250-90a7-94ae89cb2db4", 2024-09-15T22:27:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:27:34.165 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:27:34.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 msgr2=0x7facf00a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 0x7facf00a5940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facfc004af0 tx=0x7facfc005fd0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 msgr2=0x7fad041186c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad041186c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7facec00e790 tx=0x7facec00ec50 comp rx=0 tx=0).stop 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 shutdown_connections 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7facf00a3480 0x7facf00a5940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad0411b140 0x7fad04111c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad04109830 0x7fad04111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 --2- 172.21.15.77:0/4199511948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad04109450 0x7fad041186c0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 >> 172.21.15.77:0/4199511948 conn(0x7fad0406e140 msgr2=0x7fad0411c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 shutdown_connections 2024-09-15T22:27:34.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:34.163+0000 7fad08baa640 1 -- 172.21.15.77:0/4199511948 wait complete. 2024-09-15T22:27:34.181 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:34 smithi077 ceph-mon[30980]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:34.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 38392 23.57 MB/sec warmup 95 sec latency 102.927 ms 2024-09-15T22:27:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:34 smithi196 ceph-mon[35109]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:34 smithi117 ceph-mon[35021]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:35.121 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:27:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:27:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:27:35.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.334 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:35 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.487 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:35.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 39059 23.66 MB/sec warmup 96 sec latency 59.191 ms 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:35 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:27:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:27:35.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:35.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:35 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:27:36.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 -- 172.21.15.77:0/2867180131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118101420 msgr2=0x7f211810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 --2- 172.21.15.77:0/2867180131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118101420 0x7f211810d950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ca0 tx=0x7f211002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 -- 172.21.15.77:0/2867180131 shutdown_connections 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 --2- 172.21.15.77:0/2867180131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118101420 0x7f211810d950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 --2- 172.21.15.77:0/2867180131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118100a80 0x7f2118100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 --2- 172.21.15.77:0/2867180131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 0x7f2118106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.033+0000 7f211df5f640 1 -- 172.21.15.77:0/2867180131 >> 172.21.15.77:0/2867180131 conn(0x7f21180fc7a0 msgr2=0x7f21180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.034+0000 7f211df5f640 1 -- 172.21.15.77:0/2867180131 shutdown_connections 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.034+0000 7f211df5f640 1 -- 172.21.15.77:0/2867180131 wait complete. 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.034+0000 7f211df5f640 1 Processor -- start 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.035+0000 7f211df5f640 1 -- start start 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118100a80 0x7f2118073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 0x7f211806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2118111f90 con 0x7f2118106a80 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2118111e10 con 0x7f2118100a80 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.036+0000 7f211df5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2118112110 con 0x7f2118101420 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2117fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 0x7f211806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51692/0 (socket says 172.21.15.77:51692) 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 -- 172.21.15.77:0/304557512 learned_addr learned my addr 172.21.15.77:0/304557512 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 -- 172.21.15.77:0/304557512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118100a80 msgr2=0x7f2118073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118100a80 0x7f2118073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 -- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 msgr2=0x7f211806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 0x7f211806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.037+0000 7f2116ffd640 1 -- 172.21.15.77:0/304557512 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211806d490 con 0x7f2118101420 2024-09-15T22:27:36.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f2116ffd640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21080058c0 tx=0x7f2108007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f2114ff9640 1 -- 172.21.15.77:0/304557512 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2108007d90 con 0x7f2118101420 2024-09-15T22:27:36.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f2114ff9640 1 -- 172.21.15.77:0/304557512 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2108013020 con 0x7f2118101420 2024-09-15T22:27:36.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f2114ff9640 1 -- 172.21.15.77:0/304557512 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210801e710 con 0x7f2118101420 2024-09-15T22:27:36.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211806d780 con 0x7f2118101420 2024-09-15T22:27:36.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.038+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2118071720 con 0x7f2118101420 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.042+0000 7f2120338740 1 Processor -- start 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 -- start start 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c021b9c2e0 con 0x55c021c36dc0 2024-09-15T22:27:36.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c021b43ce0 con 0x55c021c364a0 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f2120338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c021b982a0 con 0x55c021c41680 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.043+0000 7f21177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56222/0 (socket says 172.21.15.77:56222) 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 -- 172.21.15.77:0/234644506 learned_addr learned my addr 172.21.15.77:0/234644506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f2117fff640 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f2116ffd640 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f2114ff9640 1 -- 172.21.15.77:0/304557512 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f21080131d0 con 0x7f2118101420 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 -- 172.21.15.77:0/234644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 msgr2=0x55c021c43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c43a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 -- 172.21.15.77:0/234644506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 msgr2=0x55c021c36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c36880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.044+0000 7f21177fe640 1 -- 172.21.15.77:0/234644506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c021b28440 con 0x55c021c36dc0 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.045+0000 7f21177fe640 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f210c00ba80 tx=0x7f210c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972d550ee5a3a9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.045+0000 7f2114ff9640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 0x7f21000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.046+0000 7f2114ff9640 1 -- 172.21.15.77:0/304557512 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f21080d7b00 con 0x7f2118101420 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.046+0000 7f20fdffb640 1 -- 172.21.15.77:0/234644506 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c016070 con 0x55c021c36dc0 2024-09-15T22:27:36.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.046+0000 7f20fdffb640 1 -- 172.21.15.77:0/234644506 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005500 con 0x55c021c36dc0 2024-09-15T22:27:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.046+0000 7f20fdffb640 1 -- 172.21.15.77:0/234644506 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c010460 con 0x55c021c36dc0 2024-09-15T22:27:36.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2117fff640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 0x7f21000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 -- 172.21.15.77:0/234644506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 msgr2=0x55c021c41140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f210c00ba80 tx=0x7f210c00bf40 comp rx=0 tx=0).stop 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 -- 172.21.15.77:0/234644506 shutdown_connections 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c41140 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 --2- 172.21.15.77:0/234644506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 -- 172.21.15.77:0/234644506 >> 172.21.15.77:0/234644506 conn(0x55c021c32a30 msgr2=0x55c021c32e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 -- 172.21.15.77:0/234644506 shutdown_connections 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2117fff640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 0x7f21000a58f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f211002f530 tx=0x7f211002f3a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.047+0000 7f2120338740 1 -- 172.21.15.77:0/234644506 wait complete. 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.048+0000 7f2120338740 1 -- start start 2024-09-15T22:27:36.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.048+0000 7f2120338740 1 Processor -- start 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c021c55260 con 0x55c021c36dc0 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c021b57050 con 0x55c021c364a0 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c021c553e0 con 0x55c021c41680 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2120338740 10 client.0 _ll_register_callbacks cb 0x55c021b97160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2117fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2117fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51714/0 (socket says 172.21.15.77:51714) 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2117fff640 1 -- 172.21.15.77:0/4249762557 learned_addr learned my addr 172.21.15.77:0/4249762557 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:36.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.049+0000 7f2116ffd640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f21177fe640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 -- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 msgr2=0x55c021c40f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c40f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 -- 172.21.15.77:0/4249762557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 msgr2=0x55c021c3e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c3e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 -- 172.21.15.77:0/4249762557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02190d3c0 con 0x55c021c41680 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f21177fe640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f2117fff640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2110035000 tx=0x7f211002a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8ddbbae475f7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.050+0000 7f20f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110056060 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c005500 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f2110041030 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f211002fa20 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f2120338740 10 client.27985 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.051+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c021c60c30 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005500 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f2110051470 con 0x55c021c41680 2024-09-15T22:27:36.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 10 client.27985 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 10 client.27985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 20 client.27985 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021c62090 0x55c021c644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 4 client.27985 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:36.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.052+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c021ba9850 con 0x55c021c62090 2024-09-15T22:27:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.053+0000 7f21177fe640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021c62090 0x55c021c644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.054+0000 7f21177fe640 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021c62090 0x55c021c644e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd09f96b5f877e62 server_cookie=3ebf916caacb942c in_seq=0 out_seq=0 2024-09-15T22:27:36.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.054+0000 7f20f6ffd640 10 client.27985 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.056+0000 7f20f6ffd640 1 -- 172.21.15.77:0/4249762557 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c021ba9850 con 0x55c021c62090 2024-09-15T22:27:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.056+0000 7f20f6ffd640 10 client.27985 handle_command_reply: tid=0 2024-09-15T22:27:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.057+0000 7f2120338740 1 client.27985 shutdown 2024-09-15T22:27:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.057+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 msgr2=0x55c021c3e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.057+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2110035000 tx=0x7f211002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 shutdown_connections 2024-09-15T22:27:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021c62090 0x55c021c644e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c3e780 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c3e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 --2- 172.21.15.77:0/4249762557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c364a0 0x55c021c40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 >> 172.21.15.77:0/4249762557 conn(0x55c021c32a30 msgr2=0x55c021c38760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.058+0000 7f20f6ffd640 0 client.27985 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.059+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 shutdown_connections 2024-09-15T22:27:36.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.059+0000 7f2120338740 1 -- 172.21.15.77:0/4249762557 wait complete. 2024-09-15T22:27:36.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.059+0000 7f2120338740 20 client.27985 trim_cache size 0 max 16384 2024-09-15T22:27:36.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 Processor -- start 2024-09-15T22:27:36.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 -- start start 2024-09-15T22:27:36.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c41680 0x55c021c6a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b55d0 con 0x55c021c36dc0 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b71d0 con 0x55c021c41680 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.094+0000 7f2120338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b5ca0 con 0x55c021c364a0 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51716/0 (socket says 172.21.15.77:51716) 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 -- 172.21.15.77:0/2243450624 learned_addr learned my addr 172.21.15.77:0/2243450624 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:36.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f2117fff640 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c41680 0x55c021c6a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 -- 172.21.15.77:0/2243450624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c41680 msgr2=0x55c021c6a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c41680 0x55c021c6a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 -- 172.21.15.77:0/2243450624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 msgr2=0x55c021c69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.095+0000 7f21177fe640 1 -- 172.21.15.77:0/2243450624 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c021c41300 con 0x55c021c364a0 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.096+0000 7f21177fe640 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f210c00d770 tx=0x7f210c004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bcd65fa24a2022 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.096+0000 7f20fd7fa640 1 -- 172.21.15.77:0/2243450624 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c029070 con 0x55c021c364a0 2024-09-15T22:27:36.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.096+0000 7f20fd7fa640 1 -- 172.21.15.77:0/2243450624 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005000 con 0x55c021c364a0 2024-09-15T22:27:36.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.097+0000 7f20fd7fa640 1 -- 172.21.15.77:0/2243450624 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c016040 con 0x55c021c364a0 2024-09-15T22:27:36.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.097+0000 7f2120338740 1 -- 172.21.15.77:0/2243450624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 msgr2=0x55c021c5e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.097+0000 7f2120338740 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f210c00d770 tx=0x7f210c004b30 comp rx=0 tx=0).stop 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 -- 172.21.15.77:0/2243450624 shutdown_connections 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c41680 0x55c021c6a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c36dc0 0x55c021c69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 --2- 172.21.15.77:0/2243450624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c364a0 0x55c021c5e4c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 -- 172.21.15.77:0/2243450624 >> 172.21.15.77:0/2243450624 conn(0x55c021c32a30 msgr2=0x55c021c402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 -- 172.21.15.77:0/2243450624 shutdown_connections 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.098+0000 7f2120338740 1 -- 172.21.15.77:0/2243450624 wait complete. 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 -- start start 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 Processor -- start 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 0x55c021c5ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c36dc0 0x55c021c57e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b6a40 con 0x55c021c364a0 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b6370 con 0x55c021c36dc0 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21181b35f0 con 0x55c021c41680 2024-09-15T22:27:36.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.099+0000 7f2120338740 10 client.0 _ll_register_callbacks cb 0x7f2110009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:36.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.100+0000 7f2117fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.100+0000 7f2117fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:51732/0 (socket says 172.21.15.77:51732) 2024-09-15T22:27:36.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.100+0000 7f2117fff640 1 -- 172.21.15.77:0/139495601 learned_addr learned my addr 172.21.15.77:0/139495601 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.100+0000 7f21177fe640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 0x55c021c5ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f2117fff640 1 -- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c36dc0 msgr2=0x55c021c57e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f2117fff640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c36dc0 0x55c021c57e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f2117fff640 1 -- 172.21.15.77:0/139495601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 msgr2=0x55c021c5ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f2117fff640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 0x55c021c5ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f2117fff640 1 -- 172.21.15.77:0/139495601 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c021c58960 con 0x55c021c41680 2024-09-15T22:27:36.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.101+0000 7f21177fe640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 0x55c021c5ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:36.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.102+0000 7f2117fff640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2110005e30 tx=0x7f211002fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d432fd91724205 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:36.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.102+0000 7f20f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:27:36.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.102+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110056040 con 0x55c021c41680 2024-09-15T22:27:36.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.102+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f2110004d50 con 0x55c021c41680 2024-09-15T22:27:36.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.103+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c005000 con 0x55c021c41680 2024-09-15T22:27:36.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.103+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f211006a050 con 0x55c021c41680 2024-09-15T22:27:36.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.103+0000 7f2120338740 10 client.27991 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:36.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.103+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c021cccb70 con 0x55c021c41680 2024-09-15T22:27:36.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005000 con 0x55c021c41680 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f2110030bb0 con 0x55c021c41680 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 10 client.27991 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 10 client.27991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 20 client.27991 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021ccd2f0 0x55c021ccf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 4 client.27991 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:27:36.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.104+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c021ba3d60 con 0x55c021ccd2f0 2024-09-15T22:27:36.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.105+0000 7f21177fe640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021ccd2f0 0x55c021ccf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:36.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.107+0000 7f20f6ffd640 10 client.27991 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:36.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.107+0000 7f21177fe640 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021ccd2f0 0x55c021ccf710 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c45740c16afb0a server_cookie=bdb9a06454be9a5a in_seq=0 out_seq=0 2024-09-15T22:27:36.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.108+0000 7f20f6ffd640 1 -- 172.21.15.77:0/139495601 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c021ba3d60 con 0x55c021ccd2f0 2024-09-15T22:27:36.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.108+0000 7f20f6ffd640 10 client.27991 handle_command_reply: tid=0 2024-09-15T22:27:36.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 client.27991 shutdown 2024-09-15T22:27:36.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 msgr2=0x55c021c583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2110005e30 tx=0x7f211002fe70 comp rx=0 tx=0).stop 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 shutdown_connections 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c021ccd2f0 0x55c021ccf710 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c021c41680 0x55c021c583a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c021c36dc0 0x55c021c57e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 --2- 172.21.15.77:0/139495601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c021c364a0 0x55c021c5ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 >> 172.21.15.77:0/139495601 conn(0x55c021c32a30 msgr2=0x55c021c3dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f20f6ffd640 0 client.27991 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 shutdown_connections 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 1 -- 172.21.15.77:0/139495601 wait complete. 2024-09-15T22:27:36.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.109+0000 7f2120338740 20 client.27991 trim_cache size 0 max 16384 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout: "f6e3d023-b312-4250-90a7-94ae89cb2db4": { 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:27:36.113 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "f6e3d023-b312-4250-90a7-94ae89cb2db4", 2024-09-15T22:27:36.114 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:27:36.114 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:36.114 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:36.114 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.110+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 msgr2=0x7f21000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.110+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 0x7f21000a58f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f211002f530 tx=0x7f211002f3a0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.110+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 msgr2=0x7f2118073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.110+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21080058c0 tx=0x7f2108007590 comp rx=0 tx=0).stop 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 shutdown_connections 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f21000a3430 0x7f21000a58f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2118106a80 0x7f211806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2118101420 0x7f2118073ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 --2- 172.21.15.77:0/304557512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2118100a80 0x7f2118073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:36.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 >> 172.21.15.77:0/304557512 conn(0x7f21180fc7a0 msgr2=0x7f21180fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:36.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 shutdown_connections 2024-09-15T22:27:36.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:36.111+0000 7f211df5f640 1 -- 172.21.15.77:0/304557512 wait complete. 2024-09-15T22:27:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:36 smithi077 ceph-mon[30980]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:36.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 39738 23.85 MB/sec warmup 97 sec latency 173.149 ms 2024-09-15T22:27:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:36 smithi196 ceph-mon[35109]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:36 smithi117 ceph-mon[35021]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 578 op/s 2024-09-15T22:27:37.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 40300 23.96 MB/sec warmup 98 sec latency 116.429 ms 2024-09-15T22:27:37.790 INFO:tasks.cephfs.filesystem:scrub status for tag:f6e3d023-b312-4250-90a7-94ae89cb2db4 - {'path': '/', 'tag': 'f6e3d023-b312-4250-90a7-94ae89cb2db4', 'options': 'recursive,force'} 2024-09-15T22:27:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:38 smithi077 ceph-mon[30980]: pgmap v477: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 923 op/s 2024-09-15T22:27:38.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 40884 24.04 MB/sec warmup 99 sec latency 55.240 ms 2024-09-15T22:27:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:38 smithi196 ceph-mon[35109]: pgmap v477: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 923 op/s 2024-09-15T22:27:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:38 smithi117 ceph-mon[35021]: pgmap v477: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 923 op/s 2024-09-15T22:27:39.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 41506 24.14 MB/sec warmup 100 sec latency 85.462 ms 2024-09-15T22:27:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:40 smithi077 ceph-mon[30980]: pgmap v478: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-09-15T22:27:40.593 INFO:tasks.workunit.client.0.smithi077.stdout: 10 42002 24.20 MB/sec warmup 101 sec latency 86.695 ms 2024-09-15T22:27:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:40 smithi196 ceph-mon[35109]: pgmap v478: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-09-15T22:27:40.612 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:27:40.612 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-15T22:27:40.612 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:27:40.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:40 smithi117 ceph-mon[35021]: pgmap v478: 129 pgs: 129 active+clean; 966 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 690 op/s 2024-09-15T22:27:40.923 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:41 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:27:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:41 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:27:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:41 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:27:41.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:41 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:27:41.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 -- 172.21.15.77:0/506452091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c108290 msgr2=0x7f970c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:41.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 --2- 172.21.15.77:0/506452091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c108290 0x7f970c10ec30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9704009cd0 tx=0x7f970402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 -- 172.21.15.77:0/506452091 shutdown_connections 2024-09-15T22:27:41.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 --2- 172.21.15.77:0/506452091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c108290 0x7f970c10ec30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 --2- 172.21.15.77:0/506452091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f970c103740 0x7f970c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 --2- 172.21.15.77:0/506452091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c102d90 0x7f970c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 -- 172.21.15.77:0/506452091 >> 172.21.15.77:0/506452091 conn(0x7f970c0fc7d0 msgr2=0x7f970c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 -- 172.21.15.77:0/506452091 shutdown_connections 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.499+0000 7f9711cbe640 1 -- 172.21.15.77:0/506452091 wait complete. 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.500+0000 7f9711cbe640 1 Processor -- start 2024-09-15T22:27:41.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.500+0000 7f9711cbe640 1 -- start start 2024-09-15T22:27:41.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 0x7f970c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 0x7f970c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f970c108290 0x7f970c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f970c112090 con 0x7f970c102d90 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f970c111f10 con 0x7f970c108290 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f9711cbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f970c112210 con 0x7f970c103740 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f970affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 0x7f970c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f970b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 0x7f970c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f970affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 0x7f970c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37674/0 (socket says 172.21.15.77:37674) 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.501+0000 7f970affd640 1 -- 172.21.15.77:0/1804638034 learned_addr learned my addr 172.21.15.77:0/1804638034 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 -- 172.21.15.77:0/1804638034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 msgr2=0x7f970c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:41.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 0x7f970c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 -- 172.21.15.77:0/1804638034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f970c108290 msgr2=0x7f970c19cc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f970c108290 0x7f970c19cc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 -- 172.21.15.77:0/1804638034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c19d2e0 con 0x7f970c102d90 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f970b7fe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 0x7f970c198330 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e9f0 tx=0x7f96fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00ccc0 con 0x7f970c102d90 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc010910 con 0x7f970c102d90 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.502+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc018960 con 0x7f970c102d90 2024-09-15T22:27:41.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.503+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c19d5d0 con 0x7f970c102d90 2024-09-15T22:27:41.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.503+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1a4eb0 con 0x7f970c102d90 2024-09-15T22:27:41.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.504+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c199690 con 0x7f970c102d90 2024-09-15T22:27:41.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.506+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f96fc010ac0 con 0x7f970c102d90 2024-09-15T22:27:41.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.507+0000 7f9708ff9640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 0x7f96f40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.507+0000 7f970affd640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 0x7f96f40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.507+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f96fc0d0990 con 0x7f970c102d90 2024-09-15T22:27:41.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.508+0000 7f970affd640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 0x7f96f40a5970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f970c100380 tx=0x7f9700005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:41.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.511+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96fc09cf80 con 0x7f970c102d90 2024-09-15T22:27:41.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 42431 24.12 MB/sec warmup 102 sec latency 86.194 ms 2024-09-15T22:27:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:41 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:27:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:41 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:27:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:41 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:27:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:41 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:27:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:41 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:27:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:41 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:27:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:41 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:27:41.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:41 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:27:41.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.821+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f970c1a5080 con 0x7f970c102d90 2024-09-15T22:27:41.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.822+0000 7f9708ff9640 1 -- 172.21.15.77:0/1804638034 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7f96fc09c880 con 0x7f970c102d90 2024-09-15T22:27:41.823 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:27:41.823 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:27:41.823 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:27:41.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 msgr2=0x7f96f40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:41.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 0x7f96f40a5970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f970c100380 tx=0x7f9700005f90 comp rx=0 tx=0).stop 2024-09-15T22:27:41.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 msgr2=0x7f970c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:41.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 0x7f970c198330 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e9f0 tx=0x7f96fc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 shutdown_connections 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f96f40a34b0 0x7f96f40a5970 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f970c108290 0x7f970c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f970c103740 0x7f970c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 --2- 172.21.15.77:0/1804638034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f970c102d90 0x7f970c198330 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 >> 172.21.15.77:0/1804638034 conn(0x7f970c0fc7d0 msgr2=0x7f970c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 shutdown_connections 2024-09-15T22:27:41.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:41.824+0000 7f9711cbe640 1 -- 172.21.15.77:0/1804638034 wait complete. 2024-09-15T22:27:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:42 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:27:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:42 smithi077 ceph-mon[30980]: pgmap v479: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-09-15T22:27:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:42 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1804638034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:42.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43029 24.12 MB/sec warmup 103 sec latency 158.997 ms 2024-09-15T22:27:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:42 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:27:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:42 smithi196 ceph-mon[35109]: pgmap v479: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-09-15T22:27:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:42 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1804638034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:42 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:27:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:42 smithi117 ceph-mon[35021]: pgmap v479: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 863 op/s 2024-09-15T22:27:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:42 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1804638034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:27:42.768 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-15T22:27:42.768 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-15T22:27:43.100 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:43.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43793 24.32 MB/sec warmup 104 sec latency 87.148 ms 2024-09-15T22:27:43.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.620+0000 7f8fe6560640 1 -- 172.21.15.77:0/4061308269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0101dd0 msgr2=0x7f8fe01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 --2- 172.21.15.77:0/4061308269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0101dd0 0x7f8fe01021b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009900 tx=0x7f8fd002a410 comp rx=0 tx=0).stop 2024-09-15T22:27:43.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 -- 172.21.15.77:0/4061308269 shutdown_connections 2024-09-15T22:27:43.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 --2- 172.21.15.77:0/4061308269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0106f00 0x7f8fe010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 --2- 172.21.15.77:0/4061308269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 0x7f8fe01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 --2- 172.21.15.77:0/4061308269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0101dd0 0x7f8fe01021b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 -- 172.21.15.77:0/4061308269 >> 172.21.15.77:0/4061308269 conn(0x7f8fe00fd8b0 msgr2=0x7f8fe00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 -- 172.21.15.77:0/4061308269 shutdown_connections 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.621+0000 7f8fe6560640 1 -- 172.21.15.77:0/4061308269 wait complete. 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.622+0000 7f8fe6560640 1 Processor -- start 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.622+0000 7f8fe6560640 1 -- start start 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 0x7f8fe0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 0x7f8fe0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0111fb0 con 0x7f8fe0106f00 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0111e30 con 0x7f8fe0102780 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fe6560640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0112130 con 0x7f8fe0101dd0 2024-09-15T22:27:43.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 0x7f8fe0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37700/0 (socket says 172.21.15.77:37700) 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.623+0000 7f8fdffff640 1 -- 172.21.15.77:0/233380766 learned_addr learned my addr 172.21.15.77:0/233380766 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 0x7f8fe0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fdffff640 1 -- 172.21.15.77:0/233380766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 msgr2=0x7f8fe0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fdffff640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 0x7f8fe0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fdffff640 1 -- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 msgr2=0x7f8fe0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fdffff640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 0x7f8fe0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fdffff640 1 -- 172.21.15.77:0/233380766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe01aae20 con 0x7f8fe0101dd0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.624+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 0x7f8fe0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.625+0000 7f8fdffff640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0004770 tx=0x7f8fd0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.626+0000 7f8fdd7fa640 1 -- 172.21.15.77:0/233380766 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd003f070 con 0x7f8fe0101dd0 2024-09-15T22:27:43.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.626+0000 7f8fdd7fa640 1 -- 172.21.15.77:0/233380766 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd0031ca0 con 0x7f8fe0101dd0 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.626+0000 7f8fdd7fa640 1 -- 172.21.15.77:0/233380766 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0035030 con 0x7f8fe0101dd0 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.626+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01aaff0 con 0x7f8fe0101dd0 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.626+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe01ab450 con 0x7f8fe0101dd0 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.628+0000 7f8fdd7fa640 1 -- 172.21.15.77:0/233380766 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8fd0042070 con 0x7f8fe0101dd0 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 Processor -- start 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 -- start start 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdd7fa640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 0x7f8fc80a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698bbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b8698142a0 con 0x55b8698ae760 2024-09-15T22:27:43.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b8697bbb10 con 0x55b8698af080 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe8939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b869810260 con 0x55b8698b9940 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 0x7f8fc80a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdd7fa640 1 -- 172.21.15.77:0/233380766 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f8fd003bad0 con 0x7f8fe0101dd0 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36320/0 (socket says 172.21.15.77:36320) 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/2865177604 learned_addr learned my addr 172.21.15.77:0/2865177604 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.629+0000 7f8fdffff640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698bbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 -- 172.21.15.77:0/2865177604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 msgr2=0x55b8698bbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698bbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 -- 172.21.15.77:0/2865177604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 msgr2=0x55b8698b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 -- 172.21.15.77:0/2865177604 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b869585b90 con 0x55b8698ae760 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 0x7f8fc80a5a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005fd0 tx=0x7f8fd8012b30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fdffff640 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698aeb40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00ed950 tx=0x7f8fd003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efe4d3b7715efc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fc67fc640 1 -- 172.21.15.77:0/2865177604 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0049070 con 0x55b8698ae760 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fc67fc640 1 -- 172.21.15.77:0/2865177604 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd00c6010 con 0x55b8698ae760 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.630+0000 7f8fc67fc640 1 -- 172.21.15.77:0/2865177604 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00edbd0 con 0x55b8698ae760 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- 172.21.15.77:0/2865177604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 msgr2=0x55b8698aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698aeb40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00ed950 tx=0x7f8fd003b800 comp rx=0 tx=0).stop 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- 172.21.15.77:0/2865177604 shutdown_connections 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698bbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698af080 0x55b8698b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 --2- 172.21.15.77:0/2865177604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698aeb40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- 172.21.15.77:0/2865177604 >> 172.21.15.77:0/2865177604 conn(0x55b8698aacf0 msgr2=0x55b8698ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- 172.21.15.77:0/2865177604 shutdown_connections 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- 172.21.15.77:0/2865177604 wait complete. 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.632+0000 7f8fe8939740 1 -- start start 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 Processor -- start 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698af080 0x55b8698b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b8698bc2d0 con 0x55b8698ae760 2024-09-15T22:27:43.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b86979c6e0 con 0x55b8698b9940 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b8698bc450 con 0x55b8698af080 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.633+0000 7f8fe8939740 10 client.0 _ll_register_callbacks cb 0x55b8697c0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698af080 0x55b8698b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38516/0 (socket says 172.21.15.77:38516) 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 -- 172.21.15.77:0/606306555 learned_addr learned my addr 172.21.15.77:0/606306555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 -- 172.21.15.77:0/606306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698af080 msgr2=0x55b8698b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698af080 0x55b8698b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 -- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 msgr2=0x55b8698b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.634+0000 7f8fdffff640 1 -- 172.21.15.77:0/606306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8698c56b0 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fdffff640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00b81c0 tx=0x7f8fd0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bdaf01416526a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0049070 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f8fd00b8b50 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd00c6010 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.635+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fd00c0890 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 10 client.19200 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b8698bc2d0 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd00c6010 con 0x55b8698ae760 2024-09-15T22:27:43.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f8fd00f01e0 con 0x55b8698ae760 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 10 client.19200 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 10 client.19200 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 20 client.19200 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b8698c9160 0x55b8698cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 4 client.19200 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.636+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8698219d0 con 0x55b8698c9160 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.637+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b8698c9160 0x55b8698cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.637+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b8698c9160 0x55b8698cb580 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a69d743abe8533 server_cookie=b42e2674b307c2d5 in_seq=0 out_seq=0 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.637+0000 7f8fbf7fe640 10 client.19200 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.639+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/606306555 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8698219d0 con 0x55b8698c9160 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.639+0000 7f8fbf7fe640 10 client.19200 handle_command_reply: tid=0 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.639+0000 7f8fe8939740 1 client.19200 shutdown 2024-09-15T22:27:43.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 msgr2=0x55b8698b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00b81c0 tx=0x7f8fd0005c90 comp rx=0 tx=0).stop 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 shutdown_connections 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b8698c9160 0x55b8698cb580 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698af080 0x55b8698b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 --2- 172.21.15.77:0/606306555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698ae760 0x55b8698b6750 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 >> 172.21.15.77:0/606306555 conn(0x55b8698aacf0 msgr2=0x55b8698b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fbf7fe640 0 client.19200 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.640+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 shutdown_connections 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.641+0000 7f8fe8939740 1 -- 172.21.15.77:0/606306555 wait complete. 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.641+0000 7f8fe8939740 20 client.19200 trim_cache size 0 max 16384 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 Processor -- start 2024-09-15T22:27:43.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 -- start start 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698ae760 0x55b8698b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b55e0 con 0x55b8698af080 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b71e0 con 0x55b8698b9940 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe8939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b5cb0 con 0x55b8698ae760 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38520/0 (socket says 172.21.15.77:38520) 2024-09-15T22:27:43.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/3377819275 learned_addr learned my addr 172.21.15.77:0/3377819275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdffff640 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698ae760 0x55b8698b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/3377819275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698ae760 msgr2=0x55b8698b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698ae760 0x55b8698b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/3377819275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 msgr2=0x55b8698d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.663+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/3377819275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8698aab20 con 0x55b8698af080 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.664+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x55b8698b7d90 tx=0x7f8fd4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59f9e0e2e08e9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.664+0000 7f8fc5ffb640 1 -- 172.21.15.77:0/3377819275 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4004250 con 0x55b8698af080 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.664+0000 7f8fc5ffb640 1 -- 172.21.15.77:0/3377819275 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b8698219d0 con 0x55b8698af080 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.664+0000 7f8fc5ffb640 1 -- 172.21.15.77:0/3377819275 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd401e680 con 0x55b8698af080 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- 172.21.15.77:0/3377819275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 msgr2=0x55b8698b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x55b8698b7d90 tx=0x7f8fd4005f90 comp rx=0 tx=0).stop 2024-09-15T22:27:43.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- 172.21.15.77:0/3377819275 shutdown_connections 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698b9940 0x55b8698d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698b7020 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 --2- 172.21.15.77:0/3377819275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698ae760 0x55b8698b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- 172.21.15.77:0/3377819275 >> 172.21.15.77:0/3377819275 conn(0x55b8698aacf0 msgr2=0x55b8698b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- 172.21.15.77:0/3377819275 shutdown_connections 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- 172.21.15.77:0/3377819275 wait complete. 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.665+0000 7f8fe8939740 1 -- start start 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 Processor -- start 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 0x55b8698bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b6a50 con 0x55b8698af080 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b6380 con 0x55b8698ae760 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b35a0 con 0x55b8698b9940 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 0x55b8698bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38524/0 (socket says 172.21.15.77:38524) 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/625583472 learned_addr learned my addr 172.21.15.77:0/625583472 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe4afd640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 0x55b8698bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36340/0 (socket says 172.21.15.77:36340) 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fe8939740 10 client.0 _ll_register_callbacks cb 0x7f8fd00bcca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/625583472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 msgr2=0x55b8698c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 msgr2=0x55b8698bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 0x55b8698bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.666+0000 7f8fdf7fe640 1 -- 172.21.15.77:0/625583472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8698ab980 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fdf7fe640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005780 tx=0x7f8fd4004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb90997c965ad741 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40035f0 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b8698219d0 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7f8fd4003790 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8fd403b050 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fe8939740 10 client.19206 fetch_fsmap learned FSMap version 36 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.667+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b8698a6260 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b8698219d0 con 0x55b8698af080 2024-09-15T22:27:43.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f8fd4026070 con 0x55b8698af080 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 10 client.19206 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 10 client.19206 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 20 client.19206 populate_metadata read hostname 'smithi077' 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b869934e00 0x55b869937220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 4 client.19206 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55b86981bd20 con 0x55b869934e00 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fdffff640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b869934e00 0x55b869937220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fdffff640 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b869934e00 0x55b869937220 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92ba5333c299475 server_cookie=a59c12b23cb13484 in_seq=0 out_seq=0 2024-09-15T22:27:43.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:43.668+0000 7f8fbf7fe640 10 client.19206 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:44.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fbf7fe640 1 -- 172.21.15.77:0/625583472 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b86981bd20 con 0x55b869934e00 2024-09-15T22:27:44.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fbf7fe640 10 client.19206 handle_command_reply: tid=0 2024-09-15T22:27:44.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 client.19206 shutdown 2024-09-15T22:27:44.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:44 smithi077 ceph-mon[30980]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:44.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 msgr2=0x55b8698be3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:44.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005780 tx=0x7f8fd4004070 comp rx=0 tx=0).stop 2024-09-15T22:27:44.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 shutdown_connections 2024-09-15T22:27:44.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b869934e00 0x55b869937220 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fbf7fe640 0 client.19206 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b8698b9940 0x55b8698c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b8698af080 0x55b8698be3f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.465+0000 7f8fe8939740 1 --2- 172.21.15.77:0/625583472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b8698ae760 0x55b8698bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 >> 172.21.15.77:0/625583472 conn(0x55b8698aacf0 msgr2=0x55b8698b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 shutdown_connections 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe8939740 1 -- 172.21.15.77:0/625583472 wait complete. 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe8939740 20 client.19206 trim_cache size 0 max 16384 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 36, 2024-09-15T22:27:44.469 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 54, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "87964e6a": { 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 54, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 137.7, 2024-09-15T22:27:44.470 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:27:44.471 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 msgr2=0x7f8fc80a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.466+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 0x7f8fc80a5a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8005fd0 tx=0x7f8fd8012b30 comp rx=0 tx=0).stop 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 msgr2=0x7f8fe00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0004770 tx=0x7f8fd0003c10 comp rx=0 tx=0).stop 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 shutdown_connections 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8fc80a35b0 0x7f8fc80a5a70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fe0106f00 0x7f8fe0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe0102780 0x7f8fe0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 --2- 172.21.15.77:0/233380766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fe0101dd0 0x7f8fe00744f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:44.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 >> 172.21.15.77:0/233380766 conn(0x7f8fe00fd8b0 msgr2=0x7f8fe00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:44.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 shutdown_connections 2024-09-15T22:27:44.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:44.467+0000 7f8fe6560640 1 -- 172.21.15.77:0/233380766 wait complete. 2024-09-15T22:27:44.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 24.13 MB/sec warmup 105 sec latency 808.727 ms 2024-09-15T22:27:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:44 smithi196 ceph-mon[35109]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:44 smithi117 ceph-mon[35021]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:45.130 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '87964e6a', quiesce took 4.5 seconds. Will release after: 13.2 2024-09-15T22:27:45.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 23.90 MB/sec warmup 106 sec latency 1808.763 ms 2024-09-15T22:27:46.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:46 smithi077 ceph-mon[30980]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:46.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 23.68 MB/sec warmup 107 sec latency 2808.887 ms 2024-09-15T22:27:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:46 smithi196 ceph-mon[35109]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:46 smithi117 ceph-mon[35021]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 517 op/s 2024-09-15T22:27:47.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 23.46 MB/sec warmup 108 sec latency 3808.975 ms 2024-09-15T22:27:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:48 smithi077 ceph-mon[30980]: pgmap v482: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-15T22:27:48.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 23.24 MB/sec warmup 109 sec latency 4809.039 ms 2024-09-15T22:27:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:48 smithi196 ceph-mon[35109]: pgmap v482: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-15T22:27:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:48 smithi117 ceph-mon[35021]: pgmap v482: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 681 op/s 2024-09-15T22:27:49.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 23.03 MB/sec warmup 110 sec latency 5809.088 ms 2024-09-15T22:27:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:50 smithi077 ceph-mon[30980]: pgmap v483: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-09-15T22:27:50.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 22.83 MB/sec warmup 111 sec latency 6809.169 ms 2024-09-15T22:27:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:50 smithi196 ceph-mon[35109]: pgmap v483: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-09-15T22:27:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:50 smithi117 ceph-mon[35021]: pgmap v483: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-09-15T22:27:51.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 22.62 MB/sec warmup 112 sec latency 7809.216 ms 2024-09-15T22:27:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:52 smithi077 ceph-mon[30980]: pgmap v484: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-15T22:27:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:52 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:27:52.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:52 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:27:52.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:52 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:27:52.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:52 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:27:52.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 22.42 MB/sec warmup 113 sec latency 8809.318 ms 2024-09-15T22:27:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:52 smithi196 ceph-mon[35109]: pgmap v484: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-15T22:27:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:52 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:27:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:52 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:27:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:52 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:27:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:52 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:27:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:52 smithi117 ceph-mon[35021]: pgmap v484: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 345 op/s 2024-09-15T22:27:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:52 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:27:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:52 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:27:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:52 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:27:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:52 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:27:53.594 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 22.22 MB/sec warmup 114 sec latency 9809.368 ms 2024-09-15T22:27:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:54 smithi077 ceph-mon[30980]: pgmap v485: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:54.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 22.03 MB/sec warmup 115 sec latency 10809.480 ms 2024-09-15T22:27:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:54 smithi196 ceph-mon[35109]: pgmap v485: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:54 smithi117 ceph-mon[35021]: pgmap v485: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:55.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 21.84 MB/sec warmup 116 sec latency 11809.589 ms 2024-09-15T22:27:56.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:56 smithi077 ceph-mon[30980]: pgmap v486: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:56.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 21.66 MB/sec warmup 117 sec latency 12809.640 ms 2024-09-15T22:27:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:56 smithi196 ceph-mon[35109]: pgmap v486: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:56 smithi117 ceph-mon[35021]: pgmap v486: 129 pgs: 129 active+clean; 1003 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-09-15T22:27:57.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 21.47 MB/sec warmup 118 sec latency 13809.740 ms 2024-09-15T22:27:58.331 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:27:58.581 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:27:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:27:58 smithi077 ceph-mon[30980]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 276 op/s 2024-09-15T22:27:58.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 21.29 MB/sec warmup 119 sec latency 14809.783 ms 2024-09-15T22:27:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:27:58 smithi196 ceph-mon[35109]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 276 op/s 2024-09-15T22:27:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:27:58 smithi117 ceph-mon[35021]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 276 op/s 2024-09-15T22:27:59.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 -- 172.21.15.77:0/30150498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 msgr2=0x7fa3981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 --2- 172.21.15.77:0/30150498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa3981029b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009890 tx=0x7fa38c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 -- 172.21.15.77:0/30150498 shutdown_connections 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 --2- 172.21.15.77:0/30150498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 --2- 172.21.15.77:0/30150498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa3981029b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 --2- 172.21.15.77:0/30150498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 -- 172.21.15.77:0/30150498 >> 172.21.15.77:0/30150498 conn(0x7fa3980fc7d0 msgr2=0x7fa3980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 -- 172.21.15.77:0/30150498 shutdown_connections 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.129+0000 7fa39dc99640 1 -- 172.21.15.77:0/30150498 wait complete. 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 Processor -- start 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 -- start start 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:59.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa398198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa398112050 con 0x7fa398102ef0 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa398111ed0 con 0x7fa398102570 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.130+0000 7fa39dc99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981121d0 con 0x7fa398100e20 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa3977fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa396ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa398198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59996/0 (socket says 172.21.15.77:59996) 2024-09-15T22:27:59.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 -- 172.21.15.77:0/2624222386 learned_addr learned my addr 172.21.15.77:0/2624222386 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 -- 172.21.15.77:0/2624222386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 msgr2=0x7fa398198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 -- 172.21.15.77:0/2624222386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 msgr2=0x7fa398198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa398198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 -- 172.21.15.77:0/2624222386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39819d4f0 con 0x7fa398102ef0 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa396ffd640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa398198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa3977fe640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa397fff640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa39000eb10 tx=0x7fa39000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.131+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390004b20 con 0x7fa398102ef0 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.132+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39819d7e0 con 0x7fa398102ef0 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.132+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa390004cc0 con 0x7fa398102ef0 2024-09-15T22:27:59.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.132+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3981a50c0 con 0x7fa398102ef0 2024-09-15T22:27:59.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.132+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390004b20 con 0x7fa398102ef0 2024-09-15T22:27:59.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.132+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3981021b0 con 0x7fa398102ef0 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.134+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa390005000 con 0x7fa398102ef0 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.135+0000 7fa394ff9640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 0x7fa3800a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.135+0000 7fa3977fe640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 0x7fa3800a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.135+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fa3900d04d0 con 0x7fa398102ef0 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.135+0000 7fa3977fe640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 0x7fa3800a5a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa38800bfd0 tx=0x7fa38800b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:27:59.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.138+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3900541b0 con 0x7fa398102ef0 2024-09-15T22:27:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.340+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa398109090 con 0x7fa398102ef0 2024-09-15T22:27:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.341+0000 7fa394ff9640 1 -- 172.21.15.77:0/2624222386 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7fa39009cb20 con 0x7fa398102ef0 2024-09-15T22:27:59.343 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:27:59.343 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:27:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 msgr2=0x7fa3800a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 0x7fa3800a5a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa38800bfd0 tx=0x7fa38800b390 comp rx=0 tx=0).stop 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 msgr2=0x7fa39819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa39000eb10 tx=0x7fa39000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 shutdown_connections 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3800a35b0 0x7fa3800a5a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa398102ef0 0x7fa39819ce10 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa398102570 0x7fa398198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 --2- 172.21.15.77:0/2624222386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa398100e20 0x7fa398198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:27:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 >> 172.21.15.77:0/2624222386 conn(0x7fa3980fc7d0 msgr2=0x7fa3981048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:27:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 shutdown_connections 2024-09-15T22:27:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:27:59.344+0000 7fa39dc99640 1 -- 172.21.15.77:0/2624222386 wait complete. 2024-09-15T22:28:00.239 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 87964e6a --release --await' 2024-09-15T22:28:00.239 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 87964e6a --release --await 2024-09-15T22:28:00.491 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:00 smithi077 ceph-mon[30980]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 112 op/s 2024-09-15T22:28:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2624222386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:00.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 43887 0.00 MB/sec execute 1 sec latency 16809.863 ms 2024-09-15T22:28:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:00 smithi196 ceph-mon[35109]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 112 op/s 2024-09-15T22:28:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2624222386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:00 smithi117 ceph-mon[35021]: pgmap v488: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.4 MiB/s wr, 112 op/s 2024-09-15T22:28:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2624222386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 -- 172.21.15.77:0/1007106308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00100a80 msgr2=0x7fbc00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/1007106308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00100a80 0x7fbc00100ee0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009930 tx=0x7fbbec02a420 comp rx=0 tx=0).stop 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 -- 172.21.15.77:0/1007106308 shutdown_connections 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/1007106308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 0x7fbc0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/1007106308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00100a80 0x7fbc00100ee0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/1007106308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00106a80 0x7fbc00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 -- 172.21.15.77:0/1007106308 >> 172.21.15.77:0/1007106308 conn(0x7fbc000fc7a0 msgr2=0x7fbc000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 -- 172.21.15.77:0/1007106308 shutdown_connections 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.980+0000 7fbc06cf3640 1 -- 172.21.15.77:0/1007106308 wait complete. 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 Processor -- start 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 -- start start 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 0x7fbc00197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 0x7fbc001978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00110e80 con 0x7fbc00106a80 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00110d00 con 0x7fbc00100a80 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc06cf3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00111000 con 0x7fbc00101420 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60016/0 (socket says 172.21.15.77:60016) 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 -- 172.21.15.77:0/111138972 learned_addr learned my addr 172.21.15.77:0/111138972 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbbfffff640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 0x7fbc001978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 0x7fbc00197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 -- 172.21.15.77:0/111138972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 msgr2=0x7fbc001978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 0x7fbc001978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 -- 172.21.15.77:0/111138972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 msgr2=0x7fbc00197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 0x7fbc00197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 -- 172.21.15.77:0/111138972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0019c310 con 0x7fbc00106a80 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 0x7fbc00197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.981+0000 7fbc05290640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800eb10 tx=0x7fbbf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.982+0000 7fbbfdffb640 1 -- 172.21.15.77:0/111138972 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8004b20 con 0x7fbc00106a80 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.982+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0019c600 con 0x7fbc00106a80 2024-09-15T22:28:01.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.982+0000 7fbbfdffb640 1 -- 172.21.15.77:0/111138972 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8004cc0 con 0x7fbc00106a80 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.982+0000 7fbbfdffb640 1 -- 172.21.15.77:0/111138972 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8010760 con 0x7fbc00106a80 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.982+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001a3ee0 con 0x7fbc00106a80 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.984+0000 7fbbfdffb640 1 -- 172.21.15.77:0/111138972 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbbf8005020 con 0x7fbc00106a80 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.984+0000 7fbc090cc740 1 Processor -- start 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 -- start start 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfdffb640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 0x7fbbf00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ed071f620 con 0x555ed07b9f40 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ed06c6e50 con 0x555ed07ba860 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc090cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ed071b5e0 con 0x555ed07c7450 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60026/0 (socket says 172.21.15.77:60026) 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 -- 172.21.15.77:0/2166586641 learned_addr learned my addr 172.21.15.77:0/2166586641 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 0x7fbbf00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfdffb640 1 -- 172.21.15.77:0/111138972 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fbbf80d0900 con 0x7fbc00106a80 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbc05290640 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 -- 172.21.15.77:0/2166586641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 msgr2=0x555ed07c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 -- 172.21.15.77:0/2166586641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 msgr2=0x555ed07c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.985+0000 7fbbfffff640 1 -- 172.21.15.77:0/2166586641 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed06b7d40 con 0x555ed07b9f40 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbbfffff640 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0057c0 tx=0x7fbbec0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef6e7c20bfa56cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbbfffff640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 0x7fbbf00a5870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a940 tx=0x7fbbec046000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbbeaffd640 1 -- 172.21.15.77:0/2166586641 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec035040 con 0x555ed07b9f40 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbbeaffd640 1 -- 172.21.15.77:0/2166586641 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec003cd0 con 0x555ed07b9f40 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbbeaffd640 1 -- 172.21.15.77:0/2166586641 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0488f0 con 0x555ed07b9f40 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 -- 172.21.15.77:0/2166586641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 msgr2=0x555ed07ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0057c0 tx=0x7fbbec0319f0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 -- 172.21.15.77:0/2166586641 shutdown_connections 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 --2- 172.21.15.77:0/2166586641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07ba320 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.986+0000 7fbc090cc740 1 -- 172.21.15.77:0/2166586641 >> 172.21.15.77:0/2166586641 conn(0x555ed07b65f0 msgr2=0x555ed07b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- 172.21.15.77:0/2166586641 shutdown_connections 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- 172.21.15.77:0/2166586641 wait complete. 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- start start 2024-09-15T22:28:01.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 Processor -- start 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07b9f40 0x555ed07c4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ed07ca020 con 0x555ed07c7450 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ed06a7f10 con 0x555ed07ba860 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ed07ca1a0 con 0x555ed07b9f40 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc090cc740 10 client.0 _ll_register_callbacks cb 0x555ed06cbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60028/0 (socket says 172.21.15.77:60028) 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.987+0000 7fbc05290640 1 -- 172.21.15.77:0/768338042 learned_addr learned my addr 172.21.15.77:0/768338042 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbfffff640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07b9f40 0x555ed07c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 -- 172.21.15.77:0/768338042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07b9f40 msgr2=0x555ed07c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07b9f40 0x555ed07c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 -- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 msgr2=0x555ed07c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 -- 172.21.15.77:0/768338042 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed042e580 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc05290640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8000f80 tx=0x7fbbf809c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bd9a036ca220cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbdffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80ce0e0 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fbbf8005310 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbec003cd0 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbbf80a0ba0 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc090cc740 10 client.19224 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555ed04f8960 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.988+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec003cd0 con 0x555ed07c7450 2024-09-15T22:28:01.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fbbf809bc40 con 0x555ed07c7450 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 10 client.19224 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 10 client.19224 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 20 client.19224 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed07d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 4 client.19224 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ed072cb90 con 0x555ed07d70b0 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed07d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbbdffff640 10 client.19224 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.989+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed07d94f0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb2563e01afff3e server_cookie=9b1ad71011d1c544 in_seq=0 out_seq=0 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbbdffff640 1 -- 172.21.15.77:0/768338042 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ed072cb90 con 0x555ed07d70b0 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbbdffff640 10 client.19224 handle_command_reply: tid=0 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 client.19224 shutdown 2024-09-15T22:28:01.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 msgr2=0x555ed07c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8000f80 tx=0x7fbbf809c210 comp rx=0 tx=0).stop 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 shutdown_connections 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed07d94f0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07c4e90 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbbdffff640 0 client.19224 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 --2- 172.21.15.77:0/768338042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07b9f40 0x555ed07c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 >> 172.21.15.77:0/768338042 conn(0x555ed07b65f0 msgr2=0x555ed07bc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 shutdown_connections 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 1 -- 172.21.15.77:0/768338042 wait complete. 2024-09-15T22:28:01.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:00.990+0000 7fbc090cc740 20 client.19224 trim_cache size 0 max 16384 2024-09-15T22:28:01.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 Processor -- start 2024-09-15T22:28:01.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 -- start start 2024-09-15T22:28:01.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 0x555ed07c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07ba860 0x555ed07dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001acd60 con 0x555ed07c7450 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001ae960 con 0x555ed07b9f40 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc090cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001ad430 con 0x555ed07ba860 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.007+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60042/0 (socket says 172.21.15.77:60042) 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 -- 172.21.15.77:0/4203144738 learned_addr learned my addr 172.21.15.77:0/4203144738 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbbfffff640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07ba860 0x555ed07dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 0x555ed07c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 -- 172.21.15.77:0/4203144738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07ba860 msgr2=0x555ed07dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07ba860 0x555ed07dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 -- 172.21.15.77:0/4203144738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 msgr2=0x555ed07c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 0x555ed07c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 -- 172.21.15.77:0/4203144738 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ed07bb400 con 0x555ed07c7450 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 0x555ed07c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbc05290640 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009270 tx=0x7fbbf80870e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f72a00a5f095d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbbea7fc640 1 -- 172.21.15.77:0/4203144738 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80a5460 con 0x555ed07c7450 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbbea7fc640 1 -- 172.21.15.77:0/4203144738 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf809bc40 con 0x555ed07c7450 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.008+0000 7fbbea7fc640 1 -- 172.21.15.77:0/4203144738 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8005570 con 0x555ed07c7450 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- 172.21.15.77:0/4203144738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 msgr2=0x555ed07df340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009270 tx=0x7fbbf80870e0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- 172.21.15.77:0/4203144738 shutdown_connections 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07c7450 0x555ed07df340 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07ba860 0x555ed07dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 --2- 172.21.15.77:0/4203144738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07b9f40 0x555ed07c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- 172.21.15.77:0/4203144738 >> 172.21.15.77:0/4203144738 conn(0x555ed07b65f0 msgr2=0x555ed07c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- 172.21.15.77:0/4203144738 shutdown_connections 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- 172.21.15.77:0/4203144738 wait complete. 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 -- start start 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.009+0000 7fbc090cc740 1 Processor -- start 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001ae1d0 con 0x555ed07b9f40 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001adb00 con 0x555ed07ba860 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001aad60 con 0x555ed07c7450 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 10 client.0 _ll_register_callbacks cb 0x7fbbf809c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:01.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60056/0 (socket says 172.21.15.77:60056) 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 -- 172.21.15.77:0/3256116149 learned_addr learned my addr 172.21.15.77:0/3256116149 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc05290640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbbfffff640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 -- 172.21.15.77:0/3256116149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 msgr2=0x555ed07cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 -- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 msgr2=0x555ed07cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 -- 172.21.15.77:0/3256116149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ed0841070 con 0x555ed07b9f40 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc04a8f640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009780 tx=0x7fbbf4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af811bb74c83aac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbbdffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:01.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4004230 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fbbf40043d0 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf809bc40 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbbf402e050 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.010+0000 7fbc090cc740 10 client.19230 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555ed0842380 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf809bc40 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fbbf400f040 con 0x555ed07b9f40 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 10 client.19230 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:01.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 10 client.19230 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 20 client.19230 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed0844bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 4 client.19230 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "87964e6a", "release": true, "await": true}] 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "87964e6a", "release": true, "await": true}) -- 0x555ed07270a0 con 0x555ed07d70b0 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbbfffff640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed0844bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbbfffff640 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed0844bf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e1d4c3d9298ccb server_cookie=c6649ca355400c1b in_seq=0 out_seq=0 2024-09-15T22:28:01.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.011+0000 7fbbdffff640 10 client.19230 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:01.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbbdffff640 1 -- 172.21.15.77:0/3256116149 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ed07270a0 con 0x555ed07d70b0 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbbdffff640 10 client.19230 handle_command_reply: tid=0 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 client.19230 shutdown 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 msgr2=0x555ed07cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009780 tx=0x7fbbf4012820 comp rx=0 tx=0).stop 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 shutdown_connections 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ed07d70b0 0x555ed0844bf0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ed07c7450 0x555ed07cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ed07ba860 0x555ed07cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 --2- 172.21.15.77:0/3256116149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ed07b9f40 0x555ed07cb4a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbbdffff640 0 client.19230 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 >> 172.21.15.77:0/3256116149 conn(0x555ed07b65f0 msgr2=0x555ed07b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 shutdown_connections 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.083+0000 7fbc090cc740 1 -- 172.21.15.77:0/3256116149 wait complete. 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.084+0000 7fbc090cc740 20 client.19230 trim_cache size 0 max 16384 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 36, 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 56, 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "87964e6a": { 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 56, 2024-09-15T22:28:01.087 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 137.7, 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:28:01.088 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:01.089 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:01.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 msgr2=0x7fbbf00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 0x7fbbf00a5870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a940 tx=0x7fbbec046000 comp rx=0 tx=0).stop 2024-09-15T22:28:01.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 msgr2=0x7fbc0019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:01.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800eb10 tx=0x7fbbf800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 shutdown_connections 2024-09-15T22:28:01.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbbf00a33b0 0x7fbbf00a5870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc00106a80 0x7fbc0019bc30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00101420 0x7fbc001978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 --2- 172.21.15.77:0/111138972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc00100a80 0x7fbc00197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 >> 172.21.15.77:0/111138972 conn(0x7fbc000fc7a0 msgr2=0x7fbc000fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 shutdown_connections 2024-09-15T22:28:01.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:01.085+0000 7fbc06cf3640 1 -- 172.21.15.77:0/111138972 wait complete. 2024-09-15T22:28:01.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 44228 10.20 MB/sec execute 2 sec latency 17326.681 ms 2024-09-15T22:28:01.822 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '87964e6a', total seconds elapsed: 21.2 2024-09-15T22:28:01.823 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-15T22:28:01.823 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:28:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:02.467 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:02 smithi077 ceph-mon[30980]: pgmap v489: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.6 MiB/s wr, 191 op/s 2024-09-15T22:28:02.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 44832 18.68 MB/sec execute 3 sec latency 76.442 ms 2024-09-15T22:28:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:02 smithi196 ceph-mon[35109]: pgmap v489: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.6 MiB/s wr, 191 op/s 2024-09-15T22:28:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:02 smithi117 ceph-mon[35021]: pgmap v489: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 5.6 MiB/s wr, 191 op/s 2024-09-15T22:28:02.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 -- 172.21.15.77:0/2254464697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c0ffc70 msgr2=0x7fc81c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 --2- 172.21.15.77:0/2254464697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c0ffc70 0x7fc81c100640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009900 tx=0x7fc80c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 -- 172.21.15.77:0/2254464697 shutdown_connections 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 --2- 172.21.15.77:0/2254464697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc81c100fc0 0x7fc81c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 --2- 172.21.15.77:0/2254464697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 --2- 172.21.15.77:0/2254464697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c0ffc70 0x7fc81c100640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 -- 172.21.15.77:0/2254464697 >> 172.21.15.77:0/2254464697 conn(0x7fc81c0fba90 msgr2=0x7fc81c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 -- 172.21.15.77:0/2254464697 shutdown_connections 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.725+0000 7fc821c1f640 1 -- 172.21.15.77:0/2254464697 wait complete. 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.726+0000 7fc821c1f640 1 Processor -- start 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.726+0000 7fc821c1f640 1 -- start start 2024-09-15T22:28:02.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc81c0ffc70 0x7fc81c10d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c10fdb0 con 0x7fc81c0ffc70 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c10fc30 con 0x7fc81c100b80 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc821c1f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c10ff30 con 0x7fc81c100fc0 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:52900/0 (socket says 172.21.15.77:52900) 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 -- 172.21.15.77:0/473264656 learned_addr learned my addr 172.21.15.77:0/473264656 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:02.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c10dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35308/0 (socket says 172.21.15.77:35308) 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 -- 172.21.15.77:0/473264656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 msgr2=0x7fc81c10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c10dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 -- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc81c0ffc70 msgr2=0x7fc81c10d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc81c0ffc70 0x7fc81c10d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.727+0000 7fc81bfff640 1 -- 172.21.15.77:0/473264656 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c10a0e0 con 0x7fc81c100fc0 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc81bfff640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc814009c90 tx=0x7fc81400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814003b70 con 0x7fc81c100fc0 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c10a3d0 con 0x7fc81c100fc0 2024-09-15T22:28:02.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1ab320 con 0x7fc81c100fc0 2024-09-15T22:28:02.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814021910 con 0x7fc81c100fc0 2024-09-15T22:28:02.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.728+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814010690 con 0x7fc81c100fc0 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.729+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc808005290 con 0x7fc81c100fc0 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.731+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc814021430 con 0x7fc81c100fc0 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.732+0000 7fc818ff9640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 0x7fc8040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.732+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fc8140d0580 con 0x7fc81c100fc0 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.732+0000 7fc81b7fe640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 0x7fc8040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:02.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.733+0000 7fc81b7fe640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 0x7fc8040a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0096c0 tx=0x7fc80c0314b0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:02.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:02.736+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc81409cbd0 con 0x7fc81c100fc0 2024-09-15T22:28:03.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.011+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc808005850 con 0x7fc81c100fc0 2024-09-15T22:28:03.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.012+0000 7fc818ff9640 1 -- 172.21.15.77:0/473264656 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7fc81409c4d0 con 0x7fc81c100fc0 2024-09-15T22:28:03.013 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:28:03.014 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:28:03.014 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:28:03.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 msgr2=0x7fc8040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:03.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 0x7fc8040a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0096c0 tx=0x7fc80c0314b0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 msgr2=0x7fc81c109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc814009c90 tx=0x7fc81400c6a0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 shutdown_connections 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8040a33b0 0x7fc8040a5870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc81c100fc0 0x7fc81c109820 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc81c100b80 0x7fc81c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 --2- 172.21.15.77:0/473264656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc81c0ffc70 0x7fc81c10d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:03.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.015+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 >> 172.21.15.77:0/473264656 conn(0x7fc81c0fba90 msgr2=0x7fc81c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:03.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.016+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 shutdown_connections 2024-09-15T22:28:03.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:03.016+0000 7fc821c1f640 1 -- 172.21.15.77:0/473264656 wait complete. 2024-09-15T22:28:03.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:03 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/473264656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:03.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 45441 23.89 MB/sec execute 4 sec latency 45.092 ms 2024-09-15T22:28:03.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:03 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/473264656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:03.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:03 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/473264656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:03.755 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-15T22:28:03.755 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-15T22:28:04.085 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:04.508 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:04 smithi077 ceph-mon[30980]: pgmap v490: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:04.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 45969 24.30 MB/sec execute 5 sec latency 55.800 ms 2024-09-15T22:28:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:04 smithi196 ceph-mon[35109]: pgmap v490: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:04 smithi117 ceph-mon[35021]: pgmap v490: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:04.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 -- 172.21.15.77:0/2896753686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 msgr2=0x7fcf4c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/2896753686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c10bb60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002a480 comp rx=0 tx=0).stop 2024-09-15T22:28:04.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 -- 172.21.15.77:0/2896753686 shutdown_connections 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/2896753686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c10c180 0x7fcf4c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/2896753686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c10bb60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/2896753686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c102d50 0x7fcf4c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.622+0000 7fcf53b1d640 1 -- 172.21.15.77:0/2896753686 >> 172.21.15.77:0/2896753686 conn(0x7fcf4c0fc7d0 msgr2=0x7fcf4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- 172.21.15.77:0/2896753686 shutdown_connections 2024-09-15T22:28:04.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- 172.21.15.77:0/2896753686 wait complete. 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 Processor -- start 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- start start 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c10c180 0x7fcf4c101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c111fd0 con 0x7fcf4c102d50 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c111e50 con 0x7fcf4c10c180 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.623+0000 7fcf53b1d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c112150 con 0x7fcf4c103670 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60100/0 (socket says 172.21.15.77:60100) 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 -- 172.21.15.77:0/1542901437 learned_addr learned my addr 172.21.15.77:0/1542901437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c10c180 0x7fcf4c101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf510b8640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 -- 172.21.15.77:0/1542901437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 msgr2=0x7fcf4c100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 -- 172.21.15.77:0/1542901437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c10c180 msgr2=0x7fcf4c101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c10c180 0x7fcf4c101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 -- 172.21.15.77:0/1542901437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1a69d0 con 0x7fcf4c102d50 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf518b9640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9f0 tx=0x7fcf3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.624+0000 7fcf4affd640 1 -- 172.21.15.77:0/1542901437 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00ccc0 con 0x7fcf4c102d50 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.625+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c1a6cc0 con 0x7fcf4c102d50 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.625+0000 7fcf4affd640 1 -- 172.21.15.77:0/1542901437 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c010910 con 0x7fcf4c102d50 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.625+0000 7fcf4affd640 1 -- 172.21.15.77:0/1542901437 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0188a0 con 0x7fcf4c102d50 2024-09-15T22:28:04.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.625+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1a71c0 con 0x7fcf4c102d50 2024-09-15T22:28:04.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.628+0000 7fcf55ef6740 1 Processor -- start 2024-09-15T22:28:04.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.628+0000 7fcf55ef6740 1 -- start start 2024-09-15T22:28:04.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.628+0000 7fcf4affd640 1 -- 172.21.15.77:0/1542901437 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcf3c010ac0 con 0x7fcf4c102d50 2024-09-15T22:28:04.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.628+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.628+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35324/0 (socket says 172.21.15.77:35324) 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf518b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 -- 172.21.15.77:0/1229531598 learned_addr learned my addr 172.21.15.77:0/1229531598 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586f8d802a0 con 0x5586f8e25940 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586f8d27b10 con 0x5586f8e1a760 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5586f8d7c260 con 0x5586f8e1b080 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 -- 172.21.15.77:0/1229531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 msgr2=0x5586f8e25400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e25400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 -- 172.21.15.77:0/1229531598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 msgr2=0x5586f8e27d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf520ba640 1 -- 172.21.15.77:0/1229531598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f8af1b90 con 0x5586f8e1a760 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.629+0000 7fcf4affd640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 0x7fcf340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009f60 tx=0x7fcf44009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204c68e4efc3e3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf4affd640 1 -- 172.21.15.77:0/1542901437 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fcf3c0d0410 con 0x7fcf4c102d50 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf518b9640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 0x7fcf340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf33fff640 1 -- 172.21.15.77:0/1229531598 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4401e070 con 0x5586f8e1a760 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf33fff640 1 -- 172.21.15.77:0/1229531598 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf44003e50 con 0x5586f8e1a760 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.630+0000 7fcf33fff640 1 -- 172.21.15.77:0/1229531598 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4400f7a0 con 0x5586f8e1a760 2024-09-15T22:28:04.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 msgr2=0x5586f8e1ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009f60 tx=0x7fcf44009f90 comp rx=0 tx=0).stop 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf518b9640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 0x7fcf340a58f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0cd880 tx=0x7fcf3c009700 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 shutdown_connections 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e25400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1229531598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e1ab40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 >> 172.21.15.77:0/1229531598 conn(0x5586f8e16cf0 msgr2=0x5586f8e170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 shutdown_connections 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.631+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1229531598 wait complete. 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 -- start start 2024-09-15T22:28:04.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 Processor -- start 2024-09-15T22:28:04.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 0x5586f8e22820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586f8e28510 con 0x5586f8e25940 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586f8d086e0 con 0x5586f8e1b080 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5586f8e28690 con 0x5586f8e1a760 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf510b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.632+0000 7fcf55ef6740 10 client.0 _ll_register_callbacks cb 0x5586f8d2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:35330/0 (socket says 172.21.15.77:35330) 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 -- 172.21.15.77:0/965846745 learned_addr learned my addr 172.21.15.77:0/965846745 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf520ba640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 0x5586f8e22820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf520ba640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 -- 172.21.15.77:0/965846745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 msgr2=0x5586f8e22820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 0x5586f8e22820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 -- 172.21.15.77:0/965846745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 msgr2=0x5586f8e232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 -- 172.21.15.77:0/965846745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f8e31a00 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf520ba640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 0x5586f8e22820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf510b8640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf40004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30c43bf683906a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf317fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40035040 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.633+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fcf40005240 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.634+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf44003e50 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.634+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf40005440 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.634+0000 7fcf55ef6740 10 client.28130 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.634+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586f8e34370 con 0x5586f8e1b080 2024-09-15T22:28:04.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.634+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf44003e50 con 0x5586f8e1b080 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fcf400037b0 con 0x5586f8e1b080 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 10 client.28130 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 10 client.28130 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 20 client.28130 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8e358f0 0x5586f8e37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 4 client.28130 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f8d8d9d0 con 0x5586f8e358f0 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.635+0000 7fcf520ba640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8e358f0 0x5586f8e37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.636+0000 7fcf317fa640 10 client.28130 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:04.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.636+0000 7fcf520ba640 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8e358f0 0x5586f8e37ce0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9428008b4c0bae3 server_cookie=9689cfa2091850c9 in_seq=0 out_seq=0 2024-09-15T22:28:04.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf317fa640 1 -- 172.21.15.77:0/965846745 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586f8d8d9d0 con 0x5586f8e358f0 2024-09-15T22:28:04.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf317fa640 10 client.28130 handle_command_reply: tid=0 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 client.28130 shutdown 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 msgr2=0x5586f8e22d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf40004270 comp rx=0 tx=0).stop 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 shutdown_connections 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8e358f0 0x5586f8e37ce0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e22d60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/965846745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1a760 0x5586f8e22820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf317fa640 0 client.28130 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.642+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 >> 172.21.15.77:0/965846745 conn(0x5586f8e16cf0 msgr2=0x5586f8e1c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.643+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 shutdown_connections 2024-09-15T22:28:04.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.643+0000 7fcf55ef6740 1 -- 172.21.15.77:0/965846745 wait complete. 2024-09-15T22:28:04.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.643+0000 7fcf55ef6740 20 client.28130 trim_cache size 0 max 16384 2024-09-15T22:28:04.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 Processor -- start 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 -- start start 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b1340 con 0x5586f8e25940 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b2f40 con 0x5586f8e1a760 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b1a10 con 0x5586f8e1b080 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60114/0 (socket says 172.21.15.77:60114) 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 -- 172.21.15.77:0/1679141143 learned_addr learned my addr 172.21.15.77:0/1679141143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf510b8640 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 -- 172.21.15.77:0/1679141143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 msgr2=0x5586f8e3d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e3d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 -- 172.21.15.77:0/1679141143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 msgr2=0x5586f8e230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.664+0000 7fcf520ba640 1 -- 172.21.15.77:0/1679141143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f8e17da0 con 0x5586f8e25940 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.665+0000 7fcf520ba640 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fcf44002230 tx=0x7fcf44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80718a9d3ac2b7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.665+0000 7fcf337fe640 1 -- 172.21.15.77:0/1679141143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4401e040 con 0x5586f8e25940 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.665+0000 7fcf337fe640 1 -- 172.21.15.77:0/1679141143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf44004db0 con 0x5586f8e25940 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.665+0000 7fcf337fe640 1 -- 172.21.15.77:0/1679141143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4400fbe0 con 0x5586f8e25940 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1679141143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 msgr2=0x5586f8e3daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fcf44002230 tx=0x7fcf44005fd0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1679141143 shutdown_connections 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e25940 0x5586f8e3daf0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e1b080 0x5586f8e3d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/1679141143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1a760 0x5586f8e230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1679141143 >> 172.21.15.77:0/1679141143 conn(0x5586f8e16cf0 msgr2=0x5586f8e248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:04.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1679141143 shutdown_connections 2024-09-15T22:28:04.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- 172.21.15.77:0/1679141143 wait complete. 2024-09-15T22:28:04.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 -- start start 2024-09-15T22:28:04.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.666+0000 7fcf55ef6740 1 Processor -- start 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e1f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e25940 0x5586f8e1fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b27b0 con 0x5586f8e1a760 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b20e0 con 0x5586f8e1b080 2024-09-15T22:28:04.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1af340 con 0x5586f8e25940 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf55ef6740 10 client.0 _ll_register_callbacks cb 0x7fcf40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf520ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60116/0 (socket says 172.21.15.77:60116) 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf520ba640 1 -- 172.21.15.77:0/2219148462 learned_addr learned my addr 172.21.15.77:0/2219148462 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e1f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf520ba640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e25940 0x5586f8e1fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 -- 172.21.15.77:0/2219148462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e25940 msgr2=0x5586f8e1fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e25940 0x5586f8e1fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 -- 172.21.15.77:0/2219148462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 msgr2=0x5586f8e2e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf510b8640 1 -- 172.21.15.77:0/2219148462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f8e9f580 con 0x5586f8e1b080 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.667+0000 7fcf520ba640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:04.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf510b8640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e1f6c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcf40003c00 tx=0x7fcf40000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f818c0c061d537f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf317fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4005b060 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf44004db0 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9438+0+0 (secure 0 0 0) 0x7fcf400316d0 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf40062050 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf55ef6740 10 client.28136 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586f8ea06b0 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.668+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf44004db0 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fcf4003c560 con 0x5586f8e1b080 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 10 client.28136 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 10 client.28136 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 20 client.28136 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:04.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8ea0f90 0x5586f8ea33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:04.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 4 client.28136 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-15T22:28:04.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x5586f8d87d20 con 0x5586f8ea0f90 2024-09-15T22:28:04.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf520ba640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8ea0f90 0x5586f8ea33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:04.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf317fa640 10 client.28136 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:04.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:04.669+0000 7fcf520ba640 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8ea0f90 0x5586f8ea33b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec48f995a507374 server_cookie=e362154eeb450f90 in_seq=0 out_seq=0 2024-09-15T22:28:05.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf317fa640 1 -- 172.21.15.77:0/2219148462 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586f8d87d20 con 0x5586f8ea0f90 2024-09-15T22:28:05.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf317fa640 10 client.28136 handle_command_reply: tid=0 2024-09-15T22:28:05.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 client.28136 shutdown 2024-09-15T22:28:05.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 msgr2=0x5586f8e1f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e1f6c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcf40003c00 tx=0x7fcf40000bc0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 shutdown_connections 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5586f8ea0f90 0x5586f8ea33b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5586f8e25940 0x5586f8e1fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf317fa640 0 client.28136 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586f8e1b080 0x5586f8e1f6c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 --2- 172.21.15.77:0/2219148462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586f8e1a760 0x5586f8e2e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.449+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 >> 172.21.15.77:0/2219148462 conn(0x5586f8e16cf0 msgr2=0x5586f8e15650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.450+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 shutdown_connections 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.450+0000 7fcf55ef6740 1 -- 172.21.15.77:0/2219148462 wait complete. 2024-09-15T22:28:05.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.450+0000 7fcf55ef6740 20 client.28136 trim_cache size 0 max 16384 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 36, 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 58, 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "830e7afa": { 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 58, 2024-09-15T22:28:05.453 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 137.7, 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:28:05.454 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:05.455 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:05.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 msgr2=0x7fcf340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:05.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 0x7fcf340a58f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0cd880 tx=0x7fcf3c009700 comp rx=0 tx=0).stop 2024-09-15T22:28:05.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 msgr2=0x7fcf4c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:05.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9f0 tx=0x7fcf3c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 shutdown_connections 2024-09-15T22:28:05.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcf340a3430 0x7fcf340a58f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf4c10c180 0x7fcf4c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c103670 0x7fcf4c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 --2- 172.21.15.77:0/1542901437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf4c102d50 0x7fcf4c1009b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 >> 172.21.15.77:0/1542901437 conn(0x7fcf4c0fc7d0 msgr2=0x7fcf4c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 shutdown_connections 2024-09-15T22:28:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:05.451+0000 7fcf53b1d640 1 -- 172.21.15.77:0/1542901437 wait complete. 2024-09-15T22:28:05.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 20.55 MB/sec execute 6 sec latency 828.538 ms 2024-09-15T22:28:06.287 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '830e7afa', quiesce took 4.5 seconds. Will release after: 13.2 2024-09-15T22:28:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:06 smithi077 ceph-mon[30980]: pgmap v491: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:06.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 17.62 MB/sec execute 7 sec latency 1828.602 ms 2024-09-15T22:28:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:06 smithi196 ceph-mon[35109]: pgmap v491: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:06 smithi117 ceph-mon[35021]: pgmap v491: 129 pgs: 129 active+clean; 1008 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T22:28:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:07 smithi077 ceph-mon[30980]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:28:07.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 15.41 MB/sec execute 8 sec latency 2828.672 ms 2024-09-15T22:28:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:07 smithi196 ceph-mon[35109]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:28:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:07 smithi117 ceph-mon[35021]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:28:07.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:28:08.041 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:08.477 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:08 smithi077 ceph-mon[30980]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:08.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 13.70 MB/sec execute 9 sec latency 3828.700 ms 2024-09-15T22:28:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:08 smithi196 ceph-mon[35109]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:08 smithi117 ceph-mon[35021]: pgmap v493: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:08.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.557+0000 7fbd87611640 1 -- 172.21.15.77:0/275664265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80102dd0 msgr2=0x7fbd801031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.557+0000 7fbd87611640 1 --2- 172.21.15.77:0/275664265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80102dd0 0x7fbd801031b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009cd0 tx=0x7fbd7002a8d0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- 172.21.15.77:0/275664265 shutdown_connections 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- 172.21.15.77:0/275664265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80108300 0x7fbd8010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- 172.21.15.77:0/275664265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 0x7fbd80107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- 172.21.15.77:0/275664265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80102dd0 0x7fbd801031b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- 172.21.15.77:0/275664265 >> 172.21.15.77:0/275664265 conn(0x7fbd800fc7d0 msgr2=0x7fbd800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- 172.21.15.77:0/275664265 shutdown_connections 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- 172.21.15.77:0/275664265 wait complete. 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 Processor -- start 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- start start 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 0x7fbd80100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 0x7fbd80100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80112090 con 0x7fbd80108300 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80111f10 con 0x7fbd80102dd0 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.558+0000 7fbd87611640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80112210 con 0x7fbd80103780 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60120/0 (socket says 172.21.15.77:60120) 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 -- 172.21.15.77:0/2036214082 learned_addr learned my addr 172.21.15.77:0/2036214082 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd84bac640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 0x7fbd80100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd853ad640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 0x7fbd80100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 -- 172.21.15.77:0/2036214082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 msgr2=0x7fbd80100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 0x7fbd80100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 -- 172.21.15.77:0/2036214082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 msgr2=0x7fbd80100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 0x7fbd80100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 -- 172.21.15.77:0/2036214082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801a6a10 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd853ad640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 0x7fbd80100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd85bae640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c970 tx=0x7fbd7800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd7e7fc640 1 -- 172.21.15.77:0/2036214082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800ae60 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd7e7fc640 1 -- 172.21.15.77:0/2036214082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd78005500 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801a6d00 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd7e7fc640 1 -- 172.21.15.77:0/2036214082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800f450 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.559+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801a71f0 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.561+0000 7fbd7e7fc640 1 -- 172.21.15.77:0/2036214082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbd78004ee0 con 0x7fbd80108300 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 Processor -- start 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 -- start start 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f46140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db48ea12e0 con 0x55db48f3b4a0 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db48e48ce0 con 0x55db48f3bdc0 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd899ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db48e9d2a0 con 0x55db48f46680 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd7e7fc640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 0x7fbd680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60124/0 (socket says 172.21.15.77:60124) 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd84bac640 1 -- 172.21.15.77:0/350722365 learned_addr learned my addr 172.21.15.77:0/350722365 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd85bae640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 0x7fbd680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.562+0000 7fbd853ad640 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f46140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd7e7fc640 1 -- 172.21.15.77:0/2036214082 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7fbd780cfd60 con 0x7fbd80108300 2024-09-15T22:28:08.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 -- 172.21.15.77:0/350722365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 msgr2=0x55db48f48a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 -- 172.21.15.77:0/350722365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 msgr2=0x55db48f46140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f46140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 -- 172.21.15.77:0/350722365 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db48e2d440 con 0x55db48f3b4a0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd853ad640 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f46140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd84bac640 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400ac90 tx=0x7fbd740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08c0a8b31e2ef76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd85bae640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 0x7fbd680a5870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd7809b950 tx=0x7fbd7806d9d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd677fe640 1 -- 172.21.15.77:0/350722365 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74012450 con 0x55db48f3b4a0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd677fe640 1 -- 172.21.15.77:0/350722365 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd74012ad0 con 0x55db48f3b4a0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.563+0000 7fbd677fe640 1 -- 172.21.15.77:0/350722365 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7400ea90 con 0x55db48f3b4a0 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- 172.21.15.77:0/350722365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 msgr2=0x55db48f3b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400ac90 tx=0x7fbd740117b0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- 172.21.15.77:0/350722365 shutdown_connections 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f46140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 --2- 172.21.15.77:0/350722365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f3b880 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- 172.21.15.77:0/350722365 >> 172.21.15.77:0/350722365 conn(0x55db48f37a30 msgr2=0x55db48f37e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- 172.21.15.77:0/350722365 shutdown_connections 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- 172.21.15.77:0/350722365 wait complete. 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 -- start start 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.564+0000 7fbd899ea740 1 Processor -- start 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 0x55db48f432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db48f49250 con 0x55db48f46680 2024-09-15T22:28:08.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db48e5c050 con 0x55db48f3b4a0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db48f493d0 con 0x55db48f3bdc0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 10 client.0 _ll_register_callbacks cb 0x55db48e9c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60140/0 (socket says 172.21.15.77:60140) 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 -- 172.21.15.77:0/3831024629 learned_addr learned my addr 172.21.15.77:0/3831024629 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd853ad640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 0x55db48f432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 -- 172.21.15.77:0/3831024629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 msgr2=0x55db48f432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 0x55db48f432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 -- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 msgr2=0x55db48f46020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f46020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 -- 172.21.15.77:0/3831024629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db48c123c0 con 0x55db48f46680 2024-09-15T22:28:08.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd853ad640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 0x55db48f432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:08.635 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:08.635 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:28:08.635 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:28:08.635 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd84bac640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbd740113b0 tx=0x7fbd7400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324186908ecd8e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74023040 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7fbd7400e810 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.565+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd74012ad0 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd74032050 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 10 client.19260 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55db48f54c90 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd74012ad0 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fbd7400f800 con 0x55db48f46680 2024-09-15T22:28:08.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 10 client.19260 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 20 client.19260 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48f56040 0x55db48f58430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 4 client.19260 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db48eae850 con 0x55db48f56040 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.566+0000 7fbd853ad640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48f56040 0x55db48f58430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.567+0000 7fbd64ff9640 10 client.19260 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.567+0000 7fbd853ad640 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48f56040 0x55db48f58430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a1b9fdd6ef86f9 server_cookie=f3464d7f4bfe7108 in_seq=0 out_seq=0 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.567+0000 7fbd64ff9640 1 -- 172.21.15.77:0/3831024629 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db48eae850 con 0x55db48f56040 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.567+0000 7fbd64ff9640 10 client.19260 handle_command_reply: tid=0 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.567+0000 7fbd899ea740 1 client.19260 shutdown 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 msgr2=0x55db48f43870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbd740113b0 tx=0x7fbd7400e670 comp rx=0 tx=0).stop 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 shutdown_connections 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48f56040 0x55db48f58430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f46680 0x55db48f43870 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd64ff9640 0 client.19260 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f3bdc0 0x55db48f432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 --2- 172.21.15.77:0/3831024629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 >> 172.21.15.77:0/3831024629 conn(0x55db48f37a30 msgr2=0x55db48f3d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 shutdown_connections 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 1 -- 172.21.15.77:0/3831024629 wait complete. 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.568+0000 7fbd899ea740 20 client.19260 trim_cache size 0 max 16384 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 Processor -- start 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 -- start start 2024-09-15T22:28:08.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f40550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b1350 con 0x55db48f3b4a0 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b2f50 con 0x55db48f3bdc0 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd899ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b1a20 con 0x55db48f46680 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60148/0 (socket says 172.21.15.77:60148) 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 -- 172.21.15.77:0/1763071395 learned_addr learned my addr 172.21.15.77:0/1763071395 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 -- 172.21.15.77:0/1763071395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 msgr2=0x55db48f40550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f40550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 -- 172.21.15.77:0/1763071395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 msgr2=0x55db48f5e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.585+0000 7fbd84bac640 1 -- 172.21.15.77:0/1763071395 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db48f3bbf0 con 0x55db48f3b4a0 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd84bac640 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400afd0 tx=0x7fbd7400a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac330f0ee779c529 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd66ffd640 1 -- 172.21.15.77:0/1763071395 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd740110d0 con 0x55db48f3b4a0 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd66ffd640 1 -- 172.21.15.77:0/1763071395 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7400f420 con 0x55db48f3b4a0 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd66ffd640 1 -- 172.21.15.77:0/1763071395 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7402ca40 con 0x55db48f3b4a0 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 -- 172.21.15.77:0/1763071395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 msgr2=0x55db48f5dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400afd0 tx=0x7fbd7400a850 comp rx=0 tx=0).stop 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 -- 172.21.15.77:0/1763071395 shutdown_connections 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f40550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3bdc0 0x55db48f5e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 --2- 172.21.15.77:0/1763071395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3b4a0 0x55db48f5dc30 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 -- 172.21.15.77:0/1763071395 >> 172.21.15.77:0/1763071395 conn(0x55db48f37a30 msgr2=0x55db48f36b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 -- 172.21.15.77:0/1763071395 shutdown_connections 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.586+0000 7fbd899ea740 1 -- 172.21.15.77:0/1763071395 wait complete. 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 -- start start 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 Processor -- start 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f4ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f4c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b27c0 con 0x55db48f3bdc0 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801b20f0 con 0x55db48f3b4a0 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801af350 con 0x55db48f46680 2024-09-15T22:28:08.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd899ea740 10 client.0 _ll_register_callbacks cb 0x7fbd7400ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60158/0 (socket says 172.21.15.77:60158) 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 -- 172.21.15.77:0/2491431394 learned_addr learned my addr 172.21.15.77:0/2491431394 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd84bac640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f4ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 -- 172.21.15.77:0/2491431394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 msgr2=0x55db48f4c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f4c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 -- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 msgr2=0x55db48f4ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f4ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.587+0000 7fbd853ad640 1 -- 172.21.15.77:0/2491431394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db48f4cf40 con 0x55db48f3bdc0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd853ad640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ca0 tx=0x7fbd70032ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73de9db86bb229 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031420 con 0x55db48f3bdc0 2024-09-15T22:28:08.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7fbd700315c0 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7400f420 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd70057050 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 10 client.19266 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55db48fc0d50 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7400f420 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fbd7002f630 con 0x55db48f3bdc0 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 10 client.19266 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 10 client.19266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 20 client.19266 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48fc1610 0x55db48fc3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 4 client.19266 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.588+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db48ea8d60 con 0x55db48fc1610 2024-09-15T22:28:08.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd84bac640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48fc1610 0x55db48fc3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd84bac640 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48fc1610 0x55db48fc3a30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1aefef1c926145 server_cookie=f491e2e059484366 in_seq=0 out_seq=0 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd64ff9640 10 client.19266 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd64ff9640 1 -- 172.21.15.77:0/2491431394 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db48ea8d60 con 0x55db48fc1610 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd64ff9640 10 client.19266 handle_command_reply: tid=0 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 client.19266 shutdown 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 msgr2=0x55db48f4c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009ca0 tx=0x7fbd70032ca0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 shutdown_connections 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db48fc1610 0x55db48fc3a30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db48f46680 0x55db48f4c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd64ff9640 0 client.19266 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db48f3bdc0 0x55db48f4c190 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 --2- 172.21.15.77:0/2491431394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db48f3b4a0 0x55db48f4ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 >> 172.21.15.77:0/2491431394 conn(0x55db48f37a30 msgr2=0x55db48f36350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 shutdown_connections 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.589+0000 7fbd899ea740 1 -- 172.21.15.77:0/2491431394 wait complete. 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd899ea740 20 client.19266 trim_cache size 0 max 16384 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 msgr2=0x7fbd680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 0x7fbd680a5870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd7809b950 tx=0x7fbd7806d9d0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 msgr2=0x7fbd801014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c970 tx=0x7fbd7800cd70 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 shutdown_connections 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbd680a33b0 0x7fbd680a5870 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd80108300 0x7fbd801014b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd80103780 0x7fbd80100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 --2- 172.21.15.77:0/2036214082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd80102dd0 0x7fbd80100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:08.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 >> 172.21.15.77:0/2036214082 conn(0x7fbd800fc7d0 msgr2=0x7fbd80105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:08.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 shutdown_connections 2024-09-15T22:28:08.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:08.590+0000 7fbd87611640 1 -- 172.21.15.77:0/2036214082 wait complete. 2024-09-15T22:28:09.582 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:28:09.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:28:09.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 12.33 MB/sec execute 10 sec latency 4828.729 ms 2024-09-15T22:28:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:10.514 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:10 smithi077 ceph-mon[30980]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:10.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 11.21 MB/sec execute 11 sec latency 5828.761 ms 2024-09-15T22:28:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:10 smithi196 ceph-mon[35109]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:10 smithi117 ceph-mon[35021]: pgmap v494: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-09-15T22:28:10.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.594+0000 7f10ee774640 1 -- 172.21.15.77:0/2491947446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 msgr2=0x7f10e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:10.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.594+0000 7f10ee774640 1 --2- 172.21.15.77:0/2491947446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8103120 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d802a480 comp rx=0 tx=0).stop 2024-09-15T22:28:10.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 -- 172.21.15.77:0/2491947446 shutdown_connections 2024-09-15T22:28:10.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 --2- 172.21.15.77:0/2491947446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 --2- 172.21.15.77:0/2491947446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 0x7f10e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 --2- 172.21.15.77:0/2491947446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8103120 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 -- 172.21.15.77:0/2491947446 >> 172.21.15.77:0/2491947446 conn(0x7f10e80fc7a0 msgr2=0x7f10e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 -- 172.21.15.77:0/2491947446 shutdown_connections 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 -- 172.21.15.77:0/2491947446 wait complete. 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 Processor -- start 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.595+0000 7f10ee774640 1 -- start start 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 0x7f10e81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8111f70 con 0x7f10e8102d40 2024-09-15T22:28:10.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8111df0 con 0x7f10e810c170 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ee774640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81120f0 con 0x7f10e8103660 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50776/0 (socket says 172.21.15.77:50776) 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 -- 172.21.15.77:0/2683164115 learned_addr learned my addr 172.21.15.77:0/2683164115 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e77fe640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 0x7f10e81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ecd11640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 -- 172.21.15.77:0/2683164115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 msgr2=0x7f10e81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 0x7f10e81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 -- 172.21.15.77:0/2683164115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 msgr2=0x7f10e819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 -- 172.21.15.77:0/2683164115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e819d420 con 0x7f10e8102d40 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10ecd11640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:10.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.596+0000 7f10e7fff640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.597+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d803f030 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.597+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e819d6b0 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.597+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d8003ce0 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.597+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8035070 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.597+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a4ff0 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.598+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8103120 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.599+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f10d8042030 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.599+0000 7f10e57fa640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 0x7f10d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.600+0000 7f10e77fe640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 0x7f10d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.600+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7f10d80f0a20 con 0x7f10e8102d40 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.600+0000 7f10e77fe640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 0x7f10d00a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0058c0 tx=0x7f10dc005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:10.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.603+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10d80bd0d0 con 0x7f10e8102d40 2024-09-15T22:28:10.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.805+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10e81997d0 con 0x7f10e8102d40 2024-09-15T22:28:10.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.806+0000 7f10e57fa640 1 -- 172.21.15.77:0/2683164115 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11996 (secure 0 0 0) 0x7f10e81997d0 con 0x7f10e8102d40 2024-09-15T22:28:10.808 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:28:10.808 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":36,"btime":"2024-09-15T22:26:06:357374+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:26:05.420334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:28:10.808 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 36 2024-09-15T22:28:10.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 msgr2=0x7f10d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:10.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 0x7f10d00a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0058c0 tx=0x7f10dc005850 comp rx=0 tx=0).stop 2024-09-15T22:28:10.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 msgr2=0x7f10e8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f10d8006010 tx=0x7f10d80042f0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 shutdown_connections 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f10d00a33b0 0x7f10d00a5870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10e810c170 0x7f10e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.808+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e8103660 0x7f10e81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.809+0000 7f10ee774640 1 --2- 172.21.15.77:0/2683164115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10e8102d40 0x7f10e8198470 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.809+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 >> 172.21.15.77:0/2683164115 conn(0x7f10e80fc7a0 msgr2=0x7f10e80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.809+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 shutdown_connections 2024-09-15T22:28:10.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:10.809+0000 7f10ee774640 1 -- 172.21.15.77:0/2683164115 wait complete. 2024-09-15T22:28:11.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:11 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2683164115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:11.531 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:28:11.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 10.28 MB/sec execute 12 sec latency 6828.829 ms 2024-09-15T22:28:11.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:11 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2683164115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:11.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:11 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2683164115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:11.815 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- 172.21.15.77:0/1332182457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 msgr2=0x7f3f64101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/1332182457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64101200 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009900 tx=0x7f3f5402a480 comp rx=0 tx=0).stop 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- 172.21.15.77:0/1332182457 shutdown_connections 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/1332182457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102ef0 0x7f3f6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/1332182457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102570 0x7f3f641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/1332182457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64101200 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- 172.21.15.77:0/1332182457 >> 172.21.15.77:0/1332182457 conn(0x7f3f640fc7d0 msgr2=0x7f3f640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- 172.21.15.77:0/1332182457 shutdown_connections 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- 172.21.15.77:0/1332182457 wait complete. 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 Processor -- start 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.317+0000 7f3f68cd7640 1 -- start start 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102570 0x7f3f641998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102ef0 0x7f3f641a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64110fc0 con 0x7f3f64100e20 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64110e40 con 0x7f3f64102570 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64111140 con 0x7f3f64102ef0 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50796/0 (socket says 172.21.15.77:50796) 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 -- 172.21.15.77:0/665668758 learned_addr learned my addr 172.21.15.77:0/665668758 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102ef0 0x7f3f641a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102570 0x7f3f641998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 -- 172.21.15.77:0/665668758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102ef0 msgr2=0x7f3f641a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102ef0 0x7f3f641a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 -- 172.21.15.77:0/665668758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102570 msgr2=0x7f3f641998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102570 0x7f3f641998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 -- 172.21.15.77:0/665668758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f641a4920 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f6259c640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006010 tx=0x7f3f54003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f5b7fe640 1 -- 172.21.15.77:0/665668758 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5403f070 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f641a4af0 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f5b7fe640 1 -- 172.21.15.77:0/665668758 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f54031c30 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f5b7fe640 1 -- 172.21.15.77:0/665668758 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54035030 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.318+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641a4fa0 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f5b7fe640 1 -- 172.21.15.77:0/665668758 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3f54042070 con 0x7f3f64100e20 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 Processor -- start 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 -- start start 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed9077090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed9082370 0x560ed9084760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ed8fdd2f0 con 0x560ed90775d0 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560ed8f84cf0 con 0x560ed9078b40 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.321+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ed8fd92b0 con 0x560ed9082370 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50806/0 (socket says 172.21.15.77:50806) 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 -- 172.21.15.77:0/2241414251 learned_addr learned my addr 172.21.15.77:0/2241414251 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f5b7fe640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 0x7f3f500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed9082370 0x560ed9084760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed9077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 0x7f3f500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 -- 172.21.15.77:0/2241414251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed9082370 msgr2=0x560ed9084760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed9082370 0x560ed9084760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 -- 172.21.15.77:0/2241414251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 msgr2=0x560ed9077090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed9077090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f5b7fe640 1 -- 172.21.15.77:0/665668758 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7f3f540f0970 con 0x7f3f64100e20 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 -- 172.21.15.77:0/2241414251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed8f693a0 con 0x560ed90775d0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f6259c640 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3f540bc560 tx=0x7f3f54071c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce7b400ec18f7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 0x7f3f500a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009900 tx=0x7f3f5c009650 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f58ff9640 1 -- 172.21.15.77:0/2241414251 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540bcf60 con 0x560ed90775d0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.322+0000 7f3f58ff9640 1 -- 172.21.15.77:0/2241414251 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f54031de0 con 0x560ed90775d0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f58ff9640 1 -- 172.21.15.77:0/2241414251 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540b8ed0 con 0x560ed90775d0 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2241414251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 msgr2=0x560ed9077a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3f540bc560 tx=0x7f3f54071c10 comp rx=0 tx=0).stop 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2241414251 shutdown_connections 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed9082370 0x560ed9084760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed90775d0 0x560ed9077a10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2241414251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed9077090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2241414251 >> 172.21.15.77:0/2241414251 conn(0x560ed9040310 msgr2=0x560ed9073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2241414251 shutdown_connections 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.323+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2241414251 wait complete. 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 -- start start 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 Processor -- start 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed9081d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ed9084e30 con 0x560ed9082370 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560ed8f98060 con 0x560ed9078b40 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560ed9084fb0 con 0x560ed90775d0 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6b0b0740 10 client.0 _ll_register_callbacks cb 0x560ed8ceb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:12.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50808/0 (socket says 172.21.15.77:50808) 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 -- 172.21.15.77:0/3840094695 learned_addr learned my addr 172.21.15.77:0/3840094695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6259c640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed9081d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 -- 172.21.15.77:0/3840094695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 msgr2=0x560ed9081d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed9081d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 -- 172.21.15.77:0/3840094695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 msgr2=0x560ed907f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f62d9d640 1 -- 172.21.15.77:0/3840094695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed8d4e3c0 con 0x560ed9082370 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.324+0000 7f3f6259c640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed9081d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c017bd0 tx=0x7f3f5c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd05bb43d294681 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:12.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c02b060 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7f3f5c01b9c0 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f54031de0 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f5c032050 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 10 client.19284 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560ed9090820 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f54031de0 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f3f5c019380 con 0x560ed9082370 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 10 client.19284 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 10 client.19284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 20 client.19284 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed9091ce0 0x560ed9094160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 4 client.19284 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.325+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ed8fea860 con 0x560ed9091ce0 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f6259c640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed9091ce0 0x560ed9094160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f49ffb640 10 client.19284 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f6259c640 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed9091ce0 0x560ed9094160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43210bf750ae25da server_cookie=ca0ae18af892b2ea in_seq=0 out_seq=0 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f49ffb640 1 -- 172.21.15.77:0/3840094695 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ed8fea860 con 0x560ed9091ce0 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f49ffb640 10 client.19284 handle_command_reply: tid=0 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f6b0b0740 1 client.19284 shutdown 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 msgr2=0x560ed907f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.326+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c017bd0 tx=0x7f3f5c004b50 comp rx=0 tx=0).stop 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 shutdown_connections 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed9091ce0 0x560ed9094160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907f600 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f49ffb640 0 client.19284 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/3840094695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed9081d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 >> 172.21.15.77:0/3840094695 conn(0x560ed9040310 msgr2=0x560ed9079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 shutdown_connections 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/3840094695 wait complete. 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.327+0000 7f3f6b0b0740 20 client.19284 trim_cache size 0 max 16384 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.339+0000 7f3f6b0b0740 1 Processor -- start 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.339+0000 7f3f6b0b0740 1 -- start start 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed907df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9082370 0x560ed907a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641af120 con 0x560ed9078b40 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641b0d20 con 0x560ed9082370 2024-09-15T22:28:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641af7f0 con 0x560ed90775d0 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50824/0 (socket says 172.21.15.77:50824) 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 -- 172.21.15.77:0/2545257101 learned_addr learned my addr 172.21.15.77:0/2545257101 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9082370 0x560ed907a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6259c640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed907df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 -- 172.21.15.77:0/2545257101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 msgr2=0x560ed907df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed907df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 -- 172.21.15.77:0/2545257101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9082370 msgr2=0x560ed907a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9082370 0x560ed907a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 -- 172.21.15.77:0/2545257101 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed90947e0 con 0x560ed9078b40 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f6259c640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed907df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406ab00 tx=0x7f3f4c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4fc9e00f8396e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f4bfff640 1 -- 172.21.15.77:0/2545257101 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c020070 con 0x560ed9078b40 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f4bfff640 1 -- 172.21.15.77:0/2545257101 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c00bd80 con 0x560ed9078b40 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.340+0000 7f3f4bfff640 1 -- 172.21.15.77:0/2545257101 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0107d0 con 0x560ed9078b40 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2545257101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 msgr2=0x560ed9072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3f6406ab00 tx=0x7f3f4c012820 comp rx=0 tx=0).stop 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2545257101 shutdown_connections 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9082370 0x560ed907a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9078b40 0x560ed9072bf0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/2545257101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed907df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2545257101 >> 172.21.15.77:0/2545257101 conn(0x560ed9040310 msgr2=0x560ed90724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2545257101 shutdown_connections 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/2545257101 wait complete. 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 -- start start 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 Processor -- start 2024-09-15T22:28:12.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed908a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.341+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641b0590 con 0x560ed9082370 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641afec0 con 0x560ed9078b40 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641ad120 con 0x560ed90775d0 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 10 client.0 _ll_register_callbacks cb 0x7f3f5c00b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50826/0 (socket says 172.21.15.77:50826) 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 -- 172.21.15.77:0/722991816 learned_addr learned my addr 172.21.15.77:0/722991816 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6259c640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed908a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 -- 172.21.15.77:0/722991816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 msgr2=0x560ed908a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed908a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 -- 172.21.15.77:0/722991816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 msgr2=0x560ed907bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 -- 172.21.15.77:0/722991816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed907cb50 con 0x560ed9082370 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6259c640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed908a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f61d9b640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f62d9d640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c018010 tx=0x7f3f5c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b2a9be7a83413b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0199c0 con 0x560ed9082370 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9526+0+0 (secure 0 0 0) 0x7f3f5c019b60 con 0x560ed9082370 2024-09-15T22:28:12.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f4c00bd80 con 0x560ed9082370 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f5c046050 con 0x560ed9082370 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 10 client.19290 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.342+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560ed90fca00 con 0x560ed9082370 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c00bd80 con 0x560ed9082370 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f3f5c038960 con 0x560ed9082370 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 10 client.19290 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 10 client.19290 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 20 client.19290 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed90fd170 0x560ed90ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 4 client.19290 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ed8fe4d70 con 0x560ed90fd170 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6259c640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed90fd170 0x560ed90ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f49ffb640 10 client.19290 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:28:12.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6259c640 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed90fd170 0x560ed90ff5b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3cc68eefac8ae304 server_cookie=6960089d9a2cca5c in_seq=0 out_seq=0 2024-09-15T22:28:12.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f49ffb640 1 -- 172.21.15.77:0/722991816 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ed8fe4d70 con 0x560ed90fd170 2024-09-15T22:28:12.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f49ffb640 10 client.19290 handle_command_reply: tid=0 2024-09-15T22:28:12.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 1 client.19290 shutdown 2024-09-15T22:28:12.365 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:28:12.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 msgr2=0x560ed907c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.343+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c018010 tx=0x7f3f5c00b920 comp rx=0 tx=0).stop 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 shutdown_connections 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x560ed90fd170 0x560ed90ff5b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ed9082370 0x560ed907c320 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ed9078b40 0x560ed907bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 --2- 172.21.15.77:0/722991816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560ed90775d0 0x560ed908a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 >> 172.21.15.77:0/722991816 conn(0x560ed9040310 msgr2=0x560ed9081090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f49ffb640 0 client.19290 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 shutdown_connections 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 1 -- 172.21.15.77:0/722991816 wait complete. 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f6b0b0740 20 client.19290 trim_cache size 0 max 16384 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 msgr2=0x7f3f500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 0x7f3f500a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009900 tx=0x7f3f5c009650 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 msgr2=0x7f3f64199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006010 tx=0x7f3f54003c10 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 shutdown_connections 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3f500a33b0 0x7f3f500a5870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f64102ef0 0x7f3f641a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f64102570 0x7f3f641998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 --2- 172.21.15.77:0/665668758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f64100e20 0x7f3f64199360 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 >> 172.21.15.77:0/665668758 conn(0x7f3f640fc7d0 msgr2=0x7f3f640fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 shutdown_connections 2024-09-15T22:28:12.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:12.344+0000 7f3f68cd7640 1 -- 172.21.15.77:0/665668758 wait complete. 2024-09-15T22:28:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:12 smithi077 ceph-mon[30980]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:12.595 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 9.49 MB/sec execute 13 sec latency 7828.868 ms 2024-09-15T22:28:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:12 smithi196 ceph-mon[35109]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:12.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:12 smithi117 ceph-mon[35021]: pgmap v495: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:13.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 8.81 MB/sec execute 14 sec latency 8828.938 ms 2024-09-15T22:28:13.612 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:28:13.903 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:14 smithi077 ceph-mon[30980]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:14 smithi077 ceph-mon[30980]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:28:14.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 8.22 MB/sec execute 15 sec latency 9828.998 ms 2024-09-15T22:28:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:14 smithi196 ceph-mon[35109]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:14 smithi196 ceph-mon[35109]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:28:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:14 smithi117 ceph-mon[35021]: pgmap v496: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-09-15T22:28:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:14 smithi117 ceph-mon[35021]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:28:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 -- 172.21.15.77:0/2845279830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30101df0 msgr2=0x7f1a301021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 --2- 172.21.15.77:0/2845279830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30101df0 0x7f1a301021d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009900 tx=0x7f1a2002a410 comp rx=0 tx=0).stop 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 -- 172.21.15.77:0/2845279830 shutdown_connections 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 --2- 172.21.15.77:0/2845279830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 --2- 172.21.15.77:0/2845279830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a301027a0 0x7f1a301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 --2- 172.21.15.77:0/2845279830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a30101df0 0x7f1a301021d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 -- 172.21.15.77:0/2845279830 >> 172.21.15.77:0/2845279830 conn(0x7f1a300fd8b0 msgr2=0x7f1a300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.800+0000 7f1a368c8640 1 -- 172.21.15.77:0/2845279830 shutdown_connections 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.801+0000 7f1a368c8640 1 -- 172.21.15.77:0/2845279830 wait complete. 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.801+0000 7f1a368c8640 1 Processor -- start 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.801+0000 7f1a368c8640 1 -- start start 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30101df0 0x7f1a30198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30111f80 con 0x7f1a301027a0 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30111e00 con 0x7f1a30101df0 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a368c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30112100 con 0x7f1a30106f20 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50840/0 (socket says 172.21.15.77:50840) 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/1039456139 learned_addr learned my addr 172.21.15.77:0/1039456139 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30101df0 0x7f1a30198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a34e65640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/1039456139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 msgr2=0x7f1a3019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/1039456139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30101df0 msgr2=0x7f1a30198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30101df0 0x7f1a30198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/1039456139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3019d410 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.802+0000 7f1a34e65640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400d950 tx=0x7f1a2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a2d7fa640 1 -- 172.21.15.77:0/1039456139 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a2d7fa640 1 -- 172.21.15.77:0/1039456139 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24009c30 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3019d700 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a2d7fa640 1 -- 172.21.15.77:0/1039456139 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400f6b0 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.803+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a4fe0 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a2d7fa640 1 -- 172.21.15.77:0/1039456139 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1a240109d0 con 0x7f1a301027a0 2024-09-15T22:28:14.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 Processor -- start 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 -- start start 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42e97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42e97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b42dfd2f0 con 0x561b42ea2370 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b42da4cf0 con 0x561b42e98b40 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.806+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b42df92b0 con 0x561b42e975d0 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50846/0 (socket says 172.21.15.77:50846) 2024-09-15T22:28:14.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/2989255597 learned_addr learned my addr 172.21.15.77:0/2989255597 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2d7fa640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 0x7f1a180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42e97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a34e65640 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42e97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a34e65640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 0x7f1a180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2d7fa640 1 -- 172.21.15.77:0/1039456139 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1a240cfd40 con 0x7f1a301027a0 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/2989255597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 msgr2=0x561b42e97a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42e97a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/2989255597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 msgr2=0x561b42e97090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42e97090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/2989255597 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b42d893a0 con 0x561b42ea2370 2024-09-15T22:28:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1a24098660 tx=0x7f1a24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeab791a1cbe844 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a34e65640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 0x7f1a180a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a28005c40 tx=0x7f1a28009280 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a167fc640 1 -- 172.21.15.77:0/2989255597 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24020070 con 0x561b42ea2370 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a167fc640 1 -- 172.21.15.77:0/2989255597 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24009de0 con 0x561b42ea2370 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.807+0000 7f1a167fc640 1 -- 172.21.15.77:0/2989255597 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24098300 con 0x561b42ea2370 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2989255597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 msgr2=0x561b42ea4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1a24098660 tx=0x7f1a24000bc0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.908 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:28:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2989255597 shutdown_connections 2024-09-15T22:28:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42ea2370 0x561b42ea4760 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42e97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2989255597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42e97090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2989255597 >> 172.21.15.77:0/2989255597 conn(0x561b42e60310 msgr2=0x561b42e93c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2989255597 shutdown_connections 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.808+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2989255597 wait complete. 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 -- start start 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 Processor -- start 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42ea1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42e9f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b42ea4fb0 con 0x561b42e98b40 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b42db8060 con 0x561b42ea2370 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b42ea5130 con 0x561b42e975d0 2024-09-15T22:28:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a38ca1740 10 client.0 _ll_register_callbacks cb 0x561b42b0b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50848/0 (socket says 172.21.15.77:50848) 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/573421436 learned_addr learned my addr 172.21.15.77:0/573421436 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a34e65640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42e9f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.809+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42ea1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 msgr2=0x561b42ea1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42ea1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/573421436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 msgr2=0x561b42e9f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42e9f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 -- 172.21.15.77:0/573421436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b42b6e3c0 con 0x561b42e98b40 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f1a240cd700 tx=0x7f1a2409b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b052f5adebb5d05a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24020070 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1a240979e0 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a24009de0 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a24097f30 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 10 client.19302 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561b42eb0a80 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24009de0 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f1a240d3020 con 0x561b42e98b40 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 10 client.19302 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 10 client.19302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 20 client.19302 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42eb1dd0 0x561b42eb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 4 client.19302 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.810+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b42e0a860 con 0x561b42eb1dd0 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.811+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42eb1dd0 0x561b42eb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.811+0000 7f1a0f7fe640 10 client.19302 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.811+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42eb1dd0 0x561b42eb41c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db6f7a53f9d5c9db server_cookie=a681f0169f6da1fd in_seq=0 out_seq=0 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/573421436 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b42e0a860 con 0x561b42eb1dd0 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a0f7fe640 10 client.19302 handle_command_reply: tid=0 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 client.19302 shutdown 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 msgr2=0x561b42e9f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f1a240cd700 tx=0x7f1a2409b670 comp rx=0 tx=0).stop 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 shutdown_connections 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42eb1dd0 0x561b42eb41c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42e9f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a0f7fe640 0 client.19302 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e98b40 0x561b42e9f030 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/573421436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e975d0 0x561b42ea1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 >> 172.21.15.77:0/573421436 conn(0x561b42e60310 msgr2=0x561b42e99990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 shutdown_connections 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.812+0000 7f1a38ca1740 1 -- 172.21.15.77:0/573421436 wait complete. 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.813+0000 7f1a38ca1740 20 client.19302 trim_cache size 0 max 16384 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.825+0000 7f1a38ca1740 1 Processor -- start 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.825+0000 7f1a38ca1740 1 -- start start 2024-09-15T22:28:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 0x561b42eb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42eb7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301aefb0 con 0x561b42e975d0 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b0bb0 con 0x561b42ea2370 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301af680 con 0x561b42e98b40 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50850/0 (socket says 172.21.15.77:50850) 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 -- 172.21.15.77:0/2462548068 learned_addr learned my addr 172.21.15.77:0/2462548068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a34e65640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42eb7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 0x561b42eb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 -- 172.21.15.77:0/2462548068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 msgr2=0x561b42eb71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 0x561b42eb71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 -- 172.21.15.77:0/2462548068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 msgr2=0x561b42eb7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42eb7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 -- 172.21.15.77:0/2462548068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b42e9f3c0 con 0x561b42e975d0 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 0x561b42eb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a34e65640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42eb7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a200058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76982916252a9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.826+0000 7f1a15ffb640 1 -- 172.21.15.77:0/2462548068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2004a070 con 0x561b42e975d0 2024-09-15T22:28:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a15ffb640 1 -- 172.21.15.77:0/2462548068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a20005500 con 0x561b42e975d0 2024-09-15T22:28:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a15ffb640 1 -- 172.21.15.77:0/2462548068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20035040 con 0x561b42e975d0 2024-09-15T22:28:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2462548068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 msgr2=0x561b42eae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c00 tx=0x7f1a200058d0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2462548068 shutdown_connections 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42ea2370 0x561b42eb7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42e98b40 0x561b42eb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/2462548068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eae3d0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2462548068 >> 172.21.15.77:0/2462548068 conn(0x561b42e60310 msgr2=0x561b42e9e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2462548068 shutdown_connections 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.827+0000 7f1a38ca1740 1 -- 172.21.15.77:0/2462548068 wait complete. 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 -- start start 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 Processor -- start 2024-09-15T22:28:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42eb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42ea2370 0x561b42f1b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301b0420 con 0x561b42e975d0 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301afd50 con 0x561b42e98b40 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301acfb0 con 0x561b42ea2370 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a38ca1740 10 client.0 _ll_register_callbacks cb 0x7f1a28009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50852/0 (socket says 172.21.15.77:50852) 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 -- 172.21.15.77:0/3931368149 learned_addr learned my addr 172.21.15.77:0/3931368149 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42eb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 -- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42ea2370 msgr2=0x561b42f1b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42ea2370 0x561b42f1b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 -- 172.21.15.77:0/3931368149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 msgr2=0x561b42eb58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.828+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42eb58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a2ffff640 1 -- 172.21.15.77:0/3931368149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b42f1bc50 con 0x561b42e975d0 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a2ffff640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009900 tx=0x7f1a20005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c6f20b25604d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2004a050 con 0x561b42e975d0 2024-09-15T22:28:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1a20004240 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a20005500 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a20035040 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 10 client.19308 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561b42f1cd80 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a20005500 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f1a20004c10 con 0x561b42e975d0 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 10 client.19308 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 10 client.19308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:28:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.829+0000 7f1a38ca1740 20 client.19308 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42f1d4c0 0x561b42f1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 4 client.19308 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b42e04d70 con 0x561b42f1d4c0 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42f1d4c0 0x561b42f1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a2f7fe640 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42f1d4c0 0x561b42f1f900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d11e12fad6b323 server_cookie=82383725687a30cf in_seq=0 out_seq=0 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a0f7fe640 10 client.19308 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a0f7fe640 1 -- 172.21.15.77:0/3931368149 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b42e04d70 con 0x561b42f1d4c0 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a0f7fe640 10 client.19308 handle_command_reply: tid=0 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 client.19308 shutdown 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 msgr2=0x561b42eb5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009900 tx=0x7f1a20005730 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 shutdown_connections 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b42f1d4c0 0x561b42f1f900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b42ea2370 0x561b42f1b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b42e98b40 0x561b42eb58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.830+0000 7f1a38ca1740 1 --2- 172.21.15.77:0/3931368149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b42e975d0 0x561b42eb5390 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 >> 172.21.15.77:0/3931368149 conn(0x561b42e60310 msgr2=0x561b42e924e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a0f7fe640 0 client.19308 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 shutdown_connections 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a38ca1740 1 -- 172.21.15.77:0/3931368149 wait complete. 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a38ca1740 20 client.19308 trim_cache size 0 max 16384 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 msgr2=0x7f1a180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 0x7f1a180a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a28005c40 tx=0x7f1a28009280 comp rx=0 tx=0).stop 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 msgr2=0x7f1a301989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:14.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400d950 tx=0x7f1a2400de10 comp rx=0 tx=0).stop 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 shutdown_connections 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a180a33b0 0x7f1a180a5870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a30106f20 0x7f1a3019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a301027a0 0x7f1a301989a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 --2- 172.21.15.77:0/1039456139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30101df0 0x7f1a30198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 >> 172.21.15.77:0/1039456139 conn(0x7f1a300fd8b0 msgr2=0x7f1a300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 shutdown_connections 2024-09-15T22:28:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:14.831+0000 7f1a368c8640 1 -- 172.21.15.77:0/1039456139 wait complete. 2024-09-15T22:28:15.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 7.71 MB/sec execute 16 sec latency 10829.051 ms 2024-09-15T22:28:15.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:28:16.154 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:16 smithi077 ceph-mon[30980]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-15T22:28:16.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 7.25 MB/sec execute 17 sec latency 11829.104 ms 2024-09-15T22:28:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:16 smithi196 ceph-mon[35109]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-15T22:28:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:16 smithi117 ceph-mon[35021]: pgmap v498: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-09-15T22:28:16.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- 172.21.15.77:0/2851570189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc101de0 msgr2=0x7f48cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 --2- 172.21.15.77:0/2851570189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc101de0 0x7f48cc1021c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009900 tx=0x7f48c802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- 172.21.15.77:0/2851570189 shutdown_connections 2024-09-15T22:28:16.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 --2- 172.21.15.77:0/2851570189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc106f10 0x7f48cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 --2- 172.21.15.77:0/2851570189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc102790 0x7f48cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 --2- 172.21.15.77:0/2851570189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc101de0 0x7f48cc1021c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- 172.21.15.77:0/2851570189 >> 172.21.15.77:0/2851570189 conn(0x7f48cc0fd8c0 msgr2=0x7f48cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- 172.21.15.77:0/2851570189 shutdown_connections 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- 172.21.15.77:0/2851570189 wait complete. 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 Processor -- start 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.669+0000 7f48d32d1640 1 -- start start 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc101de0 0x7f48cc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc102790 0x7f48cc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc111f70 con 0x7f48cc106f10 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc111df0 con 0x7f48cc102790 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1120f0 con 0x7f48cc101de0 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50876/0 (socket says 172.21.15.77:50876) 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 -- 172.21.15.77:0/3957001226 learned_addr learned my addr 172.21.15.77:0/3957001226 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:16.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d086c640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc102790 0x7f48cc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d106d640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc101de0 0x7f48cc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 -- 172.21.15.77:0/3957001226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc101de0 msgr2=0x7f48cc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc101de0 0x7f48cc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 -- 172.21.15.77:0/3957001226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc102790 msgr2=0x7f48cc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc102790 0x7f48cc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 -- 172.21.15.77:0/3957001226 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc19d380 con 0x7f48cc106f10 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d186e640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f48c400cce0 tx=0x7f48c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48c27fc640 1 -- 172.21.15.77:0/3957001226 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400f450 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48c27fc640 1 -- 172.21.15.77:0/3957001226 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c400fad0 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc19d670 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.670+0000 7f48c27fc640 1 -- 172.21.15.77:0/3957001226 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4018c10 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.671+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1a4f50 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48c27fc640 1 -- 172.21.15.77:0/3957001226 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f48c400f5f0 con 0x7f48cc106f10 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 Processor -- start 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 -- start start 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 0x5597b1703760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5597b165c2f0 con 0x5597b16f65d0 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597b1603cf0 con 0x5597b16f7b40 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d56aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597b16582b0 con 0x5597b1701370 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d106d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48c27fc640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 0x7f48b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d106d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50884/0 (socket says 172.21.15.77:50884) 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.673+0000 7f48d106d640 1 -- 172.21.15.77:0/1816649750 learned_addr learned my addr 172.21.15.77:0/1816649750 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d186e640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 0x7f48b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d086c640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d086c640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 0x5597b1703760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 -- 172.21.15.77:0/1816649750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 msgr2=0x5597b1703760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 0x5597b1703760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 -- 172.21.15.77:0/1816649750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 msgr2=0x5597b16f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 -- 172.21.15.77:0/1816649750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b15e83a0 con 0x5597b16f65d0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d086c640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 0x5597b1703760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48c27fc640 1 -- 172.21.15.77:0/3957001226 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f48c40d0630 con 0x7f48cc106f10 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d106d640 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009900 tx=0x7f48c8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e593ac986e05ab35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48d186e640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 0x7f48b80a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48c40cdff0 tx=0x7f48c400af30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48b77fe640 1 -- 172.21.15.77:0/1816649750 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8040070 con 0x5597b16f65d0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48b77fe640 1 -- 172.21.15.77:0/1816649750 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c8005020 con 0x5597b16f65d0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.674+0000 7f48b77fe640 1 -- 172.21.15.77:0/1816649750 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8035040 con 0x5597b16f65d0 2024-09-15T22:28:16.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- 172.21.15.77:0/1816649750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 msgr2=0x5597b16f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009900 tx=0x7f48c8031630 comp rx=0 tx=0).stop 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- 172.21.15.77:0/1816649750 shutdown_connections 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b1701370 0x5597b1703760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b16f6a10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 --2- 172.21.15.77:0/1816649750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- 172.21.15.77:0/1816649750 >> 172.21.15.77:0/1816649750 conn(0x5597b16bf310 msgr2=0x5597b16f2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- 172.21.15.77:0/1816649750 shutdown_connections 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- 172.21.15.77:0/1816649750 wait complete. 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 -- start start 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.675+0000 7f48d56aa740 1 Processor -- start 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f7b40 0x5597b16fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b1701370 0x5597b16fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5597b1704010 con 0x5597b16f65d0 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597b1617060 con 0x5597b1701370 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5597b1704190 con 0x5597b16f7b40 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 10 client.0 _ll_register_callbacks cb 0x5597b136a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50888/0 (socket says 172.21.15.77:50888) 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 -- 172.21.15.77:0/2564667874 learned_addr learned my addr 172.21.15.77:0/2564667874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d106d640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f7b40 0x5597b16fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 -- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f7b40 msgr2=0x5597b16fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f7b40 0x5597b16fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 -- 172.21.15.77:0/2564667874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b1701370 msgr2=0x5597b16fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b1701370 0x5597b16fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 -- 172.21.15.77:0/2564667874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b13cd3c0 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d086c640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00bfb0 tx=0x7f48bc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52dfcb8c7a1340c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc018070 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f48bc004230 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.676+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c8005020 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f48bc024050 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 10 client.19320 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5597b170f9f0 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c8005020 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f48bc003a60 con 0x5597b16f65d0 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 10 client.19320 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:16.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 10 client.19320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 20 client.19320 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1710eb0 0x5597b17132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 4 client.19320 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597b1669860 con 0x5597b1710eb0 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.677+0000 7f48d106d640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1710eb0 0x5597b17132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.678+0000 7f48b4ff9640 10 client.19320 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.678+0000 7f48d106d640 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1710eb0 0x5597b17132a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a491aa851c600f71 server_cookie=ff6bf3bd92f2870f in_seq=0 out_seq=0 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.678+0000 7f48b4ff9640 1 -- 172.21.15.77:0/2564667874 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597b1669860 con 0x5597b1710eb0 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.678+0000 7f48b4ff9640 10 client.19320 handle_command_reply: tid=0 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.678+0000 7f48d56aa740 1 client.19320 shutdown 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 msgr2=0x5597b1700ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00bfb0 tx=0x7f48bc0128e0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 shutdown_connections 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1710eb0 0x5597b17132a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b1701370 0x5597b16fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f7b40 0x5597b16fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48b4ff9640 0 client.19320 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 --2- 172.21.15.77:0/2564667874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16f65d0 0x5597b1700ce0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 >> 172.21.15.77:0/2564667874 conn(0x5597b16bf310 msgr2=0x5597b16f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 shutdown_connections 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 1 -- 172.21.15.77:0/2564667874 wait complete. 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.679+0000 7f48d56aa740 20 client.19320 trim_cache size 0 max 16384 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 Processor -- start 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 -- start start 2024-09-15T22:28:16.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b170d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1aef40 con 0x5597b16bf310 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b0b40 con 0x5597b16f7b40 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d56aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1af610 con 0x5597b16f65d0 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50890/0 (socket says 172.21.15.77:50890) 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 -- 172.21.15.77:0/3547599988 learned_addr learned my addr 172.21.15.77:0/3547599988 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 -- 172.21.15.77:0/3547599988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 msgr2=0x5597b170d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b170d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 -- 172.21.15.77:0/3547599988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 msgr2=0x5597b16edad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 -- 172.21.15.77:0/3547599988 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b16fc4e0 con 0x5597b16bf310 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48d086c640 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f48bc003e00 tx=0x7f48bc00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc995972fd3290b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48b6ffd640 1 -- 172.21.15.77:0/3547599988 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc02c070 con 0x5597b16bf310 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48b6ffd640 1 -- 172.21.15.77:0/3547599988 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc021040 con 0x5597b16bf310 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.692+0000 7f48b6ffd640 1 -- 172.21.15.77:0/3547599988 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc018040 con 0x5597b16bf310 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- 172.21.15.77:0/3547599988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 msgr2=0x5597b16f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f48bc003e00 tx=0x7f48bc00b9e0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- 172.21.15.77:0/3547599988 shutdown_connections 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b170d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3547599988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b16f2e10 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- 172.21.15.77:0/3547599988 >> 172.21.15.77:0/3547599988 conn(0x5597b16f99a0 msgr2=0x5597b16f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- 172.21.15.77:0/3547599988 shutdown_connections 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- 172.21.15.77:0/3547599988 wait complete. 2024-09-15T22:28:16.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 -- start start 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.693+0000 7f48d56aa740 1 Processor -- start 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b16facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b03b0 con 0x5597b16bf310 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1afce0 con 0x5597b16f7b40 2024-09-15T22:28:16.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1acf40 con 0x5597b16f65d0 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d56aa740 10 client.0 _ll_register_callbacks cb 0x7f48bc00b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50900/0 (socket says 172.21.15.77:50900) 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 -- 172.21.15.77:0/3522254294 learned_addr learned my addr 172.21.15.77:0/3522254294 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 -- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 msgr2=0x5597b16facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b16facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 -- 172.21.15.77:0/3522254294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 msgr2=0x5597b16fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:16.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16fb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 -- 172.21.15.77:0/3522254294 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b16fba20 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48d086c640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f48bc004490 tx=0x7f48bc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9399f79e7633c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.694+0000 7f48b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc018040 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f48bc01ccd0 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc021040 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f48bc042050 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 10 client.19326 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5597b177ba60 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc021040 con 0x5597b16bf310 2024-09-15T22:28:16.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f48bc003820 con 0x5597b16bf310 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 10 client.19326 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 10 client.19326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 20 client.19326 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1701370 0x5597b177e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 4 client.19326 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597b1663d70 con 0x5597b1701370 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.695+0000 7f48d106d640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1701370 0x5597b177e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d106d640 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1701370 0x5597b177e0b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a25600806f2058b4 server_cookie=1c08f6cb30e1d7bc in_seq=0 out_seq=0 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48b4ff9640 10 client.19326 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48b4ff9640 1 -- 172.21.15.77:0/3522254294 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597b1663d70 con 0x5597b1701370 2024-09-15T22:28:16.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48b4ff9640 10 client.19326 handle_command_reply: tid=0 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 client.19326 shutdown 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 msgr2=0x5597b1709b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f48bc004490 tx=0x7f48bc00b7c0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 shutdown_connections 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5597b1701370 0x5597b177e0b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48b4ff9640 0 client.19326 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597b16f7b40 0x5597b16fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5597b16f65d0 0x5597b16facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 --2- 172.21.15.77:0/3522254294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597b16bf310 0x5597b1709b60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 >> 172.21.15.77:0/3522254294 conn(0x5597b16f99a0 msgr2=0x5597b16f26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 shutdown_connections 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.696+0000 7f48d56aa740 1 -- 172.21.15.77:0/3522254294 wait complete. 2024-09-15T22:28:16.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d56aa740 20 client.19326 trim_cache size 0 max 16384 2024-09-15T22:28:16.725 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 msgr2=0x7f48b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 0x7f48b80a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48c40cdff0 tx=0x7f48c400af30 comp rx=0 tx=0).stop 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 msgr2=0x7f48cc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f48c400cce0 tx=0x7f48c400a620 comp rx=0 tx=0).stop 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 shutdown_connections 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48b80a33b0 0x7f48b80a5870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48cc106f10 0x7f48cc19cca0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48cc102790 0x7f48cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 --2- 172.21.15.77:0/3957001226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48cc101de0 0x7f48cc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:16.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 >> 172.21.15.77:0/3957001226 conn(0x7f48cc0fd8c0 msgr2=0x7f48cc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:16.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 shutdown_connections 2024-09-15T22:28:16.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:16.697+0000 7f48d32d1640 1 -- 172.21.15.77:0/3957001226 wait complete. 2024-09-15T22:28:17.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 6.85 MB/sec execute 18 sec latency 12829.138 ms 2024-09-15T22:28:17.598 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:28:17.857 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:18.517 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:18 smithi077 ceph-mon[30980]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- 172.21.15.77:0/3825875221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 msgr2=0x7f551410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 --2- 172.21.15.77:0/3825875221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f551410ea90 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f550c009ca0 tx=0x7f550c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- 172.21.15.77:0/3825875221 shutdown_connections 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 --2- 172.21.15.77:0/3825875221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f551410ea90 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 --2- 172.21.15.77:0/3825875221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 0x7f55141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 --2- 172.21.15.77:0/3825875221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 0x7f55141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- 172.21.15.77:0/3825875221 >> 172.21.15.77:0/3825875221 conn(0x7f55140fd8c0 msgr2=0x7f55140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- 172.21.15.77:0/3825875221 shutdown_connections 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- 172.21.15.77:0/3825875221 wait complete. 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 Processor -- start 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.514+0000 7f551c171640 1 -- start start 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 0x7f55140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 0x7f5514074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5514112010 con 0x7f5514106f10 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5514111e90 con 0x7f5514101de0 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551c171640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5514112190 con 0x7f5514102790 2024-09-15T22:28:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50928/0 (socket says 172.21.15.77:50928) 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 -- 172.21.15.77:0/2243633878 learned_addr learned my addr 172.21.15.77:0/2243633878 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551970c640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 0x7f5514074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f5519f0d640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 0x7f55140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 -- 172.21.15.77:0/2243633878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 msgr2=0x7f5514074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 0x7f5514074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 -- 172.21.15.77:0/2243633878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 msgr2=0x7f55140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 0x7f55140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 -- 172.21.15.77:0/2243633878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141a6ab0 con 0x7f5514106f10 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.515+0000 7f551a70e640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f550c031a60 tx=0x7f550c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.516+0000 7f550affd640 1 -- 172.21.15.77:0/2243633878 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c003b70 con 0x7f5514106f10 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.516+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55141a6d40 con 0x7f5514106f10 2024-09-15T22:28:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.516+0000 7f550affd640 1 -- 172.21.15.77:0/2243633878 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c003d10 con 0x7f5514106f10 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.516+0000 7f550affd640 1 -- 172.21.15.77:0/2243633878 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c004770 con 0x7f5514106f10 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.516+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141a7280 con 0x7f5514106f10 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.518+0000 7f550affd640 1 -- 172.21.15.77:0/2243633878 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f550c02f790 con 0x7f5514106f10 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.518+0000 7f551e54a740 1 Processor -- start 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.518+0000 7f551e54a740 1 -- start start 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f550affd640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 0x7f55000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bf7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0b5d2f0 con 0x55f3d0c02370 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0b04cf0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551e54a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0b592b0 con 0x55f3d0bf75d0 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551970c640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 0x7f55000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f550affd640 1 -- 172.21.15.77:0/2243633878 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f550c0f1330 con 0x7f5514106f10 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551970c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50934/0 (socket says 172.21.15.77:50934) 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 -- 172.21.15.77:0/1179614616 learned_addr learned my addr 172.21.15.77:0/1179614616 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f5519f0d640 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bf7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 -- 172.21.15.77:0/1179614616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 msgr2=0x55f3d0bf7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 -- 172.21.15.77:0/1179614616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf8b40 msgr2=0x55f3d0bf7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bf7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 -- 172.21.15.77:0/1179614616 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d0ae93a0 con 0x55f3d0c02370 2024-09-15T22:28:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551970c640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 0x7f55000a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5504006e10 tx=0x7f5504009930 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.519+0000 7f551a70e640 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f550c0ee800 tx=0x7f550c0bd1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73542b59e885548f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f54fffff640 1 -- 172.21.15.77:0/1179614616 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c02fe90 con 0x55f3d0c02370 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f54fffff640 1 -- 172.21.15.77:0/1179614616 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c036070 con 0x55f3d0c02370 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f54fffff640 1 -- 172.21.15.77:0/1179614616 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0bb630 con 0x55f3d0c02370 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 -- 172.21.15.77:0/1179614616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 msgr2=0x55f3d0c04760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f550c0ee800 tx=0x7f550c0bd1c0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 -- 172.21.15.77:0/1179614616 shutdown_connections 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0c02370 0x55f3d0c04760 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 --2- 172.21.15.77:0/1179614616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bf7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 -- 172.21.15.77:0/1179614616 >> 172.21.15.77:0/1179614616 conn(0x55f3d0bc0310 msgr2=0x55f3d0bf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.520+0000 7f551e54a740 1 -- 172.21.15.77:0/1179614616 shutdown_connections 2024-09-15T22:28:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 -- 172.21.15.77:0/1179614616 wait complete. 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 -- start start 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 Processor -- start 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0c02370 0x55f3d0bff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0b5d2f0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0b18060 con 0x55f3d0bf75d0 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3d0c04e30 con 0x55f3d0c02370 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551e54a740 10 client.0 _ll_register_callbacks cb 0x55f3d086b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551970c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551970c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50942/0 (socket says 172.21.15.77:50942) 2024-09-15T22:28:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.521+0000 7f551970c640 1 -- 172.21.15.77:0/1824640533 learned_addr learned my addr 172.21.15.77:0/1824640533 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f5519f0d640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551a70e640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0c02370 0x55f3d0bff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 -- 172.21.15.77:0/1824640533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0c02370 msgr2=0x55f3d0bff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0c02370 0x55f3d0bff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 -- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf75d0 msgr2=0x55f3d0c01db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c01db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 -- 172.21.15.77:0/1824640533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d08ce3c0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551970c640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5504017cb0 tx=0x7f55040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964861867ad66a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f54fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504026070 con 0x55f3d0bf8b40 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f5504022460 con 0x55f3d0bf8b40 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550c036070 con 0x55f3d0bf8b40 2024-09-15T22:28:18.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5504032050 con 0x55f3d0bf8b40 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551e54a740 10 client.19338 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f3d0c106e0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.522+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c036070 con 0x55f3d0bf8b40 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f55040215e0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 10 client.19338 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 10 client.19338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 20 client.19338 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c11af0 0x55f3d0c13f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 4 client.19338 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d0b6a860 con 0x55f3d0c11af0 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f5519f0d640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c11af0 0x55f3d0c13f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f5519f0d640 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c11af0 0x55f3d0c13f40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ed8d20d3e2ab63b server_cookie=f165363e120731ab in_seq=0 out_seq=0 2024-09-15T22:28:18.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.523+0000 7f54fd7fa640 10 client.19338 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f54fd7fa640 1 -- 172.21.15.77:0/1824640533 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3d0b6a860 con 0x55f3d0c11af0 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f54fd7fa640 10 client.19338 handle_command_reply: tid=0 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 client.19338 shutdown 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 msgr2=0x55f3d0bff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5504017cb0 tx=0x7f55040046c0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 shutdown_connections 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c11af0 0x55f3d0c13f40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0c02370 0x55f3d0bff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bff060 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 --2- 172.21.15.77:0/1824640533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c01db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f54fd7fa640 0 client.19338 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:28:18.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 >> 172.21.15.77:0/1824640533 conn(0x55f3d0bc0310 msgr2=0x55f3d0bf9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 shutdown_connections 2024-09-15T22:28:18.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.524+0000 7f551e54a740 1 -- 172.21.15.77:0/1824640533 wait complete. 2024-09-15T22:28:18.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.525+0000 7f551e54a740 20 client.19338 trim_cache size 0 max 16384 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 Processor -- start 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 -- start start 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bfe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55141b12f0 con 0x55f3d0bf75d0 2024-09-15T22:28:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55141b2ef0 con 0x55f3d0c02370 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f551e54a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55141b19c0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f5519f0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f5519f0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50948/0 (socket says 172.21.15.77:50948) 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.537+0000 7f5519f0d640 1 -- 172.21.15.77:0/3255421181 learned_addr learned my addr 172.21.15.77:0/3255421181 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f551970c640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bfe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 -- 172.21.15.77:0/3255421181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 msgr2=0x55f3d0bfe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bfe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 -- 172.21.15.77:0/3255421181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 msgr2=0x55f3d0c198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f551a70e640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c198a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 -- 172.21.15.77:0/3255421181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d0c145c0 con 0x55f3d0bf75d0 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f551970c640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bfe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f5519f0d640 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55f3d0c14840 tx=0x7f5510008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de288948f622791 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f54ff7fe640 1 -- 172.21.15.77:0/3255421181 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551000bd80 con 0x55f3d0bf75d0 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f54ff7fe640 1 -- 172.21.15.77:0/3255421181 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551001c730 con 0x55f3d0bf75d0 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.538+0000 7f54ff7fe640 1 -- 172.21.15.77:0/3255421181 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510013730 con 0x55f3d0bf75d0 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- 172.21.15.77:0/3255421181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 msgr2=0x55f3d0bf3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55f3d0c14840 tx=0x7f5510008040 comp rx=0 tx=0).stop 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- 172.21.15.77:0/3255421181 shutdown_connections 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf8b40 0x55f3d0bfe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 --2- 172.21.15.77:0/3255421181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf75d0 0x55f3d0bf3e10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- 172.21.15.77:0/3255421181 >> 172.21.15.77:0/3255421181 conn(0x55f3d0bc0310 msgr2=0x55f3d0bf3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- 172.21.15.77:0/3255421181 shutdown_connections 2024-09-15T22:28:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- 172.21.15.77:0/3255421181 wait complete. 2024-09-15T22:28:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 -- start start 2024-09-15T22:28:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.539+0000 7f551e54a740 1 Processor -- start 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c06080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55141b2760 con 0x55f3d0bf8b40 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55141b2090 con 0x55f3d0c02370 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55141af2f0 con 0x55f3d0bf75d0 2024-09-15T22:28:18.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 10 client.0 _ll_register_callbacks cb 0x7f5504017960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50958/0 (socket says 172.21.15.77:50958) 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 -- 172.21.15.77:0/82424805 learned_addr learned my addr 172.21.15.77:0/82424805 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551a70e640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f5519f0d640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c06080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 -- 172.21.15.77:0/82424805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 msgr2=0x55f3d0c06080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c06080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 -- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 msgr2=0x55f3d0c06b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c06b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 -- 172.21.15.77:0/82424805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d0c7bac0 con 0x55f3d0bf8b40 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f5519f0d640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c06080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551970c640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f5504009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbf996284e32182 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f54fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550403b060 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f551001c730 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.540+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f5504022460 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5504042050 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 10 client.19344 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f3d0c7cd10 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551001c730 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f550402b440 con 0x55f3d0bf8b40 2024-09-15T22:28:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 10 client.19344 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 10 client.19344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 20 client.19344 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c7d460 0x55f3d0c7f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 4 client.19344 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3d0b64d70 con 0x55f3d0c7d460 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.541+0000 7f5519f0d640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c7d460 0x55f3d0c7f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f5519f0d640 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c7d460 0x55f3d0c7f880 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=151c34ca892b3957 server_cookie=afe726932191211d in_seq=0 out_seq=0 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f54fd7fa640 10 client.19344 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f54fd7fa640 1 -- 172.21.15.77:0/82424805 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3d0b64d70 con 0x55f3d0c7d460 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f54fd7fa640 10 client.19344 handle_command_reply: tid=0 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 client.19344 shutdown 2024-09-15T22:28:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 msgr2=0x55f3d0c065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f5504009570 comp rx=0 tx=0).stop 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 shutdown_connections 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f3d0c7d460 0x55f3d0c7f880 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3d0c02370 0x55f3d0c06b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3d0bf8b40 0x55f3d0c065c0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 --2- 172.21.15.77:0/82424805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3d0bf75d0 0x55f3d0c06080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f54fd7fa640 0 client.19344 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 >> 172.21.15.77:0/82424805 conn(0x55f3d0bc0310 msgr2=0x55f3d0bf24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 shutdown_connections 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 1 -- 172.21.15.77:0/82424805 wait complete. 2024-09-15T22:28:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.542+0000 7f551e54a740 20 client.19344 trim_cache size 0 max 16384 2024-09-15T22:28:18.555 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 msgr2=0x7f55000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 0x7f55000a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5504006e10 tx=0x7f5504009930 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 msgr2=0x7f5514074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f550c031a60 tx=0x7f550c031a90 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 shutdown_connections 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55000a33b0 0x7f55000a5870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5514106f10 0x7f5514074f70 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5514102790 0x7f5514074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 --2- 172.21.15.77:0/2243633878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5514101de0 0x7f55140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 >> 172.21.15.77:0/2243633878 conn(0x7f55140fd8c0 msgr2=0x7f55140fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 shutdown_connections 2024-09-15T22:28:18.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:18.543+0000 7f551c171640 1 -- 172.21.15.77:0/2243633878 wait complete. 2024-09-15T22:28:18.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 6.49 MB/sec execute 19 sec latency 13829.244 ms 2024-09-15T22:28:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:18 smithi196 ceph-mon[35109]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:18 smithi117 ceph-mon[35021]: pgmap v499: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:19.459 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:28:19.488 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:28:19.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 6.17 MB/sec execute 20 sec latency 14829.314 ms 2024-09-15T22:28:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:20.405 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:20.529 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:20 smithi077 ceph-mon[30980]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:20.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 5.87 MB/sec execute 21 sec latency 15829.346 ms 2024-09-15T22:28:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:20 smithi196 ceph-mon[35109]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:20 smithi117 ceph-mon[35021]: pgmap v500: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 909 KiB/s wr, 34 op/s 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 -- 172.21.15.77:0/1227600326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 msgr2=0x7fb150103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 --2- 172.21.15.77:0/1227600326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb150103110 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb140009930 tx=0x7fb14002a440 comp rx=0 tx=0).stop 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 -- 172.21.15.77:0/1227600326 shutdown_connections 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 --2- 172.21.15.77:0/1227600326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 0x7fb15010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 --2- 172.21.15.77:0/1227600326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 0x7fb150107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 --2- 172.21.15.77:0/1227600326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb150103110 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 -- 172.21.15.77:0/1227600326 >> 172.21.15.77:0/1227600326 conn(0x7fb1500fc7d0 msgr2=0x7fb1500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 -- 172.21.15.77:0/1227600326 shutdown_connections 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.659+0000 7fb158197640 1 -- 172.21.15.77:0/1227600326 wait complete. 2024-09-15T22:28:20.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 Processor -- start 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 -- start start 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 0x7fb150198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 0x7fb15019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501120e0 con 0x7fb150102d30 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb150111f60 con 0x7fb1501082a0 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb158197640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb150112260 con 0x7fb1501036e0 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb155f33640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb155f33640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59692/0 (socket says 172.21.15.77:59692) 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb155f33640 1 -- 172.21.15.77:0/736043976 learned_addr learned my addr 172.21.15.77:0/736043976 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb155732640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 0x7fb150198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.660+0000 7fb156734640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 0x7fb15019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 -- 172.21.15.77:0/736043976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 msgr2=0x7fb150198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 0x7fb150198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 -- 172.21.15.77:0/736043976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 msgr2=0x7fb15019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 0x7fb15019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 -- 172.21.15.77:0/736043976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15019d560 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb155f33640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb140009a50 tx=0x7fb140004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb146ffd640 1 -- 172.21.15.77:0/736043976 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14003f030 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb146ffd640 1 -- 172.21.15.77:0/736043976 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb140003ad0 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb146ffd640 1 -- 172.21.15.77:0/736043976 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140035070 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15019d7f0 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.661+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15019dc80 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.663+0000 7fb146ffd640 1 -- 172.21.15.77:0/736043976 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb140042030 con 0x7fb150102d30 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 Processor -- start 2024-09-15T22:28:20.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 -- start start 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb146ffd640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 0x7fb13c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b4fcb62f0 con 0x557b4fd5b370 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b4fc5dcf0 con 0x557b4fd505d0 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb15a570740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b4fcb22b0 con 0x557b4fd51b40 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb156734640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 0x7fb13c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb155f33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb156734640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb146ffd640 1 -- 172.21.15.77:0/736043976 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fb1400f0600 con 0x7fb150102d30 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb155f33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45496/0 (socket says 172.21.15.77:45496) 2024-09-15T22:28:20.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb155f33640 1 -- 172.21.15.77:0/1713040363 learned_addr learned my addr 172.21.15.77:0/1713040363 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.664+0000 7fb155732640 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 -- 172.21.15.77:0/1713040363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 msgr2=0x557b4fd50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 -- 172.21.15.77:0/1713040363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 msgr2=0x557b4fd5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 -- 172.21.15.77:0/1713040363 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4fc423a0 con 0x557b4fd51b40 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb156734640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 0x7fb13c0a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb148007310 tx=0x7fb148007df0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb155f33640 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1400c4c30 tx=0x7fb140040fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2672035b7674a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb13bfff640 1 -- 172.21.15.77:0/1713040363 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140049070 con 0x557b4fd51b40 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb13bfff640 1 -- 172.21.15.77:0/1713040363 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1400bcbf0 con 0x557b4fd51b40 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.665+0000 7fb13bfff640 1 -- 172.21.15.77:0/1713040363 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1400bc4f0 con 0x557b4fd51b40 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- 172.21.15.77:0/1713040363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 msgr2=0x557b4fd50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1400c4c30 tx=0x7fb140040fb0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- 172.21.15.77:0/1713040363 shutdown_connections 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 --2- 172.21.15.77:0/1713040363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd51b40 0x557b4fd50090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- 172.21.15.77:0/1713040363 >> 172.21.15.77:0/1713040363 conn(0x557b4fd19310 msgr2=0x557b4fd4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- 172.21.15.77:0/1713040363 shutdown_connections 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- 172.21.15.77:0/1713040363 wait complete. 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 -- start start 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.666+0000 7fb15a570740 1 Processor -- start 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd57fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b4fd62240 con 0x557b4fd5b370 2024-09-15T22:28:20.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b4fc71060 con 0x557b4fd51b40 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b4fd623c0 con 0x557b4fd505d0 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 10 client.0 _ll_register_callbacks cb 0x557b4f9c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb155f33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59724/0 (socket says 172.21.15.77:59724) 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 -- 172.21.15.77:0/3997881102 learned_addr learned my addr 172.21.15.77:0/3997881102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb155732640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd57fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 -- 172.21.15.77:0/3997881102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 msgr2=0x557b4fd5ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd5ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 -- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 msgr2=0x557b4fd57fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd57fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 -- 172.21.15.77:0/3997881102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4fa273c0 con 0x557b4fd5b370 2024-09-15T22:28:20.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb156734640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb1480126b0 tx=0x7fb148007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1672e7e749e2d819 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb1397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148026070 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1400bcbf0 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.667+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fb148020660 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb148032050 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 10 client.19356 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557b4fd6dbc0 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1400bcbf0 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb14800f810 con 0x557b4fd5b370 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 10 client.19356 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 10 client.19356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 20 client.19356 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fd6f0f0 0x557b4fd71530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 4 client.19356 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4fcc3860 con 0x557b4fd6f0f0 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.668+0000 7fb155f33640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fd6f0f0 0x557b4fd71530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb155f33640 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fd6f0f0 0x557b4fd71530 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168448dd3386c147 server_cookie=c39a21cae9dbbd29 in_seq=0 out_seq=0 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb1397fa640 10 client.19356 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb1397fa640 1 -- 172.21.15.77:0/3997881102 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b4fcc3860 con 0x557b4fd6f0f0 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb1397fa640 10 client.19356 handle_command_reply: tid=0 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb15a570740 1 client.19356 shutdown 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 msgr2=0x557b4fd58570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb1480126b0 tx=0x7fb148007520 comp rx=0 tx=0).stop 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 shutdown_connections 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.669+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fd6f0f0 0x557b4fd71530 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd58570 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd57fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb1397fa640 0 client.19356 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 --2- 172.21.15.77:0/3997881102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd5ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 >> 172.21.15.77:0/3997881102 conn(0x557b4fd19310 msgr2=0x557b4fd52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 shutdown_connections 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 1 -- 172.21.15.77:0/3997881102 wait complete. 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.670+0000 7fb15a570740 20 client.19356 trim_cache size 0 max 16384 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.682+0000 7fb15a570740 1 Processor -- start 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.682+0000 7fb15a570740 1 -- start start 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd5b370 0x557b4fd53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501acde0 con 0x557b4fd51b40 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501ae9e0 con 0x557b4fd505d0 2024-09-15T22:28:20.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb15a570740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501ad4b0 con 0x557b4fd5b370 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59732/0 (socket says 172.21.15.77:59732) 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 -- 172.21.15.77:0/2627124455 learned_addr learned my addr 172.21.15.77:0/2627124455 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155f33640 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb156734640 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd5b370 0x557b4fd53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 -- 172.21.15.77:0/2627124455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd5b370 msgr2=0x557b4fd53660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd5b370 0x557b4fd53660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 -- 172.21.15.77:0/2627124455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 msgr2=0x557b4fd6b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd6b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 -- 172.21.15.77:0/2627124455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4fd56520 con 0x557b4fd51b40 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb155732640 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb15006a370 tx=0x7fb14c014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1ef169a07e27e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.683+0000 7fb13b7fe640 1 -- 172.21.15.77:0/2627124455 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c022070 con 0x557b4fd51b40 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb13b7fe640 1 -- 172.21.15.77:0/2627124455 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c01d920 con 0x557b4fd51b40 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb13b7fe640 1 -- 172.21.15.77:0/2627124455 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0158a0 con 0x557b4fd51b40 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- 172.21.15.77:0/2627124455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 msgr2=0x557b4fd53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb15006a370 tx=0x7fb14c014780 comp rx=0 tx=0).stop 2024-09-15T22:28:20.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- 172.21.15.77:0/2627124455 shutdown_connections 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd5b370 0x557b4fd53660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd51b40 0x557b4fd53120 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 --2- 172.21.15.77:0/2627124455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd505d0 0x557b4fd6b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- 172.21.15.77:0/2627124455 >> 172.21.15.77:0/2627124455 conn(0x557b4fd19310 msgr2=0x557b4fd57f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- 172.21.15.77:0/2627124455 shutdown_connections 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- 172.21.15.77:0/2627124455 wait complete. 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.684+0000 7fb15a570740 1 -- start start 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 Processor -- start 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd67b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501ae250 con 0x557b4fd5b370 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501adb80 con 0x557b4fd51b40 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501aade0 con 0x557b4fd505d0 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb15a570740 10 client.0 _ll_register_callbacks cb 0x7fb148014280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb156734640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb155732640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb156734640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59742/0 (socket says 172.21.15.77:59742) 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb156734640 1 -- 172.21.15.77:0/3682697974 learned_addr learned my addr 172.21.15.77:0/3682697974 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb155f33640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd67b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb156734640 1 -- 172.21.15.77:0/3682697974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 msgr2=0x557b4fd67b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.685+0000 7fb156734640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd67b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb156734640 1 -- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 msgr2=0x557b4fd54dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb156734640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd54dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb156734640 1 -- 172.21.15.77:0/3682697974 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4fd55b30 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb156734640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb1480072e0 tx=0x7fb148009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e23f52f1d60e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14803a020 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fb14800f5d0 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14c01d920 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb148045050 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 10 client.19362 fetch_fsmap learned FSMap version 36 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557b4fdd9de0 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c01d920 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fb148020c60 con 0x557b4fd5b370 2024-09-15T22:28:20.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 10 client.19362 fetch_fsmap finished waiting for FSMap version 36 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 10 client.19362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 20 client.19362 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fdda5e0 0x557b4fddca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 4 client.19362 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.686+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b4fcbdd70 con 0x557b4fdda5e0 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb155f33640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fdda5e0 0x557b4fddca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb1397fa640 10 client.19362 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb155f33640 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fdda5e0 0x557b4fddca00 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3385b147ce5dcef6 server_cookie=ab7edab9fc9b24de in_seq=0 out_seq=0 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb1397fa640 1 -- 172.21.15.77:0/3682697974 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b4fcbdd70 con 0x557b4fdda5e0 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb1397fa640 10 client.19362 handle_command_reply: tid=0 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 client.19362 shutdown 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 msgr2=0x557b4fd55300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb1480072e0 tx=0x7fb148009000 comp rx=0 tx=0).stop 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 shutdown_connections 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557b4fdda5e0 0x557b4fddca00 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b4fd5b370 0x557b4fd55300 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b4fd51b40 0x557b4fd54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 --2- 172.21.15.77:0/3682697974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b4fd505d0 0x557b4fd67b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb1397fa640 0 client.19362 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.687+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 >> 172.21.15.77:0/3682697974 conn(0x557b4fd19310 msgr2=0x557b4fd59830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 shutdown_connections 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb15a570740 1 -- 172.21.15.77:0/3682697974 wait complete. 2024-09-15T22:28:20.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb15a570740 20 client.19362 trim_cache size 0 max 16384 2024-09-15T22:28:20.708 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:28:20.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 msgr2=0x7fb13c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 0x7fb13c0a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb148007310 tx=0x7fb148007df0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 msgr2=0x7fb1501985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb140009a50 tx=0x7fb140004270 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 shutdown_connections 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb13c0a33b0 0x7fb13c0a5870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1501082a0 0x7fb15019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1501036e0 0x7fb150198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 --2- 172.21.15.77:0/736043976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb150102d30 0x7fb1501985b0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 >> 172.21.15.77:0/736043976 conn(0x7fb1500fc7d0 msgr2=0x7fb150105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:20.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 shutdown_connections 2024-09-15T22:28:20.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:20.688+0000 7fb158197640 1 -- 172.21.15.77:0/736043976 wait complete. 2024-09-15T22:28:21.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 5.60 MB/sec execute 22 sec latency 16829.391 ms 2024-09-15T22:28:22.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 -- 172.21.15.77:0/4267004054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 msgr2=0x7f00dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:22.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 --2- 172.21.15.77:0/4267004054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc101200 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009900 tx=0x7f00cc02a480 comp rx=0 tx=0).stop 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 -- 172.21.15.77:0/4267004054 shutdown_connections 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 --2- 172.21.15.77:0/4267004054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102ef0 0x7f00dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 --2- 172.21.15.77:0/4267004054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102570 0x7f00dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 --2- 172.21.15.77:0/4267004054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc101200 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 -- 172.21.15.77:0/4267004054 >> 172.21.15.77:0/4267004054 conn(0x7f00dc0fc7d0 msgr2=0x7f00dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:22.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 -- 172.21.15.77:0/4267004054 shutdown_connections 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.049+0000 7f00e108b640 1 -- 172.21.15.77:0/4267004054 wait complete. 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.050+0000 7f00e108b640 1 Processor -- start 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.050+0000 7f00e108b640 1 -- start start 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102570 0x7f00dc198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:22.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 0x7f00dc19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:22.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc111020 con 0x7f00dc100e20 2024-09-15T22:28:22.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc110ea0 con 0x7f00dc102570 2024-09-15T22:28:22.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1111a0 con 0x7f00dc102ef0 2024-09-15T22:28:22.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:22.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59758/0 (socket says 172.21.15.77:59758) 2024-09-15T22:28:22.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 -- 172.21.15.77:0/3786801795 learned_addr learned my addr 172.21.15.77:0/3786801795 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:22.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00da59c640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102570 0x7f00dc198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:22.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00db59e640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 0x7f00dc19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:22.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 -- 172.21.15.77:0/3786801795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 msgr2=0x7f00dc19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 0x7f00dc19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 -- 172.21.15.77:0/3786801795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102570 msgr2=0x7f00dc198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102570 0x7f00dc198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 -- 172.21.15.77:0/3786801795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc19d620 con 0x7f00dc100e20 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00db59e640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 0x7f00dc19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:22.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00dad9d640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006010 tx=0x7f00cc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:22.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc03f070 con 0x7f00dc100e20 2024-09-15T22:28:22.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00cc031e30 con 0x7f00dc100e20 2024-09-15T22:28:22.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc19d8b0 con 0x7f00dc100e20 2024-09-15T22:28:22.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc035030 con 0x7f00dc100e20 2024-09-15T22:28:22.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.051+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc1a51f0 con 0x7f00dc100e20 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.052+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc101280 con 0x7f00dc100e20 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.054+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f00cc042070 con 0x7f00dc100e20 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.054+0000 7f00d3fff640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 0x7f00c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.054+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f00cc0f11e0 con 0x7f00dc100e20 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.055+0000 7f00da59c640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 0x7f00c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:22.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.055+0000 7f00da59c640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 0x7f00c80a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0fef00 tx=0x7f00c4009110 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:22.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.058+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00cc0bd830 con 0x7f00dc100e20 2024-09-15T22:28:22.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.262+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00dc1a53c0 con 0x7f00dc100e20 2024-09-15T22:28:22.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.263+0000 7f00d3fff640 1 -- 172.21.15.77:0/3786801795 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11994 (secure 0 0 0) 0x7f00cc0bd130 con 0x7f00dc100e20 2024-09-15T22:28:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:28:22.266 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":37,"btime":"2024-09-15T22:28:21:247964+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:20.441526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:28:22.266 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 37 2024-09-15T22:28:22.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 msgr2=0x7f00c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 0x7f00c80a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0fef00 tx=0x7f00c4009110 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 msgr2=0x7f00dc198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006010 tx=0x7f00cc003c80 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 shutdown_connections 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00c80a33b0 0x7f00c80a5870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00dc102ef0 0x7f00dc19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00dc102570 0x7f00dc198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 --2- 172.21.15.77:0/3786801795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00dc100e20 0x7f00dc198670 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 >> 172.21.15.77:0/3786801795 conn(0x7f00dc0fc7d0 msgr2=0x7f00dc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 shutdown_connections 2024-09-15T22:28:22.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:22.265+0000 7f00e108b640 1 -- 172.21.15.77:0/3786801795 wait complete. 2024-09-15T22:28:22.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:28:22.515 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:28:22.539 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:22 smithi077 ceph-mon[30980]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:22.540 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:22 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:22.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 5.36 MB/sec execute 23 sec latency 17829.427 ms 2024-09-15T22:28:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:22 smithi196 ceph-mon[35109]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:22 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:22 smithi117 ceph-mon[35021]: pgmap v501: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:22 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:22.962 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 830e7afa --release --await' 2024-09-15T22:28:22.962 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 830e7afa --release --await 2024-09-15T22:28:22.996 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:23.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:23 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3786801795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 -- 172.21.15.77:0/1173065836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c100e20 msgr2=0x7fd00c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 --2- 172.21.15.77:0/1173065836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c100e20 0x7fd00c101200 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009900 tx=0x7fcffc02a480 comp rx=0 tx=0).stop 2024-09-15T22:28:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 -- 172.21.15.77:0/1173065836 shutdown_connections 2024-09-15T22:28:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 --2- 172.21.15.77:0/1173065836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c102ef0 0x7fd00c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 --2- 172.21.15.77:0/1173065836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 0x7fd00c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 --2- 172.21.15.77:0/1173065836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c100e20 0x7fd00c101200 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 -- 172.21.15.77:0/1173065836 >> 172.21.15.77:0/1173065836 conn(0x7fd00c0fc7d0 msgr2=0x7fd00c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 -- 172.21.15.77:0/1173065836 shutdown_connections 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.491+0000 7fd011f13640 1 -- 172.21.15.77:0/1173065836 wait complete. 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 Processor -- start 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 -- start start 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 0x7fd00c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 0x7fd00c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c110fa0 con 0x7fd00c102ef0 2024-09-15T22:28:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c110e20 con 0x7fd00c100e20 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd011f13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c111120 con 0x7fd00c102570 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59784/0 (socket says 172.21.15.77:59784) 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 -- 172.21.15.77:0/357165431 learned_addr learned my addr 172.21.15.77:0/357165431 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 0x7fd00c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00affd640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 0x7fd00c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 -- 172.21.15.77:0/357165431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 msgr2=0x7fd00c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 0x7fd00c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 -- 172.21.15.77:0/357165431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 msgr2=0x7fd00c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 0x7fd00c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00bfff640 1 -- 172.21.15.77:0/357165431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c19d540 con 0x7fd00c102ef0 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.492+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 0x7fd00c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd00bfff640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd00400cce0 tx=0x7fd00400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd008ff9640 1 -- 172.21.15.77:0/357165431 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00400f450 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c19d830 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd008ff9640 1 -- 172.21.15.77:0/357165431 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00400fad0 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd008ff9640 1 -- 172.21.15.77:0/357165431 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004018c10 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.493+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c1a5110 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.495+0000 7fd008ff9640 1 -- 172.21.15.77:0/357165431 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd00400f5f0 con 0x7fd00c102ef0 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.495+0000 7fd0142ec740 1 Processor -- start 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.495+0000 7fd0142ec740 1 -- start start 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.495+0000 7fd008ff9640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 0x7fcff40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.495+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd0142ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a9264e280 con 0x556a926e9480 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd0142ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a925f5b10 con 0x556a926f3d40 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd0142ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a9264a240 con 0x556a926e8b60 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 0x7fcff40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd008ff9640 1 -- 172.21.15.77:0/357165431 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd0040d08a0 con 0x7fd00c102ef0 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59798/0 (socket says 172.21.15.77:59798) 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 -- 172.21.15.77:0/3707095511 learned_addr learned my addr 172.21.15.77:0/3707095511 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 -- 172.21.15.77:0/3707095511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 msgr2=0x556a926e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 -- 172.21.15.77:0/3707095511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 msgr2=0x556a926f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 -- 172.21.15.77:0/3707095511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a926021f0 con 0x556a926e9480 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00bfff640 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 0x7fcff40a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd000014670 tx=0x7fd00000a940 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fd00affd640 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd000006aa0 tx=0x7fd000019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ffbae682936727 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.496+0000 7fcff1ffb640 1 -- 172.21.15.77:0/3707095511 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000025070 con 0x556a926e9480 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fcff1ffb640 1 -- 172.21.15.77:0/3707095511 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00001c070 con 0x556a926e9480 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fcff1ffb640 1 -- 172.21.15.77:0/3707095511 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000020680 con 0x556a926e9480 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 -- 172.21.15.77:0/3707095511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 msgr2=0x556a926f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd000006aa0 tx=0x7fd000019000 comp rx=0 tx=0).stop 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 -- 172.21.15.77:0/3707095511 shutdown_connections 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f3800 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3707095511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 -- 172.21.15.77:0/3707095511 >> 172.21.15.77:0/3707095511 conn(0x556a926e50f0 msgr2=0x556a926e54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 -- 172.21.15.77:0/3707095511 shutdown_connections 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.497+0000 7fd0142ec740 1 -- 172.21.15.77:0/3707095511 wait complete. 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 -- start start 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 Processor -- start 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a926f6910 con 0x556a926e9480 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556a92608f10 con 0x556a926f3d40 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556a926f6a90 con 0x556a926e8b60 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd0142ec740 10 client.0 _ll_register_callbacks cb 0x556a925d1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd00affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd00affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59800/0 (socket says 172.21.15.77:59800) 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd00affd640 1 -- 172.21.15.77:0/1490886181 learned_addr learned my addr 172.21.15.77:0/1490886181 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.498+0000 7fd00bfff640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 -- 172.21.15.77:0/1490886181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 msgr2=0x556a926f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 -- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 msgr2=0x556a926f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 -- 172.21.15.77:0/1490886181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a923bf6a0 con 0x556a926e9480 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00bfff640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd00affd640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd00000a110 tx=0x7fd000000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f0757b01a3767 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fcfeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:23.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000025040 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd000020420 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00001c070 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd000037050 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd0142ec740 10 client.19380 fetch_fsmap learned FSMap version 37 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.499+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556a92702510 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00001c070 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fd000028030 con 0x556a926e9480 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 10 client.19380 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 10 client.19380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 20 client.19380 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92703860 0x556a92705c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 4 client.19380 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:23.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a9265b7f0 con 0x556a92703860 2024-09-15T22:28:23.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.500+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92703860 0x556a92705c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fcfeaffd640 10 client.19380 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:23.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92703860 0x556a92705c50 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4751d4037d58e81 server_cookie=38e64e0dcc8477b6 in_seq=0 out_seq=0 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fcfeaffd640 1 -- 172.21.15.77:0/1490886181 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a9265b7f0 con 0x556a92703860 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fcfeaffd640 10 client.19380 handle_command_reply: tid=0 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fd0142ec740 1 client.19380 shutdown 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 msgr2=0x556a926f1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.501+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd00000a110 tx=0x7fd000000bc0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 shutdown_connections 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92703860 0x556a92705c50 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fcfeaffd640 0 client.19380 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a926f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e9480 0x556a926f1160 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 --2- 172.21.15.77:0/1490886181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e8b60 0x556a926f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 >> 172.21.15.77:0/1490886181 conn(0x556a926e50f0 msgr2=0x556a926eae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 shutdown_connections 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 1 -- 172.21.15.77:0/1490886181 wait complete. 2024-09-15T22:28:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.502+0000 7fd0142ec740 20 client.19380 trim_cache size 0 max 16384 2024-09-15T22:28:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.520+0000 7fd0142ec740 1 Processor -- start 2024-09-15T22:28:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.520+0000 7fd0142ec740 1 -- start start 2024-09-15T22:28:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.520+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e9480 0x556a9270b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a9270b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59808/0 (socket says 172.21.15.77:59808) 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 -- 172.21.15.77:0/3472471820 learned_addr learned my addr 172.21.15.77:0/3472471820 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1af000 con 0x556a926e8b60 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1b0bd0 con 0x556a926f3d40 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1af6d0 con 0x556a926e9480 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 -- 172.21.15.77:0/3472471820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e9480 msgr2=0x556a9270b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e9480 0x556a9270b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 -- 172.21.15.77:0/3472471820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 msgr2=0x556a9270b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a9270b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 -- 172.21.15.77:0/3472471820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a926ef170 con 0x556a926e8b60 2024-09-15T22:28:23.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.521+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005c40 tx=0x7fcffc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0dc3947110ec79 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fcff17fa640 1 -- 172.21.15.77:0/3472471820 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc035040 con 0x556a926e8b60 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fcff17fa640 1 -- 172.21.15.77:0/3472471820 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcffc0046c0 con 0x556a926e8b60 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fcff17fa640 1 -- 172.21.15.77:0/3472471820 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc046710 con 0x556a926e8b60 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 msgr2=0x556a926f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005c40 tx=0x7fcffc005b50 comp rx=0 tx=0).stop 2024-09-15T22:28:23.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 shutdown_connections 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926f3d40 0x556a9270b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926e9480 0x556a9270b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3472471820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a926e8b60 0x556a926f14f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.522+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 >> 172.21.15.77:0/3472471820 conn(0x556a926e50f0 msgr2=0x556a926f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.523+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 shutdown_connections 2024-09-15T22:28:23.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.523+0000 7fd0142ec740 1 -- 172.21.15.77:0/3472471820 wait complete. 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.523+0000 7fd0142ec740 1 -- start start 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 Processor -- start 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 0x556a92767800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926f3d40 0x556a92767d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 0x556a926fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1b0470 con 0x556a92703860 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1afda0 con 0x556a926e9480 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c1ad000 con 0x556a926f3d40 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd0142ec740 10 client.0 _ll_register_callbacks cb 0x7fd000023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:23.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 0x556a926fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 0x556a92767800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 0x556a92767800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37174/0 (socket says 172.21.15.77:37174) 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.524+0000 7fd00b7fe640 1 -- 172.21.15.77:0/3834973973 learned_addr learned my addr 172.21.15.77:0/3834973973 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00affd640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926f3d40 0x556a92767d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 -- 172.21.15.77:0/3834973973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926f3d40 msgr2=0x556a92767d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926f3d40 0x556a92767d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 -- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 msgr2=0x556a92767800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 0x556a92767800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 -- 172.21.15.77:0/3834973973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a926fc220 con 0x556a92703860 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd00bfff640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 0x556a926fbce0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd0040cdc00 tx=0x7fd00409c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7337804ab4b3351c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fcfeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040ce080 con 0x556a92703860 2024-09-15T22:28:23.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd0040a4f70 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcffc0046c0 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd0040a0b40 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd0142ec740 10 client.19386 fetch_fsmap learned FSMap version 37 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.525+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556a927730a0 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0040a5400 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fd00409bc70 con 0x556a92703860 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 10 client.19386 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 10 client.19386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 20 client.19386 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92773820 0x556a92775c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 4 client.19386 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fcffc0046c0 con 0x556a92773820 2024-09-15T22:28:23.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.526+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92773820 0x556a92775c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.527+0000 7fcfeaffd640 10 client.19386 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.527+0000 7fd00b7fe640 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92773820 0x556a92775c40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb35f50859ff8ee4 server_cookie=51efcfd066ab1e6a in_seq=0 out_seq=0 2024-09-15T22:28:23.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fcfeaffd640 1 -- 172.21.15.77:0/3834973973 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcffc0046c0 con 0x556a92773820 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fcfeaffd640 10 client.19386 handle_command_reply: tid=0 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 client.19386 shutdown 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 msgr2=0x556a926fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 0x556a926fbce0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd0040cdc00 tx=0x7fd00409c0f0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 shutdown_connections 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x556a92773820 0x556a92775c40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a92703860 0x556a926fbce0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fcfeaffd640 0 client.19386 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556a926f3d40 0x556a92767d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 --2- 172.21.15.77:0/3834973973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556a926e9480 0x556a92767800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.528+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 >> 172.21.15.77:0/3834973973 conn(0x556a926e50f0 msgr2=0x556a926ea600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 shutdown_connections 2024-09-15T22:28:23.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd0142ec740 1 -- 172.21.15.77:0/3834973973 wait complete. 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd0142ec740 20 client.19386 trim_cache size 0 max 16384 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "c0444a6a-3af3-4110-bba0-b043d17e3fc3", 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:28:23.534 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 msgr2=0x7fcff40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 0x7fcff40a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd000014670 tx=0x7fd00000a940 comp rx=0 tx=0).stop 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 msgr2=0x7fd00c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd00400cce0 tx=0x7fd00400a620 comp rx=0 tx=0).stop 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 shutdown_connections 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcff40a33b0 0x7fcff40a5870 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd00c102ef0 0x7fd00c19ce60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd00c102570 0x7fd00c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 --2- 172.21.15.77:0/357165431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd00c100e20 0x7fd00c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:23.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.529+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 >> 172.21.15.77:0/357165431 conn(0x7fd00c0fc7d0 msgr2=0x7fd00c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:23.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.530+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 shutdown_connections 2024-09-15T22:28:23.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:23.530+0000 7fd011f13640 1 -- 172.21.15.77:0/357165431 wait complete. 2024-09-15T22:28:23.540 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:23.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 5.14 MB/sec execute 24 sec latency 18829.460 ms 2024-09-15T22:28:23.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:23 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3786801795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:23.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:23 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3786801795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:28:24.417 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:28:24.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:24.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:24 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46037 4.93 MB/sec execute 25 sec latency 19829.487 ms 2024-09-15T22:28:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:24 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: pgmap v502: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:24.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:24 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 -- 172.21.15.77:0/3010888743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 msgr2=0x7f0054100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 --2- 172.21.15.77:0/3010888743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f0054100ee0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f003c009930 tx=0x7f003c02a420 comp rx=0 tx=0).stop 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 -- 172.21.15.77:0/3010888743 shutdown_connections 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 --2- 172.21.15.77:0/3010888743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054101420 0x7f005410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 --2- 172.21.15.77:0/3010888743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f0054100ee0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 --2- 172.21.15.77:0/3010888743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054106a80 0x7f0054106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 -- 172.21.15.77:0/3010888743 >> 172.21.15.77:0/3010888743 conn(0x7f00540fc7a0 msgr2=0x7f00540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 -- 172.21.15.77:0/3010888743 shutdown_connections 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.920+0000 7f005a20d640 1 -- 172.21.15.77:0/3010888743 wait complete. 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 Processor -- start 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 -- start start 2024-09-15T22:28:24.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054101420 0x7f00541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054106a80 0x7f005419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0054111f60 con 0x7f0054100a80 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0054111de0 con 0x7f0054101420 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f005a20d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00541120e0 con 0x7f0054106a80 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59830/0 (socket says 172.21.15.77:59830) 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f00537fe640 1 -- 172.21.15.77:0/2655799376 learned_addr learned my addr 172.21.15.77:0/2655799376 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f0053fff640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054106a80 0x7f005419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.921+0000 7f004bfff640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054101420 0x7f00541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 -- 172.21.15.77:0/2655799376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054106a80 msgr2=0x7f005419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054106a80 0x7f005419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 -- 172.21.15.77:0/2655799376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054101420 msgr2=0x7f00541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054101420 0x7f00541988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 -- 172.21.15.77:0/2655799376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005419d360 con 0x7f0054100a80 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00537fe640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f004400c920 tx=0x7f004400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00517fa640 1 -- 172.21.15.77:0/2655799376 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044007c70 con 0x7f0054100a80 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00517fa640 1 -- 172.21.15.77:0/2655799376 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0044007e10 con 0x7f0054100a80 2024-09-15T22:28:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005419d650 con 0x7f0054100a80 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f00517fa640 1 -- 172.21.15.77:0/2655799376 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044018890 con 0x7f0054100a80 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.922+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00541a4f30 con 0x7f0054100a80 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.924+0000 7f00517fa640 1 -- 172.21.15.77:0/2655799376 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0044018a50 con 0x7f0054100a80 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.924+0000 7f005c5e6740 1 Processor -- start 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 -- start start 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630fea1d620 con 0x5630feab8060 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630fe9c4e50 con 0x5630feab8980 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f005c5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5630fea195e0 con 0x5630feac3240 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f00517fa640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 0x7f00400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59846/0 (socket says 172.21.15.77:59846) 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 -- 172.21.15.77:0/3409848977 learned_addr learned my addr 172.21.15.77:0/3409848977 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f00517fa640 1 -- 172.21.15.77:0/2655799376 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f00440d0290 con 0x7f0054100a80 2024-09-15T22:28:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f004bfff640 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f00537fe640 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 0x7f00400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 -- 172.21.15.77:0/3409848977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 msgr2=0x5630feac5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 -- 172.21.15.77:0/3409848977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 msgr2=0x5630feac2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.925+0000 7f0053fff640 1 -- 172.21.15.77:0/3409848977 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fe9b5d40 con 0x5630feab8060 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f0053fff640 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f004c012d00 tx=0x7f004c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc1bd8c8a33479 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f0053fff640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 0x7f00400a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f004c009fd0 tx=0x7f004c017000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f0049ffb640 1 -- 172.21.15.77:0/3409848977 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c022070 con 0x5630feab8060 2024-09-15T22:28:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f0049ffb640 1 -- 172.21.15.77:0/3409848977 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0124f0 con 0x5630feab8060 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f0049ffb640 1 -- 172.21.15.77:0/3409848977 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c00cd40 con 0x5630feab8060 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f005c5e6740 1 -- 172.21.15.77:0/3409848977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 msgr2=0x5630feab8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.926+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f004c012d00 tx=0x7f004c010740 comp rx=0 tx=0).stop 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- 172.21.15.77:0/3409848977 shutdown_connections 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3409848977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feab8440 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- 172.21.15.77:0/3409848977 >> 172.21.15.77:0/3409848977 conn(0x5630feab45f0 msgr2=0x5630feab49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- 172.21.15.77:0/3409848977 shutdown_connections 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- 172.21.15.77:0/3409848977 wait complete. 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- start start 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 Processor -- start 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630feac5e10 con 0x5630feab8060 2024-09-15T22:28:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630fe9a5f10 con 0x5630feab8980 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5630feac5f90 con 0x5630feac3240 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f005c5e6740 10 client.0 _ll_register_callbacks cb 0x5630fe9c9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.927+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59850/0 (socket says 172.21.15.77:59850) 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 -- 172.21.15.77:0/3464432697 learned_addr learned my addr 172.21.15.77:0/3464432697 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f004bfff640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 -- 172.21.15.77:0/3464432697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 msgr2=0x5630feac0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0053fff640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 -- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 msgr2=0x5630feac0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 -- 172.21.15.77:0/3464432697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fe72c580 con 0x5630feab8060 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f004bfff640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f00537fe640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f004409be80 tx=0x7f004409c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a98b592f5106b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440b4030 con 0x5630feab8060 2024-09-15T22:28:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f004400fb30 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c0124f0 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f00440a0530 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f005c5e6740 10 client.19398 fetch_fsmap learned FSMap version 37 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5630fe7f6960 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.928+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0124f0 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f00440bc0b0 con 0x5630feab8060 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 10 client.19398 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 10 client.19398 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 20 client.19398 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630fead2de0 0x5630fead5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 4 client.19398 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630fea2ab90 con 0x5630fead2de0 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f004bfff640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630fead2de0 0x5630fead5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f004bfff640 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630fead2de0 0x5630fead5250 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d6d870b6de23c2 server_cookie=914b86c8a5cff009 in_seq=0 out_seq=0 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.929+0000 7f0032ffd640 10 client.19398 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f0032ffd640 1 -- 172.21.15.77:0/3464432697 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630fea2ab90 con 0x5630fead2de0 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f0032ffd640 10 client.19398 handle_command_reply: tid=0 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 client.19398 shutdown 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 msgr2=0x5630feac01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f004409be80 tx=0x7f004409c9b0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 shutdown_connections 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630fead2de0 0x5630fead5250 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f0032ffd640 0 client.19398 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3464432697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feac01c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 >> 172.21.15.77:0/3464432697 conn(0x5630feab45f0 msgr2=0x5630feaba320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 shutdown_connections 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 1 -- 172.21.15.77:0/3464432697 wait complete. 2024-09-15T22:28:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.930+0000 7f005c5e6740 20 client.19398 trim_cache size 0 max 16384 2024-09-15T22:28:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 Processor -- start 2024-09-15T22:28:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 -- start start 2024-09-15T22:28:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 0x5630feab3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feab8980 0x5630feadab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00541aefd0 con 0x5630feac3240 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.947+0000 7f005c5e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00541b0ba0 con 0x5630feab8060 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f005c5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00541af6a0 con 0x5630feab8980 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59862/0 (socket says 172.21.15.77:59862) 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 -- 172.21.15.77:0/4236721017 learned_addr learned my addr 172.21.15.77:0/4236721017 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f004bfff640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feab8980 0x5630feadab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f00537fe640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 0x5630feab3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 -- 172.21.15.77:0/4236721017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feab8980 msgr2=0x5630feadab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feab8980 0x5630feadab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 -- 172.21.15.77:0/4236721017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 msgr2=0x5630feab3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 0x5630feab3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 -- 172.21.15.77:0/4236721017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630feab56a0 con 0x5630feac3240 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f00537fe640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 0x5630feab3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f0053fff640 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f004c006010 tx=0x7f004c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c1b79d6af68df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f00497fa640 1 -- 172.21.15.77:0/4236721017 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c02a030 con 0x5630feac3240 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f00497fa640 1 -- 172.21.15.77:0/4236721017 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0127e0 con 0x5630feac3240 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.948+0000 7f00497fa640 1 -- 172.21.15.77:0/4236721017 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c022040 con 0x5630feac3240 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 -- 172.21.15.77:0/4236721017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 msgr2=0x5630feadb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f004c006010 tx=0x7f004c009260 comp rx=0 tx=0).stop 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 -- 172.21.15.77:0/4236721017 shutdown_connections 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feac3240 0x5630feadb060 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feab8980 0x5630feadab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 --2- 172.21.15.77:0/4236721017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8060 0x5630feab3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 -- 172.21.15.77:0/4236721017 >> 172.21.15.77:0/4236721017 conn(0x5630feab45f0 msgr2=0x5630feab2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 -- 172.21.15.77:0/4236721017 shutdown_connections 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.949+0000 7f005c5e6740 1 -- 172.21.15.77:0/4236721017 wait complete. 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 -- start start 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 Processor -- start 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00541b0440 con 0x5630feab8060 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00541afd70 con 0x5630feab8980 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00541acfd0 con 0x5630feac3240 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f005c5e6740 10 client.0 _ll_register_callbacks cb 0x7f004409bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f00537fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59878/0 (socket says 172.21.15.77:59878) 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f00537fe640 1 -- 172.21.15.77:0/3793903566 learned_addr learned my addr 172.21.15.77:0/3793903566 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f004bfff640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.950+0000 7f0053fff640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 -- 172.21.15.77:0/3793903566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 msgr2=0x5630feac86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 -- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 msgr2=0x5630feac81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 -- 172.21.15.77:0/3793903566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630feac8c80 con 0x5630feab8060 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0053fff640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f00537fe640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00440bc010 tx=0x7f004400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ad84038cb7a721 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440b4070 con 0x5630feab8060 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f00440a4960 con 0x5630feab8060 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c0127e0 con 0x5630feab8060 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f00440a4df0 con 0x5630feab8060 2024-09-15T22:28:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f005c5e6740 10 client.19404 fetch_fsmap learned FSMap version 37 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5630feb3e870 con 0x5630feab8060 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.951+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0127e0 con 0x5630feab8060 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f004407f020 con 0x5630feab8060 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 10 client.19404 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 10 client.19404 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 20 client.19404 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630feb3ee90 0x5630feb412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 4 client.19404 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "830e7afa", "release": true, "await": true}] 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "830e7afa", "release": true, "await": true}) -- 0x5630fea250a0 con 0x5630feb3ee90 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f004bfff640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630feb3ee90 0x5630feb412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f004bfff640 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630feb3ee90 0x5630feb412b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1250f7a6ba225fb7 server_cookie=b133762f0e5cf98a in_seq=0 out_seq=0 2024-09-15T22:28:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:24.952+0000 7f0032ffd640 10 client.19404 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:24.988 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:25.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.017+0000 7f0032ffd640 1 -- 172.21.15.77:0/3793903566 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5630fea250a0 con 0x5630feb3ee90 2024-09-15T22:28:25.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.017+0000 7f0032ffd640 10 client.19404 handle_command_reply: tid=0 2024-09-15T22:28:25.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.017+0000 7f005c5e6740 1 client.19404 shutdown 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 msgr2=0x5630feacb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00440bc010 tx=0x7f004400cf80 comp rx=0 tx=0).stop 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 shutdown_connections 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630feb3ee90 0x5630feb412b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f0032ffd640 0 client.19404 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630feac3240 0x5630feac86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630feab8980 0x5630feac81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 --2- 172.21.15.77:0/3793903566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630feab8060 0x5630feacb670 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 >> 172.21.15.77:0/3793903566 conn(0x5630feab45f0 msgr2=0x5630feac09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 shutdown_connections 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 1 -- 172.21.15.77:0/3793903566 wait complete. 2024-09-15T22:28:25.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.018+0000 7f005c5e6740 20 client.19404 trim_cache size 0 max 16384 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 37, 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 60, 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "830e7afa": { 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 60, 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:25.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 137.7, 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:25.023 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:25.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 msgr2=0x7f00400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:25.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 0x7f00400a5870 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f004c009fd0 tx=0x7f004c017000 comp rx=0 tx=0).stop 2024-09-15T22:28:25.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 msgr2=0x7f00541983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f004400c920 tx=0x7f004400cde0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 shutdown_connections 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f00400a33b0 0x7f00400a5870 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0054106a80 0x7f005419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0054101420 0x7f00541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 --2- 172.21.15.77:0/2655799376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0054100a80 0x7f00541983b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 >> 172.21.15.77:0/2655799376 conn(0x7f00540fc7a0 msgr2=0x7f00540fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 shutdown_connections 2024-09-15T22:28:25.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:25.019+0000 7f005a20d640 1 -- 172.21.15.77:0/2655799376 wait complete. 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46301 5.31 MB/sec execute 26 sec latency 20281.812 ms 2024-09-15T22:28:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:28:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:28:26.175 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '830e7afa', total seconds elapsed: 24.4 2024-09-15T22:28:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 -- 172.21.15.77:0/2807052593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81036e0 msgr2=0x7fa7b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/2807052593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81036e0 0x7fa7b8107b90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009930 tx=0x7fa7a802a420 comp rx=0 tx=0).stop 2024-09-15T22:28:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 -- 172.21.15.77:0/2807052593 shutdown_connections 2024-09-15T22:28:26.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/2807052593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b81082a0 0x7fa7b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/2807052593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81036e0 0x7fa7b8107b90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/2807052593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b8102d30 0x7fa7b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 -- 172.21.15.77:0/2807052593 >> 172.21.15.77:0/2807052593 conn(0x7fa7b80fc7d0 msgr2=0x7fa7b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 -- 172.21.15.77:0/2807052593 shutdown_connections 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.339+0000 7fa7bf59c640 1 -- 172.21.15.77:0/2807052593 wait complete. 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 Processor -- start 2024-09-15T22:28:26.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 -- start start 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b8102d30 0x7fa7b81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 0x7fa7b819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8112060 con 0x7fa7b81082a0 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8111ee0 con 0x7fa7b8102d30 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bf59c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81121e0 con 0x7fa7b81036e0 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45638/0 (socket says 172.21.15.77:45638) 2024-09-15T22:28:26.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.340+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1423633962 learned_addr learned my addr 172.21.15.77:0/1423633962 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b8102d30 0x7fa7b81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 0x7fa7b819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1423633962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b8102d30 msgr2=0x7fa7b81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b8102d30 0x7fa7b81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 msgr2=0x7fa7b819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 0x7fa7b819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1423633962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b819d550 con 0x7fa7b81036e0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 0x7fa7b819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005940 tx=0x7fa7a8031b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.341+0000 7fa7ae7fc640 1 -- 172.21.15.77:0/1423633962 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a803f030 con 0x7fa7b81036e0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.342+0000 7fa7ae7fc640 1 -- 172.21.15.77:0/1423633962 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80043d0 con 0x7fa7b81036e0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.342+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819d7e0 con 0x7fa7b81036e0 2024-09-15T22:28:26.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.342+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b819dc50 con 0x7fa7b81036e0 2024-09-15T22:28:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.342+0000 7fa7ae7fc640 1 -- 172.21.15.77:0/1423633962 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8035070 con 0x7fa7b81036e0 2024-09-15T22:28:26.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.345+0000 7fa7ae7fc640 1 -- 172.21.15.77:0/1423633962 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa7a8042030 con 0x7fa7b81036e0 2024-09-15T22:28:26.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.346+0000 7fa7ae7fc640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 0x7fa7a40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.346+0000 7fa7ae7fc640 1 -- 172.21.15.77:0/1423633962 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fa7a80f03e0 con 0x7fa7b81036e0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.346+0000 7fa7c1975740 1 Processor -- start 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 -- start start 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 0x7fa7a40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a43532e0 con 0x5640a43f8680 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640a42face0 con 0x5640a43ed4a0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7c1975740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5640a434f2a0 con 0x5640a43eddc0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bd338640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45652/0 (socket says 172.21.15.77:45652) 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3141881720 learned_addr learned my addr 172.21.15.77:0/3141881720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.347+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3141881720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 msgr2=0x5640a43ed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43ed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3141881720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 msgr2=0x5640a43faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3141881720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a42df440 con 0x5640a43eddc0 2024-09-15T22:28:26.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 0x7fa7a40a5970 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40095f0 tx=0x7fa7b4011810 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.348+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8032c00 tx=0x7fa7a80bbcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2852354cb90a3a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.349+0000 7fa7a37fe640 1 -- 172.21.15.77:0/3141881720 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8049040 con 0x5640a43eddc0 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.349+0000 7fa7a37fe640 1 -- 172.21.15.77:0/3141881720 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80c0680 con 0x5640a43eddc0 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.349+0000 7fa7a37fe640 1 -- 172.21.15.77:0/3141881720 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80ed9c0 con 0x5640a43eddc0 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 -- 172.21.15.77:0/3141881720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 msgr2=0x5640a43f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8032c00 tx=0x7fa7a80bbcf0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 -- 172.21.15.77:0/3141881720 shutdown_connections 2024-09-15T22:28:26.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43f8140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3141881720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43ed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.350+0000 7fa7c1975740 1 -- 172.21.15.77:0/3141881720 >> 172.21.15.77:0/3141881720 conn(0x5640a43e9a30 msgr2=0x5640a43e9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.351+0000 7fa7c1975740 1 -- 172.21.15.77:0/3141881720 shutdown_connections 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.351+0000 7fa7c1975740 1 -- 172.21.15.77:0/3141881720 wait complete. 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.351+0000 7fa7c1975740 1 -- start start 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 Processor -- start 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43fc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a4400be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a43fb010 con 0x5640a43f8680 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640a430e050 con 0x5640a43ed4a0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7c1975740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5640a43fb190 con 0x5640a43eddc0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.352+0000 7fa7bd338640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7c1975740 10 client.0 _ll_register_callbacks cb 0x5640a434e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45666/0 (socket says 172.21.15.77:45666) 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3877610817 learned_addr learned my addr 172.21.15.77:0/3877610817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a4400be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 msgr2=0x5640a43fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3877610817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 msgr2=0x5640a4400be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a4400be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3877610817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a40c43c0 con 0x5640a43eddc0 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a4400be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.353+0000 7fa7bd338640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004650 tx=0x7fa7a80bbd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8442c7f667a30251 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:28:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80d4020 con 0x5640a43eddc0 2024-09-15T22:28:26.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a80c0680 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fa7a80d5220 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa7a8049070 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7c1975740 10 client.28165 fetch_fsmap learned FSMap version 38 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.354+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5640a43fb010 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80c0680 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fa7a80bb720 con 0x5640a43eddc0 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 10 client.28165 fetch_fsmap finished waiting for FSMap version 38 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 10 client.28165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:26.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 20 client.28165 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a440be60 0x5640a440e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 4 client.28165 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.355+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a4360850 con 0x5640a440be60 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.356+0000 7fa7bd338640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a440be60 0x5640a440e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.356+0000 7fa7bd338640 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a440be60 0x5640a440e250 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d8bef13a603227 server_cookie=6c6354bba914281f in_seq=0 out_seq=0 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.356+0000 7fa7a0ff9640 10 client.28165 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/3877610817 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640a4360850 con 0x5640a440be60 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7a0ff9640 10 client.28165 handle_command_reply: tid=0 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 client.28165 shutdown 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 msgr2=0x5640a43fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004650 tx=0x7fa7a80bbd60 comp rx=0 tx=0).stop 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 shutdown_connections 2024-09-15T22:28:26.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a440be60 0x5640a440e250 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a4400be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43eddc0 0x5640a43fca70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7a0ff9640 0 client.28165 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3877610817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43ed4a0 0x5640a43fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 >> 172.21.15.77:0/3877610817 conn(0x5640a43e9a30 msgr2=0x5640a43ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.358+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 shutdown_connections 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.359+0000 7fa7c1975740 1 -- 172.21.15.77:0/3877610817 wait complete. 2024-09-15T22:28:26.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.359+0000 7fa7c1975740 20 client.28165 trim_cache size 0 max 16384 2024-09-15T22:28:26.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 Processor -- start 2024-09-15T22:28:26.388 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:26 smithi077 ceph-mon[30980]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 30 op/s 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 -- start start 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43ed4a0 0x5640a44139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ace80 con 0x5640a43f8680 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aea20 con 0x5640a43eddc0 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7c1975740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ad550 con 0x5640a43ed4a0 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bd338640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43ed4a0 0x5640a44139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37288/0 (socket says 172.21.15.77:37288) 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3514455501 learned_addr learned my addr 172.21.15.77:0/3514455501 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:26.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3514455501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43ed4a0 msgr2=0x5640a44139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43ed4a0 0x5640a44139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3514455501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 msgr2=0x5640a43f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.386+0000 7fa7bcb37640 1 -- 172.21.15.77:0/3514455501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a43ed2d0 con 0x5640a43eddc0 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.387+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009900 tx=0x7fa7a80ba7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52b9bf30f6b23db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.387+0000 7fa7a2ffd640 1 -- 172.21.15.77:0/3514455501 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80d4070 con 0x5640a43eddc0 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.387+0000 7fa7a2ffd640 1 -- 172.21.15.77:0/3514455501 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80dc150 con 0x5640a43eddc0 2024-09-15T22:28:26.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.387+0000 7fa7a2ffd640 1 -- 172.21.15.77:0/3514455501 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80b8940 con 0x5640a43eddc0 2024-09-15T22:28:26.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- 172.21.15.77:0/3514455501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 msgr2=0x5640a4413f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009900 tx=0x7fa7a80ba7e0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- 172.21.15.77:0/3514455501 shutdown_connections 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43f8680 0x5640a43f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4413f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 --2- 172.21.15.77:0/3514455501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43ed4a0 0x5640a44139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- 172.21.15.77:0/3514455501 >> 172.21.15.77:0/3514455501 conn(0x5640a43e9a30 msgr2=0x5640a43ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- 172.21.15.77:0/3514455501 shutdown_connections 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- 172.21.15.77:0/3514455501 wait complete. 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.388+0000 7fa7c1975740 1 -- start start 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 Processor -- start 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 0x5640a44048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 0x5640a4475990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7bdb39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 0x5640a4475990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7bcb37640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37294/0 (socket says 172.21.15.77:37294) 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1971652520 learned_addr learned my addr 172.21.15.77:0/1971652520 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ae2f0 con 0x5640a43ed4a0 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81adc20 con 0x5640a43eddc0 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.389+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81aae80 con 0x5640a43f8680 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 0x5640a44048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7c1975740 10 client.0 _ll_register_callbacks cb 0x7fa7a80bbe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1971652520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 msgr2=0x5640a4475990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 0x5640a4475990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1971652520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 msgr2=0x5640a44048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 0x5640a44048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bcb37640 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a43fcde0 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bdb39640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 0x5640a4475990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.390+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 0x5640a44048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7bcb37640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80bbaf0 tx=0x7fa7a80bcb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd6916c62735dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80d4070 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a80dc150 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fa7a80ed9c0 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa7a80c0680 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7c1975740 10 client.28256 fetch_fsmap learned FSMap version 38 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5640a43fcde0 con 0x5640a43eddc0 2024-09-15T22:28:26.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.391+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a80dc150 con 0x5640a43eddc0 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fa7a80a1070 con 0x5640a43eddc0 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 10 client.28256 fetch_fsmap finished waiting for FSMap version 38 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 10 client.28256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 20 client.28256 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a4477440 0x5640a4479860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 4 client.28256 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640a435ad60 con 0x5640a4477440 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.392+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a4477440 0x5640a4479860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.393+0000 7fa7bd338640 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a4477440 0x5640a4479860 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c577a0434f1f9c server_cookie=42da84fb25382cba in_seq=0 out_seq=0 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.393+0000 7fa7a0ff9640 10 client.28256 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7a0ff9640 1 -- 172.21.15.77:0/1971652520 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5640a435ad60 con 0x5640a4477440 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7a0ff9640 10 client.28256 handle_command_reply: tid=0 2024-09-15T22:28:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 client.28256 shutdown 2024-09-15T22:28:26.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 msgr2=0x5640a4404de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80bbaf0 tx=0x7fa7a80bcb10 comp rx=0 tx=0).stop 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 shutdown_connections 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5640a4477440 0x5640a4479860 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640a43f8680 0x5640a4475990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a43eddc0 0x5640a4404de0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 --2- 172.21.15.77:0/1971652520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a43ed4a0 0x5640a44048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 >> 172.21.15.77:0/1971652520 conn(0x5640a43e9a30 msgr2=0x5640a43e8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7a0ff9640 0 client.28256 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:26.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 shutdown_connections 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.394+0000 7fa7c1975740 1 -- 172.21.15.77:0/1971652520 wait complete. 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.395+0000 7fa7c1975740 20 client.28256 trim_cache size 0 max 16384 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:28:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "c0444a6a-3af3-4110-bba0-b043d17e3fc3": { 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "c0444a6a-3af3-4110-bba0-b043d17e3fc3", 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:26.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 msgr2=0x7fa7a40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 0x7fa7a40a5970 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40095f0 tx=0x7fa7b4011810 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 msgr2=0x7fa7b8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005940 tx=0x7fa7a8031b50 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 shutdown_connections 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa7a40a34b0 0x7fa7a40a5970 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7b81082a0 0x7fa7b819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81036e0 0x7fa7b8198ae0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 --2- 172.21.15.77:0/1423633962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7b8102d30 0x7fa7b81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.396+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 >> 172.21.15.77:0/1423633962 conn(0x7fa7b80fc7d0 msgr2=0x7fa7b8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.397+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 shutdown_connections 2024-09-15T22:28:26.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:26.397+0000 7fa7bf59c640 1 -- 172.21.15.77:0/1423633962 wait complete. 2024-09-15T22:28:26.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 46881 6.06 MB/sec execute 27 sec latency 57.674 ms 2024-09-15T22:28:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:26 smithi196 ceph-mon[35109]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 30 op/s 2024-09-15T22:28:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:26 smithi117 ceph-mon[35021]: pgmap v503: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.0 MiB/s wr, 30 op/s 2024-09-15T22:28:27.145 INFO:tasks.cephfs.filesystem:scrub status for tag:c0444a6a-3af3-4110-bba0-b043d17e3fc3 - {'path': '/', 'tag': 'c0444a6a-3af3-4110-bba0-b043d17e3fc3', 'options': 'recursive,force'} 2024-09-15T22:28:27.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:27 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:27.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 47516 7.07 MB/sec execute 28 sec latency 52.713 ms 2024-09-15T22:28:27.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:27 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:27.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:27 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:28 smithi077 ceph-mon[30980]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-15T22:28:28.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 48137 7.89 MB/sec execute 29 sec latency 52.554 ms 2024-09-15T22:28:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:28 smithi196 ceph-mon[35109]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-15T22:28:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:28 smithi117 ceph-mon[35021]: pgmap v504: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-15T22:28:29.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 48729 8.87 MB/sec execute 30 sec latency 60.466 ms 2024-09-15T22:28:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:30 smithi077 ceph-mon[30980]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 37 op/s 2024-09-15T22:28:30.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 49385 9.75 MB/sec execute 31 sec latency 59.081 ms 2024-09-15T22:28:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:30 smithi196 ceph-mon[35109]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 37 op/s 2024-09-15T22:28:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:30 smithi117 ceph-mon[35021]: pgmap v505: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 546 KiB/s wr, 37 op/s 2024-09-15T22:28:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:28:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:28:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:28:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:28:31.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 49920 10.41 MB/sec execute 32 sec latency 70.870 ms 2024-09-15T22:28:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:28:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:28:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:28:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:28:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:28:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:28:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:28:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:28:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:32 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:28:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:32 smithi077 ceph-mon[30980]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-15T22:28:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:32 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:32.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 50430 10.98 MB/sec execute 33 sec latency 131.558 ms 2024-09-15T22:28:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:32 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:28:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:32 smithi196 ceph-mon[35109]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-15T22:28:32.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:32 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:32 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:28:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:32 smithi117 ceph-mon[35021]: pgmap v506: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-09-15T22:28:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:32 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:28:33.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 50982 11.49 MB/sec execute 34 sec latency 70.141 ms 2024-09-15T22:28:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:34 smithi077 ceph-mon[30980]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:34.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 51475 11.92 MB/sec execute 35 sec latency 69.688 ms 2024-09-15T22:28:34.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:34 smithi196 ceph-mon[35109]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:34 smithi117 ceph-mon[35021]: pgmap v507: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:35.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 52134 12.34 MB/sec execute 36 sec latency 60.781 ms 2024-09-15T22:28:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:36 smithi077 ceph-mon[30980]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:36.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 52890 13.12 MB/sec execute 37 sec latency 52.050 ms 2024-09-15T22:28:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:36 smithi196 ceph-mon[35109]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:36 smithi117 ceph-mon[35021]: pgmap v508: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2024-09-15T22:28:37.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 53416 13.76 MB/sec execute 38 sec latency 176.716 ms 2024-09-15T22:28:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:38 smithi077 ceph-mon[30980]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:28:38.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 54019 14.13 MB/sec execute 39 sec latency 64.722 ms 2024-09-15T22:28:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:38 smithi196 ceph-mon[35109]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:28:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:38 smithi117 ceph-mon[35021]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-09-15T22:28:39.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 54647 14.67 MB/sec execute 40 sec latency 67.551 ms 2024-09-15T22:28:40.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:40 smithi077 ceph-mon[30980]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 407 op/s 2024-09-15T22:28:40.596 INFO:tasks.workunit.client.0.smithi077.stdout: 10 55061 15.04 MB/sec execute 41 sec latency 158.201 ms 2024-09-15T22:28:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:40 smithi196 ceph-mon[35109]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 407 op/s 2024-09-15T22:28:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:40 smithi117 ceph-mon[35021]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.5 MiB/s wr, 407 op/s 2024-09-15T22:28:41.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 55515 15.00 MB/sec execute 42 sec latency 150.328 ms 2024-09-15T22:28:42.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:42 smithi077 ceph-mon[30980]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 8.9 MiB/s wr, 616 op/s 2024-09-15T22:28:42.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 56079 15.31 MB/sec execute 43 sec latency 98.579 ms 2024-09-15T22:28:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:42 smithi196 ceph-mon[35109]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 8.9 MiB/s wr, 616 op/s 2024-09-15T22:28:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:42 smithi117 ceph-mon[35021]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 8.9 MiB/s wr, 616 op/s 2024-09-15T22:28:43.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 56797 15.93 MB/sec execute 44 sec latency 64.215 ms 2024-09-15T22:28:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:44 smithi077 ceph-mon[30980]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:44.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 57377 16.19 MB/sec execute 45 sec latency 65.567 ms 2024-09-15T22:28:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:44 smithi196 ceph-mon[35109]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:44 smithi117 ceph-mon[35021]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:45.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 58058 16.71 MB/sec execute 46 sec latency 53.334 ms 2024-09-15T22:28:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:46 smithi077 ceph-mon[30980]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:46.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 58756 17.18 MB/sec execute 47 sec latency 64.317 ms 2024-09-15T22:28:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:46 smithi196 ceph-mon[35109]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:46 smithi117 ceph-mon[35021]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 404 op/s 2024-09-15T22:28:47.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 59182 17.40 MB/sec execute 48 sec latency 177.285 ms 2024-09-15T22:28:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:48 smithi077 ceph-mon[30980]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 594 op/s 2024-09-15T22:28:48.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 59208 17.08 MB/sec execute 49 sec latency 1159.742 ms 2024-09-15T22:28:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:48 smithi196 ceph-mon[35109]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 594 op/s 2024-09-15T22:28:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:48 smithi117 ceph-mon[35021]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 594 op/s 2024-09-15T22:28:49.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 59363 16.90 MB/sec execute 50 sec latency 1351.961 ms 2024-09-15T22:28:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:50 smithi077 ceph-mon[30980]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 400 op/s 2024-09-15T22:28:50.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 59705 16.94 MB/sec execute 51 sec latency 81.382 ms 2024-09-15T22:28:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:50 smithi196 ceph-mon[35109]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 400 op/s 2024-09-15T22:28:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:50 smithi117 ceph-mon[35021]: pgmap v515: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 400 op/s 2024-09-15T22:28:51.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 60096 16.91 MB/sec execute 52 sec latency 168.382 ms 2024-09-15T22:28:52.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 60812 17.22 MB/sec execute 53 sec latency 58.698 ms 2024-09-15T22:28:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:52 smithi196 ceph-mon[35109]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 784 op/s 2024-09-15T22:28:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:52 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:52 smithi117 ceph-mon[35021]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 784 op/s 2024-09-15T22:28:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:52 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:52.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:52 smithi077 ceph-mon[30980]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 784 op/s 2024-09-15T22:28:52.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:52 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:53.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 61327 17.49 MB/sec execute 54 sec latency 143.571 ms 2024-09-15T22:28:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:53 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:28:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:53 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:28:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:53 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:28:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:53 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:28:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:53 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:28:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:53 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:28:53.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:53 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:28:53.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:53 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:28:53.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:53 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:28:54.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 61997 17.71 MB/sec execute 55 sec latency 58.883 ms 2024-09-15T22:28:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:54 smithi196 ceph-mon[35109]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:54 smithi117 ceph-mon[35021]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:54.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:54 smithi077 ceph-mon[30980]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:55.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 62726 18.23 MB/sec execute 56 sec latency 63.598 ms 2024-09-15T22:28:56.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 63279 18.49 MB/sec execute 57 sec latency 46.278 ms 2024-09-15T22:28:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:56 smithi196 ceph-mon[35109]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:56 smithi117 ceph-mon[35021]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:56.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:56 smithi077 ceph-mon[30980]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 574 op/s 2024-09-15T22:28:57.146 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:28:57.458 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:28:57.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 63678 18.56 MB/sec execute 58 sec latency 193.378 ms 2024-09-15T22:28:58.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 -- 172.21.15.77:0/3976396831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 msgr2=0x7f1ec810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3976396831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec810ea90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009ca0 tx=0x7f1ec002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 -- 172.21.15.77:0/3976396831 shutdown_connections 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3976396831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec810ea90 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3976396831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8102790 0x7f1ec81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3976396831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8101de0 0x7f1ec81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 -- 172.21.15.77:0/3976396831 >> 172.21.15.77:0/3976396831 conn(0x7f1ec80fd880 msgr2=0x7f1ec80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 -- 172.21.15.77:0/3976396831 shutdown_connections 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.150+0000 7f1ecde60640 1 -- 172.21.15.77:0/3976396831 wait complete. 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 Processor -- start 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 -- start start 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 0x7f1ec8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111f70 con 0x7f1ec8106f10 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111df0 con 0x7f1ec8101de0 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.151+0000 7f1ecde60640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81120f0 con 0x7f1ec8102790 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 0x7f1ec8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60580/0 (socket says 172.21.15.77:60580) 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3523253587 learned_addr learned my addr 172.21.15.77:0/3523253587 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3523253587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 msgr2=0x7f1ec8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 0x7f1ec8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 msgr2=0x7f1ec819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3523253587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec819d380 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec7fff640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 0x7f1ec8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800e9f0 tx=0x7f1eb800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.152+0000 7f1ec4ff9640 1 -- 172.21.15.77:0/3523253587 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb800cd80 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.153+0000 7f1ec4ff9640 1 -- 172.21.15.77:0/3523253587 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8005500 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.153+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec819d670 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.153+0000 7f1ec4ff9640 1 -- 172.21.15.77:0/3523253587 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8010690 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.153+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a4f50 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.156+0000 7f1ec4ff9640 1 -- 172.21.15.77:0/3523253587 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1eb80046c0 con 0x7f1ec8101de0 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.157+0000 7f1ed0239740 1 Processor -- start 2024-09-15T22:28:58.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.157+0000 7f1ed0239740 1 -- start start 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.157+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d4278797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.157+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.157+0000 7f1ec4ff9640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 0x7f1eb00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d4278797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47824/0 (socket says 172.21.15.77:47824) 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ed0239740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4277db2e0 con 0x55d427879d10 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d427782ce0 con 0x55d42787ade0 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 -- 172.21.15.77:0/918927902 learned_addr learned my addr 172.21.15.77:0/918927902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4277d72a0 con 0x55d427888d20 2024-09-15T22:28:58.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec4ff9640 1 -- 172.21.15.77:0/3523253587 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1eb80d04b0 con 0x7f1ec8101de0 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 -- 172.21.15.77:0/918927902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 msgr2=0x55d42788b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 -- 172.21.15.77:0/918927902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 msgr2=0x55d4278797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d4278797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.158+0000 7f1ec77fe640 1 -- 172.21.15.77:0/918927902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d427767440 con 0x55d427879d10 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.159+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80a4ae0 tx=0x7f1eb8004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aaeef4d9b2b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.159+0000 7f1eadffb640 1 -- 172.21.15.77:0/918927902 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb809caa0 con 0x55d427879d10 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.159+0000 7f1eadffb640 1 -- 172.21.15.77:0/918927902 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80a5010 con 0x55d427879d10 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.159+0000 7f1eadffb640 1 -- 172.21.15.77:0/918927902 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb809c860 con 0x55d427879d10 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.159+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 0x7f1eb00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 0x7f1eb00a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1ec81088e0 tx=0x7f1ebc005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 msgr2=0x55d42787a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80a4ae0 tx=0x7f1eb8004ba0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 shutdown_connections 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d42787a130 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 --2- 172.21.15.77:0/918927902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d4278797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 >> 172.21.15.77:0/918927902 conn(0x55d427875e40 msgr2=0x55d427876220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 shutdown_connections 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.160+0000 7f1ed0239740 1 -- 172.21.15.77:0/918927902 wait complete. 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 -- start start 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 Processor -- start 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d427872060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d42788b9f0 con 0x55d427888d20 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d427796050 con 0x55d42787ade0 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d42788bb70 con 0x55d427879d10 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ed0239740 10 client.0 _ll_register_callbacks cb 0x55d4277d6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:28:58.260 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d427872060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47838/0 (socket says 172.21.15.77:47838) 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38010/0 (socket says 172.21.15.77:38010) 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.161+0000 7f1ec7fff640 1 -- 172.21.15.77:0/4050672589 learned_addr learned my addr 172.21.15.77:0/4050672589 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 -- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 msgr2=0x55d427872060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d427872060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 -- 172.21.15.77:0/4050672589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 msgr2=0x55d4278725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 -- 172.21.15.77:0/4050672589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42754c3c0 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec7fff640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d42788b680 tx=0x7f1eb809cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd351a6d7e171dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80a0750 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1eb809c810 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb80a5010 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1eb80d00d0 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ed0239740 10 client.28180 fetch_fsmap learned FSMap version 39 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.162+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d42788b9f0 con 0x55d427879d10 2024-09-15T22:28:58.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80d0270 con 0x55d427879d10 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1eb809b260 con 0x55d427879d10 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 10 client.28180 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 10 client.28180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 20 client.28180 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427898590 0x55d42789a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 4 client.28180 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1eb80a5010 con 0x55d427898590 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427898590 0x55d42789a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427898590 0x55d42789a980 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c50566c8b7fd2d server_cookie=f413719880c3ac14 in_seq=0 out_seq=0 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.163+0000 7f1ea6ffd640 10 client.28180 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/4050672589 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1eb80a5010 con 0x55d427898590 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ea6ffd640 10 client.28180 handle_command_reply: tid=0 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 client.28180 shutdown 2024-09-15T22:28:58.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 msgr2=0x55d427874db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d42788b680 tx=0x7f1eb809cbe0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 shutdown_connections 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427898590 0x55d42789a980 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427888d20 0x55d4278725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d427872060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ea6ffd640 0 client.28180 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 --2- 172.21.15.77:0/4050672589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427879d10 0x55d427874db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 >> 172.21.15.77:0/4050672589 conn(0x55d427875e40 msgr2=0x55d42787bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 shutdown_connections 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 1 -- 172.21.15.77:0/4050672589 wait complete. 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.169+0000 7f1ed0239740 20 client.28180 trim_cache size 0 max 16384 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.199+0000 7f1ed0239740 1 Processor -- start 2024-09-15T22:28:58.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.199+0000 7f1ed0239740 1 -- start start 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42787ade0 0x55d427884d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d427888d20 0x55d42789ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81acef0 con 0x55d427879d10 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81aea30 con 0x55d427888d20 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ed0239740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ad5c0 con 0x55d42787ade0 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47844/0 (socket says 172.21.15.77:47844) 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3544788000 learned_addr learned my addr 172.21.15.77:0/3544788000 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec7fff640 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d427888d20 0x55d42789ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42787ade0 0x55d427884d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3544788000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42787ade0 msgr2=0x55d427884d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42787ade0 0x55d427884d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3544788000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d427888d20 msgr2=0x55d42789ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d427888d20 0x55d42789ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 -- 172.21.15.77:0/3544788000 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d427871030 con 0x55d427879d10 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1eb809c330 tx=0x7f1eb809cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f85783ae91c5aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.200+0000 7f1ead7fa640 1 -- 172.21.15.77:0/3544788000 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb809c170 con 0x55d427879d10 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.201+0000 7f1ead7fa640 1 -- 172.21.15.77:0/3544788000 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80d0270 con 0x55d427879d10 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.201+0000 7f1ead7fa640 1 -- 172.21.15.77:0/3544788000 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb809dd70 con 0x55d427879d10 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.201+0000 7f1ed0239740 1 -- 172.21.15.77:0/3544788000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 msgr2=0x55d427894d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.201+0000 7f1ed0239740 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f1eb809c330 tx=0x7f1eb809cac0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 -- 172.21.15.77:0/3544788000 shutdown_connections 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d427888d20 0x55d42789ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42787ade0 0x55d427884d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 --2- 172.21.15.77:0/3544788000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427894d00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 -- 172.21.15.77:0/3544788000 >> 172.21.15.77:0/3544788000 conn(0x55d427875e40 msgr2=0x55d42787bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 -- 172.21.15.77:0/3544788000 shutdown_connections 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 -- 172.21.15.77:0/3544788000 wait complete. 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 -- start start 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.202+0000 7f1ed0239740 1 Processor -- start 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427891770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d42788caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ae360 con 0x55d427879d10 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81adc90 con 0x55d42787ade0 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81aaef0 con 0x55d427888d20 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ed0239740 10 client.0 _ll_register_callbacks cb 0x7f1eb80cdd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d42788caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427891770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d42788caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60642/0 (socket says 172.21.15.77:60642) 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec6ffd640 1 -- 172.21.15.77:0/1162709709 learned_addr learned my addr 172.21.15.77:0/1162709709 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec7fff640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 -- 172.21.15.77:0/1162709709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 msgr2=0x55d42788d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 -- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 msgr2=0x55d42788caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d42788caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 -- 172.21.15.77:0/1162709709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d427901f80 con 0x55d427879d10 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ec77fe640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427891770 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1eb809c560 tx=0x7f1eb800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b023178edf49bed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.203+0000 7f1ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb809c170 con 0x55d427879d10 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f1eb809a6a0 con 0x55d427879d10 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb80d0270 con 0x55d427879d10 2024-09-15T22:28:58.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1eb80a5010 con 0x55d427879d10 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 10 client.19434 fetch_fsmap learned FSMap version 39 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d427901f80 con 0x55d427879d10 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80d0270 con 0x55d427879d10 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1eb809b7f0 con 0x55d427879d10 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 10 client.19434 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 10 client.19434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 20 client.19434 populate_metadata read hostname 'smithi077' 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427903450 0x55d427905840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 4 client.19434 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.204+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4277e8850 con 0x55d427903450 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.205+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427903450 0x55d427905840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.205+0000 7f1ec6ffd640 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427903450 0x55d427905840 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aafb856121f7ed9d server_cookie=e5304982f5970618 in_seq=0 out_seq=0 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.205+0000 7f1ea6ffd640 10 client.19434 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:58.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ea6ffd640 1 -- 172.21.15.77:0/1162709709 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d4277e8850 con 0x55d427903450 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ea6ffd640 10 client.19434 handle_command_reply: tid=0 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 client.19434 shutdown 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 msgr2=0x55d427891770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427891770 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1eb809c560 tx=0x7f1eb800e940 comp rx=0 tx=0).stop 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 shutdown_connections 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d427903450 0x55d427905840 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.207+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d427888d20 0x55d42788d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d42787ade0 0x55d42788caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ea6ffd640 0 client.19434 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 1 --2- 172.21.15.77:0/1162709709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d427879d10 0x55d427891770 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 >> 172.21.15.77:0/1162709709 conn(0x55d427875e40 msgr2=0x55d4278738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 shutdown_connections 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 1 -- 172.21.15.77:0/1162709709 wait complete. 2024-09-15T22:28:58.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ed0239740 20 client.19434 trim_cache size 0 max 16384 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 msgr2=0x7f1eb00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 0x7f1eb00a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1ec81088e0 tx=0x7f1ebc005f90 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 msgr2=0x7f1ec81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800e9f0 tx=0x7f1eb800edf0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 shutdown_connections 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1eb00a33b0 0x7f1eb00a5870 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ec8106f10 0x7f1ec819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ec8102790 0x7f1ec8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 --2- 172.21.15.77:0/3523253587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec8101de0 0x7f1ec81983d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 >> 172.21.15.77:0/3523253587 conn(0x7f1ec80fd880 msgr2=0x7f1ec80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.208+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 shutdown_connections 2024-09-15T22:28:58.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:28:58.209+0000 7f1ecde60640 1 -- 172.21.15.77:0/3523253587 wait complete. 2024-09-15T22:28:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:28:58 smithi077 ceph-mon[30980]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 800 op/s 2024-09-15T22:28:58.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 64222 18.73 MB/sec execute 59 sec latency 190.763 ms 2024-09-15T22:28:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:28:58 smithi196 ceph-mon[35109]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 800 op/s 2024-09-15T22:28:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:28:58 smithi117 ceph-mon[35021]: pgmap v519: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 800 op/s 2024-09-15T22:28:59.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 64779 18.92 MB/sec execute 60 sec latency 86.216 ms 2024-09-15T22:28:59.973 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:28:59.974 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:29:00.337 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:00.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65327 18.92 MB/sec execute 61 sec latency 112.956 ms 2024-09-15T22:29:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:00 smithi196 ceph-mon[35109]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 610 op/s 2024-09-15T22:29:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:00 smithi117 ceph-mon[35021]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 610 op/s 2024-09-15T22:29:00.651 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:00 smithi077 ceph-mon[30980]: pgmap v520: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 610 op/s 2024-09-15T22:29:00.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 -- 172.21.15.77:0/421871611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 msgr2=0x7f29101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 --2- 172.21.15.77:0/421871611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f29101029b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2904009900 tx=0x7f290402a480 comp rx=0 tx=0).stop 2024-09-15T22:29:00.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 -- 172.21.15.77:0/421871611 shutdown_connections 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 --2- 172.21.15.77:0/421871611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910102ef0 0x7f291010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 --2- 172.21.15.77:0/421871611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f29101029b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 --2- 172.21.15.77:0/421871611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910100e20 0x7f2910101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 -- 172.21.15.77:0/421871611 >> 172.21.15.77:0/421871611 conn(0x7f29100fc7d0 msgr2=0x7f29100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 -- 172.21.15.77:0/421871611 shutdown_connections 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.955+0000 7f2915110640 1 -- 172.21.15.77:0/421871611 wait complete. 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 Processor -- start 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 -- start start 2024-09-15T22:29:00.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910100e20 0x7f291019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 0x7f291019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2910110f80 con 0x7f2910102570 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2910110e00 con 0x7f2910100e20 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f2915110640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2910111100 con 0x7f2910102ef0 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f290e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f290e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51024/0 (socket says 172.21.15.77:51024) 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f290e59c640 1 -- 172.21.15.77:0/3815146687 learned_addr learned my addr 172.21.15.77:0/3815146687 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f290ed9d640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910100e20 0x7f291019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.956+0000 7f290f59e640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 0x7f291019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 -- 172.21.15.77:0/3815146687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 msgr2=0x7f291019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 0x7f291019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 -- 172.21.15.77:0/3815146687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910100e20 msgr2=0x7f291019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910100e20 0x7f291019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 -- 172.21.15.77:0/3815146687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291019f660 con 0x7f2910102570 2024-09-15T22:29:00.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290f59e640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 0x7f291019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f290e59c640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f2904003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290403f070 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2904031c30 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904035030 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291019f8f0 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.957+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a7230 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.958+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29101021b0 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.959+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2904042070 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.960+0000 7f28f7fff640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 0x7f28f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.960+0000 7f290ed9d640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 0x7f28f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.960+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f29040f0cf0 con 0x7f2910102570 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.961+0000 7f290ed9d640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 0x7f28f80a5870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2908009bd0 tx=0x7f2908005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:00.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:00.963+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29040bd2e0 con 0x7f2910102570 2024-09-15T22:29:01.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.191+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2910101200 con 0x7f2910102570 2024-09-15T22:29:01.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.191+0000 7f28f7fff640 1 -- 172.21.15.77:0/3815146687 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f2910101200 con 0x7f2910102570 2024-09-15T22:29:01.194 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:29:01.194 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:29:01.195 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:29:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 msgr2=0x7f28f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:01.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 0x7f28f80a5870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2908009bd0 tx=0x7f2908005f90 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 msgr2=0x7f291019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2904006010 tx=0x7f2904003c10 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 shutdown_connections 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f28f80a33b0 0x7f28f80a5870 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2910102ef0 0x7f291019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2910102570 0x7f291019abf0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 --2- 172.21.15.77:0/3815146687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2910100e20 0x7f291019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 >> 172.21.15.77:0/3815146687 conn(0x7f29100fc7d0 msgr2=0x7f291006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 shutdown_connections 2024-09-15T22:29:01.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:01.194+0000 7f2915110640 1 -- 172.21.15.77:0/3815146687 wait complete. 2024-09-15T22:29:01.597 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65360 18.65 MB/sec execute 62 sec latency 951.519 ms 2024-09-15T22:29:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:01 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3815146687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:01.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:01 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3815146687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:01.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:01 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3815146687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:02.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65360 18.36 MB/sec execute 63 sec latency 1951.614 ms 2024-09-15T22:29:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:02 smithi117 ceph-mon[35021]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 772 op/s 2024-09-15T22:29:02.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:02 smithi077 ceph-mon[30980]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 772 op/s 2024-09-15T22:29:02.708 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:29:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:02 smithi196 ceph-mon[35109]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 772 op/s 2024-09-15T22:29:02.961 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:03.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65360 18.07 MB/sec execute 64 sec latency 2951.765 ms 2024-09-15T22:29:03.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 -- 172.21.15.77:0/941355765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c100a80 msgr2=0x7fcb1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 --2- 172.21.15.77:0/941355765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c100a80 0x7fcb1c100ee0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009930 tx=0x7fcb1002a420 comp rx=0 tx=0).stop 2024-09-15T22:29:03.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 -- 172.21.15.77:0/941355765 shutdown_connections 2024-09-15T22:29:03.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 --2- 172.21.15.77:0/941355765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c101420 0x7fcb1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 --2- 172.21.15.77:0/941355765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c100a80 0x7fcb1c100ee0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 --2- 172.21.15.77:0/941355765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 0x7fcb1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 -- 172.21.15.77:0/941355765 >> 172.21.15.77:0/941355765 conn(0x7fcb1c0fc7a0 msgr2=0x7fcb1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 -- 172.21.15.77:0/941355765 shutdown_connections 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.869+0000 7fcb209a1640 1 -- 172.21.15.77:0/941355765 wait complete. 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.870+0000 7fcb209a1640 1 Processor -- start 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.870+0000 7fcb209a1640 1 -- start start 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 0x7fcb1c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 0x7fcb1c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c110e80 con 0x7fcb1c101420 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c110d00 con 0x7fcb1c100a80 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb209a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c111000 con 0x7fcb1c106a80 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb19d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 0x7fcb1c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb19d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51036/0 (socket says 172.21.15.77:51036) 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 0x7fcb1c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb19d9b640 1 -- 172.21.15.77:0/1195984683 learned_addr learned my addr 172.21.15.77:0/1195984683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:03.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 -- 172.21.15.77:0/1195984683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 msgr2=0x7fcb1c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 0x7fcb1c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 -- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 msgr2=0x7fcb1c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.871+0000 7fcb1a59c640 1 -- 172.21.15.77:0/1195984683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c19f620 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.872+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 0x7fcb1c19a670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb0b7fe640 1 -- 172.21.15.77:0/1195984683 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ccc0 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c19f910 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a71f0 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb0b7fe640 1 -- 172.21.15.77:0/1195984683 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0c005020 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.873+0000 7fcb0b7fe640 1 -- 172.21.15.77:0/1195984683 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0049e0 con 0x7fcb1c100a80 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 Processor -- start 2024-09-15T22:29:03.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 -- start start 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb0b7fe640 1 -- 172.21.15.77:0/1195984683 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcb0c004cc0 con 0x7fcb1c100a80 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db57370 0x557d8db59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d8dab22f0 con 0x557d8db4db40 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d8da59cf0 con 0x557d8db57370 2024-09-15T22:29:03.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.877+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d8daae2b0 con 0x557d8db4c5d0 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51044/0 (socket says 172.21.15.77:51044) 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/2287401371 learned_addr learned my addr 172.21.15.77:0/2287401371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/2287401371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 msgr2=0x557d8db4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db4ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/2287401371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db57370 msgr2=0x557d8db59760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db57370 0x557d8db59760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.878+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/2287401371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8da3e3a0 con 0x557d8db4db40 2024-09-15T22:29:03.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.879+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b0d82277badaac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.879+0000 7fcb0b7fe640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 0x7fcb040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.879+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 0x7fcb040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.879+0000 7fcb0b7fe640 1 -- 172.21.15.77:0/1195984683 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcb0c0cfe80 con 0x7fcb1c100a80 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.880+0000 7fcb08ff9640 1 -- 172.21.15.77:0/2287401371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004d20 con 0x557d8db4db40 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.880+0000 7fcb08ff9640 1 -- 172.21.15.77:0/2287401371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14004050 con 0x557d8db4db40 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.880+0000 7fcb08ff9640 1 -- 172.21.15.77:0/2287401371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14005260 con 0x557d8db4db40 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 -- 172.21.15.77:0/2287401371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 msgr2=0x557d8db4c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb1a59c640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 0x7fcb040a58f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557d8db49ac0 tx=0x7fcb0c0d3930 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb140118d0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 -- 172.21.15.77:0/2287401371 shutdown_connections 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db57370 0x557d8db59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/2287401371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4db40 0x557d8db4c090 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 -- 172.21.15.77:0/2287401371 >> 172.21.15.77:0/2287401371 conn(0x557d8db15310 msgr2=0x557d8db48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 -- 172.21.15.77:0/2287401371 shutdown_connections 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.881+0000 7fcb22d7a740 1 -- 172.21.15.77:0/2287401371 wait complete. 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 -- start start 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 Processor -- start 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d8db6b080 con 0x557d8db4c5d0 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d8da6d060 con 0x557d8db4db40 2024-09-15T22:29:03.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d8db6b200 con 0x557d8db57370 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb22d7a740 10 client.0 _ll_register_callbacks cb 0x557d8d7c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51050/0 (socket says 172.21.15.77:51050) 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/3413931856 learned_addr learned my addr 172.21.15.77:0/3413931856 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.882+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/3413931856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 msgr2=0x557d8db545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/3413931856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 msgr2=0x557d8db54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/3413931856 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8d8233c0 con 0x557d8db4c5d0 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009de0 tx=0x7fcb14009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686c8c703b3f9bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004640 con 0x557d8db4c5d0 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcb1400f4e0 con 0x557d8db4c5d0 2024-09-15T22:29:03.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.883+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb14004050 con 0x557d8db4c5d0 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb14034050 con 0x557d8db4c5d0 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 10 client.19452 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d8db76b60 con 0x557d8db4c5d0 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14004050 con 0x557d8db4c5d0 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcb14019730 con 0x557d8db4c5d0 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 10 client.19452 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 10 client.19452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 20 client.19452 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8db77eb0 0x557d8db7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 4 client.19452 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:03.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.884+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d8dabf860 con 0x557d8db77eb0 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.886+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8db77eb0 0x557d8db7a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.887+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8db77eb0 0x557d8db7a2a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed5db64e11646229 server_cookie=47b2b3a8ab987859 in_seq=0 out_seq=0 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.887+0000 7fcb01ffb640 10 client.19452 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.888+0000 7fcb01ffb640 1 -- 172.21.15.77:0/3413931856 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d8dabf860 con 0x557d8db77eb0 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.888+0000 7fcb01ffb640 10 client.19452 handle_command_reply: tid=0 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 client.19452 shutdown 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 msgr2=0x557d8db56d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009de0 tx=0x7fcb14009d30 comp rx=0 tx=0).stop 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 shutdown_connections 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8db77eb0 0x557d8db7a2a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/3413931856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db56d50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 >> 172.21.15.77:0/3413931856 conn(0x557d8db15310 msgr2=0x557d8db4e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb01ffb640 0 client.19452 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:03.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.889+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 shutdown_connections 2024-09-15T22:29:03.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.890+0000 7fcb22d7a740 1 -- 172.21.15.77:0/3413931856 wait complete. 2024-09-15T22:29:03.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.890+0000 7fcb22d7a740 20 client.19452 trim_cache size 0 max 16384 2024-09-15T22:29:03.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 Processor -- start 2024-09-15T22:29:03.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 -- start start 2024-09-15T22:29:03.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db47c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db57370 0x557d8db4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc00ff60 con 0x557d8db57370 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc011aa0 con 0x557d8db4db40 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.914+0000 7fcb22d7a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc010630 con 0x557d8db4c5d0 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb19d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db47c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47420/0 (socket says 172.21.15.77:47420) 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1536590109 learned_addr learned my addr 172.21.15.77:0/1536590109 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb19d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db47c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39938/0 (socket says 172.21.15.77:39938) 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db57370 0x557d8db4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1536590109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 msgr2=0x557d8db47c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db47c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1536590109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db57370 msgr2=0x557d8db4f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db57370 0x557d8db4f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.915+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1536590109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d8db56970 con 0x557d8db4c5d0 2024-09-15T22:29:03.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.916+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb14006010 tx=0x7fcb14014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf390ba577a02fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.916+0000 7fcb03fff640 1 -- 172.21.15.77:0/1536590109 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14004000 con 0x557d8db4c5d0 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.916+0000 7fcb03fff640 1 -- 172.21.15.77:0/1536590109 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14019910 con 0x557d8db4c5d0 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.916+0000 7fcb03fff640 1 -- 172.21.15.77:0/1536590109 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14025560 con 0x557d8db4c5d0 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1536590109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 msgr2=0x557d8db48df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb14006010 tx=0x7fcb14014040 comp rx=0 tx=0).stop 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1536590109 shutdown_connections 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db57370 0x557d8db4f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db47c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1536590109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db4c5d0 0x557d8db48df0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1536590109 >> 172.21.15.77:0/1536590109 conn(0x557d8db15310 msgr2=0x557d8db47500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1536590109 shutdown_connections 2024-09-15T22:29:03.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.918+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1536590109 wait complete. 2024-09-15T22:29:03.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.919+0000 7fcb22d7a740 1 -- start start 2024-09-15T22:29:03.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.919+0000 7fcb22d7a740 1 Processor -- start 2024-09-15T22:29:03.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.920+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.920+0000 7fcb22d7a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.920+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.920+0000 7fcb1ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51068/0 (socket says 172.21.15.77:51068) 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.920+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1796569801 learned_addr learned my addr 172.21.15.77:0/1796569801 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0113d0 con 0x557d8db4c5d0 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc010d00 con 0x557d8db4db40 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc00df60 con 0x557d8db57370 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb22d7a740 10 client.0 _ll_register_callbacks cb 0x7fcb0c0d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1796569801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 msgr2=0x557d8db50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1796569801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 msgr2=0x557d8db70a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:03.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db70a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d8db50b70 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db70a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb1ad9d640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcb14011910 tx=0x7fcb14009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa8cacd069330a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.921+0000 7fcb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1403f040 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcb14004000 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb14019910 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb14019640 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb22d7a740 10 client.19458 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.922+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d8dbea7c0 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14019910 con 0x557d8db4c5d0 2024-09-15T22:29:03.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcb14025bc0 con 0x557d8db4c5d0 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 10 client.19458 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 10 client.19458 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 20 client.19458 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8dbeadd0 0x557d8dbed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 4 client.19458 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.923+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d8dab9d70 con 0x557d8dbeadd0 2024-09-15T22:29:03.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.924+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8dbeadd0 0x557d8dbed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.924+0000 7fcb19d9b640 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8dbeadd0 0x557d8dbed1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53807a65e1c6efbc server_cookie=ac3f4290295bab7f in_seq=0 out_seq=0 2024-09-15T22:29:03.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.924+0000 7fcb01ffb640 10 client.19458 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:03.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.924+0000 7fcb01ffb640 1 -- 172.21.15.77:0/1796569801 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d8dab9d70 con 0x557d8dbeadd0 2024-09-15T22:29:03.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.924+0000 7fcb01ffb640 10 client.19458 handle_command_reply: tid=0 2024-09-15T22:29:03.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 client.19458 shutdown 2024-09-15T22:29:03.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 msgr2=0x557d8db704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcb14011910 tx=0x7fcb14009fd0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 shutdown_connections 2024-09-15T22:29:03.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557d8dbeadd0 0x557d8dbed1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d8db57370 0x557d8db50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d8db4db40 0x557d8db70a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 --2- 172.21.15.77:0/1796569801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d8db4c5d0 0x557d8db704f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 >> 172.21.15.77:0/1796569801 conn(0x557d8db15310 msgr2=0x557d8db47d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.926+0000 7fcb01ffb640 0 client.19458 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.927+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 shutdown_connections 2024-09-15T22:29:03.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.927+0000 7fcb22d7a740 1 -- 172.21.15.77:0/1796569801 wait complete. 2024-09-15T22:29:03.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.927+0000 7fcb22d7a740 20 client.19458 trim_cache size 0 max 16384 2024-09-15T22:29:03.934 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:03.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 msgr2=0x7fcb040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 0x7fcb040a58f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x557d8db49ac0 tx=0x7fcb0c0d3930 comp rx=0 tx=0).stop 2024-09-15T22:29:03.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 msgr2=0x7fcb1c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 0x7fcb1c19a670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e9f0 tx=0x7fcb0c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 shutdown_connections 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcb040a3430 0x7fcb040a58f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb1c106a80 0x7fcb1c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb1c101420 0x7fcb1c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 --2- 172.21.15.77:0/1195984683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb1c100a80 0x7fcb1c19a670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 >> 172.21.15.77:0/1195984683 conn(0x7fcb1c0fc7a0 msgr2=0x7fcb1c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 shutdown_connections 2024-09-15T22:29:03.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:03.928+0000 7fcb209a1640 1 -- 172.21.15.77:0/1195984683 wait complete. 2024-09-15T22:29:04.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65453 17.84 MB/sec execute 65 sec latency 3232.917 ms 2024-09-15T22:29:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:04 smithi117 ceph-mon[35021]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:04.727 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:04 smithi077 ceph-mon[30980]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:04.765 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:29:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:04 smithi196 ceph-mon[35109]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:05.064 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 -- 172.21.15.77:0/530902430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 msgr2=0x7fd1b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/530902430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b8106e60 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009930 tx=0x7fd1a802a420 comp rx=0 tx=0).stop 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 -- 172.21.15.77:0/530902430 shutdown_connections 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/530902430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8101420 0x7fd1b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/530902430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8100a80 0x7fd1b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/530902430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b8106e60 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 -- 172.21.15.77:0/530902430 >> 172.21.15.77:0/530902430 conn(0x7fd1b80fc7a0 msgr2=0x7fd1b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 -- 172.21.15.77:0/530902430 shutdown_connections 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.537+0000 7fd1be8e5640 1 -- 172.21.15.77:0/530902430 wait complete. 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 Processor -- start 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 -- start start 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8100a80 0x7fd1b8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8101420 0x7fd1b8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8110e40 con 0x7fd1b8106a80 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8110cc0 con 0x7fd1b8100a80 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1be8e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b8110fc0 con 0x7fd1b8101420 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.538+0000 7fd1bce82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8100a80 0x7fd1b8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51086/0 (socket says 172.21.15.77:51086) 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 -- 172.21.15.77:0/4072164728 learned_addr learned my addr 172.21.15.77:0/4072164728 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1affff640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8101420 0x7fd1b8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 -- 172.21.15.77:0/4072164728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8101420 msgr2=0x7fd1b8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8101420 0x7fd1b8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 -- 172.21.15.77:0/4072164728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8100a80 msgr2=0x7fd1b8197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8100a80 0x7fd1b8197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 -- 172.21.15.77:0/4072164728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b819c2d0 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1bce82640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000cad0 tx=0x7fd1b000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1b5ffb640 1 -- 172.21.15.77:0/4072164728 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000ad60 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b819c5c0 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1b5ffb640 1 -- 172.21.15.77:0/4072164728 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b0005020 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1b5ffb640 1 -- 172.21.15.77:0/4072164728 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b000f6b0 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.539+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b81a3ea0 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 Processor -- start 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1b5ffb640 1 -- 172.21.15.77:0/4072164728 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd1b0005560 con 0x7fd1b8106a80 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 -- start start 2024-09-15T22:29:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55748abed2f0 con 0x55748ac94580 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55748ab94cf0 con 0x55748ac897e0 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55748abe92b0 con 0x55748ac8ad50 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51102/0 (socket says 172.21.15.77:51102) 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 -- 172.21.15.77:0/1197560096 learned_addr learned my addr 172.21.15.77:0/1197560096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac89c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 -- 172.21.15.77:0/1197560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 msgr2=0x55748ac892a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac892a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 -- 172.21.15.77:0/1197560096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 msgr2=0x55748ac89c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac89c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1affff640 1 -- 172.21.15.77:0/1197560096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748ab793a0 con 0x55748ac94580 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.542+0000 7fd1b5ffb640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1affff640 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0011560 tx=0x7fd1a000a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a98f50675998450 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1ae7fc640 1 -- 172.21.15.77:0/1197560096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0005220 con 0x55748ac94580 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1b5ffb640 1 -- 172.21.15.77:0/4072164728 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd1b00cfff0 con 0x7fd1b8106a80 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1ae7fc640 1 -- 172.21.15.77:0/1197560096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a0004c80 con 0x55748ac94580 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1ae7fc640 1 -- 172.21.15.77:0/1197560096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000fcb0 con 0x55748ac94580 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/1197560096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 msgr2=0x55748ac96970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 0x7fd1a40a5870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009900 tx=0x7fd1a80058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.543+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0011560 tx=0x7fd1a000a810 comp rx=0 tx=0).stop 2024-09-15T22:29:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/1197560096 shutdown_connections 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748ac96970 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac89c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/1197560096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/1197560096 >> 172.21.15.77:0/1197560096 conn(0x55748ac50310 msgr2=0x55748ac85e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/1197560096 shutdown_connections 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/1197560096 wait complete. 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 -- start start 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 Processor -- start 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.544+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac94580 0x55748ac919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55748ac97280 con 0x55748ac897e0 2024-09-15T22:29:05.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55748aba8060 con 0x55748ac94580 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55748ac97400 con 0x55748ac8ad50 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 10 client.0 _ll_register_callbacks cb 0x55748a8fb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51108/0 (socket says 172.21.15.77:51108) 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3500273154 learned_addr learned my addr 172.21.15.77:0/3500273154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1affff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1bce82640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac94580 0x55748ac919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 msgr2=0x55748ac91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3500273154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac94580 msgr2=0x55748ac919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac94580 0x55748ac919f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 -- 172.21.15.77:0/3500273154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748a95e3c0 con 0x55748ac897e0 2024-09-15T22:29:05.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009930 tx=0x7fd1a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15836ccdf8ab3544 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1977fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a804e070 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd1a804a450 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a0004c80 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd1a805a050 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 10 client.19470 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.545+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55748aca2d60 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a0004c80 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd1a803d7b0 con 0x55748ac897e0 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 10 client.19470 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 10 client.19470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:29:05.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 20 client.19470 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748aca4170 0x55748aca65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 4 client.19470 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55748abfa860 con 0x55748aca4170 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1affff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748aca4170 0x55748aca65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1977fe640 10 client.19470 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.546+0000 7fd1affff640 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748aca4170 0x55748aca65a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=362e17c4bf7d68d2 server_cookie=270e7ce9b41c435b in_seq=0 out_seq=0 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1977fe640 1 -- 172.21.15.77:0/3500273154 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55748abfa860 con 0x55748aca4170 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1977fe640 10 client.19470 handle_command_reply: tid=0 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 client.19470 shutdown 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 msgr2=0x55748ac93fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009930 tx=0x7fd1a8009260 comp rx=0 tx=0).stop 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 shutdown_connections 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748aca4170 0x55748aca65a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac94580 0x55748ac919f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac8ad50 0x55748ac91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/3500273154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac897e0 0x55748ac93fb0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 >> 172.21.15.77:0/3500273154 conn(0x55748ac50310 msgr2=0x55748ac8bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1977fe640 0 client.19470 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 shutdown_connections 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.547+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/3500273154 wait complete. 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.548+0000 7fd1c0cbe740 20 client.19470 trim_cache size 0 max 16384 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.560+0000 7fd1c0cbe740 1 Processor -- start 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.560+0000 7fd1c0cbe740 1 -- start start 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.560+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac897e0 0x55748ac85910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.560+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac8ad50 0x55748ac85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81acd70 con 0x55748ac94580 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81ae8b0 con 0x55748ac8ad50 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81ad440 con 0x55748ac897e0 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51122/0 (socket says 172.21.15.77:51122) 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 -- 172.21.15.77:0/2168009096 learned_addr learned my addr 172.21.15.77:0/2168009096 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1affff640 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac8ad50 0x55748ac85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 -- 172.21.15.77:0/2168009096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac897e0 msgr2=0x55748ac85910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac897e0 0x55748ac85910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 -- 172.21.15.77:0/2168009096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac8ad50 msgr2=0x55748ac85e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac8ad50 0x55748ac85e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 -- 172.21.15.77:0/2168009096 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748aca0a00 con 0x55748ac94580 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1bce82640 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd1b009bed0 tx=0x7fd1b0098260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39992b3b07666e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1adffb640 1 -- 172.21.15.77:0/2168009096 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b001d040 con 0x55748ac94580 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1adffb640 1 -- 172.21.15.77:0/2168009096 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b00a4bb0 con 0x55748ac94580 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.561+0000 7fd1adffb640 1 -- 172.21.15.77:0/2168009096 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00a4720 con 0x55748ac94580 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2168009096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 msgr2=0x55748acabec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd1b009bed0 tx=0x7fd1b0098260 comp rx=0 tx=0).stop 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2168009096 shutdown_connections 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac94580 0x55748acabec0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac8ad50 0x55748ac85e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2168009096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac897e0 0x55748ac85910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2168009096 >> 172.21.15.77:0/2168009096 conn(0x55748ac50310 msgr2=0x55748ac8fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2168009096 shutdown_connections 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2168009096 wait complete. 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 -- start start 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.562+0000 7fd1c0cbe740 1 Processor -- start 2024-09-15T22:29:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac9ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac94580 0x55748ac8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81ae1e0 con 0x55748ac8ad50 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81adb10 con 0x55748ac897e0 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b81aad70 con 0x55748ac94580 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 10 client.0 _ll_register_callbacks cb 0x7fd1a8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51134/0 (socket says 172.21.15.77:51134) 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 -- 172.21.15.77:0/2562343525 learned_addr learned my addr 172.21.15.77:0/2562343525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:05.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac9ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1bce82640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac94580 0x55748ac8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 -- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac94580 msgr2=0x55748ac8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac94580 0x55748ac8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 -- 172.21.15.77:0/2562343525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 msgr2=0x55748ac9ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac9ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 -- 172.21.15.77:0/2562343525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748ac8e700 con 0x55748ac8ad50 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac9ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1affff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000f780 tx=0x7fd1a000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d77771a067d182b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1977fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000fad0 con 0x55748ac8ad50 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fd1a0021040 con 0x55748ac8ad50 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.563+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b00a4bb0 con 0x55748ac8ad50 2024-09-15T22:29:05.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd1a0039050 con 0x55748ac8ad50 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 10 client.19476 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55748ad0eb80 con 0x55748ac8ad50 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b00a4bb0 con 0x55748ac8ad50 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd1a001b070 con 0x55748ac8ad50 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 10 client.19476 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 10 client.19476 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 20 client.19476 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748ad0f110 0x55748ad11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 4 client.19476 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55748abf4d70 con 0x55748ad0f110 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.564+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748ad0f110 0x55748ad11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1977fe640 10 client.19476 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1b7fff640 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748ad0f110 0x55748ad11530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60e1c10c870cdebc server_cookie=b059d9e5747c2eec in_seq=0 out_seq=0 2024-09-15T22:29:05.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1977fe640 1 -- 172.21.15.77:0/2562343525 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55748abf4d70 con 0x55748ad0f110 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1977fe640 10 client.19476 handle_command_reply: tid=0 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 client.19476 shutdown 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 msgr2=0x55748ac9cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000f780 tx=0x7fd1a000af70 comp rx=0 tx=0).stop 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 shutdown_connections 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55748ad0f110 0x55748ad11530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55748ac94580 0x55748ac8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1977fe640 0 client.19476 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55748ac8ad50 0x55748ac9cf80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 --2- 172.21.15.77:0/2562343525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55748ac897e0 0x55748ac9ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 >> 172.21.15.77:0/2562343525 conn(0x55748ac50310 msgr2=0x55748ac8fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 shutdown_connections 2024-09-15T22:29:05.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 1 -- 172.21.15.77:0/2562343525 wait complete. 2024-09-15T22:29:05.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.565+0000 7fd1c0cbe740 20 client.19476 trim_cache size 0 max 16384 2024-09-15T22:29:05.585 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:05.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 msgr2=0x7fd1a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 0x7fd1a40a5870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009900 tx=0x7fd1a80058d0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 msgr2=0x7fd1b819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000cad0 tx=0x7fd1b000ced0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 shutdown_connections 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd1a40a33b0 0x7fd1a40a5870 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1b8106a80 0x7fd1b819bbf0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1b8101420 0x7fd1b8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 --2- 172.21.15.77:0/4072164728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1b8100a80 0x7fd1b8197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 >> 172.21.15.77:0/4072164728 conn(0x7fd1b80fc7a0 msgr2=0x7fd1b80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 shutdown_connections 2024-09-15T22:29:05.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:05.566+0000 7fd1be8e5640 1 -- 172.21.15.77:0/4072164728 wait complete. 2024-09-15T22:29:05.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65455 17.58 MB/sec execute 66 sec latency 1564.801 ms 2024-09-15T22:29:06.448 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:29:06.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65455 17.32 MB/sec execute 67 sec latency 2564.871 ms 2024-09-15T22:29:06.691 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:06.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:06 smithi077 ceph-mon[30980]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:06 smithi196 ceph-mon[35109]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:06.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:06 smithi117 ceph-mon[35021]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.3 MiB/s wr, 387 op/s 2024-09-15T22:29:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 -- 172.21.15.77:0/3746649721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 msgr2=0x7fcbe0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 --2- 172.21.15.77:0/3746649721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe0105eb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-09-15T22:29:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 -- 172.21.15.77:0/3746649721 shutdown_connections 2024-09-15T22:29:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 --2- 172.21.15.77:0/3746649721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 0x7fcbe010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 --2- 172.21.15.77:0/3746649721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 0x7fcbe0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 --2- 172.21.15.77:0/3746649721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe0105eb0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 -- 172.21.15.77:0/3746649721 >> 172.21.15.77:0/3746649721 conn(0x7fcbe0077210 msgr2=0x7fcbe00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 -- 172.21.15.77:0/3746649721 shutdown_connections 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.210+0000 7fcbe5033640 1 -- 172.21.15.77:0/3746649721 wait complete. 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 Processor -- start 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 -- start start 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 0x7fcbe0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 0x7fcbe0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0112000 con 0x7fcbe0105ad0 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0111e80 con 0x7fcbe01011c0 2024-09-15T22:29:07.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbe5033640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0112180 con 0x7fcbe0101b60 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51152/0 (socket says 172.21.15.77:51152) 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 -- 172.21.15.77:0/545582213 learned_addr learned my addr 172.21.15.77:0/545582213 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbded9d640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 0x7fcbe0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbde59c640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 0x7fcbe0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 -- 172.21.15.77:0/545582213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 msgr2=0x7fcbe0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 0x7fcbe0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 -- 172.21.15.77:0/545582213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 msgr2=0x7fcbe0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 0x7fcbe0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.211+0000 7fcbdf59e640 1 -- 172.21.15.77:0/545582213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe019d3f0 con 0x7fcbe0105ad0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbdf59e640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800b9e0 tx=0x7fcbd800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbc7fff640 1 -- 172.21.15.77:0/545582213 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8019070 con 0x7fcbe0105ad0 2024-09-15T22:29:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbc7fff640 1 -- 172.21.15.77:0/545582213 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd8014440 con 0x7fcbe0105ad0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe019d6e0 con 0x7fcbe0105ad0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbc7fff640 1 -- 172.21.15.77:0/545582213 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80125c0 con 0x7fcbe0105ad0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.212+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a4fc0 con 0x7fcbe0105ad0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.214+0000 7fcbc7fff640 1 -- 172.21.15.77:0/545582213 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcbd801c070 con 0x7fcbe0105ad0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 Processor -- start 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 -- start start 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd2729090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd2736760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556dd268f2f0 con 0x556dd27295d0 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556dd2636cf0 con 0x556dd272ab40 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbe740c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556dd268b2b0 con 0x556dd2734370 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbc7fff640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 0x7fcbc80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51156/0 (socket says 172.21.15.77:51156) 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 -- 172.21.15.77:0/3901798974 learned_addr learned my addr 172.21.15.77:0/3901798974 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbde59c640 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd2736760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbdf59e640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 0x7fcbc80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbde59c640 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd2729090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbc7fff640 1 -- 172.21.15.77:0/545582213 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcbd80cf4c0 con 0x7fcbe0105ad0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 -- 172.21.15.77:0/3901798974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 msgr2=0x556dd2736760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd2736760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 -- 172.21.15.77:0/3901798974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 msgr2=0x556dd2729090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd2729090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.215+0000 7fcbded9d640 1 -- 172.21.15.77:0/3901798974 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd261b3a0 con 0x556dd27295d0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbded9d640 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0696dcdb1ad4c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbdf59e640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 0x7fcbc80a5870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbd809b560 tx=0x7fcbd806d350 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbc57fa640 1 -- 172.21.15.77:0/3901798974 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0035030 con 0x556dd27295d0 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbc57fa640 1 -- 172.21.15.77:0/3901798974 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0040070 con 0x556dd27295d0 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbc57fa640 1 -- 172.21.15.77:0/3901798974 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0031a40 con 0x556dd27295d0 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbe740c740 1 -- 172.21.15.77:0/3901798974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 msgr2=0x556dd2729a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.216+0000 7fcbe740c740 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd0004170 comp rx=0 tx=0).stop 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- 172.21.15.77:0/3901798974 shutdown_connections 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd2736760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2729a10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- 172.21.15.77:0/3901798974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd2729090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- 172.21.15.77:0/3901798974 >> 172.21.15.77:0/3901798974 conn(0x556dd26f2310 msgr2=0x556dd2725c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- 172.21.15.77:0/3901798974 shutdown_connections 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- 172.21.15.77:0/3901798974 wait complete. 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- start start 2024-09-15T22:29:07.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 Processor -- start 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd27295d0 0x556dd2733d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd272ab40 0x556dd2730ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556dd2737010 con 0x556dd2734370 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556dd264a060 con 0x556dd27295d0 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556dd2737190 con 0x556dd272ab40 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbe740c740 10 client.0 _ll_register_callbacks cb 0x556dd239d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51164/0 (socket says 172.21.15.77:51164) 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.217+0000 7fcbde59c640 1 -- 172.21.15.77:0/2785912883 learned_addr learned my addr 172.21.15.77:0/2785912883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd27295d0 0x556dd2733d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbded9d640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd272ab40 0x556dd2730ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 -- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd272ab40 msgr2=0x556dd2730ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd272ab40 0x556dd2730ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 -- 172.21.15.77:0/2785912883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd27295d0 msgr2=0x556dd2733d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd27295d0 0x556dd2733d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 -- 172.21.15.77:0/2785912883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd24003c0 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4011460 tx=0x7fcbd400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb609dea1ac07946 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004030 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcbd40041d0 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd0040070 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4028050 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbe740c740 10 client.19488 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556dd2742ae0 con 0x556dd2734370 2024-09-15T22:29:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.218+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0040070 con 0x556dd2734370 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd400b4e0 con 0x556dd2734370 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 10 client.19488 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 10 client.19488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 20 client.19488 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd2743ef0 0x556dd27462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 4 client.19488 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dd269c860 con 0x556dd2743ef0 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbded9d640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd2743ef0 0x556dd27462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbded9d640 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd2743ef0 0x556dd27462e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=607446a1e1c2ba5d server_cookie=20f4ce534c58c218 in_seq=0 out_seq=0 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.219+0000 7fcbc2ff5640 10 client.19488 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/2785912883 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556dd269c860 con 0x556dd2743ef0 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbc2ff5640 10 client.19488 handle_command_reply: tid=0 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 client.19488 shutdown 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 msgr2=0x556dd27315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4011460 tx=0x7fcbd400cb30 comp rx=0 tx=0).stop 2024-09-15T22:29:07.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 shutdown_connections 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd2743ef0 0x556dd27462e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd2734370 0x556dd27315e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd272ab40 0x556dd2730ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2785912883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd27295d0 0x556dd2733d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbc2ff5640 0 client.19488 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 >> 172.21.15.77:0/2785912883 conn(0x556dd26f2310 msgr2=0x556dd272b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 shutdown_connections 2024-09-15T22:29:07.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 1 -- 172.21.15.77:0/2785912883 wait complete. 2024-09-15T22:29:07.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.220+0000 7fcbe740c740 20 client.19488 trim_cache size 0 max 16384 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.233+0000 7fcbe740c740 1 Processor -- start 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.233+0000 7fcbe740c740 1 -- start start 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01af070 con 0x556dd27295d0 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b0bb0 con 0x556dd272ab40 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbe740c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01af740 con 0x556dd2734370 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51172/0 (socket says 172.21.15.77:51172) 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 -- 172.21.15.77:0/2266235770 learned_addr learned my addr 172.21.15.77:0/2266235770 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbded9d640 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 -- 172.21.15.77:0/2266235770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 msgr2=0x556dd272c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 -- 172.21.15.77:0/2266235770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 msgr2=0x556dd272c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 -- 172.21.15.77:0/2266235770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd2746650 con 0x556dd27295d0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbded9d640 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbde59c640 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4012bd0 tx=0x7fcbd4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c0dcb1f2b25982 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbc4ff9640 1 -- 172.21.15.77:0/2266235770 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031070 con 0x556dd27295d0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbc4ff9640 1 -- 172.21.15.77:0/2266235770 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd400b670 con 0x556dd27295d0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.234+0000 7fcbc4ff9640 1 -- 172.21.15.77:0/2266235770 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40052d0 con 0x556dd27295d0 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- 172.21.15.77:0/2266235770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 msgr2=0x556dd2724bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4012bd0 tx=0x7fcbd4005c50 comp rx=0 tx=0).stop 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- 172.21.15.77:0/2266235770 shutdown_connections 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 --2- 172.21.15.77:0/2266235770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd2724bf0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- 172.21.15.77:0/2266235770 >> 172.21.15.77:0/2266235770 conn(0x556dd26f2310 msgr2=0x556dd27244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- 172.21.15.77:0/2266235770 shutdown_connections 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- 172.21.15.77:0/2266235770 wait complete. 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 -- start start 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.235+0000 7fcbe740c740 1 Processor -- start 2024-09-15T22:29:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b04e0 con 0x556dd27295d0 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01afe10 con 0x556dd272ab40 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01ad070 con 0x556dd2734370 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbe740c740 10 client.0 _ll_register_callbacks cb 0x7fcbd400cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51184/0 (socket says 172.21.15.77:51184) 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 -- 172.21.15.77:0/1298596240 learned_addr learned my addr 172.21.15.77:0/1298596240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbded9d640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 -- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 msgr2=0x556dd272e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 -- 172.21.15.77:0/1298596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 msgr2=0x556dd272dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 -- 172.21.15.77:0/1298596240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd272e6a0 con 0x556dd27295d0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbded9d640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbde59c640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c470 tx=0x7fcbd400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c588c244150f7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.236+0000 7fcbc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031050 con 0x556dd27295d0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7fcbd4004030 con 0x556dd27295d0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd400b670 con 0x556dd27295d0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbd400bd10 con 0x556dd27295d0 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 10 client.19494 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556dd27ae850 con 0x556dd27295d0 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd400b670 con 0x556dd27295d0 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcbd4021b10 con 0x556dd27295d0 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 10 client.19494 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 10 client.19494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 20 client.19494 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd27aee90 0x556dd27b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 4 client.19494 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.237+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dd2696d70 con 0x556dd27aee90 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbded9d640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd27aee90 0x556dd27b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbc2ff5640 10 client.19494 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbded9d640 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd27aee90 0x556dd27b12b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=940891b82d0daa93 server_cookie=9b71ad361a81603f in_seq=0 out_seq=0 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbc2ff5640 1 -- 172.21.15.77:0/1298596240 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dd2696d70 con 0x556dd27aee90 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbc2ff5640 10 client.19494 handle_command_reply: tid=0 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 client.19494 shutdown 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 msgr2=0x556dd273c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c470 tx=0x7fcbd400c570 comp rx=0 tx=0).stop 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 shutdown_connections 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x556dd27aee90 0x556dd27b12b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dd2734370 0x556dd272e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556dd272ab40 0x556dd272dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 --2- 172.21.15.77:0/1298596240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556dd27295d0 0x556dd273c910 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.238+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 >> 172.21.15.77:0/1298596240 conn(0x556dd26f2310 msgr2=0x556dd2733070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbc2ff5640 0 client.19494 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 shutdown_connections 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe740c740 1 -- 172.21.15.77:0/1298596240 wait complete. 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe740c740 20 client.19494 trim_cache size 0 max 16384 2024-09-15T22:29:07.258 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 msgr2=0x7fcbc80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 0x7fcbc80a5870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcbd809b560 tx=0x7fcbd806d350 comp rx=0 tx=0).stop 2024-09-15T22:29:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 msgr2=0x7fcbe019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800b9e0 tx=0x7fcbd800bea0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 shutdown_connections 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fcbc80a33b0 0x7fcbc80a5870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbe0105ad0 0x7fcbe019cd10 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0101b60 0x7fcbe0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 --2- 172.21.15.77:0/545582213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe01011c0 0x7fcbe0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 >> 172.21.15.77:0/545582213 conn(0x7fcbe0077210 msgr2=0x7fcbe0107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 shutdown_connections 2024-09-15T22:29:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:07.239+0000 7fcbe5033640 1 -- 172.21.15.77:0/545582213 wait complete. 2024-09-15T22:29:07.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65455 17.06 MB/sec execute 68 sec latency 3564.910 ms 2024-09-15T22:29:08.076 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:29:08.442 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:08.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 65983 17.26 MB/sec execute 69 sec latency 3701.181 ms 2024-09-15T22:29:08.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:08 smithi077 ceph-mon[30980]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-15T22:29:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:08 smithi196 ceph-mon[35109]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-15T22:29:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:08 smithi117 ceph-mon[35021]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 -- 172.21.15.77:0/611618523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 msgr2=0x7f3fdc109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 --2- 172.21.15.77:0/611618523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc109830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009900 tx=0x7f3fcc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 -- 172.21.15.77:0/611618523 shutdown_connections 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 --2- 172.21.15.77:0/611618523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 --2- 172.21.15.77:0/611618523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc109e00 0x7f3fdc10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 --2- 172.21.15.77:0/611618523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc109830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:08.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 -- 172.21.15.77:0/611618523 >> 172.21.15.77:0/611618523 conn(0x7f3fdc073660 msgr2=0x7f3fdc073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:08.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 -- 172.21.15.77:0/611618523 shutdown_connections 2024-09-15T22:29:08.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.995+0000 7f3fe2214640 1 -- 172.21.15.77:0/611618523 wait complete. 2024-09-15T22:29:09.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.996+0000 7f3fe2214640 1 Processor -- start 2024-09-15T22:29:09.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.996+0000 7f3fe2214640 1 -- start start 2024-09-15T22:29:09.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.996+0000 7f3fe2214640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc116c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.996+0000 7f3fe2214640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc117140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe2214640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 0x7f3fdc117680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe2214640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc11a5d0 con 0x7f3fdc069ab0 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe2214640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc11a450 con 0x7f3fdc109450 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe2214640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc11a750 con 0x7f3fdc109e00 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc117140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc117140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37322/0 (socket says 172.21.15.77:37322) 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe0a11640 1 -- 172.21.15.77:0/1236547118 learned_addr learned my addr 172.21.15.77:0/1236547118 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc116c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 0x7f3fdc117680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 -- 172.21.15.77:0/1236547118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 msgr2=0x7f3fdc117680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 0x7f3fdc117680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 -- 172.21.15.77:0/1236547118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 msgr2=0x7f3fdc117140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc117140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.997+0000 7f3fe1212640 1 -- 172.21.15.77:0/1236547118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc1b3440 con 0x7f3fdc069ab0 2024-09-15T22:29:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.998+0000 7f3fe1212640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc116c00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005940 tx=0x7f3fcc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.998+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 0x7f3fdc117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:09.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.998+0000 7f3fd27fc640 1 -- 172.21.15.77:0/1236547118 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc03f030 con 0x7f3fdc069ab0 2024-09-15T22:29:09.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.998+0000 7f3fd27fc640 1 -- 172.21.15.77:0/1236547118 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc0044c0 con 0x7f3fdc069ab0 2024-09-15T22:29:09.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.999+0000 7f3fd27fc640 1 -- 172.21.15.77:0/1236547118 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc035070 con 0x7f3fdc069ab0 2024-09-15T22:29:09.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.999+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc1b3610 con 0x7f3fdc069ab0 2024-09-15T22:29:09.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:08.999+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc1b3a90 con 0x7f3fdc069ab0 2024-09-15T22:29:09.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 Processor -- start 2024-09-15T22:29:09.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 -- start start 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cec5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cec5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55884ce2b2f0 con 0x55884ced0550 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55884cdd2cf0 con 0x55884cec6d20 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.003+0000 7f3fe584f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55884ce272b0 con 0x55884cec57b0 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe1a13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cec5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fd27fc640 1 -- 172.21.15.77:0/1236547118 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3fcc042030 con 0x7f3fdc069ab0 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe1a13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cec5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37324/0 (socket says 172.21.15.77:37324) 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe1a13640 1 -- 172.21.15.77:0/2398897286 learned_addr learned my addr 172.21.15.77:0/2398897286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:09.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49780/0 (socket says 172.21.15.77:49780) 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe1212640 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cec5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 -- 172.21.15.77:0/2398897286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 msgr2=0x55884cec5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cec5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 -- 172.21.15.77:0/2398897286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 msgr2=0x55884cec5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cec5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 -- 172.21.15.77:0/2398897286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884cdb73a0 con 0x55884ced0550 2024-09-15T22:29:09.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.004+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc118450 tx=0x7f3fd8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6381b07c21064f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fc77fe640 1 -- 172.21.15.77:0/2398897286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8004250 con 0x55884ced0550 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fc77fe640 1 -- 172.21.15.77:0/2398897286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd8009e40 con 0x55884ced0550 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fc77fe640 1 -- 172.21.15.77:0/2398897286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80197a0 con 0x55884ced0550 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fd27fc640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 0x7f3fc80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fd27fc640 1 -- 172.21.15.77:0/1236547118 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f3fcc0efe70 con 0x7f3fdc069ab0 2024-09-15T22:29:09.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.005+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 0x7f3fc80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 -- 172.21.15.77:0/2398897286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 msgr2=0x55884ced2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc118450 tx=0x7f3fd8011810 comp rx=0 tx=0).stop 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 -- 172.21.15.77:0/2398897286 shutdown_connections 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced2940 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cec5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2398897286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cec5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 0x7f3fc80a5a40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4003f80 tx=0x7f3fd4009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 -- 172.21.15.77:0/2398897286 >> 172.21.15.77:0/2398897286 conn(0x55884ce8e310 msgr2=0x55884cec1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 -- 172.21.15.77:0/2398897286 shutdown_connections 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.006+0000 7f3fe584f740 1 -- 172.21.15.77:0/2398897286 wait complete. 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 -- start start 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 Processor -- start 2024-09-15T22:29:09.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cecfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cecd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55884ced3250 con 0x55884ced0550 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55884cde6060 con 0x55884cec6d20 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55884ced33d0 con 0x55884cec57b0 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe584f740 10 client.0 _ll_register_callbacks cb 0x55884cb398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.007+0000 7f3fe1a13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49786/0 (socket says 172.21.15.77:49786) 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 -- 172.21.15.77:0/2410220255 learned_addr learned my addr 172.21.15.77:0/2410220255 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1212640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cecfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cecd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 -- 172.21.15.77:0/2410220255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 msgr2=0x55884cecfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cecfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 -- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 msgr2=0x55884cecd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cecd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 -- 172.21.15.77:0/2410220255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884cb9c3c0 con 0x55884ced0550 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1212640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cecfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.008+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009fb0 tx=0x7f3fd4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e78bb41b6eeaf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd402c060 con 0x55884ced0550 2024-09-15T22:29:09.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f3fd4017030 con 0x55884ced0550 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55884ce38860 con 0x55884ced0550 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fd401f810 con 0x55884ced0550 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fe584f740 10 client.19506 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.009+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55884cedecd0 con 0x55884ced0550 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd4010430 con 0x55884ced0550 2024-09-15T22:29:09.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3fd4027410 con 0x55884ced0550 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 10 client.19506 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 10 client.19506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 20 client.19506 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cee04c0 0x55884cee2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 4 client.19506 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55884ce38860 con 0x55884cee04c0 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.010+0000 7f3fe1212640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cee04c0 0x55884cee2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.011+0000 7f3fc4ff9640 10 client.19506 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.011+0000 7f3fe1212640 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cee04c0 0x55884cee2940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5396395f08c98ecc server_cookie=23b7f0f1f37823e4 in_seq=0 out_seq=0 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.012+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/2410220255 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55884ce38860 con 0x55884cee04c0 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.012+0000 7f3fc4ff9640 10 client.19506 handle_command_reply: tid=0 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.012+0000 7f3fe584f740 1 client.19506 shutdown 2024-09-15T22:29:09.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.012+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 msgr2=0x55884cecd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.012+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009fb0 tx=0x7f3fd4004ab0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 shutdown_connections 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cee04c0 0x55884cee2940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884cecd660 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec6d20 0x55884cecd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 --2- 172.21.15.77:0/2410220255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cecfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 >> 172.21.15.77:0/2410220255 conn(0x55884ce8e310 msgr2=0x55884cec7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fc4ff9640 0 client.19506 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 shutdown_connections 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 1 -- 172.21.15.77:0/2410220255 wait complete. 2024-09-15T22:29:09.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.013+0000 7f3fe584f740 20 client.19506 trim_cache size 0 max 16384 2024-09-15T22:29:09.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.031+0000 7f3fe584f740 1 Processor -- start 2024-09-15T22:29:09.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 -- start start 2024-09-15T22:29:09.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cedc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884ced0550 0x55884cecd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1bdc60 con 0x55884cec6d20 2024-09-15T22:29:09.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1bf7a0 con 0x55884ced0550 2024-09-15T22:29:09.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe584f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1be330 con 0x55884cec57b0 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49788/0 (socket says 172.21.15.77:49788) 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 -- 172.21.15.77:0/3783427317 learned_addr learned my addr 172.21.15.77:0/3783427317 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe1212640 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cedc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe1a13640 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884ced0550 0x55884cecd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 -- 172.21.15.77:0/3783427317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 msgr2=0x55884cedc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cedc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 -- 172.21.15.77:0/3783427317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884ced0550 msgr2=0x55884cecd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.032+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884ced0550 0x55884cecd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.033+0000 7f3fe0a11640 1 -- 172.21.15.77:0/3783427317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884cec6940 con 0x55884cec6d20 2024-09-15T22:29:09.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.033+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x55884cec6250 tx=0x7f3fd8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81db8023dc078581 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.033+0000 7f3fc6ffd640 1 -- 172.21.15.77:0/3783427317 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd801f040 con 0x55884cec6d20 2024-09-15T22:29:09.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.033+0000 7f3fc6ffd640 1 -- 172.21.15.77:0/3783427317 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd803a070 con 0x55884cec6d20 2024-09-15T22:29:09.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.033+0000 7f3fc6ffd640 1 -- 172.21.15.77:0/3783427317 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd801a6a0 con 0x55884cec6d20 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.034+0000 7f3fe584f740 1 -- 172.21.15.77:0/3783427317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 msgr2=0x55884cecfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x55884cec6250 tx=0x7f3fd8009b30 comp rx=0 tx=0).stop 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 -- 172.21.15.77:0/3783427317 shutdown_connections 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884ced0550 0x55884cecd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884cec6d20 0x55884cecfe70 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 --2- 172.21.15.77:0/3783427317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec57b0 0x55884cedc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 -- 172.21.15.77:0/3783427317 >> 172.21.15.77:0/3783427317 conn(0x55884ce8e310 msgr2=0x55884cec84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 -- 172.21.15.77:0/3783427317 shutdown_connections 2024-09-15T22:29:09.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.035+0000 7f3fe584f740 1 -- 172.21.15.77:0/3783427317 wait complete. 2024-09-15T22:29:09.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.036+0000 7f3fe584f740 1 -- start start 2024-09-15T22:29:09.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.036+0000 7f3fe584f740 1 Processor -- start 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 0x55884ced8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe0a11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60918/0 (socket says 172.21.15.77:60918) 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe0a11640 1 -- 172.21.15.77:0/4113482339 learned_addr learned my addr 172.21.15.77:0/4113482339 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1bf0d0 con 0x55884ced0550 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1bea00 con 0x55884cec57b0 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1bbc60 con 0x55884cec6d20 2024-09-15T22:29:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.037+0000 7f3fe584f740 10 client.0 _ll_register_callbacks cb 0x7f3fd4003bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:09.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe1212640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 0x55884ced8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 -- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 msgr2=0x55884ced8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 0x55884ced8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 -- 172.21.15.77:0/4113482339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 msgr2=0x55884ced3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884ced4470 con 0x55884cec6d20 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe1212640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 0x55884ced8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe0a11640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8011850 tx=0x7f3fd8009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff64cbd86782dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80035f0 con 0x55884cec6d20 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd803a070 con 0x55884cec6d20 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.038+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f3fd8003790 con 0x55884cec6d20 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fd804e050 con 0x55884cec6d20 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 10 client.28252 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:09.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55884cf4a2f0 con 0x55884cec6d20 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd803a070 con 0x55884cec6d20 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3fd8018070 con 0x55884cec6d20 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 10 client.28252 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 10 client.28252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 20 client.28252 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.039+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cf4a7c0 0x55884cf4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.040+0000 7f3fe584f740 4 client.28252 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.040+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55884ce32d70 con 0x55884cf4a7c0 2024-09-15T22:29:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.040+0000 7f3fe1212640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cf4a7c0 0x55884cf4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.040+0000 7f3fe1212640 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cf4a7c0 0x55884cf4cbe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccc43e039cb665 server_cookie=d580b7e11550b587 in_seq=0 out_seq=0 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.040+0000 7f3fc4ff9640 10 client.28252 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fc4ff9640 1 -- 172.21.15.77:0/4113482339 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55884ce32d70 con 0x55884cf4a7c0 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fc4ff9640 10 client.28252 handle_command_reply: tid=0 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fe584f740 1 client.28252 shutdown 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 msgr2=0x55884ced3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8011850 tx=0x7f3fd8009530 comp rx=0 tx=0).stop 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 shutdown_connections 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55884cf4a7c0 0x55884cf4cbe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.041+0000 7f3fc4ff9640 0 client.28252 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:09.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.042+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55884ced0550 0x55884ced3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.042+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55884cec6d20 0x55884ced3680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.042+0000 7f3fe584f740 1 --2- 172.21.15.77:0/4113482339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55884cec57b0 0x55884ced8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.042+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 >> 172.21.15.77:0/4113482339 conn(0x55884ce8e310 msgr2=0x55884cecb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.042+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 shutdown_connections 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.044+0000 7f3fe584f740 1 -- 172.21.15.77:0/4113482339 wait complete. 2024-09-15T22:29:09.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.045+0000 7f3fe584f740 20 client.28252 trim_cache size 0 max 16384 2024-09-15T22:29:09.048 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:09.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 msgr2=0x7f3fc80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 0x7f3fc80a5a40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4003f80 tx=0x7f3fd4009500 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 msgr2=0x7f3fdc116c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc116c00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005940 tx=0x7f3fcc0040c0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 shutdown_connections 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3fc80a3580 0x7f3fc80a5a40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fdc109e00 0x7f3fdc117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fdc109450 0x7f3fdc117140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 --2- 172.21.15.77:0/1236547118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fdc069ab0 0x7f3fdc116c00 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 >> 172.21.15.77:0/1236547118 conn(0x7f3fdc073660 msgr2=0x7f3fdc10c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 shutdown_connections 2024-09-15T22:29:09.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:09.046+0000 7f3fe2214640 1 -- 172.21.15.77:0/1236547118 wait complete. 2024-09-15T22:29:09.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 66212 17.13 MB/sec execute 70 sec latency 431.234 ms 2024-09-15T22:29:09.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:29:10.384 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:10.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 66223 16.93 MB/sec execute 71 sec latency 515.508 ms 2024-09-15T22:29:10.759 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:10 smithi077 ceph-mon[30980]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-09-15T22:29:10.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:10 smithi196 ceph-mon[35109]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-09-15T22:29:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:10 smithi117 ceph-mon[35021]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 -- 172.21.15.77:0/1075258875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 msgr2=0x7f63ac09c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/1075258875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac09c950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009860 tx=0x7f63a402a340 comp rx=0 tx=0).stop 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 -- 172.21.15.77:0/1075258875 shutdown_connections 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/1075258875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac09ce90 0x7f63ac0a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/1075258875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac09c950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/1075258875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac0a1d50 0x7f63ac0a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.157+0000 7f63ba8b3640 1 -- 172.21.15.77:0/1075258875 >> 172.21.15.77:0/1075258875 conn(0x7f63ac009fc0 msgr2=0x7f63ac00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.158+0000 7f63ba8b3640 1 -- 172.21.15.77:0/1075258875 shutdown_connections 2024-09-15T22:29:11.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.159+0000 7f63ba8b3640 1 -- 172.21.15.77:0/1075258875 wait complete. 2024-09-15T22:29:11.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.159+0000 7f63ba8b3640 1 Processor -- start 2024-09-15T22:29:11.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 -- start start 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac132590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 0x7f63ac132ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0ac0b0 con 0x7f63ac09ce90 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0abf30 con 0x7f63ac0a1d50 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.160+0000 7f63ba8b3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac0ac230 con 0x7f63ac09c4d0 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.161+0000 7f63ba0b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.161+0000 7f63ba0b2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37378/0 (socket says 172.21.15.77:37378) 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.161+0000 7f63ba0b2640 1 -- 172.21.15.77:0/2022871517 learned_addr learned my addr 172.21.15.77:0/2022871517 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:11.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.161+0000 7f63b98b1640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac132590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 -- 172.21.15.77:0/2022871517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 msgr2=0x7f63ac132590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac132590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 -- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 msgr2=0x7f63ac132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63b90b0640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 0x7f63ac132ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 0x7f63ac132ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 -- 172.21.15.77:0/2022871517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac137560 con 0x7f63ac0a1d50 2024-09-15T22:29:11.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63b90b0640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 0x7f63ac132ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba0b2640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63b4051220 tx=0x7f63b406db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63aaffd640 1 -- 172.21.15.77:0/2022871517 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b406b800 con 0x7f63ac0a1d50 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.162+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac137850 con 0x7f63ac0a1d50 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.163+0000 7f63aaffd640 1 -- 172.21.15.77:0/2022871517 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b406be80 con 0x7f63ac0a1d50 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.163+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac150120 con 0x7f63ac0a1d50 2024-09-15T22:29:11.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.163+0000 7f63aaffd640 1 -- 172.21.15.77:0/2022871517 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b406f410 con 0x7f63ac0a1d50 2024-09-15T22:29:11.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.166+0000 7f63bdeee740 1 Processor -- start 2024-09-15T22:29:11.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.167+0000 7f63aaffd640 1 -- 172.21.15.77:0/2022871517 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f63b406f5b0 con 0x7f63ac0a1d50 2024-09-15T22:29:11.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.169+0000 7f63bdeee740 1 -- start start 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63aaffd640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 0x7f63a00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf53910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea0beac2f0 con 0x55ea0bf47cf0 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63b98b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea0be53cf0 con 0x55ea0bf51520 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63bdeee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea0bea82b0 con 0x55ea0bf46780 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63b98b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60954/0 (socket says 172.21.15.77:60954) 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63b98b1640 1 -- 172.21.15.77:0/603972941 learned_addr learned my addr 172.21.15.77:0/603972941 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.170+0000 7f63aaffd640 1 -- 172.21.15.77:0/2022871517 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f63b41344b0 con 0x7f63ac0a1d50 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b90b0640 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf53910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63ba0b2640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 0x7f63a00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 -- 172.21.15.77:0/603972941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 msgr2=0x55ea0bf53910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf53910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 -- 172.21.15.77:0/603972941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 msgr2=0x55ea0bf46240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 -- 172.21.15.77:0/603972941 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0be383a0 con 0x55ea0bf46780 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.171+0000 7f63b98b1640 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009c00 tx=0x7f63b00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5a092c70ba321c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.172+0000 7f639ffff640 1 -- 172.21.15.77:0/603972941 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0012450 con 0x55ea0bf46780 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.172+0000 7f639ffff640 1 -- 172.21.15.77:0/603972941 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b0012ad0 con 0x55ea0bf46780 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.172+0000 7f63ba0b2640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 0x7f63a00a59c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f63b40fcce0 tx=0x7f63b4067fe0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.173+0000 7f639ffff640 1 -- 172.21.15.77:0/603972941 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000fbf0 con 0x55ea0bf46780 2024-09-15T22:29:11.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.173+0000 7f63bdeee740 1 -- 172.21.15.77:0/603972941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 msgr2=0x55ea0bf46bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.173+0000 7f63bdeee740 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009c00 tx=0x7f63b00117b0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.174+0000 7f63bdeee740 1 -- 172.21.15.77:0/603972941 shutdown_connections 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.174+0000 7f63bdeee740 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.174+0000 7f63bdeee740 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf46bc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.174+0000 7f63bdeee740 1 --2- 172.21.15.77:0/603972941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf46240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.174+0000 7f63bdeee740 1 -- 172.21.15.77:0/603972941 >> 172.21.15.77:0/603972941 conn(0x55ea0bf0f310 msgr2=0x55ea0bf42e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.175+0000 7f63bdeee740 1 -- 172.21.15.77:0/603972941 shutdown_connections 2024-09-15T22:29:11.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.175+0000 7f63bdeee740 1 -- 172.21.15.77:0/603972941 wait complete. 2024-09-15T22:29:11.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.175+0000 7f63bdeee740 1 -- start start 2024-09-15T22:29:11.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 Processor -- start 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf4e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea0bf54270 con 0x55ea0bf46780 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea0be67060 con 0x55ea0bf51520 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea0bf543f0 con 0x55ea0bf47cf0 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63bdeee740 10 client.0 _ll_register_callbacks cb 0x55ea0bbba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63b90b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63b90b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49832/0 (socket says 172.21.15.77:49832) 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63b90b0640 1 -- 172.21.15.77:0/3121083891 learned_addr learned my addr 172.21.15.77:0/3121083891 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:11.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.176+0000 7f63b90b0640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf4e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 -- 172.21.15.77:0/3121083891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf47cf0 msgr2=0x55ea0bf4e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf4e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 -- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 msgr2=0x55ea0bf4e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf4e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 -- 172.21.15.77:0/3121083891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0bc1d3c0 con 0x55ea0bf46780 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.177+0000 7f63b90b0640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005b60 tx=0x7f63a402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4418d31066e86d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f639d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4059060 con 0x55ea0bf46780 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f63a4004460 con 0x55ea0bf46780 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b0012ad0 con 0x55ea0bf46780 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63a4037070 con 0x55ea0bf46780 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f63bdeee740 10 client.19515 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:11.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.178+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0bf5fce0 con 0x55ea0bf46780 2024-09-15T22:29:11.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b0012ad0 con 0x55ea0bf46780 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f63a4003890 con 0x55ea0bf46780 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 10 client.19515 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 10 client.19515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 20 client.19515 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bf611a0 0x55ea0bf635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 4 client.19515 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0beb9860 con 0x55ea0bf611a0 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.179+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bf611a0 0x55ea0bf635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.181+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bf611a0 0x55ea0bf635f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18975ac48f34af2 server_cookie=6516a25fcda37e0a in_seq=0 out_seq=0 2024-09-15T22:29:11.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.181+0000 7f639d7fa640 10 client.19515 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:11.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.183+0000 7f639d7fa640 1 -- 172.21.15.77:0/3121083891 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0beb9860 con 0x55ea0bf611a0 2024-09-15T22:29:11.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.183+0000 7f639d7fa640 10 client.19515 handle_command_reply: tid=0 2024-09-15T22:29:11.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.184+0000 7f63bdeee740 1 client.19515 shutdown 2024-09-15T22:29:11.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.184+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 msgr2=0x55ea0bf50db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.184+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005b60 tx=0x7f63a402c040 comp rx=0 tx=0).stop 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 shutdown_connections 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bf611a0 0x55ea0bf635f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf4e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf4e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3121083891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf50db0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.186+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 >> 172.21.15.77:0/3121083891 conn(0x55ea0bf0f310 msgr2=0x55ea0bf48ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.188+0000 7f639d7fa640 0 client.19515 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:11.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.190+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 shutdown_connections 2024-09-15T22:29:11.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.190+0000 7f63bdeee740 1 -- 172.21.15.77:0/3121083891 wait complete. 2024-09-15T22:29:11.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.190+0000 7f63bdeee740 20 client.19515 trim_cache size 0 max 16384 2024-09-15T22:29:11.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 Processor -- start 2024-09-15T22:29:11.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 -- start start 2024-09-15T22:29:11.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf5d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf3db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf52910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6398031fc0 con 0x55ea0bf47cf0 2024-09-15T22:29:11.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6398033b00 con 0x55ea0bf51520 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63bdeee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6398032690 con 0x55ea0bf46780 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63b90b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf5d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63b90b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf5d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60980/0 (socket says 172.21.15.77:60980) 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63b90b0640 1 -- 172.21.15.77:0/3174643246 learned_addr learned my addr 172.21.15.77:0/3174643246 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.216+0000 7f63b90b0640 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf52910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf3db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 -- 172.21.15.77:0/3174643246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 msgr2=0x55ea0bf5d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf5d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 -- 172.21.15.77:0/3174643246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 msgr2=0x55ea0bf52910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf52910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 -- 172.21.15.77:0/3174643246 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0bf4c600 con 0x55ea0bf47cf0 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.217+0000 7f63b98b1640 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf3db90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f63b0004650 tx=0x7f63b000f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa0da739448f40a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.218+0000 7f639f7fe640 1 -- 172.21.15.77:0/3174643246 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0030070 con 0x55ea0bf47cf0 2024-09-15T22:29:11.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.218+0000 7f639f7fe640 1 -- 172.21.15.77:0/3174643246 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b0009e30 con 0x55ea0bf47cf0 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.219+0000 7f63bdeee740 1 -- 172.21.15.77:0/3174643246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 msgr2=0x55ea0bf3db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.219+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf3db90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f63b0004650 tx=0x7f63b000f6d0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.219+0000 7f63bdeee740 1 -- 172.21.15.77:0/3174643246 shutdown_connections 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf51520 0x55ea0bf52910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf3db90 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 --2- 172.21.15.77:0/3174643246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf46780 0x55ea0bf5d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 -- 172.21.15.77:0/3174643246 >> 172.21.15.77:0/3174643246 conn(0x55ea0bf0f310 msgr2=0x55ea0bf4c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 -- 172.21.15.77:0/3174643246 shutdown_connections 2024-09-15T22:29:11.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.220+0000 7f63bdeee740 1 -- 172.21.15.77:0/3174643246 wait complete. 2024-09-15T22:29:11.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 -- start start 2024-09-15T22:29:11.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 Processor -- start 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf59ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf51520 0x55ea0bf68d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6398033430 con 0x55ea0bf46780 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6398032d60 con 0x55ea0bf47cf0 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f639802ffc0 con 0x55ea0bf51520 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.221+0000 7f63bdeee740 10 client.0 _ll_register_callbacks cb 0x7f63b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:11.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37426/0 (socket says 172.21.15.77:37426) 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 -- 172.21.15.77:0/2511786263 learned_addr learned my addr 172.21.15.77:0/2511786263 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 -- 172.21.15.77:0/2511786263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf51520 msgr2=0x55ea0bf68d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf51520 0x55ea0bf68d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 -- 172.21.15.77:0/2511786263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 msgr2=0x55ea0bf59ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf59ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 -- 172.21.15.77:0/2511786263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0bf69590 con 0x55ea0bf47cf0 2024-09-15T22:29:11.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.222+0000 7f63b98b1640 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f63b00099c0 tx=0x7f63b0009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d2a45562feae0f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.223+0000 7f639d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:29:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.223+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0030070 con 0x55ea0bf47cf0 2024-09-15T22:29:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.223+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9438+0+0 (secure 0 0 0) 0x7f63b0005240 con 0x55ea0bf47cf0 2024-09-15T22:29:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.223+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b0009e30 con 0x55ea0bf47cf0 2024-09-15T22:29:11.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.224+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63b003f050 con 0x55ea0bf47cf0 2024-09-15T22:29:11.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.224+0000 7f63bdeee740 10 client.28355 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:11.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.224+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0bfcbf30 con 0x55ea0bf47cf0 2024-09-15T22:29:11.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b0009e30 con 0x55ea0bf47cf0 2024-09-15T22:29:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f63b0012450 con 0x55ea0bf47cf0 2024-09-15T22:29:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 10 client.28355 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 10 client.28355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 20 client.28355 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:11.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bfcc300 0x55ea0bfce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:11.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 4 client.28355 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:11.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.226+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0beb3d70 con 0x55ea0bfcc300 2024-09-15T22:29:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.229+0000 7f63b90b0640 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bfcc300 0x55ea0bfce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.230+0000 7f63b90b0640 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bfcc300 0x55ea0bfce740 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83141d6f2f0d746 server_cookie=562fcd9ce00a2563 in_seq=0 out_seq=0 2024-09-15T22:29:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.230+0000 7f639d7fa640 10 client.28355 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.232+0000 7f639d7fa640 1 -- 172.21.15.77:0/2511786263 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0beb3d70 con 0x55ea0bfcc300 2024-09-15T22:29:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.232+0000 7f639d7fa640 10 client.28355 handle_command_reply: tid=0 2024-09-15T22:29:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.232+0000 7f63bdeee740 1 client.28355 shutdown 2024-09-15T22:29:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 msgr2=0x55ea0bf68820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f63b00099c0 tx=0x7f63b0009630 comp rx=0 tx=0).stop 2024-09-15T22:29:11.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 shutdown_connections 2024-09-15T22:29:11.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0bfcc300 0x55ea0bfce740 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0bf51520 0x55ea0bf68d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0bf47cf0 0x55ea0bf68820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 --2- 172.21.15.77:0/2511786263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0bf46780 0x55ea0bf59ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.233+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 >> 172.21.15.77:0/2511786263 conn(0x55ea0bf0f310 msgr2=0x55ea0bf416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.234+0000 7f639d7fa640 0 client.28355 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.234+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 shutdown_connections 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.234+0000 7f63bdeee740 1 -- 172.21.15.77:0/2511786263 wait complete. 2024-09-15T22:29:11.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.234+0000 7f63bdeee740 20 client.28355 trim_cache size 0 max 16384 2024-09-15T22:29:11.238 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 msgr2=0x7f63a00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 0x7f63a00a59c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f63b40fcce0 tx=0x7f63b4067fe0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 msgr2=0x7f63ac136e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63b4051220 tx=0x7f63b406db30 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 shutdown_connections 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63a00a3500 0x7f63a00a59c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63ac0a1d50 0x7f63ac136e60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63ac09ce90 0x7f63ac132ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 --2- 172.21.15.77:0/2022871517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63ac09c4d0 0x7f63ac132590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 >> 172.21.15.77:0/2022871517 conn(0x7f63ac009fc0 msgr2=0x7f63ac09a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 shutdown_connections 2024-09-15T22:29:11.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:11.237+0000 7f63ba8b3640 1 -- 172.21.15.77:0/2022871517 wait complete. 2024-09-15T22:29:11.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 66356 16.85 MB/sec execute 72 sec latency 299.623 ms 2024-09-15T22:29:12.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 66863 17.04 MB/sec execute 73 sec latency 304.609 ms 2024-09-15T22:29:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:12 smithi077 ceph-mon[30980]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-09-15T22:29:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:12 smithi077 ceph-mon[30980]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:29:12.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:12 smithi196 ceph-mon[35109]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-09-15T22:29:12.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:12 smithi196 ceph-mon[35109]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:29:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:12 smithi117 ceph-mon[35021]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-09-15T22:29:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:12 smithi117 ceph-mon[35021]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:29:13.156 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:29:13.157 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:29:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:13.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 67471 17.22 MB/sec execute 74 sec latency 72.334 ms 2024-09-15T22:29:14.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 -- 172.21.15.77:0/291772147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 msgr2=0x7f6d641021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/291772147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d641021b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009900 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 -- 172.21.15.77:0/291772147 shutdown_connections 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/291772147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64106f00 0x7f6d6410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/291772147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64102780 0x7f6d641067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/291772147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d641021b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 -- 172.21.15.77:0/291772147 >> 172.21.15.77:0/291772147 conn(0x7f6d640fd8b0 msgr2=0x7f6d640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 -- 172.21.15.77:0/291772147 shutdown_connections 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.057+0000 7f6d68b1f640 1 -- 172.21.15.77:0/291772147 wait complete. 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.058+0000 7f6d68b1f640 1 Processor -- start 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.058+0000 7f6d68b1f640 1 -- start start 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 0x7f6d641989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 0x7f6d6419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64111f60 con 0x7f6d64101dd0 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49864/0 (socket says 172.21.15.77:49864) 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 -- 172.21.15.77:0/1685846075 learned_addr learned my addr 172.21.15.77:0/1685846075 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64111de0 con 0x7f6d64102780 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641120e0 con 0x7f6d64106f00 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 0x7f6d6419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 0x7f6d641989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 -- 172.21.15.77:0/1685846075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 msgr2=0x7f6d6419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 0x7f6d6419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 -- 172.21.15.77:0/1685846075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 msgr2=0x7f6d641989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 0x7f6d641989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d6259c640 1 -- 172.21.15.77:0/1685846075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6419d440 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 0x7f6d641989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.059+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 0x7f6d6419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d6259c640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d54004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d537fe640 1 -- 172.21.15.77:0/1685846075 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5403f030 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d537fe640 1 -- 172.21.15.77:0/1685846075 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d54004440 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6419d6d0 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d537fe640 1 -- 172.21.15.77:0/1685846075 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54035070 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.060+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6419db40 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.063+0000 7f6d537fe640 1 -- 172.21.15.77:0/1685846075 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6d54042030 con 0x7f6d64101dd0 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.063+0000 7f6d6aef8740 1 Processor -- start 2024-09-15T22:29:14.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.063+0000 7f6d6aef8740 1 -- start start 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d537fe640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 0x7f6d4c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c362f29280 con 0x55c362fced40 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c362ed0b10 con 0x55c362fc3b60 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c362f25240 con 0x55c362fc4480 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 0x7f6d4c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37450/0 (socket says 172.21.15.77:37450) 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 -- 172.21.15.77:0/114027162 learned_addr learned my addr 172.21.15.77:0/114027162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:14.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d537fe640 1 -- 172.21.15.77:0/1685846075 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f6d540f0440 con 0x7f6d64101dd0 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 -- 172.21.15.77:0/114027162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 msgr2=0x55c362fce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 -- 172.21.15.77:0/114027162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 msgr2=0x55c362fd1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fd1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.064+0000 7f6d6259c640 1 -- 172.21.15.77:0/114027162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c362edd1f0 con 0x55c362fc3b60 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.065+0000 7f6d6259c640 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d540bc310 tx=0x7f6d5402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab9d2cb509674fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.065+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 0x7f6d4c0a58f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55c362fc1470 tx=0x7f6d5c009f60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.065+0000 7f6d50ff9640 1 -- 172.21.15.77:0/114027162 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54049070 con 0x55c362fc3b60 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.065+0000 7f6d50ff9640 1 -- 172.21.15.77:0/114027162 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d540c0680 con 0x55c362fc3b60 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.065+0000 7f6d50ff9640 1 -- 172.21.15.77:0/114027162 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54003df0 con 0x55c362fc3b60 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- 172.21.15.77:0/114027162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 msgr2=0x55c362fc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d540bc310 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- 172.21.15.77:0/114027162 shutdown_connections 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fd1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/114027162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fc3f40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- 172.21.15.77:0/114027162 >> 172.21.15.77:0/114027162 conn(0x55c362fc00f0 msgr2=0x55c362fc04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- 172.21.15.77:0/114027162 shutdown_connections 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- 172.21.15.77:0/114027162 wait complete. 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 -- start start 2024-09-15T22:29:14.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.066+0000 7f6d6aef8740 1 Processor -- start 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fcbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fcc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c362fd16d0 con 0x55c362fced40 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c362ee3f10 con 0x55c362fc3b60 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c362fd1850 con 0x55c362fc4480 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 10 client.0 _ll_register_callbacks cb 0x55c362eaca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fcbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fcc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49892/0 (socket says 172.21.15.77:49892) 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fcc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d62d9d640 1 -- 172.21.15.77:0/4087779661 learned_addr learned my addr 172.21.15.77:0/4087779661 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 -- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 msgr2=0x55c362fcbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fcbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 -- 172.21.15.77:0/4087779661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 msgr2=0x55c362fcc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fcc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 -- 172.21.15.77:0/4087779661 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c362c9a6a0 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fcc140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d640ffe00 tx=0x7f6d5800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca78161ffa0c256c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58012110 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f6d58004050 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.067+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d540c0680 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6d580122b0 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 10 client.28279 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c362fdd280 con 0x55c362fc4480 2024-09-15T22:29:14.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d540c0680 con 0x55c362fc4480 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6d580104c0 con 0x55c362fc4480 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 10 client.28279 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 10 client.28279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 20 client.28279 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c362fde790 0x55c362fe0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 4 client.28279 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c362f367f0 con 0x55c362fde790 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.068+0000 7f6d6259c640 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c362fde790 0x55c362fe0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.069+0000 7f6d49ffb640 10 client.28279 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.069+0000 7f6d6259c640 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c362fde790 0x55c362fe0b80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5e317030873bcc server_cookie=4a74a6b6b708e739 in_seq=0 out_seq=0 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.070+0000 7f6d49ffb640 1 -- 172.21.15.77:0/4087779661 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c362f367f0 con 0x55c362fde790 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.070+0000 7f6d49ffb640 10 client.28279 handle_command_reply: tid=0 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.070+0000 7f6d6aef8740 1 client.28279 shutdown 2024-09-15T22:29:14.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 msgr2=0x55c362fcc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fcc140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d640ffe00 tx=0x7f6d5800f790 comp rx=0 tx=0).stop 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 shutdown_connections 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c362fde790 0x55c362fe0b80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fced40 0x55c362fcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fc4480 0x55c362fcc140 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/4087779661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc3b60 0x55c362fcbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 >> 172.21.15.77:0/4087779661 conn(0x55c362fc00f0 msgr2=0x55c362fc5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d49ffb640 0 client.28279 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 shutdown_connections 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 1 -- 172.21.15.77:0/4087779661 wait complete. 2024-09-15T22:29:14.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.071+0000 7f6d6aef8740 20 client.28279 trim_cache size 0 max 16384 2024-09-15T22:29:14.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 Processor -- start 2024-09-15T22:29:14.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 -- start start 2024-09-15T22:29:14.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fcc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fe6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641af0a0 con 0x55c362fc3b60 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641b0be0 con 0x55c362fc4480 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641af770 con 0x55c362fced40 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fcc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fcc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49902/0 (socket says 172.21.15.77:49902) 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.093+0000 7f6d6259c640 1 -- 172.21.15.77:0/3318705630 learned_addr learned my addr 172.21.15.77:0/3318705630 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fe6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 -- 172.21.15.77:0/3318705630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 msgr2=0x55c362fe6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fe6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 -- 172.21.15.77:0/3318705630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 msgr2=0x55c362fcc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fcc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 -- 172.21.15.77:0/3318705630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c362fbff20 con 0x55c362fc4480 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.094+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc5cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d58010930 tx=0x7f6d58005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266a526182e45f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.095+0000 7f6d4bfff640 1 -- 172.21.15.77:0/3318705630 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802e070 con 0x55c362fc4480 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.095+0000 7f6d4bfff640 1 -- 172.21.15.77:0/3318705630 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d580055c0 con 0x55c362fc4480 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.095+0000 7f6d4bfff640 1 -- 172.21.15.77:0/3318705630 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58003850 con 0x55c362fc4480 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- 172.21.15.77:0/3318705630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 msgr2=0x55c362fc5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc5cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d58010930 tx=0x7f6d58005d30 comp rx=0 tx=0).stop 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- 172.21.15.77:0/3318705630 shutdown_connections 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fe6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc5cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/3318705630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fcc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- 172.21.15.77:0/3318705630 >> 172.21.15.77:0/3318705630 conn(0x55c362fc00f0 msgr2=0x55c362fc55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- 172.21.15.77:0/3318705630 shutdown_connections 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- 172.21.15.77:0/3318705630 wait complete. 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 -- start start 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.096+0000 7f6d6aef8740 1 Processor -- start 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fd7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641b0510 con 0x55c362fc3b60 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641afe40 con 0x55c362fc4480 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641ad0a0 con 0x55c362fced40 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6aef8740 10 client.0 _ll_register_callbacks cb 0x7f6d58009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fd7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d61d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d62d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:32824/0 (socket says 172.21.15.77:32824) 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d62d9d640 1 -- 172.21.15.77:0/2465834518 learned_addr learned my addr 172.21.15.77:0/2465834518 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 -- 172.21.15.77:0/2465834518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 msgr2=0x55c362fc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 -- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 msgr2=0x55c362fc8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 -- 172.21.15.77:0/2465834518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c362fc11a0 con 0x55c362fc3b60 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d62d9d640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d6259c640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fd7200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6d54000c00 tx=0x7f6d540bc500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8941b1344aad2d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.097+0000 7f6d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54049040 con 0x55c362fc3b60 2024-09-15T22:29:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f6d54003cd0 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d580055c0 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6d540e0210 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 10 client.19536 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c363049320 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d580055c0 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6d540ed9c0 con 0x55c362fc3b60 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 10 client.19536 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 10 client.19536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 20 client.19536 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:14.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.098+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c3630498b0 0x55c36304bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:14.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.099+0000 7f6d6aef8740 4 client.19536 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:29:14.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.099+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c362f30d00 con 0x55c3630498b0 2024-09-15T22:29:14.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.099+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c3630498b0 0x55c36304bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:14.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.099+0000 7f6d49ffb640 10 client.19536 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:14.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.099+0000 7f6d61d9b640 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c3630498b0 0x55c36304bca0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f08e9df8845f85 server_cookie=df02361b18572bb1 in_seq=0 out_seq=0 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d49ffb640 1 -- 172.21.15.77:0/2465834518 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c362f30d00 con 0x55c3630498b0 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d49ffb640 10 client.19536 handle_command_reply: tid=0 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 client.19536 shutdown 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 msgr2=0x55c362fd7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fd7200 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6d54000c00 tx=0x7f6d540bc500 comp rx=0 tx=0).stop 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 shutdown_connections 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c3630498b0 0x55c36304bca0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c362fced40 0x55c362fc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c362fc4480 0x55c362fc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 --2- 172.21.15.77:0/2465834518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c362fc3b60 0x55c362fd7200 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 >> 172.21.15.77:0/2465834518 conn(0x55c362fc00f0 msgr2=0x55c362fbf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d49ffb640 0 client.19536 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 shutdown_connections 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.101+0000 7f6d6aef8740 1 -- 172.21.15.77:0/2465834518 wait complete. 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d6aef8740 20 client.19536 trim_cache size 0 max 16384 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "9b3ebd49-c68a-404d-889a-41b9b86eacb5", 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:29:14.107 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 msgr2=0x7f6d4c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 0x7f6d4c0a58f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55c362fc1470 tx=0x7f6d5c009f60 comp rx=0 tx=0).stop 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 msgr2=0x7f6d64198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d54004040 comp rx=0 tx=0).stop 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 shutdown_connections 2024-09-15T22:29:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6d4c0a3430 0x7f6d4c0a58f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d64106f00 0x7f6d6419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d64102780 0x7f6d641989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 --2- 172.21.15.77:0/1685846075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d64101dd0 0x7f6d64198490 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 >> 172.21.15.77:0/1685846075 conn(0x7f6d640fd8b0 msgr2=0x7f6d640fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 shutdown_connections 2024-09-15T22:29:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:14.102+0000 7f6d68b1f640 1 -- 172.21.15.77:0/1685846075 wait complete. 2024-09-15T22:29:14.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 67979 17.41 MB/sec execute 75 sec latency 163.756 ms 2024-09-15T22:29:14.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:14 smithi077 ceph-mon[30980]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:14 smithi196 ceph-mon[35109]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:14 smithi117 ceph-mon[35021]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:14.966 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:29:15.266 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:15.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 68418 17.55 MB/sec execute 76 sec latency 103.583 ms 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:29:15.707 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:15 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.811+0000 7f0dba44c640 1 -- 172.21.15.77:0/2164081055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db4106f20 msgr2=0x7f0db410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.811+0000 7f0dba44c640 1 --2- 172.21.15.77:0/2164081055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db4106f20 0x7f0db410eaa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac02a580 comp rx=0 tx=0).stop 2024-09-15T22:29:15.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 -- 172.21.15.77:0/2164081055 shutdown_connections 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 --2- 172.21.15.77:0/2164081055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db4106f20 0x7f0db410eaa0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 --2- 172.21.15.77:0/2164081055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db41027a0 0x7f0db41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 --2- 172.21.15.77:0/2164081055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 -- 172.21.15.77:0/2164081055 >> 172.21.15.77:0/2164081055 conn(0x7f0db40fd8f0 msgr2=0x7f0db40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 -- 172.21.15.77:0/2164081055 shutdown_connections 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 -- 172.21.15.77:0/2164081055 wait complete. 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 Processor -- start 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.812+0000 7f0dba44c640 1 -- start start 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 0x7f0db4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4106f20 0x7f0db419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410fd70 con 0x7f0db41027a0 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410fbf0 con 0x7f0db4106f20 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0dba44c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410fef0 con 0x7f0db4101df0 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 0x7f0db4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:32850/0 (socket says 172.21.15.77:32850) 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 -- 172.21.15.77:0/3189791886 learned_addr learned my addr 172.21.15.77:0/3189791886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db89e9640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4106f20 0x7f0db419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 -- 172.21.15.77:0/3189791886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4106f20 msgr2=0x7f0db419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4106f20 0x7f0db419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 -- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 msgr2=0x7f0db4196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 0x7f0db4196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db3fff640 1 -- 172.21.15.77:0/3189791886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db419b170 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.813+0000 7f0db37fe640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 0x7f0db4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009fd0 tx=0x7f0da4007650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0db17fa640 1 -- 172.21.15.77:0/3189791886 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da401c760 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db419b460 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db41a2d40 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0db17fa640 1 -- 172.21.15.77:0/3189791886 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da4022070 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.814+0000 7f0db17fa640 1 -- 172.21.15.77:0/3189791886 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da401dc30 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 Processor -- start 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0db17fa640 1 -- 172.21.15.77:0/3189791886 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0da4027460 con 0x7f0db4101df0 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 -- start start 2024-09-15T22:29:15.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 0x56114f26c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 0x56114f277140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 0x56114f279a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56114f1d22e0 con 0x56114f277680 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56114f179ce0 con 0x56114f26cdc0 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.818+0000 7f0dbc825740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56114f1ce2a0 con 0x56114f26c4a0 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db89e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 0x56114f26c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db89e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 0x56114f26c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:32858/0 (socket says 172.21.15.77:32858) 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db89e9640 1 -- 172.21.15.77:0/3421576742 learned_addr learned my addr 172.21.15.77:0/3421576742 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 0x56114f279a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 0x56114f277140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 -- 172.21.15.77:0/3421576742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 msgr2=0x56114f26c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 0x56114f26c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 -- 172.21.15.77:0/3421576742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 msgr2=0x56114f277140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 0x56114f277140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db37fe640 1 -- 172.21.15.77:0/3421576742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114f15e440 con 0x56114f277680 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.819+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 0x56114f277140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0db37fe640 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 0x56114f279a70 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009f20 tx=0x7f0da80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0e8bd28dc180c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0db17fa640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 0x7f0d9c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0db89e9640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 0x7f0d9c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0db17fa640 1 -- 172.21.15.77:0/3189791886 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f0da40d82f0 con 0x7f0db4101df0 2024-09-15T22:29:15.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0d9a7fc640 1 -- 172.21.15.77:0/3421576742 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8004030 con 0x56114f277680 2024-09-15T22:29:15.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0d9a7fc640 1 -- 172.21.15.77:0/3421576742 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8005500 con 0x56114f277680 2024-09-15T22:29:15.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.820+0000 7f0d9a7fc640 1 -- 172.21.15.77:0/3421576742 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8004e20 con 0x56114f277680 2024-09-15T22:29:15.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.821+0000 7f0db89e9640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 0x7f0d9c0a5970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005e30 tx=0x7f0dac009360 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.822+0000 7f0dbc825740 1 -- 172.21.15.77:0/3421576742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 msgr2=0x56114f279a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.822+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 0x56114f279a70 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009f20 tx=0x7f0da80118d0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.823+0000 7f0dbc825740 1 -- 172.21.15.77:0/3421576742 shutdown_connections 2024-09-15T22:29:15.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.823+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f277680 0x56114f279a70 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.823+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26cdc0 0x56114f277140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.823+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3421576742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26c4a0 0x56114f26c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.823+0000 7f0dbc825740 1 -- 172.21.15.77:0/3421576742 >> 172.21.15.77:0/3421576742 conn(0x56114f268a30 msgr2=0x56114f268e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.824+0000 7f0dbc825740 1 -- 172.21.15.77:0/3421576742 shutdown_connections 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.825+0000 7f0dbc825740 1 -- 172.21.15.77:0/3421576742 wait complete. 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.825+0000 7f0dbc825740 1 -- start start 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.826+0000 7f0dbc825740 1 Processor -- start 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.826+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 0x56114f276f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.826+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f2741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.826+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 0x56114f274700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.826+0000 7f0dbc825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56114f27a250 con 0x56114f26c4a0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 0x56114f276f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0dbc825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56114f18d050 con 0x56114f277680 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0dbc825740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56114f27a3d0 con 0x56114f26cdc0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0dbc825740 10 client.0 _ll_register_callbacks cb 0x56114f1cd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 0x56114f276f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49938/0 (socket says 172.21.15.77:49938) 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0db3fff640 1 -- 172.21.15.77:0/1093282862 learned_addr learned my addr 172.21.15.77:0/1093282862 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0db37fe640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f2741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.827+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 0x56114f274700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 -- 172.21.15.77:0/1093282862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 msgr2=0x56114f2741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f2741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 -- 172.21.15.77:0/1093282862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 msgr2=0x56114f276f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 0x56114f276f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 -- 172.21.15.77:0/1093282862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114ef433c0 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 0x56114f274700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac02a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2862533c7d04b1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac057060 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.828+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da8005500 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.829+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f0dac0316d0 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.829+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0dac05e050 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.829+0000 7f0dbc825740 10 client.28382 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.829+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56114f285b60 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dac0318d0 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0dac051030 con 0x56114f277680 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 10 client.28382 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:15.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 10 client.28382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 20 client.28382 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f2874c0 0x56114f289940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 4 client.28382 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.830+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0da8005500 con 0x56114f2874c0 2024-09-15T22:29:15.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.831+0000 7f0db3fff640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f2874c0 0x56114f289940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.831+0000 7f0d937fe640 10 client.28382 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.831+0000 7f0db3fff640 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f2874c0 0x56114f289940 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51411211c94a410 server_cookie=4efae27236672290 in_seq=0 out_seq=0 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.832+0000 7f0d937fe640 1 -- 172.21.15.77:0/1093282862 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0da8005500 con 0x56114f2874c0 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.832+0000 7f0d937fe640 10 client.28382 handle_command_reply: tid=0 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.832+0000 7f0dbc825740 1 client.28382 shutdown 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 msgr2=0x56114f274700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 0x56114f274700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ca0 tx=0x7f0dac02a580 comp rx=0 tx=0).stop 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 shutdown_connections 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f2874c0 0x56114f289940 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f277680 0x56114f274700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f2741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1093282862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26c4a0 0x56114f276f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 >> 172.21.15.77:0/1093282862 conn(0x56114f268a30 msgr2=0x56114f26e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0d937fe640 0 client.28382 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:15.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 shutdown_connections 2024-09-15T22:29:15.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 1 -- 172.21.15.77:0/1093282862 wait complete. 2024-09-15T22:29:15.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.833+0000 7f0dbc825740 20 client.28382 trim_cache size 0 max 16384 2024-09-15T22:29:15.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:29:15.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:29:15.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:15 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 Processor -- start 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 -- start start 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f267a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 0x56114f28f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0010080 con 0x56114f26cdc0 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0011bc0 con 0x56114f26c4a0 2024-09-15T22:29:15.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0dbc825740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0010750 con 0x56114f268a30 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0db3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:32870/0 (socket says 172.21.15.77:32870) 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0db3fff640 1 -- 172.21.15.77:0/3512778228 learned_addr learned my addr 172.21.15.77:0/3512778228 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.860+0000 7f0db89e9640 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 0x56114f28f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 -- 172.21.15.77:0/3512778228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 msgr2=0x56114f267a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f267a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 -- 172.21.15.77:0/3512778228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 msgr2=0x56114f28f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 0x56114f28f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 -- 172.21.15.77:0/3512778228 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114f268860 con 0x56114f268a30 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db89e9640 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 0x56114f28f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:15.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0db3fff640 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0da4013b60 tx=0x7f0da40da6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d8b2a7e2110680 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.861+0000 7f0d99ffb640 1 -- 172.21.15.77:0/3512778228 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40a8530 con 0x56114f268a30 2024-09-15T22:29:15.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:29:15.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.865 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:29:15.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:15 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.862+0000 7f0d99ffb640 1 -- 172.21.15.77:0/3512778228 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da40acdf0 con 0x56114f268a30 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.862+0000 7f0d99ffb640 1 -- 172.21.15.77:0/3512778228 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da401d3f0 con 0x56114f268a30 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 -- 172.21.15.77:0/3512778228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 msgr2=0x56114f274180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0da4013b60 tx=0x7f0da40da6c0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 -- 172.21.15.77:0/3512778228 shutdown_connections 2024-09-15T22:29:15.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f26cdc0 0x56114f28f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f267a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 --2- 172.21.15.77:0/3512778228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f268a30 0x56114f274180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.863+0000 7f0dbc825740 1 -- 172.21.15.77:0/3512778228 >> 172.21.15.77:0/3512778228 conn(0x56114f274860 msgr2=0x56114f267350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.864+0000 7f0dbc825740 1 -- 172.21.15.77:0/3512778228 shutdown_connections 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.865+0000 7f0dbc825740 1 -- 172.21.15.77:0/3512778228 wait complete. 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.865+0000 7f0dbc825740 1 -- start start 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.865+0000 7f0dbc825740 1 Processor -- start 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f268a30 0x56114f27d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f27daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f27e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0da00114f0 con 0x56114f268a30 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0010e20 con 0x56114f26c4a0 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0da000e080 con 0x56114f26cdc0 2024-09-15T22:29:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0dbc825740 10 client.0 _ll_register_callbacks cb 0x7f0dac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0db37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f27daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0db37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f27daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:37500/0 (socket says 172.21.15.77:37500) 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0db37fe640 1 -- 172.21.15.77:0/1504041774 learned_addr learned my addr 172.21.15.77:0/1504041774 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f27e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.866+0000 7f0db3fff640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f268a30 0x56114f27d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 -- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 msgr2=0x56114f27daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f27daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 -- 172.21.15.77:0/1504041774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f268a30 msgr2=0x56114f27d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f268a30 0x56114f27d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 -- 172.21.15.77:0/1504041774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114f2f13d0 con 0x56114f26cdc0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0db89e9640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f27e030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dac004440 tx=0x7f0dac009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e9937c4a006c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.867+0000 7f0d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.868+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac057040 con 0x56114f26cdc0 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.868+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9526+0+0 (secure 0 0 0) 0x7f0dac0306d0 con 0x56114f26cdc0 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.868+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da40acdf0 con 0x56114f26cdc0 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.868+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0dac069050 con 0x56114f26cdc0 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.869+0000 7f0dbc825740 10 client.28303 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.869+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56114f2f2670 con 0x56114f26cdc0 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da40acdf0 con 0x56114f26cdc0 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0dac061020 con 0x56114f26cdc0 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0dbc825740 10 client.28303 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0dbc825740 10 client.28303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0dbc825740 20 client.28303 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.870+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f277680 0x56114f2f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.871+0000 7f0dbc825740 4 client.28303 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.871+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56114f1df850 con 0x56114f277680 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.871+0000 7f0db3fff640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f277680 0x56114f2f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.872+0000 7f0d937fe640 10 client.28303 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:15.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.872+0000 7f0db3fff640 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f277680 0x56114f2f4e00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306d28af5b604634 server_cookie=1f696381026269cf in_seq=0 out_seq=0 2024-09-15T22:29:15.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.873+0000 7f0d937fe640 1 -- 172.21.15.77:0/1504041774 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56114f1df850 con 0x56114f277680 2024-09-15T22:29:15.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.873+0000 7f0d937fe640 10 client.28303 handle_command_reply: tid=0 2024-09-15T22:29:15.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.873+0000 7f0dbc825740 1 client.28303 shutdown 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 msgr2=0x56114f27e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f27e030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0dac004440 tx=0x7f0dac009800 comp rx=0 tx=0).stop 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 shutdown_connections 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56114f277680 0x56114f2f4e00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56114f26cdc0 0x56114f27e030 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114f26c4a0 0x56114f27daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 --2- 172.21.15.77:0/1504041774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56114f268a30 0x56114f27d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 >> 172.21.15.77:0/1504041774 conn(0x56114f274860 msgr2=0x56114f276240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.874+0000 7f0d937fe640 0 client.28303 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:15.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.875+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 shutdown_connections 2024-09-15T22:29:15.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.876+0000 7f0dbc825740 1 -- 172.21.15.77:0/1504041774 wait complete. 2024-09-15T22:29:15.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.876+0000 7f0dbc825740 20 client.28303 trim_cache size 0 max 16384 2024-09-15T22:29:15.878 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "9b3ebd49-c68a-404d-889a-41b9b86eacb5": { 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "9b3ebd49-c68a-404d-889a-41b9b86eacb5", 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 msgr2=0x7f0d9c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 0x7f0d9c0a5970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005e30 tx=0x7f0dac009360 comp rx=0 tx=0).stop 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 msgr2=0x7f0db41961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009fd0 tx=0x7f0da4007650 comp rx=0 tx=0).stop 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 shutdown_connections 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d9c0a34b0 0x7f0d9c0a5970 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4106f20 0x7f0db419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0db41027a0 0x7f0db4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 --2- 172.21.15.77:0/3189791886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4101df0 0x7f0db41961c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.878+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 >> 172.21.15.77:0/3189791886 conn(0x7f0db40fd8f0 msgr2=0x7f0db40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.879+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 shutdown_connections 2024-09-15T22:29:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:15.879+0000 7f0dba44c640 1 -- 172.21.15.77:0/3189791886 wait complete. 2024-09-15T22:29:16.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 68860 17.59 MB/sec execute 77 sec latency 74.685 ms 2024-09-15T22:29:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:16 smithi077 ceph-mon[30980]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:16.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:16 smithi196 ceph-mon[35109]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:16.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:16 smithi117 ceph-mon[35021]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-09-15T22:29:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:29:17.502 INFO:tasks.cephfs.filesystem:scrub status for tag:9b3ebd49-c68a-404d-889a-41b9b86eacb5 - {'path': '/', 'tag': '9b3ebd49-c68a-404d-889a-41b9b86eacb5', 'options': 'recursive,force'} 2024-09-15T22:29:17.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 69275 17.58 MB/sec execute 78 sec latency 119.919 ms 2024-09-15T22:29:18.598 INFO:tasks.workunit.client.0.smithi077.stdout: 10 69937 17.75 MB/sec execute 79 sec latency 64.793 ms 2024-09-15T22:29:18.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:18 smithi077 ceph-mon[30980]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:18.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:18 smithi196 ceph-mon[35109]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:18 smithi117 ceph-mon[35021]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:19.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 70661 18.11 MB/sec execute 80 sec latency 89.643 ms 2024-09-15T22:29:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:19 smithi077 ceph-mon[30980]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:19 smithi077 ceph-mon[30980]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:29:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:19 smithi196 ceph-mon[35109]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:20.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:19 smithi196 ceph-mon[35109]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:29:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:19 smithi117 ceph-mon[35021]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 451 op/s 2024-09-15T22:29:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:19 smithi117 ceph-mon[35021]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:29:20.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 71282 18.28 MB/sec execute 81 sec latency 107.958 ms 2024-09-15T22:29:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:29:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:29:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:29:21.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:20 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:29:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:29:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:29:21.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:29:21.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:20 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:29:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:29:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:29:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:29:21.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:20 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:29:21.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 71857 18.52 MB/sec execute 82 sec latency 132.885 ms 2024-09-15T22:29:22.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:21 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:29:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:21 smithi077 ceph-mon[30980]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2024-09-15T22:29:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:21 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:29:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:21 smithi196 ceph-mon[35109]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2024-09-15T22:29:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:21 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:29:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:21 smithi117 ceph-mon[35021]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 497 op/s 2024-09-15T22:29:22.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 72418 18.67 MB/sec execute 83 sec latency 205.854 ms 2024-09-15T22:29:23.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 72699 18.64 MB/sec execute 84 sec latency 270.114 ms 2024-09-15T22:29:24.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:23 smithi077 ceph-mon[30980]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:24.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:23 smithi196 ceph-mon[35109]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:23 smithi117 ceph-mon[35021]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:24.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 72976 18.53 MB/sec execute 85 sec latency 207.281 ms 2024-09-15T22:29:25.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 73447 18.57 MB/sec execute 86 sec latency 101.264 ms 2024-09-15T22:29:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:25 smithi077 ceph-mon[30980]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:25 smithi196 ceph-mon[35109]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:25 smithi117 ceph-mon[35021]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2024-09-15T22:29:26.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 74207 18.74 MB/sec execute 87 sec latency 56.157 ms 2024-09-15T22:29:27.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 74745 18.91 MB/sec execute 88 sec latency 233.743 ms 2024-09-15T22:29:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:27 smithi077 ceph-mon[30980]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:27 smithi196 ceph-mon[35109]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:27 smithi117 ceph-mon[35021]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:28.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 75209 19.03 MB/sec execute 89 sec latency 239.209 ms 2024-09-15T22:29:29.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 75618 18.99 MB/sec execute 90 sec latency 244.709 ms 2024-09-15T22:29:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:29 smithi077 ceph-mon[30980]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:29 smithi196 ceph-mon[35109]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:29 smithi117 ceph-mon[35021]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.8 MiB/s wr, 470 op/s 2024-09-15T22:29:30.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 76148 19.15 MB/sec execute 91 sec latency 113.192 ms 2024-09-15T22:29:31.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 76698 19.30 MB/sec execute 92 sec latency 127.183 ms 2024-09-15T22:29:31.919 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:29:31.919 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.95 2024-09-15T22:29:31.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:29:32.225 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:31 smithi077 ceph-mon[30980]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-09-15T22:29:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:31 smithi196 ceph-mon[35109]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-09-15T22:29:32.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:31 smithi117 ceph-mon[35021]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-09-15T22:29:32.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 77211 19.43 MB/sec execute 93 sec latency 59.957 ms 2024-09-15T22:29:32.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 -- 172.21.15.77:0/2882729439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 msgr2=0x7f48d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:32.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 --2- 172.21.15.77:0/2882729439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d410ea90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009ca0 tx=0x7f48cc02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 -- 172.21.15.77:0/2882729439 shutdown_connections 2024-09-15T22:29:32.812 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 --2- 172.21.15.77:0/2882729439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d410ea90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 --2- 172.21.15.77:0/2882729439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4102790 0x7f48d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 --2- 172.21.15.77:0/2882729439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4101de0 0x7f48d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 -- 172.21.15.77:0/2882729439 >> 172.21.15.77:0/2882729439 conn(0x7f48d40fd8e0 msgr2=0x7f48d40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 -- 172.21.15.77:0/2882729439 shutdown_connections 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.778+0000 7f48d9470640 1 -- 172.21.15.77:0/2882729439 wait complete. 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 Processor -- start 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 -- start start 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4101de0 0x7f48d419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4102790 0x7f48d419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111f70 con 0x7f48d4106f10 2024-09-15T22:29:32.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111df0 con 0x7f48d4101de0 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d9470640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41120f0 con 0x7f48d4102790 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45668/0 (socket says 172.21.15.77:45668) 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d37fe640 1 -- 172.21.15.77:0/574377003 learned_addr learned my addr 172.21.15.77:0/574377003 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48d2ffd640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4101de0 0x7f48d419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.779+0000 7f48ca7fc640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4102790 0x7f48d419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 -- 172.21.15.77:0/574377003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4102790 msgr2=0x7f48d419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4102790 0x7f48d419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 -- 172.21.15.77:0/574377003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4101de0 msgr2=0x7f48d419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:32.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4101de0 0x7f48d419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 -- 172.21.15.77:0/574377003 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d419f5c0 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d37fe640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f48cc031a20 tx=0x7f48cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc003c30 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc003dd0 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d419f850 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc02fc40 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.780+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a7190 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.781+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d41021c0 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.783+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f48cc004b40 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.784+0000 7f48d0ff9640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 0x7f48c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.784+0000 7f48d2ffd640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 0x7f48c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.784+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f48cc0f12f0 con 0x7f48d4106f10 2024-09-15T22:29:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.784+0000 7f48d2ffd640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 0x7f48c00a5870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005fd0 tx=0x7f48c400b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:32.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:32.787+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48cc0bdae0 con 0x7f48d4106f10 2024-09-15T22:29:33.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.045+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48d41a73c0 con 0x7f48d4106f10 2024-09-15T22:29:33.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.046+0000 7f48d0ff9640 1 -- 172.21.15.77:0/574377003 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f48cc0bd3e0 con 0x7f48d4106f10 2024-09-15T22:29:33.048 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:29:33.048 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:29:33.048 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:29:33.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 msgr2=0x7f48c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 0x7f48c00a5870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f48c4005fd0 tx=0x7f48c400b390 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 msgr2=0x7f48d419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f48cc031a20 tx=0x7f48cc031a50 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 shutdown_connections 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f48c00a33b0 0x7f48c00a5870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48d4106f10 0x7f48d419eee0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48d4102790 0x7f48d419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 --2- 172.21.15.77:0/574377003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48d4101de0 0x7f48d419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 >> 172.21.15.77:0/574377003 conn(0x7f48d40fd8e0 msgr2=0x7f48d40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 shutdown_connections 2024-09-15T22:29:33.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:33.049+0000 7f48d9470640 1 -- 172.21.15.77:0/574377003 wait complete. 2024-09-15T22:29:33.599 INFO:tasks.workunit.client.0.smithi077.stdout: 10 77699 19.49 MB/sec execute 94 sec latency 114.251 ms 2024-09-15T22:29:34.137 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0' 2024-09-15T22:29:34.137 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0 2024-09-15T22:29:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:33 smithi077 ceph-mon[30980]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:34.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:33 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/574377003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:33 smithi196 ceph-mon[35109]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:33 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/574377003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:33 smithi117 ceph-mon[35021]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:33 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/574377003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:34.517 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:34.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78095 19.50 MB/sec execute 95 sec latency 61.372 ms 2024-09-15T22:29:35.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78477 19.45 MB/sec execute 96 sec latency 164.075 ms 2024-09-15T22:29:36.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.136+0000 7fb09b59e640 1 -- 172.21.15.77:0/1291341022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c100c60 msgr2=0x7fb09c104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.136+0000 7fb09b59e640 1 --2- 172.21.15.77:0/1291341022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c100c60 0x7fb09c104cb0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb09000aa10 tx=0x7fb09002b740 comp rx=0 tx=0).stop 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 -- 172.21.15.77:0/1291341022 shutdown_connections 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 --2- 172.21.15.77:0/1291341022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 --2- 172.21.15.77:0/1291341022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c100c60 0x7fb09c104cb0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 --2- 172.21.15.77:0/1291341022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c1002b0 0x7fb09c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 -- 172.21.15.77:0/1291341022 >> 172.21.15.77:0/1291341022 conn(0x7fb09c077070 msgr2=0x7fb09c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 -- 172.21.15.77:0/1291341022 shutdown_connections 2024-09-15T22:29:36.139 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.137+0000 7fb09b59e640 1 -- 172.21.15.77:0/1291341022 wait complete. 2024-09-15T22:29:36.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 Processor -- start 2024-09-15T22:29:36.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 -- start start 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 0x7fb09c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 0x7fb09c196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10fcd0 con 0x7fb09c1002b0 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10fb50 con 0x7fb09c1053e0 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09b59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c10fe50 con 0x7fb09c100c60 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb099d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 0x7fb09c196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb099d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 0x7fb09c196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56734/0 (socket says 172.21.15.77:56734) 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb099d9b640 1 -- 172.21.15.77:0/3179914004 learned_addr learned my addr 172.21.15.77:0/3179914004 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.138+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 0x7fb09c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 -- 172.21.15.77:0/3179914004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 msgr2=0x7fb09c196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 0x7fb09c196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 -- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 msgr2=0x7fb09c196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 0x7fb09c196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 -- 172.21.15.77:0/3179914004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c19b200 con 0x7fb09c1053e0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 0x7fb09c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c19ab20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb09400be20 tx=0x7fb094009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb08b7fe640 1 -- 172.21.15.77:0/3179914004 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094019070 con 0x7fb09c1053e0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c19b4f0 con 0x7fb09c1053e0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb08b7fe640 1 -- 172.21.15.77:0/3179914004 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09400cdb0 con 0x7fb09c1053e0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.139+0000 7fb08b7fe640 1 -- 172.21.15.77:0/3179914004 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094012970 con 0x7fb09c1053e0 2024-09-15T22:29:36.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.140+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c1a2dd0 con 0x7fb09c1053e0 2024-09-15T22:29:36.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.143+0000 7fb08b7fe640 1 -- 172.21.15.77:0/3179914004 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb09401c070 con 0x7fb09c1053e0 2024-09-15T22:29:36.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.144+0000 7fb08b7fe640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 0x7fb0840a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.144+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 0x7fb0840a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.144+0000 7fb08b7fe640 1 -- 172.21.15.77:0/3179914004 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fb0940cf6e0 con 0x7fb09c1053e0 2024-09-15T22:29:36.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.144+0000 7fb0a319a740 1 Processor -- start 2024-09-15T22:29:36.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.144+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 0x7fb0840a5940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d60 tx=0x7fb08c009110 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 -- start start 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb96c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb96d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ecb8c7160 con 0x559ecb971c60 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ecb86e720 con 0x559ecb96cc20 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb0a319a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ecb8c3120 con 0x559ecb972580 2024-09-15T22:29:36.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45692/0 (socket says 172.21.15.77:45692) 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 -- 172.21.15.77:0/3532493596 learned_addr learned my addr 172.21.15.77:0/3532493596 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb96d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb099d9b640 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb96c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 -- 172.21.15.77:0/3532493596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 msgr2=0x559ecb96c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb96c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 -- 172.21.15.77:0/3532493596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 msgr2=0x559ecb96d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb96d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.145+0000 7fb09a59c640 1 -- 172.21.15.77:0/3532493596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecb638ba0 con 0x559ecb971c60 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.146+0000 7fb09a59c640 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00f000 tx=0x7fb08c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2684d3719a98de1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.146+0000 7fb088ff9640 1 -- 172.21.15.77:0/3532493596 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c01b040 con 0x559ecb971c60 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.146+0000 7fb088ff9640 1 -- 172.21.15.77:0/3532493596 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c024070 con 0x559ecb971c60 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.146+0000 7fb088ff9640 1 -- 172.21.15.77:0/3532493596 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c01f750 con 0x559ecb971c60 2024-09-15T22:29:36.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 -- 172.21.15.77:0/3532493596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 msgr2=0x559ecb972040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00f000 tx=0x7fb08c011000 comp rx=0 tx=0).stop 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 -- 172.21.15.77:0/3532493596 shutdown_connections 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb96d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb96c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 --2- 172.21.15.77:0/3532493596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb971c60 0x559ecb972040 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 -- 172.21.15.77:0/3532493596 >> 172.21.15.77:0/3532493596 conn(0x559ecb95ea60 msgr2=0x559ecb95ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 -- 172.21.15.77:0/3532493596 shutdown_connections 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.147+0000 7fb0a319a740 1 -- 172.21.15.77:0/3532493596 wait complete. 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 -- start start 2024-09-15T22:29:36.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 Processor -- start 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb975fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb976520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ecb978fa0 con 0x559ecb96cc20 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ecb84f900 con 0x559ecb971c60 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ecb979120 con 0x559ecb972580 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb0a319a740 10 client.0 _ll_register_callbacks cb 0x559ecb84d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45706/0 (socket says 172.21.15.77:45706) 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb09a59c640 1 -- 172.21.15.77:0/957376127 learned_addr learned my addr 172.21.15.77:0/957376127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.148+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb976520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 -- 172.21.15.77:0/957376127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 msgr2=0x559ecb976520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb976520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 -- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 msgr2=0x559ecb975fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb975fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 -- 172.21.15.77:0/957376127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecb88fc50 con 0x559ecb96cc20 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb09a59c640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0137c0 tx=0x7fb08c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020eadac1b4b629 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.149+0000 7fb081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c02d070 con 0x559ecb96cc20 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fb08c013d30 con 0x559ecb96cc20 2024-09-15T22:29:36.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08c024070 con 0x559ecb96cc20 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb08c009a70 con 0x559ecb96cc20 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb0a319a740 10 client.19569 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.150+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559ecb984d30 con 0x559ecb96cc20 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.151+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c024070 con 0x559ecb96cc20 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb08c01e030 con 0x559ecb96cc20 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 10 client.19569 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 10 client.19569 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 20 client.19569 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 4 client.19569 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.152+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ecb8d4890 con 0x559ecb9861f0 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.153+0000 7fb099d9b640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.153+0000 7fb099d9b640 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9885e0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbfa82f987a9fc6 server_cookie=28dacee9c721f933 in_seq=0 out_seq=0 2024-09-15T22:29:36.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.153+0000 7fb081ffb640 10 client.19569 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:36.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb081ffb640 1 -- 172.21.15.77:0/957376127 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ecb8d4890 con 0x559ecb9861f0 2024-09-15T22:29:36.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb081ffb640 10 client.19569 handle_command_reply: tid=0 2024-09-15T22:29:36.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 client.19569 shutdown 2024-09-15T22:29:36.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 msgr2=0x559ecb975aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0137c0 tx=0x7fb08c0047d0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 shutdown_connections 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9885e0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb081ffb640 0 client.19569 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb972580 0x559ecb976520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb975fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 --2- 172.21.15.77:0/957376127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb96cc20 0x559ecb975aa0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 >> 172.21.15.77:0/957376127 conn(0x559ecb95ea60 msgr2=0x559ecb95e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.158+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 shutdown_connections 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.159+0000 7fb0a319a740 1 -- 172.21.15.77:0/957376127 wait complete. 2024-09-15T22:29:36.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.159+0000 7fb0a319a740 20 client.19569 trim_cache size 0 max 16384 2024-09-15T22:29:36.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 Processor -- start 2024-09-15T22:29:36.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 -- start start 2024-09-15T22:29:36.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 0x559ecb96c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb95df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1acdb0 con 0x559ecb972580 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ae8f0 con 0x559ecb971c60 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb0a319a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ad480 con 0x559ecb96cc20 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 0x559ecb96c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb099d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb95df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 0x559ecb96c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56768/0 (socket says 172.21.15.77:56768) 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb09a59c640 1 -- 172.21.15.77:0/173514248 learned_addr learned my addr 172.21.15.77:0/173514248 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.184+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 -- 172.21.15.77:0/173514248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 msgr2=0x559ecb96c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 0x559ecb96c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 -- 172.21.15.77:0/173514248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 msgr2=0x559ecb98dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 -- 172.21.15.77:0/173514248 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ecb95a2d0 con 0x559ecb971c60 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb099d9b640 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb95df40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c00 tx=0x7fb090005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50543cc6d374bf85 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb083fff640 1 -- 172.21.15.77:0/173514248 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090036040 con 0x559ecb971c60 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb083fff640 1 -- 172.21.15.77:0/173514248 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb090048070 con 0x559ecb971c60 2024-09-15T22:29:36.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.185+0000 7fb083fff640 1 -- 172.21.15.77:0/173514248 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0900306c0 con 0x559ecb971c60 2024-09-15T22:29:36.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- 172.21.15.77:0/173514248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 msgr2=0x559ecb95df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb95df40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c00 tx=0x7fb090005c00 comp rx=0 tx=0).stop 2024-09-15T22:29:36.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- 172.21.15.77:0/173514248 shutdown_connections 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb971c60 0x559ecb95df40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 --2- 172.21.15.77:0/173514248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb96cc20 0x559ecb96c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- 172.21.15.77:0/173514248 >> 172.21.15.77:0/173514248 conn(0x559ecb95ea60 msgr2=0x559ecb95d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- 172.21.15.77:0/173514248 shutdown_connections 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- 172.21.15.77:0/173514248 wait complete. 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.186+0000 7fb0a319a740 1 -- start start 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 Processor -- start 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb97e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ae220 con 0x559ecb972580 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1adb50 con 0x559ecb96cc20 2024-09-15T22:29:36.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1aadb0 con 0x559ecb971c60 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb0a319a740 10 client.0 _ll_register_callbacks cb 0x7fb08c015f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb97e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb09a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb97e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:52902/0 (socket says 172.21.15.77:52902) 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb09a59c640 1 -- 172.21.15.77:0/2000142793 learned_addr learned my addr 172.21.15.77:0/2000142793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:56782/0 (socket says 172.21.15.77:56782) 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb09ad9d640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 -- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 msgr2=0x559ecb97e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb97e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 -- 172.21.15.77:0/2000142793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 msgr2=0x559ecb98d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.194 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.187+0000 7fb099d9b640 1 -- 172.21.15.77:0/2000142793 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ecb98d9d0 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb099d9b640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb09002bd30 tx=0x7fb090004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60cb63940ef331d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090036050 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb090048070 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fb0900306c0 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0900308c0 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb0a319a740 10 client.28324 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.188+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559ecb9f06e0 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb090048070 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb09000b040 con 0x559ecb971c60 2024-09-15T22:29:36.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 10 client.28324 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 10 client.28324 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 20 client.28324 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 4 client.28324 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}] 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}) -- 0x559ecb8cebe0 con 0x559ecb9861f0 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb09a59c640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb09a59c640 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9f2ee0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70e26ac27e43c53 server_cookie=dcff2deef6480743 in_seq=0 out_seq=0 2024-09-15T22:29:36.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.189+0000 7fb081ffb640 10 client.28324 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:35 smithi077 ceph-mon[30980]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:35 smithi196 ceph-mon[35109]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:35 smithi117 ceph-mon[35021]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 443 op/s 2024-09-15T22:29:36.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78913 19.42 MB/sec execute 97 sec latency 393.737 ms 2024-09-15T22:29:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.833+0000 7fb081ffb640 1 -- 172.21.15.77:0/2000142793 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559ecb8cebe0 con 0x559ecb9861f0 2024-09-15T22:29:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.833+0000 7fb081ffb640 10 client.28324 handle_command_reply: tid=0 2024-09-15T22:29:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.833+0000 7fb0a319a740 1 client.28324 shutdown 2024-09-15T22:29:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 msgr2=0x559ecb97ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb09002bd30 tx=0x7fb090004510 comp rx=0 tx=0).stop 2024-09-15T22:29:36.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 shutdown_connections 2024-09-15T22:29:36.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559ecb9861f0 0x559ecb9f2ee0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ecb972580 0x559ecb98d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb081ffb640 0 client.28324 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ecb971c60 0x559ecb97ead0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 --2- 172.21.15.77:0/2000142793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ecb96cc20 0x559ecb97e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 >> 172.21.15.77:0/2000142793 conn(0x559ecb95ea60 msgr2=0x559ecb95e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.834+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 shutdown_connections 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.835+0000 7fb0a319a740 1 -- 172.21.15.77:0/2000142793 wait complete. 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.835+0000 7fb0a319a740 20 client.28324 trim_cache size 0 max 16384 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:29:36.837 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 62, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "93278e8f": { 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 62, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:29:36.838 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.9, 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:29:36.839 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:29:36.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 msgr2=0x7fb0840a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 0x7fb0840a5940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d60 tx=0x7fb08c009110 comp rx=0 tx=0).stop 2024-09-15T22:29:36.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 msgr2=0x7fb09c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:36.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c19ab20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb09400be20 tx=0x7fb094009ef0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 shutdown_connections 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb0840a3480 0x7fb0840a5940 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb09c1053e0 0x7fb09c19ab20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb09c100c60 0x7fb09c196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 --2- 172.21.15.77:0/3179914004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb09c1002b0 0x7fb09c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 >> 172.21.15.77:0/3179914004 conn(0x7fb09c077070 msgr2=0x7fb09c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 shutdown_connections 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:36.836+0000 7fb09b59e640 1 -- 172.21.15.77:0/3179914004 wait complete. 2024-09-15T22:29:37.592 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93278e8f', quiesce took 5.7 seconds. Will release after: 17.25 2024-09-15T22:29:37.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 19.22 MB/sec execute 98 sec latency 1312.530 ms 2024-09-15T22:29:38.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:37 smithi077 ceph-mon[30980]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 623 op/s 2024-09-15T22:29:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:37 smithi196 ceph-mon[35109]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 623 op/s 2024-09-15T22:29:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:37 smithi117 ceph-mon[35021]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 623 op/s 2024-09-15T22:29:38.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 19.03 MB/sec execute 99 sec latency 2312.577 ms 2024-09-15T22:29:39.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 18.84 MB/sec execute 100 sec latency 3312.656 ms 2024-09-15T22:29:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:39 smithi077 ceph-mon[30980]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 411 op/s 2024-09-15T22:29:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:39 smithi196 ceph-mon[35109]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 411 op/s 2024-09-15T22:29:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:39 smithi117 ceph-mon[35021]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.9 MiB/s wr, 411 op/s 2024-09-15T22:29:40.600 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 18.65 MB/sec execute 101 sec latency 4312.697 ms 2024-09-15T22:29:41.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 18.47 MB/sec execute 102 sec latency 5312.799 ms 2024-09-15T22:29:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:41 smithi077 ceph-mon[30980]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 496 op/s 2024-09-15T22:29:42.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:41 smithi196 ceph-mon[35109]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 496 op/s 2024-09-15T22:29:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:41 smithi117 ceph-mon[35021]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.3 MiB/s wr, 496 op/s 2024-09-15T22:29:42.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 18.29 MB/sec execute 103 sec latency 6312.859 ms 2024-09-15T22:29:43.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 18.11 MB/sec execute 104 sec latency 7312.990 ms 2024-09-15T22:29:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:43 smithi077 ceph-mon[30980]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:43 smithi196 ceph-mon[35109]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:43 smithi117 ceph-mon[35021]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:44.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.94 MB/sec execute 105 sec latency 8313.039 ms 2024-09-15T22:29:45.603 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.77 MB/sec execute 106 sec latency 9313.127 ms 2024-09-15T22:29:46.250 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:45 smithi077 ceph-mon[30980]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:45 smithi196 ceph-mon[35109]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:45 smithi117 ceph-mon[35021]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-09-15T22:29:46.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.61 MB/sec execute 107 sec latency 10313.217 ms 2024-09-15T22:29:47.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:29:47.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.44 MB/sec execute 108 sec latency 11313.247 ms 2024-09-15T22:29:47.750 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:48.160 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:47 smithi077 ceph-mon[30980]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-15T22:29:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- 172.21.15.77:0/2317995294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 msgr2=0x7f809810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- 172.21.15.77:0/2317995294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f809810bb50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f808c009900 tx=0x7f808c02a480 comp rx=0 tx=0).stop 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- 172.21.15.77:0/2317995294 shutdown_connections 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- 172.21.15.77:0/2317995294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f809810c170 0x7f809810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- 172.21.15.77:0/2317995294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f809810bb50 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- 172.21.15.77:0/2317995294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8098102d40 0x7f8098103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- 172.21.15.77:0/2317995294 >> 172.21.15.77:0/2317995294 conn(0x7f80980fc7a0 msgr2=0x7f80980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- 172.21.15.77:0/2317995294 shutdown_connections 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- 172.21.15.77:0/2317995294 wait complete. 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 Processor -- start 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- start start 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8098102d40 0x7f8098198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 0x7f809819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8098111fb0 con 0x7f8098103660 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8098111e30 con 0x7f809810c170 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.241+0000 7f809ed1a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8098112130 con 0x7f8098102d40 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46488/0 (socket says 172.21.15.77:46488) 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 -- 172.21.15.77:0/984041713 learned_addr learned my addr 172.21.15.77:0/984041713 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f809cab6640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8098102d40 0x7f8098198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f809d2b7640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 0x7f809819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 -- 172.21.15.77:0/984041713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8098102d40 msgr2=0x7f8098198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8098102d40 0x7f8098198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 -- 172.21.15.77:0/984041713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 msgr2=0x7f809819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 0x7f809819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 -- 172.21.15.77:0/984041713 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809819d410 con 0x7f8098103660 2024-09-15T22:29:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f809d2b7640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 0x7f809819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8097fff640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f808c005ab0 tx=0x7f808c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8095ffb640 1 -- 172.21.15.77:0/984041713 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c03f030 con 0x7f8098103660 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809819d6a0 con 0x7f8098103660 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8095ffb640 1 -- 172.21.15.77:0/984041713 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c003ce0 con 0x7f8098103660 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.242+0000 7f8095ffb640 1 -- 172.21.15.77:0/984041713 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c035070 con 0x7f8098103660 2024-09-15T22:29:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.243+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981a4fe0 con 0x7f8098103660 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.245+0000 7f8095ffb640 1 -- 172.21.15.77:0/984041713 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f808c042030 con 0x7f8098103660 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.245+0000 7f80a10f3740 1 Processor -- start 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.245+0000 7f80a10f3740 1 -- start start 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57fe6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f8095ffb640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 0x7f80800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b57f4c2e0 con 0x555b57ff5a90 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b57ef3ce0 con 0x555b57fe6cc0 2024-09-15T22:29:48.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f80a10f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b57f482a0 con 0x555b57fe63a0 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809cab6640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 0x7f80800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46492/0 (socket says 172.21.15.77:46492) 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 -- 172.21.15.77:0/4184083800 learned_addr learned my addr 172.21.15.77:0/4184083800 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f8097fff640 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57fe6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f8095ffb640 1 -- 172.21.15.77:0/984041713 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f808c0f09c0 con 0x7f8098103660 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 -- 172.21.15.77:0/4184083800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 msgr2=0x555b57fe6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57fe6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 -- 172.21.15.77:0/4184083800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 msgr2=0x555b57ff5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 -- 172.21.15.77:0/4184083800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b57ed8440 con 0x555b57ff5a90 2024-09-15T22:29:48.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809d2b7640 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8090008fb0 tx=0x7f8090008850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9709158a4ea88001 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f809cab6640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 0x7f80800a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8088009890 tx=0x7f8088008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.246+0000 7f807effd640 1 -- 172.21.15.77:0/4184083800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090014e50 con 0x555b57ff5a90 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f807effd640 1 -- 172.21.15.77:0/4184083800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000f8e0 con 0x555b57ff5a90 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f807effd640 1 -- 172.21.15.77:0/4184083800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900054e0 con 0x555b57ff5a90 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 -- 172.21.15.77:0/4184083800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 msgr2=0x555b57ff7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8090008fb0 tx=0x7f8090008850 comp rx=0 tx=0).stop 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 -- 172.21.15.77:0/4184083800 shutdown_connections 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff7e80 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 --2- 172.21.15.77:0/4184083800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57fe6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 -- 172.21.15.77:0/4184083800 >> 172.21.15.77:0/4184083800 conn(0x555b57fe2a30 msgr2=0x555b57fe2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 -- 172.21.15.77:0/4184083800 shutdown_connections 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.247+0000 7f80a10f3740 1 -- 172.21.15.77:0/4184083800 wait complete. 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 -- start start 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 Processor -- start 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57ff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555b57ff86c0 con 0x555b57ff5a90 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b57f07050 con 0x555b57fe6cc0 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555b57ff8840 con 0x555b57fe63a0 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f80a10f3740 10 client.0 _ll_register_callbacks cb 0x555b57f47160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f809d2b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f809d2b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46494/0 (socket says 172.21.15.77:46494) 2024-09-15T22:29:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f809d2b7640 1 -- 172.21.15.77:0/979121847 learned_addr learned my addr 172.21.15.77:0/979121847 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f809cab6640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57ff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.248+0000 7f8097fff640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 -- 172.21.15.77:0/979121847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 msgr2=0x555b57ff54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57ff54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 -- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 msgr2=0x555b57ff2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 -- 172.21.15.77:0/979121847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b57cbd3c0 con 0x555b57ff5a90 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f809d2b7640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8090008f80 tx=0x7f8090004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53bd6a482291a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f8077fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090008230 con 0x555b57ff5a90 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f8090002030 con 0x555b57ff5a90 2024-09-15T22:29:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809000f8e0 con 0x555b57ff5a90 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f809002f050 con 0x555b57ff5a90 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f80a10f3740 10 client.19587 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555b580041b0 con 0x555b57ff5a90 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.249+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000f8e0 con 0x555b57ff5a90 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8090012610 con 0x555b57ff5a90 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 10 client.19587 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 10 client.19587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 20 client.19587 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58005610 0x555b58007a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 4 client.19587 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b57f59850 con 0x555b58005610 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f809cab6640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58005610 0x555b58007a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f809cab6640 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58005610 0x555b58007a00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1898a5cb3aa75 server_cookie=879665892770cb39 in_seq=0 out_seq=0 2024-09-15T22:29:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.250+0000 7f8077fff640 10 client.19587 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.251+0000 7f8077fff640 1 -- 172.21.15.77:0/979121847 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b57f59850 con 0x555b58005610 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.251+0000 7f8077fff640 10 client.19587 handle_command_reply: tid=0 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.251+0000 7f80a10f3740 1 client.19587 shutdown 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 msgr2=0x555b57ff2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8090008f80 tx=0x7f8090004c90 comp rx=0 tx=0).stop 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 shutdown_connections 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58005610 0x555b58007a00 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f8077fff640 0 client.19587 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57ff5a90 0x555b57ff2f10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ff2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 --2- 172.21.15.77:0/979121847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b57ff54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 >> 172.21.15.77:0/979121847 conn(0x555b57fe2a30 msgr2=0x555b57fe8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 shutdown_connections 2024-09-15T22:29:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 1 -- 172.21.15.77:0/979121847 wait complete. 2024-09-15T22:29:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.252+0000 7f80a10f3740 20 client.19587 trim_cache size 0 max 16384 2024-09-15T22:29:48.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 Processor -- start 2024-09-15T22:29:48.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 -- start start 2024-09-15T22:29:48.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b5800cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57ff5a90 0x555b57fefa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80981acd90 con 0x555b57fe6cc0 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80981ae8d0 con 0x555b57ff5a90 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f80a10f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80981ad460 con 0x555b57fe63a0 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46500/0 (socket says 172.21.15.77:46500) 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 -- 172.21.15.77:0/2270640345 learned_addr learned my addr 172.21.15.77:0/2270640345 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f809cab6640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b5800cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f809d2b7640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57ff5a90 0x555b57fefa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 -- 172.21.15.77:0/2270640345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 msgr2=0x555b5800cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b5800cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 -- 172.21.15.77:0/2270640345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57ff5a90 msgr2=0x555b57fefa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57ff5a90 0x555b57fefa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 -- 172.21.15.77:0/2270640345 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b58007d70 con 0x555b57fe6cc0 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f809cab6640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b5800cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.269+0000 7f8097fff640 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ede30 tx=0x7f808c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0387009d1cf3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f807e7fc640 1 -- 172.21.15.77:0/2270640345 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c049040 con 0x555b57fe6cc0 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f807e7fc640 1 -- 172.21.15.77:0/2270640345 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c0bd010 con 0x555b57fe6cc0 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f807e7fc640 1 -- 172.21.15.77:0/2270640345 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0bde70 con 0x555b57fe6cc0 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 -- 172.21.15.77:0/2270640345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 msgr2=0x555b5800d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ede30 tx=0x7f808c0058b0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 -- 172.21.15.77:0/2270640345 shutdown_connections 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57ff5a90 0x555b57fefa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe6cc0 0x555b5800d4e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2270640345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57fe63a0 0x555b5800cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 -- 172.21.15.77:0/2270640345 >> 172.21.15.77:0/2270640345 conn(0x555b57fe2a30 msgr2=0x555b57ff4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 -- 172.21.15.77:0/2270640345 shutdown_connections 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.270+0000 7f80a10f3740 1 -- 172.21.15.77:0/2270640345 wait complete. 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 -- start start 2024-09-15T22:29:48.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 Processor -- start 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ffaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57ff5a90 0x555b57ffafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f80981ae200 con 0x555b57fe63a0 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80981adb30 con 0x555b57fe6cc0 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80981aad90 con 0x555b57ff5a90 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f80a10f3740 10 client.0 _ll_register_callbacks cb 0x7f8090014720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f809cab6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f809cab6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46508/0 (socket says 172.21.15.77:46508) 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f809cab6640 1 -- 172.21.15.77:0/2016775933 learned_addr learned my addr 172.21.15.77:0/2016775933 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.271+0000 7f809d2b7640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57ff5a90 0x555b57ffafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8097fff640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ffaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 -- 172.21.15.77:0/2016775933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57ff5a90 msgr2=0x555b57ffafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57ff5a90 0x555b57ffafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 -- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 msgr2=0x555b57ffaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ffaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 -- 172.21.15.77:0/2016775933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b57ffb5a0 con 0x555b57fe63a0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8097fff640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ffaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f809cab6640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x555b57ff8960 tx=0x7f80880047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c91bf7497529e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808802b400 con 0x555b57fe63a0 2024-09-15T22:29:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f808800f040 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f808c0bd010 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8088036050 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f80a10f3740 10 client.19593 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555b58070d10 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c0bd010 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.272+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8088013070 con 0x555b57fe63a0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 10 client.19593 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 10 client.19593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 20 client.19593 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58071490 0x555b580738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 4 client.19593 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b57f53d60 con 0x555b58071490 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f8097fff640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58071490 0x555b580738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f8077fff640 10 client.19593 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f8097fff640 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58071490 0x555b580738b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c15d15648951f9 server_cookie=e71810ba9d880497 in_seq=0 out_seq=0 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f8077fff640 1 -- 172.21.15.77:0/2016775933 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555b57f53d60 con 0x555b58071490 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f8077fff640 10 client.19593 handle_command_reply: tid=0 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 1 client.19593 shutdown 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 msgr2=0x555b57ffdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.273+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x555b57ff8960 tx=0x7f80880047d0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 shutdown_connections 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555b58071490 0x555b580738b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555b57ff5a90 0x555b57ffafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b57fe6cc0 0x555b57ffaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 --2- 172.21.15.77:0/2016775933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555b57fe63a0 0x555b57ffdf50 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f8077fff640 0 client.19593 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 >> 172.21.15.77:0/2016775933 conn(0x555b57fe2a30 msgr2=0x555b57fed4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 shutdown_connections 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 1 -- 172.21.15.77:0/2016775933 wait complete. 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f80a10f3740 20 client.19593 trim_cache size 0 max 16384 2024-09-15T22:29:48.282 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:29:48.283 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:29:48.283 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:29:48.283 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:29:48.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 msgr2=0x7f80800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 0x7f80800a5870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8088009890 tx=0x7f8088008040 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 msgr2=0x7f80981989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f808c005ab0 tx=0x7f808c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 shutdown_connections 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f80800a33b0 0x7f80800a5870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809810c170 0x7f809819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8098103660 0x7f80981989a0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 --2- 172.21.15.77:0/984041713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8098102d40 0x7f8098198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.274+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 >> 172.21.15.77:0/984041713 conn(0x7f80980fc7a0 msgr2=0x7f80980fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.275+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 shutdown_connections 2024-09-15T22:29:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:48.275+0000 7f809ed1a640 1 -- 172.21.15.77:0/984041713 wait complete. 2024-09-15T22:29:48.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:47 smithi196 ceph-mon[35109]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-15T22:29:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:47 smithi117 ceph-mon[35021]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 316 op/s 2024-09-15T22:29:48.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.28 MB/sec execute 109 sec latency 12313.275 ms 2024-09-15T22:29:49.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:29:49.186 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:29:49.464 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:49.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 17.13 MB/sec execute 110 sec latency 13313.319 ms 2024-09-15T22:29:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- 172.21.15.77:0/1125837250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 msgr2=0x7f0180106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 --2- 172.21.15.77:0/1125837250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f0180106e60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0174009930 tx=0x7f017402a420 comp rx=0 tx=0).stop 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- 172.21.15.77:0/1125837250 shutdown_connections 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 --2- 172.21.15.77:0/1125837250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 0x7f018010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 --2- 172.21.15.77:0/1125837250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 0x7f0180100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 --2- 172.21.15.77:0/1125837250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f0180106e60 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- 172.21.15.77:0/1125837250 >> 172.21.15.77:0/1125837250 conn(0x7f01800fc7a0 msgr2=0x7f01800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- 172.21.15.77:0/1125837250 shutdown_connections 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- 172.21.15.77:0/1125837250 wait complete. 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 Processor -- start 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.050+0000 7f0188770640 1 -- start start 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 0x7f0180199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 0x7f0180199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0180110ea0 con 0x7f0180106a80 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0180110d20 con 0x7f0180101420 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0188770640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0180111020 con 0x7f0180100a80 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f018650c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 0x7f0180199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51666/0 (socket says 172.21.15.77:51666) 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 -- 172.21.15.77:0/2709287475 learned_addr learned my addr 172.21.15.77:0/2709287475 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0185d0b640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 0x7f0180199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 -- 172.21.15.77:0/2709287475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 msgr2=0x7f0180199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 0x7f0180199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 -- 172.21.15.77:0/2709287475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 msgr2=0x7f0180199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 0x7f0180199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 -- 172.21.15.77:0/2709287475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01801a48f0 con 0x7f0180106a80 2024-09-15T22:29:50.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f0186d0d640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f017800eaa0 tx=0x7f017800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.051+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178004d20 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.052+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0178005020 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.052+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01780116d0 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.052+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01801a4ac0 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.052+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01801a4f80 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.053+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0180100ee0 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.054+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f01780051d0 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.055+0000 7f01737fe640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 0x7f016c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.055+0000 7f018650c640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 0x7f016c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.055+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f01780d0290 con 0x7f0180106a80 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.055+0000 7f018650c640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 0x7f016c0a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f01740058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:50.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.059+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0178016030 con 0x7f0180106a80 2024-09-15T22:29:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.258+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0180108ff0 con 0x7f0180106a80 2024-09-15T22:29:50.261 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:49 smithi077 ceph-mon[30980]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-09-15T22:29:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.259+0000 7f01737fe640 1 -- 172.21.15.77:0/2709287475 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f01780bae20 con 0x7f0180106a80 2024-09-15T22:29:50.262 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:29:50.262 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:29:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:29:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 msgr2=0x7f016c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 0x7f016c0a5870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f01740058d0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 msgr2=0x7f01801a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f017800eaa0 tx=0x7f017800eea0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 shutdown_connections 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f016c0a33b0 0x7f016c0a5870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0180106a80 0x7f01801a43b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0180101420 0x7f0180199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 --2- 172.21.15.77:0/2709287475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0180100a80 0x7f0180199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.261+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 >> 172.21.15.77:0/2709287475 conn(0x7f01800fc7a0 msgr2=0x7f01800fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.262+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 shutdown_connections 2024-09-15T22:29:50.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:50.262+0000 7f0188770640 1 -- 172.21.15.77:0/2709287475 wait complete. 2024-09-15T22:29:50.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:49 smithi196 ceph-mon[35109]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-09-15T22:29:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:49 smithi117 ceph-mon[35021]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 136 op/s 2024-09-15T22:29:50.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.97 MB/sec execute 111 sec latency 14313.359 ms 2024-09-15T22:29:51.080 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:29:51.330 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:51.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:51 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2709287475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:51.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:51 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2709287475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:51.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:51 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2709287475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:51.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.82 MB/sec execute 112 sec latency 15313.400 ms 2024-09-15T22:29:51.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.811+0000 7f222c32e640 1 -- 172.21.15.77:0/1003481174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224101de0 msgr2=0x7f22241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.811+0000 7f222c32e640 1 --2- 172.21.15.77:0/1003481174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224101de0 0x7f22241021c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2220009900 tx=0x7f222002a410 comp rx=0 tx=0).stop 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- 172.21.15.77:0/1003481174 shutdown_connections 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- 172.21.15.77:0/1003481174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224106f10 0x7f222410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- 172.21.15.77:0/1003481174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224102790 0x7f22241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- 172.21.15.77:0/1003481174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224101de0 0x7f22241021c0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- 172.21.15.77:0/1003481174 >> 172.21.15.77:0/1003481174 conn(0x7f22240fd8c0 msgr2=0x7f22240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- 172.21.15.77:0/1003481174 shutdown_connections 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- 172.21.15.77:0/1003481174 wait complete. 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 Processor -- start 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- start start 2024-09-15T22:29:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224101de0 0x7f22240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224102790 0x7f2224074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111f90 con 0x7f2224106f10 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111e10 con 0x7f2224102790 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.812+0000 7f222c32e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2224112110 con 0x7f2224101de0 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51688/0 (socket says 172.21.15.77:51688) 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 -- 172.21.15.77:0/2712890904 learned_addr learned my addr 172.21.15.77:0/2712890904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f22298c9640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224102790 0x7f2224074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224101de0 0x7f22240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 -- 172.21.15.77:0/2712890904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224101de0 msgr2=0x7f22240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224101de0 0x7f22240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 -- 172.21.15.77:0/2712890904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224102790 msgr2=0x7f2224074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224102790 0x7f2224074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 -- 172.21.15.77:0/2712890904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241a6920 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222a8cb640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f221c00cce0 tx=0x7f221c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f221b7fe640 1 -- 172.21.15.77:0/2712890904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c00f450 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f221b7fe640 1 -- 172.21.15.77:0/2712890904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c00fad0 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241a6c10 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f221b7fe640 1 -- 172.21.15.77:0/2712890904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c018c10 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.813+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a7150 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f221b7fe640 1 -- 172.21.15.77:0/2712890904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f221c00f5f0 con 0x7f2224106f10 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 Processor -- start 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 -- start start 2024-09-15T22:29:51.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557ed401e2f0 con 0x557ed40b85d0 2024-09-15T22:29:51.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ed3fc5cf0 con 0x557ed40b9b40 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222e707740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ed401a2b0 con 0x557ed40c3370 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.816+0000 7f222a0ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51698/0 (socket says 172.21.15.77:51698) 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 -- 172.21.15.77:0/1098646030 learned_addr learned my addr 172.21.15.77:0/1098646030 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f22298c9640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f22298c9640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f221b7fe640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 0x7f22100a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a8cb640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 0x7f22100a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 -- 172.21.15.77:0/1098646030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 msgr2=0x557ed40c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 -- 172.21.15.77:0/1098646030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 msgr2=0x557ed40b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 -- 172.21.15.77:0/1098646030 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed3faa3a0 con 0x557ed40b85d0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f22298c9640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f221b7fe640 1 -- 172.21.15.77:0/2712890904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f221c0d0800 con 0x7f2224106f10 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a0ca640 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2220005ab0 tx=0x7f2220031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eb36691ad882e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f222a8cb640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 0x7f22100a58f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f221c0990c0 tx=0x7f221c06e470 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f2218dea640 1 -- 172.21.15.77:0/1098646030 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222003f070 con 0x557ed40b85d0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f2218dea640 1 -- 172.21.15.77:0/1098646030 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2220031a20 con 0x557ed40b85d0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.817+0000 7f2218dea640 1 -- 172.21.15.77:0/1098646030 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220035070 con 0x557ed40b85d0 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- 172.21.15.77:0/1098646030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 msgr2=0x557ed40b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2220005ab0 tx=0x7f2220031630 comp rx=0 tx=0).stop 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- 172.21.15.77:0/1098646030 shutdown_connections 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40b8a10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 --2- 172.21.15.77:0/1098646030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- 172.21.15.77:0/1098646030 >> 172.21.15.77:0/1098646030 conn(0x557ed4081310 msgr2=0x557ed40b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- 172.21.15.77:0/1098646030 shutdown_connections 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- 172.21.15.77:0/1098646030 wait complete. 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.818+0000 7f222e707740 1 -- start start 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 Processor -- start 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557ed40c6010 con 0x557ed40b85d0 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ed3fd9060 con 0x557ed40b9b40 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ed40c6190 con 0x557ed40c3370 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222e707740 10 client.0 _ll_register_callbacks cb 0x557ed3d2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51706/0 (socket says 172.21.15.77:51706) 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 -- 172.21.15.77:0/2803408106 learned_addr learned my addr 172.21.15.77:0/2803408106 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 -- 172.21.15.77:0/2803408106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 msgr2=0x557ed40c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 -- 172.21.15.77:0/2803408106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 msgr2=0x557ed40bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 -- 172.21.15.77:0/2803408106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed3d8f3c0 con 0x557ed40b85d0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f22298c9640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f221400af10 tx=0x7f2214008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432875de1e03f7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.819+0000 7f220dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214020060 con 0x557ed40b85d0 2024-09-15T22:29:51.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f2214018450 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2220031a20 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2214027050 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 10 client.19611 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ed40d1a50 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2220031a20 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2214015040 con 0x557ed40b85d0 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 10 client.19611 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 10 client.19611 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 20 client.19611 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed40d2f10 0x557ed40d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 4 client.19611 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed402b860 con 0x557ed40d2f10 2024-09-15T22:29:51.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.820+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed40d2f10 0x557ed40d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f220dffb640 10 client.19611 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed40d2f10 0x557ed40d5300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5622d3a2b89d040 server_cookie=9ad01dab3f59c06b in_seq=0 out_seq=0 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f220dffb640 1 -- 172.21.15.77:0/2803408106 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ed402b860 con 0x557ed40d2f10 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f220dffb640 10 client.19611 handle_command_reply: tid=0 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 client.19611 shutdown 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 msgr2=0x557ed40c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f221400af10 tx=0x7f2214008040 comp rx=0 tx=0).stop 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 shutdown_connections 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed40d2f10 0x557ed40d5300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40c3370 0x557ed40c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b9b40 0x557ed40bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f220dffb640 0 client.19611 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:51.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 --2- 172.21.15.77:0/2803408106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b85d0 0x557ed40c2cc0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.821+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 >> 172.21.15.77:0/2803408106 conn(0x557ed4081310 msgr2=0x557ed40ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.822+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 shutdown_connections 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.822+0000 7f222e707740 1 -- 172.21.15.77:0/2803408106 wait complete. 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.822+0000 7f222e707740 20 client.19611 trim_cache size 0 max 16384 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 Processor -- start 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 -- start start 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b85d0 0x557ed40b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 0x557ed40afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b10a0 con 0x557ed40c3370 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b2be0 con 0x557ed40b85d0 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.834+0000 7f222e707740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b1770 con 0x557ed40b9b40 2024-09-15T22:29:51.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f222a0ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 0x557ed40afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51718/0 (socket says 172.21.15.77:51718) 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 -- 172.21.15.77:0/1957392467 learned_addr learned my addr 172.21.15.77:0/1957392467 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f222a0ca640 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 0x557ed40afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35390/0 (socket says 172.21.15.77:35390) 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b85d0 0x557ed40b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 -- 172.21.15.77:0/1957392467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 msgr2=0x557ed40afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 0x557ed40afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 -- 172.21.15.77:0/1957392467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b85d0 msgr2=0x557ed40b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b85d0 0x557ed40b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 -- 172.21.15.77:0/1957392467 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed40be4e0 con 0x557ed40c3370 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f22298c9640 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2214017fc0 tx=0x7f22140047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c988d3dd45fece8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f220ffff640 1 -- 172.21.15.77:0/1957392467 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214020070 con 0x557ed40c3370 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f220ffff640 1 -- 172.21.15.77:0/1957392467 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221400ac90 con 0x557ed40c3370 2024-09-15T22:29:51.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.835+0000 7f220ffff640 1 -- 172.21.15.77:0/1957392467 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214017130 con 0x557ed40c3370 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- 172.21.15.77:0/1957392467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 msgr2=0x557ed40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2214017fc0 tx=0x7f22140047d0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- 172.21.15.77:0/1957392467 shutdown_connections 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40c3370 0x557ed40b3c10 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b9b40 0x557ed40afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 --2- 172.21.15.77:0/1957392467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40b85d0 0x557ed40b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- 172.21.15.77:0/1957392467 >> 172.21.15.77:0/1957392467 conn(0x557ed4081310 msgr2=0x557ed40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- 172.21.15.77:0/1957392467 shutdown_connections 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- 172.21.15.77:0/1957392467 wait complete. 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 -- start start 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.836+0000 7f222e707740 1 Processor -- start 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b85d0 0x557ed40cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 0x557ed40bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b2510 con 0x557ed40b9b40 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b1e40 con 0x557ed40c3370 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22241af0a0 con 0x557ed40b85d0 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 10 client.0 _ll_register_callbacks cb 0x7f221401a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51734/0 (socket says 172.21.15.77:51734) 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 -- 172.21.15.77:0/2530713499 learned_addr learned my addr 172.21.15.77:0/2530713499 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f22298c9640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b85d0 0x557ed40cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f22298c9640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 0x557ed40bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 -- 172.21.15.77:0/2530713499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b85d0 msgr2=0x557ed40cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b85d0 0x557ed40cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 -- 172.21.15.77:0/2530713499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 msgr2=0x557ed40bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:51.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 0x557ed40bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 -- 172.21.15.77:0/2530713499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed40bcb00 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f22298c9640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 0x557ed40bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222a0ca640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f22200096c0 tx=0x7f2220005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d506f957259f0326 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f220dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220035040 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f2220004820 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.837+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221400ac90 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f222005d050 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 10 client.19617 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ed413d380 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221400ac90 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f222004eaf0 con 0x557ed40b9b40 2024-09-15T22:29:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 10 client.19617 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 10 client.19617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:29:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 20 client.19617 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:51.866 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed413d6e0 0x557ed413fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 4 client.19617 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ed4025d70 con 0x557ed413d6e0 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f22298c9640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed413d6e0 0x557ed413fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f220dffb640 10 client.19617 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.838+0000 7f22298c9640 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed413d6e0 0x557ed413fb00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a05bed82e51f20e6 server_cookie=5fe0dd3ec8389a28 in_seq=0 out_seq=0 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.839+0000 7f220dffb640 1 -- 172.21.15.77:0/2530713499 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ed4025d70 con 0x557ed413d6e0 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.839+0000 7f220dffb640 10 client.19617 handle_command_reply: tid=0 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.839+0000 7f222e707740 1 client.19617 shutdown 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 msgr2=0x557ed40cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f22200096c0 tx=0x7f2220005520 comp rx=0 tx=0).stop 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 shutdown_connections 2024-09-15T22:29:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x557ed413d6e0 0x557ed413fb00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f220dffb640 0 client.19617 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ed40c3370 0x557ed40bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ed40b9b40 0x557ed40cb990 secure :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f22200096c0 tx=0x7f2220005520 comp rx=0 tx=0).stop 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 --2- 172.21.15.77:0/2530713499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ed40b85d0 0x557ed40cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.840+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 >> 172.21.15.77:0/2530713499 conn(0x557ed4081310 msgr2=0x557ed40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.841+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 shutdown_connections 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.841+0000 7f222e707740 1 -- 172.21.15.77:0/2530713499 wait complete. 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.841+0000 7f222e707740 20 client.19617 trim_cache size 0 max 16384 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 msgr2=0x7f22100a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 0x7f22100a58f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f221c0990c0 tx=0x7f221c06e470 comp rx=0 tx=0).stop 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 msgr2=0x7f2224074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f221c00cce0 tx=0x7f221c00a620 comp rx=0 tx=0).stop 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 shutdown_connections 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f22100a3430 0x7f22100a58f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2224106f10 0x7f2224074f70 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2224102790 0x7f2224074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 --2- 172.21.15.77:0/2712890904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2224101de0 0x7f22240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 >> 172.21.15.77:0/2712890904 conn(0x7f22240fd8c0 msgr2=0x7f22240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 shutdown_connections 2024-09-15T22:29:51.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:51.842+0000 7f222c32e640 1 -- 172.21.15.77:0/2712890904 wait complete. 2024-09-15T22:29:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:52 smithi077 ceph-mon[30980]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 199 op/s 2024-09-15T22:29:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:52 smithi196 ceph-mon[35109]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 199 op/s 2024-09-15T22:29:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:52 smithi117 ceph-mon[35021]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 199 op/s 2024-09-15T22:29:52.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.67 MB/sec execute 113 sec latency 16313.431 ms 2024-09-15T22:29:52.751 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:29:52.995 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- 172.21.15.77:0/1816429094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660106f10 msgr2=0x7fe66010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 --2- 172.21.15.77:0/1816429094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660106f10 0x7fe66010ea90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe658009ca0 tx=0x7fe65802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- 172.21.15.77:0/1816429094 shutdown_connections 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 --2- 172.21.15.77:0/1816429094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660106f10 0x7fe66010ea90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 --2- 172.21.15.77:0/1816429094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660102790 0x7fe6601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 --2- 172.21.15.77:0/1816429094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660101de0 0x7fe6601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- 172.21.15.77:0/1816429094 >> 172.21.15.77:0/1816429094 conn(0x7fe6600fd8c0 msgr2=0x7fe6600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- 172.21.15.77:0/1816429094 shutdown_connections 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- 172.21.15.77:0/1816429094 wait complete. 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 Processor -- start 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.589+0000 7fe66801d640 1 -- start start 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660102790 0x7fe660198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660106f10 0x7fe66019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe660111f70 con 0x7fe660101de0 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe660111df0 con 0x7fe660102790 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601120f0 con 0x7fe660106f10 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51752/0 (socket says 172.21.15.77:51752) 2024-09-15T22:29:53.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 -- 172.21.15.77:0/3231197862 learned_addr learned my addr 172.21.15.77:0/3231197862 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe6655b8640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660102790 0x7fe660198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe6665ba640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660106f10 0x7fe66019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 -- 172.21.15.77:0/3231197862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660106f10 msgr2=0x7fe66019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660106f10 0x7fe66019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 -- 172.21.15.77:0/3231197862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660102790 msgr2=0x7fe660198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660102790 0x7fe660198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 -- 172.21.15.77:0/3231197862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66019d3c0 con 0x7fe660101de0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe665db9640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe65000e9f0 tx=0x7fe65000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe656ffd640 1 -- 172.21.15.77:0/3231197862 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65000cd80 con 0x7fe660101de0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe656ffd640 1 -- 172.21.15.77:0/3231197862 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe650010910 con 0x7fe660101de0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66019d6b0 con 0x7fe660101de0 2024-09-15T22:29:53.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe656ffd640 1 -- 172.21.15.77:0/3231197862 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650018990 con 0x7fe660101de0 2024-09-15T22:29:53.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.590+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6601a4f90 con 0x7fe660101de0 2024-09-15T22:29:53.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe656ffd640 1 -- 172.21.15.77:0/3231197862 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe650010ac0 con 0x7fe660101de0 2024-09-15T22:29:53.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 Processor -- start 2024-09-15T22:29:53.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 -- start start 2024-09-15T22:29:53.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba8752b2f0 con 0x55ba875c55d0 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba874d2cf0 con 0x55ba875c6b40 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba875272b0 con 0x55ba875d0370 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe665db9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe665db9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51762/0 (socket says 172.21.15.77:51762) 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe665db9640 1 -- 172.21.15.77:0/2135564340 learned_addr learned my addr 172.21.15.77:0/2135564340 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.593+0000 7fe6665ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6655b8640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe656ffd640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 0x7fe64c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 -- 172.21.15.77:0/2135564340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 msgr2=0x55ba875d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 -- 172.21.15.77:0/2135564340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 msgr2=0x55ba875c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 -- 172.21.15.77:0/2135564340 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba874b73a0 con 0x55ba875c6b40 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe665db9640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6655b8640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe656ffd640 1 -- 172.21.15.77:0/3231197862 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fe6500d0470 con 0x7fe660101de0 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 0x7fe64c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875c5090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe658006010 tx=0x7fe658009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce099f5c7b57ca12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe64bfff640 1 -- 172.21.15.77:0/2135564340 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658041040 con 0x55ba875c6b40 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe6665ba640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 0x7fe64c0a5870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ba875c7170 tx=0x7fe65804d000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe64bfff640 1 -- 172.21.15.77:0/2135564340 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658036070 con 0x55ba875c6b40 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.594+0000 7fe64bfff640 1 -- 172.21.15.77:0/2135564340 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65802f8e0 con 0x55ba875c6b40 2024-09-15T22:29:53.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- 172.21.15.77:0/2135564340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 msgr2=0x55ba875c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875c5090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe658006010 tx=0x7fe658009800 comp rx=0 tx=0).stop 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- 172.21.15.77:0/2135564340 shutdown_connections 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c55d0 0x55ba875c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/2135564340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875c5090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- 172.21.15.77:0/2135564340 >> 172.21.15.77:0/2135564340 conn(0x55ba8758e310 msgr2=0x55ba875c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- 172.21.15.77:0/2135564340 shutdown_connections 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- 172.21.15.77:0/2135564340 wait complete. 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.595+0000 7fe66a3f6740 1 -- start start 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 Processor -- start 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 0x55ba875cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba875d7480 con 0x55ba875c6b40 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba874e6060 con 0x55ba875c55d0 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba875d7600 con 0x55ba875d0370 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe66a3f6740 10 client.0 _ll_register_callbacks cb 0x55ba872398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:52138/0 (socket says 172.21.15.77:52138) 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 -- 172.21.15.77:0/4090282543 learned_addr learned my addr 172.21.15.77:0/4090282543 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe6655b8640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 0x55ba875cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe6665ba640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 -- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 msgr2=0x55ba875cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.596+0000 7fe665db9640 1 -- 172.21.15.77:0/4090282543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 msgr2=0x55ba875cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe665db9640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 0x55ba875cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe665db9640 1 -- 172.21.15.77:0/4090282543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8729c3c0 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6655b8640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 0x55ba875cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe665db9640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe650098820 tx=0x7fe650000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff40d94877f2c06b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65009bf00 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fe650018b30 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe658036070 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe65009bd20 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 10 client.28442 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba875d2d50 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658036070 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe65009b810 con 0x55ba875c55d0 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 10 client.28442 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:53.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 10 client.28442 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 20 client.28442 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba875d4160 0x55ba875d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 4 client.28442 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.597+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba87538860 con 0x55ba875d4160 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.598+0000 7fe6655b8640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba875d4160 0x55ba875d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.598+0000 7fe6497fa640 10 client.28442 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.598+0000 7fe6655b8640 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba875d4160 0x55ba875d65d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32920ad959c5268b server_cookie=e20c4e09a3812540 in_seq=0 out_seq=0 2024-09-15T22:29:53.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe6497fa640 1 -- 172.21.15.77:0/4090282543 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba87538860 con 0x55ba875d4160 2024-09-15T22:29:53.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.52 MB/sec execute 114 sec latency 17313.460 ms 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe6497fa640 10 client.28442 handle_command_reply: tid=0 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 client.28442 shutdown 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 msgr2=0x55ba875cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe650098820 tx=0x7fe650000bc0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 shutdown_connections 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba875d4160 0x55ba875d65d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875d0370 0x55ba875cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875c6b40 0x55ba875cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe6497fa640 0 client.28442 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/4090282543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c55d0 0x55ba875cfda0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 >> 172.21.15.77:0/4090282543 conn(0x55ba8758e310 msgr2=0x55ba875c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 shutdown_connections 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 1 -- 172.21.15.77:0/4090282543 wait complete. 2024-09-15T22:29:53.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.599+0000 7fe66a3f6740 20 client.28442 trim_cache size 0 max 16384 2024-09-15T22:29:53.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 Processor -- start 2024-09-15T22:29:53.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 -- start start 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601ace80 con 0x55ba875d0370 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601ae9c0 con 0x55ba875c6b40 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601ad550 con 0x55ba875c55d0 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe6665ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe6665ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51780/0 (socket says 172.21.15.77:51780) 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe6665ba640 1 -- 172.21.15.77:0/1513913666 learned_addr learned my addr 172.21.15.77:0/1513913666 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe6655b8640 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.612+0000 7fe665db9640 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 -- 172.21.15.77:0/1513913666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 msgr2=0x55ba875dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 -- 172.21.15.77:0/1513913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 msgr2=0x55ba875ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 -- 172.21.15.77:0/1513913666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba875c63a0 con 0x55ba875d0370 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe6665ba640 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe65802ade0 tx=0x7fe6580095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874a066e7cb2bdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe64b7fe640 1 -- 172.21.15.77:0/1513913666 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658041050 con 0x55ba875d0370 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe64b7fe640 1 -- 172.21.15.77:0/1513913666 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658036040 con 0x55ba875d0370 2024-09-15T22:29:53.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.613+0000 7fe64b7fe640 1 -- 172.21.15.77:0/1513913666 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658030940 con 0x55ba875d0370 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1513913666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 msgr2=0x55ba875ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe65802ade0 tx=0x7fe6580095b0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1513913666 shutdown_connections 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875ca7f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1513913666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1513913666 >> 172.21.15.77:0/1513913666 conn(0x55ba8758e310 msgr2=0x55ba875df890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1513913666 shutdown_connections 2024-09-15T22:29:53.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1513913666 wait complete. 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 -- start start 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.614+0000 7fe66a3f6740 1 Processor -- start 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601ae2f0 con 0x55ba875d0370 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601adc20 con 0x55ba875c6b40 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6601aae80 con 0x55ba875c55d0 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 10 client.0 _ll_register_callbacks cb 0x7fe65000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe665db9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:52156/0 (socket says 172.21.15.77:52156) 2024-09-15T22:29:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe665db9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35434/0 (socket says 172.21.15.77:35434) 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 -- 172.21.15.77:0/1803375136 learned_addr learned my addr 172.21.15.77:0/1803375136 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6665ba640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 -- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 msgr2=0x55ba875e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 -- 172.21.15.77:0/1803375136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 msgr2=0x55ba875e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 -- 172.21.15.77:0/1803375136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba875c5be0 con 0x55ba875c6b40 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe665db9640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6665ba640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6655b8640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ba875c6070 tx=0x7fe65c012d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d40340a4f659dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c01faa0 con 0x55ba875c6b40 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fe65c01fc40 con 0x55ba875c6b40 2024-09-15T22:29:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.615+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe658036040 con 0x55ba875c6b40 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe65c01fe40 con 0x55ba875c6b40 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 10 client.28448 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ba875c2960 con 0x55ba875c6b40 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe658036040 con 0x55ba875c6b40 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe65c010550 con 0x55ba875c6b40 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 10 client.28448 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 10 client.28448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 20 client.28448 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba8764b980 0x55ba8764dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 4 client.28448 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba87532d70 con 0x55ba8764b980 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.616+0000 7fe665db9640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba8764b980 0x55ba8764dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:53.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe6497fa640 10 client.28448 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe665db9640 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba8764b980 0x55ba8764dda0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c4cb6c4efb50938 server_cookie=b866d8156e50410e in_seq=0 out_seq=0 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe6497fa640 1 -- 172.21.15.77:0/1803375136 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba87532d70 con 0x55ba8764b980 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe6497fa640 10 client.28448 handle_command_reply: tid=0 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 client.28448 shutdown 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 msgr2=0x55ba875e64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ba875c6070 tx=0x7fe65c012d10 comp rx=0 tx=0).stop 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 shutdown_connections 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55ba8764b980 0x55ba8764dda0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe6497fa640 0 client.28448 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba875d0370 0x55ba875e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba875c6b40 0x55ba875e64f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 --2- 172.21.15.77:0/1803375136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba875c55d0 0x55ba875e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 >> 172.21.15.77:0/1803375136 conn(0x55ba8758e310 msgr2=0x55ba875e00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 shutdown_connections 2024-09-15T22:29:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 1 -- 172.21.15.77:0/1803375136 wait complete. 2024-09-15T22:29:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.617+0000 7fe66a3f6740 20 client.28448 trim_cache size 0 max 16384 2024-09-15T22:29:53.626 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 msgr2=0x7fe64c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 0x7fe64c0a5870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55ba875c7170 tx=0x7fe65804d000 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 msgr2=0x7fe660198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe65000e9f0 tx=0x7fe65000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 shutdown_connections 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe64c0a33b0 0x7fe64c0a5870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe660106f10 0x7fe66019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe660102790 0x7fe660198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 --2- 172.21.15.77:0/3231197862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe660101de0 0x7fe660198410 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 >> 172.21.15.77:0/3231197862 conn(0x7fe6600fd8c0 msgr2=0x7fe6600feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 shutdown_connections 2024-09-15T22:29:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:53.618+0000 7fe66801d640 1 -- 172.21.15.77:0/3231197862 wait complete. 2024-09-15T22:29:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:54 smithi077 ceph-mon[30980]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:54 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:54 smithi196 ceph-mon[35109]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:54 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:54 smithi117 ceph-mon[35021]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:54 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:54.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:29:54.601 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.38 MB/sec execute 115 sec latency 18313.501 ms 2024-09-15T22:29:54.731 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:54.843 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:29:55.112 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:55.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:55.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:29:55.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:29:55.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:55.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:29:55.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.24 MB/sec execute 116 sec latency 19313.573 ms 2024-09-15T22:29:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 -- 172.21.15.77:0/2732586135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc83810c170 msgr2=0x7fc83810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/2732586135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc83810c170 0x7fc83810e610 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc830009900 tx=0x7fc83002a480 comp rx=0 tx=0).stop 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 -- 172.21.15.77:0/2732586135 shutdown_connections 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/2732586135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc83810c170 0x7fc83810e610 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/2732586135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc838103660 0x7fc83810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/2732586135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 0x7fc838103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 -- 172.21.15.77:0/2732586135 >> 172.21.15.77:0/2732586135 conn(0x7fc8380fc7a0 msgr2=0x7fc8380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 -- 172.21.15.77:0/2732586135 shutdown_connections 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.747+0000 7fc83ee8e640 1 -- 172.21.15.77:0/2732586135 wait complete. 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 Processor -- start 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 -- start start 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 0x7fc838198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc83810c170 0x7fc83819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc838111f50 con 0x7fc838103660 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc838111dd0 con 0x7fc838102d40 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83ee8e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381120d0 con 0x7fc83810c170 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51820/0 (socket says 172.21.15.77:51820) 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc837fff640 1 -- 172.21.15.77:0/148632540 learned_addr learned my addr 172.21.15.77:0/148632540 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.748+0000 7fc83d42b640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc83810c170 0x7fc83819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 0x7fc838198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 -- 172.21.15.77:0/148632540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc83810c170 msgr2=0x7fc83819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc83810c170 0x7fc83819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 -- 172.21.15.77:0/148632540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 msgr2=0x7fc838198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 0x7fc838198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 -- 172.21.15.77:0/148632540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83819d410 con 0x7fc838103660 2024-09-15T22:29:55.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc837fff640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc82400d750 tx=0x7fc82400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc835ffb640 1 -- 172.21.15.77:0/148632540 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824017070 con 0x7fc838103660 2024-09-15T22:29:55.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83819d700 con 0x7fc838103660 2024-09-15T22:29:55.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc835ffb640 1 -- 172.21.15.77:0/148632540 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc824005020 con 0x7fc838103660 2024-09-15T22:29:55.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc835ffb640 1 -- 172.21.15.77:0/148632540 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824004d00 con 0x7fc838103660 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.749+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8381a4fe0 con 0x7fc838103660 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.751+0000 7fc835ffb640 1 -- 172.21.15.77:0/148632540 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc824009a30 con 0x7fc838103660 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 Processor -- start 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 -- start start 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c2b40 0x559eeb9c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc835ffb640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 0x7fc8280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559eeb9272f0 con 0x559eeb9c15d0 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eeb8cecf0 con 0x559eeb9c2b40 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc841267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eeb9232b0 con 0x559eeb9d0780 2024-09-15T22:29:55.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83d42b640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 0x7fc8280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51836/0 (socket says 172.21.15.77:51836) 2024-09-15T22:29:55.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 -- 172.21.15.77:0/4120086487 learned_addr learned my addr 172.21.15.77:0/4120086487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:55.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83d42b640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c2b40 0x559eeb9c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc835ffb640 1 -- 172.21.15.77:0/148632540 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fc8240cfad0 con 0x7fc838103660 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc837fff640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 -- 172.21.15.77:0/4120086487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 msgr2=0x559eeb9d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 -- 172.21.15.77:0/4120086487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c2b40 msgr2=0x559eeb9c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c2b40 0x559eeb9c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc83cc2a640 1 -- 172.21.15.77:0/4120086487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eeb8b33a0 con 0x559eeb9c15d0 2024-09-15T22:29:55.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.752+0000 7fc837fff640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009fc0 tx=0x7fc82c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a3cd5dbb71d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc83d42b640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 0x7fc8280a5870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc830031550 tx=0x7fc830005d50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc822ffd640 1 -- 172.21.15.77:0/4120086487 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c004d20 con 0x559eeb9c15d0 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc822ffd640 1 -- 172.21.15.77:0/4120086487 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc82c0119e0 con 0x559eeb9c15d0 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc822ffd640 1 -- 172.21.15.77:0/4120086487 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c005580 con 0x559eeb9c15d0 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 -- 172.21.15.77:0/4120086487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 msgr2=0x559eeb9c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009fc0 tx=0x7fc82c00f7f0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 -- 172.21.15.77:0/4120086487 shutdown_connections 2024-09-15T22:29:55.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9c1a10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 --2- 172.21.15.77:0/4120086487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c2b40 0x559eeb9c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.753+0000 7fc841267740 1 -- 172.21.15.77:0/4120086487 >> 172.21.15.77:0/4120086487 conn(0x559eeb98a310 msgr2=0x559eeb9bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- 172.21.15.77:0/4120086487 shutdown_connections 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- 172.21.15.77:0/4120086487 wait complete. 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- start start 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 Processor -- start 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559eeb9d3480 con 0x559eeb9c2b40 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eeb8e2060 con 0x559eeb9c15d0 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559eeb9d3600 con 0x559eeb9d0780 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc841267740 10 client.0 _ll_register_callbacks cb 0x559eeb6358c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.754+0000 7fc837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51844/0 (socket says 172.21.15.77:51844) 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 -- 172.21.15.77:0/2559371711 learned_addr learned my addr 172.21.15.77:0/2559371711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 -- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 msgr2=0x559eeb9cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 -- 172.21.15.77:0/2559371711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 msgr2=0x559eeb9d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 -- 172.21.15.77:0/2559371711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eeb6983c0 con 0x559eeb9c2b40 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc837fff640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc82409b950 tx=0x7fc8240b4e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ee53687392f1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc817fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824020040 con 0x559eeb9c2b40 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fc824099cc0 con 0x559eeb9c2b40 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82c0119e0 con 0x559eeb9c2b40 2024-09-15T22:29:55.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc82409b9c0 con 0x559eeb9c2b40 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc841267740 10 client.19647 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559eeb9dede0 con 0x559eeb9c2b40 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.755+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc82c0119e0 con 0x559eeb9c2b40 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc824098030 con 0x559eeb9c2b40 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 10 client.19647 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 10 client.19647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 20 client.19647 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeb9e0240 0x559eeb9e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 4 client.19647 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:55.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eeb934860 con 0x559eeb9e0240 2024-09-15T22:29:55.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeb9e0240 0x559eeb9e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc817fff640 10 client.19647 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:55.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.756+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeb9e0240 0x559eeb9e2690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1fc11260faa559 server_cookie=ada91e7f3d808d08 in_seq=0 out_seq=0 2024-09-15T22:29:55.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc817fff640 1 -- 172.21.15.77:0/2559371711 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eeb934860 con 0x559eeb9e0240 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc817fff640 10 client.19647 handle_command_reply: tid=0 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 client.19647 shutdown 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 msgr2=0x559eeb9cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc82409b950 tx=0x7fc8240b4e60 comp rx=0 tx=0).stop 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 shutdown_connections 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeb9e0240 0x559eeb9e2690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9d0780 0x559eeb9cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c2b40 0x559eeb9cd370 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc817fff640 0 client.19647 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 --2- 172.21.15.77:0/2559371711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 >> 172.21.15.77:0/2559371711 conn(0x559eeb98a310 msgr2=0x559eeb9c3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 shutdown_connections 2024-09-15T22:29:55.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 1 -- 172.21.15.77:0/2559371711 wait complete. 2024-09-15T22:29:55.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.757+0000 7fc841267740 20 client.19647 trim_cache size 0 max 16384 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 Processor -- start 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 -- start start 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9dc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9d0780 0x559eeb9e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381acd20 con 0x559eeb9c15d0 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ae860 con 0x559eeb9d0780 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc841267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ad3f0 con 0x559eeb9c2b40 2024-09-15T22:29:55.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51848/0 (socket says 172.21.15.77:51848) 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 -- 172.21.15.77:0/2710480621 learned_addr learned my addr 172.21.15.77:0/2710480621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83d42b640 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9d0780 0x559eeb9e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc837fff640 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9dc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 -- 172.21.15.77:0/2710480621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 msgr2=0x559eeb9dc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9dc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 -- 172.21.15.77:0/2710480621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9d0780 msgr2=0x559eeb9e7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9d0780 0x559eeb9e7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.771+0000 7fc83cc2a640 1 -- 172.21.15.77:0/2710480621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eeb9cb8c0 con 0x559eeb9c15d0 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.772+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009320 tx=0x7fc82c0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d378a211dc6ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.772+0000 7fc8227fc640 1 -- 172.21.15.77:0/2710480621 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c004e70 con 0x559eeb9c15d0 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.772+0000 7fc8227fc640 1 -- 172.21.15.77:0/2710480621 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc82c003e50 con 0x559eeb9c15d0 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.772+0000 7fc8227fc640 1 -- 172.21.15.77:0/2710480621 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c004020 con 0x559eeb9c15d0 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- 172.21.15.77:0/2710480621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 msgr2=0x559eeb9bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009320 tx=0x7fc82c0098c0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- 172.21.15.77:0/2710480621 shutdown_connections 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9d0780 0x559eeb9e7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 --2- 172.21.15.77:0/2710480621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9c15d0 0x559eeb9bde10 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- 172.21.15.77:0/2710480621 >> 172.21.15.77:0/2710480621 conn(0x559eeb98a310 msgr2=0x559eeb9bd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- 172.21.15.77:0/2710480621 shutdown_connections 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- 172.21.15.77:0/2710480621 wait complete. 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 -- start start 2024-09-15T22:29:55.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.773+0000 7fc841267740 1 Processor -- start 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381ae190 con 0x559eeb9d0780 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381adac0 con 0x559eeb9c15d0 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8381aad20 con 0x559eeb9c2b40 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc841267740 10 client.0 _ll_register_callbacks cb 0x7fc824004ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:55.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51852/0 (socket says 172.21.15.77:51852) 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83cc2a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 -- 172.21.15.77:0/3436014460 learned_addr learned my addr 172.21.15.77:0/3436014460 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc837fff640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 -- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 msgr2=0x559eeb9d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 -- 172.21.15.77:0/3436014460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 msgr2=0x559eeb9d8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 -- 172.21.15.77:0/3436014460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eeb9d5060 con 0x559eeb9d0780 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.774+0000 7fc83d42b640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc83002a9a0 tx=0x7fc830046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0544131f5c6b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830050070 con 0x559eeb9d0780 2024-09-15T22:29:55.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fc8300047a0 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82c003e50 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc83005e050 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 10 client.19653 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559eeba4af20 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc82c003e50 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc830003b50 con 0x559eeb9d0780 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 10 client.19653 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 10 client.19653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 20 client.19653 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeba4b670 0x559eeba4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 4 client.19653 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.775+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eeb92ed70 con 0x559eeba4b670 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeba4b670 0x559eeba4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:55.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc817fff640 10 client.19653 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc83cc2a640 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeba4b670 0x559eeba4da90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=419b3adf66006b5a server_cookie=47b7ea375805290f in_seq=0 out_seq=0 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc817fff640 1 -- 172.21.15.77:0/3436014460 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eeb92ed70 con 0x559eeba4b670 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc817fff640 10 client.19653 handle_command_reply: tid=0 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc841267740 1 client.19653 shutdown 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 msgr2=0x559eeb9d4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.776+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc83002a9a0 tx=0x7fc830046000 comp rx=0 tx=0).stop 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 shutdown_connections 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559eeba4b670 0x559eeba4da90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559eeb9d0780 0x559eeb9d4830 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559eeb9c2b40 0x559eeb9d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 --2- 172.21.15.77:0/3436014460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eeb9c15d0 0x559eeb9d8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 >> 172.21.15.77:0/3436014460 conn(0x559eeb98a310 msgr2=0x559eeb9bc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc817fff640 0 client.19653 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 shutdown_connections 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 1 -- 172.21.15.77:0/3436014460 wait complete. 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc841267740 20 client.19653 trim_cache size 0 max 16384 2024-09-15T22:29:55.796 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 msgr2=0x7fc8280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 0x7fc8280a5870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc830031550 tx=0x7fc830005d50 comp rx=0 tx=0).stop 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 msgr2=0x7fc8381989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.777+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc82400d750 tx=0x7fc82400dc10 comp rx=0 tx=0).stop 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 shutdown_connections 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc8280a33b0 0x7fc8280a5870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc83810c170 0x7fc83819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc838103660 0x7fc8381989a0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 --2- 172.21.15.77:0/148632540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc838102d40 0x7fc838198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:55.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 >> 172.21.15.77:0/148632540 conn(0x7fc8380fc7a0 msgr2=0x7fc8380fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:55.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 shutdown_connections 2024-09-15T22:29:55.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:55.778+0000 7fc83ee8e640 1 -- 172.21.15.77:0/148632540 wait complete. 2024-09-15T22:29:56.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 16.10 MB/sec execute 117 sec latency 20313.876 ms 2024-09-15T22:29:56.618 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:29:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:56.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:56 smithi077 ceph-mon[30980]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:56.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:56 smithi196 ceph-mon[35109]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:56.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:56 smithi117 ceph-mon[35021]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-15T22:29:57.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 -- 172.21.15.77:0/2416455677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 msgr2=0x7f0948105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 --2- 172.21.15.77:0/2416455677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f0948105eb0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f0940009ca0 tx=0x7f094002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 -- 172.21.15.77:0/2416455677 shutdown_connections 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 --2- 172.21.15.77:0/2416455677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f094810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 --2- 172.21.15.77:0/2416455677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 0x7f0948101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 --2- 172.21.15.77:0/2416455677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f0948105eb0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 -- 172.21.15.77:0/2416455677 >> 172.21.15.77:0/2416455677 conn(0x7f0948077210 msgr2=0x7f09480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.198+0000 7f094cf4e640 1 -- 172.21.15.77:0/2416455677 shutdown_connections 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.199+0000 7f094cf4e640 1 -- 172.21.15.77:0/2416455677 wait complete. 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.199+0000 7f094cf4e640 1 Processor -- start 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.199+0000 7f094cf4e640 1 -- start start 2024-09-15T22:29:57.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 0x7f0948110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f09481112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f094810fda0 con 0x7f0948105ad0 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f0945d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 0x7f0948110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f0945d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:52208/0 (socket says 172.21.15.77:52208) 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f0945d9b640 1 -- 172.21.15.77:0/2411164968 learned_addr learned my addr 172.21.15.77:0/2411164968 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f0946d9d640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f09481112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f094810fc20 con 0x7f0948101ba0 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.200+0000 7f094cf4e640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f094810ff20 con 0x7f0948101200 2024-09-15T22:29:57.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 -- 172.21.15.77:0/2411164968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 msgr2=0x7f0948110840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 0x7f0948110840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 -- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 msgr2=0x7f09481112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f09481112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09480728b0 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f0945d9b640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f093000d950 tx=0x7f093000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930017030 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.201+0000 7f094cf4e640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0948072ba0 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.202+0000 7f094cf4e640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0948073080 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.203+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930004020 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.203+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300047a0 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.205+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0948102210 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.205+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f09300049c0 con 0x7f0948101ba0 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.206+0000 7f093f7fe640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 0x7f09340a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.206+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f09300cfaa0 con 0x7f0948101ba0 2024-09-15T22:29:57.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.208+0000 7f094659c640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 0x7f09340a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.208+0000 7f094659c640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 0x7f09340a5a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0940006010 tx=0x7f0940009820 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.214+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f093009c090 con 0x7f0948101ba0 2024-09-15T22:29:57.263 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.438+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0948101660 con 0x7f0948101ba0 2024-09-15T22:29:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.440+0000 7f093f7fe640 1 -- 172.21.15.77:0/2411164968 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f093009b990 con 0x7f0948101ba0 2024-09-15T22:29:57.442 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:29:57.442 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:29:57.443 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 msgr2=0x7f09340a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 0x7f09340a5a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0940006010 tx=0x7f0940009820 comp rx=0 tx=0).stop 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 msgr2=0x7f0948110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f093000d950 tx=0x7f093000de10 comp rx=0 tx=0).stop 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 shutdown_connections 2024-09-15T22:29:57.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09340a3580 0x7f09340a5a40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0948105ad0 0x7f09481112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0948101ba0 0x7f0948110d80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 --2- 172.21.15.77:0/2411164968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0948101200 0x7f0948110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 >> 172.21.15.77:0/2411164968 conn(0x7f0948077210 msgr2=0x7f09481036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 shutdown_connections 2024-09-15T22:29:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.442+0000 7f093d7fa640 1 -- 172.21.15.77:0/2411164968 wait complete. 2024-09-15T22:29:57.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 15.96 MB/sec execute 118 sec latency 21313.913 ms 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.929+0000 7f9d6931b640 1 -- 172.21.15.77:0/1986854279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641036e0 msgr2=0x7f9d64107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.929+0000 7f9d6931b640 1 --2- 172.21.15.77:0/1986854279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641036e0 0x7f9d64107b90 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009930 tx=0x7f9d5802a420 comp rx=0 tx=0).stop 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 -- 172.21.15.77:0/1986854279 shutdown_connections 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 --2- 172.21.15.77:0/1986854279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641082a0 0x7f9d6410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 --2- 172.21.15.77:0/1986854279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641036e0 0x7f9d64107b90 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 --2- 172.21.15.77:0/1986854279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 0x7f9d64103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 -- 172.21.15.77:0/1986854279 >> 172.21.15.77:0/1986854279 conn(0x7f9d640fc7d0 msgr2=0x7f9d640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 -- 172.21.15.77:0/1986854279 shutdown_connections 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 -- 172.21.15.77:0/1986854279 wait complete. 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 Processor -- start 2024-09-15T22:29:57.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.930+0000 7f9d6931b640 1 -- start start 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 0x7f9d641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 0x7f9d64198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641120a0 con 0x7f9d641082a0 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64111f20 con 0x7f9d64102d30 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64112220 con 0x7f9d641036e0 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51878/0 (socket says 172.21.15.77:51878) 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 -- 172.21.15.77:0/797917904 learned_addr learned my addr 172.21.15.77:0/797917904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 0x7f9d641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d627fc640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 0x7f9d64198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 -- 172.21.15.77:0/797917904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 msgr2=0x7f9d64198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 0x7f9d64198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 -- 172.21.15.77:0/797917904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 msgr2=0x7f9d641985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 0x7f9d641985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 -- 172.21.15.77:0/797917904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6419d560 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d627fc640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 0x7f9d64198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d637fe640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d750 tx=0x7f9d5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d4bfff640 1 -- 172.21.15.77:0/797917904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c017070 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6419d850 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d4bfff640 1 -- 172.21.15.77:0/797917904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5c005500 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d4bfff640 1 -- 172.21.15.77:0/797917904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c004d90 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.931+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641a5130 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.933+0000 7f9d4bfff640 1 -- 172.21.15.77:0/797917904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9d5c004f30 con 0x7f9d641082a0 2024-09-15T22:29:57.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 Processor -- start 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 -- start start 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 0x557b58e51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b58daa2f0 con 0x557b58e445d0 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b58d51cf0 con 0x557b58e45b40 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b58da62b0 con 0x557b58e4f370 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d4bfff640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 0x7f9d4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51882/0 (socket says 172.21.15.77:51882) 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3498629708 learned_addr learned my addr 172.21.15.77:0/3498629708 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d637fe640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 0x7f9d4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d4bfff640 1 -- 172.21.15.77:0/797917904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f9d5c0cf9f0 con 0x7f9d641082a0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3498629708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 msgr2=0x557b58e51760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.934+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 0x557b58e51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 0x557b58e51760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3498629708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 msgr2=0x557b58e44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e44090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3498629708 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b58d363a0 con 0x557b58e445d0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 0x557b58e51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9d540039f0 tx=0x7f9d54003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c871a82619fd6bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d637fe640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 0x7f9d4c0a5870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c09b8d0 tx=0x7f9d5c086040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d497fa640 1 -- 172.21.15.77:0/3498629708 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004b20 con 0x557b58e445d0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d497fa640 1 -- 172.21.15.77:0/3498629708 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d54004cc0 con 0x557b58e445d0 2024-09-15T22:29:57.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.935+0000 7f9d497fa640 1 -- 172.21.15.77:0/3498629708 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540127d0 con 0x557b58e445d0 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3498629708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 msgr2=0x557b58e44a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9d540039f0 tx=0x7f9d54003eb0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3498629708 shutdown_connections 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e4f370 0x557b58e51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e445d0 0x557b58e44a10 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3498629708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3498629708 >> 172.21.15.77:0/3498629708 conn(0x557b58e0d310 msgr2=0x557b58e40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3498629708 shutdown_connections 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3498629708 wait complete. 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- start start 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 Processor -- start 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 0x557b58e4c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b58e52070 con 0x557b58e45b40 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b58d65060 con 0x557b58e4f370 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b58e521f0 con 0x557b58e445d0 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.936+0000 7f9d6b6f4740 10 client.0 _ll_register_callbacks cb 0x557b58ab88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51898/0 (socket says 172.21.15.77:51898) 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3194680549 learned_addr learned my addr 172.21.15.77:0/3194680549 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 0x557b58e4c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3194680549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 msgr2=0x557b58e4ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e4ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 msgr2=0x557b58e4c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 0x557b58e4c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 -- 172.21.15.77:0/3194680549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b58b1b3c0 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 0x557b58e4c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9d54004740 tx=0x7f9d54005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c851ceb0c46d6e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54012960 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f9d54012b00 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d54004cc0 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9d54012d40 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d6b6f4740 10 client.19671 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b58e5da30 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.937+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d54004cc0 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9d5400f040 con 0x557b58e45b40 2024-09-15T22:29:57.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 10 client.19671 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 10 client.19671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 20 client.19671 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58e5ede0 0x557b58e611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 4 client.19671 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b58db7860 con 0x557b58e5ede0 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58e5ede0 0x557b58e611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58e5ede0 0x557b58e611d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79351a83b5ecfabb server_cookie=8c675234f4f13de2 in_seq=0 out_seq=0 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.938+0000 7f9d46ff5640 10 client.19671 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d46ff5640 1 -- 172.21.15.77:0/3194680549 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b58db7860 con 0x557b58e5ede0 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d46ff5640 10 client.19671 handle_command_reply: tid=0 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 client.19671 shutdown 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 msgr2=0x557b58e4bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9d54004740 tx=0x7f9d54005fd0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 shutdown_connections 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58e5ede0 0x557b58e611d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e4f370 0x557b58e4c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e45b40 0x557b58e4bfc0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d46ff5640 0 client.19671 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3194680549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e4ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 >> 172.21.15.77:0/3194680549 conn(0x557b58e0d310 msgr2=0x557b58e46960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 shutdown_connections 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3194680549 wait complete. 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.939+0000 7f9d6b6f4740 20 client.19671 trim_cache size 0 max 16384 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 Processor -- start 2024-09-15T22:29:57.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 -- start start 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e5b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e40700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641af110 con 0x557b58e4f370 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0c50 con 0x557b58e45b40 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641af7e0 con 0x557b58e445d0 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.952+0000 7f9d627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51912/0 (socket says 172.21.15.77:51912) 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 -- 172.21.15.77:0/3156987307 learned_addr learned my addr 172.21.15.77:0/3156987307 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 -- 172.21.15.77:0/3156987307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 msgr2=0x557b58e5b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e5b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 -- 172.21.15.77:0/3156987307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 msgr2=0x557b58e40700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e40700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 -- 172.21.15.77:0/3156987307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b58e4c8d0 con 0x557b58e4f370 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d627fc640 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9d640697d0 tx=0x7f9d58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c45d87eea103fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d48ff9640 1 -- 172.21.15.77:0/3156987307 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58035040 con 0x557b58e4f370 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d48ff9640 1 -- 172.21.15.77:0/3156987307 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58049840 con 0x557b58e4f370 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d48ff9640 1 -- 172.21.15.77:0/3156987307 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580527d0 con 0x557b58e4f370 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.953+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3156987307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 msgr2=0x557b58e57860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9d640697d0 tx=0x7f9d58000bc0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3156987307 shutdown_connections 2024-09-15T22:29:57.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e57860 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e40700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/3156987307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e5b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3156987307 >> 172.21.15.77:0/3156987307 conn(0x557b58e0d310 msgr2=0x557b58e3fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3156987307 shutdown_connections 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/3156987307 wait complete. 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- start start 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 Processor -- start 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0580 con 0x557b58e4f370 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641afeb0 con 0x557b58e45b40 2024-09-15T22:29:57.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ad110 con 0x557b58e445d0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d6b6f4740 10 client.0 _ll_register_callbacks cb 0x7f9d54009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d627fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51920/0 (socket says 172.21.15.77:51920) 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d627fc640 1 -- 172.21.15.77:0/2060954529 learned_addr learned my addr 172.21.15.77:0/2060954529 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.954+0000 7f9d627fc640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 -- 172.21.15.77:0/2060954529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 msgr2=0x557b58e49420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e49420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 -- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 msgr2=0x557b58e49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 -- 172.21.15.77:0/2060954529 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b58e660e0 con 0x557b58e4f370 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d627fc640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9d58031520 tx=0x7f9d58005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d5e00d957d7cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:57.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5805d060 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f9d58052450 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d58049840 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9d58035030 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 10 client.19677 fetch_fsmap learned FSMap version 39 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b58e673e0 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58049840 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9d5803a040 con 0x557b58e4f370 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 10 client.19677 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 10 client.19677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 20 client.19677 populate_metadata read hostname 'smithi077' 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58ecec70 0x557b58ed1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:57.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 4 client.19677 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.955+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b58db1d70 con 0x557b58ecec70 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58ecec70 0x557b58ed1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d62ffd640 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58ecec70 0x557b58ed1090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=880dac39d3178c38 server_cookie=cf4d1ed963179c80 in_seq=0 out_seq=0 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d46ff5640 10 client.19677 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d46ff5640 1 -- 172.21.15.77:0/2060954529 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b58db1d70 con 0x557b58ecec70 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d46ff5640 10 client.19677 handle_command_reply: tid=0 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 client.19677 shutdown 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 msgr2=0x557b58e659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9d58031520 tx=0x7f9d58005b50 comp rx=0 tx=0).stop 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 shutdown_connections 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x557b58ecec70 0x557b58ed1090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b58e4f370 0x557b58e659f0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b58e45b40 0x557b58e49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 --2- 172.21.15.77:0/2060954529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b58e445d0 0x557b58e49420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 >> 172.21.15.77:0/2060954529 conn(0x557b58e0d310 msgr2=0x557b58e4a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d46ff5640 0 client.19677 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 shutdown_connections 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.956+0000 7f9d6b6f4740 1 -- 172.21.15.77:0/2060954529 wait complete. 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6b6f4740 20 client.19677 trim_cache size 0 max 16384 2024-09-15T22:29:57.977 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:29:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 msgr2=0x7f9d4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 0x7f9d4c0a5870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c09b8d0 tx=0x7f9d5c086040 comp rx=0 tx=0).stop 2024-09-15T22:29:57.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 msgr2=0x7f9d6419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d750 tx=0x7f9d5c00dc10 comp rx=0 tx=0).stop 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 shutdown_connections 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9d4c0a33b0 0x7f9d4c0a5870 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d641082a0 0x7f9d6419ce80 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d641036e0 0x7f9d64198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 --2- 172.21.15.77:0/797917904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d64102d30 0x7f9d641985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 >> 172.21.15.77:0/797917904 conn(0x7f9d640fc7d0 msgr2=0x7f9d64105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 shutdown_connections 2024-09-15T22:29:57.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:57.957+0000 7f9d6931b640 1 -- 172.21.15.77:0/797917904 wait complete. 2024-09-15T22:29:58.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 15.83 MB/sec execute 119 sec latency 22313.965 ms 2024-09-15T22:29:58.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:58 smithi077 ceph-mon[30980]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 5.3 MiB/s wr, 174 op/s 2024-09-15T22:29:58.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:29:58 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2411164968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:58.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:58 smithi196 ceph-mon[35109]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 5.3 MiB/s wr, 174 op/s 2024-09-15T22:29:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:29:58 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2411164968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:58 smithi117 ceph-mon[35021]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 5.3 MiB/s wr, 174 op/s 2024-09-15T22:29:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:29:58 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2411164968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:29:58.876 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:29:59.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 93278e8f --release --await' 2024-09-15T22:29:59.371 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 93278e8f --release --await 2024-09-15T22:29:59.457 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:59.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 15.70 MB/sec execute 120 sec latency 23313.995 ms 2024-09-15T22:30:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 -- 172.21.15.77:0/1573126867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8106f50 msgr2=0x7f41a810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 --2- 172.21.15.77:0/1573126867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8106f50 0x7f41a810eaf0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009cd0 tx=0x7f41a002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 -- 172.21.15.77:0/1573126867 shutdown_connections 2024-09-15T22:30:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 --2- 172.21.15.77:0/1573126867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8106f50 0x7f41a810eaf0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 --2- 172.21.15.77:0/1573126867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 0x7f41a8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 --2- 172.21.15.77:0/1573126867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8100350 0x7f41a8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 -- 172.21.15.77:0/1573126867 >> 172.21.15.77:0/1573126867 conn(0x7f41a8077210 msgr2=0x7f41a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 -- 172.21.15.77:0/1573126867 shutdown_connections 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.977+0000 7f41aff83640 1 -- 172.21.15.77:0/1573126867 wait complete. 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 Processor -- start 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 -- start start 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 0x7f41a8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f50 0x7f41a819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111fa0 con 0x7f41a8100350 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8111e20 con 0x7f41a8100d00 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41aff83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8112120 con 0x7f41a8106f50 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47206/0 (socket says 172.21.15.77:47206) 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 -- 172.21.15.77:0/3515742048 learned_addr learned my addr 172.21.15.77:0/3515742048 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:00.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41ad51e640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 0x7f41a8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 -- 172.21.15.77:0/3515742048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f50 msgr2=0x7f41a819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f50 0x7f41a819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 -- 172.21.15.77:0/3515742048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 msgr2=0x7f41a8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 0x7f41a8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.978+0000 7f41add1f640 1 -- 172.21.15.77:0/3515742048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a819d3b0 con 0x7f41a8100350 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f41add1f640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c920 tx=0x7f41a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f419effd640 1 -- 172.21.15.77:0/3515742048 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4007c70 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f419effd640 1 -- 172.21.15.77:0/3515742048 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4007e10 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a819d6a0 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f419effd640 1 -- 172.21.15.77:0/3515742048 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4018890 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.979+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a81a4f80 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.981+0000 7f419effd640 1 -- 172.21.15.77:0/3515742048 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f41a4018a70 con 0x7f41a8100350 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.981+0000 7f41b235c740 1 Processor -- start 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.981+0000 7f41b235c740 1 -- start start 2024-09-15T22:30:00.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f07b40 0x55f507f06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f06a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f507e6c2f0 con 0x55f507f11370 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f507e13cf0 con 0x55f507f065d0 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41b235c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f507e682b0 con 0x55f507f07b40 2024-09-15T22:30:00.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f419effd640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41add1f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f06a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47210/0 (socket says 172.21.15.77:47210) 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 -- 172.21.15.77:0/829421380 learned_addr learned my addr 172.21.15.77:0/829421380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ae520640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ae520640 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f07b40 0x55f507f06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f419effd640 1 -- 172.21.15.77:0/3515742048 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f41a40d02b0 con 0x7f41a8100350 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 -- 172.21.15.77:0/829421380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f07b40 msgr2=0x55f507f06090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f07b40 0x55f507f06090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 -- 172.21.15.77:0/829421380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 msgr2=0x55f507f06a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f06a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 -- 172.21.15.77:0/829421380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f507df83a0 con 0x55f507f11370 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ae520640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 0x7f41940a5870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f41a0031860 tx=0x7f41a00093a0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.982+0000 7f41ad51e640 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f419800cb90 tx=0x7f419800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1229769df7af77f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f4193fff640 1 -- 172.21.15.77:0/829421380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198005220 con 0x55f507f11370 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f4193fff640 1 -- 172.21.15.77:0/829421380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198004c80 con 0x55f507f11370 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f4193fff640 1 -- 172.21.15.77:0/829421380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198010890 con 0x55f507f11370 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 -- 172.21.15.77:0/829421380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 msgr2=0x55f507f13760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f419800cb90 tx=0x7f419800e720 comp rx=0 tx=0).stop 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 -- 172.21.15.77:0/829421380 shutdown_connections 2024-09-15T22:30:00.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f13760 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f06a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 --2- 172.21.15.77:0/829421380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f07b40 0x55f507f06090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 -- 172.21.15.77:0/829421380 >> 172.21.15.77:0/829421380 conn(0x55f507ecf310 msgr2=0x55f507f02c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 -- 172.21.15.77:0/829421380 shutdown_connections 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.983+0000 7f41b235c740 1 -- 172.21.15.77:0/829421380 wait complete. 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 -- start start 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 Processor -- start 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f507f14070 con 0x55f507f07b40 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f507e27060 con 0x55f507f065d0 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f507f141f0 con 0x55f507f11370 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41b235c740 10 client.0 _ll_register_callbacks cb 0x55f507b7a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47212/0 (socket says 172.21.15.77:47212) 2024-09-15T22:30:00.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.984+0000 7f41ad51e640 1 -- 172.21.15.77:0/3251925624 learned_addr learned my addr 172.21.15.77:0/3251925624 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ae520640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 -- 172.21.15.77:0/3251925624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 msgr2=0x55f507f0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 -- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 msgr2=0x55f507f10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41add1f640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f10c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 -- 172.21.15.77:0/3251925624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f507bdd3c0 con 0x55f507f07b40 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ae520640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41add1f640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41ad51e640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f419800ec20 tx=0x7f419800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c1f097d258804 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41917fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419800ce90 con 0x55f507f07b40 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f41980048d0 con 0x55f507f07b40 2024-09-15T22:30:00.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4198004c80 con 0x55f507f07b40 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f419802f050 con 0x55f507f07b40 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41b235c740 10 client.19689 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f507f1fa60 con 0x55f507f07b40 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.985+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198004c80 con 0x55f507f07b40 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4198003aa0 con 0x55f507f07b40 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 10 client.19689 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 10 client.19689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 20 client.19689 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:00.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f20e60 0x55f507f23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 4 client.19689 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f507e79860 con 0x55f507f20e60 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41add1f640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f20e60 0x55f507f23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41917fa640 10 client.19689 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.986+0000 7f41add1f640 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f20e60 0x55f507f23250 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5faa5a87d5e931 server_cookie=b684dbc3bedf9b11 in_seq=0 out_seq=0 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41917fa640 1 -- 172.21.15.77:0/3251925624 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f507e79860 con 0x55f507f20e60 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41917fa640 10 client.19689 handle_command_reply: tid=0 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 client.19689 shutdown 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 msgr2=0x55f507f0df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f419800ec20 tx=0x7f419800c8d0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 shutdown_connections 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f20e60 0x55f507f23250 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f0df60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41917fa640 0 client.19689 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 --2- 172.21.15.77:0/3251925624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 >> 172.21.15.77:0/3251925624 conn(0x55f507ecf310 msgr2=0x55f507f08960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 shutdown_connections 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 1 -- 172.21.15.77:0/3251925624 wait complete. 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:29:59.987+0000 7f41b235c740 20 client.19689 trim_cache size 0 max 16384 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 Processor -- start 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 -- start start 2024-09-15T22:30:00.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 0x55f507f1d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f07b40 0x55f507f288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81aef00 con 0x55f507f11370 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81b0a40 con 0x55f507f07b40 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41b235c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81af5d0 con 0x55f507f065d0 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41ae520640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41add1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 0x55f507f1d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f07b40 0x55f507f288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41add1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 0x55f507f1d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60252/0 (socket says 172.21.15.77:60252) 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41ae520640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47218/0 (socket says 172.21.15.77:47218) 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.000+0000 7f41add1f640 1 -- 172.21.15.77:0/3241805281 learned_addr learned my addr 172.21.15.77:0/3241805281 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:00.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 -- 172.21.15.77:0/3241805281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 msgr2=0x55f507f1d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 0x55f507f1d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 -- 172.21.15.77:0/3241805281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f07b40 msgr2=0x55f507f288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f07b40 0x55f507f288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 -- 172.21.15.77:0/3241805281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f507f06d50 con 0x55f507f11370 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41ae520640 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f41a002ae00 tx=0x7f41a0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235f659c8a931c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41937fe640 1 -- 172.21.15.77:0/3241805281 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0049620 con 0x55f507f11370 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41937fe640 1 -- 172.21.15.77:0/3241805281 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0036050 con 0x55f507f11370 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.001+0000 7f41937fe640 1 -- 172.21.15.77:0/3241805281 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0053700 con 0x55f507f11370 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- 172.21.15.77:0/3241805281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 msgr2=0x55f507f28df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f41a002ae00 tx=0x7f41a0041000 comp rx=0 tx=0).stop 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- 172.21.15.77:0/3241805281 shutdown_connections 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f11370 0x55f507f28df0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f07b40 0x55f507f288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 --2- 172.21.15.77:0/3241805281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f065d0 0x55f507f1d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- 172.21.15.77:0/3241805281 >> 172.21.15.77:0/3241805281 conn(0x55f507ecf310 msgr2=0x55f507f0c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- 172.21.15.77:0/3241805281 shutdown_connections 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- 172.21.15.77:0/3241805281 wait complete. 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 -- start start 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.002+0000 7f41b235c740 1 Processor -- start 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f19d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f15810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81b0370 con 0x55f507f07b40 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81afca0 con 0x55f507f065d0 2024-09-15T22:30:00.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81acf00 con 0x55f507f11370 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 10 client.0 _ll_register_callbacks cb 0x7f4198014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47224/0 (socket says 172.21.15.77:47224) 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 -- 172.21.15.77:0/1094111556 learned_addr learned my addr 172.21.15.77:0/1094111556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41add1f640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f19d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ae520640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 -- 172.21.15.77:0/1094111556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 msgr2=0x55f507f15810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f15810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 -- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 msgr2=0x55f507f19d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f19d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 -- 172.21.15.77:0/1094111556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f507f8a880 con 0x55f507f07b40 2024-09-15T22:30:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41ad51e640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x55f507f0f510 tx=0x7f419800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93356ec6e5270f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41917fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41980048d0 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f4198021040 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a0036050 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4198043050 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 10 client.19695 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.003+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f507f8ba10 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0036050 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4198003850 con 0x55f507f07b40 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 10 client.19695 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 20 client.19695 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f8c0d0 0x55f507f8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 4 client.19695 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f507e73d70 con 0x55f507f8c0d0 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.004+0000 7f41add1f640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f8c0d0 0x55f507f8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41add1f640 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f8c0d0 0x55f507f8e4f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dde5ece2cccdc8c server_cookie=f58087e5bcb5c872 in_seq=0 out_seq=0 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41917fa640 10 client.19695 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41917fa640 1 -- 172.21.15.77:0/1094111556 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f507e73d70 con 0x55f507f8c0d0 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41917fa640 10 client.19695 handle_command_reply: tid=0 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41b235c740 1 client.19695 shutdown 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 msgr2=0x55f507f15080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.005+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x55f507f0f510 tx=0x7f419800cf10 comp rx=0 tx=0).stop 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 shutdown_connections 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f507f8c0d0 0x55f507f8e4f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f507f11370 0x55f507f15810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f507f07b40 0x55f507f15080 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41917fa640 0 client.19695 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 --2- 172.21.15.77:0/1094111556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f507f065d0 0x55f507f19d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 >> 172.21.15.77:0/1094111556 conn(0x55f507ecf310 msgr2=0x55f507f0f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 shutdown_connections 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 1 -- 172.21.15.77:0/1094111556 wait complete. 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41b235c740 20 client.19695 trim_cache size 0 max 16384 2024-09-15T22:30:00.031 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 msgr2=0x7f41940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 0x7f41940a5870 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f41a0031860 tx=0x7f41a00093a0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 msgr2=0x7f41a8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c920 tx=0x7f41a400cde0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 shutdown_connections 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41a8106f50 0x7f41a819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41a8100d00 0x7f41a8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 --2- 172.21.15.77:0/3515742048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41a8100350 0x7f41a8198400 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.006+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 >> 172.21.15.77:0/3515742048 conn(0x7f41a8077210 msgr2=0x7f41a8105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.007+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 shutdown_connections 2024-09-15T22:30:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:00.007+0000 7f41aff83640 1 -- 172.21.15.77:0/3515742048 wait complete. 2024-09-15T22:30:00.227 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:00.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 15.57 MB/sec execute 121 sec latency 24314.030 ms 2024-09-15T22:30:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:00 smithi077 ceph-mon[30980]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 122 op/s 2024-09-15T22:30:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:00 smithi077 ceph-mon[30980]: overall HEALTH_OK 2024-09-15T22:30:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:00 smithi196 ceph-mon[35109]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 122 op/s 2024-09-15T22:30:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:00 smithi196 ceph-mon[35109]: overall HEALTH_OK 2024-09-15T22:30:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:00 smithi117 ceph-mon[35021]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 122 op/s 2024-09-15T22:30:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:00 smithi117 ceph-mon[35021]: overall HEALTH_OK 2024-09-15T22:30:01.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 78918 15.44 MB/sec execute 122 sec latency 25314.152 ms 2024-09-15T22:30:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- 172.21.15.77:0/1173003849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148101420 msgr2=0x7f614810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 --2- 172.21.15.77:0/1173003849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148101420 0x7f614810d950 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6140009cd0 tx=0x7f614002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- 172.21.15.77:0/1173003849 shutdown_connections 2024-09-15T22:30:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 --2- 172.21.15.77:0/1173003849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148101420 0x7f614810d950 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 --2- 172.21.15.77:0/1173003849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 --2- 172.21.15.77:0/1173003849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148106a80 0x7f6148106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- 172.21.15.77:0/1173003849 >> 172.21.15.77:0/1173003849 conn(0x7f61480fc7a0 msgr2=0x7f61480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- 172.21.15.77:0/1173003849 shutdown_connections 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- 172.21.15.77:0/1173003849 wait complete. 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 Processor -- start 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.095+0000 7f614f26a640 1 -- start start 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148101420 0x7f61481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6148110e80 con 0x7f6148106a80 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6148110d00 con 0x7f6148100a80 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6148111000 con 0x7f6148101420 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47242/0 (socket says 172.21.15.77:47242) 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 -- 172.21.15.77:0/2083965065 learned_addr learned my addr 172.21.15.77:0/2083965065 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614c805640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148101420 0x7f61481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d006640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 -- 172.21.15.77:0/2083965065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148101420 msgr2=0x7f61481997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148101420 0x7f61481997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 -- 172.21.15.77:0/2083965065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 msgr2=0x7f6148199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 -- 172.21.15.77:0/2083965065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61481a4850 con 0x7f6148106a80 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d006640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614d807640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6140005cc0 tx=0x7f6140003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f613e7fc640 1 -- 172.21.15.77:0/2083965065 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140004040 con 0x7f6148106a80 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f613e7fc640 1 -- 172.21.15.77:0/2083965065 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61400046c0 con 0x7f6148106a80 2024-09-15T22:30:02.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61481a4a20 con 0x7f6148106a80 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f613e7fc640 1 -- 172.21.15.77:0/2083965065 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140046480 con 0x7f6148106a80 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.096+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61481a4ed0 con 0x7f6148106a80 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.098+0000 7f613e7fc640 1 -- 172.21.15.77:0/2083965065 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61400305a0 con 0x7f6148106a80 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 Processor -- start 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 -- start start 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f463c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f464da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3f45a1620 con 0x55f3f464b650 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3f4548e50 con 0x55f3f463bf40 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f6151643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3f459d5e0 con 0x55f3f463c860 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f613e7fc640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 0x7f61340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.099+0000 7f614d807640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 0x7f61340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614d006640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f464da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614d006640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60298/0 (socket says 172.21.15.77:60298) 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614d006640 1 -- 172.21.15.77:0/2288541565 learned_addr learned my addr 172.21.15.77:0/2288541565 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f613e7fc640 1 -- 172.21.15.77:0/2083965065 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f614002e030 con 0x7f6148106a80 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f463c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 -- 172.21.15.77:0/2288541565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 msgr2=0x55f3f464b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 -- 172.21.15.77:0/2288541565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 msgr2=0x55f3f463c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f463c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 -- 172.21.15.77:0/2288541565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f4539d40 con 0x55f3f464b650 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614d006640 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:02.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614d807640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 0x7f61340a5870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f61400eef50 tx=0x7f61400095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f614c805640 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f464da40 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f61380058c0 tx=0x7f613800a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22141670214a72a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f61337fe640 1 -- 172.21.15.77:0/2288541565 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138004220 con 0x55f3f464b650 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f61337fe640 1 -- 172.21.15.77:0/2288541565 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61380043c0 con 0x55f3f464b650 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.100+0000 7f61337fe640 1 -- 172.21.15.77:0/2288541565 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613800f8a0 con 0x55f3f464b650 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- 172.21.15.77:0/2288541565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 msgr2=0x55f3f464da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f464da40 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f61380058c0 tx=0x7f613800a6f0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- 172.21.15.77:0/2288541565 shutdown_connections 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f464da40 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 --2- 172.21.15.77:0/2288541565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f463c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- 172.21.15.77:0/2288541565 >> 172.21.15.77:0/2288541565 conn(0x55f3f46385f0 msgr2=0x55f3f46389d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- 172.21.15.77:0/2288541565 shutdown_connections 2024-09-15T22:30:02.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- 172.21.15.77:0/2288541565 wait complete. 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.101+0000 7f6151643740 1 -- start start 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 Processor -- start 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f4648520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f464b650 0x55f3f4648fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3f464e220 con 0x55f3f463c860 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3f4529f10 con 0x55f3f463bf40 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3f464e3a0 con 0x55f3f464b650 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f6151643740 10 client.0 _ll_register_callbacks cb 0x55f3f454ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47262/0 (socket says 172.21.15.77:47262) 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 -- 172.21.15.77:0/48312716 learned_addr learned my addr 172.21.15.77:0/48312716 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:02.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614c805640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f464b650 0x55f3f4648fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614c805640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f4648520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 -- 172.21.15.77:0/48312716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f464b650 msgr2=0x55f3f4648fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f464b650 0x55f3f4648fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 -- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 msgr2=0x55f3f4648520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f4648520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614d006640 1 -- 172.21.15.77:0/48312716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f42b0580 con 0x55f3f463c860 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.102+0000 7f614c805640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f4648520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f614d006640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6144012450 tx=0x7f6144012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fabb78206a456f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144005220 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f61440053c0 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61380043c0 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6144025050 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 10 client.19707 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3f437a960 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61380043c0 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6144019030 con 0x55f3f463c860 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 10 client.19707 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:02.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 10 client.19707 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 20 client.19707 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f465b330 0x55f3f465d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 4 client.19707 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.103+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3f45aeb90 con 0x55f3f465b330 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.104+0000 7f614c805640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f465b330 0x55f3f465d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.104+0000 7f614c805640 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f465b330 0x55f3f465d7d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185d5e9c5267c962 server_cookie=e14d817149a7f6e7 in_seq=0 out_seq=0 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.104+0000 7f6130ff9640 10 client.19707 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6130ff9640 1 -- 172.21.15.77:0/48312716 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3f45aeb90 con 0x55f3f465b330 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6130ff9640 10 client.19707 handle_command_reply: tid=0 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 client.19707 shutdown 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 msgr2=0x55f3f4648a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6144012450 tx=0x7f6144012910 comp rx=0 tx=0).stop 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 shutdown_connections 2024-09-15T22:30:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f465b330 0x55f3f465d7d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f464b650 0x55f3f4648fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f463c860 0x55f3f4648a60 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6130ff9640 0 client.19707 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 --2- 172.21.15.77:0/48312716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f4648520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 >> 172.21.15.77:0/48312716 conn(0x55f3f46385f0 msgr2=0x55f3f463e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 shutdown_connections 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 1 -- 172.21.15.77:0/48312716 wait complete. 2024-09-15T22:30:02.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.105+0000 7f6151643740 20 client.19707 trim_cache size 0 max 16384 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 Processor -- start 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 -- start start 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f46332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f4637640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61481acc80 con 0x55f3f464b650 2024-09-15T22:30:02.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ae7c0 con 0x55f3f463bf40 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f6151643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ad350 con 0x55f3f463c860 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47264/0 (socket says 172.21.15.77:47264) 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 -- 172.21.15.77:0/1633854150 learned_addr learned my addr 172.21.15.77:0/1633854150 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614d006640 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f4637640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 -- 172.21.15.77:0/1633854150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 msgr2=0x55f3f4637640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f4637640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 -- 172.21.15.77:0/1633854150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 msgr2=0x55f3f46332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f46332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.122+0000 7f614c805640 1 -- 172.21.15.77:0/1633854150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f464a460 con 0x55f3f464b650 2024-09-15T22:30:02.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f614c805640 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6138004610 tx=0x7f613800ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f2b944acfa52b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6132ffd640 1 -- 172.21.15.77:0/1633854150 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138022030 con 0x55f3f464b650 2024-09-15T22:30:02.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6132ffd640 1 -- 172.21.15.77:0/1633854150 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613800fb20 con 0x55f3f464b650 2024-09-15T22:30:02.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6132ffd640 1 -- 172.21.15.77:0/1633854150 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138026040 con 0x55f3f464b650 2024-09-15T22:30:02.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 -- 172.21.15.77:0/1633854150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 msgr2=0x55f3f46630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6138004610 tx=0x7f613800ae30 comp rx=0 tx=0).stop 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 -- 172.21.15.77:0/1633854150 shutdown_connections 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46630a0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f4637640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 --2- 172.21.15.77:0/1633854150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f46332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.123+0000 7f6151643740 1 -- 172.21.15.77:0/1633854150 >> 172.21.15.77:0/1633854150 conn(0x55f3f46385f0 msgr2=0x55f3f4636f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- 172.21.15.77:0/1633854150 shutdown_connections 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- 172.21.15.77:0/1633854150 wait complete. 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- start start 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 Processor -- start 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f464f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ae0f0 con 0x55f3f464b650 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ada20 con 0x55f3f463bf40 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61481aac80 con 0x55f3f463c860 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f6151643740 10 client.0 _ll_register_callbacks cb 0x7f614400baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f614c805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.124+0000 7f614c805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47272/0 (socket says 172.21.15.77:47272) 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 -- 172.21.15.77:0/4121518272 learned_addr learned my addr 172.21.15.77:0/4121518272 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614d006640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f464f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 -- 172.21.15.77:0/4121518272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 msgr2=0x55f3f464fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 -- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 msgr2=0x55f3f464f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:02.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f464f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 -- 172.21.15.77:0/4121518272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f46c5360 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f614c805640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f61380058c0 tx=0x7f613802e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000987c2e343c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138022070 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f6138011050 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f613800fb20 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6138011250 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6151643740 10 client.19713 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3f46c6620 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.125+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f613800fb20 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f613801b030 con 0x55f3f464b650 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 10 client.19713 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:02.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 10 client.19713 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 20 client.19713 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f46c6ae0 0x55f3f46c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 4 client.19713 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93278e8f", "release": true, "await": true}] 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93278e8f", "release": true, "await": true}) -- 0x55f3f45a90a0 con 0x55f3f46c6ae0 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f614d006640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f46c6ae0 0x55f3f46c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f614d006640 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f46c6ae0 0x55f3f46c8ed0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca546743ad5d3c server_cookie=f3a14992754907e3 in_seq=0 out_seq=0 2024-09-15T22:30:02.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.126+0000 7f6130ff9640 10 client.19713 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6130ff9640 1 -- 172.21.15.77:0/4121518272 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f3f45a90a0 con 0x55f3f46c6ae0 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6130ff9640 10 client.19713 handle_command_reply: tid=0 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 client.19713 shutdown 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 msgr2=0x55f3f46500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f61380058c0 tx=0x7f613802e4c0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 shutdown_connections 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3f46c6ae0 0x55f3f46c8ed0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3f464b650 0x55f3f46500f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6130ff9640 0 client.19713 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3f463c860 0x55f3f464fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 --2- 172.21.15.77:0/4121518272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3f463bf40 0x55f3f464f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 >> 172.21.15.77:0/4121518272 conn(0x55f3f46385f0 msgr2=0x55f3f4648ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 shutdown_connections 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.195+0000 7f6151643740 1 -- 172.21.15.77:0/4121518272 wait complete. 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.196+0000 7f6151643740 20 client.19713 trim_cache size 0 max 16384 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 64, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "93278e8f": { 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 64, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.9, 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:30:02.200 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.196+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 msgr2=0x7f61340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.196+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 0x7f61340a5870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f61400eef50 tx=0x7f61400095c0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.196+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 msgr2=0x7f61481a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.196+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6140005cc0 tx=0x7f6140003970 comp rx=0 tx=0).stop 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 shutdown_connections 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61340a33b0 0x7f61340a5870 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6148106a80 0x7f61481a4310 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6148101420 0x7f61481997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 --2- 172.21.15.77:0/2083965065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6148100a80 0x7f6148199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 >> 172.21.15.77:0/2083965065 conn(0x7f61480fc7a0 msgr2=0x7f61480fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 shutdown_connections 2024-09-15T22:30:02.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:02.197+0000 7f614f26a640 1 -- 172.21.15.77:0/2083965065 wait complete. 2024-09-15T22:30:02.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:30:02.248 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:30:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:02 smithi077 ceph-mon[30980]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 6.0 MiB/s wr, 220 op/s 2024-09-15T22:30:02.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 79085 15.40 MB/sec execute 123 sec latency 25904.393 ms 2024-09-15T22:30:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:02 smithi196 ceph-mon[35109]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 6.0 MiB/s wr, 220 op/s 2024-09-15T22:30:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:02 smithi117 ceph-mon[35021]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 6.0 MiB/s wr, 220 op/s 2024-09-15T22:30:03.162 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:03.170 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93278e8f', total seconds elapsed: 31.3 2024-09-15T22:30:03.171 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.95 2024-09-15T22:30:03.171 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:30:03.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 79735 15.62 MB/sec execute 124 sec latency 207.330 ms 2024-09-15T22:30:03.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.711+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/1998125142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 msgr2=0x7fd79c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.712+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/1998125142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c10ec30 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd794009cd0 tx=0x7fd79402a5b0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.712+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/1998125142 shutdown_connections 2024-09-15T22:30:03.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.712+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/1998125142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c10ec30 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.712+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/1998125142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c103740 0x7fd79c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.713+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/1998125142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c102d90 0x7fd79c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.713+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/1998125142 >> 172.21.15.77:0/1998125142 conn(0x7fd79c0fc7d0 msgr2=0x7fd79c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.713+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/1998125142 shutdown_connections 2024-09-15T22:30:03.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.713+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/1998125142 wait complete. 2024-09-15T22:30:03.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.714+0000 7fd7a1aa4640 1 Processor -- start 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.714+0000 7fd7a1aa4640 1 -- start start 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.714+0000 7fd7a1aa4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c102d90 0x7fd79c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.714+0000 7fd7a1aa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.714+0000 7fd7a1aa4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd7a1aa4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c10fef0 con 0x7fd79c108290 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd7a1aa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c10fd70 con 0x7fd79c103740 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd7a1aa4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c110070 con 0x7fd79c102d90 2024-09-15T22:30:03.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd79affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd79affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60780/0 (socket says 172.21.15.77:60780) 2024-09-15T22:30:03.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd79affd640 1 -- 172.21.15.77:0/848788569 learned_addr learned my addr 172.21.15.77:0/848788569 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:03.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c102d90 0x7fd79c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.715+0000 7fd79bfff640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 -- 172.21.15.77:0/848788569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c102d90 msgr2=0x7fd79c196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c102d90 0x7fd79c196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 -- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 msgr2=0x7fd79c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 -- 172.21.15.77:0/848788569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c19b360 con 0x7fd79c103740 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.716+0000 7fd79affd640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd790009fd0 tx=0x7fd79000e800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.718+0000 7fd798ff9640 1 -- 172.21.15.77:0/848788569 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900134e0 con 0x7fd79c103740 2024-09-15T22:30:03.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.718+0000 7fd798ff9640 1 -- 172.21.15.77:0/848788569 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd790005500 con 0x7fd79c103740 2024-09-15T22:30:03.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.718+0000 7fd798ff9640 1 -- 172.21.15.77:0/848788569 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790004d70 con 0x7fd79c103740 2024-09-15T22:30:03.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.718+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c19b650 con 0x7fd79c103740 2024-09-15T22:30:03.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.718+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1b3f20 con 0x7fd79c103740 2024-09-15T22:30:03.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 Processor -- start 2024-09-15T22:30:03.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 -- start start 2024-09-15T22:30:03.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd798ff9640 1 -- 172.21.15.77:0/848788569 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd790004f10 con 0x7fd79c103740 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559c9e631280 con 0x559c9e6d6d40 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c9e5d8b10 con 0x559c9e6cc480 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559c9e62d240 con 0x559c9e6cbb60 2024-09-15T22:30:03.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd79affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60356/0 (socket says 172.21.15.77:60356) 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd79bfff640 1 -- 172.21.15.77:0/2917595563 learned_addr learned my addr 172.21.15.77:0/2917595563 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.723+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2917595563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 msgr2=0x559c9e6cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2917595563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 msgr2=0x559c9e6d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6d9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2917595563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9e5e51f0 con 0x559c9e6cc480 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d6800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009c20 tx=0x7fd78c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f347fdfe29dff01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd798ff9640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 0x7fd7840a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd781ffb640 1 -- 172.21.15.77:0/2917595563 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c009dd0 con 0x559c9e6cc480 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd781ffb640 1 -- 172.21.15.77:0/2917595563 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c005020 con 0x559c9e6cc480 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd781ffb640 1 -- 172.21.15.77:0/2917595563 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c004c40 con 0x559c9e6cc480 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.724+0000 7fd798ff9640 1 -- 172.21.15.77:0/848788569 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fd7900d0330 con 0x7fd79c103740 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.725+0000 7fd79bfff640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 0x7fd7840a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.725+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2917595563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 msgr2=0x559c9e6d6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.725+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d6800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009c20 tx=0x7fd78c011810 comp rx=0 tx=0).stop 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2917595563 shutdown_connections 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d6800 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2917595563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2917595563 >> 172.21.15.77:0/2917595563 conn(0x559c9e6c80f0 msgr2=0x559c9e6c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd79bfff640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 0x7fd7840a5970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd794009cd0 tx=0x7fd794030000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2917595563 shutdown_connections 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2917595563 wait complete. 2024-09-15T22:30:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.726+0000 7fd7a3e7d740 1 -- start start 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.727+0000 7fd7a3e7d740 1 Processor -- start 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.727+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 0x559c9e6d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cc480 0x559c9e6d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 0x559c9e6d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 0x559c9e6d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 0x559c9e6d3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47302/0 (socket says 172.21.15.77:47302) 2024-09-15T22:30:03.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559c9e6d9910 con 0x559c9e6cbb60 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c9e5ebf10 con 0x559c9e6d6d40 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559c9e6d9a90 con 0x559c9e6cc480 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79b7fe640 1 -- 172.21.15.77:0/4282090897 learned_addr learned my addr 172.21.15.77:0/4282090897 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd7a3e7d740 10 client.0 _ll_register_callbacks cb 0x559c9e5b4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 0x559c9e6d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79affd640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cc480 0x559c9e6d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 -- 172.21.15.77:0/4282090897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cc480 msgr2=0x559c9e6d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cc480 0x559c9e6d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 -- 172.21.15.77:0/4282090897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 msgr2=0x559c9e6d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 0x559c9e6d3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.728+0000 7fd79bfff640 1 -- 172.21.15.77:0/4282090897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9e3a26a0 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd79bfff640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 0x559c9e6d4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd794005e30 tx=0x7fd79402a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73df6c1d52ea1821 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd77affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940316d0 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c005020 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fd794031870 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd794031a70 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd7a3e7d740 10 client.28523 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.729+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c9e6e5500 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c005020 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd794051070 con 0x559c9e6d6d40 2024-09-15T22:30:03.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 10 client.28523 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 10 client.28523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 20 client.28523 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e6e6910 0x559c9e6e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 4 client.28523 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:03.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c9e63e7f0 con 0x559c9e6e6910 2024-09-15T22:30:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.730+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e6e6910 0x559c9e6e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.731+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e6e6910 0x559c9e6e8d40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620ef1b67e03ef15 server_cookie=34e8129f0c9d7887 in_seq=0 out_seq=0 2024-09-15T22:30:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.731+0000 7fd77affd640 10 client.28523 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.732+0000 7fd77affd640 1 -- 172.21.15.77:0/4282090897 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c9e63e7f0 con 0x559c9e6e6910 2024-09-15T22:30:03.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.732+0000 7fd77affd640 10 client.28523 handle_command_reply: tid=0 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.732+0000 7fd7a3e7d740 1 client.28523 shutdown 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 msgr2=0x559c9e6d4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 0x559c9e6d4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd794005e30 tx=0x7fd79402a5b0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 shutdown_connections 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e6e6910 0x559c9e6e8d40 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6d6d40 0x559c9e6d4610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cc480 0x559c9e6d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/4282090897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cbb60 0x559c9e6d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 >> 172.21.15.77:0/4282090897 conn(0x559c9e6c80f0 msgr2=0x559c9e6cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd77affd640 0 client.28523 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 shutdown_connections 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/4282090897 wait complete. 2024-09-15T22:30:03.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.733+0000 7fd7a3e7d740 20 client.28523 trim_cache size 0 max 16384 2024-09-15T22:30:03.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.766+0000 7fd7a3e7d740 1 Processor -- start 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.766+0000 7fd7a3e7d740 1 -- start start 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60364/0 (socket says 172.21.15.77:60364) 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2027623888 learned_addr learned my addr 172.21.15.77:0/2027623888 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:03.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1bdf30 con 0x559c9e6d6d40 2024-09-15T22:30:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1bfa70 con 0x559c9e6cc480 2024-09-15T22:30:03.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1be600 con 0x559c9e6cbb60 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.767+0000 7fd79bfff640 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79affd640 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2027623888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 msgr2=0x559c9e6d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2027623888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 msgr2=0x559c9e6ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 -- 172.21.15.77:0/2027623888 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9e6c7f20 con 0x559c9e6cbb60 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.768+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0057c0 tx=0x7fd78c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5152e7b3ee89700c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.769+0000 7fd7817fa640 1 -- 172.21.15.77:0/2027623888 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c031070 con 0x559c9e6cbb60 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.769+0000 7fd7817fa640 1 -- 172.21.15.77:0/2027623888 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004520 con 0x559c9e6cbb60 2024-09-15T22:30:03.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.769+0000 7fd7817fa640 1 -- 172.21.15.77:0/2027623888 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c004f10 con 0x559c9e6cbb60 2024-09-15T22:30:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.769+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 msgr2=0x559c9e6ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0057c0 tx=0x7fd78c005f90 comp rx=0 tx=0).stop 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 shutdown_connections 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6d6d40 0x559c9e6ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cc480 0x559c9e6d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/2027623888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6cbb60 0x559c9e6ce5c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 >> 172.21.15.77:0/2027623888 conn(0x559c9e6c80f0 msgr2=0x559c9e6d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 shutdown_connections 2024-09-15T22:30:03.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/2027623888 wait complete. 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.770+0000 7fd7a3e7d740 1 -- start start 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 Processor -- start 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cc480 0x559c9e6d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6d6d40 0x559c9e6d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1bf3a0 con 0x559c9e6cc480 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1becd0 con 0x559c9e6cbb60 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1bbf30 con 0x559c9e6d6d40 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd79affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cc480 0x559c9e6d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd7a3e7d740 10 client.0 _ll_register_callbacks cb 0x7fd790005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60830/0 (socket says 172.21.15.77:60830) 2024-09-15T22:30:03.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd79b7fe640 1 -- 172.21.15.77:0/847627963 learned_addr learned my addr 172.21.15.77:0/847627963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.771+0000 7fd79bfff640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6d6d40 0x559c9e6d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 -- 172.21.15.77:0/847627963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6d6d40 msgr2=0x559c9e6d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6d6d40 0x559c9e6d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 -- 172.21.15.77:0/847627963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cc480 msgr2=0x559c9e6d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cc480 0x559c9e6d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 -- 172.21.15.77:0/847627963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9e6c3830 con 0x559c9e6cbb60 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd79b7fe640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0042f0 tx=0x7fd78c004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6a84e6aed073cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd77affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c03a060 con 0x559c9e6cbb60 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7fd78c004da0 con 0x559c9e6cbb60 2024-09-15T22:30:03.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c004520 con 0x559c9e6cbb60 2024-09-15T22:30:03.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd78c031070 con 0x559c9e6cbb60 2024-09-15T22:30:03.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd7a3e7d740 10 client.28529 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.772+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559c9e7516f0 con 0x559c9e6cbb60 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004520 con 0x559c9e6cbb60 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd78c003ad0 con 0x559c9e6cbb60 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 10 client.28529 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 10 client.28529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 20 client.28529 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e751b00 0x559c9e753f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 4 client.28529 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c9e638d00 con 0x559c9e751b00 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.773+0000 7fd79affd640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e751b00 0x559c9e753f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.774+0000 7fd77affd640 10 client.28529 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.774+0000 7fd79affd640 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e751b00 0x559c9e753f20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae9b186fd661557 server_cookie=6ff9abd2a11a1d73 in_seq=0 out_seq=0 2024-09-15T22:30:03.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd77affd640 1 -- 172.21.15.77:0/847627963 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c9e638d00 con 0x559c9e751b00 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd77affd640 10 client.28529 handle_command_reply: tid=0 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 client.28529 shutdown 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 msgr2=0x559c9e6d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0042f0 tx=0x7fd78c004320 comp rx=0 tx=0).stop 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 shutdown_connections 2024-09-15T22:30:03.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559c9e751b00 0x559c9e753f20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559c9e6d6d40 0x559c9e6d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559c9e6cc480 0x559c9e6d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 --2- 172.21.15.77:0/847627963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c9e6cbb60 0x559c9e6d0bb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 >> 172.21.15.77:0/847627963 conn(0x559c9e6c80f0 msgr2=0x559c9e6d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.776+0000 7fd77affd640 0 client.28529 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 shutdown_connections 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a3e7d740 1 -- 172.21.15.77:0/847627963 wait complete. 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a3e7d740 20 client.28529 trim_cache size 0 max 16384 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:03.780 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:30:03.781 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "ca6b70cd-c74e-4b0f-8345-abfc04279738", 2024-09-15T22:30:03.781 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:30:03.781 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:03.781 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:03.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 msgr2=0x7fd7840a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 0x7fd7840a5970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd794009cd0 tx=0x7fd794030000 comp rx=0 tx=0).stop 2024-09-15T22:30:03.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 msgr2=0x7fd79c1968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:03.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.777+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd790009fd0 tx=0x7fd79000e800 comp rx=0 tx=0).stop 2024-09-15T22:30:03.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 shutdown_connections 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7840a34b0 0x7fd7840a5970 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd79c108290 0x7fd79c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd79c103740 0x7fd79c1968d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 --2- 172.21.15.77:0/848788569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd79c102d90 0x7fd79c196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 >> 172.21.15.77:0/848788569 conn(0x7fd79c0fc7d0 msgr2=0x7fd79c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 shutdown_connections 2024-09-15T22:30:03.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:03.778+0000 7fd7a1aa4640 1 -- 172.21.15.77:0/848788569 wait complete. 2024-09-15T22:30:04.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 80183 15.68 MB/sec execute 125 sec latency 168.903 ms 2024-09-15T22:30:04.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.739 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.739 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:04.739 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:04 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:04.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:04 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:04.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:04 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 -- 172.21.15.77:0/1701957281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 msgr2=0x7f56cc107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 --2- 172.21.15.77:0/1701957281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc107a50 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009930 tx=0x7f56c002a420 comp rx=0 tx=0).stop 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 -- 172.21.15.77:0/1701957281 shutdown_connections 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 --2- 172.21.15.77:0/1701957281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 0x7f56cc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 --2- 172.21.15.77:0/1701957281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc107a50 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 --2- 172.21.15.77:0/1701957281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 -- 172.21.15.77:0/1701957281 >> 172.21.15.77:0/1701957281 conn(0x7f56cc077210 msgr2=0x7f56cc077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 -- 172.21.15.77:0/1701957281 shutdown_connections 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.162+0000 7f56d27b9640 1 -- 172.21.15.77:0/1701957281 wait complete. 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.163+0000 7f56d27b9640 1 Processor -- start 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.163+0000 7f56d27b9640 1 -- start start 2024-09-15T22:30:05.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 0x7f56cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc111fb0 con 0x7f56cc101290 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc111e30 con 0x7f56cc108160 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc112130 con 0x7f56cc1008e0 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47338/0 (socket says 172.21.15.77:47338) 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 -- 172.21.15.77:0/2862930034 learned_addr learned my addr 172.21.15.77:0/2862930034 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d0d56640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 0x7f56cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 -- 172.21.15.77:0/2862930034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 msgr2=0x7f56cc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 -- 172.21.15.77:0/2862930034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 msgr2=0x7f56cc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 0x7f56cc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 -- 172.21.15.77:0/2862930034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc19d480 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cbfff640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56cb7fe640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005940 tx=0x7f56c0031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c003f030 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc19d710 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c00043d0 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.164+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0035070 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.165+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1a5050 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.166+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc06a110 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.167+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f56c0042030 con 0x7f56cc101290 2024-09-15T22:30:05.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.167+0000 7f56c97fa640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 0x7f56b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:05.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.167+0000 7f56cbfff640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 0x7f56b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:05.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.168+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9438+0+0 (secure 0 0 0) 0x7f56c00f02f0 con 0x7f56cc101290 2024-09-15T22:30:05.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.168+0000 7f56cbfff640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 0x7f56b40a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fc0 tx=0x7f56bc005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:05.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.172+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56c00bc2f0 con 0x7f56cc101290 2024-09-15T22:30:05.246 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:05.570 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:30:05.570 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:30:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:30:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.563+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56cc10a170 con 0x7f56cc101290 2024-09-15T22:30:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.565+0000 7f56c97fa640 1 -- 172.21.15.77:0/2862930034 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f56c00c05f0 con 0x7f56cc101290 2024-09-15T22:30:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.569+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 msgr2=0x7f56b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:05.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.569+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 0x7f56b40a5870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fc0 tx=0x7f56bc005f90 comp rx=0 tx=0).stop 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.569+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 msgr2=0x7f56cc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.569+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005940 tx=0x7f56c0031b50 comp rx=0 tx=0).stop 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.570+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 shutdown_connections 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.570+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f56b40a33b0 0x7f56b40a5870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.570+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56cc108160 0x7f56cc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.571+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56cc101290 0x7f56cc198a10 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.571+0000 7f56d27b9640 1 --2- 172.21.15.77:0/2862930034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56cc1008e0 0x7f56cc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.571+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 >> 172.21.15.77:0/2862930034 conn(0x7f56cc077210 msgr2=0x7f56cc105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.571+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 shutdown_connections 2024-09-15T22:30:05.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:05.571+0000 7f56d27b9640 1 -- 172.21.15.77:0/2862930034 wait complete. 2024-09-15T22:30:05.603 INFO:tasks.workunit.client.0.smithi077.stdout: 10 80699 15.76 MB/sec execute 126 sec latency 86.740 ms 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:30:05.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:05.837 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:05.837 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.837 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:30:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 -- 172.21.15.77:0/2461618418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 msgr2=0x7fa33410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/2461618418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa33410da40 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ca0 tx=0x7fa32c02a950 comp rx=0 tx=0).stop 2024-09-15T22:30:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 -- 172.21.15.77:0/2461618418 shutdown_connections 2024-09-15T22:30:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/2461618418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa33410da40 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/2461618418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334102570 0x7fa3341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/2461618418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334100e20 0x7fa334101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.001+0000 7fa33bb0e640 1 -- 172.21.15.77:0/2461618418 >> 172.21.15.77:0/2461618418 conn(0x7fa3340fc7d0 msgr2=0x7fa3340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.002+0000 7fa33bb0e640 1 -- 172.21.15.77:0/2461618418 shutdown_connections 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.002+0000 7fa33bb0e640 1 -- 172.21.15.77:0/2461618418 wait complete. 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 Processor -- start 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 -- start start 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 0x7fa3341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa334199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa334110f80 con 0x7fa334102ef0 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa334110e00 con 0x7fa334100e20 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3390a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 0x7fa3341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33bb0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa334111100 con 0x7fa334102570 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:60868/0 (socket says 172.21.15.77:60868) 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 -- 172.21.15.77:0/1261221475 learned_addr learned my addr 172.21.15.77:0/1261221475 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3390a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 0x7fa3341998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60392/0 (socket says 172.21.15.77:60392) 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa334199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 -- 172.21.15.77:0/1261221475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 msgr2=0x7fa3341998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 0x7fa3341998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 -- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 msgr2=0x7fa334199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.003+0000 7fa3398aa640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa334199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa3398aa640 1 -- 172.21.15.77:0/1261221475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3341a47b0 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa3398aa640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa33000e9f0 tx=0x7fa33000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa32affd640 1 -- 172.21.15.77:0/1261221475 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33000cd80 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa334197200 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa334197790 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa32affd640 1 -- 172.21.15.77:0/1261221475 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330010910 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.004+0000 7fa32affd640 1 -- 172.21.15.77:0/1261221475 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300188f0 con 0x7fa334100e20 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.007+0000 7fa33dee7740 1 Processor -- start 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.007+0000 7fa33dee7740 1 -- start start 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e398704dc0 0x55e39870f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e39870f680 0x55e398711a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e39866a2e0 con 0x55e3987044a0 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e398611ce0 con 0x55e39870f680 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33dee7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3986662a0 con 0x55e398704dc0 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa32affd640 1 -- 172.21.15.77:0/1261221475 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa330018b70 con 0x7fa334100e20 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47348/0 (socket says 172.21.15.77:47348) 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2238455149 learned_addr learned my addr 172.21.15.77:0/2238455149 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa3390a9640 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e39870f680 0x55e398711a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa3398aa640 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e398704dc0 0x55e39870f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2238455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e398704dc0 msgr2=0x55e39870f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e398704dc0 0x55e39870f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2238455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e39870f680 msgr2=0x55e398711a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e39870f680 0x55e398711a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.008+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2238455149 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3985f6440 con 0x55e3987044a0 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ca0 tx=0x7fa32c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301820bcbfbae244 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa32affd640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 0x7fa3200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa31ffff640 1 -- 172.21.15.77:0/2238455149 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c048070 con 0x55e3987044a0 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa31ffff640 1 -- 172.21.15.77:0/2238455149 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32c02f930 con 0x55e3987044a0 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa31ffff640 1 -- 172.21.15.77:0/2238455149 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c034040 con 0x55e3987044a0 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.009+0000 7fa32affd640 1 -- 172.21.15.77:0/1261221475 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fa3300d0300 con 0x7fa334100e20 2024-09-15T22:30:06.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.010+0000 7fa3398aa640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 0x7fa3200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- 172.21.15.77:0/2238455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 msgr2=0x55e398704880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa3398aa640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 0x7fa3200a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3300cd710 tx=0x7fa330000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ca0 tx=0x7fa32c0093e0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- 172.21.15.77:0/2238455149 shutdown_connections 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e39870f680 0x55e398711a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e398704dc0 0x55e39870f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2238455149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e398704880 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- 172.21.15.77:0/2238455149 >> 172.21.15.77:0/2238455149 conn(0x55e398700a30 msgr2=0x55e398700e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- 172.21.15.77:0/2238455149 shutdown_connections 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- 172.21.15.77:0/2238455149 wait complete. 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 -- start start 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.011+0000 7fa33dee7740 1 Processor -- start 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e39870ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e39870c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e39870c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e398712250 con 0x55e3987044a0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e398625050 con 0x55e398704dc0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3987123d0 con 0x55e39870f680 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33dee7740 10 client.0 _ll_register_callbacks cb 0x55e398665160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e39870ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e39870ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47352/0 (socket says 172.21.15.77:47352) 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33a0ab640 1 -- 172.21.15.77:0/1744744882 learned_addr learned my addr 172.21.15.77:0/1744744882 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e39870c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.012+0000 7fa3390a9640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e39870c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.014+0000 7fa33a0ab640 1 -- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 msgr2=0x55e39870c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.014+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e39870c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.014+0000 7fa33a0ab640 1 -- 172.21.15.77:0/1744744882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 msgr2=0x55e39870ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.014+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e39870ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.014+0000 7fa33a0ab640 1 -- 172.21.15.77:0/1744744882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3983db3c0 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.015+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e39870c760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa32c003d30 tx=0x7fa32c02a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5909476b7d9b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.015+0000 7fa31d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c048070 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fa32c03c0d0 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c02f930 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa32c034070 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa33dee7740 10 client.28462 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.016+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e398712250 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32c03cb60 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa32c0308d0 con 0x55e39870f680 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 10 client.28462 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:06.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 10 client.28462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 20 client.28462 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e39871eda0 0x55e398721190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 4 client.28462 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.017+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa32c02f930 con 0x55e39871eda0 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.018+0000 7fa3390a9640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e39871eda0 0x55e398721190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.023+0000 7fa3390a9640 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e39871eda0 0x55e398721190 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d29b05e834e18da server_cookie=a0ba68f5d99dff0 in_seq=0 out_seq=0 2024-09-15T22:30:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.023+0000 7fa31d7fa640 10 client.28462 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.026+0000 7fa31d7fa640 1 -- 172.21.15.77:0/1744744882 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa32c02f930 con 0x55e39871eda0 2024-09-15T22:30:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.026+0000 7fa31d7fa640 10 client.28462 handle_command_reply: tid=0 2024-09-15T22:30:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.026+0000 7fa33dee7740 1 client.28462 shutdown 2024-09-15T22:30:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 msgr2=0x55e39870c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e39870c760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa32c003d30 tx=0x7fa32c02a950 comp rx=0 tx=0).stop 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 shutdown_connections 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e39871eda0 0x55e398721190 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e39870c760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e39870c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 --2- 172.21.15.77:0/1744744882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e3987044a0 0x55e39870ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.027+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 >> 172.21.15.77:0/1744744882 conn(0x55e398700a30 msgr2=0x55e398706760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.028+0000 7fa31d7fa640 0 client.28462 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.029+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 shutdown_connections 2024-09-15T22:30:06.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.031+0000 7fa33dee7740 1 -- 172.21.15.77:0/1744744882 wait complete. 2024-09-15T22:30:06.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.034+0000 7fa33dee7740 20 client.28462 trim_cache size 0 max 16384 2024-09-15T22:30:06.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 Processor -- start 2024-09-15T22:30:06.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 -- start start 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3987044a0 0x55e398726760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e3987094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa318010080 con 0x55e398704dc0 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa318011bc0 con 0x55e3987044a0 2024-09-15T22:30:06.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.064+0000 7fa33dee7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa318010750 con 0x55e39870f680 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.065+0000 7fa3390a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.065+0000 7fa3390a9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47364/0 (socket says 172.21.15.77:47364) 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.065+0000 7fa3390a9640 1 -- 172.21.15.77:0/3006663271 learned_addr learned my addr 172.21.15.77:0/3006663271 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.065+0000 7fa3390a9640 1 -- 172.21.15.77:0/3006663271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 msgr2=0x55e3987094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.065+0000 7fa3390a9640 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e3987094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.066+0000 7fa3390a9640 1 -- 172.21.15.77:0/3006663271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3987044a0 msgr2=0x55e398726760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.066+0000 7fa3390a9640 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3987044a0 0x55e398726760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.066+0000 7fa3390a9640 1 -- 172.21.15.77:0/3006663271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39870f2e0 con 0x55e398704dc0 2024-09-15T22:30:06.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.066+0000 7fa3390a9640 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x55e39870bab0 tx=0x7fa324005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea834e15faed970c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.067+0000 7fa31f7fe640 1 -- 172.21.15.77:0/3006663271 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324009c70 con 0x55e398704dc0 2024-09-15T22:30:06.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.067+0000 7fa31f7fe640 1 -- 172.21.15.77:0/3006663271 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324009e10 con 0x55e398704dc0 2024-09-15T22:30:06.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.067+0000 7fa31f7fe640 1 -- 172.21.15.77:0/3006663271 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324004a00 con 0x55e398704dc0 2024-09-15T22:30:06.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 -- 172.21.15.77:0/3006663271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 msgr2=0x55e398726ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x55e39870bab0 tx=0x7fa324005f90 comp rx=0 tx=0).stop 2024-09-15T22:30:06.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 -- 172.21.15.77:0/3006663271 shutdown_connections 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e39870f680 0x55e3987094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e398704dc0 0x55e398726ca0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 --2- 172.21.15.77:0/3006663271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e3987044a0 0x55e398726760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.068+0000 7fa33dee7740 1 -- 172.21.15.77:0/3006663271 >> 172.21.15.77:0/3006663271 conn(0x55e398700a30 msgr2=0x55e39870c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.069+0000 7fa33dee7740 1 -- 172.21.15.77:0/3006663271 shutdown_connections 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.069+0000 7fa33dee7740 1 -- 172.21.15.77:0/3006663271 wait complete. 2024-09-15T22:30:06.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.069+0000 7fa33dee7740 1 -- start start 2024-09-15T22:30:06.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.069+0000 7fa33dee7740 1 Processor -- start 2024-09-15T22:30:06.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e398715f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e39870f680 0x55e3987a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33a0ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:60414/0 (socket says 172.21.15.77:60414) 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2800896274 learned_addr learned my addr 172.21.15.77:0/2800896274 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180114f0 con 0x55e39870f680 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa3390a9640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e398715f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa318010e20 con 0x55e398704dc0 2024-09-15T22:30:06.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa31800e080 con 0x55e3987044a0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.070+0000 7fa33dee7740 10 client.0 _ll_register_callbacks cb 0x7fa32c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 msgr2=0x55e398715f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e398715f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2800896274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e39870f680 msgr2=0x55e3987a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e39870f680 0x55e3987a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3987a24a0 con 0x55e3987044a0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.071+0000 7fa33a0ab640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02b140 tx=0x7fa32c02f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922076d05a316b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.072+0000 7fa31d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.072+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c060060 con 0x55e3987044a0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.072+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa324009e10 con 0x55e3987044a0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.072+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fa32c003af0 con 0x55e3987044a0 2024-09-15T22:30:06.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.072+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa32c067050 con 0x55e3987044a0 2024-09-15T22:30:06.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.076+0000 7fa33dee7740 10 client.28465 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:06.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.076+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e3987a3690 con 0x55e3987044a0 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.076+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa324009e10 con 0x55e3987044a0 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.076+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa32c03c8c0 con 0x55e3987044a0 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 10 client.28465 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 10 client.28465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 20 client.28465 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e3987a3d00 0x55e3987a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:06.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 4 client.28465 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:30:06.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e398677850 con 0x55e3987a3d00 2024-09-15T22:30:06.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.077+0000 7fa3390a9640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e3987a3d00 0x55e3987a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:06.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.080+0000 7fa3390a9640 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e3987a3d00 0x55e3987a6120 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a32e461fdce296 server_cookie=705017317306c442 in_seq=0 out_seq=0 2024-09-15T22:30:06.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.080+0000 7fa31d7fa640 10 client.28465 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:06.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.085+0000 7fa31d7fa640 1 -- 172.21.15.77:0/2800896274 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e398677850 con 0x55e3987a3d00 2024-09-15T22:30:06.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.085+0000 7fa31d7fa640 10 client.28465 handle_command_reply: tid=0 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 client.28465 shutdown 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 msgr2=0x55e398715a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02b140 tx=0x7fa32c02f720 comp rx=0 tx=0).stop 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 shutdown_connections 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e3987a3d00 0x55e3987a6120 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e39870f680 0x55e3987a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e398704dc0 0x55e398715f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 --2- 172.21.15.77:0/2800896274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3987044a0 0x55e398715a30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.086+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 >> 172.21.15.77:0/2800896274 conn(0x55e398700a30 msgr2=0x55e39870da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.087+0000 7fa31d7fa640 0 client.28465 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.087+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 shutdown_connections 2024-09-15T22:30:06.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.088+0000 7fa33dee7740 1 -- 172.21.15.77:0/2800896274 wait complete. 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.088+0000 7fa33dee7740 20 client.28465 trim_cache size 0 max 16384 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "ca6b70cd-c74e-4b0f-8345-abfc04279738": { 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "ca6b70cd-c74e-4b0f-8345-abfc04279738", 2024-09-15T22:30:06.090 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.090+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 msgr2=0x7fa3200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.090+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 0x7fa3200a5870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa3300cd710 tx=0x7fa330000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.090+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 msgr2=0x7fa3341993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:06.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.090+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa33000e9f0 tx=0x7fa33000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 shutdown_connections 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3200a33b0 0x7fa3200a5870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa334102ef0 0x7fa334199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa334102570 0x7fa3341998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 --2- 172.21.15.77:0/1261221475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa334100e20 0x7fa3341993b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.091+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 >> 172.21.15.77:0/1261221475 conn(0x7fa3340fc7d0 msgr2=0x7fa3341068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.092+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 shutdown_connections 2024-09-15T22:30:06.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:06.092+0000 7fa33bb0e640 1 -- 172.21.15.77:0/1261221475 wait complete. 2024-09-15T22:30:06.602 INFO:tasks.workunit.client.0.smithi077.stdout: 10 81137 15.85 MB/sec execute 127 sec latency 204.640 ms 2024-09-15T22:30:06.788 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:06 smithi077 ceph-mon[30980]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:06.789 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:06 smithi077 ceph-mon[30980]: osdmap e114: 12 total, 12 up, 12 in 2024-09-15T22:30:06.789 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:06 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2862930034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:06 smithi196 ceph-mon[35109]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:06.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:06 smithi196 ceph-mon[35109]: osdmap e114: 12 total, 12 up, 12 in 2024-09-15T22:30:06.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:06 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2862930034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:06.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:06 smithi117 ceph-mon[35021]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.3 MiB/s wr, 158 op/s 2024-09-15T22:30:06.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:06 smithi117 ceph-mon[35021]: osdmap e114: 12 total, 12 up, 12 in 2024-09-15T22:30:06.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:06 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2862930034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:07.603 INFO:tasks.workunit.client.0.smithi077.stdout: 10 81631 15.99 MB/sec execute 128 sec latency 95.348 ms 2024-09-15T22:30:08.333 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0' 2024-09-15T22:30:08.333 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 142.95 --await-for 45.0 2024-09-15T22:30:08.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:08 smithi077 ceph-mon[30980]: pgmap v557: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:08.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 81988 16.05 MB/sec execute 129 sec latency 136.196 ms 2024-09-15T22:30:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:08 smithi196 ceph-mon[35109]: pgmap v557: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:08 smithi117 ceph-mon[35021]: pgmap v557: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:09.027 INFO:tasks.cephfs.filesystem:scrub status for tag:ca6b70cd-c74e-4b0f-8345-abfc04279738 - {'path': '/', 'tag': 'ca6b70cd-c74e-4b0f-8345-abfc04279738', 'options': 'recursive,force'} 2024-09-15T22:30:09.102 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:09.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82295 15.98 MB/sec execute 130 sec latency 516.745 ms 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/3653263646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 msgr2=0x7fb5f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/3653263646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01029b0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a480 comp rx=0 tx=0).stop 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/3653263646 shutdown_connections 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/3653263646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 0x7fb5f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/3653263646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01029b0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/3653263646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/3653263646 >> 172.21.15.77:0/3653263646 conn(0x7fb5f00fc7d0 msgr2=0x7fb5f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/3653263646 shutdown_connections 2024-09-15T22:30:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.700+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/3653263646 wait complete. 2024-09-15T22:30:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 Processor -- start 2024-09-15T22:30:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 -- start start 2024-09-15T22:30:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 0x7fb5f01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0110f50 con 0x7fb5f0102570 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0110dd0 con 0x7fb5f0102ef0 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5f4bd1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01110d0 con 0x7fb5f0100e20 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5ee59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34328/0 (socket says 172.21.15.77:34328) 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5e7fff640 1 -- 172.21.15.77:0/21463157 learned_addr learned my addr 172.21.15.77:0/21463157 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.701+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 0x7fb5f01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 -- 172.21.15.77:0/21463157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 msgr2=0x7fb5f01a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 0x7fb5f01a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 -- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 msgr2=0x7fb5f01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 -- 172.21.15.77:0/21463157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a4920 con 0x7fb5f0100e20 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0199360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000d6d0 tx=0x7fb5e000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5e77fe640 1 -- 172.21.15.77:0/21463157 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0017070 con 0x7fb5f0100e20 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a4af0 con 0x7fb5f0100e20 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.702+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01a5080 con 0x7fb5f0100e20 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.703+0000 7fb5e77fe640 1 -- 172.21.15.77:0/21463157 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e0009d00 con 0x7fb5f0100e20 2024-09-15T22:30:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.703+0000 7fb5e77fe640 1 -- 172.21.15.77:0/21463157 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0020060 con 0x7fb5f0100e20 2024-09-15T22:30:09.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.705+0000 7fb5e77fe640 1 -- 172.21.15.77:0/21463157 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb5e0010460 con 0x7fb5f0100e20 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.705+0000 7fb5f6faa740 1 Processor -- start 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 -- start start 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562bb611c160 con 0x562bb61c2a30 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562bb60c3720 con 0x562bb61b7850 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562bb6118120 con 0x562bb61b8170 2024-09-15T22:30:09.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:46224/0 (socket says 172.21.15.77:46224) 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1689040832 learned_addr learned my addr 172.21.15.77:0/1689040832 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5e77fe640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 0x7fb5dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.706+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1689040832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 msgr2=0x562bb61c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1689040832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 msgr2=0x562bb61c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1689040832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb5e8dba0 con 0x562bb61b7850 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5e77fe640 1 -- 172.21.15.77:0/21463157 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fb5e00cfae0 con 0x7fb5f0100e20 2024-09-15T22:30:09.713 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 0x7fb5dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009f30 tx=0x7fb5e80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edeb83d64ab28c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5e4ff9640 1 -- 172.21.15.77:0/1689040832 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800f480 con 0x562bb61b7850 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5e4ff9640 1 -- 172.21.15.77:0/1689040832 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e800fb50 con 0x562bb61b7850 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.707+0000 7fb5e4ff9640 1 -- 172.21.15.77:0/1689040832 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e801b930 con 0x562bb61b7850 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 0x7fb5dc0a5870 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8020000 tx=0x7fb5e8021000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1689040832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 msgr2=0x562bb61b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009f30 tx=0x7fb5e80118d0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1689040832 shutdown_connections 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1689040832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61b7c30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1689040832 >> 172.21.15.77:0/1689040832 conn(0x562bb61b3de0 msgr2=0x562bb61b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1689040832 shutdown_connections 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.708+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1689040832 wait complete. 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 -- start start 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 Processor -- start 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61c2a30 0x562bb61c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.709+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562bb61c5600 con 0x562bb61b8170 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562bb60a4900 con 0x562bb61b7850 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562bb61c5780 con 0x562bb61c2a30 2024-09-15T22:30:09.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34340/0 (socket says 172.21.15.77:34340) 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 -- 172.21.15.77:0/3334432964 learned_addr learned my addr 172.21.15.77:0/3334432964 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5f6faa740 10 client.0 _ll_register_callbacks cb 0x562bb60a2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61c2a30 0x562bb61c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 -- 172.21.15.77:0/3334432964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61c2a30 msgr2=0x562bb61c0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61c2a30 0x562bb61c0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 -- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 msgr2=0x562bb61bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 -- 172.21.15.77:0/3334432964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb60e4c50 con 0x562bb61b8170 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009a20 tx=0x7fb5d8003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9919e487016f8c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.710+0000 7fb5d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035040 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e800fb50 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fb5d8003b50 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5d804f050 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5f6faa740 10 client.19764 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562bb61d5620 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e800fb50 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb5d8046470 con 0x562bb61b8170 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.711+0000 7fb5f6faa740 10 client.19764 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:09.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5f6faa740 10 client.19764 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5f6faa740 20 client.19764 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb61d6a80 0x562bb61d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5f6faa740 4 client.19764 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb6129890 con 0x562bb61d6a80 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb61d6a80 0x562bb61d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5d5ffb640 10 client.19764 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.712+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb61d6a80 0x562bb61d8f20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f00eaee2e7dbef6 server_cookie=6e83d6a368f36f5e in_seq=0 out_seq=0 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/3334432964 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bb6129890 con 0x562bb61d6a80 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5d5ffb640 10 client.19764 handle_command_reply: tid=0 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 client.19764 shutdown 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 msgr2=0x562bb61bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009a20 tx=0x7fb5d8003670 comp rx=0 tx=0).stop 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 shutdown_connections 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb61d6a80 0x562bb61d8f20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61c2a30 0x562bb61c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61bfdc0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/3334432964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 >> 172.21.15.77:0/3334432964 conn(0x562bb61b3de0 msgr2=0x562bb61b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5d5ffb640 0 client.19764 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 shutdown_connections 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 1 -- 172.21.15.77:0/3334432964 wait complete. 2024-09-15T22:30:09.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.715+0000 7fb5f6faa740 20 client.19764 trim_cache size 0 max 16384 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.733+0000 7fb5f6faa740 1 Processor -- start 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.733+0000 7fb5f6faa740 1 -- start start 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61ba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d000ff60 con 0x562bb61c2a30 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0011aa0 con 0x562bb61b7850 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0010630 con 0x562bb61b8170 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34346/0 (socket says 172.21.15.77:34346) 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1797401815 learned_addr learned my addr 172.21.15.77:0/1797401815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5ee59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1797401815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 msgr2=0x562bb61de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1797401815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 msgr2=0x562bb61ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 -- 172.21.15.77:0/1797401815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb61b7fa0 con 0x562bb61c2a30 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.734+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009f30 tx=0x7fb5e8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28624af878c7820e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5d7fff640 1 -- 172.21.15.77:0/1797401815 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8029040 con 0x562bb61c2a30 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5d7fff640 1 -- 172.21.15.77:0/1797401815 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e802a040 con 0x562bb61c2a30 2024-09-15T22:30:09.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5d7fff640 1 -- 172.21.15.77:0/1797401815 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800f8d0 con 0x562bb61c2a30 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1797401815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 msgr2=0x562bb61ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009f30 tx=0x7fb5e8027000 comp rx=0 tx=0).stop 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1797401815 shutdown_connections 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61c2a30 0x562bb61ded70 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b8170 0x562bb61de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/1797401815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61b7850 0x562bb61ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.735+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1797401815 >> 172.21.15.77:0/1797401815 conn(0x562bb61b3de0 msgr2=0x562bb61be150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:09.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1797401815 shutdown_connections 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- 172.21.15.77:0/1797401815 wait complete. 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- start start 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 Processor -- start 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b7850 0x562bb61caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61c2a30 0x562bb61c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d00113d0 con 0x562bb61b8170 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0010d00 con 0x562bb61c2a30 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d000df60 con 0x562bb61b7850 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5f6faa740 10 client.0 _ll_register_callbacks cb 0x7fb5d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34360/0 (socket says 172.21.15.77:34360) 2024-09-15T22:30:09.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5e7fff640 1 -- 172.21.15.77:0/2727726309 learned_addr learned my addr 172.21.15.77:0/2727726309 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b7850 0x562bb61caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.736+0000 7fb5eed9d640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61c2a30 0x562bb61c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 -- 172.21.15.77:0/2727726309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b7850 msgr2=0x562bb61caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b7850 0x562bb61caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 -- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61c2a30 msgr2=0x562bb61c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61c2a30 0x562bb61c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 -- 172.21.15.77:0/2727726309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb61c8010 con 0x562bb61b8170 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5e7fff640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80059d0 tx=0x7fb5d8009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13402f556904504 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035040 con 0x562bb61b8170 2024-09-15T22:30:09.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e802a040 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9526+0+0 (secure 0 0 0) 0x7fb5d8003950 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb5d8061050 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5f6faa740 10 client.19770 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562bb62420f0 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.737+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e802a040 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb5d80318c0 con 0x562bb61b8170 2024-09-15T22:30:09.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 10 client.19770 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 10 client.19770 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 20 client.19770 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb6242900 0x562bb6244d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 4 client.19770 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}] 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.95, "await_for": 45.0}) -- 0x562bb61af410 con 0x562bb6242900 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb6242900 0x562bb6244d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5d5ffb640 10 client.19770 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:09.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:09.738+0000 7fb5ee59c640 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb6242900 0x562bb6244d20 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3768a48f10757a78 server_cookie=623f16c2aee552b in_seq=0 out_seq=0 2024-09-15T22:30:10.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5d5ffb640 1 -- 172.21.15.77:0/2727726309 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562bb61af410 con 0x562bb6242900 2024-09-15T22:30:10.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5d5ffb640 10 client.19770 handle_command_reply: tid=0 2024-09-15T22:30:10.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5f6faa740 1 client.19770 shutdown 2024-09-15T22:30:10.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 msgr2=0x562bb61caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80059d0 tx=0x7fb5d8009890 comp rx=0 tx=0).stop 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.568+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 shutdown_connections 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562bb6242900 0x562bb6244d20 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562bb61c2a30 0x562bb61c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bb61b8170 0x562bb61caf40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 --2- 172.21.15.77:0/2727726309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562bb61b7850 0x562bb61caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 >> 172.21.15.77:0/2727726309 conn(0x562bb61b3de0 msgr2=0x562bb61bf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5d5ffb640 0 client.19770 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 shutdown_connections 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 1 -- 172.21.15.77:0/2727726309 wait complete. 2024-09-15T22:30:10.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.569+0000 7fb5f6faa740 20 client.19770 trim_cache size 0 max 16384 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 66, 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "9de00ec3": { 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 66, 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:10.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.9, 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:10.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 msgr2=0x7fb5dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 0x7fb5dc0a5870 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8020000 tx=0x7fb5e8021000 comp rx=0 tx=0).stop 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 msgr2=0x7fb5f0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:10.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0199360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000d6d0 tx=0x7fb5e000db90 comp rx=0 tx=0).stop 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 shutdown_connections 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb5dc0a33b0 0x7fb5dc0a5870 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f0102ef0 0x7fb5f01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5f0102570 0x7fb5f01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 --2- 172.21.15.77:0/21463157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f0100e20 0x7fb5f0199360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 >> 172.21.15.77:0/21463157 conn(0x7fb5f00fc7d0 msgr2=0x7fb5f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 shutdown_connections 2024-09-15T22:30:10.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:10.570+0000 7fb5f4bd1640 1 -- 172.21.15.77:0/21463157 wait complete. 2024-09-15T22:30:10.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.87 MB/sec execute 131 sec latency 1489.618 ms 2024-09-15T22:30:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:10 smithi077 ceph-mon[30980]: pgmap v558: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:10 smithi196 ceph-mon[35109]: pgmap v558: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:10 smithi117 ceph-mon[35021]: pgmap v558: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 1.2 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.6 MiB/s wr, 337 op/s 2024-09-15T22:30:11.286 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9de00ec3', quiesce took 8.1 seconds. Will release after: 14.85 2024-09-15T22:30:11.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.75 MB/sec execute 132 sec latency 1823.187 ms 2024-09-15T22:30:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:30:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:30:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:30:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:30:11.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:11 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:30:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:30:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:30:11.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:30:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:30:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:11 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:30:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:30:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:30:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:30:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:30:11.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:11 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:30:12.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.64 MB/sec execute 133 sec latency 2823.294 ms 2024-09-15T22:30:12.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:12 smithi077 ceph-mon[30980]: pgmap v559: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:12.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:12 smithi196 ceph-mon[35109]: pgmap v559: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:12 smithi117 ceph-mon[35021]: pgmap v559: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:13.604 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.52 MB/sec execute 134 sec latency 3823.418 ms 2024-09-15T22:30:14.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.40 MB/sec execute 135 sec latency 4823.477 ms 2024-09-15T22:30:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:14 smithi077 ceph-mon[30980]: pgmap v560: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:14 smithi077 ceph-mon[30980]: osdmap e115: 12 total, 12 up, 12 in 2024-09-15T22:30:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:14 smithi196 ceph-mon[35109]: pgmap v560: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:14 smithi196 ceph-mon[35109]: osdmap e115: 12 total, 12 up, 12 in 2024-09-15T22:30:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:14 smithi117 ceph-mon[35021]: pgmap v560: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.3 MiB/s wr, 448 op/s 2024-09-15T22:30:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:14 smithi117 ceph-mon[35021]: osdmap e115: 12 total, 12 up, 12 in 2024-09-15T22:30:15.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.29 MB/sec execute 136 sec latency 5823.515 ms 2024-09-15T22:30:16.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.18 MB/sec execute 137 sec latency 6823.584 ms 2024-09-15T22:30:16.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:16 smithi077 ceph-mon[30980]: pgmap v562: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-09-15T22:30:16.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:16 smithi196 ceph-mon[35109]: pgmap v562: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-09-15T22:30:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:16 smithi117 ceph-mon[35021]: pgmap v562: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 488 op/s 2024-09-15T22:30:17.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 15.07 MB/sec execute 138 sec latency 7823.642 ms 2024-09-15T22:30:18.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.96 MB/sec execute 139 sec latency 8823.747 ms 2024-09-15T22:30:18.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:18 smithi077 ceph-mon[30980]: pgmap v563: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:18.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:18 smithi196 ceph-mon[35109]: pgmap v563: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:18 smithi117 ceph-mon[35021]: pgmap v563: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:19.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.85 MB/sec execute 140 sec latency 9823.780 ms 2024-09-15T22:30:20.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.75 MB/sec execute 141 sec latency 10823.821 ms 2024-09-15T22:30:20.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:20 smithi077 ceph-mon[30980]: pgmap v564: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:20.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:20 smithi196 ceph-mon[35109]: pgmap v564: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:20.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:20 smithi117 ceph-mon[35021]: pgmap v564: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-09-15T22:30:21.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.64 MB/sec execute 142 sec latency 11823.912 ms 2024-09-15T22:30:22.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.54 MB/sec execute 143 sec latency 12824.016 ms 2024-09-15T22:30:22.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:22 smithi077 ceph-mon[30980]: pgmap v565: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:22.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:22 smithi196 ceph-mon[35109]: pgmap v565: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:22.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:22 smithi117 ceph-mon[35021]: pgmap v565: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:23.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.44 MB/sec execute 144 sec latency 13824.120 ms 2024-09-15T22:30:24.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.34 MB/sec execute 145 sec latency 14824.195 ms 2024-09-15T22:30:24.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:24 smithi077 ceph-mon[30980]: pgmap v566: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:24.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:24 smithi196 ceph-mon[35109]: pgmap v566: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:24 smithi117 ceph-mon[35021]: pgmap v566: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 914 KiB/s wr, 36 op/s 2024-09-15T22:30:25.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.24 MB/sec execute 146 sec latency 15824.245 ms 2024-09-15T22:30:25.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:25 smithi077 ceph-mon[30980]: pgmap v567: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 854 KiB/s wr, 33 op/s 2024-09-15T22:30:25.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:25 smithi196 ceph-mon[35109]: pgmap v567: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 854 KiB/s wr, 33 op/s 2024-09-15T22:30:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:25 smithi117 ceph-mon[35021]: pgmap v567: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 854 KiB/s wr, 33 op/s 2024-09-15T22:30:26.138 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:30:26.399 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:26.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.15 MB/sec execute 147 sec latency 16824.318 ms 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 -- 172.21.15.77:0/2522485839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c41027a0 msgr2=0x7f09c41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 --2- 172.21.15.77:0/2522485839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c41027a0 0x7f09c41067f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009ca0 tx=0x7f09b802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 -- 172.21.15.77:0/2522485839 shutdown_connections 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 --2- 172.21.15.77:0/2522485839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c4106f20 0x7f09c410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 --2- 172.21.15.77:0/2522485839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c41027a0 0x7f09c41067f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 --2- 172.21.15.77:0/2522485839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 0x7f09c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 -- 172.21.15.77:0/2522485839 >> 172.21.15.77:0/2522485839 conn(0x7f09c40fd8b0 msgr2=0x7f09c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:26.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 -- 172.21.15.77:0/2522485839 shutdown_connections 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.902+0000 7f09cbad9640 1 -- 172.21.15.77:0/2522485839 wait complete. 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 Processor -- start 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 -- start start 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 0x7f09c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c41027a0 0x7f09c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4111fa0 con 0x7f09c4106f20 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4111e20 con 0x7f09c4101df0 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09cbad9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4112120 con 0x7f09c41027a0 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:26.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58732/0 (socket says 172.21.15.77:58732) 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 -- 172.21.15.77:0/3958215547 learned_addr learned my addr 172.21.15.77:0/3958215547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09c9875640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 0x7f09c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 -- 172.21.15.77:0/3958215547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c41027a0 msgr2=0x7f09c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09c9074640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c41027a0 0x7f09c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c41027a0 0x7f09c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 -- 172.21.15.77:0/3958215547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 msgr2=0x7f09c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 0x7f09c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.903+0000 7f09ca076640 1 -- 172.21.15.77:0/3958215547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41a6900 con 0x7f09c4106f20 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09ca076640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009fd0 tx=0x7f09b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011830 con 0x7f09c4106f20 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4016070 con 0x7f09c4106f20 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41a6bf0 con 0x7f09c4106f20 2024-09-15T22:30:26.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400f2e0 con 0x7f09c4106f20 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.904+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a7130 con 0x7f09c4106f20 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.905+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c41021d0 con 0x7f09c4106f20 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.906+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f09b40119d0 con 0x7f09c4106f20 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.907+0000 7f09c2ffd640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 0x7f09ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.907+0000 7f09c9875640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 0x7f09ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.907+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f09b40cf620 con 0x7f09c4106f20 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.907+0000 7f09c9875640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 0x7f09ac0a5870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00bbe0 tx=0x7f09bc00b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:26.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:26.911+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09b409bd20 con 0x7f09c4106f20 2024-09-15T22:30:27.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.113+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c4109bd0 con 0x7f09c4106f20 2024-09-15T22:30:27.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.114+0000 7f09c2ffd640 1 -- 172.21.15.77:0/3958215547 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f09b409b620 con 0x7f09c4106f20 2024-09-15T22:30:27.116 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:30:27.116 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:30:27.116 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:30:27.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 msgr2=0x7f09ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:27.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 0x7f09ac0a5870 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00bbe0 tx=0x7f09bc00b390 comp rx=0 tx=0).stop 2024-09-15T22:30:27.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 msgr2=0x7f09c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:27.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009fd0 tx=0x7f09b400b8e0 comp rx=0 tx=0).stop 2024-09-15T22:30:27.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 shutdown_connections 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f09ac0a33b0 0x7f09ac0a5870 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.116+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09c4106f20 0x7f09c4074f70 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.117+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09c41027a0 0x7f09c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.117+0000 7f09cbad9640 1 --2- 172.21.15.77:0/3958215547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4101df0 0x7f09c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.117+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 >> 172.21.15.77:0/3958215547 conn(0x7f09c40fd8b0 msgr2=0x7f09c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.117+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 shutdown_connections 2024-09-15T22:30:27.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:27.117+0000 7f09cbad9640 1 -- 172.21.15.77:0/3958215547 wait complete. 2024-09-15T22:30:27.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 14.05 MB/sec execute 148 sec latency 17824.391 ms 2024-09-15T22:30:27.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 9de00ec3 --release --await' 2024-09-15T22:30:27.821 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 9de00ec3 --release --await 2024-09-15T22:30:28.072 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:28.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:27 smithi077 ceph-mon[30980]: pgmap v568: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 983 KiB/s wr, 39 op/s 2024-09-15T22:30:28.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:27 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3958215547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:28.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:27 smithi196 ceph-mon[35109]: pgmap v568: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 983 KiB/s wr, 39 op/s 2024-09-15T22:30:28.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:27 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3958215547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:27 smithi117 ceph-mon[35021]: pgmap v568: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 983 KiB/s wr, 39 op/s 2024-09-15T22:30:28.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:27 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3958215547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 -- 172.21.15.77:0/4294038450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 msgr2=0x7ffa2c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 --2- 172.21.15.77:0/4294038450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c10d930 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009cd0 tx=0x7ffa2402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 -- 172.21.15.77:0/4294038450 shutdown_connections 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 --2- 172.21.15.77:0/4294038450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c10d930 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 --2- 172.21.15.77:0/4294038450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 --2- 172.21.15.77:0/4294038450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 0x7ffa2c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 -- 172.21.15.77:0/4294038450 >> 172.21.15.77:0/4294038450 conn(0x7ffa2c0fc780 msgr2=0x7ffa2c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 -- 172.21.15.77:0/4294038450 shutdown_connections 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.558+0000 7ffa33bec640 1 -- 172.21.15.77:0/4294038450 wait complete. 2024-09-15T22:30:28.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 Processor -- start 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 -- start start 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 0x7ffa2c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c110e80 con 0x7ffa2c101400 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c110d00 con 0x7ffa2c106a60 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa33bec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c111000 con 0x7ffa2c100a60 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58748/0 (socket says 172.21.15.77:58748) 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.559+0000 7ffa31187640 1 -- 172.21.15.77:0/945807477 learned_addr learned my addr 172.21.15.77:0/945807477 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31988640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa32189640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 0x7ffa2c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 -- 172.21.15.77:0/945807477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 msgr2=0x7ffa2c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 -- 172.21.15.77:0/945807477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 msgr2=0x7ffa2c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 0x7ffa2c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 -- 172.21.15.77:0/945807477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c19d480 con 0x7ffa2c101400 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31988640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa31187640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b4a0 tx=0x7ffa1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa22ffd640 1 -- 172.21.15.77:0/945807477 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c014070 con 0x7ffa2c101400 2024-09-15T22:30:28.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa22ffd640 1 -- 172.21.15.77:0/945807477 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c010af0 con 0x7ffa2c101400 2024-09-15T22:30:28.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa22ffd640 1 -- 172.21.15.77:0/945807477 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c017d10 con 0x7ffa2c101400 2024-09-15T22:30:28.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c19d770 con 0x7ffa2c101400 2024-09-15T22:30:28.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.560+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c1a5050 con 0x7ffa2c101400 2024-09-15T22:30:28.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.562+0000 7ffa22ffd640 1 -- 172.21.15.77:0/945807477 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa1c017070 con 0x7ffa2c101400 2024-09-15T22:30:28.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 Processor -- start 2024-09-15T22:30:28.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 -- start start 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa22ffd640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 0x7ffa180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f04f20 0x55a3f6f05300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f14650 0x55a3f6f16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6e6a620 con 0x55a3f6f05840 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6e11e50 con 0x55a3f6f14650 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6e665e0 con 0x55a3f6f04f20 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.563+0000 7ffa32189640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 0x7ffa180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58756/0 (socket says 172.21.15.77:58756) 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 -- 172.21.15.77:0/3176727518 learned_addr learned my addr 172.21.15.77:0/3176727518 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:28.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa22ffd640 1 -- 172.21.15.77:0/945807477 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7ffa1c0cf410 con 0x7ffa2c101400 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31988640 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f04f20 0x55a3f6f05300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 -- 172.21.15.77:0/3176727518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f04f20 msgr2=0x55a3f6f05300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f04f20 0x55a3f6f05300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 -- 172.21.15.77:0/3176727518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f14650 msgr2=0x55a3f6f16a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f14650 0x55a3f6f16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 -- 172.21.15.77:0/3176727518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6e02d40 con 0x55a3f6f05840 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa32189640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 0x7ffa180a58f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa24009900 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa31187640 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c097870 tx=0x7ffa1c017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9b378a104a4f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa17fff640 1 -- 172.21.15.77:0/3176727518 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c01e070 con 0x55a3f6f05840 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa17fff640 1 -- 172.21.15.77:0/3176727518 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a4f40 con 0x55a3f6f05840 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.564+0000 7ffa17fff640 1 -- 172.21.15.77:0/3176727518 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c09f7c0 con 0x55a3f6f05840 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3176727518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 msgr2=0x55a3f6f14110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c097870 tx=0x7ffa1c017730 comp rx=0 tx=0).stop 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3176727518 shutdown_connections 2024-09-15T22:30:28.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f14650 0x55a3f6f16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f14110 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3176727518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f04f20 0x55a3f6f05300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3176727518 >> 172.21.15.77:0/3176727518 conn(0x55a3f6f015f0 msgr2=0x55a3f6f019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3176727518 shutdown_connections 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3176727518 wait complete. 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.565+0000 7ffa35fc5740 1 -- start start 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 Processor -- start 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 0x55a3f6f11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6f17220 con 0x55a3f6f04f20 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6df2f10 con 0x55a3f6f05840 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6f173a0 con 0x55a3f6f14650 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa35fc5740 10 client.0 _ll_register_callbacks cb 0x55a3f6e16db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58764/0 (socket says 172.21.15.77:58764) 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 -- 172.21.15.77:0/3119348145 learned_addr learned my addr 172.21.15.77:0/3119348145 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31187640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 0x55a3f6f11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa32189640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 -- 172.21.15.77:0/3119348145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 msgr2=0x55a3f6f11fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f11fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 -- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 msgr2=0x55a3f6f11a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 0x55a3f6f11a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31988640 1 -- 172.21.15.77:0/3119348145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f6b79580 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.566+0000 7ffa31187640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 0x55a3f6f11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa31988640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ffa28010a70 tx=0x7ffa28010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8478abbd8ac5f0b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800eb90 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7ffa2800ed30 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c0a4f40 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa28011440 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 10 client.19788 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3f6c43960 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a4f40 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffa2801a690 con 0x55a3f6f04f20 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 10 client.19788 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:28.574 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 10 client.19788 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 20 client.19788 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f242b0 0x55a3f6f26740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 4 client.19788 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.567+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f6e77b90 con 0x55a3f6f242b0 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.568+0000 7ffa31187640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f242b0 0x55a3f6f26740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.568+0000 7ffa31187640 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f242b0 0x55a3f6f26740 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40223d108f2660a7 server_cookie=23dfcfb6e5fd1659 in_seq=0 out_seq=0 2024-09-15T22:30:28.575 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.568+0000 7ffa157fa640 10 client.19788 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa157fa640 1 -- 172.21.15.77:0/3119348145 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3f6e77b90 con 0x55a3f6f242b0 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa157fa640 10 client.19788 handle_command_reply: tid=0 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 client.19788 shutdown 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 msgr2=0x55a3f6f11530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ffa28010a70 tx=0x7ffa28010f30 comp rx=0 tx=0).stop 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 shutdown_connections 2024-09-15T22:30:28.576 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f242b0 0x55a3f6f26740 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f05840 0x55a3f6f11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/3119348145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11530 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa157fa640 0 client.19788 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 >> 172.21.15.77:0/3119348145 conn(0x55a3f6f015f0 msgr2=0x55a3f6f07350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 shutdown_connections 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 1 -- 172.21.15.77:0/3119348145 wait complete. 2024-09-15T22:30:28.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.569+0000 7ffa35fc5740 20 client.19788 trim_cache size 0 max 16384 2024-09-15T22:30:28.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.585+0000 7ffa35fc5740 1 Processor -- start 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.585+0000 7ffa35fc5740 1 -- start start 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f2c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ace20 con 0x55a3f6f05840 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ae960 con 0x55a3f6f04f20 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ad4f0 con 0x55a3f6f14650 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58774/0 (socket says 172.21.15.77:58774) 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 -- 172.21.15.77:0/255084424 learned_addr learned my addr 172.21.15.77:0/255084424 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:28.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31988640 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 -- 172.21.15.77:0/255084424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 msgr2=0x55a3f6f2c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f2c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 -- 172.21.15.77:0/255084424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 msgr2=0x55a3f6f11c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 -- 172.21.15.77:0/255084424 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6f17390 con 0x55a3f6f05840 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa31187640 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c09b110 tx=0x7ffa1c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b6409675bcc9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa177fe640 1 -- 172.21.15.77:0/255084424 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c01e040 con 0x55a3f6f05840 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa177fe640 1 -- 172.21.15.77:0/255084424 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a4080 con 0x55a3f6f05840 2024-09-15T22:30:28.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.586+0000 7ffa177fe640 1 -- 172.21.15.77:0/255084424 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c09c740 con 0x55a3f6f05840 2024-09-15T22:30:28.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- 172.21.15.77:0/255084424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 msgr2=0x55a3f6f12140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c09b110 tx=0x7ffa1c097040 comp rx=0 tx=0).stop 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- 172.21.15.77:0/255084424 shutdown_connections 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f14650 0x55a3f6f2c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f05840 0x55a3f6f12140 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/255084424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f11c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- 172.21.15.77:0/255084424 >> 172.21.15.77:0/255084424 conn(0x55a3f6f015f0 msgr2=0x55a3f6f114f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- 172.21.15.77:0/255084424 shutdown_connections 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- 172.21.15.77:0/255084424 wait complete. 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 -- start start 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.587+0000 7ffa35fc5740 1 Processor -- start 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 0x55a3f6f19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ae290 con 0x55a3f6f14650 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1adbc0 con 0x55a3f6f04f20 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1aae20 con 0x55a3f6f05840 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa35fc5740 10 client.0 _ll_register_callbacks cb 0x7ffa2800cd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58776/0 (socket says 172.21.15.77:58776) 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa31988640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 -- 172.21.15.77:0/1776830648 learned_addr learned my addr 172.21.15.77:0/1776830648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa31187640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 0x55a3f6f19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 -- 172.21.15.77:0/1776830648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 msgr2=0x55a3f6f19020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 0x55a3f6f19020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 -- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 msgr2=0x55a3f6f18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:28.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 -- 172.21.15.77:0/1776830648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f6f8e520 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa31187640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 0x55a3f6f19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa32189640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ffa24035000 tx=0x7ffa240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a46e2e2e279cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.588+0000 7ffa157fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2404e070 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7ffa240305a0 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c0a4080 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa2405b050 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 10 client.19794 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a3f6f8f860 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a4080 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffa24030070 con 0x55a3f6f14650 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 10 client.19794 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:28.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 10 client.19794 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 20 client.19794 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f90040 0x55a3f6f92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 4 client.19794 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9de00ec3", "release": true, "await": true}] 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9de00ec3", "release": true, "await": true}) -- 0x55a3f6e720a0 con 0x55a3f6f90040 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.589+0000 7ffa31988640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f90040 0x55a3f6f92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.590+0000 7ffa157fa640 10 client.19794 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:28.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.590+0000 7ffa31988640 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f90040 0x55a3f6f92460 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0100668b7eca934 server_cookie=af8e32bc8313e141 in_seq=0 out_seq=0 2024-09-15T22:30:28.605 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82351 13.96 MB/sec execute 149 sec latency 18822.745 ms 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa157fa640 1 -- 172.21.15.77:0/1776830648 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a3f6e720a0 con 0x55a3f6f90040 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa157fa640 10 client.19794 handle_command_reply: tid=0 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 client.19794 shutdown 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 msgr2=0x55a3f6f8dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ffa24035000 tx=0x7ffa240095c0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 shutdown_connections 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55a3f6f90040 0x55a3f6f92460 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f6f14650 0x55a3f6f8dfe0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a3f6f05840 0x55a3f6f19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa157fa640 0 client.19794 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 --2- 172.21.15.77:0/1776830648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a3f6f04f20 0x55a3f6f18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 >> 172.21.15.77:0/1776830648 conn(0x55a3f6f015f0 msgr2=0x55a3f6f11cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 shutdown_connections 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 1 -- 172.21.15.77:0/1776830648 wait complete. 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.648+0000 7ffa35fc5740 20 client.19794 trim_cache size 0 max 16384 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:30:28.651 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 68, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "9de00ec3": { 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 68, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 142.9, 2024-09-15T22:30:28.652 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:28.653 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:28.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.649+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 msgr2=0x7ffa180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.649+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 0x7ffa180a58f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006010 tx=0x7ffa24009900 comp rx=0 tx=0).stop 2024-09-15T22:30:28.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.649+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 msgr2=0x7ffa2c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:28.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.649+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b4a0 tx=0x7ffa1c00b960 comp rx=0 tx=0).stop 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 shutdown_connections 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa180a3430 0x7ffa180a58f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c106a60 0x7ffa2c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa2c101400 0x7ffa2c198a10 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 --2- 172.21.15.77:0/945807477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa2c100a60 0x7ffa2c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 >> 172.21.15.77:0/945807477 conn(0x7ffa2c0fc780 msgr2=0x7ffa2c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 shutdown_connections 2024-09-15T22:30:28.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:28.650+0000 7ffa33bec640 1 -- 172.21.15.77:0/945807477 wait complete. 2024-09-15T22:30:29.383 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9de00ec3', total seconds elapsed: 26.2 2024-09-15T22:30:29.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 82818 14.02 MB/sec execute 150 sec latency 18895.294 ms 2024-09-15T22:30:30.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:29 smithi077 ceph-mon[30980]: pgmap v569: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 526 KiB/s wr, 21 op/s 2024-09-15T22:30:30.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:29 smithi196 ceph-mon[35109]: pgmap v569: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 526 KiB/s wr, 21 op/s 2024-09-15T22:30:30.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:29 smithi117 ceph-mon[35021]: pgmap v569: 129 pgs: 129 active+clean; 1.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 526 KiB/s wr, 21 op/s 2024-09-15T22:30:30.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 83397 14.08 MB/sec execute 151 sec latency 49.615 ms 2024-09-15T22:30:31.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 84061 14.24 MB/sec execute 152 sec latency 47.585 ms 2024-09-15T22:30:32.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:31 smithi077 ceph-mon[30980]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-09-15T22:30:32.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:31 smithi196 ceph-mon[35109]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-09-15T22:30:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:31 smithi117 ceph-mon[35021]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-09-15T22:30:32.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 84617 14.35 MB/sec execute 153 sec latency 366.605 ms 2024-09-15T22:30:33.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 85286 14.53 MB/sec execute 154 sec latency 373.267 ms 2024-09-15T22:30:34.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:33 smithi077 ceph-mon[30980]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:34.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:33 smithi196 ceph-mon[35109]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:34.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:33 smithi117 ceph-mon[35021]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:34.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 85771 14.67 MB/sec execute 155 sec latency 56.527 ms 2024-09-15T22:30:35.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 86042 14.63 MB/sec execute 156 sec latency 134.389 ms 2024-09-15T22:30:36.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:35 smithi077 ceph-mon[30980]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:35 smithi196 ceph-mon[35109]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:35 smithi117 ceph-mon[35021]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-09-15T22:30:36.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 86319 14.63 MB/sec execute 157 sec latency 606.136 ms 2024-09-15T22:30:37.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 86739 14.64 MB/sec execute 158 sec latency 1606.171 ms 2024-09-15T22:30:38.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:37 smithi077 ceph-mon[30980]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.5 MiB/s wr, 290 op/s 2024-09-15T22:30:38.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:37 smithi196 ceph-mon[35109]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.5 MiB/s wr, 290 op/s 2024-09-15T22:30:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:37 smithi117 ceph-mon[35021]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.5 MiB/s wr, 290 op/s 2024-09-15T22:30:38.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 87324 14.73 MB/sec execute 159 sec latency 2606.205 ms 2024-09-15T22:30:39.028 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:30:39.345 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:39.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 87841 14.89 MB/sec execute 160 sec latency 3606.236 ms 2024-09-15T22:30:40.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:39 smithi077 ceph-mon[30980]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-15T22:30:40.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:39 smithi196 ceph-mon[35109]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-15T22:30:40.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:39 smithi117 ceph-mon[35021]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-15T22:30:40.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 88067 14.82 MB/sec execute 161 sec latency 4606.271 ms 2024-09-15T22:30:41.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 88356 14.83 MB/sec execute 162 sec latency 4788.661 ms 2024-09-15T22:30:41.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- 172.21.15.77:0/2542728329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 msgr2=0x7f354c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 --2- 172.21.15.77:0/2542728329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c107b80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f353c009930 tx=0x7f353c02a420 comp rx=0 tx=0).stop 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- 172.21.15.77:0/2542728329 shutdown_connections 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 --2- 172.21.15.77:0/2542728329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 0x7f354c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 --2- 172.21.15.77:0/2542728329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c107b80 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 --2- 172.21.15.77:0/2542728329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- 172.21.15.77:0/2542728329 >> 172.21.15.77:0/2542728329 conn(0x7f354c0fc7d0 msgr2=0x7f354c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- 172.21.15.77:0/2542728329 shutdown_connections 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- 172.21.15.77:0/2542728329 wait complete. 2024-09-15T22:30:41.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 Processor -- start 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.205+0000 7f3550a13640 1 -- start start 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 0x7f354c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112090 con 0x7f354c103740 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f354c111f10 con 0x7f354c108290 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3550a13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112210 con 0x7f354c102d90 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f354a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f354a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57578/0 (socket says 172.21.15.77:57578) 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f354a59c640 1 -- 172.21.15.77:0/1669180650 learned_addr learned my addr 172.21.15.77:0/1669180650 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f354ad9d640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 0x7f354c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 -- 172.21.15.77:0/1669180650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 msgr2=0x7f354c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 -- 172.21.15.77:0/1669180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 msgr2=0x7f354c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 0x7f354c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 -- 172.21.15.77:0/1669180650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c19d5c0 con 0x7f354c103740 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f3549d9b640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c198b50 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f353c009230 tx=0x7f353c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f35437fe640 1 -- 172.21.15.77:0/1669180650 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c03f070 con 0x7f354c103740 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.206+0000 7f35437fe640 1 -- 172.21.15.77:0/1669180650 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353c0044c0 con 0x7f354c103740 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.207+0000 7f35437fe640 1 -- 172.21.15.77:0/1669180650 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c035070 con 0x7f354c103740 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.207+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c19d850 con 0x7f354c103740 2024-09-15T22:30:41.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.207+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c19dcb0 con 0x7f354c103740 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f35437fe640 1 -- 172.21.15.77:0/1669180650 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f353c03b460 con 0x7f354c103740 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 Processor -- start 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 -- start start 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffae9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598ffa422e0 con 0x5598ffadc4a0 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5598ff9e9ce0 con 0x5598ffae7680 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3552dec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffa3e2a0 con 0x5598ffadcdc0 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37064/0 (socket says 172.21.15.77:37064) 2024-09-15T22:30:41.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f354ad9d640 1 -- 172.21.15.77:0/1567744914 learned_addr learned my addr 172.21.15.77:0/1567744914 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f354a59c640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.210+0000 7f3549d9b640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffae9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f35437fe640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 0x7f35380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 -- 172.21.15.77:0/1567744914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 msgr2=0x5598ffae9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffae9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 -- 172.21.15.77:0/1567744914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 msgr2=0x5598ffadc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 -- 172.21.15.77:0/1567744914 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ff9ce440 con 0x5598ffadcdc0 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354ad9d640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f3549d9b640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffae9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:41.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f35437fe640 1 -- 172.21.15.77:0/1669180650 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f353c0f05b0 con 0x7f354c103740 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354ad9d640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 0x7f35380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f354a59c640 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae7140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3534009cf0 tx=0x7f3534011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87df8700a8e8eee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f3540ff9640 1 -- 172.21.15.77:0/1567744914 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534005220 con 0x5598ffadcdc0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f3540ff9640 1 -- 172.21.15.77:0/1567744914 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3534004c80 con 0x5598ffadcdc0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.211+0000 7f3540ff9640 1 -- 172.21.15.77:0/1567744914 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534019940 con 0x5598ffadcdc0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- 172.21.15.77:0/1567744914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 msgr2=0x5598ffae7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f354ad9d640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 0x7f35380a5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3544005f40 tx=0x7f3544004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae7140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3534009cf0 tx=0x7f3534011810 comp rx=0 tx=0).stop 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- 172.21.15.77:0/1567744914 shutdown_connections 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffae9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae7140 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 --2- 172.21.15.77:0/1567744914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadc4a0 0x5598ffadc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- 172.21.15.77:0/1567744914 >> 172.21.15.77:0/1567744914 conn(0x5598ffad8a30 msgr2=0x5598ffad8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- 172.21.15.77:0/1567744914 shutdown_connections 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- 172.21.15.77:0/1567744914 wait complete. 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.212+0000 7f3552dec740 1 -- start start 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 Processor -- start 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadc4a0 0x5598ffae6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598ffaee660 con 0x5598ffae7680 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5598ff9fd050 con 0x5598ffadc4a0 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffaee7e0 con 0x5598ffadcdc0 2024-09-15T22:30:41.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37070/0 (socket says 172.21.15.77:37070) 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3552dec740 10 client.0 _ll_register_callbacks cb 0x5598ffa3d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 -- 172.21.15.77:0/133459153 learned_addr learned my addr 172.21.15.77:0/133459153 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354a59c640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadc4a0 0x5598ffae6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f3549d9b640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 -- 172.21.15.77:0/133459153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 msgr2=0x5598ffae4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 -- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadc4a0 msgr2=0x5598ffae6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadc4a0 0x5598ffae6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.213+0000 7f354ad9d640 1 -- 172.21.15.77:0/133459153 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ff7b33c0 con 0x5598ffae7680 2024-09-15T22:30:41.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f354ad9d640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3544005fd0 tx=0x7f3544004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74dd3746fb71a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544011840 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f35440119e0 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3534004c80 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3544011be0 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 10 client.19806 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5598ffafa010 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3534004c80 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f354401c3c0 con 0x5598ffae7680 2024-09-15T22:30:41.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 10 client.19806 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 10 client.19806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 20 client.19806 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffafb4d0 0x5598ffafd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 4 client.19806 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.214+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598ffa4f850 con 0x5598ffafb4d0 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.215+0000 7f354a59c640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffafb4d0 0x5598ffafd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.215+0000 7f354a59c640 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffafb4d0 0x5598ffafd8c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2dff02e4cdb12e server_cookie=15e46b9a37178245 in_seq=0 out_seq=0 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.215+0000 7f3531ffb640 10 client.19806 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3531ffb640 1 -- 172.21.15.77:0/133459153 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598ffa4f850 con 0x5598ffafb4d0 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3531ffb640 10 client.19806 handle_command_reply: tid=0 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 client.19806 shutdown 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 msgr2=0x5598ffae4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:42.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3544005fd0 tx=0x7f3544004ab0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 shutdown_connections 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffafb4d0 0x5598ffafd8c0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffae7680 0x5598ffae4740 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3531ffb640 0 client.19806 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadcdc0 0x5598ffae4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 --2- 172.21.15.77:0/133459153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadc4a0 0x5598ffae6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 >> 172.21.15.77:0/133459153 conn(0x5598ffad8a30 msgr2=0x5598ffade760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 shutdown_connections 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 1 -- 172.21.15.77:0/133459153 wait complete. 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.216+0000 7f3552dec740 20 client.19806 trim_cache size 0 max 16384 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.240+0000 7f3552dec740 1 Processor -- start 2024-09-15T22:30:42.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.240+0000 7f3552dec740 1 -- start start 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 0x5598ffae4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffb030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1acdc0 con 0x5598ffadcdc0 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ae900 con 0x5598ffae7680 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3552dec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ad490 con 0x5598ffadc4a0 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3549d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 0x5598ffae4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57614/0 (socket says 172.21.15.77:57614) 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f3549d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 0x5598ffae4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37078/0 (socket says 172.21.15.77:37078) 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 -- 172.21.15.77:0/3798489066 learned_addr learned my addr 172.21.15.77:0/3798489066 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 -- 172.21.15.77:0/3798489066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 msgr2=0x5598ffb030f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:30:42.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffb030f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 -- 172.21.15.77:0/3798489066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 msgr2=0x5598ffae4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 0x5598ffae4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.241+0000 7f354a59c640 1 -- 172.21.15.77:0/3798489066 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ffafdc30 con 0x5598ffadc4a0 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.242+0000 7f354a59c640 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5598ffad9890 tx=0x7f3534005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7149dacf8a532aee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.242+0000 7f3533fff640 1 -- 172.21.15.77:0/3798489066 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534005200 con 0x5598ffadc4a0 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.242+0000 7f3533fff640 1 -- 172.21.15.77:0/3798489066 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35340053a0 con 0x5598ffadc4a0 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.242+0000 7f3533fff640 1 -- 172.21.15.77:0/3798489066 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534019930 con 0x5598ffadc4a0 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 -- 172.21.15.77:0/3798489066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 msgr2=0x5598ffae3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5598ffad9890 tx=0x7f3534005f90 comp rx=0 tx=0).stop 2024-09-15T22:30:42.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 -- 172.21.15.77:0/3798489066 shutdown_connections 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffae7680 0x5598ffb030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffadcdc0 0x5598ffae4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 --2- 172.21.15.77:0/3798489066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffadc4a0 0x5598ffae3140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 -- 172.21.15.77:0/3798489066 >> 172.21.15.77:0/3798489066 conn(0x5598ffad8a30 msgr2=0x5598ffae6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 -- 172.21.15.77:0/3798489066 shutdown_connections 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.243+0000 7f3552dec740 1 -- 172.21.15.77:0/3798489066 wait complete. 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.244+0000 7f3552dec740 1 -- start start 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.244+0000 7f3552dec740 1 Processor -- start 2024-09-15T22:30:42.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.244+0000 7f3552dec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 0x5598ffaf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.244+0000 7f3552dec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 0x5598ffaecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.244+0000 7f3552dec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffaed620 0x5598ffaf1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3552dec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ae230 con 0x5598ffaed620 2024-09-15T22:30:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3552dec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1adb60 con 0x5598ffadcdc0 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3552dec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1aadc0 con 0x5598ffae7680 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3549d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 0x5598ffaecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3549d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 0x5598ffaecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57630/0 (socket says 172.21.15.77:57630) 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3549d9b640 1 -- 172.21.15.77:0/2236909946 learned_addr learned my addr 172.21.15.77:0/2236909946 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f354ad9d640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffaed620 0x5598ffaf1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f354a59c640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 0x5598ffaf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.245+0000 7f3552dec740 10 client.0 _ll_register_callbacks cb 0x7f3544009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 -- 172.21.15.77:0/2236909946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 msgr2=0x5598ffaecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 0x5598ffaecf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 -- 172.21.15.77:0/2236909946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffaed620 msgr2=0x5598ffaf1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffaed620 0x5598ffaf1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 -- 172.21.15.77:0/2236909946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ffaedd90 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f354a59c640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 0x5598ffaf4320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5598ffae75b0 tx=0x7f3534009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bad45a2cac0aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.246+0000 7f3531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.247+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353403a020 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.247+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f3534004880 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.247+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35340053a0 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.248+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3534004a80 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.248+0000 7f3552dec740 10 client.28595 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.248+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5598ffb6adf0 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.248+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35340053a0 con 0x5598ffadcdc0 2024-09-15T22:30:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f35340194b0 con 0x5598ffadcdc0 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 10 client.28595 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 10 client.28595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 20 client.28595 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffb6b170 0x5598ffb6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 4 client.28595 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598ffa49d60 con 0x5598ffb6b170 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.249+0000 7f3549d9b640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffb6b170 0x5598ffb6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.252+0000 7f3549d9b640 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffb6b170 0x5598ffb6d560 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e3d2b816b0bcd1 server_cookie=23431538054071a8 in_seq=0 out_seq=0 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.252+0000 7f3531ffb640 10 client.28595 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.260+0000 7f3531ffb640 1 -- 172.21.15.77:0/2236909946 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5598ffa49d60 con 0x5598ffb6b170 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.260+0000 7f3531ffb640 10 client.28595 handle_command_reply: tid=0 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.260+0000 7f3552dec740 1 client.28595 shutdown 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:30:42.009 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.260+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 msgr2=0x5598ffaf4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.260+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 0x5598ffaf4320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5598ffae75b0 tx=0x7f3534009900 comp rx=0 tx=0).stop 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 shutdown_connections 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598ffb6b170 0x5598ffb6d560 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598ffaed620 0x5598ffaf1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffae7680 0x5598ffaecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 --2- 172.21.15.77:0/2236909946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5598ffadcdc0 0x5598ffaf4320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 >> 172.21.15.77:0/2236909946 conn(0x5598ffad8a30 msgr2=0x5598ffad7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3531ffb640 0 client.28595 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 shutdown_connections 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 1 -- 172.21.15.77:0/2236909946 wait complete. 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.261+0000 7f3552dec740 20 client.28595 trim_cache size 0 max 16384 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 msgr2=0x7f35380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 0x7f35380a5870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3544005f40 tx=0x7f3544004a40 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 msgr2=0x7f354c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c198b50 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f353c009230 tx=0x7f353c0040c0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 shutdown_connections 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35380a33b0 0x7f35380a5870 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354c108290 0x7f354c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f354c103740 0x7f354c198b50 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 --2- 172.21.15.77:0/1669180650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f354c102d90 0x7f354c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 >> 172.21.15.77:0/1669180650 conn(0x7f354c0fc7d0 msgr2=0x7f354c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 shutdown_connections 2024-09-15T22:30:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:41.262+0000 7f3550a13640 1 -- 172.21.15.77:0/1669180650 wait complete. 2024-09-15T22:30:42.022 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:41 smithi077 ceph-mon[30980]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 471 op/s 2024-09-15T22:30:42.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:41 smithi196 ceph-mon[35109]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 471 op/s 2024-09-15T22:30:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:41 smithi117 ceph-mon[35021]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.9 MiB/s wr, 471 op/s 2024-09-15T22:30:42.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 88767 14.84 MB/sec execute 163 sec latency 195.383 ms 2024-09-15T22:30:42.819 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:30:42.820 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:30:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:43.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 89329 14.97 MB/sec execute 164 sec latency 193.956 ms 2024-09-15T22:30:44.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:43 smithi077 ceph-mon[30980]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:44.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:43 smithi196 ceph-mon[35109]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:44.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:43 smithi117 ceph-mon[35021]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:44.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 -- 172.21.15.77:0/3320942199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 msgr2=0x7f544010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:44.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 --2- 172.21.15.77:0/3320942199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f544010da40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5438009be0 tx=0x7f543802a750 comp rx=0 tx=0).stop 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 -- 172.21.15.77:0/3320942199 shutdown_connections 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 --2- 172.21.15.77:0/3320942199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f544010da40 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 --2- 172.21.15.77:0/3320942199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440102570 0x7f54401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 --2- 172.21.15.77:0/3320942199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440100e20 0x7f5440101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 -- 172.21.15.77:0/3320942199 >> 172.21.15.77:0/3320942199 conn(0x7f54400fc7d0 msgr2=0x7f54400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 -- 172.21.15.77:0/3320942199 shutdown_connections 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.496+0000 7f5444bac640 1 -- 172.21.15.77:0/3320942199 wait complete. 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 Processor -- start 2024-09-15T22:30:44.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 -- start start 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440100e20 0x7f54401993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440102570 0x7f54401998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5440110f80 con 0x7f5440102ef0 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5440110e00 con 0x7f5440100e20 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.497+0000 7f5444bac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5440111100 con 0x7f5440102570 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37102/0 (socket says 172.21.15.77:37102) 2024-09-15T22:30:44.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 -- 172.21.15.77:0/379657925 learned_addr learned my addr 172.21.15.77:0/379657925 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440100e20 0x7f54401993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f5435d9b640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440102570 0x7f54401998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 -- 172.21.15.77:0/379657925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440102570 msgr2=0x7f54401998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440102570 0x7f54401998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 -- 172.21.15.77:0/379657925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440100e20 msgr2=0x7f54401993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440100e20 0x7f54401993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 -- 172.21.15.77:0/379657925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401a47b0 con 0x7f5440102ef0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.498+0000 7f543ed9d640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5438009b30 tx=0x7f5438030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.499+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438041020 con 0x7f5440102ef0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.499+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438003e50 con 0x7f5440102ef0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.499+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438036040 con 0x7f5440102ef0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.499+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401a4a40 con 0x7f5440102ef0 2024-09-15T22:30:44.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.499+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a4ed0 con 0x7f5440102ef0 2024-09-15T22:30:44.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.500+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440101200 con 0x7f5440102ef0 2024-09-15T22:30:44.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.502+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5438044020 con 0x7f5440102ef0 2024-09-15T22:30:44.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.503+0000 7f5437fff640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 0x7f542c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:44.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.503+0000 7f543e59c640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 0x7f542c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:44.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.504+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f54380f0be0 con 0x7f5440102ef0 2024-09-15T22:30:44.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.504+0000 7f543e59c640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 0x7f542c0a58f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5430009530 tx=0x7f5430009260 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:44.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.508+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543804b020 con 0x7f5440102ef0 2024-09-15T22:30:44.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 89809 15.07 MB/sec execute 165 sec latency 143.831 ms 2024-09-15T22:30:44.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.797+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5440109090 con 0x7f5440102ef0 2024-09-15T22:30:44.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.798+0000 7f5437fff640 1 -- 172.21.15.77:0/379657925 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f54380bd230 con 0x7f5440102ef0 2024-09-15T22:30:44.801 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:30:44.801 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:30:44.801 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 msgr2=0x7f542c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 0x7f542c0a58f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5430009530 tx=0x7f5430009260 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 msgr2=0x7f5440199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5438009b30 tx=0x7f5438030c50 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 shutdown_connections 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f542c0a3430 0x7f542c0a58f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5440102ef0 0x7f5440199e70 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5440102570 0x7f54401998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 --2- 172.21.15.77:0/379657925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5440100e20 0x7f54401993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:44.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.800+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 >> 172.21.15.77:0/379657925 conn(0x7f54400fc7d0 msgr2=0x7f54401068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.801+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 shutdown_connections 2024-09-15T22:30:44.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:44.801+0000 7f5444bac640 1 -- 172.21.15.77:0/379657925 wait complete. 2024-09-15T22:30:45.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 90274 15.10 MB/sec execute 166 sec latency 169.757 ms 2024-09-15T22:30:45.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:30:45.996 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:46.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:45 smithi077 ceph-mon[30980]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:46.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:45 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/379657925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:46.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:45 smithi196 ceph-mon[35109]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:46.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:45 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/379657925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:46.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:45 smithi117 ceph-mon[35021]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.9 MiB/s wr, 385 op/s 2024-09-15T22:30:46.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:45 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/379657925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:30:46.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 -- 172.21.15.77:0/3191492473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81008f0 msgr2=0x7f66b8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 --2- 172.21.15.77:0/3191492473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81008f0 0x7f66b8100d50 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009930 tx=0x7f66ac02a420 comp rx=0 tx=0).stop 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 -- 172.21.15.77:0/3191492473 shutdown_connections 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 --2- 172.21.15.77:0/3191492473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b8101290 0x7f66b810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 --2- 172.21.15.77:0/3191492473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81008f0 0x7f66b8100d50 secure :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009930 tx=0x7f66ac02a420 comp rx=0 tx=0).stop 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 --2- 172.21.15.77:0/3191492473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b81068f0 0x7f66b8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.589+0000 7f66be5ee640 1 -- 172.21.15.77:0/3191492473 >> 172.21.15.77:0/3191492473 conn(0x7f66b80fc630 msgr2=0x7f66b80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.590+0000 7f66be5ee640 1 -- 172.21.15.77:0/3191492473 shutdown_connections 2024-09-15T22:30:46.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.591+0000 7f66be5ee640 1 -- 172.21.15.77:0/3191492473 wait complete. 2024-09-15T22:30:46.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.592+0000 7f66be5ee640 1 Processor -- start 2024-09-15T22:30:46.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.592+0000 7f66be5ee640 1 -- start start 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 0x7f66b80fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 0x7f66b80fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b80fe940 0x7f66b80fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8110d40 con 0x7f66b81068f0 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8110bc0 con 0x7f66b8101290 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66be5ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8110ec0 con 0x7f66b80fe940 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 0x7f66b80fd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 0x7f66b80fd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37120/0 (socket says 172.21.15.77:37120) 2024-09-15T22:30:46.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.593+0000 7f66b77fe640 1 -- 172.21.15.77:0/2217340949 learned_addr learned my addr 172.21.15.77:0/2217340949 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:46.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.594+0000 7f66b7fff640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 0x7f66b80fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66bcb8b640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b80fe940 0x7f66b80fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66b7fff640 1 -- 172.21.15.77:0/2217340949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b80fe940 msgr2=0x7f66b80fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66b7fff640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b80fe940 0x7f66b80fd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66b7fff640 1 -- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 msgr2=0x7f66b80fd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66b7fff640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 0x7f66b80fd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.595+0000 7f66b7fff640 1 -- 172.21.15.77:0/2217340949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b81ab1f0 con 0x7f66b8101290 2024-09-15T22:30:46.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.597+0000 7f66b7fff640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 0x7f66b80fccb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d6d0 tx=0x7f66a800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.602+0000 7f66b57fa640 1 -- 172.21.15.77:0/2217340949 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8017070 con 0x7f66b8101290 2024-09-15T22:30:46.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 90641 15.14 MB/sec execute 167 sec latency 185.528 ms 2024-09-15T22:30:46.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.602+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b81ab4e0 con 0x7f66b8101290 2024-09-15T22:30:46.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.602+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b81ab9f0 con 0x7f66b8101290 2024-09-15T22:30:46.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.605+0000 7f66c09c7740 1 Processor -- start 2024-09-15T22:30:46.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.606+0000 7f66b57fa640 1 -- 172.21.15.77:0/2217340949 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a8009820 con 0x7f66b8101290 2024-09-15T22:30:46.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.606+0000 7f66b57fa640 1 -- 172.21.15.77:0/2217340949 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004d00 con 0x7f66b8101290 2024-09-15T22:30:46.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.607+0000 7f66c09c7740 1 -- start start 2024-09-15T22:30:46.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.607+0000 7f66b57fa640 1 -- 172.21.15.77:0/2217340949 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f66a8010460 con 0x7f66b8101290 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.607+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.607+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884437bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.607+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x558884444910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66c09c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55888439d2f0 con 0x558884438cf0 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66c09c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558884344cf0 con 0x558884442520 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66c09c7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588843992b0 con 0x558884437780 2024-09-15T22:30:46.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66bcb8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66bcb8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37136/0 (socket says 172.21.15.77:37136) 2024-09-15T22:30:46.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.608+0000 7f66bcb8b640 1 -- 172.21.15.77:0/2661815302 learned_addr learned my addr 172.21.15.77:0/2661815302 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:46.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.609+0000 7f66b77fe640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x558884444910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.609+0000 7f66b7fff640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884437bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b57fa640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 0x7f66a00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 -- 172.21.15.77:0/2661815302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 msgr2=0x558884437bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884437bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 -- 172.21.15.77:0/2661815302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 msgr2=0x558884437240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 -- 172.21.15.77:0/2661815302 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588843293a0 con 0x558884442520 2024-09-15T22:30:46.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66bcb8b640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:46.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66b77fe640 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x558884444910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02a990 tx=0x7f66ac004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b014c53106ebd4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.610+0000 7f66bcb8b640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 0x7f66a00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.611+0000 7f669e7fc640 1 -- 172.21.15.77:0/2661815302 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac03f070 con 0x558884442520 2024-09-15T22:30:46.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.611+0000 7f66b57fa640 1 -- 172.21.15.77:0/2217340949 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f66a80d09d0 con 0x7f66b8101290 2024-09-15T22:30:46.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.611+0000 7f669e7fc640 1 -- 172.21.15.77:0/2661815302 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ac004430 con 0x558884442520 2024-09-15T22:30:46.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.612+0000 7f66c09c7740 1 -- 172.21.15.77:0/2661815302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 msgr2=0x558884444910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.612+0000 7f66c09c7740 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x558884444910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02a990 tx=0x7f66ac004040 comp rx=0 tx=0).stop 2024-09-15T22:30:46.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.613+0000 7f66bcb8b640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 0x7f66a00a5a80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66b0012640 tx=0x7f66b00096c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.617+0000 7f66c09c7740 1 -- 172.21.15.77:0/2661815302 shutdown_connections 2024-09-15T22:30:46.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.617+0000 7f66c09c7740 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x558884444910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.617+0000 7f66c09c7740 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884437bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.617+0000 7f66c09c7740 1 --2- 172.21.15.77:0/2661815302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x558884437240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.617+0000 7f66c09c7740 1 -- 172.21.15.77:0/2661815302 >> 172.21.15.77:0/2661815302 conn(0x558884400310 msgr2=0x558884433e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.619+0000 7f66c09c7740 1 -- 172.21.15.77:0/2661815302 shutdown_connections 2024-09-15T22:30:46.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.619+0000 7f66c09c7740 1 -- 172.21.15.77:0/2661815302 wait complete. 2024-09-15T22:30:46.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.620+0000 7f66c09c7740 1 -- start start 2024-09-15T22:30:46.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.620+0000 7f66c09c7740 1 Processor -- start 2024-09-15T22:30:46.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x55888443f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884442520 0x55888443f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558884445260 con 0x558884442520 2024-09-15T22:30:46.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57688/0 (socket says 172.21.15.77:57688) 2024-09-15T22:30:46.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66bcb8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884442520 0x55888443f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66b7fff640 1 -- 172.21.15.77:0/1196792307 learned_addr learned my addr 172.21.15.77:0/1196792307 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:46.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558884358060 con 0x558884438cf0 2024-09-15T22:30:46.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.621+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588844453e0 con 0x558884437780 2024-09-15T22:30:46.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66c09c7740 10 client.0 _ll_register_callbacks cb 0x5588840ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:46.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66b7fff640 1 -- 172.21.15.77:0/1196792307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 msgr2=0x55888443f020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66b7fff640 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x55888443f020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66b7fff640 1 -- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884442520 msgr2=0x55888443f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66b7fff640 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884442520 0x55888443f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.622+0000 7f66b7fff640 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55888410e3c0 con 0x558884437780 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.623+0000 7f66b7fff640 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a8099290 tx=0x7f66a8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e339dfae4c4e540 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.623+0000 7f66977fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:46.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.623+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8020040 con 0x558884437780 2024-09-15T22:30:46.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.623+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ac004430 con 0x558884437780 2024-09-15T22:30:46.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.623+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f66a809bd10 con 0x558884437780 2024-09-15T22:30:46.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.633+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f66a809c220 con 0x558884437780 2024-09-15T22:30:46.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.633+0000 7f66c09c7740 10 client.28522 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:46.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.634+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558884450c50 con 0x558884437780 2024-09-15T22:30:46.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a809cfc0 con 0x558884437780 2024-09-15T22:30:46.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f66a8098610 con 0x558884437780 2024-09-15T22:30:46.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 10 client.28522 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:46.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 10 client.28522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 20 client.28522 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558884452110 0x5588844545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 4 client.28522 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f66ac004430 con 0x558884452110 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.635+0000 7f66b77fe640 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558884452110 0x5588844545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.636+0000 7f66b77fe640 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558884452110 0x5588844545b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b479750d520484b7 server_cookie=2c2c72fe03b1e808 in_seq=0 out_seq=0 2024-09-15T22:30:46.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.636+0000 7f66977fe640 10 client.28522 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:30:46.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.637+0000 7f66977fe640 1 -- 172.21.15.77:0/1196792307 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f66ac004430 con 0x558884452110 2024-09-15T22:30:46.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.637+0000 7f66977fe640 10 client.28522 handle_command_reply: tid=0 2024-09-15T22:30:46.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.637+0000 7f66c09c7740 1 client.28522 shutdown 2024-09-15T22:30:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 msgr2=0x558884441d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a8099290 tx=0x7f66a8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 shutdown_connections 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x558884452110 0x5588844545b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884442520 0x55888443f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x55888443f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.639+0000 7f66c09c7740 1 --2- 172.21.15.77:0/1196792307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884441d70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.640+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 >> 172.21.15.77:0/1196792307 conn(0x558884400310 msgr2=0x558884439ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.640+0000 7f66977fe640 0 client.28522 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:30:46.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.640+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 shutdown_connections 2024-09-15T22:30:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.642+0000 7f66c09c7740 1 -- 172.21.15.77:0/1196792307 wait complete. 2024-09-15T22:30:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.643+0000 7f66c09c7740 20 client.28522 trim_cache size 0 max 16384 2024-09-15T22:30:46.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.667+0000 7f66c09c7740 1 Processor -- start 2024-09-15T22:30:46.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.669+0000 7f66c09c7740 1 -- start start 2024-09-15T22:30:46.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.669+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.669+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x558884443910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.669+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37152/0 (socket says 172.21.15.77:37152) 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66b7fff640 1 -- 172.21.15.77:0/852963942 learned_addr learned my addr 172.21.15.77:0/852963942 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66c09c7740 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884442520 0x558884443e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66a400fff0 con 0x558884437780 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4011b30 con 0x558884438cf0 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66b77fe640 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x558884443910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.670+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40106c0 con 0x558884442520 2024-09-15T22:30:46.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.671+0000 7f66b7fff640 1 -- 172.21.15.77:0/852963942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884442520 msgr2=0x558884443e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.671+0000 7f66b7fff640 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884442520 0x558884443e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.671+0000 7f66b7fff640 1 -- 172.21.15.77:0/852963942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 msgr2=0x558884443910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.671+0000 7f66b7fff640 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x558884443910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.671+0000 7f66b7fff640 1 -- 172.21.15.77:0/852963942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558884434350 con 0x558884437780 2024-09-15T22:30:46.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.672+0000 7f66b7fff640 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f66a8098d20 tx=0x7f66a801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a381fa1ac6909011 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.673+0000 7f669dffb640 1 -- 172.21.15.77:0/852963942 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8020040 con 0x558884437780 2024-09-15T22:30:46.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.673+0000 7f669dffb640 1 -- 172.21.15.77:0/852963942 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a809cfc0 con 0x558884437780 2024-09-15T22:30:46.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.673+0000 7f669dffb640 1 -- 172.21.15.77:0/852963942 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a809c8c0 con 0x558884437780 2024-09-15T22:30:46.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.675+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 msgr2=0x55888444e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.675+0000 7f66c09c7740 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f66a8098d20 tx=0x7f66a801e000 comp rx=0 tx=0).stop 2024-09-15T22:30:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 shutdown_connections 2024-09-15T22:30:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884442520 0x558884443e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884438cf0 0x558884443910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 --2- 172.21.15.77:0/852963942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884437780 0x55888444e580 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 >> 172.21.15.77:0/852963942 conn(0x558884400310 msgr2=0x55888443efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 shutdown_connections 2024-09-15T22:30:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 -- 172.21.15.77:0/852963942 wait complete. 2024-09-15T22:30:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.677+0000 7f66c09c7740 1 -- start start 2024-09-15T22:30:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.678+0000 7f66c09c7740 1 Processor -- start 2024-09-15T22:30:46.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.678+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.678+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x5588844599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66c09c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x5588844496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57708/0 (socket says 172.21.15.77:57708) 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 -- 172.21.15.77:0/3135434172 learned_addr learned my addr 172.21.15.77:0/3135434172 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4011460 con 0x558884438cf0 2024-09-15T22:30:46.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4010d90 con 0x558884442520 2024-09-15T22:30:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66a400dff0 con 0x558884437780 2024-09-15T22:30:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66c09c7740 10 client.0 _ll_register_callbacks cb 0x7f66a8004ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66bcb8b640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x5588844496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 -- 172.21.15.77:0/3135434172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 msgr2=0x5588844496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x5588844496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 -- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 msgr2=0x5588844599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x5588844599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.679+0000 7f66b7fff640 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558884449c30 con 0x558884437780 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.680+0000 7f66b7fff640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f66a809c5c0 tx=0x7f66a8009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30d911b532d0796 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.680+0000 7f66977fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.680+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80a0c70 con 0x558884437780 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.680+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a809cfc0 con 0x558884437780 2024-09-15T22:30:46.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.680+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f66a80a1e00 con 0x558884437780 2024-09-15T22:30:46.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.681+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f66a80bb2b0 con 0x558884437780 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.681+0000 7f66c09c7740 10 client.28525 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.681+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558884449c30 con 0x558884437780 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.681+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a809cfc0 con 0x558884437780 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.681+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f66a80a2000 con 0x558884437780 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 10 client.28525 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 10 client.28525 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 20 client.28525 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5588844bce90 0x5588844bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 4 client.28525 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:46.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588843aa860 con 0x5588844bce90 2024-09-15T22:30:46.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.682+0000 7f66b77fe640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5588844bce90 0x5588844bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:46.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.683+0000 7f66977fe640 10 client.28525 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:30:46.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.684+0000 7f66b77fe640 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5588844bce90 0x5588844bf280 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9cbcee25fca7055 server_cookie=28cdb3b10e6a3151 in_seq=0 out_seq=0 2024-09-15T22:30:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.684+0000 7f66977fe640 1 -- 172.21.15.77:0/3135434172 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588843aa860 con 0x5588844bce90 2024-09-15T22:30:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.685+0000 7f66977fe640 10 client.28525 handle_command_reply: tid=0 2024-09-15T22:30:46.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.685+0000 7f66c09c7740 1 client.28525 shutdown 2024-09-15T22:30:46.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.685+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 msgr2=0x558884459470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.685+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f66a809c5c0 tx=0x7f66a8009d00 comp rx=0 tx=0).stop 2024-09-15T22:30:46.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 shutdown_connections 2024-09-15T22:30:46.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5588844bce90 0x5588844bf280 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558884442520 0x5588844496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558884438cf0 0x5588844599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 --2- 172.21.15.77:0/3135434172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558884437780 0x558884459470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 >> 172.21.15.77:0/3135434172 conn(0x558884400310 msgr2=0x55888443d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66977fe640 0 client.28525 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 shutdown_connections 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.686+0000 7f66c09c7740 1 -- 172.21.15.77:0/3135434172 wait complete. 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.687+0000 7f66c09c7740 20 client.28525 trim_cache size 0 max 16384 2024-09-15T22:30:46.690 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 msgr2=0x7f66a00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 0x7f66a00a5a80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66b0012640 tx=0x7f66b00096c0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 msgr2=0x7f66b80fccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 0x7f66b80fccb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d6d0 tx=0x7f66a800db90 comp rx=0 tx=0).stop 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 shutdown_connections 2024-09-15T22:30:46.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f66a00a35c0 0x7f66a00a5a80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b80fe940 0x7f66b80fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66b81068f0 0x7f66b80fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 --2- 172.21.15.77:0/2217340949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8101290 0x7f66b80fccb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 >> 172.21.15.77:0/2217340949 conn(0x7f66b80fc630 msgr2=0x7f66b806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 shutdown_connections 2024-09-15T22:30:46.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:46.688+0000 7f66be5ee640 1 -- 172.21.15.77:0/2217340949 wait complete. 2024-09-15T22:30:47.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 91063 15.18 MB/sec execute 168 sec latency 179.384 ms 2024-09-15T22:30:47.648 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:30:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:48.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:47 smithi077 ceph-mon[30980]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 546 op/s 2024-09-15T22:30:48.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:47 smithi196 ceph-mon[35109]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 546 op/s 2024-09-15T22:30:48.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:47 smithi117 ceph-mon[35021]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.9 MiB/s wr, 546 op/s 2024-09-15T22:30:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.447+0000 7f030d5e5640 1 -- 172.21.15.77:0/2815915376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0308101e10 msgr2=0x7f03081021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.447+0000 7f030d5e5640 1 --2- 172.21.15.77:0/2815915376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0308101e10 0x7f03081021f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009900 tx=0x7f02f802a410 comp rx=0 tx=0).stop 2024-09-15T22:30:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 -- 172.21.15.77:0/2815915376 shutdown_connections 2024-09-15T22:30:48.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 --2- 172.21.15.77:0/2815915376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 0x7f030810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 --2- 172.21.15.77:0/2815915376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03081027c0 0x7f0308106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 --2- 172.21.15.77:0/2815915376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0308101e10 0x7f03081021f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 -- 172.21.15.77:0/2815915376 >> 172.21.15.77:0/2815915376 conn(0x7f03080fd8d0 msgr2=0x7f03080ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.448+0000 7f030d5e5640 1 -- 172.21.15.77:0/2815915376 shutdown_connections 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.449+0000 7f030d5e5640 1 -- 172.21.15.77:0/2815915376 wait complete. 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.449+0000 7f030d5e5640 1 Processor -- start 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.449+0000 7f030d5e5640 1 -- start start 2024-09-15T22:30:48.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0308101e10 0x7f0308198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 0x7f030819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0308111f60 con 0x7f03081027c0 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0308111de0 con 0x7f0308106f40 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03081120e0 con 0x7f0308101e10 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37198/0 (socket says 172.21.15.77:37198) 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 -- 172.21.15.77:0/3055431580 learned_addr learned my addr 172.21.15.77:0/3055431580 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03077fe640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 0x7f030819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f0306ffd640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0308101e10 0x7f0308198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 -- 172.21.15.77:0/3055431580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0308101e10 msgr2=0x7f0308198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0308101e10 0x7f0308198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 -- 172.21.15.77:0/3055431580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 msgr2=0x7f030819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 0x7f030819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 -- 172.21.15.77:0/3055431580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030819d410 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f03067fc640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00bd30 tx=0x7f02fc009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f02effff640 1 -- 172.21.15.77:0/3055431580 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004020 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030819d700 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.450+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0308074420 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.451+0000 7f02effff640 1 -- 172.21.15.77:0/3055431580 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc004c80 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.451+0000 7f02effff640 1 -- 172.21.15.77:0/3055431580 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc021060 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.453+0000 7f02effff640 1 -- 172.21.15.77:0/3055431580 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f02fc005350 con 0x7f03081027c0 2024-09-15T22:30:48.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.453+0000 7f030f9be740 1 Processor -- start 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.453+0000 7f030f9be740 1 -- start start 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 0x563e26ef2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26efd370 0x563e26eff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563e26e582f0 con 0x563e26ef25d0 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e26dffcf0 con 0x563e26ef3b40 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f030f9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e26e542b0 con 0x563e26efd370 2024-09-15T22:30:48.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f02effff640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26efd370 0x563e26eff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37214/0 (socket says 172.21.15.77:37214) 2024-09-15T22:30:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f03077fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 0x563e26ef2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 -- 172.21.15.77:0/2068156792 learned_addr learned my addr 172.21.15.77:0/2068156792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f03077fe640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f02effff640 1 -- 172.21.15.77:0/3055431580 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f02fc0d3cd0 con 0x7f03081027c0 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 -- 172.21.15.77:0/2068156792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26efd370 msgr2=0x563e26eff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26efd370 0x563e26eff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 -- 172.21.15.77:0/2068156792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 msgr2=0x563e26ef2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 0x563e26ef2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f0306ffd640 1 -- 172.21.15.77:0/2068156792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e26de43a0 con 0x563e26ef25d0 2024-09-15T22:30:48.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.454+0000 7f03077fe640 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 0x563e26ef2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.455+0000 7f03077fe640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 0x7f02f00a5870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0300005fd0 tx=0x7f0300009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.455+0000 7f0306ffd640 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009900 tx=0x7f02f8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff85c763549139 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.455+0000 7f02ed7fa640 1 -- 172.21.15.77:0/2068156792 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f803f070 con 0x563e26ef25d0 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.455+0000 7f02ed7fa640 1 -- 172.21.15.77:0/2068156792 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8004030 con 0x563e26ef25d0 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.455+0000 7f02ed7fa640 1 -- 172.21.15.77:0/2068156792 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8035070 con 0x563e26ef25d0 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- 172.21.15.77:0/2068156792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 msgr2=0x563e26ef2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009900 tx=0x7f02f8005dd0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- 172.21.15.77:0/2068156792 shutdown_connections 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26efd370 0x563e26eff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26ef2a10 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 --2- 172.21.15.77:0/2068156792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef3b40 0x563e26ef2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- 172.21.15.77:0/2068156792 >> 172.21.15.77:0/2068156792 conn(0x563e26ebb310 msgr2=0x563e26eeec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- 172.21.15.77:0/2068156792 shutdown_connections 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- 172.21.15.77:0/2068156792 wait complete. 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 -- start start 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.456+0000 7f030f9be740 1 Processor -- start 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef25d0 0x563e26efcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26efd370 0x563e26efa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563e26f00010 con 0x563e26efd370 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e26e13060 con 0x563e26ef25d0 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e26f00190 con 0x563e26ef3b40 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f030f9be740 10 client.0 _ll_register_callbacks cb 0x563e26b668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:48.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57740/0 (socket says 172.21.15.77:57740) 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 -- 172.21.15.77:0/3940474626 learned_addr learned my addr 172.21.15.77:0/3940474626 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f0306ffd640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef25d0 0x563e26efcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03077fe640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26efd370 0x563e26efa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 -- 172.21.15.77:0/3940474626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef25d0 msgr2=0x563e26efcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef25d0 0x563e26efcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 -- 172.21.15.77:0/3940474626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26efd370 msgr2=0x563e26efa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26efd370 0x563e26efa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.457+0000 7f03067fc640 1 -- 172.21.15.77:0/3940474626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e26bc93c0 con 0x563e26ef3b40 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f03067fc640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02fc09f920 tx=0x7f02fc06f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35618bbc58dcd6e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc09fc20 con 0x563e26ef3b40 2024-09-15T22:30:48.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f02fc0d1310 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f8004030 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02fc0a3fd0 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 10 client.28537 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e26f0ba30 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8004030 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f02fc09f070 con 0x563e26ef3b40 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 10 client.28537 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 10 client.28537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 20 client.28537 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f0ce90 0x563e26f0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 4 client.28537 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e26e65860 con 0x563e26f0ce90 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.458+0000 7f0306ffd640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f0ce90 0x563e26f0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.459+0000 7f02eaff5640 10 client.28537 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.459+0000 7f0306ffd640 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f0ce90 0x563e26f0f2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d924530c5b22e47 server_cookie=fcd0ce6b6e008309 in_seq=0 out_seq=0 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.459+0000 7f02eaff5640 1 -- 172.21.15.77:0/3940474626 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e26e65860 con 0x563e26f0ce90 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.459+0000 7f02eaff5640 10 client.28537 handle_command_reply: tid=0 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 client.28537 shutdown 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 msgr2=0x563e26ef9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02fc09f920 tx=0x7f02fc06f840 comp rx=0 tx=0).stop 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 shutdown_connections 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f0ce90 0x563e26f0f2b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26efd370 0x563e26efa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26ef9f40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 --2- 172.21.15.77:0/3940474626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26ef25d0 0x563e26efcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 >> 172.21.15.77:0/3940474626 conn(0x563e26ebb310 msgr2=0x563e26ef4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f02eaff5640 0 client.28537 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:30:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 shutdown_connections 2024-09-15T22:30:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 1 -- 172.21.15.77:0/3940474626 wait complete. 2024-09-15T22:30:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.460+0000 7f030f9be740 20 client.28537 trim_cache size 0 max 16384 2024-09-15T22:30:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 Processor -- start 2024-09-15T22:30:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 -- start start 2024-09-15T22:30:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26eeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02f400fda0 con 0x563e26ef25d0 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02f40118e0 con 0x563e26efd370 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.477+0000 7f030f9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4010470 con 0x563e26ef3b40 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f0306ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57742/0 (socket says 172.21.15.77:57742) 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 -- 172.21.15.77:0/195089874 learned_addr learned my addr 172.21.15.77:0/195089874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f0306ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f09310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37232/0 (socket says 172.21.15.77:37232) 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03077fe640 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26eeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 -- 172.21.15.77:0/195089874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 msgr2=0x563e26eeec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26eeec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 -- 172.21.15.77:0/195089874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 msgr2=0x563e26f09310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f09310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 -- 172.21.15.77:0/195089874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e26eef180 con 0x563e26ef3b40 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f03067fc640 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02fc09c030 tx=0x7f02fc09e130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d696fd364b3b81dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.478+0000 7f02ecff9640 1 -- 172.21.15.77:0/195089874 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc09fc20 con 0x563e26ef3b40 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f02ecff9640 1 -- 172.21.15.77:0/195089874 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc0d5040 con 0x563e26ef3b40 2024-09-15T22:30:48.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f02ecff9640 1 -- 172.21.15.77:0/195089874 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc09c290 con 0x563e26ef3b40 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 -- 172.21.15.77:0/195089874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 msgr2=0x563e26eee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02fc09c030 tx=0x7f02fc09e130 comp rx=0 tx=0).stop 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 -- 172.21.15.77:0/195089874 shutdown_connections 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26eeec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26eee700 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 --2- 172.21.15.77:0/195089874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f09310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.479+0000 7f030f9be740 1 -- 172.21.15.77:0/195089874 >> 172.21.15.77:0/195089874 conn(0x563e26ebb310 msgr2=0x563e26ef9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- 172.21.15.77:0/195089874 shutdown_connections 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- 172.21.15.77:0/195089874 wait complete. 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- start start 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 Processor -- start 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f05d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26f01840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4011210 con 0x563e26ef25d0 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4010b40 con 0x563e26efd370 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02f400dda0 con 0x563e26ef3b40 2024-09-15T22:30:48.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f030f9be740 10 client.0 _ll_register_callbacks cb 0x7f02fc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f0306ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f05d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f03067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57756/0 (socket says 172.21.15.77:57756) 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.480+0000 7f03067fc640 1 -- 172.21.15.77:0/693061317 learned_addr learned my addr 172.21.15.77:0/693061317 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 -- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 msgr2=0x563e26f010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 -- 172.21.15.77:0/693061317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 msgr2=0x563e26f01840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26f01840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 -- 172.21.15.77:0/693061317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e26f76b50 con 0x563e26ef25d0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f03067fc640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f0306ffd640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f05d80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006010 tx=0x7f02f8032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576bec967f44c746 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f02eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:48.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8035050 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02fc0d5040 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f02f80317c0 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.481+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02f803f040 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 10 client.19848 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e26f77d10 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc0d5040 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f02f80046c0 con 0x563e26ef25d0 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 10 client.19848 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:48.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 10 client.19848 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 20 client.19848 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f78000 0x563e26f7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 4 client.19848 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.482+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e26e5fd70 con 0x563e26f78000 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f03067fc640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f78000 0x563e26f7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f03067fc640 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f78000 0x563e26f7a3f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd9d4588ea0e9800 server_cookie=49814c88e98b8c99 in_seq=0 out_seq=0 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f02eaff5640 10 client.19848 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f02eaff5640 1 -- 172.21.15.77:0/693061317 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e26e5fd70 con 0x563e26f78000 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f02eaff5640 10 client.19848 handle_command_reply: tid=0 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 client.19848 shutdown 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 msgr2=0x563e26f05d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f05d80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006010 tx=0x7f02f8032c30 comp rx=0 tx=0).stop 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 shutdown_connections 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x563e26f78000 0x563e26f7a3f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e26efd370 0x563e26f01840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e26ef3b40 0x563e26f010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 --2- 172.21.15.77:0/693061317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563e26ef25d0 0x563e26f05d80 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 >> 172.21.15.77:0/693061317 conn(0x563e26ebb310 msgr2=0x563e26ef9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.483+0000 7f02eaff5640 0 client.19848 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:30:48.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 shutdown_connections 2024-09-15T22:30:48.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030f9be740 1 -- 172.21.15.77:0/693061317 wait complete. 2024-09-15T22:30:48.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030f9be740 20 client.19848 trim_cache size 0 max 16384 2024-09-15T22:30:48.494 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 msgr2=0x7f02f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 0x7f02f00a5870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0300005fd0 tx=0x7f0300009500 comp rx=0 tx=0).stop 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 msgr2=0x7f03081989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00bd30 tx=0x7f02fc009610 comp rx=0 tx=0).stop 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 shutdown_connections 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f02f00a33b0 0x7f02f00a5870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0308106f40 0x7f030819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03081027c0 0x7f03081989a0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 --2- 172.21.15.77:0/3055431580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0308101e10 0x7f0308198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:48.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.484+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 >> 172.21.15.77:0/3055431580 conn(0x7f03080fd8d0 msgr2=0x7f03080fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:48.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.485+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 shutdown_connections 2024-09-15T22:30:48.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:48.485+0000 7f030d5e5640 1 -- 172.21.15.77:0/3055431580 wait complete. 2024-09-15T22:30:48.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 91475 15.23 MB/sec execute 169 sec latency 180.683 ms 2024-09-15T22:30:49.381 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:30:49.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 91817 15.20 MB/sec execute 170 sec latency 162.350 ms 2024-09-15T22:30:49.692 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:50.023 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:49 smithi077 ceph-mon[30980]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.7 MiB/s wr, 351 op/s 2024-09-15T22:30:50.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:49 smithi196 ceph-mon[35109]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.7 MiB/s wr, 351 op/s 2024-09-15T22:30:50.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:49 smithi117 ceph-mon[35021]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.7 MiB/s wr, 351 op/s 2024-09-15T22:30:50.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 -- 172.21.15.77:0/1036654644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 msgr2=0x7fed8c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 --2- 172.21.15.77:0/1036654644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c10da40 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fed84009ca0 tx=0x7fed8402a950 comp rx=0 tx=0).stop 2024-09-15T22:30:50.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 -- 172.21.15.77:0/1036654644 shutdown_connections 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 --2- 172.21.15.77:0/1036654644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c10da40 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 --2- 172.21.15.77:0/1036654644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 --2- 172.21.15.77:0/1036654644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 -- 172.21.15.77:0/1036654644 >> 172.21.15.77:0/1036654644 conn(0x7fed8c0fc7d0 msgr2=0x7fed8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 -- 172.21.15.77:0/1036654644 shutdown_connections 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 -- 172.21.15.77:0/1036654644 wait complete. 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.248+0000 7fed9275c640 1 Processor -- start 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 -- start start 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c110f80 con 0x7fed8c102ef0 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c110e00 con 0x7fed8c100e20 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed9275c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c111100 con 0x7fed8c102570 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed90cf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed90cf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54854/0 (socket says 172.21.15.77:54854) 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed90cf9640 1 -- 172.21.15.77:0/2710907633 learned_addr learned my addr 172.21.15.77:0/2710907633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.249+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 -- 172.21.15.77:0/2710907633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 msgr2=0x7fed8c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 -- 172.21.15.77:0/2710907633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 msgr2=0x7fed8c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 -- 172.21.15.77:0/2710907633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c1a69d0 con 0x7fed8c102ef0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed90cf9640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fed84004af0 tx=0x7fed8402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed897fa640 1 -- 172.21.15.77:0/2710907633 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84034040 con 0x7fed8c102ef0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed897fa640 1 -- 172.21.15.77:0/2710907633 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed84030920 con 0x7fed8c102ef0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c1a6c60 con 0x7fed8c102ef0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed897fa640 1 -- 172.21.15.77:0/2710907633 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84004630 con 0x7fed8c102ef0 2024-09-15T22:30:50.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.250+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1a7190 con 0x7fed8c102ef0 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.253+0000 7fed897fa640 1 -- 172.21.15.77:0/2710907633 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fed84037020 con 0x7fed8c102ef0 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 Processor -- start 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 -- start start 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 0x559d82305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82310370 0x559d82312760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559d8226b2f0 con 0x559d823055d0 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d82212cf0 con 0x559d82306b40 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed94b35740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d822672b0 con 0x559d82310370 2024-09-15T22:30:50.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54860/0 (socket says 172.21.15.77:54860) 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.254+0000 7fed8bfff640 1 -- 172.21.15.77:0/3834076355 learned_addr learned my addr 172.21.15.77:0/3834076355 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 0x559d82305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82310370 0x559d82312760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed897fa640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 0x7fed740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed90cf9640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 0x7fed740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 -- 172.21.15.77:0/3834076355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82310370 msgr2=0x559d82312760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82310370 0x559d82312760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 -- 172.21.15.77:0/3834076355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 msgr2=0x559d82305a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8b7fe640 1 -- 172.21.15.77:0/3834076355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d821f73a0 con 0x559d82306b40 2024-09-15T22:30:50.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.255+0000 7fed8bfff640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.256+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 0x559d82305090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed800058c0 tx=0x7fed80005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38eae0bca51ed60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.256+0000 7fed897fa640 1 -- 172.21.15.77:0/2710907633 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fed840f0ce0 con 0x7fed8c102ef0 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.256+0000 7fed727fc640 1 -- 172.21.15.77:0/3834076355 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8001cad0 con 0x559d82306b40 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.256+0000 7fed727fc640 1 -- 172.21.15.77:0/3834076355 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed80005500 con 0x559d82306b40 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.256+0000 7fed727fc640 1 -- 172.21.15.77:0/3834076355 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8001e0d0 con 0x559d82306b40 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 -- 172.21.15.77:0/3834076355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 msgr2=0x559d82305090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed90cf9640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 0x7fed740a58f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed840ee260 tx=0x7fed840a7040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 0x559d82305090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed800058c0 tx=0x7fed80005850 comp rx=0 tx=0).stop 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 -- 172.21.15.77:0/3834076355 shutdown_connections 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82310370 0x559d82312760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82305a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 --2- 172.21.15.77:0/3834076355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82306b40 0x559d82305090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 -- 172.21.15.77:0/3834076355 >> 172.21.15.77:0/3834076355 conn(0x559d822ce310 msgr2=0x559d82301c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 -- 172.21.15.77:0/3834076355 shutdown_connections 2024-09-15T22:30:50.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.257+0000 7fed94b35740 1 -- 172.21.15.77:0/3834076355 wait complete. 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.258+0000 7fed94b35740 1 -- start start 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.258+0000 7fed94b35740 1 Processor -- start 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.258+0000 7fed94b35740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 0x559d8230d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559d82313010 con 0x559d82306b40 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d82226060 con 0x559d82310370 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559d82313190 con 0x559d823055d0 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed94b35740 10 client.0 _ll_register_callbacks cb 0x559d81f798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49706/0 (socket says 172.21.15.77:49706) 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2483336359 learned_addr learned my addr 172.21.15.77:0/2483336359 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2483336359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 msgr2=0x559d8230d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 0x559d8230d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2483336359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 msgr2=0x559d8230d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 0x559d8230d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2483336359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d81fdc3c0 con 0x559d823055d0 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.259+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 0x559d8230d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed8001ef20 tx=0x7fed80004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd89020f65ac5d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:50.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80004030 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fed800041d0 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed80005500 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed80039050 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed94b35740 10 client.28555 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d8231eb10 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.260+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed80005500 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fed8001e5c0 con 0x559d823055d0 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 10 client.28555 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 10 client.28555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 20 client.28555 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d82322250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 4 client.28555 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:50.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d82278860 con 0x559d8231fe60 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d82322250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.261+0000 7fed6b7fe640 10 client.28555 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.262+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d82322250 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74fcfdba46f987a8 server_cookie=6133afc28be301ac in_seq=0 out_seq=0 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.262+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2483336359 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d82278860 con 0x559d8231fe60 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.262+0000 7fed6b7fe640 10 client.28555 handle_command_reply: tid=0 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.262+0000 7fed94b35740 1 client.28555 shutdown 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 msgr2=0x559d8230fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed8001ef20 tx=0x7fed80004ac0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 shutdown_connections 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d82322250 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed6b7fe640 0 client.28555 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:30:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82306b40 0x559d8230d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 --2- 172.21.15.77:0/2483336359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d823055d0 0x559d8230fd50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 >> 172.21.15.77:0/2483336359 conn(0x559d822ce310 msgr2=0x559d82307930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 shutdown_connections 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 1 -- 172.21.15.77:0/2483336359 wait complete. 2024-09-15T22:30:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.263+0000 7fed94b35740 20 client.28555 trim_cache size 0 max 16384 2024-09-15T22:30:50.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 Processor -- start 2024-09-15T22:30:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 -- start start 2024-09-15T22:30:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82308120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82308660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1acd90 con 0x559d823055d0 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1ae8d0 con 0x559d82310370 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.280+0000 7fed94b35740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1ad460 con 0x559d82306b40 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82308120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82308660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:33092/0 (socket says 172.21.15.77:33092) 2024-09-15T22:30:50.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2849186545 learned_addr learned my addr 172.21.15.77:0/2849186545 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2849186545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 msgr2=0x559d82308660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82308660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2849186545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 msgr2=0x559d82308120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82308120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.281+0000 7fed8b7fe640 1 -- 172.21.15.77:0/2849186545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d823100c0 con 0x559d82310370 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.282+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed80004a90 tx=0x7fed80000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69ee9099b33085a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.282+0000 7fed71ffb640 1 -- 172.21.15.77:0/2849186545 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8002f040 con 0x559d82310370 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.282+0000 7fed71ffb640 1 -- 172.21.15.77:0/2849186545 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed8001cdb0 con 0x559d82310370 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.282+0000 7fed71ffb640 1 -- 172.21.15.77:0/2849186545 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8001e410 con 0x559d82310370 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- 172.21.15.77:0/2849186545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 msgr2=0x559d8230a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed80004a90 tx=0x7fed80000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- 172.21.15.77:0/2849186545 shutdown_connections 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d82310370 0x559d8230a2d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82308660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 --2- 172.21.15.77:0/2849186545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d823055d0 0x559d82308120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- 172.21.15.77:0/2849186545 >> 172.21.15.77:0/2849186545 conn(0x559d822ce310 msgr2=0x559d82300d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- 172.21.15.77:0/2849186545 shutdown_connections 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- 172.21.15.77:0/2849186545 wait complete. 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.283+0000 7fed94b35740 1 -- start start 2024-09-15T22:30:50.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 Processor -- start 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d823055d0 0x559d82325ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82310370 0x559d82323440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1ae200 con 0x559d82310370 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1adb30 con 0x559d823055d0 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1aadf0 con 0x559d82306b40 2024-09-15T22:30:50.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed94b35740 10 client.0 _ll_register_callbacks cb 0x7fed7c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49736/0 (socket says 172.21.15.77:49736) 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed8bfff640 1 -- 172.21.15.77:0/2708437883 learned_addr learned my addr 172.21.15.77:0/2708437883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d823055d0 0x559d82325ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.284+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82310370 0x559d82323440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 -- 172.21.15.77:0/2708437883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d823055d0 msgr2=0x559d82325ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d823055d0 0x559d82325ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 -- 172.21.15.77:0/2708437883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82310370 msgr2=0x559d82323440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82310370 0x559d82323440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 -- 172.21.15.77:0/2708437883 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d82323c70 con 0x559d82306b40 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed8bfff640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x559d8230c8e0 tx=0x7fed7c012d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1511a0409c19ab68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0124e0 con 0x559d82306b40 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fed7c004030 con 0x559d82306b40 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed8001cdb0 con 0x559d82306b40 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed7c032050 con 0x559d82306b40 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed94b35740 10 client.28561 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:50.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.285+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559d8238a930 con 0x559d82306b40 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed8001cdb0 con 0x559d82306b40 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fed7c02d030 con 0x559d82306b40 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 10 client.28561 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 10 client.28561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 20 client.28561 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d8238d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 4 client.28561 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d82272d70 con 0x559d8231fe60 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d8238d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed6b7fe640 10 client.28561 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:30:50.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed8b7fe640 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d8238d120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d99a76f8f19caed server_cookie=2e2d98f7ae7590b5 in_seq=0 out_seq=0 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed6b7fe640 1 -- 172.21.15.77:0/2708437883 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d82272d70 con 0x559d8231fe60 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed6b7fe640 10 client.28561 handle_command_reply: tid=0 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.286+0000 7fed94b35740 1 client.28561 shutdown 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 msgr2=0x559d82322e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x559d8230c8e0 tx=0x7fed7c012d30 comp rx=0 tx=0).stop 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 shutdown_connections 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x559d8231fe60 0x559d8238d120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559d82310370 0x559d82323440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559d82306b40 0x559d82322e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 --2- 172.21.15.77:0/2708437883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d823055d0 0x559d82325ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 >> 172.21.15.77:0/2708437883 conn(0x559d822ce310 msgr2=0x559d8230e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed6b7fe640 0 client.28561 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:30:50.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 shutdown_connections 2024-09-15T22:30:50.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 1 -- 172.21.15.77:0/2708437883 wait complete. 2024-09-15T22:30:50.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.287+0000 7fed94b35740 20 client.28561 trim_cache size 0 max 16384 2024-09-15T22:30:50.296 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.288+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 msgr2=0x7fed740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.288+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 0x7fed740a58f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed840ee260 tx=0x7fed840a7040 comp rx=0 tx=0).stop 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.288+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 msgr2=0x7fed8c197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.288+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fed84004af0 tx=0x7fed8402f3a0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 shutdown_connections 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed740a3430 0x7fed740a58f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed8c102ef0 0x7fed8c197be0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.297 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c102570 0x7fed8c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 --2- 172.21.15.77:0/2710907633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8c100e20 0x7fed8c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:50.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 >> 172.21.15.77:0/2710907633 conn(0x7fed8c0fc7d0 msgr2=0x7fed8c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:50.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 shutdown_connections 2024-09-15T22:30:50.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:50.289+0000 7fed9275c640 1 -- 172.21.15.77:0/2710907633 wait complete. 2024-09-15T22:30:50.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 92229 15.24 MB/sec execute 171 sec latency 153.049 ms 2024-09-15T22:30:51.210 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:30:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:51.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 92737 15.28 MB/sec execute 172 sec latency 161.908 ms 2024-09-15T22:30:51.927 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:51 smithi077 ceph-mon[30980]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-09-15T22:30:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 -- 172.21.15.77:0/1562638540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc100e20 msgr2=0x7f35dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 --2- 172.21.15.77:0/1562638540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc100e20 0x7f35dc101200 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802a480 comp rx=0 tx=0).stop 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 -- 172.21.15.77:0/1562638540 shutdown_connections 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 --2- 172.21.15.77:0/1562638540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc102ef0 0x7f35dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 --2- 172.21.15.77:0/1562638540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102570 0x7f35dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 --2- 172.21.15.77:0/1562638540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc100e20 0x7f35dc101200 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 -- 172.21.15.77:0/1562638540 >> 172.21.15.77:0/1562638540 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 -- 172.21.15.77:0/1562638540 shutdown_connections 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.018+0000 7f35e3629640 1 -- 172.21.15.77:0/1562638540 wait complete. 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 Processor -- start 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 -- start start 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc100e20 0x7f35dc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102ef0 0x7f35dc1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc110f80 con 0x7f35dc102570 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc110e00 con 0x7f35dc102ef0 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e3629640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc111100 con 0x7f35dc100e20 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54910/0 (socket says 172.21.15.77:54910) 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e0bc4640 1 -- 172.21.15.77:0/2007946809 learned_addr learned my addr 172.21.15.77:0/2007946809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e13c5640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc100e20 0x7f35dc103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.019+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102ef0 0x7f35dc1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 -- 172.21.15.77:0/2007946809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc100e20 msgr2=0x7f35dc103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc100e20 0x7f35dc103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 -- 172.21.15.77:0/2007946809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102ef0 msgr2=0x7f35dc1041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102ef0 0x7f35dc1041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 -- 172.21.15.77:0/2007946809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc1a69c0 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00e9f0 tx=0x7f35cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35d27fc640 1 -- 172.21.15.77:0/2007946809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00cd80 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc1a6cb0 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35d27fc640 1 -- 172.21.15.77:0/2007946809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc010910 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35d27fc640 1 -- 172.21.15.77:0/2007946809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0188f0 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.020+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a71e0 con 0x7f35dc102570 2024-09-15T22:30:52.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.023+0000 7f35d27fc640 1 -- 172.21.15.77:0/2007946809 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f35cc010ac0 con 0x7f35dc102570 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.023+0000 7f35e5a02740 1 Processor -- start 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.023+0000 7f35e5a02740 1 -- start start 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35d27fc640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 0x7f35c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 0x7f35c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35d27fc640 1 -- 172.21.15.77:0/2007946809 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f35cc0d03f0 con 0x7f35dc102570 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657dd2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657ddf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c657d382f0 con 0x55c657ddd370 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c657cdfcf0 con 0x55c657dd25d0 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e5a02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c657d342b0 con 0x55c657dd3b40 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657dd2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49770/0 (socket says 172.21.15.77:49770) 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 -- 172.21.15.77:0/608404192 learned_addr learned my addr 172.21.15.77:0/608404192 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657dd2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:33136/0 (socket says 172.21.15.77:33136) 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657ddf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 -- 172.21.15.77:0/608404192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 msgr2=0x55c657dd2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657dd2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 -- 172.21.15.77:0/608404192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 msgr2=0x55c657ddf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657ddf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e13c5640 1 -- 172.21.15.77:0/608404192 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c657cc43a0 con 0x55c657dd3b40 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.024+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 0x7f35c80a58f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4009aa0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e13c5640 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35d802a9a0 tx=0x7f35d8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8edf1aab7762de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35c77fe640 1 -- 172.21.15.77:0/608404192 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8035030 con 0x55c657dd3b40 2024-09-15T22:30:52.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35c77fe640 1 -- 172.21.15.77:0/608404192 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d8040070 con 0x55c657dd3b40 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35c77fe640 1 -- 172.21.15.77:0/608404192 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80318c0 con 0x55c657dd3b40 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 -- 172.21.15.77:0/608404192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 msgr2=0x55c657dd2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35d802a9a0 tx=0x7f35d8005dd0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 -- 172.21.15.77:0/608404192 shutdown_connections 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657ddf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657dd2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 --2- 172.21.15.77:0/608404192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657dd2090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.025+0000 7f35e5a02740 1 -- 172.21.15.77:0/608404192 >> 172.21.15.77:0/608404192 conn(0x55c657d9b310 msgr2=0x55c657dcec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- 172.21.15.77:0/608404192 shutdown_connections 2024-09-15T22:30:52.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- 172.21.15.77:0/608404192 wait complete. 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- start start 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 Processor -- start 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 0x55c657ddccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 0x55c657dd9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657dda4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c657de0010 con 0x55c657ddd370 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c657cf3060 con 0x55c657dd3b40 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c657de0190 con 0x55c657dd25d0 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.026+0000 7f35e5a02740 10 client.0 _ll_register_callbacks cb 0x55c657a468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e13c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 0x55c657ddccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e1bc6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657dda4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 0x55c657dd9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e13c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 0x55c657ddccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49772/0 (socket says 172.21.15.77:49772) 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e13c5640 1 -- 172.21.15.77:0/1499157636 learned_addr learned my addr 172.21.15.77:0/1499157636 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 -- 172.21.15.77:0/1499157636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 msgr2=0x55c657ddccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 0x55c657ddccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 -- 172.21.15.77:0/1499157636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 msgr2=0x55c657dda4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657dda4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 -- 172.21.15.77:0/1499157636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c657aa93c0 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e0bc4640 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 0x55c657dd9f80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0cd7b0 tx=0x7f35cc09c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cac0778f1a1427 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc09b840 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d8040070 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f35cc09b9e0 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35cc0b5240 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e5a02740 10 client.28661 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.027+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c657deba50 con 0x55c657dd3b40 2024-09-15T22:30:52.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d8040070 con 0x55c657dd3b40 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f35cc0bc0b0 con 0x55c657dd3b40 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 10 client.28661 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 10 client.28661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 20 client.28661 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657def280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 4 client.28661 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c657d45860 con 0x55c657dece40 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.028+0000 7f35e13c5640 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657def280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.029+0000 7f35e13c5640 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657def280 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c859805e3692932b server_cookie=e6cd77ecdbc66ebe in_seq=0 out_seq=0 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.029+0000 7f35c4ff9640 10 client.28661 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35c4ff9640 1 -- 172.21.15.77:0/1499157636 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c657d45860 con 0x55c657dece40 2024-09-15T22:30:52.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35c4ff9640 10 client.28661 handle_command_reply: tid=0 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 client.28661 shutdown 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 msgr2=0x55c657dd9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 0x55c657dd9f80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0cd7b0 tx=0x7f35cc09c4b0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 shutdown_connections 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657def280 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657dda4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd3b40 0x55c657dd9f80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35c4ff9640 0 client.28661 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 --2- 172.21.15.77:0/1499157636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd25d0 0x55c657ddccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 >> 172.21.15.77:0/1499157636 conn(0x55c657d9b310 msgr2=0x55c657dd4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 shutdown_connections 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 1 -- 172.21.15.77:0/1499157636 wait complete. 2024-09-15T22:30:52.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.030+0000 7f35e5a02740 20 client.28661 trim_cache size 0 max 16384 2024-09-15T22:30:52.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 Processor -- start 2024-09-15T22:30:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 -- start start 2024-09-15T22:30:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657dd3b40 0x55c657dd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b1250 con 0x55c657dd3b40 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b2d90 con 0x55c657dd25d0 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e5a02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b1920 con 0x55c657ddd370 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e1bc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.046+0000 7f35e1bc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49780/0 (socket says 172.21.15.77:49780) 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2131269793 learned_addr learned my addr 172.21.15.77:0/2131269793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e13c5640 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2131269793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 msgr2=0x55c657de92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2131269793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657dd3b40 msgr2=0x55c657dd9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657dd3b40 0x55c657dd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2131269793 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c657dda850 con 0x55c657ddd370 2024-09-15T22:30:52.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35d4003b90 tx=0x7f35d4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ffa331fa687803 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.047+0000 7f35c6ffd640 1 -- 172.21.15.77:0/2131269793 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40046d0 con 0x55c657ddd370 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.048+0000 7f35c6ffd640 1 -- 172.21.15.77:0/2131269793 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40120d0 con 0x55c657ddd370 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.048+0000 7f35c6ffd640 1 -- 172.21.15.77:0/2131269793 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4010670 con 0x55c657ddd370 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- 172.21.15.77:0/2131269793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 msgr2=0x55c657dd9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f35d4003b90 tx=0x7f35d4016000 comp rx=0 tx=0).stop 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- 172.21.15.77:0/2131269793 shutdown_connections 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657ddd370 0x55c657dd9d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657dd3b40 0x55c657dd9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2131269793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- 172.21.15.77:0/2131269793 >> 172.21.15.77:0/2131269793 conn(0x55c657d9b310 msgr2=0x55c657dd52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- 172.21.15.77:0/2131269793 shutdown_connections 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- 172.21.15.77:0/2131269793 wait complete. 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 -- start start 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.049+0000 7f35e5a02740 1 Processor -- start 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657de0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657de1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b26c0 con 0x55c657ddd370 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b1ff0 con 0x55c657dd25d0 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1af250 con 0x55c657dd3b40 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e5a02740 10 client.0 _ll_register_callbacks cb 0x7f35cc09bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:52.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657de0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e1bc6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657de1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e0bc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657de0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49796/0 (socket says 172.21.15.77:49796) 2024-09-15T22:30:52.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.050+0000 7f35e0bc4640 1 -- 172.21.15.77:0/2342206384 learned_addr learned my addr 172.21.15.77:0/2342206384 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:52.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2342206384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 msgr2=0x55c657de0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657de0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 msgr2=0x55c657de5c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de5c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 -- 172.21.15.77:0/2342206384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c657de1d30 con 0x55c657ddd370 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e1bc6640 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657de1500 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009e00 tx=0x7f35d400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9d794566944d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400fe40 con 0x55c657ddd370 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f35d4004c50 con 0x55c657ddd370 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.051+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d40120d0 con 0x55c657ddd370 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35d4004e50 con 0x55c657ddd370 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 10 client.19878 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:52.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c657e57bc0 con 0x55c657ddd370 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d4012280 con 0x55c657ddd370 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f35d4031570 con 0x55c657ddd370 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 10 client.19878 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 10 client.19878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 20 client.19878 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657e5a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 4 client.19878 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.052+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f35d40120d0 con 0x55c657dece40 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.053+0000 7f35e13c5640 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657e5a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.053+0000 7f35c4ff9640 10 client.19878 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.053+0000 7f35e13c5640 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657e5a240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f70ea0f9e6a9d00 server_cookie=5d248f82a202eb02 in_seq=0 out_seq=0 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35c4ff9640 1 -- 172.21.15.77:0/2342206384 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f35d40120d0 con 0x55c657dece40 2024-09-15T22:30:52.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35c4ff9640 10 client.19878 handle_command_reply: tid=0 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 client.19878 shutdown 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 msgr2=0x55c657de1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657de1500 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009e00 tx=0x7f35d400f9b0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 shutdown_connections 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c657dece40 0x55c657e5a240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c657ddd370 0x55c657de1500 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35c4ff9640 0 client.19878 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c657dd3b40 0x55c657de0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 --2- 172.21.15.77:0/2342206384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c657dd25d0 0x55c657de5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.054+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 >> 172.21.15.77:0/2342206384 conn(0x55c657d9b310 msgr2=0x55c657dd8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.055+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 shutdown_connections 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.055+0000 7f35e5a02740 1 -- 172.21.15.77:0/2342206384 wait complete. 2024-09-15T22:30:52.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.055+0000 7f35e5a02740 20 client.19878 trim_cache size 0 max 16384 2024-09-15T22:30:52.060 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 msgr2=0x7f35c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 0x7f35c80a58f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4009aa0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 msgr2=0x7f35dc103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00e9f0 tx=0x7f35cc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 shutdown_connections 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f35c80a3430 0x7f35c80a58f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc102ef0 0x7f35dc1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc102570 0x7f35dc103c90 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 --2- 172.21.15.77:0/2007946809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35dc100e20 0x7f35dc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 >> 172.21.15.77:0/2007946809 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 shutdown_connections 2024-09-15T22:30:52.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:52.056+0000 7f35e3629640 1 -- 172.21.15.77:0/2007946809 wait complete. 2024-09-15T22:30:52.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:51 smithi196 ceph-mon[35109]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-09-15T22:30:52.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:51 smithi117 ceph-mon[35021]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 501 op/s 2024-09-15T22:30:52.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 93312 15.39 MB/sec execute 173 sec latency 135.864 ms 2024-09-15T22:30:52.985 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:30:53.267 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:53.606 INFO:tasks.workunit.client.0.smithi077.stdout: 10 93801 15.51 MB/sec execute 174 sec latency 191.256 ms 2024-09-15T22:30:53.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.778+0000 7fd5e4525640 1 -- 172.21.15.77:0/2661450915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc106f10 msgr2=0x7fd5dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.778+0000 7fd5e4525640 1 --2- 172.21.15.77:0/2661450915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc106f10 0x7fd5dc10ea90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009ca0 tx=0x7fd5d402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.779+0000 7fd5e4525640 1 -- 172.21.15.77:0/2661450915 shutdown_connections 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.779+0000 7fd5e4525640 1 --2- 172.21.15.77:0/2661450915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc106f10 0x7fd5dc10ea90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.779+0000 7fd5e4525640 1 --2- 172.21.15.77:0/2661450915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc102790 0x7fd5dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.779+0000 7fd5e4525640 1 --2- 172.21.15.77:0/2661450915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc101de0 0x7fd5dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.779+0000 7fd5e4525640 1 -- 172.21.15.77:0/2661450915 >> 172.21.15.77:0/2661450915 conn(0x7fd5dc0fd8e0 msgr2=0x7fd5dc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.780+0000 7fd5e4525640 1 -- 172.21.15.77:0/2661450915 shutdown_connections 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.780+0000 7fd5e4525640 1 -- 172.21.15.77:0/2661450915 wait complete. 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.781+0000 7fd5e4525640 1 Processor -- start 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.781+0000 7fd5e4525640 1 -- start start 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc102790 0x7fd5dc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc106f10 0x7fd5dc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc114140 con 0x7fd5dc101de0 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc113fc0 con 0x7fd5dc106f10 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e4525640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1142c0 con 0x7fd5dc102790 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e1ac0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc102790 0x7fd5dc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54986/0 (socket says 172.21.15.77:54986) 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 -- 172.21.15.77:0/96971433 learned_addr learned my addr 172.21.15.77:0/96971433 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc106f10 0x7fd5dc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 -- 172.21.15.77:0/96971433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc102790 msgr2=0x7fd5dc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc102790 0x7fd5dc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 -- 172.21.15.77:0/96971433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc106f10 msgr2=0x7fd5dc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc106f10 0x7fd5dc19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.782+0000 7fd5e22c1640 1 -- 172.21.15.77:0/96971433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc19f570 con 0x7fd5dc101de0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e9f0 tx=0x7fd5d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5d37fe640 1 -- 172.21.15.77:0/96971433 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800cd80 con 0x7fd5dc101de0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5d37fe640 1 -- 172.21.15.77:0/96971433 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d8010910 con 0x7fd5dc101de0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5dc19f860 con 0x7fd5dc101de0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5d37fe640 1 -- 172.21.15.77:0/96971433 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8018990 con 0x7fd5dc101de0 2024-09-15T22:30:53.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.783+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5dc1a7140 con 0x7fd5dc101de0 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.786+0000 7fd5d37fe640 1 -- 172.21.15.77:0/96971433 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd5d8010ac0 con 0x7fd5dc101de0 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.786+0000 7fd5e68fe740 1 Processor -- start 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.786+0000 7fd5e68fe740 1 -- start start 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 0x55f957d39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f957c922f0 con 0x55f957d2c5d0 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f957c39cf0 con 0x55f957d37370 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f957c8e2b0 con 0x55f957d2db40 2024-09-15T22:30:53.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5d37fe640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 0x7fd5c80a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54988/0 (socket says 172.21.15.77:54988) 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e22c1640 1 -- 172.21.15.77:0/1705395586 learned_addr learned my addr 172.21.15.77:0/1705395586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 0x7fd5c80a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.787+0000 7fd5d37fe640 1 -- 172.21.15.77:0/96971433 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fd5d80d0490 con 0x7fd5dc101de0 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 0x55f957d39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/1705395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 msgr2=0x55f957d39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 0x55f957d39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/1705395586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 msgr2=0x55f957d2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/1705395586 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f957c1e3a0 con 0x55f957d2db40 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 0x55f957d39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:53.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2c090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006010 tx=0x7fd5d4031b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6babfece2348a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 0x7fd5c80a58c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009a60 tx=0x7fd5d4036000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5d0ff9640 1 -- 172.21.15.77:0/1705395586 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d404c070 con 0x55f957d2db40 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5d0ff9640 1 -- 172.21.15.77:0/1705395586 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4036070 con 0x55f957d2db40 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.788+0000 7fd5d0ff9640 1 -- 172.21.15.77:0/1705395586 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4030820 con 0x55f957d2db40 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 -- 172.21.15.77:0/1705395586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 msgr2=0x55f957d2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2c090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006010 tx=0x7fd5d4031b90 comp rx=0 tx=0).stop 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 -- 172.21.15.77:0/1705395586 shutdown_connections 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d37370 0x55f957d39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/1705395586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2c090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.789+0000 7fd5e68fe740 1 -- 172.21.15.77:0/1705395586 >> 172.21.15.77:0/1705395586 conn(0x55f957cf5310 msgr2=0x55f957d28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.790+0000 7fd5e68fe740 1 -- 172.21.15.77:0/1705395586 shutdown_connections 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.790+0000 7fd5e68fe740 1 -- 172.21.15.77:0/1705395586 wait complete. 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.790+0000 7fd5e68fe740 1 -- start start 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.790+0000 7fd5e68fe740 1 Processor -- start 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 0x55f957d36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d34080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d34810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f957d3a010 con 0x55f957d37370 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f957c4d060 con 0x55f957d2db40 2024-09-15T22:30:53.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 0x55f957d36dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 0x55f957d36dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49840/0 (socket says 172.21.15.77:49840) 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e22c1640 1 -- 172.21.15.77:0/965993420 learned_addr learned my addr 172.21.15.77:0/965993420 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d34080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d34810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f957d3a190 con 0x55f957d2c5d0 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.791+0000 7fd5e68fe740 10 client.0 _ll_register_callbacks cb 0x55f9579a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 -- 172.21.15.77:0/965993420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 msgr2=0x55f957d36dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 0x55f957d36dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 -- 172.21.15.77:0/965993420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 msgr2=0x55f957d34810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d34810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 -- 172.21.15.77:0/965993420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f957a033c0 con 0x55f957d2db40 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d34810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d34080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0fdd70 tx=0x7fd5cc013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237bc4c797799fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.798 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.792+0000 7fd5c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc0119c0 con 0x55f957d2db40 2024-09-15T22:30:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fd5cc011b60 con 0x55f957d2db40 2024-09-15T22:30:53.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d4036070 con 0x55f957d2db40 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc011d60 con 0x55f957d2db40 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5e68fe740 10 client.28673 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f957d45b00 con 0x55f957d2db40 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.793+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d4036070 con 0x55f957d2db40 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd5cc01a070 con 0x55f957d2db40 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 10 client.28673 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 10 client.28673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 20 client.28673 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957d46f10 0x55f957d49390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 4 client.28673 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f957c9f860 con 0x55f957d46f10 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.794+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957d46f10 0x55f957d49390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.795+0000 7fd5c5ffb640 10 client.28673 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.795+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957d46f10 0x55f957d49390 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f4dcaa59b68734 server_cookie=e2eedb6d5357ef6b in_seq=0 out_seq=0 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.797+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/965993420 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f957c9f860 con 0x55f957d46f10 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.797+0000 7fd5c5ffb640 10 client.28673 handle_command_reply: tid=0 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.797+0000 7fd5e68fe740 1 client.28673 shutdown 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 msgr2=0x55f957d34080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d34080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0fdd70 tx=0x7fd5cc013b70 comp rx=0 tx=0).stop 2024-09-15T22:30:53.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 shutdown_connections 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957d46f10 0x55f957d49390 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d34810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d34080 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5c5ffb640 0 client.28673 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/965993420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2c5d0 0x55f957d36dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.798+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 >> 172.21.15.77:0/965993420 conn(0x55f957cf5310 msgr2=0x55f957d2e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.799+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 shutdown_connections 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.799+0000 7fd5e68fe740 1 -- 172.21.15.77:0/965993420 wait complete. 2024-09-15T22:30:53.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.799+0000 7fd5e68fe740 20 client.28673 trim_cache size 0 max 16384 2024-09-15T22:30:53.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 Processor -- start 2024-09-15T22:30:53.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 -- start start 2024-09-15T22:30:53.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2c5d0 0x55f957d2c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1b1320 con 0x55f957d37370 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1b2e60 con 0x55f957d2c5d0 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.814+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1b19f0 con 0x55f957d2db40 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55016/0 (socket says 172.21.15.77:55016) 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/3439789352 learned_addr learned my addr 172.21.15.77:0/3439789352 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:53.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2c5d0 0x55f957d2c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/3439789352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 msgr2=0x55f957d2f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/3439789352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2c5d0 msgr2=0x55f957d2c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2c5d0 0x55f957d2c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 -- 172.21.15.77:0/3439789352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f957d49a10 con 0x55f957d37370 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009bf0 tx=0x7fd5d4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e0a12b20e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.815+0000 7fd5c7fff640 1 -- 172.21.15.77:0/3439789352 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4058070 con 0x55f957d37370 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5c7fff640 1 -- 172.21.15.77:0/3439789352 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d40305e0 con 0x55f957d37370 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5c7fff640 1 -- 172.21.15.77:0/3439789352 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d404c040 con 0x55f957d37370 2024-09-15T22:30:53.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 -- 172.21.15.77:0/3439789352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 msgr2=0x55f957d2f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009bf0 tx=0x7fd5d4003e00 comp rx=0 tx=0).stop 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 -- 172.21.15.77:0/3439789352 shutdown_connections 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d37370 0x55f957d2f660 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d2db40 0x55f957d2f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/3439789352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2c5d0 0x55f957d2c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 -- 172.21.15.77:0/3439789352 >> 172.21.15.77:0/3439789352 conn(0x55f957cf5310 msgr2=0x55f957d23a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.816+0000 7fd5e68fe740 1 -- 172.21.15.77:0/3439789352 shutdown_connections 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 -- 172.21.15.77:0/3439789352 wait complete. 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 -- start start 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 Processor -- start 2024-09-15T22:30:53.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 0x55f957d31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1b2790 con 0x55f957d2c5d0 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1b20c0 con 0x55f957d2db40 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5dc1af320 con 0x55f957d37370 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e68fe740 10 client.0 _ll_register_callbacks cb 0x7fd5cc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e22c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55018/0 (socket says 172.21.15.77:55018) 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e22c1640 1 -- 172.21.15.77:0/48382931 learned_addr learned my addr 172.21.15.77:0/48382931 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.817+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 0x55f957d31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 -- 172.21.15.77:0/48382931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 msgr2=0x55f957d31290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 0x55f957d31290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 -- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 msgr2=0x55f957d30d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d30d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 -- 172.21.15.77:0/48382931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f957d31a90 con 0x55f957d2c5d0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e2ac2640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 0x55f957d31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e22c1640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8098db0 tx=0x7fd5d8098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4eb477933ec11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80d0170 con 0x55f957d2c5d0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7fd5d80cdad0 con 0x55f957d2c5d0 2024-09-15T22:30:53.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d40305e0 con 0x55f957d2c5d0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5d80cdcd0 con 0x55f957d2c5d0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e68fe740 10 client.19896 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.818+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f957db1b40 con 0x55f957d2c5d0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d40305e0 con 0x55f957d2c5d0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd5d8018c30 con 0x55f957d2c5d0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 10 client.19896 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 10 client.19896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 20 client.19896 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957db2360 0x55f957db4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 4 client.19896 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f957c99d70 con 0x55f957db2360 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957db2360 0x55f957db4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.819+0000 7fd5e1ac0640 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957db2360 0x55f957db4780 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c9c4417fc29643 server_cookie=dffbd2fb9bc56716 in_seq=0 out_seq=0 2024-09-15T22:30:53.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.820+0000 7fd5c5ffb640 10 client.19896 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:53.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5c5ffb640 1 -- 172.21.15.77:0/48382931 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f957c99d70 con 0x55f957db2360 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5c5ffb640 10 client.19896 handle_command_reply: tid=0 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 client.19896 shutdown 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 msgr2=0x55f957d3faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8098db0 tx=0x7fd5d8098040 comp rx=0 tx=0).stop 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 shutdown_connections 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f957db2360 0x55f957db4780 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f957d37370 0x55f957d31290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f957d2db40 0x55f957d30d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 --2- 172.21.15.77:0/48382931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f957d2c5d0 0x55f957d3faf0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 >> 172.21.15.77:0/48382931 conn(0x55f957cf5310 msgr2=0x55f957d358e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.821+0000 7fd5c5ffb640 0 client.19896 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 shutdown_connections 2024-09-15T22:30:53.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e68fe740 1 -- 172.21.15.77:0/48382931 wait complete. 2024-09-15T22:30:53.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e68fe740 20 client.19896 trim_cache size 0 max 16384 2024-09-15T22:30:53.831 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 msgr2=0x7fd5c80a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 0x7fd5c80a58c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009a60 tx=0x7fd5d4036000 comp rx=0 tx=0).stop 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 msgr2=0x7fd5dc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.822+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e9f0 tx=0x7fd5d800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 shutdown_connections 2024-09-15T22:30:53.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd5c80a3400 0x7fd5c80a58c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5dc106f10 0x7fd5dc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5dc102790 0x7fd5dc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 --2- 172.21.15.77:0/96971433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5dc101de0 0x7fd5dc19a5c0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 >> 172.21.15.77:0/96971433 conn(0x7fd5dc0fd8e0 msgr2=0x7fd5dc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 shutdown_connections 2024-09-15T22:30:53.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:53.823+0000 7fd5e4525640 1 -- 172.21.15.77:0/96971433 wait complete. 2024-09-15T22:30:54.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:53 smithi077 ceph-mon[30980]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:54.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:53 smithi196 ceph-mon[35109]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:54.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:53 smithi117 ceph-mon[35021]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:54.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 94250 15.57 MB/sec execute 175 sec latency 145.186 ms 2024-09-15T22:30:55.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:54 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:55.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:54 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:55.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:54 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:55.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 94482 15.53 MB/sec execute 176 sec latency 291.561 ms 2024-09-15T22:30:55.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:30:55.728 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:30:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:56.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:55 smithi077 ceph-mon[30980]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:56.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:56.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:56.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:55 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:30:56.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:55 smithi196 ceph-mon[35109]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:56.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:55 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:30:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:55 smithi117 ceph-mon[35021]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.2 MiB/s wr, 309 op/s 2024-09-15T22:30:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:56.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:55 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.582+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1793837230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 msgr2=0x7f7b281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.582+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1793837230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b281067e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009900 tx=0x7f7b1802a410 comp rx=0 tx=0).stop 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1793837230 shutdown_connections 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1793837230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28106f10 0x7f7b2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1793837230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b281067e0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1793837230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28101de0 0x7f7b281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1793837230 >> 172.21.15.77:0/1793837230 conn(0x7f7b280fd8e0 msgr2=0x7f7b280ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1793837230 shutdown_connections 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.583+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1793837230 wait complete. 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 Processor -- start 2024-09-15T22:30:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 -- start start 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 0x7f7b2819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b2819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28106f10 0x7f7b2819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28074b60 con 0x7f7b28102790 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280749e0 con 0x7f7b28101de0 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2ff4d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28074ce0 con 0x7f7b28106f10 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 0x7f7b2819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 0x7f7b2819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:33242/0 (socket says 172.21.15.77:33242) 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2dce9640 1 -- 172.21.15.77:0/1253148419 learned_addr learned my addr 172.21.15.77:0/1253148419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.584+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b2819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28106f10 0x7f7b2819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/1253148419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28106f10 msgr2=0x7f7b2819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28106f10 0x7f7b2819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/1253148419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 msgr2=0x7f7b2819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 0x7f7b2819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/1253148419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2819f6e0 con 0x7f7b28102790 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.585+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b2819b080 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005940 tx=0x7f7b18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.586+0000 7f7b1effd640 1 -- 172.21.15.77:0/1253148419 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1803f030 con 0x7f7b28102790 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.586+0000 7f7b1effd640 1 -- 172.21.15.77:0/1253148419 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b18003ad0 con 0x7f7b28102790 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.586+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2819f970 con 0x7f7b28102790 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.586+0000 7f7b1effd640 1 -- 172.21.15.77:0/1253148419 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18035070 con 0x7f7b28102790 2024-09-15T22:30:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.586+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281a72b0 con 0x7f7b28102790 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.589+0000 7f7b1effd640 1 -- 172.21.15.77:0/1253148419 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7b18042030 con 0x7f7b28102790 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 Processor -- start 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 -- start start 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b1effd640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 0x7f7b140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc41130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55770fb99280 con 0x55770fc3ed40 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55770fb40b10 con 0x55770fc34480 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b32326740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55770fb95240 con 0x55770fc33b60 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 0x7f7b140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2e4ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2d4e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2e4ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49884/0 (socket says 172.21.15.77:49884) 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/997318211 learned_addr learned my addr 172.21.15.77:0/997318211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.590+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/997318211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 msgr2=0x55770fc3e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/997318211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 msgr2=0x55770fc41130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc41130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/997318211 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770fb4d1f0 con 0x55770fc33b60 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b1effd640 1 -- 172.21.15.77:0/1253148419 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f7b180f06a0 con 0x7f7b28102790 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009fd0 tx=0x7f7b20009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c363a1907a615 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 0x7f7b140a58f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009950 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b13fff640 1 -- 172.21.15.77:0/997318211 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011070 con 0x55770fc33b60 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b13fff640 1 -- 172.21.15.77:0/997318211 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b20011210 con 0x55770fc33b60 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.591+0000 7f7b13fff640 1 -- 172.21.15.77:0/997318211 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20005260 con 0x55770fc33b60 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 -- 172.21.15.77:0/997318211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 msgr2=0x55770fc33f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009fd0 tx=0x7f7b20009b80 comp rx=0 tx=0).stop 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 -- 172.21.15.77:0/997318211 shutdown_connections 2024-09-15T22:30:56.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc41130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 --2- 172.21.15.77:0/997318211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc33f40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.592+0000 7f7b32326740 1 -- 172.21.15.77:0/997318211 >> 172.21.15.77:0/997318211 conn(0x55770fc300f0 msgr2=0x55770fc304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.593+0000 7f7b32326740 1 -- 172.21.15.77:0/997318211 shutdown_connections 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.593+0000 7f7b32326740 1 -- 172.21.15.77:0/997318211 wait complete. 2024-09-15T22:30:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.593+0000 7f7b32326740 1 -- start start 2024-09-15T22:30:56.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.593+0000 7f7b32326740 1 Processor -- start 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc3c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc3ed40 0x55770fc3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55770fc41960 con 0x55770fc34480 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55770fb53f10 con 0x55770fc3ed40 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55770fc41ae0 con 0x55770fc33b60 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b32326740 10 client.0 _ll_register_callbacks cb 0x55770fb1ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2d4e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc3c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49898/0 (socket says 172.21.15.77:49898) 2024-09-15T22:30:56.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 -- 172.21.15.77:0/3478917506 learned_addr learned my addr 172.21.15.77:0/3478917506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc3ed40 0x55770fc3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 -- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc3ed40 msgr2=0x55770fc3c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc3ed40 0x55770fc3c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 -- 172.21.15.77:0/3478917506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 msgr2=0x55770fc3c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc3c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 -- 172.21.15.77:0/3478917506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770f90a6a0 con 0x55770fc33b60 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc3c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400dec0 tx=0x7f7b240047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8381d17f71e7b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.594+0000 7f7b117fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24024060 con 0x55770fc33b60 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f7b240125f0 con 0x55770fc33b60 2024-09-15T22:30:56.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20011210 con 0x55770fc33b60 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b240059c0 con 0x55770fc33b60 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 10 client.28606 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55770fc55e00 con 0x55770fc33b60 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b20011210 con 0x55770fc33b60 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7b24013070 con 0x55770fc33b60 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 10 client.28606 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 10 client.28606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 20 client.28606 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fc572c0 0x55770fc59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 4 client.28606 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.595+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55770fba67f0 con 0x55770fc572c0 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.596+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fc572c0 0x55770fc59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.596+0000 7f7b117fa640 10 client.28606 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.596+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fc572c0 0x55770fc59760 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16a430f290ad282 server_cookie=a03ca51ada06fab6 in_seq=0 out_seq=0 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b117fa640 1 -- 172.21.15.77:0/3478917506 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55770fba67f0 con 0x55770fc572c0 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b117fa640 10 client.28606 handle_command_reply: tid=0 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 client.28606 shutdown 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 msgr2=0x55770fc3bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400dec0 tx=0x7f7b240047d0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 shutdown_connections 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fc572c0 0x55770fc59760 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc3ed40 0x55770fc3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc3c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 --2- 172.21.15.77:0/3478917506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc3bc40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b117fa640 0 client.28606 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.598+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 >> 172.21.15.77:0/3478917506 conn(0x55770fc300f0 msgr2=0x55770fc35dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.599+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 shutdown_connections 2024-09-15T22:30:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.599+0000 7f7b32326740 1 -- 172.21.15.77:0/3478917506 wait complete. 2024-09-15T22:30:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.599+0000 7f7b32326740 20 client.28606 trim_cache size 0 max 16384 2024-09-15T22:30:56.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 94986 15.61 MB/sec execute 177 sec latency 165.522 ms 2024-09-15T22:30:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.631+0000 7f7b32326740 1 Processor -- start 2024-09-15T22:30:56.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.632+0000 7f7b32326740 1 -- start start 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.632+0000 7f7b32326740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b32326740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b32326740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b32326740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ace10 con 0x55770fc3ed40 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b32326740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ae950 con 0x55770fc34480 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b32326740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ad4e0 con 0x55770fc33b60 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:49912/0 (socket says 172.21.15.77:49912) 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b2dce9640 1 -- 172.21.15.77:0/1453828420 learned_addr learned my addr 172.21.15.77:0/1453828420 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b2e4ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.633+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/1453828420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 msgr2=0x55770fc365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/1453828420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 msgr2=0x55770fc3daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 -- 172.21.15.77:0/1453828420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770fc2fb90 con 0x55770fc3ed40 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc5f080 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000fe30 tx=0x7f7b2000fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c350a250b87a558 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b137fe640 1 -- 172.21.15.77:0/1453828420 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011240 con 0x55770fc3ed40 2024-09-15T22:30:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b137fe640 1 -- 172.21.15.77:0/1453828420 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b20003970 con 0x55770fc3ed40 2024-09-15T22:30:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.634+0000 7f7b137fe640 1 -- 172.21.15.77:0/1453828420 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200053c0 con 0x55770fc3ed40 2024-09-15T22:30:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.635+0000 7f7b32326740 1 -- 172.21.15.77:0/1453828420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 msgr2=0x55770fc5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.635+0000 7f7b32326740 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc5f080 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000fe30 tx=0x7f7b2000fe90 comp rx=0 tx=0).stop 2024-09-15T22:30:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 -- 172.21.15.77:0/1453828420 shutdown_connections 2024-09-15T22:30:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc3ed40 0x55770fc5f080 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc34480 0x55770fc3daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 --2- 172.21.15.77:0/1453828420 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc33b60 0x55770fc365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 -- 172.21.15.77:0/1453828420 >> 172.21.15.77:0/1453828420 conn(0x55770fc300f0 msgr2=0x55770fc3d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 -- 172.21.15.77:0/1453828420 shutdown_connections 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.636+0000 7f7b32326740 1 -- 172.21.15.77:0/1453828420 wait complete. 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.637+0000 7f7b32326740 1 -- start start 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.637+0000 7f7b32326740 1 Processor -- start 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.638+0000 7f7b32326740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc33b60 0x55770fc4f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.638+0000 7f7b32326740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.638+0000 7f7b32326740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 0x55770fc38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b32326740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ae280 con 0x55770fc34480 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b32326740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281adbb0 con 0x55770fc33b60 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b32326740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281aae30 con 0x55770fc3ed40 2024-09-15T22:30:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b32326740 10 client.0 _ll_register_callbacks cb 0x7f7b240095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55072/0 (socket says 172.21.15.77:55072) 2024-09-15T22:30:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2dce9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc33b60 0x55770fc4f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/2430783089 learned_addr learned my addr 172.21.15.77:0/2430783089 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 0x55770fc38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/2430783089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 msgr2=0x55770fc38840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 0x55770fc38840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc33b60 msgr2=0x55770fc4f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc33b60 0x55770fc4f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2d4e8640 1 -- 172.21.15.77:0/2430783089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770fc39070 con 0x55770fc34480 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.639+0000 7f7b2e4ea640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 0x55770fc38840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.640+0000 7f7b2d4e8640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7b180096c0 tx=0x7f7b18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656d53bac37453b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.640+0000 7f7b117fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.640+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180a1e80 con 0x55770fc34480 2024-09-15T22:30:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.640+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f7b180c06e0 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.640+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20003970 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.641+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b18049040 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.641+0000 7f7b32326740 10 client.19914 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.641+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55770fcc1d50 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.641+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b180bc590 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.641+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7b180b8c00 con 0x55770fc34480 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 10 client.19914 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 10 client.19914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 20 client.19914 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fcc2400 0x55770fcc4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 4 client.19914 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:30:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.642+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7b20003970 con 0x55770fcc2400 2024-09-15T22:30:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.643+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fcc2400 0x55770fcc4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.644+0000 7f7b2dce9640 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fcc2400 0x55770fcc4820 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabf746993ed0374 server_cookie=84395fc31498ca8a in_seq=0 out_seq=0 2024-09-15T22:30:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.644+0000 7f7b117fa640 10 client.19914 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b117fa640 1 -- 172.21.15.77:0/2430783089 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7b20003970 con 0x55770fcc2400 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b117fa640 10 client.19914 handle_command_reply: tid=0 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 client.19914 shutdown 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 msgr2=0x55770fc4fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7b180096c0 tx=0x7f7b18000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 shutdown_connections 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55770fcc2400 0x55770fcc4820 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55770fc3ed40 0x55770fc38840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55770fc34480 0x55770fc4fe90 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b117fa640 0 client.19914 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 --2- 172.21.15.77:0/2430783089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55770fc33b60 0x55770fc4f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 >> 172.21.15.77:0/2430783089 conn(0x55770fc300f0 msgr2=0x55770fc2f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 shutdown_connections 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 1 -- 172.21.15.77:0/2430783089 wait complete. 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.653+0000 7f7b32326740 20 client.19914 trim_cache size 0 max 16384 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:30:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "89111f66-a52c-4be3-8dc5-b80c7a0c9409", 2024-09-15T22:30:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:30:56.657 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 msgr2=0x7f7b140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 0x7f7b140a58f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009950 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 msgr2=0x7f7b2819b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b2819b080 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005940 tx=0x7f7b18004270 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 shutdown_connections 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7b140a3430 0x7f7b140a58f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b28106f10 0x7f7b2819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b28102790 0x7f7b2819b080 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 --2- 172.21.15.77:0/1253148419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b28101de0 0x7f7b2819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.654+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 >> 172.21.15.77:0/1253148419 conn(0x7f7b280fd8e0 msgr2=0x7f7b281057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:56.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.655+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 shutdown_connections 2024-09-15T22:30:56.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:56.655+0000 7f7b2ff4d640 1 -- 172.21.15.77:0/1253148419 wait complete. 2024-09-15T22:30:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:56 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:30:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:56 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:56 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:57.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:56 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:30:57.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:56 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:57.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:56 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:56 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:30:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:56 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:57.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:56 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:57.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 95389 15.63 MB/sec execute 178 sec latency 177.324 ms 2024-09-15T22:30:57.883 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.083 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:57 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:57 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:30:58.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:57 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:30:58.183 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:58.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 95914 15.71 MB/sec execute 179 sec latency 145.063 ms 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 -- 172.21.15.77:0/1807424231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70102dd0 msgr2=0x7f2a701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 --2- 172.21.15.77:0/1807424231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70102dd0 0x7f2a701031b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009cd0 tx=0x7f2a6c02a8d0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 -- 172.21.15.77:0/1807424231 shutdown_connections 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 --2- 172.21.15.77:0/1807424231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70108300 0x7f2a7010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 --2- 172.21.15.77:0/1807424231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70103780 0x7f2a70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 --2- 172.21.15.77:0/1807424231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70102dd0 0x7f2a701031b0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 -- 172.21.15.77:0/1807424231 >> 172.21.15.77:0/1807424231 conn(0x7f2a700fc7d0 msgr2=0x7f2a700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 -- 172.21.15.77:0/1807424231 shutdown_connections 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.732+0000 7f2a77688640 1 -- 172.21.15.77:0/1807424231 wait complete. 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 Processor -- start 2024-09-15T22:30:58.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 -- start start 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 0x7f2a701985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70103780 0x7f2a70198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701120e0 con 0x7f2a70108300 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70111f60 con 0x7f2a70103780 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a77688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112260 con 0x7f2a70102dd0 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a75c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a75c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55082/0 (socket says 172.21.15.77:55082) 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a75c25640 1 -- 172.21.15.77:0/2305936442 learned_addr learned my addr 172.21.15.77:0/2305936442 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a75424640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 0x7f2a701985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.733+0000 7f2a74c23640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70103780 0x7f2a70198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 -- 172.21.15.77:0/2305936442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 msgr2=0x7f2a701985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 0x7f2a701985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 -- 172.21.15.77:0/2305936442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70103780 msgr2=0x7f2a70198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70103780 0x7f2a70198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 -- 172.21.15.77:0/2305936442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7019d570 con 0x7f2a70108300 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75424640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 0x7f2a701985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.734+0000 7f2a75c25640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e970 tx=0x7f2a6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.735+0000 7f2a667fc640 1 -- 172.21.15.77:0/2305936442 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ccc0 con 0x7f2a70108300 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.735+0000 7f2a667fc640 1 -- 172.21.15.77:0/2305936442 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6800ce60 con 0x7f2a70108300 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.735+0000 7f2a667fc640 1 -- 172.21.15.77:0/2305936442 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68010490 con 0x7f2a70108300 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.735+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7019d860 con 0x7f2a70108300 2024-09-15T22:30:58.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.735+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701a5140 con 0x7f2a70108300 2024-09-15T22:30:58.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.738+0000 7f2a667fc640 1 -- 172.21.15.77:0/2305936442 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2a68005020 con 0x7f2a70108300 2024-09-15T22:30:58.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.738+0000 7f2a79a61740 1 Processor -- start 2024-09-15T22:30:58.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 -- start start 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a667fc640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 0x7f2a5c0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7613360 0x55f3a7613740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7622550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3a75792e0 con 0x55f3a7622a90 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3a7520ce0 con 0x55f3a7613360 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a79a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3a75752a0 con 0x55f3a7613c80 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7613360 0x55f3a7613740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a667fc640 1 -- 172.21.15.77:0/2305936442 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f2a680d0460 con 0x7f2a70108300 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55094/0 (socket says 172.21.15.77:55094) 2024-09-15T22:30:58.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 -- 172.21.15.77:0/3832008310 learned_addr learned my addr 172.21.15.77:0/3832008310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a74c23640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 0x7f2a5c0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 -- 172.21.15.77:0/3832008310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 msgr2=0x55f3a7622550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7622550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 -- 172.21.15.77:0/3832008310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7613360 msgr2=0x55f3a7613740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7613360 0x55f3a7613740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.739+0000 7f2a75c25640 1 -- 172.21.15.77:0/3832008310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a7505440 con 0x55f3a7622a90 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.740+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f2a68098e20 tx=0x7f2a68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a41d30dc61f205 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.740+0000 7f2a74c23640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 0x7f2a5c0a5940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058a0 tx=0x7f2a6000b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.740+0000 7f2a5b7fe640 1 -- 172.21.15.77:0/3832008310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800be30 con 0x55f3a7622a90 2024-09-15T22:30:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.740+0000 7f2a5b7fe640 1 -- 172.21.15.77:0/3832008310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6809cb50 con 0x55f3a7622a90 2024-09-15T22:30:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.740+0000 7f2a5b7fe640 1 -- 172.21.15.77:0/3832008310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801dd60 con 0x55f3a7622a90 2024-09-15T22:30:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 -- 172.21.15.77:0/3832008310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 msgr2=0x55f3a7624e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f2a68098e20 tx=0x7f2a68017000 comp rx=0 tx=0).stop 2024-09-15T22:30:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 -- 172.21.15.77:0/3832008310 shutdown_connections 2024-09-15T22:30:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7622a90 0x55f3a7624e80 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7622550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3832008310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7613360 0x55f3a7613740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 -- 172.21.15.77:0/3832008310 >> 172.21.15.77:0/3832008310 conn(0x55f3a760fa30 msgr2=0x55f3a760fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 -- 172.21.15.77:0/3832008310 shutdown_connections 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.741+0000 7f2a79a61740 1 -- 172.21.15.77:0/3832008310 wait complete. 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.742+0000 7f2a79a61740 1 -- start start 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.742+0000 7f2a79a61740 1 Processor -- start 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a76223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a761fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f3a7625660 con 0x55f3a7613c80 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f3a7534050 con 0x55f3a7622a90 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f3a76257e0 con 0x55f3a7613360 2024-09-15T22:30:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a74c23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a74c23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55110/0 (socket says 172.21.15.77:55110) 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a74c23640 1 -- 172.21.15.77:0/3145588536 learned_addr learned my addr 172.21.15.77:0/3145588536 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a79a61740 10 client.0 _ll_register_callbacks cb 0x55f3a7574160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a761fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.743+0000 7f2a75424640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a76223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 -- 172.21.15.77:0/3145588536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 msgr2=0x55f3a76223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a76223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 -- 172.21.15.77:0/3145588536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 msgr2=0x55f3a761f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 -- 172.21.15.77:0/3145588536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a72ea3c0 con 0x55f3a7622a90 2024-09-15T22:30:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a74c23640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a75c25640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a761fc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a6809c610 tx=0x7f2a68016df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28795a69cc5b9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680b5060 con 0x55f3a7622a90 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6809cb50 con 0x55f3a7622a90 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.744+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f2a680d0170 con 0x55f3a7622a90 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.745+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a680a0800 con 0x55f3a7622a90 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.745+0000 7f2a79a61740 10 client.28709 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.745+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3a76161a0 con 0x55f3a7622a90 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.745+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6809cb50 con 0x55f3a7622a90 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2a680b5200 con 0x55f3a7622a90 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 10 client.28709 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 10 client.28709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 20 client.28709 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a76175b0 0x55f3a76199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 4 client.28709 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.746+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a7586850 con 0x55f3a76175b0 2024-09-15T22:30:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.747+0000 7f2a75424640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a76175b0 0x55f3a76199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.748+0000 7f2a75424640 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a76175b0 0x55f3a76199f0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d45e317b9694c9e server_cookie=2bc9d1aaeea1cca5 in_seq=0 out_seq=0 2024-09-15T22:30:58.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.748+0000 7f2a58ff9640 10 client.28709 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.750+0000 7f2a58ff9640 1 -- 172.21.15.77:0/3145588536 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3a7586850 con 0x55f3a76175b0 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a58ff9640 10 client.28709 handle_command_reply: tid=0 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 client.28709 shutdown 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 msgr2=0x55f3a761fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a761fc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a6809c610 tx=0x7f2a68016df0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 shutdown_connections 2024-09-15T22:30:58.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a76175b0 0x55f3a76199f0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a761fc20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a761f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 --2- 172.21.15.77:0/3145588536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a76223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 >> 172.21.15.77:0/3145588536 conn(0x55f3a760fa30 msgr2=0x55f3a7615790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a58ff9640 0 client.28709 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 shutdown_connections 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.751+0000 7f2a79a61740 1 -- 172.21.15.77:0/3145588536 wait complete. 2024-09-15T22:30:58.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.752+0000 7f2a79a61740 20 client.28709 trim_cache size 0 max 16384 2024-09-15T22:30:58.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 Processor -- start 2024-09-15T22:30:58.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 -- start start 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7635fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a76364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701acdd0 con 0x55f3a7613360 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ae910 con 0x55f3a7622a90 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a79a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ad4a0 con 0x55f3a7613c80 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55120/0 (socket says 172.21.15.77:55120) 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 -- 172.21.15.77:0/791565154 learned_addr learned my addr 172.21.15.77:0/791565154 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a74c23640 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7635fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75c25640 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a76364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.777 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 -- 172.21.15.77:0/791565154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 msgr2=0x55f3a7635fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7635fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 -- 172.21.15.77:0/791565154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 msgr2=0x55f3a76364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a76364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.771+0000 7f2a75424640 1 -- 172.21.15.77:0/791565154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a760ea30 con 0x55f3a7613360 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.772+0000 7f2a75424640 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c02ae40 tx=0x7f2a6c02add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f8231bd1b3f2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.772+0000 7f2a5affd640 1 -- 172.21.15.77:0/791565154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c032d40 con 0x55f3a7613360 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.772+0000 7f2a5affd640 1 -- 172.21.15.77:0/791565154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c036040 con 0x55f3a7613360 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.772+0000 7f2a5affd640 1 -- 172.21.15.77:0/791565154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0043a0 con 0x55f3a7613360 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- 172.21.15.77:0/791565154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 msgr2=0x55f3a760e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c02ae40 tx=0x7f2a6c02add0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- 172.21.15.77:0/791565154 shutdown_connections 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a76364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613c80 0x55f3a7635fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.778 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 --2- 172.21.15.77:0/791565154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613360 0x55f3a760e350 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- 172.21.15.77:0/791565154 >> 172.21.15.77:0/791565154 conn(0x55f3a760fa30 msgr2=0x55f3a7621740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- 172.21.15.77:0/791565154 shutdown_connections 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- 172.21.15.77:0/791565154 wait complete. 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 -- start start 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.773+0000 7f2a79a61740 1 Processor -- start 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a7632ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a7633a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ae240 con 0x55f3a7613c80 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701adb70 con 0x55f3a7622a90 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701aadd0 con 0x55f3a7613360 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a79a61740 10 client.0 _ll_register_callbacks cb 0x7f2a68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.779 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55130/0 (socket says 172.21.15.77:55130) 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a75c25640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a7633a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 -- 172.21.15.77:0/1487989632 learned_addr learned my addr 172.21.15.77:0/1487989632 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a75424640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a7632ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 -- 172.21.15.77:0/1487989632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 msgr2=0x55f3a7632ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a7632ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 -- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 msgr2=0x55f3a7633a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a7633a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 -- 172.21.15.77:0/1487989632 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a7697250 con 0x55f3a7613c80 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a75c25640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a7633a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:58.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a74c23640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x55f3a761ef50 tx=0x7f2a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c1e0d15818921d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.774+0000 7f2a58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6002d060 con 0x55f3a7613c80 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9438+0+0 (secure 0 0 0) 0x7f2a6001fbd0 con 0x55f3a7613c80 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c036040 con 0x55f3a7613c80 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2a60034050 con 0x55f3a7613c80 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 10 client.19932 fetch_fsmap learned FSMap version 39 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f3a76985a0 con 0x55f3a7613c80 2024-09-15T22:30:58.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c036040 con 0x55f3a7613c80 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2a6001d3b0 con 0x55f3a7613c80 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 10 client.19932 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 10 client.19932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 20 client.19932 populate_metadata read hostname 'smithi077' 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a7698d50 0x55f3a769b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 4 client.19932 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3a7580d60 con 0x55f3a7698d50 2024-09-15T22:30:58.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.775+0000 7f2a75424640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a7698d50 0x55f3a769b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.776+0000 7f2a58ff9640 10 client.19932 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.776+0000 7f2a75424640 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a7698d50 0x55f3a769b170 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d251857402de57c1 server_cookie=ea9ea6bd8132ee13 in_seq=0 out_seq=0 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.777+0000 7f2a58ff9640 1 -- 172.21.15.77:0/1487989632 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3a7580d60 con 0x55f3a7698d50 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.777+0000 7f2a58ff9640 10 client.19932 handle_command_reply: tid=0 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.777+0000 7f2a79a61740 1 client.19932 shutdown 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.777+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 msgr2=0x55f3a7633530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.777+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x55f3a761ef50 tx=0x7f2a60004730 comp rx=0 tx=0).stop 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 shutdown_connections 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f3a7698d50 0x55f3a769b170 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f3a7622a90 0x55f3a7633a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f3a7613c80 0x55f3a7633530 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 --2- 172.21.15.77:0/1487989632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f3a7613360 0x55f3a7632ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 >> 172.21.15.77:0/1487989632 conn(0x55f3a760fa30 msgr2=0x55f3a7614e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a58ff9640 0 client.19932 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 shutdown_connections 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 1 -- 172.21.15.77:0/1487989632 wait complete. 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.778+0000 7f2a79a61740 20 client.19932 trim_cache size 0 max 16384 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stdout: "89111f66-a52c-4be3-8dc5-b80c7a0c9409": { 2024-09-15T22:30:58.785 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "89111f66-a52c-4be3-8dc5-b80c7a0c9409", 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 msgr2=0x7f2a5c0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 0x7f2a5c0a5940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058a0 tx=0x7f2a6000b390 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 msgr2=0x7f2a7019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e970 tx=0x7f2a6800ed70 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 shutdown_connections 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a5c0a3480 0x7f2a5c0a5940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a70108300 0x7f2a7019ce90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a70103780 0x7f2a70198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 --2- 172.21.15.77:0/2305936442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a70102dd0 0x7f2a701985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 >> 172.21.15.77:0/2305936442 conn(0x7f2a700fc7d0 msgr2=0x7f2a70105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 shutdown_connections 2024-09-15T22:30:58.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:30:58.779+0000 7f2a77688640 1 -- 172.21.15.77:0/2305936442 wait complete. 2024-09-15T22:30:59.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96345 15.72 MB/sec execute 180 sec latency 151.995 ms 2024-09-15T22:31:00.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:30:59 smithi077 ceph-mon[30980]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-09-15T22:31:00.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:30:59 smithi196 ceph-mon[35109]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-09-15T22:31:00.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:30:59 smithi117 ceph-mon[35021]: pgmap v584: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-09-15T22:31:00.484 INFO:tasks.cephfs.filesystem:scrub status for tag:89111f66-a52c-4be3-8dc5-b80c7a0c9409 - {'path': '/', 'tag': '89111f66-a52c-4be3-8dc5-b80c7a0c9409', 'options': 'recursive,force'} 2024-09-15T22:31:00.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96569 15.71 MB/sec execute 181 sec latency 506.991 ms 2024-09-15T22:31:01.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96569 15.62 MB/sec execute 182 sec latency 1507.035 ms 2024-09-15T22:31:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:01 smithi077 ceph-mon[30980]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-15T22:31:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:01 smithi196 ceph-mon[35109]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-15T22:31:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:01 smithi117 ceph-mon[35021]: pgmap v585: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-15T22:31:02.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96650 15.56 MB/sec execute 183 sec latency 2179.386 ms 2024-09-15T22:31:03.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96650 15.48 MB/sec execute 184 sec latency 1304.349 ms 2024-09-15T22:31:04.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:03 smithi077 ceph-mon[30980]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:04.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:03 smithi196 ceph-mon[35109]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:04.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:03 smithi117 ceph-mon[35021]: pgmap v586: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:04.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96650 15.39 MB/sec execute 185 sec latency 2304.391 ms 2024-09-15T22:31:05.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96650 15.31 MB/sec execute 186 sec latency 3304.482 ms 2024-09-15T22:31:06.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:05 smithi077 ceph-mon[30980]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:06.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:05 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:31:06.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:05 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:31:06.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:05 smithi196 ceph-mon[35109]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:06.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:05 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:31:06.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:05 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:31:06.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:05 smithi117 ceph-mon[35021]: pgmap v587: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.1 MiB/s wr, 457 op/s 2024-09-15T22:31:06.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:05 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:31:06.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:05 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:31:06.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 96743 15.26 MB/sec execute 187 sec latency 4183.795 ms 2024-09-15T22:31:07.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:06 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:31:07.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:06 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:31:07.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:06 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:31:07.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:06 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:31:07.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:06 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:31:07.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:06 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:31:07.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:06 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:31:07.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:06 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:31:07.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:06 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:31:07.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 97141 15.30 MB/sec execute 188 sec latency 144.404 ms 2024-09-15T22:31:08.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:07 smithi077 ceph-mon[30980]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-15T22:31:08.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:07 smithi196 ceph-mon[35109]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-15T22:31:08.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:07 smithi117 ceph-mon[35021]: pgmap v588: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-15T22:31:08.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 97484 15.31 MB/sec execute 189 sec latency 144.443 ms 2024-09-15T22:31:09.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 97996 15.35 MB/sec execute 190 sec latency 147.150 ms 2024-09-15T22:31:10.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:09 smithi077 ceph-mon[30980]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 563 op/s 2024-09-15T22:31:10.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:09 smithi196 ceph-mon[35109]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 563 op/s 2024-09-15T22:31:10.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:09 smithi117 ceph-mon[35021]: pgmap v589: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 563 op/s 2024-09-15T22:31:10.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 98567 15.43 MB/sec execute 191 sec latency 166.634 ms 2024-09-15T22:31:11.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 99124 15.54 MB/sec execute 192 sec latency 175.887 ms 2024-09-15T22:31:12.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:11 smithi196 ceph-mon[35109]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-09-15T22:31:12.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:11 smithi196 ceph-mon[35109]: osdmap e116: 12 total, 12 up, 12 in 2024-09-15T22:31:12.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:11 smithi117 ceph-mon[35021]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-09-15T22:31:12.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:11 smithi117 ceph-mon[35021]: osdmap e116: 12 total, 12 up, 12 in 2024-09-15T22:31:12.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:11 smithi077 ceph-mon[30980]: pgmap v590: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 605 op/s 2024-09-15T22:31:12.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:11 smithi077 ceph-mon[30980]: osdmap e116: 12 total, 12 up, 12 in 2024-09-15T22:31:12.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 99694 15.61 MB/sec execute 193 sec latency 138.482 ms 2024-09-15T22:31:13.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 100191 15.68 MB/sec execute 194 sec latency 122.904 ms 2024-09-15T22:31:14.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:13 smithi196 ceph-mon[35109]: pgmap v592: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:14.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:13 smithi117 ceph-mon[35021]: pgmap v592: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:13 smithi077 ceph-mon[30980]: pgmap v592: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:14.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 100630 15.75 MB/sec execute 195 sec latency 72.794 ms 2024-09-15T22:31:15.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 101018 15.74 MB/sec execute 196 sec latency 170.081 ms 2024-09-15T22:31:16.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:15 smithi196 ceph-mon[35109]: pgmap v593: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:16.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:15 smithi117 ceph-mon[35021]: pgmap v593: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:16.133 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:15 smithi077 ceph-mon[30980]: pgmap v593: 129 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 115 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 616 op/s 2024-09-15T22:31:16.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 101572 15.83 MB/sec execute 197 sec latency 156.093 ms 2024-09-15T22:31:17.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 101938 15.83 MB/sec execute 198 sec latency 211.496 ms 2024-09-15T22:31:18.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:17 smithi077 ceph-mon[30980]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:18.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:17 smithi196 ceph-mon[35109]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:18.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:17 smithi117 ceph-mon[35021]: pgmap v594: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:18.607 INFO:tasks.workunit.client.0.smithi077.stdout: 10 102400 15.88 MB/sec execute 199 sec latency 147.438 ms 2024-09-15T22:31:19.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 102939 15.92 MB/sec execute 200 sec latency 149.567 ms 2024-09-15T22:31:20.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:19 smithi077 ceph-mon[30980]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:20.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:19 smithi077 ceph-mon[30980]: osdmap e117: 12 total, 12 up, 12 in 2024-09-15T22:31:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:19 smithi196 ceph-mon[35109]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:19 smithi196 ceph-mon[35109]: osdmap e117: 12 total, 12 up, 12 in 2024-09-15T22:31:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:19 smithi117 ceph-mon[35021]: pgmap v595: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-15T22:31:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:19 smithi117 ceph-mon[35021]: osdmap e117: 12 total, 12 up, 12 in 2024-09-15T22:31:20.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 103451 15.98 MB/sec execute 201 sec latency 212.037 ms 2024-09-15T22:31:21.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 103880 16.02 MB/sec execute 202 sec latency 364.986 ms 2024-09-15T22:31:22.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:21 smithi077 ceph-mon[30980]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-09-15T22:31:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:21 smithi196 ceph-mon[35109]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-09-15T22:31:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:21 smithi117 ceph-mon[35021]: pgmap v597: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 362 op/s 2024-09-15T22:31:22.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 104567 16.13 MB/sec execute 203 sec latency 148.174 ms 2024-09-15T22:31:23.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 105085 16.24 MB/sec execute 204 sec latency 167.725 ms 2024-09-15T22:31:24.335 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:23 smithi077 ceph-mon[30980]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:24.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:23 smithi196 ceph-mon[35109]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:24.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:23 smithi117 ceph-mon[35021]: pgmap v598: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:24.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 105453 16.24 MB/sec execute 205 sec latency 57.434 ms 2024-09-15T22:31:25.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 105832 16.27 MB/sec execute 206 sec latency 196.426 ms 2024-09-15T22:31:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:26 smithi077 ceph-mon[30980]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:26 smithi196 ceph-mon[35109]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:26 smithi117 ceph-mon[35021]: pgmap v599: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 368 op/s 2024-09-15T22:31:26.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 106169 16.23 MB/sec execute 207 sec latency 239.063 ms 2024-09-15T22:31:27.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 106809 16.33 MB/sec execute 208 sec latency 159.011 ms 2024-09-15T22:31:28.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:28 smithi077 ceph-mon[30980]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:28 smithi196 ceph-mon[35109]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:28 smithi117 ceph-mon[35021]: pgmap v600: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:28.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 107279 16.36 MB/sec execute 209 sec latency 176.592 ms 2024-09-15T22:31:29.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 107715 16.40 MB/sec execute 210 sec latency 269.559 ms 2024-09-15T22:31:30.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:30 smithi077 ceph-mon[30980]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:30 smithi196 ceph-mon[35109]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:30 smithi117 ceph-mon[35021]: pgmap v601: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2024-09-15T22:31:30.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:31:30.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 108073 16.39 MB/sec execute 211 sec latency 260.120 ms 2024-09-15T22:31:30.811 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:31.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 -- 172.21.15.77:0/3865805218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604102ef0 msgr2=0x7f560410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3865805218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604102ef0 0x7f560410da40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009ca0 tx=0x7f55fc02a950 comp rx=0 tx=0).stop 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 -- 172.21.15.77:0/3865805218 shutdown_connections 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3865805218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604102ef0 0x7f560410da40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3865805218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 0x7f56041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3865805218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604100e20 0x7f5604101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 -- 172.21.15.77:0/3865805218 >> 172.21.15.77:0/3865805218 conn(0x7f56040fc7d0 msgr2=0x7f56040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 -- 172.21.15.77:0/3865805218 shutdown_connections 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.336+0000 7f5608aa4640 1 -- 172.21.15.77:0/3865805218 wait complete. 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 Processor -- start 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 -- start start 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 0x7f56041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 0x7f560419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110f80 con 0x7f5604100e20 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110e00 con 0x7f5604102ef0 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f5608aa4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5604111100 con 0x7f5604102570 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.337+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56544/0 (socket says 172.21.15.77:56544) 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 -- 172.21.15.77:0/3723902956 learned_addr learned my addr 172.21.15.77:0/3723902956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f5602d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 0x7f560419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f5601d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 0x7f56041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 -- 172.21.15.77:0/3723902956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 msgr2=0x7f56041989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 0x7f56041989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 -- 172.21.15.77:0/3723902956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 msgr2=0x7f560419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 0x7f560419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 -- 172.21.15.77:0/3723902956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419d460 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f5602d9d640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 0x7f560419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f560259c640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f55f37fe640 1 -- 172.21.15.77:0/3723902956 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400cd80 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f55f37fe640 1 -- 172.21.15.77:0/3723902956 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f4010910 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f55f37fe640 1 -- 172.21.15.77:0/3723902956 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4018990 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419d750 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.338+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a5030 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.341+0000 7f55f37fe640 1 -- 172.21.15.77:0/3723902956 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f55f4010ac0 con 0x7f5604100e20 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.341+0000 7f560ae7d740 1 Processor -- start 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 -- start start 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f55f37fe640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 0x7f55ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff0e680 0x55f28ff10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f28fe692e0 con 0x55f28ff034a0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f28fe10ce0 con 0x55f28ff0e680 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560ae7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f28fe652a0 con 0x55f28ff03dc0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5602d9d640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 0x7f55ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47756/0 (socket says 172.21.15.77:47756) 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 -- 172.21.15.77:0/2944156383 learned_addr learned my addr 172.21.15.77:0/2944156383 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56560/0 (socket says 172.21.15.77:56560) 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5602d9d640 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff0e680 0x55f28ff10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f55f37fe640 1 -- 172.21.15.77:0/3723902956 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f55f40d03c0 con 0x7f5604100e20 2024-09-15T22:31:31.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 -- 172.21.15.77:0/2944156383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff0e680 msgr2=0x55f28ff10a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff0e680 0x55f28ff10a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 -- 172.21.15.77:0/2944156383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 msgr2=0x55f28ff03880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f5601d9b640 1 -- 172.21.15.77:0/2944156383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28fdf5440 con 0x55f28ff03dc0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.342+0000 7f560259c640 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.343+0000 7f5601d9b640 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009c10 tx=0x7f55f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e70c29ae06d397 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.343+0000 7f5602d9d640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 0x7f55ec0a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55fc02fd80 tx=0x7f55fc005ef0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.343+0000 7f55f0ff9640 1 -- 172.21.15.77:0/2944156383 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8004230 con 0x55f28ff03dc0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.343+0000 7f55f0ff9640 1 -- 172.21.15.77:0/2944156383 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f80043d0 con 0x55f28ff03dc0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.343+0000 7f55f0ff9640 1 -- 172.21.15.77:0/2944156383 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8004e10 con 0x55f28ff03dc0 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- 172.21.15.77:0/2944156383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 msgr2=0x55f28ff0e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009c10 tx=0x7f55f8011810 comp rx=0 tx=0).stop 2024-09-15T22:31:31.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- 172.21.15.77:0/2944156383 shutdown_connections 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff0e680 0x55f28ff10a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff0e140 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2944156383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- 172.21.15.77:0/2944156383 >> 172.21.15.77:0/2944156383 conn(0x55f28feffa30 msgr2=0x55f28feffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- 172.21.15.77:0/2944156383 shutdown_connections 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- 172.21.15.77:0/2944156383 wait complete. 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 -- start start 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 Processor -- start 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.344+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 0x55f28ff0b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560ae7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f28ff11250 con 0x55f28ff034a0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560ae7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f28fe24050 con 0x55f28ff03dc0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560ae7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f28ff113d0 con 0x55f28ff0e680 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56570/0 (socket says 172.21.15.77:56570) 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 -- 172.21.15.77:0/75588879 learned_addr learned my addr 172.21.15.77:0/75588879 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560ae7d740 10 client.0 _ll_register_callbacks cb 0x55f28fe64160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f5601d9b640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f5602d9d640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 0x55f28ff0b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 -- 172.21.15.77:0/75588879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 msgr2=0x55f28ff0b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 0x55f28ff0b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 -- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 msgr2=0x55f28ff0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 -- 172.21.15.77:0/75588879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28fbda3c0 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f5602d9d640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 0x55f28ff0b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560259c640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f55f4098d90 tx=0x7f55f409cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5cb4cf035592f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f55e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f409c3c0 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f55f40ac180 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.345+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f80043d0 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55f40a9020 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 10 client.19944 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f28ff1cbd0 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f80043d0 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f55f4018b30 con 0x55f28ff034a0 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 10 client.19944 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:31.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 10 client.19944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 20 client.19944 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff1dfe0 0x55f28ff20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 4 client.19944 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f28fe76850 con 0x55f28ff1dfe0 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.346+0000 7f5601d9b640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff1dfe0 0x55f28ff20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.347+0000 7f55e9ffb640 10 client.19944 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.348+0000 7f5601d9b640 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff1dfe0 0x55f28ff20430 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267ed70888b42cbe server_cookie=3f33080261da5c59 in_seq=0 out_seq=0 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f55e9ffb640 1 -- 172.21.15.77:0/75588879 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f28fe76850 con 0x55f28ff1dfe0 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f55e9ffb640 10 client.19944 handle_command_reply: tid=0 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 client.19944 shutdown 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 msgr2=0x55f28ff0df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f55f4098d90 tx=0x7f55f409cac0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 shutdown_connections 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff1dfe0 0x55f28ff20430 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff0e680 0x55f28ff0b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f55e9ffb640 0 client.19944 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.349+0000 7f560ae7d740 1 --2- 172.21.15.77:0/75588879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff0df90 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.350+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 >> 172.21.15.77:0/75588879 conn(0x55f28feffa30 msgr2=0x55f28ff05760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.350+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 shutdown_connections 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.350+0000 7f560ae7d740 1 -- 172.21.15.77:0/75588879 wait complete. 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.350+0000 7f560ae7d740 20 client.19944 trim_cache size 0 max 16384 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 Processor -- start 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 -- start start 2024-09-15T22:31:31.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff034a0 0x55f28ff06740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56041acde0 con 0x55f28feffa30 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ae920 con 0x55f28ff034a0 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560ae7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ad4b0 con 0x55f28ff03dc0 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f5601d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff034a0 0x55f28ff06740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56574/0 (socket says 172.21.15.77:56574) 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f560259c640 1 -- 172.21.15.77:0/1631971053 learned_addr learned my addr 172.21.15.77:0/1631971053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.371+0000 7f5602d9d640 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 -- 172.21.15.77:0/1631971053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 msgr2=0x55f28ff25cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff25cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 -- 172.21.15.77:0/1631971053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff034a0 msgr2=0x55f28ff06740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff034a0 0x55f28ff06740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 -- 172.21.15.77:0/1631971053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28feff860 con 0x55f28feffa30 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f560259c640 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9c0 tx=0x7f55f409c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42106cef80aaacb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f55ebfff640 1 -- 172.21.15.77:0/1631971053 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40a06c0 con 0x55f28feffa30 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f55ebfff640 1 -- 172.21.15.77:0/1631971053 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f40a4f80 con 0x55f28feffa30 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.372+0000 7f55ebfff640 1 -- 172.21.15.77:0/1631971053 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4098390 con 0x55f28feffa30 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 -- 172.21.15.77:0/1631971053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 msgr2=0x55f28ff1a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9c0 tx=0x7f55f409c2a0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 -- 172.21.15.77:0/1631971053 shutdown_connections 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28ff03dc0 0x55f28ff25cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff034a0 0x55f28ff06740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 --2- 172.21.15.77:0/1631971053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28feffa30 0x55f28ff1a4b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 -- 172.21.15.77:0/1631971053 >> 172.21.15.77:0/1631971053 conn(0x55f28ff0b8e0 msgr2=0x55f28ff06030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 -- 172.21.15.77:0/1631971053 shutdown_connections 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.373+0000 7f560ae7d740 1 -- 172.21.15.77:0/1631971053 wait complete. 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 -- start start 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 Processor -- start 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ae250 con 0x55f28ff034a0 2024-09-15T22:31:31.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f56041adb80 con 0x55f28ff03dc0 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56041aade0 con 0x55f28feffa30 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560ae7d740 10 client.0 _ll_register_callbacks cb 0x7f55f4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f5602d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f5601d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47792/0 (socket says 172.21.15.77:47792) 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.374+0000 7f560259c640 1 -- 172.21.15.77:0/2154729488 learned_addr learned my addr 172.21.15.77:0/2154729488 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 -- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 msgr2=0x55f28ff12d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff12d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 -- 172.21.15.77:0/2154729488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 msgr2=0x55f28ff12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 -- 172.21.15.77:0/2154729488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28ff904d0 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560259c640 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c6a0 tx=0x7f55f409bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dffe1c0dc7ece47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f55e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4010640 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f55f40bc0d0 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f40a4f80 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f55f40bc2d0 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560ae7d740 10 client.28642 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.375+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f28ff91720 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f40a4f80 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f55f409b3a0 con 0x55f28feffa30 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 10 client.28642 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 10 client.28642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 20 client.28642 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:31.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff0e680 0x55f28ff93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:31.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 4 client.28642 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:31:31.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f28fe70d60 con 0x55f28ff0e680 2024-09-15T22:31:31.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f5601d9b640 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff0e680 0x55f28ff93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:31.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f5601d9b640 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff0e680 0x55f28ff93c50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8611aba2cbf21989 server_cookie=15e59b46d2d48fa3 in_seq=0 out_seq=0 2024-09-15T22:31:31.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.376+0000 7f55e9ffb640 10 client.28642 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f55e9ffb640 1 -- 172.21.15.77:0/2154729488 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f28fe70d60 con 0x55f28ff0e680 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f55e9ffb640 10 client.28642 handle_command_reply: tid=0 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 client.28642 shutdown 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 msgr2=0x55f28ff122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c6a0 tx=0x7f55f409bc30 comp rx=0 tx=0).stop 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 shutdown_connections 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f28ff0e680 0x55f28ff93c50 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f28ff03dc0 0x55f28ff12d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f28ff034a0 0x55f28ff12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f55e9ffb640 0 client.28642 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 --2- 172.21.15.77:0/2154729488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f28feffa30 0x55f28ff122c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 >> 172.21.15.77:0/2154729488 conn(0x55f28ff0b8e0 msgr2=0x55f28fefeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 shutdown_connections 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.378+0000 7f560ae7d740 1 -- 172.21.15.77:0/2154729488 wait complete. 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f560ae7d740 20 client.28642 trim_cache size 0 max 16384 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:31:31.397 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:31:31.398 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:31:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 msgr2=0x7f55ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 0x7f55ec0a5870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55fc02fd80 tx=0x7f55fc005ef0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 msgr2=0x7f56041984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f55f400e9f0 tx=0x7f55f400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 shutdown_connections 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f55ec0a33b0 0x7f55ec0a5870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604102ef0 0x7f560419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5604102570 0x7f56041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 --2- 172.21.15.77:0/3723902956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5604100e20 0x7f56041984b0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.379+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 >> 172.21.15.77:0/3723902956 conn(0x7f56040fc7d0 msgr2=0x7f560406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.380+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 shutdown_connections 2024-09-15T22:31:31.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:31.380+0000 7f5608aa4640 1 -- 172.21.15.77:0/3723902956 wait complete. 2024-09-15T22:31:31.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 108626 16.44 MB/sec execute 212 sec latency 162.301 ms 2024-09-15T22:31:32.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:32 smithi077 ceph-mon[30980]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.0 MiB/s wr, 363 op/s 2024-09-15T22:31:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:32 smithi196 ceph-mon[35109]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.0 MiB/s wr, 363 op/s 2024-09-15T22:31:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:32 smithi117 ceph-mon[35021]: pgmap v602: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.0 MiB/s wr, 363 op/s 2024-09-15T22:31:32.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 109162 16.54 MB/sec execute 213 sec latency 181.987 ms 2024-09-15T22:31:32.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:31:32.700 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:31:33.184 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:33.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 109354 16.52 MB/sec execute 214 sec latency 505.837 ms 2024-09-15T22:31:34.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:34 smithi077 ceph-mon[30980]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 370 op/s 2024-09-15T22:31:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:34 smithi196 ceph-mon[35109]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 370 op/s 2024-09-15T22:31:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:34 smithi117 ceph-mon[35021]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 370 op/s 2024-09-15T22:31:34.608 INFO:tasks.workunit.client.0.smithi077.stdout: 10 109404 16.45 MB/sec execute 215 sec latency 1365.434 ms 2024-09-15T22:31:34.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:31:34.934 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:31:34.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.952+0000 7fdb16184640 1 -- 172.21.15.77:0/2654439562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a4850 msgr2=0x7fdb080a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.952+0000 7fdb16184640 1 --2- 172.21.15.77:0/2654439562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a4850 0x7fdb080a4c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009860 tx=0x7fdb0402a410 comp rx=0 tx=0).stop 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.959+0000 7fdb16184640 1 -- 172.21.15.77:0/2654439562 shutdown_connections 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.959+0000 7fdb16184640 1 --2- 172.21.15.77:0/2654439562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080b7570 0x7fdb080b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.959+0000 7fdb16184640 1 --2- 172.21.15.77:0/2654439562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080a5170 0x7fdb080b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.959+0000 7fdb16184640 1 --2- 172.21.15.77:0/2654439562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a4850 0x7fdb080a4c30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.959+0000 7fdb16184640 1 -- 172.21.15.77:0/2654439562 >> 172.21.15.77:0/2654439562 conn(0x7fdb0801a110 msgr2=0x7fdb0801a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.963+0000 7fdb16184640 1 -- 172.21.15.77:0/2654439562 shutdown_connections 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.965+0000 7fdb16184640 1 -- 172.21.15.77:0/2654439562 wait complete. 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.966+0000 7fdb16184640 1 Processor -- start 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.970+0000 7fdb16184640 1 -- start start 2024-09-15T22:31:34.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.970+0000 7fdb16184640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 0x7fdb08143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.970+0000 7fdb16184640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a5170 0x7fdb08143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.970+0000 7fdb16184640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 0x7fdb08147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb16184640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb080bd000 con 0x7fdb080b7570 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb16184640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb080bce80 con 0x7fdb080a4850 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb16184640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb080bd180 con 0x7fdb080a5170 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 0x7fdb08147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 0x7fdb08147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56606/0 (socket says 172.21.15.77:56606) 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb0ffff640 1 -- 172.21.15.77:0/555712945 learned_addr learned my addr 172.21.15.77:0/555712945 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb0effd640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a5170 0x7fdb08143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.971+0000 7fdb0f7fe640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 0x7fdb08143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.972+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/555712945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a5170 msgr2=0x7fdb08143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.972+0000 7fdb0f7fe640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a5170 0x7fdb08143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.972+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 msgr2=0x7fdb08147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.972+0000 7fdb0f7fe640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 0x7fdb08147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.973+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/555712945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb081483f0 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.973+0000 7fdb0f7fe640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 0x7fdb08143440 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdb0402aa00 tx=0x7fdb04030940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.974+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04030a60 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.974+0000 7fdb16184640 1 -- 172.21.15.77:0/555712945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb08148680 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.974+0000 7fdb16184640 1 -- 172.21.15.77:0/555712945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0814ffc0 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.975+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb04009420 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.976+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04031400 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.977+0000 7fdb16184640 1 -- 172.21.15.77:0/555712945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb080a4c30 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.979+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdb04039040 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.980+0000 7fdb0cff9640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 0x7fdb000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.981+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7fdb040f1100 con 0x7fdb080a4850 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.981+0000 7fdb0effd640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 0x7fdb000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:34.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.982+0000 7fdb0effd640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 0x7fdb000a5790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b69a0 tx=0x7fdafc009260 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:33.988+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb04040030 con 0x7fdb080a4850 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.355+0000 7fdb16184640 1 -- 172.21.15.77:0/555712945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb081447a0 con 0x7fdb080a4850 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.356+0000 7fdb0cff9640 1 -- 172.21.15.77:0/555712945 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fdb081447a0 con 0x7fdb080a4850 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.366+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 msgr2=0x7fdb000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.366+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 0x7fdb000a5790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b69a0 tx=0x7fdafc009260 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.366+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 msgr2=0x7fdb08143440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.366+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 0x7fdb08143440 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdb0402aa00 tx=0x7fdb04030940 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 shutdown_connections 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb000a32d0 0x7fdb000a5790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb080b7570 0x7fdb08147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb080a5170 0x7fdb08143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 --2- 172.21.15.77:0/555712945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb080a4850 0x7fdb08143440 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:34.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.368+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 >> 172.21.15.77:0/555712945 conn(0x7fdb0801a110 msgr2=0x7fdb080b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.369+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 shutdown_connections 2024-09-15T22:31:34.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:34.369+0000 7fdafa7fc640 1 -- 172.21.15.77:0/555712945 wait complete. 2024-09-15T22:31:35.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:35 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/555712945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:35.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:35 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/555712945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:35.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:35 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/555712945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:35.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 109591 16.43 MB/sec execute 216 sec latency 1005.435 ms 2024-09-15T22:31:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:36 smithi077 ceph-mon[30980]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-15T22:31:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:36 smithi196 ceph-mon[35109]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-15T22:31:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:36 smithi117 ceph-mon[35021]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-09-15T22:31:36.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 109849 16.41 MB/sec execute 217 sec latency 241.698 ms 2024-09-15T22:31:37.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 110196 16.42 MB/sec execute 218 sec latency 112.092 ms 2024-09-15T22:31:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:38 smithi077 ceph-mon[30980]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-09-15T22:31:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:38 smithi196 ceph-mon[35109]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-09-15T22:31:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:38 smithi117 ceph-mon[35021]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 596 op/s 2024-09-15T22:31:38.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 110521 16.44 MB/sec execute 219 sec latency 129.100 ms 2024-09-15T22:31:39.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 110896 16.44 MB/sec execute 220 sec latency 266.854 ms 2024-09-15T22:31:40.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:40 smithi077 ceph-mon[30980]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 454 op/s 2024-09-15T22:31:40.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:40 smithi196 ceph-mon[35109]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 454 op/s 2024-09-15T22:31:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:40 smithi117 ceph-mon[35021]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 454 op/s 2024-09-15T22:31:40.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 111266 16.49 MB/sec execute 221 sec latency 234.925 ms 2024-09-15T22:31:41.054 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:31:41.391 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:41.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 111666 16.50 MB/sec execute 222 sec latency 179.895 ms 2024-09-15T22:31:42.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 -- 172.21.15.77:0/2982347950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 msgr2=0x7f9f48106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 --2- 172.21.15.77:0/2982347950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f48106890 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009900 tx=0x7f9f3802a410 comp rx=0 tx=0).stop 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 -- 172.21.15.77:0/2982347950 shutdown_connections 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 --2- 172.21.15.77:0/2982347950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48106fc0 0x7f9f4810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 --2- 172.21.15.77:0/2982347950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f48106890 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 --2- 172.21.15.77:0/2982347950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48101a30 0x7f9f48101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 -- 172.21.15.77:0/2982347950 >> 172.21.15.77:0/2982347950 conn(0x7f9f48075f70 msgr2=0x7f9f48076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 -- 172.21.15.77:0/2982347950 shutdown_connections 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.002+0000 7f9f50519640 1 -- 172.21.15.77:0/2982347950 wait complete. 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 Processor -- start 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 -- start start 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 0x7f9f4819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48110e80 con 0x7f9f481023e0 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48110d00 con 0x7f9f48101a30 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f50519640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48111000 con 0x7f9f48106fc0 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4dab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4dab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37428/0 (socket says 172.21.15.77:37428) 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41652/0 (socket says 172.21.15.77:41652) 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4dab4640 1 -- 172.21.15.77:0/2890058380 learned_addr learned my addr 172.21.15.77:0/2890058380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 0x7f9f4819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 -- 172.21.15.77:0/2890058380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 msgr2=0x7f9f4819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 0x7f9f4819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 -- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 msgr2=0x7f9f481978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.003+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f4e2b5640 1 -- 172.21.15.77:0/2890058380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4819c350 con 0x7f9f48101a30 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 0x7f9f4819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b3f0 tx=0x7f9f4400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f3f7fe640 1 -- 172.21.15.77:0/2890058380 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44015020 con 0x7f9f48101a30 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4819c640 con 0x7f9f48101a30 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f3f7fe640 1 -- 172.21.15.77:0/2890058380 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44011d10 con 0x7f9f48101a30 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f3f7fe640 1 -- 172.21.15.77:0/2890058380 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44010e50 con 0x7f9f48101a30 2024-09-15T22:31:42.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.004+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481a3f20 con 0x7f9f48101a30 2024-09-15T22:31:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 Processor -- start 2024-09-15T22:31:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 -- start start 2024-09-15T22:31:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f3f7fe640 1 -- 172.21.15.77:0/2890058380 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9f44018020 con 0x7f9f48101a30 2024-09-15T22:31:42.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ed27dc52f0 con 0x55ed27e6a370 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed27d6ccf0 con 0x55ed27e5f5d0 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f528f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed27dc12b0 con 0x55ed27e60b40 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4e2b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38434/0 (socket says 172.21.15.77:38434) 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4eab6640 1 -- 172.21.15.77:0/271799226 learned_addr learned my addr 172.21.15.77:0/271799226 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41668/0 (socket says 172.21.15.77:41668) 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.007+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 -- 172.21.15.77:0/271799226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 msgr2=0x55ed27e5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 -- 172.21.15.77:0/271799226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 msgr2=0x55ed27e6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e6c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 -- 172.21.15.77:0/271799226 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed27d513a0 con 0x55ed27e60b40 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9f400117e0 tx=0x7f9f40011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860061926e0c80d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f3f7fe640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 0x7f9f340a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.008+0000 7f9f3cff9640 1 -- 172.21.15.77:0/271799226 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004d20 con 0x55ed27e60b40 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f3cff9640 1 -- 172.21.15.77:0/271799226 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001e070 con 0x55ed27e60b40 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f3cff9640 1 -- 172.21.15.77:0/271799226 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001aab0 con 0x55ed27e60b40 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 0x7f9f340a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f3f7fe640 1 -- 172.21.15.77:0/2890058380 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f9f440cf740 con 0x7f9f48101a30 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 -- 172.21.15.77:0/271799226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 msgr2=0x55ed27e5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f4e2b5640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 0x7f9f340a5a10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f440cccc0 tx=0x7f9f440d9960 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9f400117e0 tx=0x7f9f40011ca0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 -- 172.21.15.77:0/271799226 shutdown_connections 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 --2- 172.21.15.77:0/271799226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e5f090 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.009+0000 7f9f528f2740 1 -- 172.21.15.77:0/271799226 >> 172.21.15.77:0/271799226 conn(0x55ed27e28310 msgr2=0x55ed27e5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- 172.21.15.77:0/271799226 shutdown_connections 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- 172.21.15.77:0/271799226 wait complete. 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- start start 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 Processor -- start 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ed27e6d070 con 0x55ed27e60b40 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed27d80060 con 0x55ed27e5f5d0 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed27e6d1f0 con 0x55ed27e6a370 2024-09-15T22:31:42.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.010+0000 7f9f528f2740 10 client.0 _ll_register_callbacks cb 0x55ed27ad38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4dab4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41680/0 (socket says 172.21.15.77:41680) 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2809004217 learned_addr learned my addr 172.21.15.77:0/2809004217 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2809004217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 msgr2=0x55ed27e674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 msgr2=0x55ed27e66f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e66f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2809004217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed27b363c0 con 0x55ed27e5f5d0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e66f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f40010ee0 tx=0x7f9f40010de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cec068443d392 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:31:42.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40003e60 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.011+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4001e070 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f9f4001aa60 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9f4001ac60 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 10 client.28748 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed27e78a60 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001e070 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9f4002d040 con 0x55ed27e5f5d0 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 10 client.28748 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 10 client.28748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 20 client.28748 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27e7a2f0 0x55ed27e7c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 4 client.28748 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed27dd2860 con 0x55ed27e7a2f0 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.012+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27e7a2f0 0x55ed27e7c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.013+0000 7f9f31ffb640 10 client.28748 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.013+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27e7a2f0 0x55ed27e7c700 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52f6d1464870ed94 server_cookie=9e2bbdd31aca876d in_seq=0 out_seq=0 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f31ffb640 1 -- 172.21.15.77:0/2809004217 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed27dd2860 con 0x55ed27e7a2f0 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f31ffb640 10 client.28748 handle_command_reply: tid=0 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 client.28748 shutdown 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 msgr2=0x55ed27e69cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f40010ee0 tx=0x7f9f40010de0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 shutdown_connections 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27e7a2f0 0x55ed27e7c700 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e66f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2809004217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e69cb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 >> 172.21.15.77:0/2809004217 conn(0x55ed27e28310 msgr2=0x55ed27e61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f31ffb640 0 client.28748 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 shutdown_connections 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 1 -- 172.21.15.77:0/2809004217 wait complete. 2024-09-15T22:31:42.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.015+0000 7f9f528f2740 20 client.28748 trim_cache size 0 max 16384 2024-09-15T22:31:42.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 Processor -- start 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 -- start start 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e76310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481acf60 con 0x55ed27e6a370 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481aeaa0 con 0x55ed27e5f5d0 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f528f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481ad630 con 0x55ed27e60b40 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37450/0 (socket says 172.21.15.77:37450) 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2360969724 learned_addr learned my addr 172.21.15.77:0/2360969724 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:42.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.035+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2360969724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 msgr2=0x55ed27e62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e62120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2360969724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 msgr2=0x55ed27e76310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e76310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 -- 172.21.15.77:0/2360969724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed27e62ba0 con 0x55ed27e6a370 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9f40004730 tx=0x7f9f40005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1cc6808de5d53e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f33fff640 1 -- 172.21.15.77:0/2360969724 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001aa60 con 0x55ed27e6a370 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f33fff640 1 -- 172.21.15.77:0/2360969724 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001e070 con 0x55ed27e6a370 2024-09-15T22:31:42.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.036+0000 7f9f33fff640 1 -- 172.21.15.77:0/2360969724 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004a20 con 0x55ed27e6a370 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 -- 172.21.15.77:0/2360969724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 msgr2=0x55ed27e62660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9f40004730 tx=0x7f9f40005cf0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 -- 172.21.15.77:0/2360969724 shutdown_connections 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e6a370 0x55ed27e62660 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e60b40 0x55ed27e62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 --2- 172.21.15.77:0/2360969724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 -- 172.21.15.77:0/2360969724 >> 172.21.15.77:0/2360969724 conn(0x55ed27e28310 msgr2=0x55ed27e5ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 -- 172.21.15.77:0/2360969724 shutdown_connections 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.037+0000 7f9f528f2740 1 -- 172.21.15.77:0/2360969724 wait complete. 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 -- start start 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 Processor -- start 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481ae3d0 con 0x55ed27e60b40 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481add00 con 0x55ed27e5f5d0 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.038+0000 7f9f528f2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481aaf60 con 0x55ed27e6a370 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4eab6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41700/0 (socket says 172.21.15.77:41700) 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4eab6640 1 -- 172.21.15.77:0/992366671 learned_addr learned my addr 172.21.15.77:0/992366671 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:42.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f528f2740 10 client.0 _ll_register_callbacks cb 0x7f9f38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 -- 172.21.15.77:0/992366671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 msgr2=0x55ed27e6dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e6dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 -- 172.21.15.77:0/992366671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 msgr2=0x55ed27e72d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 -- 172.21.15.77:0/992366671 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed27ee2f90 con 0x55ed27e60b40 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f4dab4640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e6d410 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009230 tx=0x7f9f38005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2320949bd378fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38035040 con 0x55ed27e60b40 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f9f38031970 con 0x55ed27e60b40 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.039+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4001e070 con 0x55ed27e60b40 2024-09-15T22:31:42.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9f38055050 con 0x55ed27e60b40 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 10 client.19974 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed27ee4150 con 0x55ed27e60b40 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001e070 con 0x55ed27e60b40 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9f38004860 con 0x55ed27e60b40 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 10 client.19974 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 10 client.19974 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 20 client.19974 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27ee4900 0x55ed27ee6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 4 client.19974 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed27dccd70 con 0x55ed27ee4900 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27ee4900 0x55ed27ee6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:42.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.040+0000 7f9f31ffb640 10 client.19974 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f4eab6640 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27ee4900 0x55ed27ee6d20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a142e8c7da7812c5 server_cookie=2d525c163eef158 in_seq=0 out_seq=0 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f31ffb640 1 -- 172.21.15.77:0/992366671 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed27dccd70 con 0x55ed27ee4900 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f31ffb640 10 client.19974 handle_command_reply: tid=0 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 client.19974 shutdown 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 msgr2=0x55ed27e6d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e6d410 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009230 tx=0x7f9f38005e40 comp rx=0 tx=0).stop 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 shutdown_connections 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55ed27ee4900 0x55ed27ee6d20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed27e6a370 0x55ed27e6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ed27e60b40 0x55ed27e6d410 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f31ffb640 0 client.19974 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 --2- 172.21.15.77:0/992366671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed27e5f5d0 0x55ed27e72d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 >> 172.21.15.77:0/992366671 conn(0x55ed27e28310 msgr2=0x55ed27e66820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 shutdown_connections 2024-09-15T22:31:42.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 1 -- 172.21.15.77:0/992366671 wait complete. 2024-09-15T22:31:42.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.041+0000 7f9f528f2740 20 client.19974 trim_cache size 0 max 16384 2024-09-15T22:31:42.048 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 msgr2=0x7f9f340a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 0x7f9f340a5a10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f440cccc0 tx=0x7f9f440d9960 comp rx=0 tx=0).stop 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 msgr2=0x7f9f481973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b3f0 tx=0x7f9f4400b8b0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 shutdown_connections 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9f340a3550 0x7f9f340a5a10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f48106fc0 0x7f9f4819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f481023e0 0x7f9f481978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 --2- 172.21.15.77:0/2890058380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f48101a30 0x7f9f481973a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:42.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 >> 172.21.15.77:0/2890058380 conn(0x7f9f48075f70 msgr2=0x7f9f48104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:42.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 shutdown_connections 2024-09-15T22:31:42.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:42.042+0000 7f9f50519640 1 -- 172.21.15.77:0/2890058380 wait complete. 2024-09-15T22:31:42.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:42 smithi077 ceph-mon[30980]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-09-15T22:31:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:42 smithi196 ceph-mon[35109]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-09-15T22:31:42.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:42 smithi117 ceph-mon[35021]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-09-15T22:31:42.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 111986 16.48 MB/sec execute 223 sec latency 164.664 ms 2024-09-15T22:31:42.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:31:43.210 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:43.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 112432 16.48 MB/sec execute 224 sec latency 171.760 ms 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2404157762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 msgr2=0x7f0ec0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2404157762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec0106e50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009930 tx=0x7f0eb002a420 comp rx=0 tx=0).stop 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2404157762 shutdown_connections 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2404157762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 0x7f0ec010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2404157762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 0x7f0ec0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2404157762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec0106e50 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2404157762 >> 172.21.15.77:0/2404157762 conn(0x7f0ec00fc790 msgr2=0x7f0ec00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2404157762 shutdown_connections 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.902+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2404157762 wait complete. 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 Processor -- start 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 -- start start 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 0x7f0ec0102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 0x7f0ec0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110ec0 con 0x7f0ec0106a70 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.903+0000 7f0ec5a6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110d40 con 0x7f0ec0100a70 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ec5a6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0111040 con 0x7f0ec0101410 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37478/0 (socket says 172.21.15.77:37478) 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 -- 172.21.15.77:0/2211092498 learned_addr learned my addr 172.21.15.77:0/2211092498 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:43.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 0x7f0ec0102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 0x7f0ec0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 -- 172.21.15.77:0/2211092498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 msgr2=0x7f0ec0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 0x7f0ec0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 -- 172.21.15.77:0/2211092498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 msgr2=0x7f0ec0102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 0x7f0ec0102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 -- 172.21.15.77:0/2211092498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec01a6930 con 0x7f0ec0106a70 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebffff640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800eb10 tx=0x7f0eb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebd7fa640 1 -- 172.21.15.77:0/2211092498 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8004b20 con 0x7f0ec0106a70 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ebd7fa640 1 -- 172.21.15.77:0/2211092498 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb8004cc0 con 0x7f0ec0106a70 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.904+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a6c20 con 0x7f0ec0106a70 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.905+0000 7f0ebd7fa640 1 -- 172.21.15.77:0/2211092498 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8010760 con 0x7f0ec0106a70 2024-09-15T22:31:43.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.905+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a7160 con 0x7f0ec0106a70 2024-09-15T22:31:43.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ebd7fa640 1 -- 172.21.15.77:0/2211092498 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0eb8005020 con 0x7f0ec0106a70 2024-09-15T22:31:43.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 Processor -- start 2024-09-15T22:31:43.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 -- start start 2024-09-15T22:31:43.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95ad7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ae2370 0x560e95ae4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e95a3d2f0 con 0x560e95ad8b40 2024-09-15T22:31:43.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e959e4cf0 con 0x560e95ae2370 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.908+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e95a392b0 con 0x560e95ad75d0 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38498/0 (socket says 172.21.15.77:38498) 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/3698223950 learned_addr learned my addr 172.21.15.77:0/3698223950 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ae2370 0x560e95ae4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95ad7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/3698223950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ae2370 msgr2=0x560e95ae4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ae2370 0x560e95ae4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/3698223950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 msgr2=0x560e95ad7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95ad7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/3698223950 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e959c93a0 con 0x560e95ad75d0 2024-09-15T22:31:43.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95ad7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.909+0000 7f0ebd7fa640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0ebd7fa640 1 -- 172.21.15.77:0/2211092498 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f0eb80d06e0 con 0x7f0ec0106a70 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009930 tx=0x7f0eb0005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c398355d6d6c087 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0eb5ffb640 1 -- 172.21.15.77:0/3698223950 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035040 con 0x560e95ad75d0 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0eb5ffb640 1 -- 172.21.15.77:0/3698223950 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0004480 con 0x560e95ad75d0 2024-09-15T22:31:43.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.910+0000 7f0eb5ffb640 1 -- 172.21.15.77:0/3698223950 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00035f0 con 0x560e95ad75d0 2024-09-15T22:31:43.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3698223950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 msgr2=0x560e95ad7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 0x7f0eac0a5870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80058a0 tx=0x7f0ea800b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009930 tx=0x7f0eb0005bf0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3698223950 shutdown_connections 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ae2370 0x560e95ae4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ad75d0 0x560e95ad7a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3698223950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95ad7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3698223950 >> 172.21.15.77:0/3698223950 conn(0x560e95aa0310 msgr2=0x560e95ad3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3698223950 shutdown_connections 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3698223950 wait complete. 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 -- start start 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.911+0000 7f0ec7e48740 1 Processor -- start 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad8b40 0x560e95adef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e95ae5010 con 0x560e95ad75d0 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e959f8060 con 0x560e95ad8b40 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560e95ae5190 con 0x560e95ae2370 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ec7e48740 10 client.0 _ll_register_callbacks cb 0x560e9574b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37502/0 (socket says 172.21.15.77:37502) 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/4246605772 learned_addr learned my addr 172.21.15.77:0/4246605772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad8b40 0x560e95adef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebffff640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 msgr2=0x560e95adf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/4246605772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad8b40 msgr2=0x560e95adef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad8b40 0x560e95adef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 -- 172.21.15.77:0/4246605772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e957ae3c0 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a940 tx=0x7f0eb002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e35e0b1b029dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0e9effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.912+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0052060 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f0eb0004280 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0004480 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0035070 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 10 client.19983 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e95af09c0 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0004480 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0eb0031b90 con 0x560e95ad75d0 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 10 client.19983 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:43.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 10 client.19983 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 20 client.19983 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95af4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 4 client.19983 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.913+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e95a4a860 con 0x560e95af1d10 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.914+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95af4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.914+0000 7f0e9effd640 10 client.19983 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.914+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95af4100 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f900c4cd5f122e server_cookie=bb24791b4286e19a in_seq=0 out_seq=0 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.915+0000 7f0e9effd640 1 -- 172.21.15.77:0/4246605772 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e95a4a860 con 0x560e95af1d10 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.915+0000 7f0e9effd640 10 client.19983 handle_command_reply: tid=0 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.915+0000 7f0ec7e48740 1 client.19983 shutdown 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 msgr2=0x560e95ae1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a940 tx=0x7f0eb002a420 comp rx=0 tx=0).stop 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 shutdown_connections 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95af4100 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0e9effd640 0 client.19983 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad8b40 0x560e95adef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/4246605772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad75d0 0x560e95ae1ce0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 >> 172.21.15.77:0/4246605772 conn(0x560e95aa0310 msgr2=0x560e95ad9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 shutdown_connections 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 1 -- 172.21.15.77:0/4246605772 wait complete. 2024-09-15T22:31:43.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.916+0000 7f0ec7e48740 20 client.19983 trim_cache size 0 max 16384 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.931+0000 7f0ec7e48740 1 Processor -- start 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.931+0000 7f0ec7e48740 1 -- start start 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95ae3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b1270 con 0x560e95ad8b40 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b2db0 con 0x560e95ad75d0 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b1940 con 0x560e95ae2370 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37518/0 (socket says 172.21.15.77:37518) 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/3312504777 learned_addr learned my addr 172.21.15.77:0/3312504777 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0ebffff640 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95ae3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/3312504777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 msgr2=0x560e95ae3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95ae3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/3312504777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 msgr2=0x560e95aee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/3312504777 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e95ad4960 con 0x560e95ad8b40 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x560e95ad8070 tx=0x7f0ea8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fd5cc7b5b89b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.932+0000 7f0eb57fa640 1 -- 172.21.15.77:0/3312504777 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8022c50 con 0x560e95ad8b40 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0eb57fa640 1 -- 172.21.15.77:0/3312504777 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea802d070 con 0x560e95ad8b40 2024-09-15T22:31:43.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0eb57fa640 1 -- 172.21.15.77:0/3312504777 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8020d60 con 0x560e95ad8b40 2024-09-15T22:31:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3312504777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 msgr2=0x560e95addf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x560e95ad8070 tx=0x7f0ea8014040 comp rx=0 tx=0).stop 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3312504777 shutdown_connections 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95ae3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95addf70 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/3312504777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.933+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3312504777 >> 172.21.15.77:0/3312504777 conn(0x560e95aa0310 msgr2=0x560e95add860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3312504777 shutdown_connections 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- 172.21.15.77:0/3312504777 wait complete. 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- start start 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 Processor -- start 2024-09-15T22:31:43.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b26e0 con 0x560e95ad8b40 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b2010 con 0x560e95ad75d0 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.934+0000 7f0ec7e48740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01af250 con 0x560e95ae2370 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0ec7e48740 10 client.0 _ll_register_callbacks cb 0x7f0eb0005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37530/0 (socket says 172.21.15.77:37530) 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/1592613548 learned_addr learned my addr 172.21.15.77:0/1592613548 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0ebffff640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 msgr2=0x560e95adc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/1592613548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 msgr2=0x560e95aea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 -- 172.21.15.77:0/1592613548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e95adcd00 con 0x560e95ad8b40 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0ebffff640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0eb6ffd640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8020600 tx=0x7f0ea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a757fb7c6537693 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0e9effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8037020 con 0x560e95ad8b40 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f0ea8020df0 con 0x560e95ad8b40 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea802d070 con 0x560e95ad8b40 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.935+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ea80220d0 con 0x560e95ad8b40 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 10 client.19989 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:43.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e95af9be0 con 0x560e95ad8b40 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea802d070 con 0x560e95ad8b40 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0ea8033460 con 0x560e95ad8b40 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 10 client.19989 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 10 client.19989 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 20 client.19989 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95b5ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 4 client.19989 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e95a44d70 con 0x560e95af1d10 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95b5ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0ebf7fe640 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95b5ecc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5f3ca0a38df42e6 server_cookie=16383b66b2a73938 in_seq=0 out_seq=0 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.936+0000 7f0e9effd640 10 client.19989 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0e9effd640 1 -- 172.21.15.77:0/1592613548 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e95a44d70 con 0x560e95af1d10 2024-09-15T22:31:43.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0e9effd640 10 client.19989 handle_command_reply: tid=0 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 client.19989 shutdown 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 msgr2=0x560e95adc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8020600 tx=0x7f0ea8004730 comp rx=0 tx=0).stop 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 shutdown_connections 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x560e95af1d10 0x560e95b5ecc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560e95ae2370 0x560e95adc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e95ad8b40 0x560e95adc280 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0e9effd640 0 client.19989 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 --2- 172.21.15.77:0/1592613548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e95ad75d0 0x560e95aea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 >> 172.21.15.77:0/1592613548 conn(0x560e95aa0310 msgr2=0x560e95ade080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 shutdown_connections 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 1 -- 172.21.15.77:0/1592613548 wait complete. 2024-09-15T22:31:43.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.937+0000 7f0ec7e48740 20 client.19989 trim_cache size 0 max 16384 2024-09-15T22:31:43.945 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:31:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 msgr2=0x7f0eac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 0x7f0eac0a5870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80058a0 tx=0x7f0ea800b390 comp rx=0 tx=0).stop 2024-09-15T22:31:43.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 msgr2=0x7f0ec01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800eb10 tx=0x7f0eb800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 shutdown_connections 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0eac0a33b0 0x7f0eac0a5870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0106a70 0x7f0ec01030c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ec0101410 0x7f0ec0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 --2- 172.21.15.77:0/2211092498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ec0100a70 0x7f0ec0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 >> 172.21.15.77:0/2211092498 conn(0x7f0ec00fc790 msgr2=0x7f0ec006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 shutdown_connections 2024-09-15T22:31:43.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:43.938+0000 7f0ec5a6f640 1 -- 172.21.15.77:0/2211092498 wait complete. 2024-09-15T22:31:44.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:44 smithi077 ceph-mon[30980]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 618 op/s 2024-09-15T22:31:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:44 smithi196 ceph-mon[35109]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 618 op/s 2024-09-15T22:31:44.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:44 smithi117 ceph-mon[35021]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 618 op/s 2024-09-15T22:31:44.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 113038 16.58 MB/sec execute 225 sec latency 149.188 ms 2024-09-15T22:31:45.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:31:45.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 113463 16.61 MB/sec execute 226 sec latency 204.565 ms 2024-09-15T22:31:45.865 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:46.293 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:46 smithi077 ceph-mon[30980]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 533 op/s 2024-09-15T22:31:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:46 smithi196 ceph-mon[35109]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 533 op/s 2024-09-15T22:31:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:46 smithi117 ceph-mon[35021]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 533 op/s 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.413+0000 7faefd7a0640 1 -- 172.21.15.77:0/446070940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 msgr2=0x7faef8106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.413+0000 7faefd7a0640 1 --2- 172.21.15.77:0/446070940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8106690 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7faeec009930 tx=0x7faeec02a420 comp rx=0 tx=0).stop 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.413+0000 7faefd7a0640 1 -- 172.21.15.77:0/446070940 shutdown_connections 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 --2- 172.21.15.77:0/446070940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef8106df0 0x7faef810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 --2- 172.21.15.77:0/446070940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8106690 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 --2- 172.21.15.77:0/446070940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef81001a0 0x7faef8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 -- 172.21.15.77:0/446070940 >> 172.21.15.77:0/446070940 conn(0x7faef8077070 msgr2=0x7faef80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 -- 172.21.15.77:0/446070940 shutdown_connections 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 -- 172.21.15.77:0/446070940 wait complete. 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 Processor -- start 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.414+0000 7faefd7a0640 1 -- start start 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef8106df0 0x7faef8101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faef8111e20 con 0x7faef8100b50 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faef8111ca0 con 0x7faef81001a0 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faefd7a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faef8111fa0 con 0x7faef8106df0 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41768/0 (socket says 172.21.15.77:41768) 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 -- 172.21.15.77:0/1732824747 learned_addr learned my addr 172.21.15.77:0/1732824747 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef77fe640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 -- 172.21.15.77:0/1732824747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef8106df0 msgr2=0x7faef8101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef8106df0 0x7faef8101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 -- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 msgr2=0x7faef8101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.415+0000 7faef7fff640 1 -- 172.21.15.77:0/1732824747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef81a6760 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.416+0000 7faef7fff640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9f0 tx=0x7faee800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.416+0000 7faef57fa640 1 -- 172.21.15.77:0/1732824747 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee800cd80 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.416+0000 7faef57fa640 1 -- 172.21.15.77:0/1732824747 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee8005500 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.416+0000 7faef57fa640 1 -- 172.21.15.77:0/1732824747 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8010690 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.417+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81a6a50 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.417+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a6f10 con 0x7faef81001a0 2024-09-15T22:31:46.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.421+0000 7faf00ddb740 1 Processor -- start 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.421+0000 7faf00ddb740 1 -- start start 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f4ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f4efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55757f4552f0 con 0x55757f502d60 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55757f3fccf0 con 0x55757f4ef7b0 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faf00ddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55757f4512b0 con 0x55757f4f0d20 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.422+0000 7faefcf9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f4ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.424+0000 7faef77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.424+0000 7faef77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37562/0 (socket says 172.21.15.77:37562) 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.424+0000 7faef77fe640 1 -- 172.21.15.77:0/2952607346 learned_addr learned my addr 172.21.15.77:0/2952607346 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.424+0000 7faef57fa640 1 -- 172.21.15.77:0/1732824747 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7faee80046c0 con 0x7faef81001a0 2024-09-15T22:31:46.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef7fff640 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f4efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 -- 172.21.15.77:0/2952607346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 msgr2=0x55757f4ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f4ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 -- 172.21.15.77:0/2952607346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 msgr2=0x55757f4efbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f4efbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 -- 172.21.15.77:0/2952607346 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f3e13a0 con 0x55757f502d60 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef7fff640 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f4efbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.425+0000 7faef77fe640 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7faeec02a990 tx=0x7faeec031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0db2309e25fa30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.426+0000 7faede7fc640 1 -- 172.21.15.77:0/2952607346 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec035030 con 0x55757f502d60 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.426+0000 7faef57fa640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 0x7faee00a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.426+0000 7faede7fc640 1 -- 172.21.15.77:0/2952607346 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec005520 con 0x55757f502d60 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.426+0000 7faede7fc640 1 -- 172.21.15.77:0/2952607346 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec004e60 con 0x55757f502d60 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.427+0000 7faefcf9f640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 0x7faee00a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.428+0000 7faf00ddb740 1 -- 172.21.15.77:0/2952607346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 msgr2=0x55757f505150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.428+0000 7faf00ddb740 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7faeec02a990 tx=0x7faeec031490 comp rx=0 tx=0).stop 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.428+0000 7faef57fa640 1 -- 172.21.15.77:0/1732824747 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7faee80d0460 con 0x7faef81001a0 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faefcf9f640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 0x7faee00a5a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faef0005fd0 tx=0x7faef00099c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 -- 172.21.15.77:0/2952607346 shutdown_connections 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f505150 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f4efbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 --2- 172.21.15.77:0/2952607346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f4ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 -- 172.21.15.77:0/2952607346 >> 172.21.15.77:0/2952607346 conn(0x55757f4b8310 msgr2=0x55757f4ebe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 -- 172.21.15.77:0/2952607346 shutdown_connections 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.429+0000 7faf00ddb740 1 -- 172.21.15.77:0/2952607346 wait complete. 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.430+0000 7faf00ddb740 1 -- start start 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.430+0000 7faf00ddb740 1 Processor -- start 2024-09-15T22:31:46.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f502710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 0x55757f4fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55757f505a00 con 0x55757f4f0d20 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55757f410060 con 0x55757f502d60 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55757f505b80 con 0x55757f4ef7b0 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faf00ddb740 10 client.0 _ll_register_callbacks cb 0x55757f1638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37566/0 (socket says 172.21.15.77:37566) 2024-09-15T22:31:46.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 -- 172.21.15.77:0/3555528803 learned_addr learned my addr 172.21.15.77:0/3555528803 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faefcf9f640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 0x55757f4fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef7fff640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f502710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 -- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 msgr2=0x55757f502710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f502710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 -- 172.21.15.77:0/3555528803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 msgr2=0x55757f4fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 0x55757f4fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faef77fe640 1 -- 172.21.15.77:0/3555528803 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757f1c63c0 con 0x55757f4f0d20 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.431+0000 7faefcf9f640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 0x55757f4fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.432+0000 7faef77fe640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7faeec02aa10 tx=0x7faeec0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9e15a216c6d29e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.432+0000 7faed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.433+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec052060 con 0x55757f4f0d20 2024-09-15T22:31:46.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.433+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7faeec004db0 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.433+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeec005520 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.434+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faeec035050 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.434+0000 7faf00ddb740 10 client.20001 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.434+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55757f5113c0 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.434+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec005520 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.434+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7faeec0035f0 con 0x55757f4f0d20 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 10 client.20001 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 10 client.20001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 20 client.20001 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f512710 0x55757f514b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 4 client.20001 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757f462860 con 0x55757f512710 2024-09-15T22:31:46.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.435+0000 7faef7fff640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f512710 0x55757f514b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.437+0000 7faef7fff640 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f512710 0x55757f514b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=877b0bd520a1c115 server_cookie=2dabfa9cb4f538e2 in_seq=0 out_seq=0 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.437+0000 7faed77fe640 10 client.20001 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.439+0000 7faed77fe640 1 -- 172.21.15.77:0/3555528803 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55757f462860 con 0x55757f512710 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.439+0000 7faed77fe640 10 client.20001 handle_command_reply: tid=0 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.439+0000 7faf00ddb740 1 client.20001 shutdown 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 msgr2=0x55757f4ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7faeec02aa10 tx=0x7faeec0043b0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 shutdown_connections 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f512710 0x55757f514b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f502d60 0x55757f4fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f4f0d20 0x55757f4ff9c0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 --2- 172.21.15.77:0/3555528803 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f502710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 >> 172.21.15.77:0/3555528803 conn(0x55757f4b8310 msgr2=0x55757f4f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faed77fe640 0 client.20001 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 shutdown_connections 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.440+0000 7faf00ddb740 1 -- 172.21.15.77:0/3555528803 wait complete. 2024-09-15T22:31:46.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.441+0000 7faf00ddb740 20 client.20001 trim_cache size 0 max 16384 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.457+0000 7faf00ddb740 1 Processor -- start 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 -- start start 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f501930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f4fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faef81acd90 con 0x55757f502d60 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41784/0 (socket says 172.21.15.77:41784) 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 -- 172.21.15.77:0/1593678827 learned_addr learned my addr 172.21.15.77:0/1593678827 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faefcf9f640 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f4fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef7fff640 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f501930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faef81ae8d0 con 0x55757f4f0d20 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faef81ad460 con 0x55757f4ef7b0 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 -- 172.21.15.77:0/1593678827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 msgr2=0x55757f501930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f501930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 -- 172.21.15.77:0/1593678827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 msgr2=0x55757f4fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f4fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.458+0000 7faef77fe640 1 -- 172.21.15.77:0/1593678827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f5054c0 con 0x55757f4f0d20 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.459+0000 7faef77fe640 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faeec003b60 tx=0x7faeec031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75af22192314672f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.459+0000 7faeddffb640 1 -- 172.21.15.77:0/1593678827 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec052030 con 0x55757f4f0d20 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.459+0000 7faeddffb640 1 -- 172.21.15.77:0/1593678827 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec003be0 con 0x55757f4f0d20 2024-09-15T22:31:46.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.459+0000 7faeddffb640 1 -- 172.21.15.77:0/1593678827 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec005290 con 0x55757f4f0d20 2024-09-15T22:31:46.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 msgr2=0x55757f4fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faeec003b60 tx=0x7faeec031490 comp rx=0 tx=0).stop 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 shutdown_connections 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f4fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4f0d20 0x55757f4fcc20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1593678827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4ef7b0 0x55757f501930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 >> 172.21.15.77:0/1593678827 conn(0x55757f4b8310 msgr2=0x55757f501220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 shutdown_connections 2024-09-15T22:31:46.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.460+0000 7faf00ddb740 1 -- 172.21.15.77:0/1593678827 wait complete. 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.461+0000 7faf00ddb740 1 -- start start 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.461+0000 7faf00ddb740 1 Processor -- start 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f57c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41792/0 (socket says 172.21.15.77:41792) 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38600/0 (socket says 172.21.15.77:38600) 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 -- 172.21.15.77:0/1707533614 learned_addr learned my addr 172.21.15.77:0/1707533614 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faef81ae200 con 0x55757f502d60 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faef81adb30 con 0x55757f4ef7b0 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faef81aadb0 con 0x55757f4f0d20 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faf00ddb740 10 client.0 _ll_register_callbacks cb 0x7faee8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:46.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faefcf9f640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f57c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 -- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 msgr2=0x55757f516160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 -- 172.21.15.77:0/1707533614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 msgr2=0x55757f57c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f57c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.462+0000 7faef7fff640 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757f57c5c0 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faef77fe640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faef7fff640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9c0 tx=0x7faee80d29e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ea6ab7426237c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8016330 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeec003be0 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.463+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7faee8098ac0 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faee809cb50 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 10 client.28784 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55757f57d8e0 con 0x55757f4ef7b0 2024-09-15T22:31:46.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec003be0 con 0x55757f4ef7b0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7faee80d0120 con 0x55757f4ef7b0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 10 client.28784 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 10 client.28784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 20 client.28784 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f57dfb0 0x55757f5803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 4 client.28784 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.464+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55757f45cd70 con 0x55757f57dfb0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.465+0000 7faef77fe640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f57dfb0 0x55757f5803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.465+0000 7faef77fe640 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f57dfb0 0x55757f5803a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c47ad4c1a66febf server_cookie=5c424701957ff1d0 in_seq=0 out_seq=0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.465+0000 7faed77fe640 10 client.28784 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faed77fe640 1 -- 172.21.15.77:0/1707533614 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55757f45cd70 con 0x55757f57dfb0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faed77fe640 10 client.28784 handle_command_reply: tid=0 2024-09-15T22:31:46.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 client.28784 shutdown 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 msgr2=0x55757f515c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9c0 tx=0x7faee80d29e0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 shutdown_connections 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55757f57dfb0 0x55757f5803a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757f502d60 0x55757f57c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55757f4f0d20 0x55757f516160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 --2- 172.21.15.77:0/1707533614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55757f4ef7b0 0x55757f515c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 >> 172.21.15.77:0/1707533614 conn(0x55757f4b8310 msgr2=0x55757f4eafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faed77fe640 0 client.28784 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 shutdown_connections 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 1 -- 172.21.15.77:0/1707533614 wait complete. 2024-09-15T22:31:46.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.466+0000 7faf00ddb740 20 client.28784 trim_cache size 0 max 16384 2024-09-15T22:31:46.474 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:31:46.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 msgr2=0x7faee00a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 0x7faee00a5a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faef0005fd0 tx=0x7faef00099c0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 msgr2=0x7faef8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faee800e9f0 tx=0x7faee800edf0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 shutdown_connections 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7faee00a3540 0x7faee00a5a00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef8106df0 0x7faef8101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faef8100b50 0x7faef8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 --2- 172.21.15.77:0/1732824747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef81001a0 0x7faef8101120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 >> 172.21.15.77:0/1732824747 conn(0x7faef8077070 msgr2=0x7faef80fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 shutdown_connections 2024-09-15T22:31:46.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:46.467+0000 7faefd7a0640 1 -- 172.21.15.77:0/1732824747 wait complete. 2024-09-15T22:31:46.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 113868 16.65 MB/sec execute 227 sec latency 250.975 ms 2024-09-15T22:31:47.323 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:31:47.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 114259 16.66 MB/sec execute 228 sec latency 153.282 ms 2024-09-15T22:31:47.691 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:48.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:48 smithi077 ceph-mon[30980]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 685 op/s 2024-09-15T22:31:48.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/1748337074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4108300 msgr2=0x7f6dd410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/1748337074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4108300 0x7f6dd410ec70 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009cd0 tx=0x7f6dcc02a8d0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/1748337074 shutdown_connections 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/1748337074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4108300 0x7f6dd410ec70 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/1748337074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4103780 0x7f6dd4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/1748337074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4102dd0 0x7f6dd41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.224+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/1748337074 >> 172.21.15.77:0/1748337074 conn(0x7f6dd40fc7d0 msgr2=0x7f6dd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.225+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/1748337074 shutdown_connections 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.225+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/1748337074 wait complete. 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.225+0000 7f6ddb9f5640 1 Processor -- start 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.225+0000 7f6ddb9f5640 1 -- start start 2024-09-15T22:31:48.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 0x7f6dd4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4108300 0x7f6dd419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41120d0 con 0x7f6dd4103780 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4111f50 con 0x7f6dd4108300 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6ddb9f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd4112250 con 0x7f6dd4102dd0 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 0x7f6dd4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 0x7f6dd4198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38624/0 (socket says 172.21.15.77:38624) 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd9791640 1 -- 172.21.15.77:0/3803648419 learned_addr learned my addr 172.21.15.77:0/3803648419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37608/0 (socket says 172.21.15.77:37608) 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4108300 0x7f6dd419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 -- 172.21.15.77:0/3803648419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 msgr2=0x7f6dd4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 0x7f6dd4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 -- 172.21.15.77:0/3803648419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4108300 msgr2=0x7f6dd419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4108300 0x7f6dd419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 -- 172.21.15.77:0/3803648419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd419d540 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.226+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40697d0 tx=0x7f6dc400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.227+0000 7f6dca7fc640 1 -- 172.21.15.77:0/3803648419 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400cc40 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.227+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd419d830 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.227+0000 7f6dca7fc640 1 -- 172.21.15.77:0/3803648419 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc400cde0 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.227+0000 7f6dca7fc640 1 -- 172.21.15.77:0/3803648419 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4004a00 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.227+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd4100960 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.229+0000 7f6dca7fc640 1 -- 172.21.15.77:0/3803648419 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6dc4004c00 con 0x7f6dd4103780 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.229+0000 7f6ddddce740 1 Processor -- start 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 -- start start 2024-09-15T22:31:48.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5a8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dc59f42f0 con 0x555dc5a8e5d0 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555dc599bcf0 con 0x555dc5a8fb40 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6ddddce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555dc59f02b0 con 0x555dc5a99370 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dca7fc640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 0x7f6dc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37610/0 (socket says 172.21.15.77:37610) 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd9791640 1 -- 172.21.15.77:0/2566972799 learned_addr learned my addr 172.21.15.77:0/2566972799 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 0x7f6dc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5a8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.230+0000 7f6dca7fc640 1 -- 172.21.15.77:0/3803648419 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f6dc40cffb0 con 0x7f6dd4103780 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 -- 172.21.15.77:0/2566972799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 msgr2=0x555dc5a9b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a9b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 -- 172.21.15.77:0/2566972799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 msgr2=0x555dc5a8e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5a8e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 -- 172.21.15.77:0/2566972799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc59803a0 con 0x555dc5a8e5d0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5a8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009dc0 tx=0x7f6dd0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91f22e011dc138a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 0x7f6dc00a5870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006010 tx=0x7f6dcc009360 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dbf7fe640 1 -- 172.21.15.77:0/2566972799 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0005220 con 0x555dc5a8e5d0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dbf7fe640 1 -- 172.21.15.77:0/2566972799 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0004c80 con 0x555dc5a8e5d0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.231+0000 7f6dbf7fe640 1 -- 172.21.15.77:0/2566972799 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0019860 con 0x555dc5a8e5d0 2024-09-15T22:31:48.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- 172.21.15.77:0/2566972799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 msgr2=0x555dc5a8ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009dc0 tx=0x7f6dd0011810 comp rx=0 tx=0).stop 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- 172.21.15.77:0/2566972799 shutdown_connections 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a8ea10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2566972799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5a8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- 172.21.15.77:0/2566972799 >> 172.21.15.77:0/2566972799 conn(0x555dc5a57310 msgr2=0x555dc5a8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- 172.21.15.77:0/2566972799 shutdown_connections 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- 172.21.15.77:0/2566972799 wait complete. 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.232+0000 7f6ddddce740 1 -- start start 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 Processor -- start 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8e5d0 0x555dc5a98ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555dc5a9c0c0 con 0x555dc5a8fb40 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6dd8f90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6dd8f90640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37618/0 (socket says 172.21.15.77:37618) 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6dd8f90640 1 -- 172.21.15.77:0/1117124082 learned_addr learned my addr 172.21.15.77:0/1117124082 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555dc59af060 con 0x555dc5a8e5d0 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555dc5a9c240 con 0x555dc5a99370 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6ddddce740 10 client.0 _ll_register_callbacks cb 0x555dc57028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8e5d0 0x555dc5a98ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.233+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 -- 172.21.15.77:0/1117124082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 msgr2=0x555dc5a964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 -- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8e5d0 msgr2=0x555dc5a98ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8e5d0 0x555dc5a98ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc57653c0 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c6a0 tx=0x7f6dc409baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676a613ed484c9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc401c040 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f6dc40cd590 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd0004c80 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6dc4081050 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6ddddce740 10 client.20019 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555dc5aa4940 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.234+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0004c80 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6dc409b2f0 con 0x555dc5a8fb40 2024-09-15T22:31:48.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 10 client.20019 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 10 client.20019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 20 client.20019 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5aba040 0x555dc5abc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 4 client.20019 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dc5a01860 con 0x555dc5aba040 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5aba040 0x555dc5abc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5aba040 0x555dc5abc430 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5280790e17168345 server_cookie=42df176afd79d11f in_seq=0 out_seq=0 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.235+0000 7f6dbcff9640 10 client.20019 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1117124082 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555dc5a01860 con 0x555dc5aba040 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6dbcff9640 10 client.20019 handle_command_reply: tid=0 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 client.20019 shutdown 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 msgr2=0x555dc5a95f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c6a0 tx=0x7f6dc409baa0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 shutdown_connections 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5aba040 0x555dc5abc430 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5a964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8fb40 0x555dc5a95f90 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1117124082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8e5d0 0x555dc5a98ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 >> 172.21.15.77:0/1117124082 conn(0x555dc5a57310 msgr2=0x555dc5a90960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6dbcff9640 0 client.20019 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 shutdown_connections 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.236+0000 7f6ddddce740 1 -- 172.21.15.77:0/1117124082 wait complete. 2024-09-15T22:31:48.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.237+0000 7f6ddddce740 20 client.20019 trim_cache size 0 max 16384 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.252+0000 7f6ddddce740 1 Processor -- start 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.252+0000 7f6ddddce740 1 -- start start 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5ac1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5ac2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41acfa0 con 0x555dc5a8e5d0 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41aeae0 con 0x555dc5a8fb40 2024-09-15T22:31:48.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6ddddce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41ad670 con 0x555dc5a99370 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37630/0 (socket says 172.21.15.77:37630) 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 -- 172.21.15.77:0/2385838925 learned_addr learned my addr 172.21.15.77:0/2385838925 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5ac1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5ac2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 -- 172.21.15.77:0/2385838925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 msgr2=0x555dc5ac2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5ac2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 -- 172.21.15.77:0/2385838925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 msgr2=0x555dc5ac1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5ac1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 -- 172.21.15.77:0/2385838925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc5a96860 con 0x555dc5a8e5d0 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5ac2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dd9791640 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004f50 tx=0x7f6dd0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e6ba02b1e25878 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.253+0000 7f6dbeffd640 1 -- 172.21.15.77:0/2385838925 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0031070 con 0x555dc5a8e5d0 2024-09-15T22:31:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.254+0000 7f6dbeffd640 1 -- 172.21.15.77:0/2385838925 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0005590 con 0x555dc5a8e5d0 2024-09-15T22:31:48.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.254+0000 7f6dbeffd640 1 -- 172.21.15.77:0/2385838925 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd002b5f0 con 0x555dc5a8e5d0 2024-09-15T22:31:48.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.254+0000 7f6ddddce740 1 -- 172.21.15.77:0/2385838925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 msgr2=0x555dc5a97f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.254+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004f50 tx=0x7f6dd0005780 comp rx=0 tx=0).stop 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- 172.21.15.77:0/2385838925 shutdown_connections 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a99370 0x555dc5ac2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a8fb40 0x555dc5ac1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- 172.21.15.77:0/2385838925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5a97f00 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- 172.21.15.77:0/2385838925 >> 172.21.15.77:0/2385838925 conn(0x555dc5a57310 msgr2=0x555dc5a977f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- 172.21.15.77:0/2385838925 shutdown_connections 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- 172.21.15.77:0/2385838925 wait complete. 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- start start 2024-09-15T22:31:48.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 Processor -- start 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 0x555dc5a9e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 0x555dc5a9e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41ae410 con 0x555dc5a8e5d0 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41add40 con 0x555dc5a99370 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.255+0000 7f6ddddce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41aafa0 con 0x555dc5a8fb40 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6ddddce740 10 client.0 _ll_register_callbacks cb 0x7f6dc4004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd8f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 0x555dc5a9e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37640/0 (socket says 172.21.15.77:37640) 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 -- 172.21.15.77:0/1125127409 learned_addr learned my addr 172.21.15.77:0/1125127409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 0x555dc5a9e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:38666/0 (socket says 172.21.15.77:38666) 2024-09-15T22:31:48.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 0x555dc5a9e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 -- 172.21.15.77:0/1125127409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 msgr2=0x555dc5a9e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 0x555dc5a9e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 -- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 msgr2=0x555dc5a9e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 0x555dc5a9e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 -- 172.21.15.77:0/1125127409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc5a9ef20 con 0x555dc5a8e5d0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9f92640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 0x555dc5a9e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dd9791640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005b10 tx=0x7f6dd0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa2440f4dbe5c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.256+0000 7f6dbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd003a060 con 0x555dc5a8e5d0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f6dd00035f0 con 0x555dc5a8e5d0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd0005590 con 0x555dc5a8e5d0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0041050 con 0x555dc5a8e5d0 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 10 client.20025 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:48.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555dc5b24900 con 0x555dc5a8e5d0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0005590 con 0x555dc5a8e5d0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6dd002b070 con 0x555dc5a8e5d0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 10 client.20025 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 10 client.20025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 20 client.20025 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5b24dd0 0x555dc5b271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 4 client.20025 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.257+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dc59fbd70 con 0x555dc5b24dd0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.258+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5b24dd0 0x555dc5b271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.258+0000 7f6dd8f90640 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5b24dd0 0x555dc5b271c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc1823c6f94c9ec6 server_cookie=e44a5e308bb5c7d3 in_seq=0 out_seq=0 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.258+0000 7f6dbcff9640 10 client.20025 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:31:48.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6dbcff9640 1 -- 172.21.15.77:0/1125127409 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dc59fbd70 con 0x555dc5b24dd0 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6dbcff9640 10 client.20025 handle_command_reply: tid=0 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 client.20025 shutdown 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 msgr2=0x555dc5aa3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0005b10 tx=0x7f6dd0009530 comp rx=0 tx=0).stop 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 shutdown_connections 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x555dc5b24dd0 0x555dc5b271c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555dc5a99370 0x555dc5a9e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555dc5a8fb40 0x555dc5a9e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 --2- 172.21.15.77:0/1125127409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555dc5a8e5d0 0x555dc5aa3090 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 >> 172.21.15.77:0/1125127409 conn(0x555dc5a57310 msgr2=0x555dc5a96070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6dbcff9640 0 client.20025 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 shutdown_connections 2024-09-15T22:31:48.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 1 -- 172.21.15.77:0/1125127409 wait complete. 2024-09-15T22:31:48.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.259+0000 7f6ddddce740 20 client.20025 trim_cache size 0 max 16384 2024-09-15T22:31:48.266 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:31:48.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 msgr2=0x7f6dc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 0x7f6dc00a5870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006010 tx=0x7f6dcc009360 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 msgr2=0x7f6dd4198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40697d0 tx=0x7f6dc400ee50 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 shutdown_connections 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6dc00a33b0 0x7f6dc00a5870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dd4108300 0x7f6dd419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6dd4103780 0x7f6dd4198ad0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 --2- 172.21.15.77:0/3803648419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dd4102dd0 0x7f6dd4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 >> 172.21.15.77:0/3803648419 conn(0x7f6dd40fc7d0 msgr2=0x7f6dd4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 shutdown_connections 2024-09-15T22:31:48.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:48.260+0000 7f6ddb9f5640 1 -- 172.21.15.77:0/3803648419 wait complete. 2024-09-15T22:31:48.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:48 smithi196 ceph-mon[35109]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 685 op/s 2024-09-15T22:31:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:48 smithi117 ceph-mon[35021]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 685 op/s 2024-09-15T22:31:48.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 114746 16.70 MB/sec execute 229 sec latency 247.697 ms 2024-09-15T22:31:49.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:31:49.499 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:49.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 115251 16.74 MB/sec execute 230 sec latency 142.751 ms 2024-09-15T22:31:50.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.044+0000 7fa54739d640 1 -- 172.21.15.77:0/3365839565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 msgr2=0x7fa540101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.044+0000 7fa54739d640 1 --2- 172.21.15.77:0/3365839565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa540101690 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009900 tx=0x7fa53c02a410 comp rx=0 tx=0).stop 2024-09-15T22:31:50.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 -- 172.21.15.77:0/3365839565 shutdown_connections 2024-09-15T22:31:50.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 --2- 172.21.15.77:0/3365839565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 0x7fa54010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 --2- 172.21.15.77:0/3365839565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 0x7fa540105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 --2- 172.21.15.77:0/3365839565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa540101690 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 -- 172.21.15.77:0/3365839565 >> 172.21.15.77:0/3365839565 conn(0x7fa540077070 msgr2=0x7fa5400fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 -- 172.21.15.77:0/3365839565 shutdown_connections 2024-09-15T22:31:50.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 -- 172.21.15.77:0/3365839565 wait complete. 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 Processor -- start 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.045+0000 7fa54739d640 1 -- start start 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 0x7fa54019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 0x7fa54019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa540110d50 con 0x7fa5401012b0 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa540110bd0 con 0x7fa540101c60 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54739d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa540110ed0 con 0x7fa540105cb0 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49116/0 (socket says 172.21.15.77:49116) 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 -- 172.21.15.77:0/4034972163 learned_addr learned my addr 172.21.15.77:0/4034972163 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:50.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa545b9a640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 0x7fa54019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa546b9c640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 0x7fa54019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 -- 172.21.15.77:0/4034972163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 msgr2=0x7fa54019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 0x7fa54019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 -- 172.21.15.77:0/4034972163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 msgr2=0x7fa54019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 0x7fa54019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 -- 172.21.15.77:0/4034972163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54019f490 con 0x7fa5401012b0 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.046+0000 7fa54639b640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006010 tx=0x7fa53c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.047+0000 7fa5377fe640 1 -- 172.21.15.77:0/4034972163 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c03f030 con 0x7fa5401012b0 2024-09-15T22:31:50.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.047+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54019f720 con 0x7fa5401012b0 2024-09-15T22:31:50.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.047+0000 7fa5377fe640 1 -- 172.21.15.77:0/4034972163 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c004440 con 0x7fa5401012b0 2024-09-15T22:31:50.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.047+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a7060 con 0x7fa5401012b0 2024-09-15T22:31:50.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.047+0000 7fa5377fe640 1 -- 172.21.15.77:0/4034972163 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c049060 con 0x7fa5401012b0 2024-09-15T22:31:50.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 Processor -- start 2024-09-15T22:31:50.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 -- start start 2024-09-15T22:31:50.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa5377fe640 1 -- 172.21.15.77:0/4034972163 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa53c042030 con 0x7fa5401012b0 2024-09-15T22:31:50.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae897b0 0x55966ae89bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae94550 0x55966ae96940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55966adef2f0 con 0x55966ae897b0 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.050+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55966ad96cf0 con 0x55966ae8ad20 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55966adeb2b0 con 0x55966ae94550 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36472/0 (socket says 172.21.15.77:36472) 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 -- 172.21.15.77:0/887990600 learned_addr learned my addr 172.21.15.77:0/887990600 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa545b9a640 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae94550 0x55966ae96940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 -- 172.21.15.77:0/887990600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae94550 msgr2=0x55966ae96940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae94550 0x55966ae96940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 -- 172.21.15.77:0/887990600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae897b0 msgr2=0x55966ae89bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae897b0 0x55966ae89bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 -- 172.21.15.77:0/887990600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966ad7b3a0 con 0x55966ae8ad20 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.051+0000 7fa546b9c640 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa538009f50 tx=0x7fa5380118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af40f9c8e89e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa5377fe640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 0x7fa52c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa534ff9640 1 -- 172.21.15.77:0/887990600 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538004d20 con 0x55966ae8ad20 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa534ff9640 1 -- 172.21.15.77:0/887990600 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538004050 con 0x55966ae8ad20 2024-09-15T22:31:50.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa534ff9640 1 -- 172.21.15.77:0/887990600 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538005280 con 0x55966ae8ad20 2024-09-15T22:31:50.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa54639b640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 0x7fa52c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.052+0000 7fa5377fe640 1 -- 172.21.15.77:0/4034972163 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7fa53c0f03c0 con 0x7fa5401012b0 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 -- 172.21.15.77:0/887990600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 msgr2=0x55966ae89270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa538009f50 tx=0x7fa5380118d0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 -- 172.21.15.77:0/887990600 shutdown_connections 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae94550 0x55966ae96940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae897b0 0x55966ae89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/887990600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae8ad20 0x55966ae89270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 -- 172.21.15.77:0/887990600 >> 172.21.15.77:0/887990600 conn(0x55966ae52310 msgr2=0x55966ae85e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 -- 172.21.15.77:0/887990600 shutdown_connections 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54a9d8740 1 -- 172.21.15.77:0/887990600 wait complete. 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.053+0000 7fa54639b640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 0x7fa52c0a5870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55966ae86ca0 tx=0x7fa53c086e70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 -- start start 2024-09-15T22:31:50.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 Processor -- start 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae8ad20 0x55966ae911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae94550 0x55966ae91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55966ae97250 con 0x55966ae8ad20 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55966adaa060 con 0x55966ae94550 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55966ae973d0 con 0x55966ae897b0 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45270/0 (socket says 172.21.15.77:45270) 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa545b9a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae8ad20 0x55966ae911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa546b9c640 1 -- 172.21.15.77:0/1901658931 learned_addr learned my addr 172.21.15.77:0/1901658931 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:50.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.054+0000 7fa54a9d8740 10 client.0 _ll_register_callbacks cb 0x55966aafd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae94550 0x55966ae91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 -- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae94550 msgr2=0x55966ae91800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae94550 0x55966ae91800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 -- 172.21.15.77:0/1901658931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae8ad20 msgr2=0x55966ae911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae8ad20 0x55966ae911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 -- 172.21.15.77:0/1901658931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966ab603c0 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa538004740 tx=0x7fa538004910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9496bcca190aed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53802c020 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa538004050 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.055+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7fa538028450 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.056+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa538028650 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.056+0000 7fa54a9d8740 10 client.28723 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.056+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55966aea2cc0 con 0x55966ae897b0 2024-09-15T22:31:50.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.056+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538004050 con 0x55966ae897b0 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa53800f940 con 0x55966ae897b0 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 10 client.28723 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 10 client.28723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 20 client.28723 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966aea4010 0x55966aea6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 4 client.28723 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55966adfc860 con 0x55966aea4010 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.057+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966aea4010 0x55966aea6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.058+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966aea4010 0x55966aea6400 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b3f35f33915e0d server_cookie=f49cc9df70a38627 in_seq=0 out_seq=0 2024-09-15T22:31:50.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.058+0000 7fa529ffb640 10 client.28723 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.059+0000 7fa529ffb640 1 -- 172.21.15.77:0/1901658931 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55966adfc860 con 0x55966aea4010 2024-09-15T22:31:50.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.059+0000 7fa529ffb640 10 client.28723 handle_command_reply: tid=0 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.059+0000 7fa54a9d8740 1 client.28723 shutdown 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.059+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 msgr2=0x55966ae93ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.059+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa538004740 tx=0x7fa538004910 comp rx=0 tx=0).stop 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 shutdown_connections 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966aea4010 0x55966aea6400 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae94550 0x55966ae91800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa529ffb640 0 client.28723 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae8ad20 0x55966ae911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1901658931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae897b0 0x55966ae93ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 >> 172.21.15.77:0/1901658931 conn(0x55966ae52310 msgr2=0x55966ae8bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 shutdown_connections 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1901658931 wait complete. 2024-09-15T22:31:50.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.060+0000 7fa54a9d8740 20 client.28723 trim_cache size 0 max 16384 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.076+0000 7fa54a9d8740 1 Processor -- start 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.076+0000 7fa54a9d8740 1 -- start start 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966aea04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966aeabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401b1070 con 0x55966ae52310 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401b2bb0 con 0x55966ae897b0 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401b1740 con 0x55966ae8ad20 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa545b9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa545b9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36494/0 (socket says 172.21.15.77:36494) 2024-09-15T22:31:50.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa545b9a640 1 -- 172.21.15.77:0/1995522619 learned_addr learned my addr 172.21.15.77:0/1995522619 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966aea04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966aeabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa545b9a640 1 -- 172.21.15.77:0/1995522619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 msgr2=0x55966aeabc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.077+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966aeabc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa545b9a640 1 -- 172.21.15.77:0/1995522619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 msgr2=0x55966aea04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966aea04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa545b9a640 1 -- 172.21.15.77:0/1995522619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966ae91b70 con 0x55966ae897b0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966aeabc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa540105a00 tx=0x7fa530005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ea250c28a62b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa52bfff640 1 -- 172.21.15.77:0/1995522619 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530022070 con 0x55966ae897b0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa52bfff640 1 -- 172.21.15.77:0/1995522619 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530005500 con 0x55966ae897b0 2024-09-15T22:31:50.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.078+0000 7fa52bfff640 1 -- 172.21.15.77:0/1995522619 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53001f690 con 0x55966ae897b0 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1995522619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 msgr2=0x55966ae84e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa540105a00 tx=0x7fa530005f90 comp rx=0 tx=0).stop 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1995522619 shutdown_connections 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966aeabc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae84e90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1995522619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966aea04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1995522619 >> 172.21.15.77:0/1995522619 conn(0x55966aea6770 msgr2=0x55966ae84780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1995522619 shutdown_connections 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1995522619 wait complete. 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.079+0000 7fa54a9d8740 1 -- start start 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 Processor -- start 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae97fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966ae984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401b24e0 con 0x55966ae52310 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401b1e10 con 0x55966ae897b0 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401af070 con 0x55966ae8ad20 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa54a9d8740 10 client.0 _ll_register_callbacks cb 0x7fa538009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa546b9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49144/0 (socket says 172.21.15.77:49144) 2024-09-15T22:31:50.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa545b9a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae97fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa546b9c640 1 -- 172.21.15.77:0/1977657023 learned_addr learned my addr 172.21.15.77:0/1977657023 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.080+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966ae984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 -- 172.21.15.77:0/1977657023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 msgr2=0x55966ae984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966ae984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 -- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 msgr2=0x55966ae97fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae97fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 -- 172.21.15.77:0/1977657023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966ae98bd0 con 0x55966ae52310 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa546b9c640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa538005d70 tx=0x7fa5380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825a528166562d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53802c070 con 0x55966ae52310 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7fa538028ac0 con 0x55966ae52310 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa530005500 con 0x55966ae52310 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa538028cc0 con 0x55966ae52310 2024-09-15T22:31:50.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa54a9d8740 10 client.20040 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.081+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55966af0eaa0 con 0x55966ae52310 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53800f960 con 0x55966ae52310 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa538026960 con 0x55966ae52310 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 10 client.20040 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 10 client.20040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 20 client.20040 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966ae94550 0x55966af11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 4 client.20040 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.082+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa530005500 con 0x55966ae94550 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.083+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966ae94550 0x55966af11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.083+0000 7fa529ffb640 10 client.20040 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:50.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.083+0000 7fa545b9a640 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966ae94550 0x55966af11110 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578361f2071a8651 server_cookie=1c149903284163f8 in_seq=0 out_seq=0 2024-09-15T22:31:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa529ffb640 1 -- 172.21.15.77:0/1977657023 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa530005500 con 0x55966ae94550 2024-09-15T22:31:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa529ffb640 10 client.20040 handle_command_reply: tid=0 2024-09-15T22:31:50.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 client.20040 shutdown 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 msgr2=0x55966ae9cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa538005d70 tx=0x7fa5380118d0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 shutdown_connections 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55966ae94550 0x55966af11110 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55966ae8ad20 0x55966ae984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55966ae897b0 0x55966ae97fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 --2- 172.21.15.77:0/1977657023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55966ae52310 0x55966ae9cc70 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 >> 172.21.15.77:0/1977657023 conn(0x55966aea6770 msgr2=0x55966ae84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa529ffb640 0 client.20040 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 shutdown_connections 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.085+0000 7fa54a9d8740 1 -- 172.21.15.77:0/1977657023 wait complete. 2024-09-15T22:31:50.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54a9d8740 20 client.20040 trim_cache size 0 max 16384 2024-09-15T22:31:50.091 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:31:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 msgr2=0x7fa52c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 0x7fa52c0a5870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55966ae86ca0 tx=0x7fa53c086e70 comp rx=0 tx=0).stop 2024-09-15T22:31:50.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 msgr2=0x7fa54019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa53c006010 tx=0x7fa53c004040 comp rx=0 tx=0).stop 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 shutdown_connections 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa52c0a33b0 0x7fa52c0a5870 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa540105cb0 0x7fa54019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa540101c60 0x7fa54019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 --2- 172.21.15.77:0/4034972163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5401012b0 0x7fa54019a4e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.086+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 >> 172.21.15.77:0/4034972163 conn(0x7fa540077070 msgr2=0x7fa5401033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.087+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 shutdown_connections 2024-09-15T22:31:50.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:50.087+0000 7fa54739d640 1 -- 172.21.15.77:0/4034972163 wait complete. 2024-09-15T22:31:50.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:50 smithi077 ceph-mon[30980]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-09-15T22:31:50.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:50 smithi196 ceph-mon[35109]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-09-15T22:31:50.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:50 smithi117 ceph-mon[35021]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 379 op/s 2024-09-15T22:31:50.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 115825 16.83 MB/sec execute 231 sec latency 157.945 ms 2024-09-15T22:31:51.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 116199 16.85 MB/sec execute 232 sec latency 170.568 ms 2024-09-15T22:31:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:52 smithi077 ceph-mon[30980]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:31:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:52 smithi196 ceph-mon[35109]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:31:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:52 smithi117 ceph-mon[35021]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-09-15T22:31:52.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:31:52.398 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:31:52.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 116605 16.84 MB/sec execute 233 sec latency 174.115 ms 2024-09-15T22:31:52.718 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:53.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.259+0000 7f7f8b595640 1 -- 172.21.15.77:0/1460334802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f841027a0 msgr2=0x7f7f841067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 --2- 172.21.15.77:0/1460334802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f841027a0 0x7f7f841067f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009900 tx=0x7f7f7402a410 comp rx=0 tx=0).stop 2024-09-15T22:31:53.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 -- 172.21.15.77:0/1460334802 shutdown_connections 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 --2- 172.21.15.77:0/1460334802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f84106f20 0x7f7f8410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 --2- 172.21.15.77:0/1460334802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f841027a0 0x7f7f841067f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 --2- 172.21.15.77:0/1460334802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84101df0 0x7f7f841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 -- 172.21.15.77:0/1460334802 >> 172.21.15.77:0/1460334802 conn(0x7f7f840fd8f0 msgr2=0x7f7f840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 -- 172.21.15.77:0/1460334802 shutdown_connections 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.260+0000 7f7f8b595640 1 -- 172.21.15.77:0/1460334802 wait complete. 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 Processor -- start 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 -- start start 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 0x7f7f84198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84106f20 0x7f7f8419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84111f80 con 0x7f7f84101df0 2024-09-15T22:31:53.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84111e00 con 0x7f7f84106f20 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f8b595640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84112100 con 0x7f7f841027a0 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f88b30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f88b30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45312/0 (socket says 172.21.15.77:45312) 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f88b30640 1 -- 172.21.15.77:0/4293583343 learned_addr learned my addr 172.21.15.77:0/4293583343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f89331640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 0x7f7f84198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.261+0000 7f7f89b32640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84106f20 0x7f7f8419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 -- 172.21.15.77:0/4293583343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84106f20 msgr2=0x7f7f8419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84106f20 0x7f7f8419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 -- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 msgr2=0x7f7f84198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 0x7f7f84198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 -- 172.21.15.77:0/4293583343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8419d3f0 con 0x7f7f841027a0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f89331640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 0x7f7f84198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f88b30640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f74004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f7a7fc640 1 -- 172.21.15.77:0/4293583343 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7403f030 con 0x7f7f841027a0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f7a7fc640 1 -- 172.21.15.77:0/4293583343 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f74004440 con 0x7f7f841027a0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8419d680 con 0x7f7f841027a0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f7a7fc640 1 -- 172.21.15.77:0/4293583343 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74035070 con 0x7f7f841027a0 2024-09-15T22:31:53.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.262+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a4fc0 con 0x7f7f841027a0 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 Processor -- start 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 -- start start 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560540111280 con 0x5605401b6d40 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5605400b8b10 con 0x5605401abb60 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56054010d240 con 0x5605401ac480 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f7a7fc640 1 -- 172.21.15.77:0/4293583343 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7f74042030 con 0x7f7f841027a0 2024-09-15T22:31:53.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f89331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f89331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45318/0 (socket says 172.21.15.77:45318) 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f89331640 1 -- 172.21.15.77:0/3430436516 learned_addr learned my addr 172.21.15.77:0/3430436516 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.266+0000 7f7f89b32640 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 -- 172.21.15.77:0/3430436516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 msgr2=0x5605401abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 -- 172.21.15.77:0/3430436516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 msgr2=0x5605401b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 -- 172.21.15.77:0/3430436516 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605400c51f0 con 0x5605401ac480 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f89331640 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005f40 tx=0x7f7f80011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d1feaf9e0799272 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f6f7fe640 1 -- 172.21.15.77:0/3430436516 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80011330 con 0x5605401ac480 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f6f7fe640 1 -- 172.21.15.77:0/3430436516 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f8000fe80 con 0x5605401ac480 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f6f7fe640 1 -- 172.21.15.77:0/3430436516 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f800193e0 con 0x5605401ac480 2024-09-15T22:31:53.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.267+0000 7f7f7a7fc640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 0x7f7f700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f7a7fc640 1 -- 172.21.15.77:0/4293583343 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f7f740f01d0 con 0x7f7f841027a0 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f89b32640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 0x7f7f700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3430436516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 msgr2=0x5605401b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005f40 tx=0x7f7f80011d30 comp rx=0 tx=0).stop 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f89b32640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 0x7f7f700a5870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009c70 tx=0x7f7f7c005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3430436516 shutdown_connections 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b6800 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3430436516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.268+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3430436516 >> 172.21.15.77:0/3430436516 conn(0x5605401a80f0 msgr2=0x5605401a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.269+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3430436516 shutdown_connections 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.269+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3430436516 wait complete. 2024-09-15T22:31:53.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.269+0000 7f7f8d96e740 1 -- start start 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.269+0000 7f7f8d96e740 1 Processor -- start 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401b6d40 0x5605401b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5605401b9910 con 0x5605401abb60 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5605400cbf10 con 0x5605401b6d40 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5605401b9a90 con 0x5605401ac480 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f89331640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f89331640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49190/0 (socket says 172.21.15.77:49190) 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f89331640 1 -- 172.21.15.77:0/828357536 learned_addr learned my addr 172.21.15.77:0/828357536 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f88b30640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f89b32640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401b6d40 0x5605401b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.270+0000 7f7f8d96e740 10 client.0 _ll_register_callbacks cb 0x560540094a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 -- 172.21.15.77:0/828357536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 msgr2=0x5605401b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 -- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401b6d40 msgr2=0x5605401b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401b6d40 0x5605401b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 -- 172.21.15.77:0/828357536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56053fe826a0 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f89331640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7f80000f80 tx=0x7f7f80011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e13c84d5154e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8001f040 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56054011e7f0 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.271+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f7f80004c40 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f80037050 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 10 client.20049 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5605401c5550 con 0x5605401abb60 2024-09-15T22:31:53.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56054011e7f0 con 0x5605401abb60 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7f80018070 con 0x5605401abb60 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 10 client.20049 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 10 client.20049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 20 client.20049 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x5605401c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 4 client.20049 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.272+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560540118d00 con 0x5605401c6960 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.273+0000 7f7f88b30640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x5605401c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.273+0000 7f7f88b30640 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x5605401c8d50 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90753cc8c000ccd3 server_cookie=1b58228481242580 in_seq=0 out_seq=0 2024-09-15T22:31:53.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.273+0000 7f7f6cff9640 10 client.20049 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.275+0000 7f7f6cff9640 1 -- 172.21.15.77:0/828357536 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560540118d00 con 0x5605401c6960 2024-09-15T22:31:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.275+0000 7f7f6cff9640 10 client.20049 handle_command_reply: tid=0 2024-09-15T22:31:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 client.20049 shutdown 2024-09-15T22:31:53.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 msgr2=0x5605401b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7f80000f80 tx=0x7f7f80011440 comp rx=0 tx=0).stop 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 shutdown_connections 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x5605401c8d50 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f6cff9640 0 client.20049 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401b6d40 0x5605401b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/828357536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401abb60 0x5605401b3c80 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 >> 172.21.15.77:0/828357536 conn(0x5605401a80f0 msgr2=0x5605401addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 shutdown_connections 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.276+0000 7f7f8d96e740 1 -- 172.21.15.77:0/828357536 wait complete. 2024-09-15T22:31:53.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.277+0000 7f7f8d96e740 20 client.20049 trim_cache size 0 max 16384 2024-09-15T22:31:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.296+0000 7f7f8d96e740 1 Processor -- start 2024-09-15T22:31:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 -- start start 2024-09-15T22:31:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401abb60 0x5605401adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401ac480 0x5605401ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841aceb0 con 0x5605401b6d40 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841ae9f0 con 0x5605401ac480 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841ad580 con 0x5605401abb60 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f89b32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f89b32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49194/0 (socket says 172.21.15.77:49194) 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f89b32640 1 -- 172.21.15.77:0/2964462300 learned_addr learned my addr 172.21.15.77:0/2964462300 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.297+0000 7f7f89331640 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401abb60 0x5605401adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 -- 172.21.15.77:0/2964462300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401abb60 msgr2=0x5605401adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401abb60 0x5605401adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 -- 172.21.15.77:0/2964462300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401ac480 msgr2=0x5605401ce490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401ac480 0x5605401ce490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 -- 172.21.15.77:0/2964462300 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605401a9280 con 0x5605401b6d40 2024-09-15T22:31:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f89b32640 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009fd0 tx=0x7f7f7c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83ebf733e1eaf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f6effd640 1 -- 172.21.15.77:0/2964462300 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004050 con 0x5605401b6d40 2024-09-15T22:31:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f6effd640 1 -- 172.21.15.77:0/2964462300 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c0041f0 con 0x5605401b6d40 2024-09-15T22:31:53.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.298+0000 7f7f6effd640 1 -- 172.21.15.77:0/2964462300 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c011900 con 0x5605401b6d40 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.299+0000 7f7f8d96e740 1 -- 172.21.15.77:0/2964462300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 msgr2=0x5605401ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.299+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009fd0 tx=0x7f7f7c01a000 comp rx=0 tx=0).stop 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 -- 172.21.15.77:0/2964462300 shutdown_connections 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401ce9d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401ac480 0x5605401ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/2964462300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401abb60 0x5605401adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 -- 172.21.15.77:0/2964462300 >> 172.21.15.77:0/2964462300 conn(0x5605401a80f0 msgr2=0x5605401ad5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 -- 172.21.15.77:0/2964462300 shutdown_connections 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 -- 172.21.15.77:0/2964462300 wait complete. 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 -- start start 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.300+0000 7f7f8d96e740 1 Processor -- start 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841ae320 con 0x5605401b6d40 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841adc50 con 0x5605401abb60 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841aaeb0 con 0x5605401ac480 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f8d96e740 10 client.0 _ll_register_callbacks cb 0x7f7f7c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89331640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89b32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89b32640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49208/0 (socket says 172.21.15.77:49208) 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45350/0 (socket says 172.21.15.77:45350) 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89331640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401bf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36584/0 (socket says 172.21.15.77:36584) 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89b32640 1 -- 172.21.15.77:0/3274448480 learned_addr learned my addr 172.21.15.77:0/3274448480 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 -- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 msgr2=0x5605401bf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401bf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 -- 172.21.15.77:0/3274448480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 msgr2=0x5605401baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 -- 172.21.15.77:0/3274448480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605401bb3d0 con 0x5605401ac480 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f89b32640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f88b30640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7f740ed590 tx=0x7f7f74009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb75453f92594683 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.301+0000 7f7f6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f740c04d0 con 0x5605401ac480 2024-09-15T22:31:53.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c0041f0 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f7f74003cd0 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f740c4d90 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 10 client.28747 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5605402423b0 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7c0041f0 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7f740edce0 con 0x5605401ac480 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 10 client.28747 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 10 client.28747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 20 client.28747 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x560540244b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 4 client.28747 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560540095110 con 0x5605401c6960 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.302+0000 7f7f89331640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x560540244b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.303+0000 7f7f6cff9640 10 client.28747 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.303+0000 7f7f89331640 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x560540244b70 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc181bbdf55842d server_cookie=86a2f58acec32f2e in_seq=0 out_seq=0 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f6cff9640 1 -- 172.21.15.77:0/3274448480 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560540095110 con 0x5605401c6960 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f6cff9640 10 client.28747 handle_command_reply: tid=0 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 client.28747 shutdown 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 msgr2=0x5605401ba660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7f740ed590 tx=0x7f7f74009880 comp rx=0 tx=0).stop 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 shutdown_connections 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5605401c6960 0x560540244b70 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5605401b6d40 0x5605401baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605401ac480 0x5605401ba660 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7f740ed590 tx=0x7f7f74009880 comp rx=0 tx=0).stop 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.305+0000 7f7f8d96e740 1 --2- 172.21.15.77:0/3274448480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605401abb60 0x5605401bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.306+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 >> 172.21.15.77:0/3274448480 conn(0x5605401a80f0 msgr2=0x5605401b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.306+0000 7f7f6cff9640 0 client.28747 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.306+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 shutdown_connections 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.306+0000 7f7f8d96e740 1 -- 172.21.15.77:0/3274448480 wait complete. 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.306+0000 7f7f8d96e740 20 client.28747 trim_cache size 0 max 16384 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "cbffa5ee-dd6b-455e-be51-6f8d40e8030c", 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:31:53.313 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 msgr2=0x7f7f700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 0x7f7f700a5870 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009c70 tx=0x7f7f7c005f90 comp rx=0 tx=0).stop 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 msgr2=0x7f7f84198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006010 tx=0x7f7f74004040 comp rx=0 tx=0).stop 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 shutdown_connections 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7f700a33b0 0x7f7f700a5870 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f84106f20 0x7f7f8419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f841027a0 0x7f7f84198980 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 --2- 172.21.15.77:0/4293583343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84101df0 0x7f7f84198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 >> 172.21.15.77:0/4293583343 conn(0x7f7f840fd8f0 msgr2=0x7f7f840fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 shutdown_connections 2024-09-15T22:31:53.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:53.307+0000 7f7f8b595640 1 -- 172.21.15.77:0/4293583343 wait complete. 2024-09-15T22:31:53.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 116982 16.86 MB/sec execute 234 sec latency 210.808 ms 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:54.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:54.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:54.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.123 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:54 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.160 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:31:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:54 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 465 op/s 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:54.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.364 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:54 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:54.488 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:54.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 117459 16.87 MB/sec execute 235 sec latency 147.654 ms 2024-09-15T22:31:54.627 INFO:tasks.quiescer.fs.[cephfs]:Total duration (56.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:31:54.627 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.15 2024-09-15T22:31:54.628 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:31:55.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 -- 172.21.15.77:0/4234161463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38441055b0 msgr2=0x7f384410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 --2- 172.21.15.77:0/4234161463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38441055b0 0x7f384410d130 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f383c00adb0 tx=0x7f383c02b7a0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 -- 172.21.15.77:0/4234161463 shutdown_connections 2024-09-15T22:31:55.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 --2- 172.21.15.77:0/4234161463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38441055b0 0x7f384410d130 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 --2- 172.21.15.77:0/4234161463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 --2- 172.21.15.77:0/4234161463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3844100480 0x7f3844100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 -- 172.21.15.77:0/4234161463 >> 172.21.15.77:0/4234161463 conn(0x7f3844077210 msgr2=0x7f38440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 -- 172.21.15.77:0/4234161463 shutdown_connections 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.050+0000 7f384968e640 1 -- 172.21.15.77:0/4234161463 wait complete. 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.051+0000 7f384968e640 1 Processor -- start 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.051+0000 7f384968e640 1 -- start start 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3844100480 0x7f3844196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 0x7f384419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fde0 con 0x7f3844100480 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fc60 con 0x7f3844100e30 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f384968e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f384410ff60 con 0x7f38441055b0 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36600/0 (socket says 172.21.15.77:36600) 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 -- 172.21.15.77:0/829407818 learned_addr learned my addr 172.21.15.77:0/829407818 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38437fe640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 0x7f384419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f3842ffd640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3844100480 0x7f3844196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 -- 172.21.15.77:0/829407818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 msgr2=0x7f384419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 0x7f384419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 -- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3844100480 msgr2=0x7f3844196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3844100480 0x7f3844196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 -- 172.21.15.77:0/829407818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384419b010 con 0x7f3844100e30 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38437fe640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 0x7f384419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.052+0000 7f38427fc640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9f0 tx=0x7f383800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.053+0000 7f382bfff640 1 -- 172.21.15.77:0/829407818 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383800cd80 con 0x7f3844100e30 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.053+0000 7f382bfff640 1 -- 172.21.15.77:0/829407818 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3838010910 con 0x7f3844100e30 2024-09-15T22:31:55.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.053+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384419b2a0 con 0x7f3844100e30 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.053+0000 7f382bfff640 1 -- 172.21.15.77:0/829407818 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018990 con 0x7f3844100e30 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.053+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384419b790 con 0x7f3844100e30 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.056+0000 7f382bfff640 1 -- 172.21.15.77:0/829407818 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3838010ac0 con 0x7f3844100e30 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.056+0000 7f384ba67740 1 Processor -- start 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.056+0000 7f384ba67740 1 -- start start 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0820950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a0823280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a07732e0 con 0x5557a0815cb0 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557a071ace0 con 0x5557a08165d0 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f384ba67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5557a076f2a0 con 0x5557a0820e90 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f38437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f38437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49230/0 (socket says 172.21.15.77:49230) 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f38437fe640 1 -- 172.21.15.77:0/1357564730 learned_addr learned my addr 172.21.15.77:0/1357564730 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f38427fc640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a0823280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.057+0000 7f3842ffd640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0820950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f382bfff640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 0x7f382c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 -- 172.21.15.77:0/1357564730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 msgr2=0x5557a0823280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a0823280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 -- 172.21.15.77:0/1357564730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 msgr2=0x5557a0820950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0820950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 -- 172.21.15.77:0/1357564730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a06ff440 con 0x5557a0815cb0 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f3842ffd640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0820950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.058+0000 7f38437fe640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 0x7f382c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.059+0000 7f38437fe640 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f383c035000 tx=0x7f383c003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a42df67efa8814e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.059+0000 7f382bfff640 1 -- 172.21.15.77:0/829407818 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f38380d03f0 con 0x7f3844100e30 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.059+0000 7f38437fe640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 0x7f382c0a5870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f383c003fd0 tx=0x7f383c048000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.062+0000 7f38297fa640 1 -- 172.21.15.77:0/1357564730 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c04e070 con 0x5557a0815cb0 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.062+0000 7f38297fa640 1 -- 172.21.15.77:0/1357564730 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c0328e0 con 0x5557a0815cb0 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.062+0000 7f38297fa640 1 -- 172.21.15.77:0/1357564730 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c030b00 con 0x5557a0815cb0 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- 172.21.15.77:0/1357564730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 msgr2=0x5557a0816090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f383c035000 tx=0x7f383c003af0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- 172.21.15.77:0/1357564730 shutdown_connections 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a0823280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0820950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 --2- 172.21.15.77:0/1357564730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0816090 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- 172.21.15.77:0/1357564730 >> 172.21.15.77:0/1357564730 conn(0x5557a0812240 msgr2=0x5557a0812620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- 172.21.15.77:0/1357564730 shutdown_connections 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- 172.21.15.77:0/1357564730 wait complete. 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 -- start start 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.063+0000 7f384ba67740 1 Processor -- start 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a081df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557a0823a60 con 0x5557a08165d0 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557a072e050 con 0x5557a0815cb0 2024-09-15T22:31:55.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5557a0823be0 con 0x5557a0820e90 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f384ba67740 10 client.0 _ll_register_callbacks cb 0x5557a076e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38437fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a081df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49232/0 (socket says 172.21.15.77:49232) 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36618/0 (socket says 172.21.15.77:36618) 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 -- 172.21.15.77:0/980665520 learned_addr learned my addr 172.21.15.77:0/980665520 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 -- 172.21.15.77:0/980665520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 msgr2=0x5557a081df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a081df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 -- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 msgr2=0x5557a0820760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 -- 172.21.15.77:0/980665520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a04e43c0 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f3842ffd640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f38427fc640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9c0 tx=0x7f383809bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ad08278f314c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.064+0000 7f3826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838026040 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c0328e0 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f3838018b30 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f383809c340 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 10 client.20067 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5557a082f430 con 0x5557a08165d0 2024-09-15T22:31:55.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c0328e0 con 0x5557a08165d0 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f383809b790 con 0x5557a08165d0 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 10 client.20067 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 10 client.20067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 20 client.20067 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a0830cc0 0x5557a0833130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 4 client.20067 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:55.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.065+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a0780850 con 0x5557a0830cc0 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.068+0000 7f3842ffd640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a0830cc0 0x5557a0833130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.069+0000 7f3842ffd640 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a0830cc0 0x5557a0833130 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6fcc58abe3b329 server_cookie=8b5af84ed55e8e53 in_seq=0 out_seq=0 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.069+0000 7f3826ff5640 10 client.20067 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.071+0000 7f3826ff5640 1 -- 172.21.15.77:0/980665520 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557a0780850 con 0x5557a0830cc0 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.071+0000 7f3826ff5640 10 client.20067 handle_command_reply: tid=0 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.071+0000 7f384ba67740 1 client.20067 shutdown 2024-09-15T22:31:55.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.071+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 msgr2=0x5557a081da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.071+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9c0 tx=0x7f383809bf40 comp rx=0 tx=0).stop 2024-09-15T22:31:55.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 shutdown_connections 2024-09-15T22:31:55.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a0830cc0 0x5557a0833130 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a081df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a08165d0 0x5557a081da10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 --2- 172.21.15.77:0/980665520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a0815cb0 0x5557a0820760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 >> 172.21.15.77:0/980665520 conn(0x5557a0812240 msgr2=0x5557a0808b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f3826ff5640 0 client.20067 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 shutdown_connections 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.072+0000 7f384ba67740 1 -- 172.21.15.77:0/980665520 wait complete. 2024-09-15T22:31:55.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.073+0000 7f384ba67740 20 client.20067 trim_cache size 0 max 16384 2024-09-15T22:31:55.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.098+0000 7f384ba67740 1 Processor -- start 2024-09-15T22:31:55.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.098+0000 7f384ba67740 1 -- start start 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 0x5557a082ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0812620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0820e90 0x5557a0819000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38441acdf0 con 0x5557a0820e90 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 0x5557a082ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 0x5557a082ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45400/0 (socket says 172.21.15.77:45400) 2024-09-15T22:31:55.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f3842ffd640 1 -- 172.21.15.77:0/965410894 learned_addr learned my addr 172.21.15.77:0/965410894 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ae930 con 0x5557a08165d0 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ad4c0 con 0x5557a0815cb0 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f38427fc640 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0812620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.099+0000 7f38437fe640 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0820e90 0x5557a0819000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.100+0000 7f38427fc640 1 -- 172.21.15.77:0/965410894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 msgr2=0x5557a082ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.100+0000 7f38427fc640 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 0x5557a082ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.100+0000 7f38427fc640 1 -- 172.21.15.77:0/965410894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0820e90 msgr2=0x5557a0819000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.100+0000 7f38427fc640 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0820e90 0x5557a0819000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.100+0000 7f38427fc640 1 -- 172.21.15.77:0/965410894 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a081c9c0 con 0x5557a08165d0 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.101+0000 7f38427fc640 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0812620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f383800cfd0 tx=0x7f3838000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d496427a5a3b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.101+0000 7f3828ff9640 1 -- 172.21.15.77:0/965410894 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38380a06f0 con 0x5557a08165d0 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.101+0000 7f3828ff9640 1 -- 172.21.15.77:0/965410894 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38380a4fb0 con 0x5557a08165d0 2024-09-15T22:31:55.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.101+0000 7f3828ff9640 1 -- 172.21.15.77:0/965410894 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838018b30 con 0x5557a08165d0 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 msgr2=0x5557a0812620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0812620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f383800cfd0 tx=0x7f3838000bc0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 shutdown_connections 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0820e90 0x5557a0819000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0812620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 --2- 172.21.15.77:0/965410894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0815cb0 0x5557a082ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 >> 172.21.15.77:0/965410894 conn(0x5557a0812240 msgr2=0x5557a08188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 shutdown_connections 2024-09-15T22:31:55.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.102+0000 7f384ba67740 1 -- 172.21.15.77:0/965410894 wait complete. 2024-09-15T22:31:55.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 -- start start 2024-09-15T22:31:55.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 Processor -- start 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0829450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a08243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ae260 con 0x5557a0815cb0 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38441adb90 con 0x5557a08165d0 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38441aadf0 con 0x5557a0820e90 2024-09-15T22:31:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f384ba67740 10 client.0 _ll_register_callbacks cb 0x7f383800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f38427fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0829450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49252/0 (socket says 172.21.15.77:49252) 2024-09-15T22:31:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f3842ffd640 1 -- 172.21.15.77:0/2362778397 learned_addr learned my addr 172.21.15.77:0/2362778397 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.103+0000 7f38437fe640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a08243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 -- 172.21.15.77:0/2362778397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 msgr2=0x5557a08243e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a08243e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 -- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 msgr2=0x5557a0829450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0829450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 -- 172.21.15.77:0/2362778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a0816400 con 0x5557a0815cb0 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f38427fc640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0829450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:55.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3842ffd640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f38340127e0 tx=0x7f3834012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a4a2083484ae0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:55.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:31:55.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834004230 con 0x5557a0815cb0 2024-09-15T22:31:55.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38380a4fb0 con 0x5557a0815cb0 2024-09-15T22:31:55.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f38340043d0 con 0x5557a0815cb0 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.104+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38340208e0 con 0x5557a0815cb0 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 10 client.20073 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5557a089b2f0 con 0x5557a0815cb0 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38380a4fb0 con 0x5557a0815cb0 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3834010430 con 0x5557a0815cb0 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 10 client.20073 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:55.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 10 client.20073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 20 client.20073 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a089ba10 0x5557a089de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 4 client.20073 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557a077ad60 con 0x5557a089ba10 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.105+0000 7f38427fc640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a089ba10 0x5557a089de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:55.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.106+0000 7f3826ff5640 10 client.20073 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:55.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.106+0000 7f38427fc640 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a089ba10 0x5557a089de30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0dc848f57963087 server_cookie=9b399e65dab86640 in_seq=0 out_seq=0 2024-09-15T22:31:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.107+0000 7f3826ff5640 1 -- 172.21.15.77:0/2362778397 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557a077ad60 con 0x5557a089ba10 2024-09-15T22:31:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.107+0000 7f3826ff5640 10 client.20073 handle_command_reply: tid=0 2024-09-15T22:31:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 client.20073 shutdown 2024-09-15T22:31:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 msgr2=0x5557a0828f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f38340127e0 tx=0x7f3834012ca0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 shutdown_connections 2024-09-15T22:31:55.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5557a089ba10 0x5557a089de30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.116 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "cbffa5ee-dd6b-455e-be51-6f8d40e8030c": { 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "cbffa5ee-dd6b-455e-be51-6f8d40e8030c", 2024-09-15T22:31:55.117 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:31:55.118 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:31:55.118 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:31:55.118 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:31:55.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5557a0820e90 0x5557a08243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557a08165d0 0x5557a0829450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 --2- 172.21.15.77:0/2362778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557a0815cb0 0x5557a0828f10 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f3826ff5640 0 client.20073 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 >> 172.21.15.77:0/2362778397 conn(0x5557a0812240 msgr2=0x5557a081fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 shutdown_connections 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 1 -- 172.21.15.77:0/2362778397 wait complete. 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.108+0000 7f384ba67740 20 client.20073 trim_cache size 0 max 16384 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 msgr2=0x7f382c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 0x7f382c0a5870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f383c003fd0 tx=0x7f383c048000 comp rx=0 tx=0).stop 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 msgr2=0x7f3844196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f383800e9f0 tx=0x7f383800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 shutdown_connections 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f382c0a33b0 0x7f382c0a5870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38441055b0 0x7f384419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100e30 0x7f3844196740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 --2- 172.21.15.77:0/829407818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3844100480 0x7f3844196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.109+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 >> 172.21.15.77:0/829407818 conn(0x7f3844077210 msgr2=0x7f3844103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.110+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 shutdown_connections 2024-09-15T22:31:55.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:55.110+0000 7f384968e640 1 -- 172.21.15.77:0/829407818 wait complete. 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:31:55.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:55 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.348 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:55.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:55.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:31:55.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:55 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:55.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:31:55.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:55 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:31:55.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 117994 16.95 MB/sec execute 236 sec latency 151.791 ms 2024-09-15T22:31:55.946 INFO:tasks.cephfs.filesystem:scrub status for tag:cbffa5ee-dd6b-455e-be51-6f8d40e8030c - {'path': '/', 'tag': 'cbffa5ee-dd6b-455e-be51-6f8d40e8030c', 'options': 'recursive,force'} 2024-09-15T22:31:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:56 smithi077 ceph-mon[30980]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 309 op/s 2024-09-15T22:31:56.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:56 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:56 smithi196 ceph-mon[35109]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 309 op/s 2024-09-15T22:31:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:56 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:56 smithi117 ceph-mon[35021]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 309 op/s 2024-09-15T22:31:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:56 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:56.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 118459 16.96 MB/sec execute 237 sec latency 164.079 ms 2024-09-15T22:31:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:57.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:31:57.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 -- 172.21.15.77:0/1622388107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910102d50 msgr2=0x7f9910103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:57.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 --2- 172.21.15.77:0/1622388107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910102d50 0x7f9910103130 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f990c009900 tx=0x7f990c02a480 comp rx=0 tx=0).stop 2024-09-15T22:31:57.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 -- 172.21.15.77:0/1622388107 shutdown_connections 2024-09-15T22:31:57.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 --2- 172.21.15.77:0/1622388107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f991010c180 0x7f991010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 --2- 172.21.15.77:0/1622388107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9910103670 0x7f991010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 --2- 172.21.15.77:0/1622388107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910102d50 0x7f9910103130 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 -- 172.21.15.77:0/1622388107 >> 172.21.15.77:0/1622388107 conn(0x7f99100fc7d0 msgr2=0x7f99100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 -- 172.21.15.77:0/1622388107 shutdown_connections 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.208+0000 7f9917286640 1 -- 172.21.15.77:0/1622388107 wait complete. 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 Processor -- start 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 -- start start 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9910102d50 0x7f9910198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 0x7f99101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9910111f80 con 0x7f991010c180 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9910111e00 con 0x7f9910103670 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9917286640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9910112100 con 0x7f9910102d50 2024-09-15T22:31:57.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9915823640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9915823640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49270/0 (socket says 172.21.15.77:49270) 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9915823640 1 -- 172.21.15.77:0/964384287 learned_addr learned my addr 172.21.15.77:0/964384287 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9914821640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 0x7f99101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.209+0000 7f9915022640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9910102d50 0x7f9910198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 -- 172.21.15.77:0/964384287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9910102d50 msgr2=0x7f9910198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9910102d50 0x7f9910198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 -- 172.21.15.77:0/964384287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 msgr2=0x7f99101989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 0x7f99101989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 -- 172.21.15.77:0/964384287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991019d410 con 0x7f991010c180 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9914821640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 0x7f99101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9915823640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f990800eb10 tx=0x7f990800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908004b20 con 0x7f991010c180 2024-09-15T22:31:57.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908004cc0 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908010760 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991019d700 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.210+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101008e0 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.212+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991006b070 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.213+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9908005020 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.213+0000 7f99067fc640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.214+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f99080d0600 con 0x7f991010c180 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.215+0000 7f9915022640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:57.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.215+0000 7f9915022640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f990c009230 tx=0x7f990c03c040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:57.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.219+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f990809cbf0 con 0x7f991010c180 2024-09-15T22:31:57.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:57.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:57.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:31:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:57.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:31:57.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.523+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99101071f0 con 0x7f991010c180 2024-09-15T22:31:57.525 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.523+0000 7f99067fc640 1 -- 172.21.15.77:0/964384287 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f990809c4f0 con 0x7f991010c180 2024-09-15T22:31:57.525 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:31:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:31:57.525 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 msgr2=0x7f98fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 0x7f98fc0a5870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f990c009230 tx=0x7f990c03c040 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 msgr2=0x7f991019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f990800eb10 tx=0x7f990800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 shutdown_connections 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f98fc0a33b0 0x7f98fc0a5870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f991010c180 0x7f991019cd30 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910103670 0x7f99101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 --2- 172.21.15.77:0/964384287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9910102d50 0x7f9910198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 >> 172.21.15.77:0/964384287 conn(0x7f99100fc7d0 msgr2=0x7f99100fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:57.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 shutdown_connections 2024-09-15T22:31:57.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:57.526+0000 7f9917286640 1 -- 172.21.15.77:0/964384287 wait complete. 2024-09-15T22:31:57.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119004 17.04 MB/sec execute 238 sec latency 157.919 ms 2024-09-15T22:31:58.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 148.15 --await-for 45.0' 2024-09-15T22:31:58.279 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 148.15 --await-for 45.0 2024-09-15T22:31:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:58 smithi077 ceph-mon[30980]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 494 op/s 2024-09-15T22:31:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:31:58 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/964384287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:58 smithi196 ceph-mon[35109]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 494 op/s 2024-09-15T22:31:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:31:58 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/964384287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:58.609 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119389 17.04 MB/sec execute 239 sec latency 180.286 ms 2024-09-15T22:31:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:58 smithi117 ceph-mon[35021]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 494 op/s 2024-09-15T22:31:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:31:58 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/964384287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:31:58.619 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 -- 172.21.15.77:0/835650643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c100e20 msgr2=0x7f475c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 --2- 172.21.15.77:0/835650643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c100e20 0x7f475c101200 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f474c009900 tx=0x7f474c02a480 comp rx=0 tx=0).stop 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 -- 172.21.15.77:0/835650643 shutdown_connections 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 --2- 172.21.15.77:0/835650643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102ef0 0x7f475c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 --2- 172.21.15.77:0/835650643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c102570 0x7f475c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 --2- 172.21.15.77:0/835650643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c100e20 0x7f475c101200 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 -- 172.21.15.77:0/835650643 >> 172.21.15.77:0/835650643 conn(0x7f475c0fc7d0 msgr2=0x7f475c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:59.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 -- 172.21.15.77:0/835650643 shutdown_connections 2024-09-15T22:31:59.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.172+0000 7f4762891640 1 -- 172.21.15.77:0/835650643 wait complete. 2024-09-15T22:31:59.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.173+0000 7f4762891640 1 Processor -- start 2024-09-15T22:31:59.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.173+0000 7f4762891640 1 -- start start 2024-09-15T22:31:59.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102570 0x7f475c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c102ef0 0x7f475c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f475c112020 con 0x7f475c102ef0 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111ea0 con 0x7f475c102570 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4762891640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1121a0 con 0x7f475c100e20 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:59014/0 (socket says 172.21.15.77:59014) 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 -- 172.21.15.77:0/1649325818 learned_addr learned my addr 172.21.15.77:0/1649325818 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475b7fe640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102570 0x7f475c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f4760e2e640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c102ef0 0x7f475c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 -- 172.21.15.77:0/1649325818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102570 msgr2=0x7f475c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102570 0x7f475c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 -- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c102ef0 msgr2=0x7f475c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c102ef0 0x7f475c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.174+0000 7f475bfff640 1 -- 172.21.15.77:0/1649325818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c19d430 con 0x7f475c100e20 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f475bfff640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f474c006010 tx=0x7f474c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f47597fa640 1 -- 172.21.15.77:0/1649325818 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c03f070 con 0x7f475c100e20 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c19d6c0 con 0x7f475c100e20 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f47597fa640 1 -- 172.21.15.77:0/1649325818 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c031c30 con 0x7f475c100e20 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f47597fa640 1 -- 172.21.15.77:0/1649325818 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c035030 con 0x7f475c100e20 2024-09-15T22:31:59.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.175+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c10e630 con 0x7f475c100e20 2024-09-15T22:31:59.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.177+0000 7f47597fa640 1 -- 172.21.15.77:0/1649325818 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f474c042070 con 0x7f475c100e20 2024-09-15T22:31:59.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.178+0000 7f47597fa640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 0x7f47440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.178+0000 7f4764c6a740 1 Processor -- start 2024-09-15T22:31:59.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.178+0000 7f475b7fe640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 0x7f47440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.178+0000 7f47597fa640 1 -- 172.21.15.77:0/1649325818 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f474c0f0710 con 0x7f475c100e20 2024-09-15T22:31:59.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 -- start start 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedaded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eceda30160 con 0x55ecedacc090 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eced9d7720 con 0x55ecedacb770 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4764c6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eceda2c120 con 0x55ecedadf240 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475b7fe640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 0x7f47440a5870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f475c0ffe20 tx=0x7f4750005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedaded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39130/0 (socket says 172.21.15.77:39130) 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 -- 172.21.15.77:0/3211401380 learned_addr learned my addr 172.21.15.77:0/3211401380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:59030/0 (socket says 172.21.15.77:59030) 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 -- 172.21.15.77:0/3211401380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 msgr2=0x55ecedae1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 -- 172.21.15.77:0/3211401380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 msgr2=0x55ecedaded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedaded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475bfff640 1 -- 172.21.15.77:0/3211401380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eced7a1ba0 con 0x55ecedacb770 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.179+0000 7f475b7fe640 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedaded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.180+0000 7f475bfff640 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b8b70 tx=0x7f474c0bc250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4d7ed5c2519c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.180+0000 7f47427fc640 1 -- 172.21.15.77:0/3211401380 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c049070 con 0x55ecedacb770 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.180+0000 7f47427fc640 1 -- 172.21.15.77:0/3211401380 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c0bc710 con 0x55ecedacb770 2024-09-15T22:31:59.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.180+0000 7f47427fc640 1 -- 172.21.15.77:0/3211401380 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0f0320 con 0x55ecedacb770 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- 172.21.15.77:0/3211401380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 msgr2=0x55ecedacbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b8b70 tx=0x7f474c0bc250 comp rx=0 tx=0).stop 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- 172.21.15.77:0/3211401380 shutdown_connections 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedaded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3211401380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedacbb50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- 172.21.15.77:0/3211401380 >> 172.21.15.77:0/3211401380 conn(0x55ecedac7de0 msgr2=0x55ecedac81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- 172.21.15.77:0/3211401380 shutdown_connections 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- 172.21.15.77:0/3211401380 wait complete. 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 -- start start 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.181+0000 7f4764c6a740 1 Processor -- start 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedadc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedadf240 0x55ecedadcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ecedae1b70 con 0x55ecedacc090 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eced9b8900 con 0x55ecedadf240 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ecedae1cf0 con 0x55ecedacb770 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedadf240 0x55ecedadcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:59042/0 (socket says 172.21.15.77:59042) 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 -- 172.21.15.77:0/259449944 learned_addr learned my addr 172.21.15.77:0/259449944 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475b7fe640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedadc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f4764c6a740 10 client.0 _ll_register_callbacks cb 0x55eced9b6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 -- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedadf240 msgr2=0x55ecedadcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedadf240 0x55ecedadcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 -- 172.21.15.77:0/259449944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 msgr2=0x55ecedadc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedadc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.182+0000 7f475bfff640 1 -- 172.21.15.77:0/259449944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eced9f8c50 con 0x55ecedacb770 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f475bfff640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f474c0c4da0 tx=0x7f474c03b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7803d940a984861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f473b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c049050 con 0x55ecedacb770 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f474c0f0320 con 0x55ecedacb770 2024-09-15T22:31:59.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c0bc710 con 0x55ecedacb770 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f474c0f0560 con 0x55ecedacb770 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f4764c6a740 10 client.28783 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ecedae1b70 con 0x55ecedacb770 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.183+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c0bc710 con 0x55ecedacb770 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f474c09c020 con 0x55ecedacb770 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 10 client.28783 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 10 client.28783 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 20 client.28783 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedaeea40 0x55ecedaf0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 4 client.28783 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eceda3d890 con 0x55ecedaeea40 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.184+0000 7f475b7fe640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedaeea40 0x55ecedaf0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.185+0000 7f475b7fe640 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedaeea40 0x55ecedaf0ea0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c83918b51843cca server_cookie=cfa9e39ff69b9bb in_seq=0 out_seq=0 2024-09-15T22:31:59.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.185+0000 7f473b7fe640 10 client.28783 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:59.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.187+0000 7f473b7fe640 1 -- 172.21.15.77:0/259449944 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eceda3d890 con 0x55ecedaeea40 2024-09-15T22:31:59.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.187+0000 7f473b7fe640 10 client.28783 handle_command_reply: tid=0 2024-09-15T22:31:59.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.187+0000 7f4764c6a740 1 client.28783 shutdown 2024-09-15T22:31:59.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 msgr2=0x55ecedadc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.192 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f474c0c4da0 tx=0x7f474c03b690 comp rx=0 tx=0).stop 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 shutdown_connections 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedaeea40 0x55ecedaf0ea0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedadf240 0x55ecedadcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedadc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 --2- 172.21.15.77:0/259449944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedacb770 0x55ecedadc1f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 >> 172.21.15.77:0/259449944 conn(0x55ecedac7de0 msgr2=0x55ecedacda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f473b7fe640 0 client.28783 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 shutdown_connections 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 1 -- 172.21.15.77:0/259449944 wait complete. 2024-09-15T22:31:59.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.188+0000 7f4764c6a740 20 client.28783 trim_cache size 0 max 16384 2024-09-15T22:31:59.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 Processor -- start 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 -- start start 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 0x55ecedaf67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedaf6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1acd00 con 0x55ecedacb770 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ae840 con 0x55ecedacc090 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4764c6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ad3d0 con 0x55ecedadf240 2024-09-15T22:31:59.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedaf6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f475bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f475bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40142/0 (socket says 172.21.15.77:40142) 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f475bfff640 1 -- 172.21.15.77:0/250647586 learned_addr learned my addr 172.21.15.77:0/250647586 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.208+0000 7f475b7fe640 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 0x55ecedaf67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 -- 172.21.15.77:0/250647586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 msgr2=0x55ecedaf6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedaf6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 -- 172.21.15.77:0/250647586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 msgr2=0x55ecedaf67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 0x55ecedaf67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 -- 172.21.15.77:0/250647586 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecedad7b60 con 0x55ecedacb770 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475b7fe640 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 0x55ecedaf67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f475bfff640 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f474c0bc410 tx=0x7f474c03b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eee4a1720cc3525 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f4741ffb640 1 -- 172.21.15.77:0/250647586 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0bc070 con 0x55ecedacb770 2024-09-15T22:31:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f4741ffb640 1 -- 172.21.15.77:0/250647586 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c0bc710 con 0x55ecedacb770 2024-09-15T22:31:59.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.209+0000 7f4741ffb640 1 -- 172.21.15.77:0/250647586 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0f0320 con 0x55ecedacb770 2024-09-15T22:31:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.210+0000 7f4764c6a740 1 -- 172.21.15.77:0/250647586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 msgr2=0x55ecedac2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:59.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.210+0000 7f4764c6a740 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f474c0bc410 tx=0x7f474c03b700 comp rx=0 tx=0).stop 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.210+0000 7f4764c6a740 1 -- 172.21.15.77:0/250647586 shutdown_connections 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedaf6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacc090 0x55ecedaf67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 --2- 172.21.15.77:0/250647586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacb770 0x55ecedac2d30 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 -- 172.21.15.77:0/250647586 >> 172.21.15.77:0/250647586 conn(0x55ecedac7de0 msgr2=0x55ecedacd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 -- 172.21.15.77:0/250647586 shutdown_connections 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 -- 172.21.15.77:0/250647586 wait complete. 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 -- start start 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.211+0000 7f4764c6a740 1 Processor -- start 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedae7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedae2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1ae170 con 0x55ecedacc090 2024-09-15T22:31:59.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1adaa0 con 0x55ecedacb770 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1aad00 con 0x55ecedadf240 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4764c6a740 10 client.0 _ll_register_callbacks cb 0x7f4754009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedae2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:59060/0 (socket says 172.21.15.77:59060) 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 -- 172.21.15.77:0/3576158909 learned_addr learned my addr 172.21.15.77:0/3576158909 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f475bfff640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedae7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 -- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 msgr2=0x55ecedae7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedae7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 -- 172.21.15.77:0/3576158909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 msgr2=0x55ecedae2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedae2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:59.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.212+0000 7f4760e2e640 1 -- 172.21.15.77:0/3576158909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecedb58780 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f4760e2e640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4754010c90 tx=0x7f475400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94e26c3ee703b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f473b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004d20 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c0bc710 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9438+0+0 (secure 0 0 0) 0x7f4754003970 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f475b7fe640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedae2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f475402a050 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f4764c6a740 10 client.28789 fetch_fsmap learned FSMap version 39 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.213+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ecedb599b0 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474c0bc710 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f475401c4b0 con 0x55ecedadf240 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 10 client.28789 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:31:59.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 10 client.28789 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 20 client.28789 populate_metadata read hostname 'smithi077' 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedb5a1c0 0x55ecedb5c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 4 client.28789 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.15, "await_for": 45.0}] 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.15, "await_for": 45.0}) -- 0x55ecedac3410 con 0x55ecedb5a1c0 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.214+0000 7f475bfff640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedb5a1c0 0x55ecedb5c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.215+0000 7f473b7fe640 10 client.28789 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:31:59.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:31:59.215+0000 7f475bfff640 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedb5a1c0 0x55ecedb5c5e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eab95ce6b43c35e server_cookie=f85f840fac342d88 in_seq=0 out_seq=0 2024-09-15T22:31:59.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119710 17.06 MB/sec execute 240 sec latency 334.452 ms 2024-09-15T22:32:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.239+0000 7f473b7fe640 1 -- 172.21.15.77:0/3576158909 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ecedac3410 con 0x55ecedb5a1c0 2024-09-15T22:32:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.239+0000 7f473b7fe640 10 client.28789 handle_command_reply: tid=0 2024-09-15T22:32:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.239+0000 7f4764c6a740 1 client.28789 shutdown 2024-09-15T22:32:00.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 msgr2=0x55ecedae3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4754010c90 tx=0x7f475400f790 comp rx=0 tx=0).stop 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 shutdown_connections 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ecedb5a1c0 0x55ecedb5c5e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f473b7fe640 0 client.28789 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ecedadf240 0x55ecedae3360 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ecedacc090 0x55ecedae2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 --2- 172.21.15.77:0/3576158909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecedacb770 0x55ecedae7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 >> 172.21.15.77:0/3576158909 conn(0x55ecedac7de0 msgr2=0x55ecedac66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 shutdown_connections 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 1 -- 172.21.15.77:0/3576158909 wait complete. 2024-09-15T22:32:00.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.240+0000 7f4764c6a740 20 client.28789 trim_cache size 0 max 16384 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 70, 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "f0a9f09": { 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 70, 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:00.244 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 148.2, 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:00.245 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:00.246 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:00.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 msgr2=0x7f47440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:00.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 0x7f47440a5870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f475c0ffe20 tx=0x7f4750005f90 comp rx=0 tx=0).stop 2024-09-15T22:32:00.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 msgr2=0x7f475c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f474c006010 tx=0x7f474c003c10 comp rx=0 tx=0).stop 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 shutdown_connections 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47440a33b0 0x7f47440a5870 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475c102ef0 0x7f475c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475c102570 0x7f475c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 --2- 172.21.15.77:0/1649325818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f475c100e20 0x7f475c198480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.241+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 >> 172.21.15.77:0/1649325818 conn(0x7f475c0fc7d0 msgr2=0x7f475c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.242+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 shutdown_connections 2024-09-15T22:32:00.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:00.242+0000 7f4762891640 1 -- 172.21.15.77:0/1649325818 wait complete. 2024-09-15T22:32:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:00 smithi077 ceph-mon[30980]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 343 op/s 2024-09-15T22:32:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:00 smithi196 ceph-mon[35109]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 343 op/s 2024-09-15T22:32:00.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.99 MB/sec execute 241 sec latency 1313.516 ms 2024-09-15T22:32:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:00 smithi117 ceph-mon[35021]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 343 op/s 2024-09-15T22:32:01.086 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f0a9f09', quiesce took 6.5 seconds. Will release after: 21.65 2024-09-15T22:32:01.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:01.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:01.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:01.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:01 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:01.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:01 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:01.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.92 MB/sec execute 242 sec latency 2313.684 ms 2024-09-15T22:32:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:01 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:02 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:02 smithi077 ceph-mon[30980]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 373 op/s 2024-09-15T22:32:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:02 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:02 smithi196 ceph-mon[35109]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 373 op/s 2024-09-15T22:32:02.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.85 MB/sec execute 243 sec latency 3313.748 ms 2024-09-15T22:32:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:02 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:02 smithi117 ceph-mon[35021]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 373 op/s 2024-09-15T22:32:03.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.78 MB/sec execute 244 sec latency 4313.800 ms 2024-09-15T22:32:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:04 smithi077 ceph-mon[30980]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T22:32:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:04 smithi196 ceph-mon[35109]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T22:32:04.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.71 MB/sec execute 245 sec latency 5313.845 ms 2024-09-15T22:32:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:04 smithi117 ceph-mon[35021]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T22:32:05.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.65 MB/sec execute 246 sec latency 6313.911 ms 2024-09-15T22:32:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:06 smithi077 ceph-mon[30980]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 MiB/s wr, 342 op/s 2024-09-15T22:32:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:06 smithi196 ceph-mon[35109]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 MiB/s wr, 342 op/s 2024-09-15T22:32:06.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.58 MB/sec execute 247 sec latency 7313.944 ms 2024-09-15T22:32:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:06 smithi117 ceph-mon[35021]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 MiB/s wr, 342 op/s 2024-09-15T22:32:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:07 smithi077 ceph-mon[30980]: osdmap e118: 12 total, 12 up, 12 in 2024-09-15T22:32:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:07 smithi196 ceph-mon[35109]: osdmap e118: 12 total, 12 up, 12 in 2024-09-15T22:32:07.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.51 MB/sec execute 248 sec latency 8314.035 ms 2024-09-15T22:32:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:07 smithi117 ceph-mon[35021]: osdmap e118: 12 total, 12 up, 12 in 2024-09-15T22:32:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:08 smithi077 ceph-mon[30980]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:08 smithi196 ceph-mon[35109]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:08.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.45 MB/sec execute 249 sec latency 9314.082 ms 2024-09-15T22:32:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:08 smithi117 ceph-mon[35021]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:09.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.38 MB/sec execute 250 sec latency 10314.126 ms 2024-09-15T22:32:10.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:10 smithi077 ceph-mon[30980]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:10 smithi196 ceph-mon[35109]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:10.610 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.31 MB/sec execute 251 sec latency 11314.220 ms 2024-09-15T22:32:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:10 smithi117 ceph-mon[35021]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 405 op/s 2024-09-15T22:32:11.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.25 MB/sec execute 252 sec latency 12314.262 ms 2024-09-15T22:32:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:12 smithi077 ceph-mon[30980]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 403 op/s 2024-09-15T22:32:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:12 smithi196 ceph-mon[35109]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 403 op/s 2024-09-15T22:32:12.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.19 MB/sec execute 253 sec latency 13314.383 ms 2024-09-15T22:32:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:12 smithi117 ceph-mon[35021]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 403 op/s 2024-09-15T22:32:13.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.12 MB/sec execute 254 sec latency 14314.440 ms 2024-09-15T22:32:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:14 smithi077 ceph-mon[30980]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.9 MiB/s wr, 326 op/s 2024-09-15T22:32:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:14 smithi077 ceph-mon[30980]: osdmap e119: 12 total, 12 up, 12 in 2024-09-15T22:32:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:14 smithi196 ceph-mon[35109]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.9 MiB/s wr, 326 op/s 2024-09-15T22:32:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:14 smithi196 ceph-mon[35109]: osdmap e119: 12 total, 12 up, 12 in 2024-09-15T22:32:14.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.06 MB/sec execute 255 sec latency 15314.569 ms 2024-09-15T22:32:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:14 smithi117 ceph-mon[35021]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.9 MiB/s wr, 326 op/s 2024-09-15T22:32:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:14 smithi117 ceph-mon[35021]: osdmap e119: 12 total, 12 up, 12 in 2024-09-15T22:32:15.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 16.00 MB/sec execute 256 sec latency 16314.662 ms 2024-09-15T22:32:16.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:16 smithi077 ceph-mon[30980]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 444 KiB/s wr, 128 op/s 2024-09-15T22:32:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:16 smithi196 ceph-mon[35109]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 444 KiB/s wr, 128 op/s 2024-09-15T22:32:16.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.93 MB/sec execute 257 sec latency 17314.773 ms 2024-09-15T22:32:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:16 smithi117 ceph-mon[35021]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 444 KiB/s wr, 128 op/s 2024-09-15T22:32:17.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.87 MB/sec execute 258 sec latency 18314.873 ms 2024-09-15T22:32:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:18 smithi077 ceph-mon[30980]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:18 smithi196 ceph-mon[35109]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:18.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.81 MB/sec execute 259 sec latency 19314.930 ms 2024-09-15T22:32:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:18 smithi117 ceph-mon[35021]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:19.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.75 MB/sec execute 260 sec latency 20315.034 ms 2024-09-15T22:32:20.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:20 smithi077 ceph-mon[30980]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:20 smithi196 ceph-mon[35109]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:20.611 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.69 MB/sec execute 261 sec latency 21315.151 ms 2024-09-15T22:32:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:20 smithi117 ceph-mon[35021]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 427 KiB/s wr, 118 op/s 2024-09-15T22:32:21.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.63 MB/sec execute 262 sec latency 22315.258 ms 2024-09-15T22:32:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:22 smithi077 ceph-mon[30980]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 355 KiB/s wr, 87 op/s 2024-09-15T22:32:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:22 smithi196 ceph-mon[35109]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 355 KiB/s wr, 87 op/s 2024-09-15T22:32:22.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.57 MB/sec execute 263 sec latency 23315.348 ms 2024-09-15T22:32:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:22 smithi117 ceph-mon[35021]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 355 KiB/s wr, 87 op/s 2024-09-15T22:32:22.738 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:32:22.996 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- 172.21.15.77:0/3062062042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 msgr2=0x7f8d441006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3062062042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d441006d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009930 tx=0x7f8d3802a420 comp rx=0 tx=0).stop 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- 172.21.15.77:0/3062062042 shutdown_connections 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3062062042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 0x7f8d4410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3062062042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 0x7f8d44106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3062062042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d441006d0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- 172.21.15.77:0/3062062042 >> 172.21.15.77:0/3062062042 conn(0x7f8d44077210 msgr2=0x7f8d440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- 172.21.15.77:0/3062062042 shutdown_connections 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- 172.21.15.77:0/3062062042 wait complete. 2024-09-15T22:32:23.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 Processor -- start 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.568+0000 7f8d4af97640 1 -- start start 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 0x7f8d44198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 0x7f8d4419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112080 con 0x7f8d441002f0 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44111f00 con 0x7f8d44106f50 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44112200 con 0x7f8d44100ca0 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60460/0 (socket says 172.21.15.77:60460) 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 -- 172.21.15.77:0/3301442827 learned_addr learned my addr 172.21.15.77:0/3301442827 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:23.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d43fff640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 0x7f8d44198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d49534640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 0x7f8d4419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 -- 172.21.15.77:0/3301442827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 msgr2=0x7f8d44198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 0x7f8d44198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 -- 172.21.15.77:0/3301442827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 msgr2=0x7f8d4419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 0x7f8d4419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 -- 172.21.15.77:0/3301442827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4419d5f0 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d48d33640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009a50 tx=0x7f8d38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3803f030 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d38003ce0 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4419d880 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.569+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38035070 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.570+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441a51c0 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.571+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d44100750 con 0x7f8d441002f0 2024-09-15T22:32:23.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.572+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8d38042030 con 0x7f8d441002f0 2024-09-15T22:32:23.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.572+0000 7f8d41ffb640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 0x7f8d340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:23.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.572+0000 7f8d43fff640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 0x7f8d340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:23.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.572+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f8d380f0720 con 0x7f8d441002f0 2024-09-15T22:32:23.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.573+0000 7f8d43fff640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 0x7f8d340a5870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ff8c0 tx=0x7f8d30008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:23.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.576+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d380bcd70 con 0x7f8d441002f0 2024-09-15T22:32:23.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.51 MB/sec execute 264 sec latency 24315.402 ms 2024-09-15T22:32:23.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.780+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d4410a120 con 0x7f8d441002f0 2024-09-15T22:32:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.781+0000 7f8d41ffb640 1 -- 172.21.15.77:0/3301442827 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f8d380bc670 con 0x7f8d441002f0 2024-09-15T22:32:23.783 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:32:23.783 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:32:23.784 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:32:23.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 msgr2=0x7f8d340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 0x7f8d340a5870 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ff8c0 tx=0x7f8d30008040 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 msgr2=0x7f8d44198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009a50 tx=0x7f8d38004270 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 shutdown_connections 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8d340a33b0 0x7f8d340a5870 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d44106f50 0x7f8d4419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d44100ca0 0x7f8d44198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 --2- 172.21.15.77:0/3301442827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d441002f0 0x7f8d44198640 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 >> 172.21.15.77:0/3301442827 conn(0x7f8d44077210 msgr2=0x7f8d44105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:23.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 shutdown_connections 2024-09-15T22:32:23.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:23.783+0000 7f8d4af97640 1 -- 172.21.15.77:0/3301442827 wait complete. 2024-09-15T22:32:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:24 smithi077 ceph-mon[30980]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-15T22:32:24.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3301442827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:24 smithi196 ceph-mon[35109]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-15T22:32:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3301442827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:24.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119718 15.45 MB/sec execute 265 sec latency 25315.478 ms 2024-09-15T22:32:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:24 smithi117 ceph-mon[35021]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-09-15T22:32:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3301442827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:24.620 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id f0a9f09 --release --await' 2024-09-15T22:32:24.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id f0a9f09 --release --await 2024-09-15T22:32:24.870 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:25.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 -- 172.21.15.77:0/1473033495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 msgr2=0x7f369c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 --2- 172.21.15.77:0/1473033495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c107b80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3690009930 tx=0x7f369002a420 comp rx=0 tx=0).stop 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 -- 172.21.15.77:0/1473033495 shutdown_connections 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 --2- 172.21.15.77:0/1473033495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 0x7f369c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 --2- 172.21.15.77:0/1473033495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c107b80 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 --2- 172.21.15.77:0/1473033495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 -- 172.21.15.77:0/1473033495 >> 172.21.15.77:0/1473033495 conn(0x7f369c0fc7d0 msgr2=0x7f369c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 -- 172.21.15.77:0/1473033495 shutdown_connections 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.379+0000 7f36a474f640 1 -- 172.21.15.77:0/1473033495 wait complete. 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 Processor -- start 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 -- start start 2024-09-15T22:32:25.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 0x7f369c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f369c112090 con 0x7f369c103740 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f369c111f10 con 0x7f369c108290 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a474f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f369c112210 con 0x7f369c102d90 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60474/0 (socket says 172.21.15.77:60474) 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 -- 172.21.15.77:0/128674935 learned_addr learned my addr 172.21.15.77:0/128674935 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a2cec640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 0x7f369c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a24eb640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 -- 172.21.15.77:0/128674935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 msgr2=0x7f369c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 -- 172.21.15.77:0/128674935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 msgr2=0x7f369c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 0x7f369c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a1cea640 1 -- 172.21.15.77:0/128674935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c19d560 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.380+0000 7f36a24eb640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f36a1cea640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3690009230 tx=0x7f36900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f369b7fe640 1 -- 172.21.15.77:0/128674935 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369003f030 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f369b7fe640 1 -- 172.21.15.77:0/128674935 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690003ce0 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c19d7f0 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f369b7fe640 1 -- 172.21.15.77:0/128674935 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690035070 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.381+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1a5130 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.383+0000 7f369b7fe640 1 -- 172.21.15.77:0/128674935 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3690042030 con 0x7f369c103740 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.383+0000 7f36a6b28740 1 Processor -- start 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 -- start start 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f2793110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2793650 0x55c4f2795a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4f26e9620 con 0x55c4f2783f40 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4f2690e50 con 0x55c4f2784860 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a6b28740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c4f26e55e0 con 0x55c4f2793650 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a2cec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a2cec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60490/0 (socket says 172.21.15.77:60490) 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a2cec640 1 -- 172.21.15.77:0/1958480893 learned_addr learned my addr 172.21.15.77:0/1958480893 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a24eb640 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f2793110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f369b7fe640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 0x7f36840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a1cea640 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2793650 0x55c4f2795a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f36a2cec640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 0x7f36840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.384+0000 7f369b7fe640 1 -- 172.21.15.77:0/128674935 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f36900f02c0 con 0x7f369c103740 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 -- 172.21.15.77:0/1958480893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2793650 msgr2=0x55c4f2795a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2793650 0x55c4f2795a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 -- 172.21.15.77:0/1958480893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 msgr2=0x55c4f2793110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f2793110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 -- 172.21.15.77:0/1958480893 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f2681d40 con 0x55c4f2783f40 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f368c013940 tx=0x7f368c013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df291ee28d1b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f3698ff9640 1 -- 172.21.15.77:0/1958480893 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c021030 con 0x55c4f2783f40 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f3698ff9640 1 -- 172.21.15.77:0/1958480893 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f368c0043a0 con 0x55c4f2783f40 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f3698ff9640 1 -- 172.21.15.77:0/1958480893 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c01da70 con 0x55c4f2783f40 2024-09-15T22:32:25.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.385+0000 7f36a2cec640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 0x7f36840a5870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f368c021000 tx=0x7f368c020000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- 172.21.15.77:0/1958480893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 msgr2=0x55c4f2784320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f368c013940 tx=0x7f368c013e00 comp rx=0 tx=0).stop 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- 172.21.15.77:0/1958480893 shutdown_connections 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2793650 0x55c4f2795a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f2793110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1958480893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f2784320 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- 172.21.15.77:0/1958480893 >> 172.21.15.77:0/1958480893 conn(0x55c4f27805f0 msgr2=0x55c4f27809d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- 172.21.15.77:0/1958480893 shutdown_connections 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- 172.21.15.77:0/1958480893 wait complete. 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 -- start start 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.386+0000 7f36a6b28740 1 Processor -- start 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f27905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f2791020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4f2796220 con 0x55c4f2784860 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4f2671f10 con 0x55c4f2793650 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c4f27963a0 con 0x55c4f2783f40 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a6b28740 10 client.0 _ll_register_callbacks cb 0x55c4f2695db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60492/0 (socket says 172.21.15.77:60492) 2024-09-15T22:32:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 -- 172.21.15.77:0/1410619205 learned_addr learned my addr 172.21.15.77:0/1410619205 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a24eb640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f27905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a2cec640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f2791020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 -- 172.21.15.77:0/1410619205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 msgr2=0x55c4f27905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f27905a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 -- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 msgr2=0x55c4f2791020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f2791020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 -- 172.21.15.77:0/1410619205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f23f8580 con 0x55c4f2784860 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a2cec640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f2791020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f36a1cea640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f36900b8670 tx=0x7f36900bc0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5309b3fc77e99843 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.387+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690049040 con 0x55c4f2784860 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f36900b8880 con 0x55c4f2784860 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f368c0043a0 con 0x55c4f2784860 2024-09-15T22:32:25.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f36900d5050 con 0x55c4f2784860 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 10 client.20115 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4f24c2960 con 0x55c4f2784860 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f368c0043a0 con 0x55c4f2784860 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f369004c060 con 0x55c4f2784860 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 10 client.20115 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 10 client.20115 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 20 client.20115 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f27a3200 0x55c4f27a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 4 client.20115 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4f26f6b90 con 0x55c4f27a3200 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.388+0000 7f36a24eb640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f27a3200 0x55c4f27a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.389+0000 7f36a24eb640 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f27a3200 0x55c4f27a5660 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2650561c2f49e2b1 server_cookie=6cbff33c28f5bb23 in_seq=0 out_seq=0 2024-09-15T22:32:25.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.389+0000 7f3681ffb640 10 client.20115 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.389+0000 7f3681ffb640 1 -- 172.21.15.77:0/1410619205 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4f26f6b90 con 0x55c4f27a3200 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.389+0000 7f3681ffb640 10 client.20115 handle_command_reply: tid=0 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 client.20115 shutdown 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 msgr2=0x55c4f2790ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f36900b8670 tx=0x7f36900bc0a0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 shutdown_connections 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f27a3200 0x55c4f27a5660 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f2791020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2784860 0x55c4f2790ae0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 --2- 172.21.15.77:0/1410619205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f27905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f3681ffb640 0 client.20115 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 >> 172.21.15.77:0/1410619205 conn(0x55c4f27805f0 msgr2=0x55c4f2786350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 shutdown_connections 2024-09-15T22:32:25.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 1 -- 172.21.15.77:0/1410619205 wait complete. 2024-09-15T22:32:25.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.390+0000 7f36a6b28740 20 client.20115 trim_cache size 0 max 16384 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 Processor -- start 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 -- start start 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 0x55c4f27ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f278d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1acda0 con 0x55c4f2783f40 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1ae8e0 con 0x55c4f2793650 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a6b28740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1ad470 con 0x55c4f2784860 2024-09-15T22:32:25.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60508/0 (socket says 172.21.15.77:60508) 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 -- 172.21.15.77:0/713981201 learned_addr learned my addr 172.21.15.77:0/713981201 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a1cea640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 0x55c4f27ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a2cec640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f278d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 -- 172.21.15.77:0/713981201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 msgr2=0x55c4f27ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 0x55c4f27ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 -- 172.21.15.77:0/713981201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 msgr2=0x55c4f278d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f278d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 -- 172.21.15.77:0/713981201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f2780090 con 0x55c4f2783f40 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a1cea640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 0x55c4f27ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a2cec640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f278d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:25.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f36a24eb640 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f36940058c0 tx=0x7f3694010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912998ba3f2ff5db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.407+0000 7f3683fff640 1 -- 172.21.15.77:0/713981201 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694004220 con 0x55c4f2783f40 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f3683fff640 1 -- 172.21.15.77:0/713981201 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36940043c0 con 0x55c4f2783f40 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f3683fff640 1 -- 172.21.15.77:0/713981201 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369401c910 con 0x55c4f2783f40 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 -- 172.21.15.77:0/713981201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 msgr2=0x55c4f27aaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f36940058c0 tx=0x7f3694010800 comp rx=0 tx=0).stop 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 -- 172.21.15.77:0/713981201 shutdown_connections 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2793650 0x55c4f278d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2784860 0x55c4f27ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 --2- 172.21.15.77:0/713981201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2783f40 0x55c4f27aaf30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 -- 172.21.15.77:0/713981201 >> 172.21.15.77:0/713981201 conn(0x55c4f27805f0 msgr2=0x55c4f2792620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 -- 172.21.15.77:0/713981201 shutdown_connections 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.408+0000 7f36a6b28740 1 -- 172.21.15.77:0/713981201 wait complete. 2024-09-15T22:32:25.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 -- start start 2024-09-15T22:32:25.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 Processor -- start 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f2797b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f27980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1ae210 con 0x55c4f2793650 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1adb40 con 0x55c4f2784860 2024-09-15T22:32:25.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1aada0 con 0x55c4f2783f40 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a6b28740 10 client.0 _ll_register_callbacks cb 0x7f3690009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60518/0 (socket says 172.21.15.77:60518) 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 -- 172.21.15.77:0/2482460633 learned_addr learned my addr 172.21.15.77:0/2482460633 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a24eb640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f2797b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a1cea640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f27980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 -- 172.21.15.77:0/2482460633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 msgr2=0x55c4f2797b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f2797b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 -- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 msgr2=0x55c4f27980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f27980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a2cec640 1 -- 172.21.15.77:0/2482460633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f280d4c0 con 0x55c4f2793650 2024-09-15T22:32:25.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a24eb640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f2797b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.409+0000 7f36a1cea640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f27980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a2cec640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f368c0136c0 tx=0x7f368c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58c198c8b9605b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c033020 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f368c01d810 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36940043c0 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f368c03e050 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 10 client.20121 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4f280e730 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36940043c0 con 0x55c4f2793650 2024-09-15T22:32:25.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f368c02e070 con 0x55c4f2793650 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 10 client.20121 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 10 client.20121 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 20 client.20121 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f280ebc0 0x55c4f2810fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 4 client.20121 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f0a9f09", "release": true, "await": true}] 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.410+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f0a9f09", "release": true, "await": true}) -- 0x55c4f277b950 con 0x55c4f280ebc0 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.411+0000 7f36a24eb640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f280ebc0 0x55c4f2810fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.411+0000 7f36a24eb640 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f280ebc0 0x55c4f2810fb0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7503dda74d8c8931 server_cookie=c9fcb1abb1f97dc0 in_seq=0 out_seq=0 2024-09-15T22:32:25.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.411+0000 7f3681ffb640 10 client.20121 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:25.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f3681ffb640 1 -- 172.21.15.77:0/2482460633 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c4f277b950 con 0x55c4f280ebc0 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f3681ffb640 10 client.20121 handle_command_reply: tid=0 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 client.20121 shutdown 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 msgr2=0x55c4f280cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f368c0136c0 tx=0x7f368c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 shutdown_connections 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c4f280ebc0 0x55c4f2810fb0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4f2793650 0x55c4f280cf00 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4f2784860 0x55c4f27980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 --2- 172.21.15.77:0/2482460633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c4f2783f40 0x55c4f2797b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 >> 172.21.15.77:0/2482460633 conn(0x55c4f27805f0 msgr2=0x55c4f277ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.488+0000 7f3681ffb640 0 client.20121 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:25.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 shutdown_connections 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a6b28740 1 -- 172.21.15.77:0/2482460633 wait complete. 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a6b28740 20 client.20121 trim_cache size 0 max 16384 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 72, 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "f0a9f09": { 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 72, 2024-09-15T22:32:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 148.2, 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 msgr2=0x7f36840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 0x7f36840a5870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f368c021000 tx=0x7f368c020000 comp rx=0 tx=0).stop 2024-09-15T22:32:25.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 msgr2=0x7f369c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3690009230 tx=0x7f36900042f0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 shutdown_connections 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f36840a33b0 0x7f36840a5870 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f369c108290 0x7f369c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103740 0x7f369c198af0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 --2- 172.21.15.77:0/128674935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f369c102d90 0x7f369c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.489+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 >> 172.21.15.77:0/128674935 conn(0x7f369c0fc7d0 msgr2=0x7f369c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.490+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 shutdown_connections 2024-09-15T22:32:25.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:25.490+0000 7f36a474f640 1 -- 172.21.15.77:0/128674935 wait complete. 2024-09-15T22:32:25.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 119749 15.40 MB/sec execute 266 sec latency 26193.802 ms 2024-09-15T22:32:25.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:32:26.303 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:26 smithi077 ceph-mon[30980]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-15T22:32:26.353 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f0a9f09', total seconds elapsed: 31.7 2024-09-15T22:32:26.353 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.15 2024-09-15T22:32:26.353 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:32:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:26.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:26 smithi196 ceph-mon[35109]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-15T22:32:26.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 120260 15.45 MB/sec execute 267 sec latency 127.816 ms 2024-09-15T22:32:26.614 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:26 smithi117 ceph-mon[35021]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-15T22:32:27.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 -- 172.21.15.77:0/1199686549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8109830 msgr2=0x7f41d8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 --2- 172.21.15.77:0/1199686549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8109830 0x7f41d8109c90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009860 tx=0x7f41c802a360 comp rx=0 tx=0).stop 2024-09-15T22:32:27.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 -- 172.21.15.77:0/1199686549 shutdown_connections 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 --2- 172.21.15.77:0/1199686549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d811b140 0x7f41d811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 --2- 172.21.15.77:0/1199686549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d8109830 0x7f41d8109c90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 --2- 172.21.15.77:0/1199686549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109450 0x7f41d8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.009+0000 7f41dff38640 1 -- 172.21.15.77:0/1199686549 >> 172.21.15.77:0/1199686549 conn(0x7f41d806e140 msgr2=0x7f41d806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.010+0000 7f41dff38640 1 -- 172.21.15.77:0/1199686549 shutdown_connections 2024-09-15T22:32:27.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.010+0000 7f41dff38640 1 -- 172.21.15.77:0/1199686549 wait complete. 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.010+0000 7f41dff38640 1 Processor -- start 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.011+0000 7f41dff38640 1 -- start start 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.011+0000 7f41dff38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 0x7f41d81a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.011+0000 7f41dff38640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109830 0x7f41d81a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.011+0000 7f41dff38640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 0x7f41d81abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.011+0000 7f41dff38640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8120df0 con 0x7f41d811b140 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.012+0000 7f41dff38640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8120c70 con 0x7f41d8109830 2024-09-15T22:32:27.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.012+0000 7f41dff38640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41d8120f70 con 0x7f41d8109450 2024-09-15T22:32:27.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.013+0000 7f41ddcd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 0x7f41d81a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.013+0000 7f41ddcd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 0x7f41d81a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42362/0 (socket says 172.21.15.77:42362) 2024-09-15T22:32:27.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.013+0000 7f41ddcd4640 1 -- 172.21.15.77:0/3938784150 learned_addr learned my addr 172.21.15.77:0/3938784150 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.014+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 0x7f41d81abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.014+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109830 0x7f41d81a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 -- 172.21.15.77:0/3938784150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 msgr2=0x7f41d81a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 0x7f41d81a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 -- 172.21.15.77:0/3938784150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109830 msgr2=0x7f41d81a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109830 0x7f41d81a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 -- 172.21.15.77:0/3938784150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81ac2a0 con 0x7f41d811b140 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 0x7f41d81abbc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ed50 tx=0x7f41d000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41ceffd640 1 -- 172.21.15.77:0/3938784150 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0004d20 con 0x7f41d811b140 2024-09-15T22:32:27.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41ceffd640 1 -- 172.21.15.77:0/3938784150 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d0005500 con 0x7f41d811b140 2024-09-15T22:32:27.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.015+0000 7f41ceffd640 1 -- 172.21.15.77:0/3938784150 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d00179b0 con 0x7f41d811b140 2024-09-15T22:32:27.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.017+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81ac590 con 0x7f41d811b140 2024-09-15T22:32:27.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.017+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81b3e70 con 0x7f41d811b140 2024-09-15T22:32:27.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.020+0000 7f41e2311740 1 Processor -- start 2024-09-15T22:32:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.023+0000 7f41ceffd640 1 -- 172.21.15.77:0/3938784150 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f41d001e020 con 0x7f41d811b140 2024-09-15T22:32:27.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.024+0000 7f41e2311740 1 -- start start 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9cebc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563d9ce222e0 con 0x563d9cec7680 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d9cdc9ce0 con 0x563d9cebcdc0 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41e2311740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d9ce1e2a0 con 0x563d9cebc4a0 2024-09-15T22:32:27.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41ceffd640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 0x7f41c40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.025+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 0x7f41c40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41ceffd640 1 -- 172.21.15.77:0/3938784150 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f41d00cff80 con 0x7f41d811b140 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41ddcd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41ddcd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39044/0 (socket says 172.21.15.77:39044) 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1538650009 learned_addr learned my addr 172.21.15.77:0/1538650009 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41de4d5640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 0x7f41c40a5a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41d009bf10 tx=0x7f41d00669f0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.026+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9cebc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1538650009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 msgr2=0x563d9cebc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9cebc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1538650009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 msgr2=0x563d9cec9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1538650009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9cdae440 con 0x563d9cebcdc0 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9cebc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.027+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009dc0 tx=0x7f41d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f4fc8e020ffdf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.031+0000 7f41c3fff640 1 -- 172.21.15.77:0/1538650009 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4005220 con 0x563d9cebcdc0 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.031+0000 7f41c3fff640 1 -- 172.21.15.77:0/1538650009 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d4004c80 con 0x563d9cebcdc0 2024-09-15T22:32:27.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.031+0000 7f41c3fff640 1 -- 172.21.15.77:0/1538650009 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4019970 con 0x563d9cebcdc0 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 -- 172.21.15.77:0/1538650009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 msgr2=0x563d9cec7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009dc0 tx=0x7f41d4011810 comp rx=0 tx=0).stop 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 -- 172.21.15.77:0/1538650009 shutdown_connections 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9cec7140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 --2- 172.21.15.77:0/1538650009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9cebc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.032+0000 7f41e2311740 1 -- 172.21.15.77:0/1538650009 >> 172.21.15.77:0/1538650009 conn(0x563d9ceb8a30 msgr2=0x563d9ceb8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.034+0000 7f41e2311740 1 -- 172.21.15.77:0/1538650009 shutdown_connections 2024-09-15T22:32:27.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.034+0000 7f41e2311740 1 -- 172.21.15.77:0/1538650009 wait complete. 2024-09-15T22:32:27.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.035+0000 7f41e2311740 1 -- start start 2024-09-15T22:32:27.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.035+0000 7f41e2311740 1 Processor -- start 2024-09-15T22:32:27.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cec42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563d9ceca250 con 0x563d9cec7680 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d9cddd050 con 0x563d9cebc4a0 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563d9ceca3d0 con 0x563d9cebcdc0 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41e2311740 10 client.0 _ll_register_callbacks cb 0x563d9ce1d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39060/0 (socket says 172.21.15.77:39060) 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41dd4d3640 1 -- 172.21.15.77:0/38244897 learned_addr learned my addr 172.21.15.77:0/38244897 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.036+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cec42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 -- 172.21.15.77:0/38244897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 msgr2=0x563d9cec42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cec42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 -- 172.21.15.77:0/38244897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 msgr2=0x563d9cec4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 -- 172.21.15.77:0/38244897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9cb933c0 con 0x563d9cebc4a0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c802a950 tx=0x7f41c802a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04dfe643fbc6d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.037+0000 7f41c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8046030 con 0x563d9cebc4a0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f41c8030960 con 0x563d9cebc4a0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41d4004c80 con 0x563d9cebc4a0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f41c8037040 con 0x563d9cebc4a0 2024-09-15T22:32:27.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41e2311740 10 client.28904 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:27.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.038+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d9ced5bb0 con 0x563d9cebc4a0 2024-09-15T22:32:27.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d4004c80 con 0x563d9cebc4a0 2024-09-15T22:32:27.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f41c803d040 con 0x563d9cebc4a0 2024-09-15T22:32:27.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 10 client.28904 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 10 client.28904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 20 client.28904 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9ced7070 0x563d9ced9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 4 client.28904 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.039+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9ce2f850 con 0x563d9ced7070 2024-09-15T22:32:27.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.040+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9ced7070 0x563d9ced9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.040+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9ced7070 0x563d9ced9460 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c6e0ec0e998920 server_cookie=4c6011e7bceff85b in_seq=0 out_seq=0 2024-09-15T22:32:27.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.040+0000 7f41c17fa640 10 client.28904 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:27.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.043+0000 7f41c17fa640 1 -- 172.21.15.77:0/38244897 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d9ce2f850 con 0x563d9ced7070 2024-09-15T22:32:27.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.044+0000 7f41c17fa640 10 client.28904 handle_command_reply: tid=0 2024-09-15T22:32:27.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.044+0000 7f41e2311740 1 client.28904 shutdown 2024-09-15T22:32:27.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 msgr2=0x563d9cec7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c802a950 tx=0x7f41c802a360 comp rx=0 tx=0).stop 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 shutdown_connections 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9ced7070 0x563d9ced9460 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cec4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cec42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 --2- 172.21.15.77:0/38244897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebc4a0 0x563d9cec7000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 >> 172.21.15.77:0/38244897 conn(0x563d9ceb8a30 msgr2=0x563d9cebe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41c17fa640 0 client.28904 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 shutdown_connections 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 1 -- 172.21.15.77:0/38244897 wait complete. 2024-09-15T22:32:27.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.047+0000 7f41e2311740 20 client.28904 trim_cache size 0 max 16384 2024-09-15T22:32:27.057 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:27.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 Processor -- start 2024-09-15T22:32:27.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 -- start start 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9ceded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cedf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81bdf10 con 0x563d9cec7680 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81bfa50 con 0x563d9cebcdc0 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41e2311740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81be5e0 con 0x563d9cebc4a0 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42404/0 (socket says 172.21.15.77:42404) 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41dd4d3640 1 -- 172.21.15.77:0/4122208602 learned_addr learned my addr 172.21.15.77:0/4122208602 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:27.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.076+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9ceded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.077+0000 7f41dd4d3640 1 -- 172.21.15.77:0/4122208602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 msgr2=0x563d9ceded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.077+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9ceded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.077+0000 7f41dd4d3640 1 -- 172.21.15.77:0/4122208602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 msgr2=0x563d9cedf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.077+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cedf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.077+0000 7f41dd4d3640 1 -- 172.21.15.77:0/4122208602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9cec3260 con 0x563d9cebc4a0 2024-09-15T22:32:27.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.078+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41c8005b10 tx=0x7f41c8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f96627775e6b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.078+0000 7f41c37fe640 1 -- 172.21.15.77:0/4122208602 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8046040 con 0x563d9cebc4a0 2024-09-15T22:32:27.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.078+0000 7f41c37fe640 1 -- 172.21.15.77:0/4122208602 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c80046c0 con 0x563d9cebc4a0 2024-09-15T22:32:27.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.078+0000 7f41c37fe640 1 -- 172.21.15.77:0/4122208602 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8005300 con 0x563d9cebc4a0 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.079+0000 7f41e2311740 1 -- 172.21.15.77:0/4122208602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 msgr2=0x563d9ceb8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.079+0000 7f41e2311740 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41c8005b10 tx=0x7f41c8005f90 comp rx=0 tx=0).stop 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.079+0000 7f41e2311740 1 -- 172.21.15.77:0/4122208602 shutdown_connections 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.080+0000 7f41e2311740 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cec7680 0x563d9cedf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.080+0000 7f41e2311740 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cebcdc0 0x563d9ceded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.080+0000 7f41e2311740 1 --2- 172.21.15.77:0/4122208602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebc4a0 0x563d9ceb8e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.080+0000 7f41e2311740 1 -- 172.21.15.77:0/4122208602 >> 172.21.15.77:0/4122208602 conn(0x563d9ceb8a30 msgr2=0x563d9ceb7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.081+0000 7f41e2311740 1 -- 172.21.15.77:0/4122208602 shutdown_connections 2024-09-15T22:32:27.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.082+0000 7f41e2311740 1 -- 172.21.15.77:0/4122208602 wait complete. 2024-09-15T22:32:27.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.082+0000 7f41e2311740 1 -- start start 2024-09-15T22:32:27.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.082+0000 7f41e2311740 1 Processor -- start 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 0x563d9cecd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cecd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cec7680 0x563d9cecde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81bf380 con 0x563d9cebc4a0 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81becb0 con 0x563d9cec7680 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 0x563d9cecd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41dd4d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 0x563d9cecd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60566/0 (socket says 172.21.15.77:60566) 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41dd4d3640 1 -- 172.21.15.77:0/1197627900 learned_addr learned my addr 172.21.15.77:0/1197627900 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:27.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.084+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81bbf10 con 0x563d9cebcdc0 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41e2311740 10 client.0 _ll_register_callbacks cb 0x7f41d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cecd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cec7680 msgr2=0x563d9cecde40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cec7680 0x563d9cecde40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1197627900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 msgr2=0x563d9cecd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 0x563d9cecd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.085+0000 7f41ddcd4640 1 -- 172.21.15.77:0/1197627900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9cf40d40 con 0x563d9cebcdc0 2024-09-15T22:32:27.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.086+0000 7f41ddcd4640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cecd900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f41d4019510 tx=0x7f41d40099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b594525d66e378 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:27.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.086+0000 7f41c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:32:27.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4030020 con 0x563d9cebcdc0 2024-09-15T22:32:27.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f41d40054d0 con 0x563d9cebcdc0 2024-09-15T22:32:27.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c80046c0 con 0x563d9cebcdc0 2024-09-15T22:32:27.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f41d40035f0 con 0x563d9cebcdc0 2024-09-15T22:32:27.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41e2311740 10 client.28822 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:27.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.087+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563d9cf41ff0 con 0x563d9cebcdc0 2024-09-15T22:32:27.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.091+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c80046c0 con 0x563d9cebcdc0 2024-09-15T22:32:27.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f41d4019910 con 0x563d9cebcdc0 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 10 client.28822 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 10 client.28822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 20 client.28822 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9cf42510 0x563d9cf44930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 4 client.28822 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:32:27.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.092+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d9ce29d60 con 0x563d9cf42510 2024-09-15T22:32:27.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.094+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9cf42510 0x563d9cf44930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:27.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.101+0000 7f41dd4d3640 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9cf42510 0x563d9cf44930 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f426d6c073a3a1 server_cookie=211836d8751107ae in_seq=0 out_seq=0 2024-09-15T22:32:27.104 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.101+0000 7f41c17fa640 10 client.28822 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41c17fa640 1 -- 172.21.15.77:0/1197627900 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d9ce29d60 con 0x563d9cf42510 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41c17fa640 10 client.28822 handle_command_reply: tid=0 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 client.28822 shutdown 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 msgr2=0x563d9cecd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cecd900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f41d4019510 tx=0x7f41d40099d0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 shutdown_connections 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x563d9cf42510 0x563d9cf44930 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41c17fa640 0 client.28822 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d9cec7680 0x563d9cecde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563d9cebcdc0 0x563d9cecd900 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f41d4019510 tx=0x7f41d40099d0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 --2- 172.21.15.77:0/1197627900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563d9cebc4a0 0x563d9cecd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.105+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 >> 172.21.15.77:0/1197627900 conn(0x563d9ceb8a30 msgr2=0x563d9cec5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.106+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 shutdown_connections 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.106+0000 7f41e2311740 1 -- 172.21.15.77:0/1197627900 wait complete. 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.106+0000 7f41e2311740 20 client.28822 trim_cache size 0 max 16384 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:27.109 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:32:27.110 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:32:27.110 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:27.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.107+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 msgr2=0x7f41c40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.107+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 0x7f41c40a5a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41d009bf10 tx=0x7f41d00669f0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.107+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 msgr2=0x7f41d81abbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 0x7f41d81abbc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f41d000ed50 tx=0x7f41d000c6e0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 shutdown_connections 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41c40a35c0 0x7f41c40a5a80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d811b140 0x7f41d81abbc0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d8109830 0x7f41d81a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 --2- 172.21.15.77:0/3938784150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41d8109450 0x7f41d81a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 >> 172.21.15.77:0/3938784150 conn(0x7f41d806e140 msgr2=0x7f41d811c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 shutdown_connections 2024-09-15T22:32:27.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:27.108+0000 7f41dff38640 1 -- 172.21.15.77:0/3938784150 wait complete. 2024-09-15T22:32:27.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 120692 15.48 MB/sec execute 268 sec latency 176.233 ms 2024-09-15T22:32:27.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:32:27.958 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:32:28.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:28 smithi077 ceph-mon[30980]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-09-15T22:32:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:28 smithi196 ceph-mon[35109]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-09-15T22:32:28.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 121055 15.48 MB/sec execute 269 sec latency 186.272 ms 2024-09-15T22:32:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:28 smithi117 ceph-mon[35021]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 188 KiB/s wr, 27 op/s 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 -- 172.21.15.77:0/3940752234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 msgr2=0x7fde5410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/3940752234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde5410ea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009be0 tx=0x7fde4c02a6c0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 -- 172.21.15.77:0/3940752234 shutdown_connections 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/3940752234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde5410ea20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/3940752234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54100100 0x7fde54100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/3940752234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54106b80 0x7fde54106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 -- 172.21.15.77:0/3940752234 >> 172.21.15.77:0/3940752234 conn(0x7fde54077230 msgr2=0x7fde540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 -- 172.21.15.77:0/3940752234 shutdown_connections 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.690+0000 7fde5b6a0640 1 -- 172.21.15.77:0/3940752234 wait complete. 2024-09-15T22:32:28.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 Processor -- start 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 -- start start 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 0x7fde5419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde54112000 con 0x7fde54100100 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fde54111e80 con 0x7fde54106b80 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5b6a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde54112180 con 0x7fde54100aa0 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.691+0000 7fde5943c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60570/0 (socket says 172.21.15.77:60570) 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 -- 172.21.15.77:0/848477473 learned_addr learned my addr 172.21.15.77:0/848477473 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde58c3b640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde59c3d640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 0x7fde5419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:28.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 -- 172.21.15.77:0/848477473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 msgr2=0x7fde541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 -- 172.21.15.77:0/848477473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 msgr2=0x7fde5419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 0x7fde5419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 -- 172.21.15.77:0/848477473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5419d430 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde59c3d640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 0x7fde5419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde5943c640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fde5000e9c0 tx=0x7fde5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.692+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000cde0 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.693+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde500035f0 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.693+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50010690 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.693+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5419d720 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.693+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5419db80 con 0x7fde54100100 2024-09-15T22:32:28.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.694+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde541997e0 con 0x7fde54100100 2024-09-15T22:32:28.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.696+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fde500037a0 con 0x7fde54100100 2024-09-15T22:32:28.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.697+0000 7fde4a7fc640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 0x7fde400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:28.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.697+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fde500d0460 con 0x7fde54100100 2024-09-15T22:32:28.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.697+0000 7fde58c3b640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 0x7fde400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:28.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.698+0000 7fde58c3b640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 0x7fde400a58f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde540ff7f0 tx=0x7fde44005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:28.701+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde5419db80 con 0x7fde54100100 2024-09-15T22:32:28.749 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:29.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.012+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde54101450 con 0x7fde54100100 2024-09-15T22:32:29.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.013+0000 7fde4a7fc640 1 -- 172.21.15.77:0/848477473 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fde5009cb40 con 0x7fde54100100 2024-09-15T22:32:29.016 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:32:29.016 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:32:29.017 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:32:29.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 msgr2=0x7fde400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:29.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 0x7fde400a58f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde540ff7f0 tx=0x7fde44005f90 comp rx=0 tx=0).stop 2024-09-15T22:32:29.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 msgr2=0x7fde54198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:29.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fde5000e9c0 tx=0x7fde5000ee80 comp rx=0 tx=0).stop 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 shutdown_connections 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fde400a3430 0x7fde400a58f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde54106b80 0x7fde5419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde54100aa0 0x7fde541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 --2- 172.21.15.77:0/848477473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde54100100 0x7fde54198480 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 >> 172.21.15.77:0/848477473 conn(0x7fde54077230 msgr2=0x7fde54104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 shutdown_connections 2024-09-15T22:32:29.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.017+0000 7fde5b6a0640 1 -- 172.21.15.77:0/848477473 wait complete. 2024-09-15T22:32:29.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:29 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/848477473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:29.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:29 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/848477473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:29.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:29 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/848477473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:29.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 121469 15.50 MB/sec execute 270 sec latency 197.547 ms 2024-09-15T22:32:29.746 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 148.15 --await-for 45.0' 2024-09-15T22:32:29.746 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 148.15 --await-for 45.0 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 -- 172.21.15.77:0/2642501364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c109b10 msgr2=0x7fb86c10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 --2- 172.21.15.77:0/2642501364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c109b10 0x7fb86c10e350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c02a420 comp rx=0 tx=0).stop 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 -- 172.21.15.77:0/2642501364 shutdown_connections 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 --2- 172.21.15.77:0/2642501364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c10e890 0x7fb86c10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 --2- 172.21.15.77:0/2642501364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c109b10 0x7fb86c10e350 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c02a420 comp rx=0 tx=0).stop 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 --2- 172.21.15.77:0/2642501364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1091f0 0x7fb86c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.981+0000 7fb871f25640 1 -- 172.21.15.77:0/2642501364 >> 172.21.15.77:0/2642501364 conn(0x7fb86c06e2d0 msgr2=0x7fb86c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.982+0000 7fb871f25640 1 -- 172.21.15.77:0/2642501364 shutdown_connections 2024-09-15T22:32:29.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.982+0000 7fb871f25640 1 -- 172.21.15.77:0/2642501364 wait complete. 2024-09-15T22:32:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.983+0000 7fb871f25640 1 Processor -- start 2024-09-15T22:32:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.984+0000 7fb871f25640 1 -- start start 2024-09-15T22:32:29.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.984+0000 7fb871f25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c1091f0 0x7fb86c1a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.984+0000 7fb871f25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 0x7fb86c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.985+0000 7fb871f25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:29.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.985+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.985+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46206/0 (socket says 172.21.15.77:46206) 2024-09-15T22:32:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.985+0000 7fb86bfff640 1 -- 172.21.15.77:0/2602653426 learned_addr learned my addr 172.21.15.77:0/2602653426 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.986+0000 7fb86b7fe640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c1091f0 0x7fb86c1a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:29.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.986+0000 7fb86affd640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 0x7fb86c1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.986+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c11cc20 con 0x7fb86c1a7a40 2024-09-15T22:32:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.986+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c112520 con 0x7fb86c1091f0 2024-09-15T22:32:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.986+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c11cda0 con 0x7fb86c10e890 2024-09-15T22:32:29.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.987+0000 7fb86bfff640 1 -- 172.21.15.77:0/2602653426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 msgr2=0x7fb86c1a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.987+0000 7fb86bfff640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 0x7fb86c1a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.987+0000 7fb86bfff640 1 -- 172.21.15.77:0/2602653426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c1091f0 msgr2=0x7fb86c1a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.987+0000 7fb86bfff640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c1091f0 0x7fb86c1a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:29.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.987+0000 7fb86bfff640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1a8500 con 0x7fb86c1a7a40 2024-09-15T22:32:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.988+0000 7fb86bfff640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:29.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.988+0000 7fb86affd640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 0x7fb86c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.992+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400cdb0 con 0x7fb86c1a7a40 2024-09-15T22:32:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.992+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb864005020 con 0x7fb86c1a7a40 2024-09-15T22:32:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.992+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1a87f0 con 0x7fb86c1a7a40 2024-09-15T22:32:29.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.992+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1aff70 con 0x7fb86c1a7a40 2024-09-15T22:32:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.993+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640116b0 con 0x7fb86c1a7a40 2024-09-15T22:32:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.994+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb858005290 con 0x7fb86c1a7a40 2024-09-15T22:32:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.997+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb864004a40 con 0x7fb86c1a7a40 2024-09-15T22:32:30.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.999+0000 7fb868ff9640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 0x7fb8540a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:30.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:29.999+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fb8640d05c0 con 0x7fb86c1a7a40 2024-09-15T22:32:30.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.001+0000 7fb86b7fe640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 0x7fb8540a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:30.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.002+0000 7fb86b7fe640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 0x7fb8540a5a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb86000a810 tx=0x7fb860008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.006+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb864016030 con 0x7fb86c1a7a40 2024-09-15T22:32:30.126 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:30.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.296+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb858005580 con 0x7fb86c1a7a40 2024-09-15T22:32:30.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.297+0000 7fb868ff9640 1 -- 172.21.15.77:0/2602653426 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fb86409ccb0 con 0x7fb86c1a7a40 2024-09-15T22:32:30.300 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:32:30.300 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:32:30.300 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:32:30.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 msgr2=0x7fb8540a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:30.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 0x7fb8540a5a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb86000a810 tx=0x7fb860008040 comp rx=0 tx=0).stop 2024-09-15T22:32:30.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 msgr2=0x7fb86c1a7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).stop 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 shutdown_connections 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb8540a35c0 0x7fb8540a5a80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb86c1a7a40 0x7fb86c1a7ef0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb86c10e890 0x7fb86c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 --2- 172.21.15.77:0/2602653426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c1091f0 0x7fb86c1a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 >> 172.21.15.77:0/2602653426 conn(0x7fb86c06e2d0 msgr2=0x7fb86c112750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 shutdown_connections 2024-09-15T22:32:30.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:30.299+0000 7fb871f25640 1 -- 172.21.15.77:0/2602653426 wait complete. 2024-09-15T22:32:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:30 smithi077 ceph-mon[30980]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 148 KiB/s wr, 20 op/s 2024-09-15T22:32:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:30 smithi196 ceph-mon[35109]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 148 KiB/s wr, 20 op/s 2024-09-15T22:32:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:30 smithi117 ceph-mon[35021]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 148 KiB/s wr, 20 op/s 2024-09-15T22:32:30.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 121910 15.54 MB/sec execute 271 sec latency 160.867 ms 2024-09-15T22:32:31.128 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:32:31.373 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:31 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2602653426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:31.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.406+0000 7fe804044640 1 -- 172.21.15.77:0/3033255109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc105ec0 msgr2=0x7fe7fc106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.406+0000 7fe804044640 1 --2- 172.21.15.77:0/3033255109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc106340 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009ca0 tx=0x7fe7f402a8b0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.407+0000 7fe804044640 1 -- 172.21.15.77:0/3033255109 shutdown_connections 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.407+0000 7fe804044640 1 --2- 172.21.15.77:0/3033255109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc106340 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.407+0000 7fe804044640 1 --2- 172.21.15.77:0/3033255109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc1011e0 0x7fe7fc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.407+0000 7fe804044640 1 --2- 172.21.15.77:0/3033255109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.407+0000 7fe804044640 1 -- 172.21.15.77:0/3033255109 >> 172.21.15.77:0/3033255109 conn(0x7fe7fc0fc7d0 msgr2=0x7fe7fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.408+0000 7fe804044640 1 -- 172.21.15.77:0/3033255109 shutdown_connections 2024-09-15T22:32:31.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.408+0000 7fe804044640 1 -- 172.21.15.77:0/3033255109 wait complete. 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.409+0000 7fe804044640 1 Processor -- start 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.409+0000 7fe804044640 1 -- start start 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.409+0000 7fe804044640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.409+0000 7fe804044640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1011e0 0x7fe7fc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe804044640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe804044640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1186e0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe804044640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc118560 con 0x7fe7fc105ec0 2024-09-15T22:32:31.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe804044640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc118860 con 0x7fe7fc1011e0 2024-09-15T22:32:31.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46236/0 (socket says 172.21.15.77:46236) 2024-09-15T22:32:31.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 -- 172.21.15.77:0/126503038 learned_addr learned my addr 172.21.15.77:0/126503038 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe8015df640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1011e0 0x7fe7fc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe8025e1640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 -- 172.21.15.77:0/126503038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1011e0 msgr2=0x7fe7fc1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1011e0 0x7fe7fc1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 -- 172.21.15.77:0/126503038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc105ec0 msgr2=0x7fe7fc19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.410+0000 7fe801de0640 1 -- 172.21.15.77:0/126503038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc19efd0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.411+0000 7fe801de0640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c890 tx=0x7fe7f800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.411+0000 7fe7f2ffd640 1 -- 172.21.15.77:0/126503038 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004220 con 0x7fe7fc1008c0 2024-09-15T22:32:31.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.411+0000 7fe7f2ffd640 1 -- 172.21.15.77:0/126503038 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f80043c0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.412+0000 7fe7f2ffd640 1 -- 172.21.15.77:0/126503038 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004a00 con 0x7fe7fc1008c0 2024-09-15T22:32:31.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.412+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc19f2c0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.412+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1ab510 con 0x7fe7fc1008c0 2024-09-15T22:32:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.415+0000 7fe80641d740 1 Processor -- start 2024-09-15T22:32:31.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 -- start start 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 0x560b55e41c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 0x560b55e4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e4ca30 0x560b55e4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b55da6160 con 0x560b55e4ca30 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b55d4d720 con 0x560b55e41850 2024-09-15T22:32:31.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.416+0000 7fe80641d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b55da2120 con 0x560b55e42170 2024-09-15T22:32:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.419+0000 7fe801de0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 0x560b55e4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.419+0000 7fe801de0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 0x560b55e4c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57708/0 (socket says 172.21.15.77:57708) 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.419+0000 7fe801de0640 1 -- 172.21.15.77:0/2986314853 learned_addr learned my addr 172.21.15.77:0/2986314853 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.419+0000 7fe7f2ffd640 1 -- 172.21.15.77:0/126503038 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe7f80079b0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.419+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 0x560b55e41c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.420+0000 7fe8025e1640 1 -- 172.21.15.77:0/2986314853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 msgr2=0x560b55e4c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.420+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 0x560b55e4c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.420+0000 7fe8025e1640 1 -- 172.21.15.77:0/2986314853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e4ca30 msgr2=0x560b55e4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.420+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e4ca30 0x560b55e4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.420+0000 7fe8025e1640 1 -- 172.21.15.77:0/2986314853 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55b17ba0 con 0x560b55e41850 2024-09-15T22:32:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.421+0000 7fe7f2ffd640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 0x7fe7e80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.421+0000 7fe7f2ffd640 1 -- 172.21.15.77:0/126503038 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fe7f80cfce0 con 0x7fe7fc1008c0 2024-09-15T22:32:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.421+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 0x560b55e41c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009bf0 tx=0x7fe7f4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3eb13454d077a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.421+0000 7fe801de0640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 0x7fe7e80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.422+0000 7fe7e7fff640 1 -- 172.21.15.77:0/2986314853 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40306a0 con 0x560b55e41850 2024-09-15T22:32:31.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.422+0000 7fe7e7fff640 1 -- 172.21.15.77:0/2986314853 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f4036070 con 0x560b55e41850 2024-09-15T22:32:31.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.422+0000 7fe7e7fff640 1 -- 172.21.15.77:0/2986314853 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40043c0 con 0x560b55e41850 2024-09-15T22:32:31.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.423+0000 7fe80641d740 1 -- 172.21.15.77:0/2986314853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 msgr2=0x560b55e41c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.423+0000 7fe80641d740 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 0x560b55e41c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009bf0 tx=0x7fe7f4009a20 comp rx=0 tx=0).stop 2024-09-15T22:32:31.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.424+0000 7fe801de0640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 0x7fe7e80a5a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f809b8d0 tx=0x7fe7f80d3750 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.425+0000 7fe80641d740 1 -- 172.21.15.77:0/2986314853 shutdown_connections 2024-09-15T22:32:31.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.425+0000 7fe80641d740 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e4ca30 0x560b55e4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.425+0000 7fe80641d740 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e42170 0x560b55e4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.425+0000 7fe80641d740 1 --2- 172.21.15.77:0/2986314853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e41850 0x560b55e41c30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.425+0000 7fe80641d740 1 -- 172.21.15.77:0/2986314853 >> 172.21.15.77:0/2986314853 conn(0x560b55e3dde0 msgr2=0x560b55e3e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.426+0000 7fe80641d740 1 -- 172.21.15.77:0/2986314853 shutdown_connections 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.426+0000 7fe80641d740 1 -- 172.21.15.77:0/2986314853 wait complete. 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.426+0000 7fe80641d740 1 -- start start 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 Processor -- start 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e49f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e4a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b55e4f600 con 0x560b55e41850 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b55d2e900 con 0x560b55e42170 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.427+0000 7fe80641d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b55e4f780 con 0x560b55e4ca30 2024-09-15T22:32:31.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.428+0000 7fe80641d740 10 client.0 _ll_register_callbacks cb 0x560b55d2c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.428+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e4a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.428+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e4a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57710/0 (socket says 172.21.15.77:57710) 2024-09-15T22:32:31.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.428+0000 7fe8025e1640 1 -- 172.21.15.77:0/169940695 learned_addr learned my addr 172.21.15.77:0/169940695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e49f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 -- 172.21.15.77:0/169940695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 msgr2=0x560b55e4a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e4a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 -- 172.21.15.77:0/169940695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 msgr2=0x560b55e499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.429+0000 7fe8015df640 1 -- 172.21.15.77:0/169940695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55d6ec50 con 0x560b55e42170 2024-09-15T22:32:31.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.430+0000 7fe8015df640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e49f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x560b55e3f160 tx=0x7fe7ec00e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f438a17f94aee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.430+0000 7fe7e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:32:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.430+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec017070 con 0x560b55e42170 2024-09-15T22:32:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.430+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f4036070 con 0x560b55e42170 2024-09-15T22:32:31.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.430+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fe7ec013680 con 0x560b55e42170 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec021050 con 0x560b55e42170 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe80641d740 10 client.28922 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b55e6c3c0 con 0x560b55e42170 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f4036070 con 0x560b55e42170 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe7ec0129f0 con 0x560b55e42170 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe80641d740 10 client.28922 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe80641d740 10 client.28922 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.434+0000 7fe80641d740 20 client.28922 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:31.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.435+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55e6dc50 0x560b55e70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.435+0000 7fe80641d740 4 client.28922 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.435+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b55db3890 con 0x560b55e6dc50 2024-09-15T22:32:31.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.436+0000 7fe8025e1640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55e6dc50 0x560b55e70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.442+0000 7fe8025e1640 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55e6dc50 0x560b55e70110 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e406bbf15e3473 server_cookie=4080f060a0482628 in_seq=0 out_seq=0 2024-09-15T22:32:31.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.442+0000 7fe7e57fa640 10 client.28922 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.458+0000 7fe7e57fa640 1 -- 172.21.15.77:0/169940695 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b55db3890 con 0x560b55e6dc50 2024-09-15T22:32:31.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.458+0000 7fe7e57fa640 10 client.28922 handle_command_reply: tid=0 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.458+0000 7fe80641d740 1 client.28922 shutdown 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.458+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 msgr2=0x560b55e49f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.458+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e49f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x560b55e3f160 tx=0x7fe7ec00e8e0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 shutdown_connections 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55e6dc50 0x560b55e70110 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e4a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e49f10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 --2- 172.21.15.77:0/169940695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 >> 172.21.15.77:0/169940695 conn(0x560b55e3dde0 msgr2=0x560b55e43ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:31.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.459+0000 7fe7e57fa640 0 client.28922 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.460+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 shutdown_connections 2024-09-15T22:32:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.460+0000 7fe80641d740 1 -- 172.21.15.77:0/169940695 wait complete. 2024-09-15T22:32:31.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.460+0000 7fe80641d740 20 client.28922 trim_cache size 0 max 16384 2024-09-15T22:32:31.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.491+0000 7fe80641d740 1 Processor -- start 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.491+0000 7fe80641d740 1 -- start start 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e42170 0x560b55e759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e4ca30 0x560b55e75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1acf30 con 0x560b55e42170 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1aea70 con 0x560b55e4ca30 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe80641d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1ad600 con 0x560b55e41850 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57726/0 (socket says 172.21.15.77:57726) 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 -- 172.21.15.77:0/2337132715 learned_addr learned my addr 172.21.15.77:0/2337132715 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:31.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8015df640 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e42170 0x560b55e759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e4ca30 0x560b55e75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 -- 172.21.15.77:0/2337132715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e4ca30 msgr2=0x560b55e75f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e4ca30 0x560b55e75f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 -- 172.21.15.77:0/2337132715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e42170 msgr2=0x560b55e759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e42170 0x560b55e759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 -- 172.21.15.77:0/2337132715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55e48980 con 0x560b55e41850 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.492+0000 7fe8025e1640 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4031000 tx=0x7fe7f4030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d63f094032d7d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.493+0000 7fe7e77fe640 1 -- 172.21.15.77:0/2337132715 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40305c0 con 0x560b55e41850 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.493+0000 7fe7e77fe640 1 -- 172.21.15.77:0/2337132715 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f4036070 con 0x560b55e41850 2024-09-15T22:32:31.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.493+0000 7fe7e77fe640 1 -- 172.21.15.77:0/2337132715 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40042a0 con 0x560b55e41850 2024-09-15T22:32:31.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 -- 172.21.15.77:0/2337132715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 msgr2=0x560b55e4b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:31.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4031000 tx=0x7fe7f4030ef0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 -- 172.21.15.77:0/2337132715 shutdown_connections 2024-09-15T22:32:31.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e4ca30 0x560b55e75f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e42170 0x560b55e759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 --2- 172.21.15.77:0/2337132715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e41850 0x560b55e4b940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.494+0000 7fe80641d740 1 -- 172.21.15.77:0/2337132715 >> 172.21.15.77:0/2337132715 conn(0x560b55e3dde0 msgr2=0x560b55e4b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.495+0000 7fe80641d740 1 -- 172.21.15.77:0/2337132715 shutdown_connections 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.495+0000 7fe80641d740 1 -- 172.21.15.77:0/2337132715 wait complete. 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.495+0000 7fe80641d740 1 -- start start 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.495+0000 7fe80641d740 1 Processor -- start 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e46840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1ae3a0 con 0x560b55e41850 2024-09-15T22:32:31.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc1adcd0 con 0x560b55e42170 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc19fda0 con 0x560b55e4ca30 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe80641d740 10 client.0 _ll_register_callbacks cb 0x7fe7ec0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe8015df640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e46840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe8025e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:57730/0 (socket says 172.21.15.77:57730) 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe8025e1640 1 -- 172.21.15.77:0/940277689 learned_addr learned my addr 172.21.15.77:0/940277689 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.496+0000 7fe8025e1640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 -- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 msgr2=0x560b55e46840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e46840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 -- 172.21.15.77:0/940277689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 msgr2=0x560b55e55270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e55270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 -- 172.21.15.77:0/940277689 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55ed79b0 con 0x560b55e4ca30 2024-09-15T22:32:31.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe8025e1640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009dc0 tx=0x7fe7f4049040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b91fafc53a1daaac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe7e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4061060 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fe7f4003c30 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f4036070 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.497+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7f4030920 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 10 client.28846 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b55ed8ae0 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f4036070 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe7f4031440 con 0x560b55e4ca30 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 10 client.28846 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 10 client.28846 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 20 client.28846 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:31.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55ed8d10 0x560b55edb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 4 client.28846 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.15, "await_for": 45.0}] 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.15, "await_for": 45.0}) -- 0x560b55dadbe0 con 0x560b55ed8d10 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.498+0000 7fe8015df640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55ed8d10 0x560b55edb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.499+0000 7fe7e57fa640 10 client.28846 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:31.499+0000 7fe8015df640 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55ed8d10 0x560b55edb100 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d93867e4a8e2ab server_cookie=45f4b53871799a30 in_seq=0 out_seq=0 2024-09-15T22:32:31.504 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:31.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:31 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2602653426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:31.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:31 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2602653426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:31.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122278 15.56 MB/sec execute 272 sec latency 198.181 ms 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 -- 172.21.15.77:0/217103825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08106f00 msgr2=0x7fea0810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/217103825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08106f00 0x7fea0810ea80 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fea00009900 tx=0x7fea0002a410 comp rx=0 tx=0).stop 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 -- 172.21.15.77:0/217103825 shutdown_connections 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/217103825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08106f00 0x7fea0810ea80 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/217103825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08102780 0x7fea081067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/217103825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08101dd0 0x7fea081021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 -- 172.21.15.77:0/217103825 >> 172.21.15.77:0/217103825 conn(0x7fea080fd8b0 msgr2=0x7fea080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 -- 172.21.15.77:0/217103825 shutdown_connections 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 -- 172.21.15.77:0/217103825 wait complete. 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.069+0000 7fea0d7a6640 1 Processor -- start 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 -- start start 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08101dd0 0x7fea080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08106f00 0x7fea08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111f60 con 0x7fea08102780 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111de0 con 0x7fea08101dd0 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea0d7a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea081120e0 con 0x7fea08106f00 2024-09-15T22:32:32.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46316/0 (socket says 172.21.15.77:46316) 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3996193271 learned_addr learned my addr 172.21.15.77:0/3996193271 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea077fe640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08106f00 0x7fea08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08101dd0 0x7fea080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3996193271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08106f00 msgr2=0x7fea08074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08106f00 0x7fea08074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3996193271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08101dd0 msgr2=0x7fea080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08101dd0 0x7fea080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.070+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3996193271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081aad30 con 0x7fea08102780 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000e9f0 tx=0x7fe9f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fea04ff9640 1 -- 172.21.15.77:0/3996193271 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000cd80 con 0x7fea08102780 2024-09-15T22:32:32.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081aaf60 con 0x7fea08102780 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fea04ff9640 1 -- 172.21.15.77:0/3996193271 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f0010910 con 0x7fea08102780 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fea04ff9640 1 -- 172.21.15.77:0/3996193271 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0018950 con 0x7fea08102780 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.071+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081ab410 con 0x7fea08102780 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.073+0000 7fea04ff9640 1 -- 172.21.15.77:0/3996193271 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe9f0010ac0 con 0x7fea08102780 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 Processor -- start 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 -- start start 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea04ff9640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 0x7fe9f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2f50142f0 con 0x55c2f50afb40 2024-09-15T22:32:32.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2f4fbbcf0 con 0x55c2f50b9370 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2f50102b0 con 0x55c2f50ae5d0 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea077fe640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 0x7fe9f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46332/0 (socket says 172.21.15.77:46332) 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea06ffd640 1 -- 172.21.15.77:0/2656747389 learned_addr learned my addr 172.21.15.77:0/2656747389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea04ff9640 1 -- 172.21.15.77:0/3996193271 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fe9f00d0370 con 0x7fea08102780 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fea077fe640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.074+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 -- 172.21.15.77:0/2656747389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 msgr2=0x55c2f50aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 -- 172.21.15.77:0/2656747389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 msgr2=0x55c2f50bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 -- 172.21.15.77:0/2656747389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f4fa03a0 con 0x55c2f50afb40 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea077fe640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 0x7fe9f40a5870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fea00006010 tx=0x7fea00009560 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fea06ffd640 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8007f90 tx=0x7fe9f800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871faa08432b9475 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fe9fd7fa640 1 -- 172.21.15.77:0/2656747389 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8004d20 con 0x55c2f50afb40 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fe9fd7fa640 1 -- 172.21.15.77:0/2656747389 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8010b20 con 0x55c2f50afb40 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.075+0000 7fe9fd7fa640 1 -- 172.21.15.77:0/2656747389 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8005720 con 0x55c2f50afb40 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- 172.21.15.77:0/2656747389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 msgr2=0x55c2f50ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8007f90 tx=0x7fe9f800f850 comp rx=0 tx=0).stop 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- 172.21.15.77:0/2656747389 shutdown_connections 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/2656747389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50ae090 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- 172.21.15.77:0/2656747389 >> 172.21.15.77:0/2656747389 conn(0x55c2f5077310 msgr2=0x55c2f50aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- 172.21.15.77:0/2656747389 shutdown_connections 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- 172.21.15.77:0/2656747389 wait complete. 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.076+0000 7fea0fb7f740 1 -- start start 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 Processor -- start 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2f50bc070 con 0x55c2f50afb40 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2f4fcf060 con 0x55c2f50b9370 2024-09-15T22:32:32.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2f50bc1f0 con 0x55c2f50ae5d0 2024-09-15T22:32:32.112 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:32:32.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea0fb7f740 10 client.0 _ll_register_callbacks cb 0x55c2f4d228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:32.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46348/0 (socket says 172.21.15.77:46348) 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3413086621 learned_addr learned my addr 172.21.15.77:0/3413086621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea077fe640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3413086621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 msgr2=0x55c2f50b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3413086621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 msgr2=0x55c2f50b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fe9fe7fc640 1 -- 172.21.15.77:0/3413086621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f4d853c0 con 0x55c2f50afb40 2024-09-15T22:32:32.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.077+0000 7fea077fe640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00cd7e0 tx=0x7fe9f0005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38661b750d918bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00213f0 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fe9f009c7d0 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f8010b20 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe9f00048a0 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 10 client.20169 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2f50c7b20 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8010b20 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe9f00b5120 con 0x55c2f50afb40 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 10 client.20169 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:32.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 10 client.20169 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 20 client.20169 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f50c8ec0 0x55c2f50cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 4 client.20169 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f5021860 con 0x55c2f50c8ec0 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.078+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f50c8ec0 0x55c2f50cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.079+0000 7fe9ee7fc640 10 client.20169 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.079+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f50c8ec0 0x55c2f50cb2d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a365cd9ad3a632f4 server_cookie=ab816fe0cfacf581 in_seq=0 out_seq=0 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.079+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3413086621 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2f5021860 con 0x55c2f50c8ec0 2024-09-15T22:32:32.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.079+0000 7fe9ee7fc640 10 client.20169 handle_command_reply: tid=0 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.079+0000 7fea0fb7f740 1 client.20169 shutdown 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 msgr2=0x55c2f50b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00cd7e0 tx=0x7fe9f0005180 comp rx=0 tx=0).stop 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 shutdown_connections 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f50c8ec0 0x55c2f50cb2d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50b9370 0x55c2f50b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50afb40 0x55c2f50b5fc0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3413086621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 >> 172.21.15.77:0/3413086621 conn(0x55c2f5077310 msgr2=0x55c2f50b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fe9ee7fc640 0 client.20169 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 shutdown_connections 2024-09-15T22:32:32.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3413086621 wait complete. 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.080+0000 7fea0fb7f740 20 client.20169 trim_cache size 0 max 16384 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.094+0000 7fea0fb7f740 1 Processor -- start 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.094+0000 7fea0fb7f740 1 -- start start 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b55e0 con 0x55c2f50b9370 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b7120 con 0x55c2f50ae5d0 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b5cb0 con 0x55c2f50afb40 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46350/0 (socket says 172.21.15.77:46350) 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 -- 172.21.15.77:0/4118532127 learned_addr learned my addr 172.21.15.77:0/4118532127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:32.117 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea06ffd640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 -- 172.21.15.77:0/4118532127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 msgr2=0x55c2f50b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 -- 172.21.15.77:0/4118532127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 msgr2=0x55c2f50b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fea077fe640 1 -- 172.21.15.77:0/4118532127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f50b4f70 con 0x55c2f50b9370 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.095+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.096+0000 7fea077fe640 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fea00009900 tx=0x7fea00047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d838bca108f057d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.096+0000 7fe9fcff9640 1 -- 172.21.15.77:0/4118532127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0004e070 con 0x55c2f50b9370 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.096+0000 7fe9fcff9640 1 -- 172.21.15.77:0/4118532127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000044d0 con 0x55c2f50b9370 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.096+0000 7fe9fcff9640 1 -- 172.21.15.77:0/4118532127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00035040 con 0x55c2f50b9370 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.096+0000 7fea0fb7f740 1 -- 172.21.15.77:0/4118532127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 msgr2=0x55c2f50b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fea00009900 tx=0x7fea00047000 comp rx=0 tx=0).stop 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- 172.21.15.77:0/4118532127 shutdown_connections 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b1120 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/4118532127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- 172.21.15.77:0/4118532127 >> 172.21.15.77:0/4118532127 conn(0x55c2f5077310 msgr2=0x55c2f50b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- 172.21.15.77:0/4118532127 shutdown_connections 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- 172.21.15.77:0/4118532127 wait complete. 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- start start 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 Processor -- start 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b6a50 con 0x55c2f50b9370 2024-09-15T22:32:32.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b6380 con 0x55c2f50ae5d0 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea081b35e0 con 0x55c2f50afb40 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea0fb7f740 10 client.0 _ll_register_callbacks cb 0x7fe9f000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.097+0000 7fea077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46358/0 (socket says 172.21.15.77:46358) 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 -- 172.21.15.77:0/3276232277 learned_addr learned my addr 172.21.15.77:0/3276232277 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fe9fe7fc640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 -- 172.21.15.77:0/3276232277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 msgr2=0x55c2f50b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 -- 172.21.15.77:0/3276232277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 msgr2=0x55c2f50b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 -- 172.21.15.77:0/3276232277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f5132d40 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea077fe640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fea0003b770 tx=0x7fea00031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bb2f8b07ac18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fe9ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00057070 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fea0003bd80 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea000044d0 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea0004e070 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea0fb7f740 10 client.20175 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.098+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2f5134000 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000044d0 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fea000037b0 con 0x55c2f50b9370 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 10 client.20175 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 10 client.20175 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:32:32.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 20 client.20175 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f51346f0 0x55c2f5136b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 4 client.20175 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f501bd70 con 0x55c2f51346f0 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f51346f0 0x55c2f5136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fe9ee7fc640 10 client.20175 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.099+0000 7fea06ffd640 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f51346f0 0x55c2f5136b10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fd73fb9b9d5b09d server_cookie=c5d40d3d3215660b in_seq=0 out_seq=0 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fe9ee7fc640 1 -- 172.21.15.77:0/3276232277 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f501bd70 con 0x55c2f51346f0 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fe9ee7fc640 10 client.20175 handle_command_reply: tid=0 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 client.20175 shutdown 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 msgr2=0x55c2f50b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fea0003b770 tx=0x7fea00031620 comp rx=0 tx=0).stop 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 shutdown_connections 2024-09-15T22:32:32.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55c2f51346f0 0x55c2f5136b10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2f50b9370 0x55c2f50b3c50 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2f50afb40 0x55c2f50b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 --2- 172.21.15.77:0/3276232277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2f50ae5d0 0x55c2f50b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fe9ee7fc640 0 client.20175 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 >> 172.21.15.77:0/3276232277 conn(0x55c2f5077310 msgr2=0x55c2f50b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 shutdown_connections 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 1 -- 172.21.15.77:0/3276232277 wait complete. 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.100+0000 7fea0fb7f740 20 client.20175 trim_cache size 0 max 16384 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 msgr2=0x7fe9f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 0x7fe9f40a5870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fea00006010 tx=0x7fea00009560 comp rx=0 tx=0).stop 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 msgr2=0x7fea08074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000e9f0 tx=0x7fe9f000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 shutdown_connections 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe9f40a33b0 0x7fe9f40a5870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea08106f00 0x7fea08074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea08102780 0x7fea08074a30 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 --2- 172.21.15.77:0/3996193271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea08101dd0 0x7fea080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 >> 172.21.15.77:0/3996193271 conn(0x7fea080fd8b0 msgr2=0x7fea080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 shutdown_connections 2024-09-15T22:32:32.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.101+0000 7fea0d7a6640 1 -- 172.21.15.77:0/3996193271 wait complete. 2024-09-15T22:32:32.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:32 smithi077 ceph-mon[30980]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-15T22:32:32.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe7e57fa640 1 -- 172.21.15.77:0/940277689 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560b55dadbe0 con 0x560b55ed8d10 2024-09-15T22:32:32.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe7e57fa640 10 client.28846 handle_command_reply: tid=0 2024-09-15T22:32:32.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 client.28846 shutdown 2024-09-15T22:32:32.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 msgr2=0x560b55e46fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009dc0 tx=0x7fe7f4049040 comp rx=0 tx=0).stop 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 shutdown_connections 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b55ed8d10 0x560b55edb100 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b55e4ca30 0x560b55e46fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b55e42170 0x560b55e46840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe7e57fa640 0 client.28846 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 --2- 172.21.15.77:0/940277689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b55e41850 0x560b55e55270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 >> 172.21.15.77:0/940277689 conn(0x560b55e3dde0 msgr2=0x560b55e43ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 shutdown_connections 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 1 -- 172.21.15.77:0/940277689 wait complete. 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.342+0000 7fe80641d740 20 client.28846 trim_cache size 0 max 16384 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 74, 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "3ce00f87": { 2024-09-15T22:32:32.345 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 74, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 148.2, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:32.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:32.347 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 msgr2=0x7fe7e80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 0x7fe7e80a5a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f809b8d0 tx=0x7fe7f80d3750 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 msgr2=0x7fe7fc1a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c890 tx=0x7fe7f800cc90 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 shutdown_connections 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7e80a3580 0x7fe7e80a5a40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7fc105ec0 0x7fe7fc19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7fc1011e0 0x7fe7fc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 --2- 172.21.15.77:0/126503038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7fc1008c0 0x7fe7fc1a46b0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 >> 172.21.15.77:0/126503038 conn(0x7fe7fc0fc7d0 msgr2=0x7fe7fc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:32.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 shutdown_connections 2024-09-15T22:32:32.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:32.343+0000 7fe804044640 1 -- 172.21.15.77:0/126503038 wait complete. 2024-09-15T22:32:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:32 smithi196 ceph-mon[35109]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-15T22:32:32.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.51 MB/sec execute 273 sec latency 1063.067 ms 2024-09-15T22:32:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:32 smithi117 ceph-mon[35021]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-09-15T22:32:33.045 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3ce00f87', quiesce took 6.7 seconds. Will release after: 21.45 2024-09-15T22:32:33.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.45 MB/sec execute 274 sec latency 2063.179 ms 2024-09-15T22:32:33.684 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:32:33.931 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:34.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:34 smithi077 ceph-mon[30980]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:32:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:34 smithi196 ceph-mon[35109]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:32:34.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.40 MB/sec execute 275 sec latency 3063.258 ms 2024-09-15T22:32:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:34 smithi117 ceph-mon[35021]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 -- 172.21.15.77:0/22704983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0106a80 msgr2=0x7fc9d0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/22704983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0106a80 0x7fc9d0106e60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009930 tx=0x7fc9c402a420 comp rx=0 tx=0).stop 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 -- 172.21.15.77:0/22704983 shutdown_connections 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/22704983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0101420 0x7fc9d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/22704983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0100a80 0x7fc9d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/22704983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0106a80 0x7fc9d0106e60 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 -- 172.21.15.77:0/22704983 >> 172.21.15.77:0/22704983 conn(0x7fc9d00fc7a0 msgr2=0x7fc9d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 -- 172.21.15.77:0/22704983 shutdown_connections 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.959+0000 7fc9d6d39640 1 -- 172.21.15.77:0/22704983 wait complete. 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 Processor -- start 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 -- start start 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 0x7fc9d01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0106a80 0x7fc9d01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110e40 con 0x7fc9d0100a80 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110cc0 con 0x7fc9d0101420 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d6d39640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110fc0 con 0x7fc9d0106a80 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46384/0 (socket says 172.21.15.77:46384) 2024-09-15T22:32:34.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/2565598411 learned_addr learned my addr 172.21.15.77:0/2565598411 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0106a80 0x7fc9d01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9cffff640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 0x7fc9d01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/2565598411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0106a80 msgr2=0x7fc9d01a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0106a80 0x7fc9d01a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/2565598411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 msgr2=0x7fc9d01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 0x7fc9d01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/2565598411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01a4850 con 0x7fc9d0100a80 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.960+0000 7fc9cffff640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 0x7fc9d01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009230 tx=0x7fc9c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9cdffb640 1 -- 172.21.15.77:0/2565598411 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c403f030 con 0x7fc9d0100a80 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a4a20 con 0x7fc9d0100a80 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9cdffb640 1 -- 172.21.15.77:0/2565598411 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c4003ad0 con 0x7fc9d0100a80 2024-09-15T22:32:34.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9cdffb640 1 -- 172.21.15.77:0/2565598411 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4035070 con 0x7fc9d0100a80 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.961+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a4ed0 con 0x7fc9d0100a80 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.963+0000 7fc9cdffb640 1 -- 172.21.15.77:0/2565598411 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc9c4042030 con 0x7fc9d0100a80 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.963+0000 7fc9d9112740 1 Processor -- start 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 -- start start 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9cdffb640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 0x7fc9c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572f3c6e2f0 con 0x5572f3d09b40 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5572f3c15cf0 con 0x5572f3d17780 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d9112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572f3c6a2b0 con 0x5572f3d085d0 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9cffff640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 0x7fc9c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46386/0 (socket says 172.21.15.77:46386) 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 -- 172.21.15.77:0/12854711 learned_addr learned my addr 172.21.15.77:0/12854711 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9cdffb640 1 -- 172.21.15.77:0/2565598411 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fc9c40f05a0 con 0x7fc9d0100a80 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 -- 172.21.15.77:0/12854711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 msgr2=0x5572f3d08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 -- 172.21.15.77:0/12854711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 msgr2=0x5572f3d19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d19b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d52d6640 1 -- 172.21.15.77:0/12854711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f3bfa3a0 con 0x5572f3d09b40 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:32:34.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.964+0000 7fc9cffff640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 0x7fc9c00a5870 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0058c0 tx=0x7fc9bc005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x5572f3d09070 tx=0x7fc9c8009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd0dabc1251260f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9baffd640 1 -- 172.21.15.77:0/12854711 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c801a030 con 0x5572f3d09b40 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9baffd640 1 -- 172.21.15.77:0/12854711 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c80120d0 con 0x5572f3d09b40 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9baffd640 1 -- 172.21.15.77:0/12854711 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8012cf0 con 0x5572f3d09b40 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 -- 172.21.15.77:0/12854711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 msgr2=0x5572f3d08090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x5572f3d09070 tx=0x7fc9c8009b10 comp rx=0 tx=0).stop 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 -- 172.21.15.77:0/12854711 shutdown_connections 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 --2- 172.21.15.77:0/12854711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d08090 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 -- 172.21.15.77:0/12854711 >> 172.21.15.77:0/12854711 conn(0x5572f3cd1310 msgr2=0x5572f3d04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 -- 172.21.15.77:0/12854711 shutdown_connections 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.965+0000 7fc9d9112740 1 -- 172.21.15.77:0/12854711 wait complete. 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 -- start start 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 Processor -- start 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d17120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572f3d1a420 con 0x5572f3d09b40 2024-09-15T22:32:34.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5572f3c29060 con 0x5572f3d17780 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572f3d1a5a0 con 0x5572f3d085d0 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d9112740 10 client.0 _ll_register_callbacks cb 0x5572f397c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46400/0 (socket says 172.21.15.77:46400) 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9cffff640 1 -- 172.21.15.77:0/3249673422 learned_addr learned my addr 172.21.15.77:0/3249673422 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.966+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d17120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 -- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 msgr2=0x5572f3d17120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d17120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 -- 172.21.15.77:0/3249673422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 msgr2=0x5572f3d14970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:34.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d14970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 -- 172.21.15.77:0/3249673422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f39df3c0 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d14970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9cffff640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc011490 tx=0x7fc9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccfb735becf1b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc022070 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fc9bc01e860 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c80120d0 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc02e050 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9d9112740 10 client.20187 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572f3d25ed0 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c80120d0 con 0x5572f3d09b40 2024-09-15T22:32:34.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.967+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc9bc010c20 con 0x5572f3d09b40 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 10 client.20187 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 10 client.20187 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 20 client.20187 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d27330 0x5572f3d29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 4 client.20187 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f3c7b860 con 0x5572f3d27330 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d27330 0x5572f3d29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9affff640 10 client.20187 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.968+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d27330 0x5572f3d29770 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5fd3facb2cdbd78 server_cookie=355a73adb5583453 in_seq=0 out_seq=0 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9affff640 1 -- 172.21.15.77:0/3249673422 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572f3c7b860 con 0x5572f3d27330 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9affff640 10 client.20187 handle_command_reply: tid=0 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 client.20187 shutdown 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 msgr2=0x5572f3d143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:34.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc011490 tx=0x7fc9bc0046c0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 shutdown_connections 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d27330 0x5572f3d29770 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d14970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d143d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9affff640 0 client.20187 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 --2- 172.21.15.77:0/3249673422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d085d0 0x5572f3d17120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 >> 172.21.15.77:0/3249673422 conn(0x5572f3cd1310 msgr2=0x5572f3d0a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 shutdown_connections 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 1 -- 172.21.15.77:0/3249673422 wait complete. 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.969+0000 7fc9d9112740 20 client.20187 trim_cache size 0 max 16384 2024-09-15T22:32:35.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 Processor -- start 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 -- start start 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d09b40 0x5572f3d133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01acdb0 con 0x5572f3d085d0 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ae8f0 con 0x5572f3d17780 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d9112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ad480 con 0x5572f3d09b40 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46414/0 (socket says 172.21.15.77:46414) 2024-09-15T22:32:35.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/1604787978 learned_addr learned my addr 172.21.15.77:0/1604787978 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/1604787978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d09b40 msgr2=0x5572f3d133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d09b40 0x5572f3d133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/1604787978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 msgr2=0x5572f3d2f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.982+0000 7fc9d4ad5640 1 -- 172.21.15.77:0/1604787978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f3d0b120 con 0x5572f3d085d0 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40bc1f0 tx=0x7fc9c40b8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112d4aafa44fb880 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9ba7fc640 1 -- 172.21.15.77:0/1604787978 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4049040 con 0x5572f3d085d0 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9ba7fc640 1 -- 172.21.15.77:0/1604787978 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c40ba590 con 0x5572f3d085d0 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9ba7fc640 1 -- 172.21.15.77:0/1604787978 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003c80 con 0x5572f3d085d0 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 -- 172.21.15.77:0/1604787978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 msgr2=0x5572f3d236c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40bc1f0 tx=0x7fc9c40b8070 comp rx=0 tx=0).stop 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 -- 172.21.15.77:0/1604787978 shutdown_connections 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d17780 0x5572f3d2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d09b40 0x5572f3d133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 --2- 172.21.15.77:0/1604787978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d085d0 0x5572f3d236c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 -- 172.21.15.77:0/1604787978 >> 172.21.15.77:0/1604787978 conn(0x5572f3cd1310 msgr2=0x5572f3d12ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 -- 172.21.15.77:0/1604787978 shutdown_connections 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.983+0000 7fc9d9112740 1 -- 172.21.15.77:0/1604787978 wait complete. 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 -- start start 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 Processor -- start 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 0x5572f3d1f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d17780 0x5572f3d1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ae220 con 0x5572f3d09b40 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01adb50 con 0x5572f3d085d0 2024-09-15T22:32:35.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aadb0 con 0x5572f3d17780 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d9112740 10 client.0 _ll_register_callbacks cb 0x7fc9bc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46428/0 (socket says 172.21.15.77:46428) 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9cffff640 1 -- 172.21.15.77:0/156807720 learned_addr learned my addr 172.21.15.77:0/156807720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 0x5572f3d1f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9cffff640 1 -- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d17780 msgr2=0x5572f3d1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9d52d6640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d17780 0x5572f3d1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.984+0000 7fc9cffff640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d17780 0x5572f3d1c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9cffff640 1 -- 172.21.15.77:0/156807720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 msgr2=0x5572f3d1f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9cffff640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 0x5572f3d1f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9cffff640 1 -- 172.21.15.77:0/156807720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f3d1cd40 con 0x5572f3d09b40 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 0x5572f3d1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9cffff640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc000c00 tx=0x7fc9bc026800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e132cc08171d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:35.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc036020 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fc9bc026dd0 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c40ba590 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc03e050 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 10 client.20193 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572f3d92780 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c40ba590 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc9bc0046c0 con 0x5572f3d09b40 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 10 client.20193 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 10 client.20193 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 20 client.20193 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:35.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d92f80 0x5572f3d953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 4 client.20193 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.985+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572f3c75d70 con 0x5572f3d92f80 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.986+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d92f80 0x5572f3d953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d4ad5640 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d92f80 0x5572f3d953a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae5d2d219509f03a server_cookie=b9cf097227b11e62 in_seq=0 out_seq=0 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9affff640 10 client.20193 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9affff640 1 -- 172.21.15.77:0/156807720 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572f3c75d70 con 0x5572f3d92f80 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9affff640 10 client.20193 handle_command_reply: tid=0 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 client.20193 shutdown 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 msgr2=0x5572f3d1fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc000c00 tx=0x7fc9bc026800 comp rx=0 tx=0).stop 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 shutdown_connections 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x5572f3d92f80 0x5572f3d953a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572f3d17780 0x5572f3d1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572f3d09b40 0x5572f3d1fd70 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 --2- 172.21.15.77:0/156807720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f3d085d0 0x5572f3d1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9affff640 0 client.20193 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 >> 172.21.15.77:0/156807720 conn(0x5572f3cd1310 msgr2=0x5572f3d134c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 shutdown_connections 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 1 -- 172.21.15.77:0/156807720 wait complete. 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.987+0000 7fc9d9112740 20 client.20193 trim_cache size 0 max 16384 2024-09-15T22:32:35.007 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 msgr2=0x7fc9c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 0x7fc9c00a5870 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0058c0 tx=0x7fc9bc005850 comp rx=0 tx=0).stop 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 msgr2=0x7fc9d0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009230 tx=0x7fc9c4004270 comp rx=0 tx=0).stop 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 shutdown_connections 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc9c00a33b0 0x7fc9c00a5870 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9d0106a80 0x7fc9d01a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9d0101420 0x7fc9d01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 --2- 172.21.15.77:0/2565598411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0100a80 0x7fc9d0199290 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:35.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 >> 172.21.15.77:0/2565598411 conn(0x7fc9d00fc7a0 msgr2=0x7fc9d00fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:35.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.988+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 shutdown_connections 2024-09-15T22:32:35.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:34.989+0000 7fc9d6d39640 1 -- 172.21.15.77:0/2565598411 wait complete. 2024-09-15T22:32:35.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.34 MB/sec execute 276 sec latency 4063.313 ms 2024-09-15T22:32:36.119 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:32:36.369 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:36.489 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:36 smithi077 ceph-mon[30980]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-09-15T22:32:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:36 smithi196 ceph-mon[35109]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-09-15T22:32:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:36 smithi117 ceph-mon[35021]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.7 MiB/s wr, 175 op/s 2024-09-15T22:32:36.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.29 MB/sec execute 277 sec latency 5063.338 ms 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- 172.21.15.77:0/1566783490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c106f20 msgr2=0x7f437c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 --2- 172.21.15.77:0/1566783490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c106f20 0x7f437c10eaa0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ca0 tx=0x7f437402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- 172.21.15.77:0/1566783490 shutdown_connections 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 --2- 172.21.15.77:0/1566783490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c106f20 0x7f437c10eaa0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 --2- 172.21.15.77:0/1566783490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c1027a0 0x7f437c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 --2- 172.21.15.77:0/1566783490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c101df0 0x7f437c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- 172.21.15.77:0/1566783490 >> 172.21.15.77:0/1566783490 conn(0x7f437c0fd8f0 msgr2=0x7f437c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- 172.21.15.77:0/1566783490 shutdown_connections 2024-09-15T22:32:37.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- 172.21.15.77:0/1566783490 wait complete. 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 Processor -- start 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.959+0000 7f4382257640 1 -- start start 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c101df0 0x7f437c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c106f20 0x7f437c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f437c111fd0 con 0x7f437c1027a0 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f437c111e50 con 0x7f437c101df0 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f437c112150 con 0x7f437c106f20 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46446/0 (socket says 172.21.15.77:46446) 2024-09-15T22:32:37.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 -- 172.21.15.77:0/462257458 learned_addr learned my addr 172.21.15.77:0/462257458 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f437bfff640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c106f20 0x7f437c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f437b7fe640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c101df0 0x7f437c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 -- 172.21.15.77:0/462257458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c106f20 msgr2=0x7f437c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c106f20 0x7f437c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 -- 172.21.15.77:0/462257458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c101df0 msgr2=0x7f437c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c101df0 0x7f437c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 -- 172.21.15.77:0/462257458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c19d440 con 0x7f437c1027a0 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4372ffd640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f436400d950 tx=0x7f436400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f43797fa640 1 -- 172.21.15.77:0/462257458 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364017070 con 0x7f437c1027a0 2024-09-15T22:32:37.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c19d730 con 0x7f437c1027a0 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f43797fa640 1 -- 172.21.15.77:0/462257458 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364009c30 con 0x7f437c1027a0 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.960+0000 7f43797fa640 1 -- 172.21.15.77:0/462257458 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436400f6b0 con 0x7f437c1027a0 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.961+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1a5010 con 0x7f437c1027a0 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.963+0000 7f43797fa640 1 -- 172.21.15.77:0/462257458 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f43640109d0 con 0x7f437c1027a0 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.963+0000 7f4384630740 1 Processor -- start 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.963+0000 7f4384630740 1 -- start start 2024-09-15T22:32:37.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de98673090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 0x55de98673a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de98684b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de985d92f0 con 0x55de98682780 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de98580cf0 con 0x55de986735d0 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4384630740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de985d52b0 con 0x55de98674b40 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de98684b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de98684b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46458/0 (socket says 172.21.15.77:46458) 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f4372ffd640 1 -- 172.21.15.77:0/3993240255 learned_addr learned my addr 172.21.15.77:0/3993240255 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f43797fa640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 0x7f43680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 0x55de98673a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437bfff640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 0x7f43680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437bfff640 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de98673090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 -- 172.21.15.77:0/3993240255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 msgr2=0x55de98673090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de98673090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f43797fa640 1 -- 172.21.15.77:0/462257458 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f43640cfdc0 con 0x7f437c1027a0 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 -- 172.21.15.77:0/3993240255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 msgr2=0x55de98684b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de98684b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 -- 172.21.15.77:0/3993240255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de985653a0 con 0x55de986735d0 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437bfff640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 0x7f43680a5870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4374004af0 tx=0x7f4374004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.964+0000 7f437b7fe640 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 0x55de98673a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f436c00bbe0 tx=0x7f436c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c57af457021a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4371ffb640 1 -- 172.21.15.77:0/3993240255 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c015070 con 0x55de986735d0 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4371ffb640 1 -- 172.21.15.77:0/3993240255 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c005500 con 0x55de986735d0 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4371ffb640 1 -- 172.21.15.77:0/3993240255 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c011440 con 0x55de986735d0 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 -- 172.21.15.77:0/3993240255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 msgr2=0x55de98673a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 0x55de98673a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f436c00bbe0 tx=0x7f436c00f730 comp rx=0 tx=0).stop 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 -- 172.21.15.77:0/3993240255 shutdown_connections 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de98684b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de986735d0 0x55de98673a10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 --2- 172.21.15.77:0/3993240255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de98673090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 -- 172.21.15.77:0/3993240255 >> 172.21.15.77:0/3993240255 conn(0x55de9863c310 msgr2=0x55de9866fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 -- 172.21.15.77:0/3993240255 shutdown_connections 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.965+0000 7f4384630740 1 -- 172.21.15.77:0/3993240255 wait complete. 2024-09-15T22:32:37.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 -- start start 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 Processor -- start 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de98682140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9867f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55de98685480 con 0x55de98674b40 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de98594060 con 0x55de98682780 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55de98685600 con 0x55de986735d0 2024-09-15T22:32:37.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4384630740 10 client.0 _ll_register_callbacks cb 0x55de982e78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46472/0 (socket says 172.21.15.77:46472) 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f4372ffd640 1 -- 172.21.15.77:0/4134761636 learned_addr learned my addr 172.21.15.77:0/4134761636 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.966+0000 7f437bfff640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9867f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f437b7fe640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de98682140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 -- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 msgr2=0x55de98682140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de98682140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 -- 172.21.15.77:0/4134761636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 msgr2=0x55de9867f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9867f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 -- 172.21.15.77:0/4134761636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9834a3c0 con 0x55de98674b40 2024-09-15T22:32:37.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4372ffd640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4364010fd0 tx=0x7f4364005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90b7d4abe72119f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436409c3b0 con 0x55de98674b40 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f4364097a00 con 0x55de98674b40 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436c005500 con 0x55de98674b40 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4364009de0 con 0x55de98674b40 2024-09-15T22:32:37.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 10 client.20205 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:37.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55de98690f40 con 0x55de98674b40 2024-09-15T22:32:37.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c005500 con 0x55de98674b40 2024-09-15T22:32:37.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f43640d3020 con 0x55de98674b40 2024-09-15T22:32:37.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 10 client.20205 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:37.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 10 client.20205 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:32:37.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 20 client.20205 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:37.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.967+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de98692350 0x55de98694780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.968+0000 7f4384630740 4 client.20205 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:37.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.968+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de985e6860 con 0x55de98692350 2024-09-15T22:32:37.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.968+0000 7f437b7fe640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de98692350 0x55de98694780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.968+0000 7f437b7fe640 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de98692350 0x55de98694780 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a362ccf7c9749ad3 server_cookie=576fafdf6d4ceae in_seq=0 out_seq=0 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.968+0000 7f435affd640 10 client.20205 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f435affd640 1 -- 172.21.15.77:0/4134761636 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de985e6860 con 0x55de98692350 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f435affd640 10 client.20205 handle_command_reply: tid=0 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 client.20205 shutdown 2024-09-15T22:32:37.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 msgr2=0x55de9867f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4364010fd0 tx=0x7f4364005880 comp rx=0 tx=0).stop 2024-09-15T22:32:37.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 shutdown_connections 2024-09-15T22:32:37.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de98692350 0x55de98694780 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9867f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98674b40 0x55de9867f3f0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f435affd640 0 client.20205 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:32:37.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 --2- 172.21.15.77:0/4134761636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de98682140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 >> 172.21.15.77:0/4134761636 conn(0x55de9863c310 msgr2=0x55de98675960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 shutdown_connections 2024-09-15T22:32:37.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 1 -- 172.21.15.77:0/4134761636 wait complete. 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.969+0000 7f4384630740 20 client.20205 trim_cache size 0 max 16384 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 Processor -- start 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 -- start start 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de9867e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9869a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ace30 con 0x55de986735d0 2024-09-15T22:32:37.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ae970 con 0x55de98682780 2024-09-15T22:32:37.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f4384630740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ad500 con 0x55de98674b40 2024-09-15T22:32:37.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f437b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f437b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46482/0 (socket says 172.21.15.77:46482) 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.982+0000 7f437b7fe640 1 -- 172.21.15.77:0/1842077445 learned_addr learned my addr 172.21.15.77:0/1842077445 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4372ffd640 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de9867e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 -- 172.21.15.77:0/1842077445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 msgr2=0x55de9867e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de9867e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 -- 172.21.15.77:0/1842077445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 msgr2=0x55de9869a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9869a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 -- 172.21.15.77:0/1842077445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de98676120 con 0x55de986735d0 2024-09-15T22:32:37.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f437b7fe640 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f436c004c00 tx=0x7f436c00b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e207643d1e862ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f43717fa640 1 -- 172.21.15.77:0/1842077445 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c015050 con 0x55de986735d0 2024-09-15T22:32:37.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f43717fa640 1 -- 172.21.15.77:0/1842077445 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c0054e0 con 0x55de986735d0 2024-09-15T22:32:37.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f43717fa640 1 -- 172.21.15.77:0/1842077445 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0256c0 con 0x55de986735d0 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 -- 172.21.15.77:0/1842077445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 msgr2=0x55de9868e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f436c004c00 tx=0x7f436c00b4d0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 -- 172.21.15.77:0/1842077445 shutdown_connections 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98682780 0x55de9869a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de98674b40 0x55de9867e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 --2- 172.21.15.77:0/1842077445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de986735d0 0x55de9868e720 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.983+0000 7f4384630740 1 -- 172.21.15.77:0/1842077445 >> 172.21.15.77:0/1842077445 conn(0x55de9863c310 msgr2=0x55de9867dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- 172.21.15.77:0/1842077445 shutdown_connections 2024-09-15T22:32:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- 172.21.15.77:0/1842077445 wait complete. 2024-09-15T22:32:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- start start 2024-09-15T22:32:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 Processor -- start 2024-09-15T22:32:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de9868b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 0x55de9867c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ae2a0 con 0x55de98682780 2024-09-15T22:32:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1adbd0 con 0x55de98674b40 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1aae30 con 0x55de986735d0 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f4384630740 10 client.0 _ll_register_callbacks cb 0x7f43640cd0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f437bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f437bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46486/0 (socket says 172.21.15.77:46486) 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f437bfff640 1 -- 172.21.15.77:0/151639482 learned_addr learned my addr 172.21.15.77:0/151639482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.984+0000 7f437b7fe640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de9868b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f4372ffd640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 0x55de9867c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 -- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 msgr2=0x55de9868b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de9868b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 -- 172.21.15.77:0/151639482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 msgr2=0x55de9867c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 0x55de9867c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 -- 172.21.15.77:0/151639482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9867cf80 con 0x55de98682780 2024-09-15T22:32:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f4372ffd640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 0x55de9867c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f437bfff640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ca0 tx=0x7f437402f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb9ae2e3bd0a2af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f435affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374051020 con 0x55de98682780 2024-09-15T22:32:37.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f4374031860 con 0x55de98682780 2024-09-15T22:32:37.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436c0054e0 con 0x55de98682780 2024-09-15T22:32:37.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4374047040 con 0x55de98682780 2024-09-15T22:32:37.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f4384630740 10 client.20211 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:37.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55de986fd0c0 con 0x55de98682780 2024-09-15T22:32:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.985+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c0054e0 con 0x55de98682780 2024-09-15T22:32:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f437402f610 con 0x55de98682780 2024-09-15T22:32:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 10 client.20211 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 10 client.20211 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:32:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 20 client.20211 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:37.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de986fd890 0x55de986ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:37.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 4 client.20211 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:32:37.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de985e0d70 con 0x55de986fd890 2024-09-15T22:32:37.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f437b7fe640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de986fd890 0x55de986ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:37.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f435affd640 10 client.20211 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:32:37.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.986+0000 7f437b7fe640 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de986fd890 0x55de986ffcb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f05426f00843da07 server_cookie=728c5371eb4532b2 in_seq=0 out_seq=0 2024-09-15T22:32:37.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f435affd640 1 -- 172.21.15.77:0/151639482 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de985e0d70 con 0x55de986fd890 2024-09-15T22:32:37.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f435affd640 10 client.20211 handle_command_reply: tid=0 2024-09-15T22:32:37.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 client.20211 shutdown 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 msgr2=0x55de9867c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ca0 tx=0x7f437402f460 comp rx=0 tx=0).stop 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 shutdown_connections 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55de986fd890 0x55de986ffcb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55de98682780 0x55de9867c750 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f435affd640 0 client.20211 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de98674b40 0x55de9867c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 --2- 172.21.15.77:0/151639482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de986735d0 0x55de9868b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 >> 172.21.15.77:0/151639482 conn(0x55de9863c310 msgr2=0x55de9867e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 shutdown_connections 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 1 -- 172.21.15.77:0/151639482 wait complete. 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4384630740 20 client.20211 trim_cache size 0 max 16384 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 msgr2=0x7f43680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 0x7f43680a5870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4374004af0 tx=0x7f4374004a40 comp rx=0 tx=0).stop 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 msgr2=0x7f437c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f436400d950 tx=0x7f436400de10 comp rx=0 tx=0).stop 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 shutdown_connections 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f43680a33b0 0x7f43680a5870 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f437c106f20 0x7f437c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f437c1027a0 0x7f437c1989d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.987+0000 7f4382257640 1 --2- 172.21.15.77:0/462257458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c101df0 0x7f437c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:37.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.988+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 >> 172.21.15.77:0/462257458 conn(0x7f437c0fd8f0 msgr2=0x7f437c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:37.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.988+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 shutdown_connections 2024-09-15T22:32:37.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:36.988+0000 7f4382257640 1 -- 172.21.15.77:0/462257458 wait complete. 2024-09-15T22:32:37.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.23 MB/sec execute 278 sec latency 6063.417 ms 2024-09-15T22:32:37.955 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:32:38.203 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:38.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:38 smithi077 ceph-mon[30980]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:32:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:38 smithi196 ceph-mon[35109]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:32:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:38 smithi117 ceph-mon[35021]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 239 op/s 2024-09-15T22:32:38.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.18 MB/sec execute 279 sec latency 7063.529 ms 2024-09-15T22:32:38.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 -- 172.21.15.77:0/1905716614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594101de0 msgr2=0x7f95941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 --2- 172.21.15.77:0/1905716614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594101de0 0x7f95941021c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9588009900 tx=0x7f958802a410 comp rx=0 tx=0).stop 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 -- 172.21.15.77:0/1905716614 shutdown_connections 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 --2- 172.21.15.77:0/1905716614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594106f10 0x7f959410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 --2- 172.21.15.77:0/1905716614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594102790 0x7f95941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 --2- 172.21.15.77:0/1905716614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594101de0 0x7f95941021c0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 -- 172.21.15.77:0/1905716614 >> 172.21.15.77:0/1905716614 conn(0x7f95940fd8c0 msgr2=0x7f95940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 -- 172.21.15.77:0/1905716614 shutdown_connections 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 -- 172.21.15.77:0/1905716614 wait complete. 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.895+0000 7f959af6f640 1 Processor -- start 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 -- start start 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594101de0 0x7f959419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594102790 0x7f959419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9594111f70 con 0x7f9594106f10 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9594111df0 con 0x7f9594101de0 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95941120f0 con 0x7f9594102790 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46498/0 (socket says 172.21.15.77:46498) 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 -- 172.21.15.77:0/1192762380 learned_addr learned my addr 172.21.15.77:0/1192762380 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f9593fff640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594102790 0x7f959419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f9598d0b640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594101de0 0x7f959419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 -- 172.21.15.77:0/1192762380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594102790 msgr2=0x7f959419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594102790 0x7f959419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 -- 172.21.15.77:0/1192762380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594101de0 msgr2=0x7f959419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594101de0 0x7f959419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 -- 172.21.15.77:0/1192762380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959419f5c0 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959950c640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f958c00eaa0 tx=0x7f958c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f9591ffb640 1 -- 172.21.15.77:0/1192762380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c004d20 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f9591ffb640 1 -- 172.21.15.77:0/1192762380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958c005020 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.896+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959419f8b0 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.897+0000 7f9591ffb640 1 -- 172.21.15.77:0/1192762380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0116d0 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.897+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941a7190 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f9591ffb640 1 -- 172.21.15.77:0/1192762380 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f958c0051d0 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f959d348740 1 Processor -- start 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f9591ffb640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 0x7f95840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f959d348740 1 -- start start 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f9593fff640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 0x7f95840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f9591ffb640 1 -- 172.21.15.77:0/1192762380 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f958c0d0170 con 0x7f9594106f10 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f959d348740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.899+0000 7f959d348740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959d348740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff7ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959d348740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639ff7432f0 con 0x5639ff7deb40 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959d348740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5639ff6eacf0 con 0x5639ff7dd5d0 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959d348740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5639ff73f2b0 con 0x5639ff7e8370 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46504/0 (socket says 172.21.15.77:46504) 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 -- 172.21.15.77:0/164114618 learned_addr learned my addr 172.21.15.77:0/164114618 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9593fff640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959950c640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff7ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9593fff640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 0x7f95840a5870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95940fdec0 tx=0x7f958000f3c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 -- 172.21.15.77:0/164114618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 msgr2=0x5639ff7ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff7ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 -- 172.21.15.77:0/164114618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 msgr2=0x5639ff7dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 -- 172.21.15.77:0/164114618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ff6cf3a0 con 0x5639ff7deb40 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f959950c640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff7ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f9598d0b640 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9588009900 tx=0x7f9588031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321d34996e5c7793 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f957effd640 1 -- 172.21.15.77:0/164114618 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588040070 con 0x5639ff7deb40 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f957effd640 1 -- 172.21.15.77:0/164114618 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9588005020 con 0x5639ff7deb40 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.900+0000 7f957effd640 1 -- 172.21.15.77:0/164114618 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588035040 con 0x5639ff7deb40 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- 172.21.15.77:0/164114618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 msgr2=0x5639ff7dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9588009900 tx=0x7f9588031630 comp rx=0 tx=0).stop 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- 172.21.15.77:0/164114618 shutdown_connections 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff7ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 --2- 172.21.15.77:0/164114618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7dd090 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- 172.21.15.77:0/164114618 >> 172.21.15.77:0/164114618 conn(0x5639ff7a6310 msgr2=0x5639ff7d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- 172.21.15.77:0/164114618 shutdown_connections 2024-09-15T22:32:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- 172.21.15.77:0/164114618 wait complete. 2024-09-15T22:32:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 -- start start 2024-09-15T22:32:38.934 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.901+0000 7f959d348740 1 Processor -- start 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7dd5d0 0x5639ff7e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 0x5639ff7e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639ff7eb010 con 0x5639ff7deb40 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5639ff6fe060 con 0x5639ff7e8370 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5639ff7eb190 con 0x5639ff7dd5d0 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959d348740 10 client.0 _ll_register_callbacks cb 0x5639ff4518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46520/0 (socket says 172.21.15.77:46520) 2024-09-15T22:32:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 -- 172.21.15.77:0/691050162 learned_addr learned my addr 172.21.15.77:0/691050162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959950c640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 0x5639ff7e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9598d0b640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7dd5d0 0x5639ff7e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 -- 172.21.15.77:0/691050162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7dd5d0 msgr2=0x5639ff7e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7dd5d0 0x5639ff7e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 -- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 msgr2=0x5639ff7e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 0x5639ff7e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 -- 172.21.15.77:0/691050162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ff4b43c0 con 0x5639ff7deb40 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f959950c640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 0x5639ff7e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9593fff640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9580010de0 tx=0x7f9580004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87dd2e7974f2e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.902+0000 7f9573fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958002a060 con 0x5639ff7deb40 2024-09-15T22:32:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f958000fe40 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9588005020 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9580031050 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 10 client.20223 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5639ff7f6ac0 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9588005020 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f95800256e0 con 0x5639ff7deb40 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 10 client.20223 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 10 client.20223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 20 client.20223 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff7fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 4 client.20223 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639ff750860 con 0x5639ff7f7fd0 2024-09-15T22:32:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.903+0000 7f9598d0b640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff7fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.904+0000 7f9573fff640 10 client.20223 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.904+0000 7f9598d0b640 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff7fa3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca68b5d19752232b server_cookie=3f6f8107e54328ae in_seq=0 out_seq=0 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f9573fff640 1 -- 172.21.15.77:0/691050162 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639ff750860 con 0x5639ff7f7fd0 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f9573fff640 10 client.20223 handle_command_reply: tid=0 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 client.20223 shutdown 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 msgr2=0x5639ff7e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9580010de0 tx=0x7f9580004730 comp rx=0 tx=0).stop 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 shutdown_connections 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff7fa3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7e8370 0x5639ff7e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f9573fff640 0 client.20223 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7e4fc0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 --2- 172.21.15.77:0/691050162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7dd5d0 0x5639ff7e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 >> 172.21.15.77:0/691050162 conn(0x5639ff7a6310 msgr2=0x5639ff7df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 shutdown_connections 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 1 -- 172.21.15.77:0/691050162 wait complete. 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.905+0000 7f959d348740 20 client.20223 trim_cache size 0 max 16384 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 Processor -- start 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 -- start start 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff800200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b11a0 con 0x5639ff7deb40 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b2ce0 con 0x5639ff7dd5d0 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959d348740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b1870 con 0x5639ff7e8370 2024-09-15T22:32:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46530/0 (socket says 172.21.15.77:46530) 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 -- 172.21.15.77:0/3720441123 learned_addr learned my addr 172.21.15.77:0/3720441123 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9598d0b640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f959950c640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff800200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 -- 172.21.15.77:0/3720441123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 msgr2=0x5639ff800200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff800200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 -- 172.21.15.77:0/3720441123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 msgr2=0x5639ff7f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 -- 172.21.15.77:0/3720441123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ff7e0120 con 0x5639ff7deb40 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9598d0b640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.918+0000 7f9593fff640 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9580010db0 tx=0x7f9580005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58609490f7f2862a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f957e7fc640 1 -- 172.21.15.77:0/3720441123 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580018070 con 0x5639ff7deb40 2024-09-15T22:32:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f957e7fc640 1 -- 172.21.15.77:0/3720441123 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958001b5c0 con 0x5639ff7deb40 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f957e7fc640 1 -- 172.21.15.77:0/3720441123 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580035640 con 0x5639ff7deb40 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 -- 172.21.15.77:0/3720441123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 msgr2=0x5639ff7ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9580010db0 tx=0x7f9580005ce0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 -- 172.21.15.77:0/3720441123 shutdown_connections 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7e8370 0x5639ff800200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7deb40 0x5639ff7ffcc0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 --2- 172.21.15.77:0/3720441123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 -- 172.21.15.77:0/3720441123 >> 172.21.15.77:0/3720441123 conn(0x5639ff7a6310 msgr2=0x5639ff7e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 -- 172.21.15.77:0/3720441123 shutdown_connections 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.919+0000 7f959d348740 1 -- 172.21.15.77:0/3720441123 wait complete. 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 -- start start 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 Processor -- start 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7deb40 0x5639ff7f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b2610 con 0x5639ff7e8370 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b1f40 con 0x5639ff7dd5d0 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95941af1a0 con 0x5639ff7deb40 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959d348740 10 client.0 _ll_register_callbacks cb 0x7f958000fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46532/0 (socket says 172.21.15.77:46532) 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 -- 172.21.15.77:0/253373892 learned_addr learned my addr 172.21.15.77:0/253373892 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f9593fff640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7deb40 0x5639ff7f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f9598d0b640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 -- 172.21.15.77:0/253373892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7deb40 msgr2=0x5639ff7f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7deb40 0x5639ff7f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 -- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 msgr2=0x5639ff7f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f959950c640 1 -- 172.21.15.77:0/253373892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ff7edc20 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.920+0000 7f9598d0b640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959950c640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f958c0cdb30 tx=0x7f958c09bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa3d129b966cc80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0b4020 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f958c0bc0b0 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f958001b5c0 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f958c01f040 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 10 client.20229 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5639ff8637e0 con 0x5639ff7e8370 2024-09-15T22:32:38.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958001b5c0 con 0x5639ff7e8370 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f958c00f040 con 0x5639ff7e8370 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 10 client.20229 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 10 client.20229 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 20 client.20229 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff865f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 4 client.20229 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.921+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639ff74ad70 con 0x5639ff7f7fd0 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9598d0b640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff865f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9573fff640 10 client.20229 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9598d0b640 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff865f10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3234bc3aff9150da server_cookie=fd94147413495d6c in_seq=0 out_seq=0 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9573fff640 1 -- 172.21.15.77:0/253373892 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639ff74ad70 con 0x5639ff7f7fd0 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9573fff640 10 client.20229 handle_command_reply: tid=0 2024-09-15T22:32:38.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 client.20229 shutdown 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 msgr2=0x5639ff7ed3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f958c0cdb30 tx=0x7f958c09bb50 comp rx=0 tx=0).stop 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 shutdown_connections 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5639ff7f7fd0 0x5639ff865f10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639ff7e8370 0x5639ff7ed3f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5639ff7deb40 0x5639ff7f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f9573fff640 0 client.20229 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 --2- 172.21.15.77:0/253373892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639ff7dd5d0 0x5639ff7f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.922+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 >> 172.21.15.77:0/253373892 conn(0x5639ff7a6310 msgr2=0x5639ff7d84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 shutdown_connections 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959d348740 1 -- 172.21.15.77:0/253373892 wait complete. 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959d348740 20 client.20229 trim_cache size 0 max 16384 2024-09-15T22:32:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 msgr2=0x7f95840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 0x7f95840a5870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f95940fdec0 tx=0x7f958000f3c0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 msgr2=0x7f959419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f958c00eaa0 tx=0x7f958c00eea0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 shutdown_connections 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f95840a33b0 0x7f95840a5870 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9594106f10 0x7f959419eee0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594102790 0x7f959419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 --2- 172.21.15.77:0/1192762380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9594101de0 0x7f959419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 >> 172.21.15.77:0/1192762380 conn(0x7f95940fd8c0 msgr2=0x7f95940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 shutdown_connections 2024-09-15T22:32:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:38.923+0000 7f959af6f640 1 -- 172.21.15.77:0/1192762380 wait complete. 2024-09-15T22:32:39.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.12 MB/sec execute 280 sec latency 8063.607 ms 2024-09-15T22:32:39.663 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:32:39.909 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:40.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:40 smithi077 ceph-mon[30980]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-15T22:32:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 -- 172.21.15.77:0/3174931284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 msgr2=0x7fdc74106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3174931284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc74106e60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009cd0 tx=0x7fdc6c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 -- 172.21.15.77:0/3174931284 shutdown_connections 2024-09-15T22:32:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3174931284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74101420 0x7fdc7410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3174931284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74100a80 0x7fdc74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3174931284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc74106e60 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 -- 172.21.15.77:0/3174931284 >> 172.21.15.77:0/3174931284 conn(0x7fdc740fc7a0 msgr2=0x7fdc740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 -- 172.21.15.77:0/3174931284 shutdown_connections 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.437+0000 7fdc7a125640 1 -- 172.21.15.77:0/3174931284 wait complete. 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 Processor -- start 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 -- start start 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74100a80 0x7fdc740fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74101420 0x7fdc740fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74110ec0 con 0x7fdc74106a80 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74110d40 con 0x7fdc74101420 2024-09-15T22:32:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc7a125640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74111040 con 0x7fdc74100a80 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.438+0000 7fdc73fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47592/0 (socket says 172.21.15.77:47592) 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 -- 172.21.15.77:0/3279410639 learned_addr learned my addr 172.21.15.77:0/3279410639 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc737fe640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74100a80 0x7fdc740fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74101420 0x7fdc740fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 -- 172.21.15.77:0/3279410639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74100a80 msgr2=0x7fdc740fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74100a80 0x7fdc740fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 -- 172.21.15.77:0/3279410639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74101420 msgr2=0x7fdc740fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74101420 0x7fdc740fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 -- 172.21.15.77:0/3279410639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc741a6920 con 0x7fdc74106a80 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc73fff640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400d750 tx=0x7fdc6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc70ff9640 1 -- 172.21.15.77:0/3279410639 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64017070 con 0x7fdc74106a80 2024-09-15T22:32:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc70ff9640 1 -- 172.21.15.77:0/3279410639 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc64005500 con 0x7fdc74106a80 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc741a6c10 con 0x7fdc74106a80 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc70ff9640 1 -- 172.21.15.77:0/3279410639 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64004d90 con 0x7fdc74106a80 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.439+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc741a7150 con 0x7fdc74106a80 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc70ff9640 1 -- 172.21.15.77:0/3279410639 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdc64004f30 con 0x7fdc74106a80 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 Processor -- start 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 -- start start 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d57e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b1d4e42f0 con 0x560b1d589370 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b1d48bcf0 con 0x560b1d57fb40 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.442+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b1d4e02b0 con 0x560b1d57e5d0 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47602/0 (socket says 172.21.15.77:47602) 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 -- 172.21.15.77:0/1768795502 learned_addr learned my addr 172.21.15.77:0/1768795502 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d57e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc737fe640 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc70ff9640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 0x7fdc600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 -- 172.21.15.77:0/1768795502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 msgr2=0x560b1d57ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 -- 172.21.15.77:0/1768795502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 msgr2=0x560b1d57e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d57e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 -- 172.21.15.77:0/1768795502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1d4703a0 con 0x560b1d589370 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc73fff640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 0x7fdc600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc70ff9640 1 -- 172.21.15.77:0/3279410639 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdc640cfb30 con 0x7fdc74106a80 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00ae00 tx=0x7fdc5c00a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42920f6ffc5a892 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc69ffb640 1 -- 172.21.15.77:0/1768795502 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01b070 con 0x560b1d589370 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc69ffb640 1 -- 172.21.15.77:0/1768795502 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c00fd70 con 0x560b1d589370 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.443+0000 7fdc69ffb640 1 -- 172.21.15.77:0/1768795502 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0146e0 con 0x560b1d589370 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1768795502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 msgr2=0x560b1d58b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00ae00 tx=0x7fdc5c00a490 comp rx=0 tx=0).stop 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc73fff640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 0x7fdc600a5870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdc640cd4f0 tx=0x7fdc64083040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1768795502 shutdown_connections 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d589370 0x560b1d58b760 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1768795502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d57e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1768795502 >> 172.21.15.77:0/1768795502 conn(0x560b1d547310 msgr2=0x560b1d57ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1768795502 shutdown_connections 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1768795502 wait complete. 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.444+0000 7fdc7c4fe740 1 -- start start 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 Processor -- start 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d588d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d586640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b1d58c070 con 0x560b1d57fb40 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b1d49f060 con 0x560b1d589370 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560b1d58c1f0 con 0x560b1d57e5d0 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc7c4fe740 10 client.0 _ll_register_callbacks cb 0x560b1d1f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:40.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47616/0 (socket says 172.21.15.77:47616) 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 -- 172.21.15.77:0/582465504 learned_addr learned my addr 172.21.15.77:0/582465504 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d586640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d588d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 -- 172.21.15.77:0/582465504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 msgr2=0x560b1d588d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d588d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 -- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 msgr2=0x560b1d586640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d586640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 -- 172.21.15.77:0/582465504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1d2553c0 con 0x560b1d57fb40 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d586640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc737fe640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c02ae00 tx=0x7fdc6c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896daecada29853 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.445+0000 7fdc52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c048060 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdc6c003970 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c00fd70 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc6c04f050 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 10 client.20241 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b1d597b70 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c00fd70 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdc6c042070 con 0x560b1d57fb40 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 10 client.20241 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 10 client.20241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 20 client.20241 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d599030 0x560b1d59b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 4 client.20241 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.446+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b1d4f1860 con 0x560b1d599030 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.447+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d599030 0x560b1d59b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.447+0000 7fdc52ffd640 10 client.20241 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.447+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d599030 0x560b1d59b470 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ec5baab13cc5bd server_cookie=f5415d89d7c67599 in_seq=0 out_seq=0 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc52ffd640 1 -- 172.21.15.77:0/582465504 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b1d4f1860 con 0x560b1d599030 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc52ffd640 10 client.20241 handle_command_reply: tid=0 2024-09-15T22:32:40.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 client.20241 shutdown 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 msgr2=0x560b1d586030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c02ae00 tx=0x7fdc6c031af0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 shutdown_connections 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d599030 0x560b1d59b470 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d586640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d586030 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc52ffd640 0 client.20241 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/582465504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d588d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 >> 172.21.15.77:0/582465504 conn(0x560b1d547310 msgr2=0x560b1d580960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 shutdown_connections 2024-09-15T22:32:40.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/582465504 wait complete. 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.448+0000 7fdc7c4fe740 20 client.20241 trim_cache size 0 max 16384 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 Processor -- start 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 -- start start 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d585050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b11c0 con 0x560b1d57fb40 2024-09-15T22:32:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b2d00 con 0x560b1d589370 2024-09-15T22:32:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b1890 con 0x560b1d57e5d0 2024-09-15T22:32:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47632/0 (socket says 172.21.15.77:47632) 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc737fe640 1 -- 172.21.15.77:0/1037948830 learned_addr learned my addr 172.21.15.77:0/1037948830 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.461+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d585050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 -- 172.21.15.77:0/1037948830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 msgr2=0x560b1d57ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 -- 172.21.15.77:0/1037948830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 msgr2=0x560b1d585050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d585050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 -- 172.21.15.77:0/1037948830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1d586fd0 con 0x560b1d57fb40 2024-09-15T22:32:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d585050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc737fe640 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c002230 tx=0x7fdc6c030910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb850c8949ffc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc697fa640 1 -- 172.21.15.77:0/1037948830 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c048040 con 0x560b1d57fb40 2024-09-15T22:32:40.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc697fa640 1 -- 172.21.15.77:0/1037948830 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c036040 con 0x560b1d57fb40 2024-09-15T22:32:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc697fa640 1 -- 172.21.15.77:0/1037948830 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0437f0 con 0x560b1d57fb40 2024-09-15T22:32:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1037948830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 msgr2=0x560b1d595340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c002230 tx=0x7fdc6c030910 comp rx=0 tx=0).stop 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1037948830 shutdown_connections 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d589370 0x560b1d585050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57fb40 0x560b1d595340 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.462+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/1037948830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d57e5d0 0x560b1d57ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1037948830 >> 172.21.15.77:0/1037948830 conn(0x560b1d547310 msgr2=0x560b1d584940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1037948830 shutdown_connections 2024-09-15T22:32:40.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/1037948830 wait complete. 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 -- start start 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 Processor -- start 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.463+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d591d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d589370 0x560b1d582c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b2630 con 0x560b1d57e5d0 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741b1f60 con 0x560b1d57fb40 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc741af1c0 con 0x560b1d589370 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc7c4fe740 10 client.0 _ll_register_callbacks cb 0x7fdc6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47636/0 (socket says 172.21.15.77:47636) 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 -- 172.21.15.77:0/2472112526 learned_addr learned my addr 172.21.15.77:0/2472112526 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:40.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d589370 0x560b1d582c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc737fe640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d591d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 -- 172.21.15.77:0/2472112526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d589370 msgr2=0x560b1d582c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d589370 0x560b1d582c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 -- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 msgr2=0x560b1d591d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.464+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d591d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc72ffd640 1 -- 172.21.15.77:0/2472112526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1d5833d0 con 0x560b1d57e5d0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc72ffd640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c016e50 tx=0x7fdc5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c9a1330dc85ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01b040 con 0x560b1d57e5d0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdc5c016620 con 0x560b1d57e5d0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6c036040 con 0x560b1d57e5d0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c00a230 con 0x560b1d57e5d0 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc7c4fe740 10 client.20247 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:40.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b1d5fc8c0 con 0x560b1d57e5d0 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c036040 con 0x560b1d57e5d0 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdc5c015040 con 0x560b1d57e5d0 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc7c4fe740 10 client.20247 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.465+0000 7fdc7c4fe740 10 client.20247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc7c4fe740 20 client.20247 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d5fcd10 0x560b1d5ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc7c4fe740 4 client.20247 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b1d4ebd70 con 0x560b1d5fcd10 2024-09-15T22:32:40.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc737fe640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d5fcd10 0x560b1d5ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:40.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc737fe640 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d5fcd10 0x560b1d5ff100 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea85bbf039fb07f server_cookie=ce3493b002c924e1 in_seq=0 out_seq=0 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc52ffd640 10 client.20247 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc52ffd640 1 -- 172.21.15.77:0/2472112526 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b1d4ebd70 con 0x560b1d5fcd10 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc52ffd640 10 client.20247 handle_command_reply: tid=0 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.466+0000 7fdc7c4fe740 1 client.20247 shutdown 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 msgr2=0x560b1d591820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c016e50 tx=0x7fdc5c008040 comp rx=0 tx=0).stop 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 shutdown_connections 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560b1d5fcd10 0x560b1d5ff100 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560b1d589370 0x560b1d582c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b1d57fb40 0x560b1d591d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc52ffd640 0 client.20247 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 --2- 172.21.15.77:0/2472112526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b1d57e5d0 0x560b1d591820 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 >> 172.21.15.77:0/2472112526 conn(0x560b1d547310 msgr2=0x560b1d580110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 shutdown_connections 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 1 -- 172.21.15.77:0/2472112526 wait complete. 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7c4fe740 20 client.20247 trim_cache size 0 max 16384 2024-09-15T22:32:40.490 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 msgr2=0x7fdc600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 0x7fdc600a5870 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdc640cd4f0 tx=0x7fdc64083040 comp rx=0 tx=0).stop 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 msgr2=0x7fdc740fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400d750 tx=0x7fdc6400dc10 comp rx=0 tx=0).stop 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 shutdown_connections 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdc600a33b0 0x7fdc600a5870 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc74106a80 0x7fdc740fd8f0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74101420 0x7fdc740fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.467+0000 7fdc7a125640 1 --2- 172.21.15.77:0/3279410639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc74100a80 0x7fdc740fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:40.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.468+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 >> 172.21.15.77:0/3279410639 conn(0x7fdc740fc7a0 msgr2=0x7fdc7406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:40.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.468+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 shutdown_connections 2024-09-15T22:32:40.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:40.468+0000 7fdc7a125640 1 -- 172.21.15.77:0/3279410639 wait complete. 2024-09-15T22:32:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:40 smithi196 ceph-mon[35109]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-15T22:32:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:40 smithi117 ceph-mon[35021]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 224 op/s 2024-09-15T22:32:40.612 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.07 MB/sec execute 281 sec latency 9063.675 ms 2024-09-15T22:32:41.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 15.02 MB/sec execute 282 sec latency 10063.765 ms 2024-09-15T22:32:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:42 smithi077 ceph-mon[30980]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-15T22:32:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:42 smithi196 ceph-mon[35109]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-15T22:32:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:42 smithi117 ceph-mon[35021]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.4 MiB/s wr, 231 op/s 2024-09-15T22:32:42.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.96 MB/sec execute 283 sec latency 11063.864 ms 2024-09-15T22:32:42.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:32:42.992 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:32:43.244 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:43.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.91 MB/sec execute 284 sec latency 12063.955 ms 2024-09-15T22:32:43.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 -- 172.21.15.77:0/695238188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 msgr2=0x7f8e6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 --2- 172.21.15.77:0/695238188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c106e60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009930 tx=0x7f8e5c02a420 comp rx=0 tx=0).stop 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 -- 172.21.15.77:0/695238188 shutdown_connections 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 --2- 172.21.15.77:0/695238188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 0x7f8e6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 --2- 172.21.15.77:0/695238188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 0x7f8e6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 --2- 172.21.15.77:0/695238188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c106e60 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 -- 172.21.15.77:0/695238188 >> 172.21.15.77:0/695238188 conn(0x7f8e6c0fc7a0 msgr2=0x7f8e6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 -- 172.21.15.77:0/695238188 shutdown_connections 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.700+0000 7f8e7226e640 1 -- 172.21.15.77:0/695238188 wait complete. 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 Processor -- start 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 -- start start 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 0x7f8e6c102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 0x7f8e6c102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c110ea0 con 0x7f8e6c106a80 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c110d20 con 0x7f8e6c100a80 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7226e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c111020 con 0x7f8e6c101420 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 0x7f8e6c102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47652/0 (socket says 172.21.15.77:47652) 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 -- 172.21.15.77:0/3199703107 learned_addr learned my addr 172.21.15.77:0/3199703107 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 0x7f8e6c102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 -- 172.21.15.77:0/3199703107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 msgr2=0x7f8e6c102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 0x7f8e6c102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 -- 172.21.15.77:0/3199703107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 msgr2=0x7f8e6c102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 0x7f8e6c102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.701+0000 7f8e7080b640 1 -- 172.21.15.77:0/3199703107 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1a6940 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e7080b640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b970 tx=0x7f8e6400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e697fa640 1 -- 172.21.15.77:0/3199703107 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64019070 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e697fa640 1 -- 172.21.15.77:0/3199703107 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6400cb40 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1a6c30 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e697fa640 1 -- 172.21.15.77:0/3199703107 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64007650 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.702+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1a71c0 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.704+0000 7f8e697fa640 1 -- 172.21.15.77:0/3199703107 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8e6401c070 con 0x7f8e6c106a80 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.704+0000 7f8e74647740 1 Processor -- start 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.704+0000 7f8e74647740 1 -- start start 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde604800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde604d40 0x55dfde607130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dfde55f280 con 0x55dfde5f9b60 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dfde506b10 con 0x55dfde5fa480 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e74647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dfde55b240 con 0x55dfde604d40 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e697fa640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 0x7f8e540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47656/0 (socket says 172.21.15.77:47656) 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/295664354 learned_addr learned my addr 172.21.15.77:0/295664354 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e7080b640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 0x7f8e540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde604800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde604d40 0x55dfde607130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e697fa640 1 -- 172.21.15.77:0/3199703107 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f8e640cf4b0 con 0x7f8e6c106a80 2024-09-15T22:32:43.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/295664354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde604d40 msgr2=0x55dfde607130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde604d40 0x55dfde607130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/295664354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 msgr2=0x55dfde604800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde604800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/295664354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfde5131f0 con 0x55dfde5f9b60 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde604800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8e600058c0 tx=0x7f8e6000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c5ef18e292a426 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.705+0000 7f8e7080b640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 0x7f8e540a58f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e640cce70 tx=0x7f8e64004b50 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e527fc640 1 -- 172.21.15.77:0/295664354 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60019030 con 0x55dfde5f9b60 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e527fc640 1 -- 172.21.15.77:0/295664354 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e600119a0 con 0x55dfde5f9b60 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e527fc640 1 -- 172.21.15.77:0/295664354 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6001d880 con 0x55dfde5f9b60 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 -- 172.21.15.77:0/295664354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 msgr2=0x55dfde5f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8e600058c0 tx=0x7f8e6000b6b0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 -- 172.21.15.77:0/295664354 shutdown_connections 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde604d40 0x55dfde607130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde604800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 --2- 172.21.15.77:0/295664354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5f9b60 0x55dfde5f9f40 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 -- 172.21.15.77:0/295664354 >> 172.21.15.77:0/295664354 conn(0x55dfde5f60f0 msgr2=0x55dfde5f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 -- 172.21.15.77:0/295664354 shutdown_connections 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.706+0000 7f8e74647740 1 -- 172.21.15.77:0/295664354 wait complete. 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 -- start start 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 Processor -- start 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde601c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde602680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dfde607910 con 0x55dfde5fa480 2024-09-15T22:32:43.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dfde519f10 con 0x55dfde604d40 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dfde607a90 con 0x55dfde5f9b60 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e74647740 10 client.0 _ll_register_callbacks cb 0x55dfde4e2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47668/0 (socket says 172.21.15.77:47668) 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e6bfff640 1 -- 172.21.15.77:0/1953572617 learned_addr learned my addr 172.21.15.77:0/1953572617 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.707+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde601c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde602680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 -- 172.21.15.77:0/1953572617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 msgr2=0x55dfde601c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde601c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 -- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 msgr2=0x55dfde602680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde602680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 -- 172.21.15.77:0/1953572617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfde2d06a0 con 0x55dfde5fa480 2024-09-15T22:32:43.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde602680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009930 tx=0x7f8e5c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b10644d3639058 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c035040 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f8e5c004240 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e600119a0 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e5c04e050 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 10 client.20259 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dfde6134f0 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e600119a0 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8e5c004ba0 con 0x55dfde5fa480 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 10 client.20259 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 10 client.20259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 20 client.20259 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:43.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde6149b0 0x55dfde616df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 4 client.20259 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.708+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfde56c7f0 con 0x55dfde6149b0 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.709+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde6149b0 0x55dfde616df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.709+0000 7f8e4b7fe640 10 client.20259 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.709+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde6149b0 0x55dfde616df0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64ceee2707eb357 server_cookie=f00124cdf62dd5de in_seq=0 out_seq=0 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/1953572617 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfde56c7f0 con 0x55dfde6149b0 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e4b7fe640 10 client.20259 handle_command_reply: tid=0 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 client.20259 shutdown 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 msgr2=0x55dfde602140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009930 tx=0x7f8e5c0317e0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 shutdown_connections 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde6149b0 0x55dfde616df0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde602680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde602140 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e4b7fe640 0 client.20259 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 --2- 172.21.15.77:0/1953572617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde601c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 >> 172.21.15.77:0/1953572617 conn(0x55dfde5f60f0 msgr2=0x55dfde5fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 shutdown_connections 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 1 -- 172.21.15.77:0/1953572617 wait complete. 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.710+0000 7f8e74647740 20 client.20259 trim_cache size 0 max 16384 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 Processor -- start 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 -- start start 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde6024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde600ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1b1270 con 0x55dfde604d40 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1b2db0 con 0x55dfde5fa480 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e74647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1b1940 con 0x55dfde5f9b60 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47682/0 (socket says 172.21.15.77:47682) 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.727+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/404733250 learned_addr learned my addr 172.21.15.77:0/404733250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde600ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde6024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/404733250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 msgr2=0x55dfde6024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde6024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/404733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 msgr2=0x55dfde600ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde600ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 -- 172.21.15.77:0/404733250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfde6041e0 con 0x55dfde604d40 2024-09-15T22:32:43.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8e6001d560 tx=0x7f8e6001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b695ec057f242ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e51ffb640 1 -- 172.21.15.77:0/404733250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60019040 con 0x55dfde604d40 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e51ffb640 1 -- 172.21.15.77:0/404733250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e600114c0 con 0x55dfde604d40 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e51ffb640 1 -- 172.21.15.77:0/404733250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600052a0 con 0x55dfde604d40 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 -- 172.21.15.77:0/404733250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 msgr2=0x55dfde61c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8e6001d560 tx=0x7f8e6001d3c0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 -- 172.21.15.77:0/404733250 shutdown_connections 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde604d40 0x55dfde61c6c0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde5fa480 0x55dfde600ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 --2- 172.21.15.77:0/404733250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde6024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.728+0000 7f8e74647740 1 -- 172.21.15.77:0/404733250 >> 172.21.15.77:0/404733250 conn(0x55dfde5f60f0 msgr2=0x55dfde6003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- 172.21.15.77:0/404733250 shutdown_connections 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- 172.21.15.77:0/404733250 wait complete. 2024-09-15T22:32:43.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- start start 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 Processor -- start 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde5fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde5ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1b26e0 con 0x55dfde5fa480 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1b2010 con 0x55dfde604d40 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1af270 con 0x55dfde5f9b60 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e74647740 10 client.0 _ll_register_callbacks cb 0x7f8e5c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47698/0 (socket says 172.21.15.77:47698) 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.729+0000 7f8e6bfff640 1 -- 172.21.15.77:0/457928791 learned_addr learned my addr 172.21.15.77:0/457928791 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde5ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.751 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 -- 172.21.15.77:0/457928791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 msgr2=0x55dfde5fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde5fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde5fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 -- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 msgr2=0x55dfde5ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde5ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 -- 172.21.15.77:0/457928791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfde67e790 con 0x55dfde5fa480 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde5ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e6bfff640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c001eb0 tx=0x7f8e5c005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bc9e22d7905b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c056070 con 0x55dfde5fa480 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f8e5c004770 con 0x55dfde5fa480 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e600114c0 con 0x55dfde5fa480 2024-09-15T22:32:43.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8e5c062050 con 0x55dfde5fa480 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e74647740 10 client.20265 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dfde67fab0 con 0x55dfde5fa480 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e600114c0 con 0x55dfde5fa480 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8e5c0458d0 con 0x55dfde5fa480 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e74647740 10 client.20265 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.730+0000 7f8e74647740 10 client.20265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e74647740 20 client.20265 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde680200 0x55dfde682620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e74647740 4 client.20265 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dfde566d00 con 0x55dfde680200 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde680200 0x55dfde682620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e6b7fe640 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde680200 0x55dfde682620 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecb1b52d5b2c6d3 server_cookie=4e40129fe80e991d in_seq=0 out_seq=0 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.731+0000 7f8e4b7fe640 10 client.20265 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.732+0000 7f8e4b7fe640 1 -- 172.21.15.77:0/457928791 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dfde566d00 con 0x55dfde680200 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.732+0000 7f8e4b7fe640 10 client.20265 handle_command_reply: tid=0 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.732+0000 7f8e74647740 1 client.20265 shutdown 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.732+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 msgr2=0x55dfde5ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.732+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c001eb0 tx=0x7f8e5c005200 comp rx=0 tx=0).stop 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 shutdown_connections 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dfde680200 0x55dfde682620 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfde604d40 0x55dfde5ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dfde5fa480 0x55dfde5ff240 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 --2- 172.21.15.77:0/457928791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dfde5f9b60 0x55dfde5fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e4b7fe640 0 client.20265 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:43.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 >> 172.21.15.77:0/457928791 conn(0x55dfde5f60f0 msgr2=0x55dfde5f5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 shutdown_connections 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 1 -- 172.21.15.77:0/457928791 wait complete. 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "19794e35-0e44-47aa-b18b-3fb6310162ac", 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:32:43.755 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e74647740 20 client.20265 trim_cache size 0 max 16384 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 msgr2=0x7f8e540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 0x7f8e540a58f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e640cce70 tx=0x7f8e64004b50 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 msgr2=0x7f8e6c103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b970 tx=0x7f8e6400be30 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 shutdown_connections 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8e540a3430 0x7f8e540a58f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e6c106a80 0x7f8e6c103520 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6c101420 0x7f8e6c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 --2- 172.21.15.77:0/3199703107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a80 0x7f8e6c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 >> 172.21.15.77:0/3199703107 conn(0x7f8e6c0fc7a0 msgr2=0x7f8e6c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 shutdown_connections 2024-09-15T22:32:43.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:43.733+0000 7f8e7226e640 1 -- 172.21.15.77:0/3199703107 wait complete. 2024-09-15T22:32:44.577 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:44.578 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:44 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:44.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:44 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.606 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:32:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-15T22:32:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:32:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:44.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:44 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:44.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.86 MB/sec execute 285 sec latency 13063.995 ms 2024-09-15T22:32:44.851 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 -- 172.21.15.77:0/245460690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c10c180 msgr2=0x7ffa3c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 --2- 172.21.15.77:0/245460690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c10c180 0x7ffa3c10e600 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009ca0 tx=0x7ffa3402a950 comp rx=0 tx=0).stop 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 -- 172.21.15.77:0/245460690 shutdown_connections 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 --2- 172.21.15.77:0/245460690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c10c180 0x7ffa3c10e600 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 --2- 172.21.15.77:0/245460690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 0x7ffa3c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 --2- 172.21.15.77:0/245460690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c102da0 0x7ffa3c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 -- 172.21.15.77:0/245460690 >> 172.21.15.77:0/245460690 conn(0x7ffa3c0fc7a0 msgr2=0x7ffa3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 -- 172.21.15.77:0/245460690 shutdown_connections 2024-09-15T22:32:45.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 -- 172.21.15.77:0/245460690 wait complete. 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.386+0000 7ffa4102c640 1 Processor -- start 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 -- start start 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 0x7ffa3c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c10c180 0x7ffa3c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111f20 con 0x7ffa3c102da0 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111da0 con 0x7ffa3c10c180 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa4102c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1120a0 con 0x7ffa3c1036c0 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47716/0 (socket says 172.21.15.77:47716) 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 -- 172.21.15.77:0/3697043158 learned_addr learned my addr 172.21.15.77:0/3697043158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c10c180 0x7ffa3c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 0x7ffa3c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 -- 172.21.15.77:0/3697043158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 msgr2=0x7ffa3c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 0x7ffa3c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 -- 172.21.15.77:0/3697043158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c10c180 msgr2=0x7ffa3c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c10c180 0x7ffa3c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.387+0000 7ffa3ad9d640 1 -- 172.21.15.77:0/3697043158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c19d250 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c920 tx=0x7ffa2c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa33fff640 1 -- 172.21.15.77:0/3697043158 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c007c70 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c19d540 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa33fff640 1 -- 172.21.15.77:0/3697043158 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2c007e10 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa33fff640 1 -- 172.21.15.77:0/3697043158 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c018890 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.388+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a4e20 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.390+0000 7ffa33fff640 1 -- 172.21.15.77:0/3697043158 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa2c018a70 con 0x7ffa3c102da0 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.390+0000 7ffa43405740 1 Processor -- start 2024-09-15T22:32:45.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.390+0000 7ffa43405740 1 -- start start 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630b3caa2e0 con 0x5630b3d4f680 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630b3c51ce0 con 0x5630b3d44dc0 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa43405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5630b3ca62a0 con 0x5630b3d444a0 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47718/0 (socket says 172.21.15.77:47718) 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 -- 172.21.15.77:0/3576207400 learned_addr learned my addr 172.21.15.77:0/3576207400 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa33fff640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 0x7ffa280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 -- 172.21.15.77:0/3576207400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 msgr2=0x5630b3d44880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d44880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 -- 172.21.15.77:0/3576207400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 msgr2=0x5630b3d4f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d4f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.391+0000 7ffa3a59c640 1 -- 172.21.15.77:0/3576207400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b3c36440 con 0x5630b3d4f680 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 0x7ffa280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa33fff640 1 -- 172.21.15.77:0/3697043158 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7ffa2c0d0260 con 0x7ffa3c102da0 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400cbb0 tx=0x7ffa2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2ecc5a841ac4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa317fa640 1 -- 172.21.15.77:0/3576207400 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24005220 con 0x5630b3d4f680 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa317fa640 1 -- 172.21.15.77:0/3576207400 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004c80 con 0x5630b3d4f680 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.392+0000 7ffa317fa640 1 -- 172.21.15.77:0/3576207400 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24010740 con 0x5630b3d4f680 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- 172.21.15.77:0/3576207400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 msgr2=0x5630b3d51a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 0x7ffa280a5870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009a60 tx=0x7ffa340095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400cbb0 tx=0x7ffa2400e720 comp rx=0 tx=0).stop 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- 172.21.15.77:0/3576207400 shutdown_connections 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d4f680 0x5630b3d51a70 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d4f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- 172.21.15.77:0/3576207400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d44880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- 172.21.15.77:0/3576207400 >> 172.21.15.77:0/3576207400 conn(0x5630b3d40a30 msgr2=0x5630b3d40e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- 172.21.15.77:0/3576207400 shutdown_connections 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- 172.21.15.77:0/3576207400 wait complete. 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- start start 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 Processor -- start 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d4eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 0x5630b3d4c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5630b3d52250 con 0x5630b3d44dc0 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630b3c65050 con 0x5630b3d4f680 2024-09-15T22:32:45.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5630b3d523d0 con 0x5630b3d444a0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa43405740 10 client.0 _ll_register_callbacks cb 0x5630b3ca5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.393+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47722/0 (socket says 172.21.15.77:47722) 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 -- 172.21.15.77:0/814287656 learned_addr learned my addr 172.21.15.77:0/814287656 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d4eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 0x5630b3d4c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 -- 172.21.15.77:0/814287656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 msgr2=0x5630b3d4eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d4eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 -- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 msgr2=0x5630b3d4c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 0x5630b3d4c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 -- 172.21.15.77:0/814287656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b3a1b3c0 con 0x5630b3d44dc0 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d4eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:45.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 0x5630b3d4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005700 tx=0x7ffa2400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a807e4a49c6b900 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa227fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24029060 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa24004c80 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7ffa24005210 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa24030050 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa43405740 10 client.20277 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.394+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5630b3d5db40 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004c80 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffa24003c50 con 0x5630b3d44dc0 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 10 client.20277 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 10 client.20277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:45.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 20 client.20277 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d5f000 0x5630b3d613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 4 client.20277 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b3cb7850 con 0x5630b3d5f000 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d5f000 0x5630b3d613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa227fc640 10 client.20277 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.395+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d5f000 0x5630b3d613f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34cd11693330be75 server_cookie=edc2fa8a0094cfe9 in_seq=0 out_seq=0 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa227fc640 1 -- 172.21.15.77:0/814287656 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630b3cb7850 con 0x5630b3d5f000 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa227fc640 10 client.20277 handle_command_reply: tid=0 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 client.20277 shutdown 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 msgr2=0x5630b3d4c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005700 tx=0x7ffa2400c920 comp rx=0 tx=0).stop 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 shutdown_connections 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d5f000 0x5630b3d613f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d4f680 0x5630b3d4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d44dc0 0x5630b3d4c190 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 --2- 172.21.15.77:0/814287656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d444a0 0x5630b3d4eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 >> 172.21.15.77:0/814287656 conn(0x5630b3d40a30 msgr2=0x5630b3d46710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.396+0000 7ffa227fc640 0 client.20277 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.397+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 shutdown_connections 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.397+0000 7ffa43405740 1 -- 172.21.15.77:0/814287656 wait complete. 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.397+0000 7ffa43405740 20 client.20277 trim_cache size 0 max 16384 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.413+0000 7ffa43405740 1 Processor -- start 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.413+0000 7ffa43405740 1 -- start start 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d40a30 0x5630b3d4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d44dc0 0x5630b3d66a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1aeea0 con 0x5630b3d444a0 2024-09-15T22:32:45.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1b09e0 con 0x5630b3d40a30 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa43405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1af570 con 0x5630b3d44dc0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47738/0 (socket says 172.21.15.77:47738) 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 -- 172.21.15.77:0/1458843276 learned_addr learned my addr 172.21.15.77:0/1458843276 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d40a30 0x5630b3d4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d44dc0 0x5630b3d66a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 -- 172.21.15.77:0/1458843276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d44dc0 msgr2=0x5630b3d66a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d44dc0 0x5630b3d66a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 -- 172.21.15.77:0/1458843276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d40a30 msgr2=0x5630b3d4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d40a30 0x5630b3d4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 -- 172.21.15.77:0/1458843276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b3d4f250 con 0x5630b3d444a0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400f010 tx=0x7ffa24010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d247b3ecf9cfb1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa30ff9640 1 -- 172.21.15.77:0/1458843276 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004c30 con 0x5630b3d444a0 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.414+0000 7ffa30ff9640 1 -- 172.21.15.77:0/1458843276 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004dd0 con 0x5630b3d444a0 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa30ff9640 1 -- 172.21.15.77:0/1458843276 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24025450 con 0x5630b3d444a0 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- 172.21.15.77:0/1458843276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 msgr2=0x5630b3d477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400f010 tx=0x7ffa24010890 comp rx=0 tx=0).stop 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- 172.21.15.77:0/1458843276 shutdown_connections 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d44dc0 0x5630b3d66a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d477f0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 --2- 172.21.15.77:0/1458843276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d40a30 0x5630b3d4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- 172.21.15.77:0/1458843276 >> 172.21.15.77:0/1458843276 conn(0x5630b3d4c830 msgr2=0x5630b3d470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- 172.21.15.77:0/1458843276 shutdown_connections 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- 172.21.15.77:0/1458843276 wait complete. 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.415+0000 7ffa43405740 1 -- start start 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 Processor -- start 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 0x5630b3d57e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1b0310 con 0x5630b3d444a0 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1afc40 con 0x5630b3d44dc0 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1acea0 con 0x5630b3d40a30 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa43405740 10 client.0 _ll_register_callbacks cb 0x7ffa24014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47742/0 (socket says 172.21.15.77:47742) 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 -- 172.21.15.77:0/2277204231 learned_addr learned my addr 172.21.15.77:0/2277204231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3b59e640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 0x5630b3d57e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 -- 172.21.15.77:0/2277204231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 msgr2=0x5630b3d57e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 0x5630b3d57e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 -- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 msgr2=0x5630b3d470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 -- 172.21.15.77:0/2277204231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b3d478d0 con 0x5630b3d444a0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 0x5630b3d57e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa3a59c640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ffa24010470 tx=0x7ffa24010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab27dae4dec733b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa227fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.416+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24042060 con 0x5630b3d444a0 2024-09-15T22:32:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7ffa24021040 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa24004dd0 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa24049050 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 10 client.20283 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5630b3dc9ae0 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa24004dd0 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffa2403d870 con 0x5630b3d444a0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 10 client.20283 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 10 client.20283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 20 client.20283 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d4f680 0x5630b3dcc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 4 client.20283 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630b3cb1d60 con 0x5630b3d4f680 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.417+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d4f680 0x5630b3dcc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa227fc640 10 client.20283 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa3ad9d640 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d4f680 0x5630b3dcc0d0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb18fbda5e1c69c4 server_cookie=2adab762cf11193a in_seq=0 out_seq=0 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa227fc640 1 -- 172.21.15.77:0/2277204231 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630b3cb1d60 con 0x5630b3d4f680 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa227fc640 10 client.20283 handle_command_reply: tid=0 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 client.20283 shutdown 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 msgr2=0x5630b3d46ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ffa24010470 tx=0x7ffa24010590 comp rx=0 tx=0).stop 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 shutdown_connections 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5630b3d4f680 0x5630b3dcc0d0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b3d44dc0 0x5630b3d470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630b3d444a0 0x5630b3d46ae0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa227fc640 0 client.20283 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 --2- 172.21.15.77:0/2277204231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5630b3d40a30 0x5630b3d57e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 >> 172.21.15.77:0/2277204231 conn(0x5630b3d4c830 msgr2=0x5630b3d4d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 shutdown_connections 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 1 -- 172.21.15.77:0/2277204231 wait complete. 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.418+0000 7ffa43405740 20 client.20283 trim_cache size 0 max 16384 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout: "19794e35-0e44-47aa-b18b-3fb6310162ac": { 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:32:45.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "19794e35-0e44-47aa-b18b-3fb6310162ac", 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 msgr2=0x7ffa280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 0x7ffa280a5870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009a60 tx=0x7ffa340095c0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 msgr2=0x7ffa3c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c920 tx=0x7ffa2c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 shutdown_connections 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa280a33b0 0x7ffa280a5870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa3c10c180 0x7ffa3c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa3c1036c0 0x7ffa3c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 --2- 172.21.15.77:0/3697043158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa3c102da0 0x7ffa3c1982a0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:45.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 >> 172.21.15.77:0/3697043158 conn(0x7ffa3c0fc7a0 msgr2=0x7ffa3c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:45.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 shutdown_connections 2024-09-15T22:32:45.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:45.419+0000 7ffa4102c640 1 -- 172.21.15.77:0/3697043158 wait complete. 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:45 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:45.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:32:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:45 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:32:45.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:45 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:32:45.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.81 MB/sec execute 286 sec latency 14064.024 ms 2024-09-15T22:32:46.125 INFO:tasks.cephfs.filesystem:scrub status for tag:19794e35-0e44-47aa-b18b-3fb6310162ac - {'path': '/', 'tag': '19794e35-0e44-47aa-b18b-3fb6310162ac', 'options': 'recursive,force'} 2024-09-15T22:32:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:46 smithi077 ceph-mon[30980]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-15T22:32:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:46 smithi196 ceph-mon[35109]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-15T22:32:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:46 smithi117 ceph-mon[35021]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-15T22:32:46.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.75 MB/sec execute 287 sec latency 15064.141 ms 2024-09-15T22:32:47.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.70 MB/sec execute 288 sec latency 16064.226 ms 2024-09-15T22:32:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:48 smithi077 ceph-mon[30980]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:32:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:48 smithi196 ceph-mon[35109]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:32:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:48 smithi117 ceph-mon[35021]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:32:48.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.65 MB/sec execute 289 sec latency 17064.316 ms 2024-09-15T22:32:49.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.60 MB/sec execute 290 sec latency 18064.415 ms 2024-09-15T22:32:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:50 smithi077 ceph-mon[30980]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 734 KiB/s wr, 65 op/s 2024-09-15T22:32:50.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:50 smithi196 ceph-mon[35109]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 734 KiB/s wr, 65 op/s 2024-09-15T22:32:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:50 smithi117 ceph-mon[35021]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 734 KiB/s wr, 65 op/s 2024-09-15T22:32:50.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.55 MB/sec execute 291 sec latency 19064.474 ms 2024-09-15T22:32:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:51.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:51 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:51.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:51 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:51 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:51.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.50 MB/sec execute 292 sec latency 20064.557 ms 2024-09-15T22:32:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:52 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:32:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:52 smithi077 ceph-mon[30980]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 769 KiB/s wr, 69 op/s 2024-09-15T22:32:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:52 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:32:52.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:52 smithi196 ceph-mon[35109]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 769 KiB/s wr, 69 op/s 2024-09-15T22:32:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:52 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:32:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:52 smithi117 ceph-mon[35021]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 769 KiB/s wr, 69 op/s 2024-09-15T22:32:52.613 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.45 MB/sec execute 293 sec latency 21064.641 ms 2024-09-15T22:32:53.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.40 MB/sec execute 294 sec latency 22064.746 ms 2024-09-15T22:32:54.496 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:32:54.524 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:54 smithi077 ceph-mon[30980]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 636 KiB/s wr, 64 op/s 2024-09-15T22:32:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:54 smithi196 ceph-mon[35109]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 636 KiB/s wr, 64 op/s 2024-09-15T22:32:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:54 smithi117 ceph-mon[35021]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 636 KiB/s wr, 64 op/s 2024-09-15T22:32:54.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.35 MB/sec execute 295 sec latency 23064.803 ms 2024-09-15T22:32:54.755 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:55.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 -- 172.21.15.77:0/2448344102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c106a80 msgr2=0x7f634c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 --2- 172.21.15.77:0/2448344102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c106a80 0x7f634c106e60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6334009930 tx=0x7f633402a420 comp rx=0 tx=0).stop 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 -- 172.21.15.77:0/2448344102 shutdown_connections 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 --2- 172.21.15.77:0/2448344102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 --2- 172.21.15.77:0/2448344102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f634c100a80 0x7f634c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 --2- 172.21.15.77:0/2448344102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c106a80 0x7f634c106e60 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 -- 172.21.15.77:0/2448344102 >> 172.21.15.77:0/2448344102 conn(0x7f634c0fc7a0 msgr2=0x7f634c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 -- 172.21.15.77:0/2448344102 shutdown_connections 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.254+0000 7f6350afb640 1 -- 172.21.15.77:0/2448344102 wait complete. 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 Processor -- start 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 -- start start 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f634c106a80 0x7f634c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f634c110e80 con 0x7f634c100a80 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f634c110d00 con 0x7f634c106a80 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6350afb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f634c111000 con 0x7f634c101420 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f634a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f634a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48502/0 (socket says 172.21.15.77:48502) 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f634a59c640 1 -- 172.21.15.77:0/1280928133 learned_addr learned my addr 172.21.15.77:0/1280928133 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.255+0000 7f6349d9b640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 -- 172.21.15.77:0/1280928133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 msgr2=0x7f634c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 -- 172.21.15.77:0/1280928133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f634c106a80 msgr2=0x7f634c19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f634c106a80 0x7f634c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 -- 172.21.15.77:0/1280928133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c19c310 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f6349d9b640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f634a59c640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6334009230 tx=0x7f6334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633403f030 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c19c5a0 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6334003ad0 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334035070 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.256+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a3ee0 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.257+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634c100ee0 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.258+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6334042030 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.259+0000 7f63437fe640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 0x7f63380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.259+0000 7f6349d9b640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 0x7f63380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.259+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7f63340f07e0 con 0x7f634c100a80 2024-09-15T22:32:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.259+0000 7f6349d9b640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 0x7f63380a5870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f633c0058c0 tx=0x7f633c005850 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.263+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63340bce30 con 0x7f634c100a80 2024-09-15T22:32:55.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.466+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f634c108ff0 con 0x7f634c100a80 2024-09-15T22:32:55.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.467+0000 7f63437fe640 1 -- 172.21.15.77:0/1280928133 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f63340bc730 con 0x7f634c100a80 2024-09-15T22:32:55.469 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:32:55.469 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:32:55.469 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 msgr2=0x7f63380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 0x7f63380a5870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f633c0058c0 tx=0x7f633c005850 comp rx=0 tx=0).stop 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 msgr2=0x7f634c197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6334009230 tx=0x7f6334004270 comp rx=0 tx=0).stop 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 shutdown_connections 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63380a33b0 0x7f63380a5870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f634c106a80 0x7f634c19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f634c101420 0x7f634c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 --2- 172.21.15.77:0/1280928133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f634c100a80 0x7f634c197360 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:55.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 >> 172.21.15.77:0/1280928133 conn(0x7f634c0fc7a0 msgr2=0x7f634c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:55.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 shutdown_connections 2024-09-15T22:32:55.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:55.469+0000 7f6350afb640 1 -- 172.21.15.77:0/1280928133 wait complete. 2024-09-15T22:32:55.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.31 MB/sec execute 296 sec latency 24064.928 ms 2024-09-15T22:32:56.177 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 3ce00f87 --release --await' 2024-09-15T22:32:56.177 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 3ce00f87 --release --await 2024-09-15T22:32:56.502 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:56 smithi077 ceph-mon[30980]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 411 KiB/s wr, 58 op/s 2024-09-15T22:32:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1280928133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:56 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:56 smithi196 ceph-mon[35109]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 411 KiB/s wr, 58 op/s 2024-09-15T22:32:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:56 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1280928133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:56 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:56 smithi117 ceph-mon[35021]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 411 KiB/s wr, 58 op/s 2024-09-15T22:32:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:56 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1280928133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:56 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:56.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122302 14.26 MB/sec execute 297 sec latency 25065.016 ms 2024-09-15T22:32:57.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/3790087906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4102790 msgr2=0x7fdbd41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/3790087906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4102790 0x7fdbd41067e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc802a410 comp rx=0 tx=0).stop 2024-09-15T22:32:57.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/3790087906 shutdown_connections 2024-09-15T22:32:57.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/3790087906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 0x7fdbd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/3790087906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4102790 0x7fdbd41067e0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/3790087906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4101de0 0x7fdbd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/3790087906 >> 172.21.15.77:0/3790087906 conn(0x7fdbd40fd8c0 msgr2=0x7fdbd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/3790087906 shutdown_connections 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/3790087906 wait complete. 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.997+0000 7fdbd9c4f640 1 Processor -- start 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 -- start start 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102790 0x7fdbd4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 0x7fdbd419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4111f70 con 0x7fdbd4101de0 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4111df0 con 0x7fdbd4106f10 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd9c4f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41120f0 con 0x7fdbd4102790 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48516/0 (socket says 172.21.15.77:48516) 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 -- 172.21.15.77:0/2938731928 learned_addr learned my addr 172.21.15.77:0/2938731928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 0x7fdbd419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102790 0x7fdbd4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 -- 172.21.15.77:0/2938731928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102790 msgr2=0x7fdbd4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102790 0x7fdbd4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 -- 172.21.15.77:0/2938731928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 msgr2=0x7fdbd419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 0x7fdbd419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 -- 172.21.15.77:0/2938731928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd419d3e0 con 0x7fdbd4101de0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.998+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9f0 tx=0x7fdbc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.999+0000 7fdbd0ff9640 1 -- 172.21.15.77:0/2938731928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400cd80 con 0x7fdbd4101de0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.999+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd419d6d0 con 0x7fdbd4101de0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.999+0000 7fdbd0ff9640 1 -- 172.21.15.77:0/2938731928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc4005500 con 0x7fdbd4101de0 2024-09-15T22:32:57.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.999+0000 7fdbd0ff9640 1 -- 172.21.15.77:0/2938731928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4010690 con 0x7fdbd4101de0 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:56.999+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a4fb0 con 0x7fdbd4101de0 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbd0ff9640 1 -- 172.21.15.77:0/2938731928 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdbc40046c0 con 0x7fdbd4101de0 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 Processor -- start 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 -- start start 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7d90 0x562ffc4b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ffc408620 con 0x562ffc4a7470 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.001+0000 7fdbdc028740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ffc3afe50 con 0x562ffc4a7d90 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbdc028740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562ffc4045e0 con 0x562ffc4b2650 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48530/0 (socket says 172.21.15.77:48530) 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 -- 172.21.15.77:0/2269388956 learned_addr learned my addr 172.21.15.77:0/2269388956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:57.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7d90 0x562ffc4b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd0ff9640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 0x7fdbbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 -- 172.21.15.77:0/2269388956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 msgr2=0x562ffc4b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 -- 172.21.15.77:0/2269388956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7d90 msgr2=0x562ffc4b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7d90 0x562ffc4b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 -- 172.21.15.77:0/2269388956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffc3a0d40 con 0x562ffc4a7470 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd0ff9640 1 -- 172.21.15.77:0/2938731928 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdbc40d04d0 con 0x7fdbd4101de0 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009f50 tx=0x7fdbcc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c6e97162f8586c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 0x7fdbbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbb9ffb640 1 -- 172.21.15.77:0/2269388956 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc01e070 con 0x562ffc4a7470 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbb9ffb640 1 -- 172.21.15.77:0/2269388956 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc004610 con 0x562ffc4a7470 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.002+0000 7fdbb9ffb640 1 -- 172.21.15.77:0/2269388956 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc019430 con 0x562ffc4a7470 2024-09-15T22:32:57.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- 172.21.15.77:0/2269388956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 msgr2=0x562ffc4a7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 0x7fdbbc0a5870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc01b000 tx=0x7fdbcc019b00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009f50 tx=0x7fdbcc0118d0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- 172.21.15.77:0/2269388956 shutdown_connections 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7d90 0x562ffc4b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 --2- 172.21.15.77:0/2269388956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4a7850 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- 172.21.15.77:0/2269388956 >> 172.21.15.77:0/2269388956 conn(0x562ffc4a3a00 msgr2=0x562ffc4a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- 172.21.15.77:0/2269388956 shutdown_connections 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- 172.21.15.77:0/2269388956 wait complete. 2024-09-15T22:32:57.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.003+0000 7fdbdc028740 1 -- start start 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 Processor -- start 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4b2650 0x562ffc4aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562ffc4b5220 con 0x562ffc4a7470 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562ffc390f10 con 0x562ffc4b2650 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562ffc4b53a0 con 0x562ffc4a7d90 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbdc028740 10 client.0 _ll_register_callbacks cb 0x562ffc3b4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48534/0 (socket says 172.21.15.77:48534) 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 -- 172.21.15.77:0/3759380904 learned_addr learned my addr 172.21.15.77:0/3759380904 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:57.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4b2650 0x562ffc4aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 -- 172.21.15.77:0/3759380904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 msgr2=0x562ffc4afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 -- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4b2650 msgr2=0x562ffc4aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4b2650 0x562ffc4aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 -- 172.21.15.77:0/3759380904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffc117580 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.004+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9c0 tx=0x7fdbc40d3c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566757d8dbd4cef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc409bf60 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdbc40a4c00 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc004610 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbc409bd80 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 10 client.20301 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ffc1e1960 con 0x562ffc4a7470 2024-09-15T22:32:57.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc004610 con 0x562ffc4a7470 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdbc40b5140 con 0x562ffc4a7470 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 10 client.20301 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 10 client.20301 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 20 client.20301 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc4c2220 0x562ffc4c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 4 client.20301 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffc415b90 con 0x562ffc4c2220 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.005+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc4c2220 0x562ffc4c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.006+0000 7fdbb2ffd640 10 client.20301 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.006+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc4c2220 0x562ffc4c4610 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b82bbd07a1b3db server_cookie=cf1e556c647e9bce in_seq=0 out_seq=0 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.006+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/3759380904 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ffc415b90 con 0x562ffc4c2220 2024-09-15T22:32:57.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.006+0000 7fdbb2ffd640 10 client.20301 handle_command_reply: tid=0 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.006+0000 7fdbdc028740 1 client.20301 shutdown 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 msgr2=0x562ffc4af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9c0 tx=0x7fdbc40d3c10 comp rx=0 tx=0).stop 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 shutdown_connections 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc4c2220 0x562ffc4c4610 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4b2650 0x562ffc4aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbb2ffd640 0 client.20301 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 --2- 172.21.15.77:0/3759380904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7470 0x562ffc4af4f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 >> 172.21.15.77:0/3759380904 conn(0x562ffc4a3a00 msgr2=0x562ffc49e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 shutdown_connections 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 1 -- 172.21.15.77:0/3759380904 wait complete. 2024-09-15T22:32:57.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.007+0000 7fdbdc028740 20 client.20301 trim_cache size 0 max 16384 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.023+0000 7fdbdc028740 1 Processor -- start 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.023+0000 7fdbdc028740 1 -- start start 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41ace30 con 0x562ffc4a7d90 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41ae970 con 0x562ffc4a7470 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbdc028740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41ad500 con 0x562ffc4b2650 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48540/0 (socket says 172.21.15.77:48540) 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 -- 172.21.15.77:0/12544820 learned_addr learned my addr 172.21.15.77:0/12544820 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 -- 172.21.15.77:0/12544820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 msgr2=0x562ffc4ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 -- 172.21.15.77:0/12544820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 msgr2=0x562ffc4b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 -- 172.21.15.77:0/12544820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffc4a3830 con 0x562ffc4a7d90 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e541a93cdead223 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbb97fa640 1 -- 172.21.15.77:0/12544820 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc804d030 con 0x562ffc4a7d90 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbb97fa640 1 -- 172.21.15.77:0/12544820 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8031a00 con 0x562ffc4a7d90 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.024+0000 7fdbb97fa640 1 -- 172.21.15.77:0/12544820 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8035040 con 0x562ffc4a7d90 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- 172.21.15.77:0/12544820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 msgr2=0x562ffc4c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc8009680 comp rx=0 tx=0).stop 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- 172.21.15.77:0/12544820 shutdown_connections 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4b2650 0x562ffc4ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4a7d90 0x562ffc4c9dc0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 --2- 172.21.15.77:0/12544820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- 172.21.15.77:0/12544820 >> 172.21.15.77:0/12544820 conn(0x562ffc4a3a00 msgr2=0x562ffc4b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- 172.21.15.77:0/12544820 shutdown_connections 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- 172.21.15.77:0/12544820 wait complete. 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.025+0000 7fdbdc028740 1 -- start start 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 Processor -- start 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41ae2a0 con 0x562ffc4b2650 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41adbd0 con 0x562ffc4a7470 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41aae30 con 0x562ffc4a7d90 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbdc028740 10 client.0 _ll_register_callbacks cb 0x7fdbc4009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:32:57.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48542/0 (socket says 172.21.15.77:48542) 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 -- 172.21.15.77:0/4203765387 learned_addr learned my addr 172.21.15.77:0/4203765387 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd2ffd640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 -- 172.21.15.77:0/4203765387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 msgr2=0x562ffc4b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 -- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 msgr2=0x562ffc4b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 -- 172.21.15.77:0/4203765387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffc4baa00 con 0x562ffc4b2650 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:32:57.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbd3fff640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ef0 tx=0x7fdbcc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449ed4211c7a121 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.026+0000 7fdbb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc01e040 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9438+0+0 (secure 0 0 0) 0x7fdbcc004050 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc8031a00 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc005bd0 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 10 client.20307 fetch_fsmap learned FSMap version 39 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ffc52d890 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc8031a00 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdbcc00ecf0 con 0x562ffc4b2650 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 10 client.20307 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 10 client.20307 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 20 client.20307 populate_metadata read hostname 'smithi077' 2024-09-15T22:32:57.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc52dea0 0x562ffc5302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:57.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 4 client.20307 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3ce00f87", "release": true, "await": true}] 2024-09-15T22:32:57.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3ce00f87", "release": true, "await": true}) -- 0x562ffc4100a0 con 0x562ffc52dea0 2024-09-15T22:32:57.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.027+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc52dea0 0x562ffc5302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:57.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.028+0000 7fdbd37fe640 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc52dea0 0x562ffc5302e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfca6fe2081dfa server_cookie=8b8d96104ab9eb50 in_seq=0 out_seq=0 2024-09-15T22:32:57.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.028+0000 7fdbb2ffd640 10 client.20307 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:57.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbb2ffd640 1 -- 172.21.15.77:0/4203765387 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562ffc4100a0 con 0x562ffc52dea0 2024-09-15T22:32:57.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbb2ffd640 10 client.20307 handle_command_reply: tid=0 2024-09-15T22:32:57.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 client.20307 shutdown 2024-09-15T22:32:57.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 msgr2=0x562ffc4ba390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ef0 tx=0x7fdbcc019000 comp rx=0 tx=0).stop 2024-09-15T22:32:57.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 shutdown_connections 2024-09-15T22:32:57.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x562ffc52dea0 0x562ffc5302e0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ffc4b2650 0x562ffc4ba390 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbb2ffd640 0 client.20307 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562ffc4a7d90 0x562ffc4b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 --2- 172.21.15.77:0/4203765387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562ffc4a7470 0x562ffc4b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 >> 172.21.15.77:0/4203765387 conn(0x562ffc4a3a00 msgr2=0x562ffc4a8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 shutdown_connections 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.082+0000 7fdbdc028740 1 -- 172.21.15.77:0/4203765387 wait complete. 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbdc028740 20 client.20307 trim_cache size 0 max 16384 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:32:57.085 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 76, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "3ce00f87": { 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 76, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 148.2, 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:32:57.086 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:32:57.087 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 msgr2=0x7fdbbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 0x7fdbbc0a5870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc01b000 tx=0x7fdbcc019b00 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 msgr2=0x7fdbd4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400e9f0 tx=0x7fdbc400edf0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 shutdown_connections 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdbbc0a33b0 0x7fdbbc0a5870 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbd4106f10 0x7fdbd419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbd4102790 0x7fdbd4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 --2- 172.21.15.77:0/2938731928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd4101de0 0x7fdbd4198430 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:57.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.083+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 >> 172.21.15.77:0/2938731928 conn(0x7fdbd40fd8c0 msgr2=0x7fdbd40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.084+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 shutdown_connections 2024-09-15T22:32:57.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:32:57.084+0000 7fdbd9c4f640 1 -- 172.21.15.77:0/2938731928 wait complete. 2024-09-15T22:32:57.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122539 14.26 MB/sec execute 298 sec latency 25488.092 ms 2024-09-15T22:32:57.914 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3ce00f87', total seconds elapsed: 31.6 2024-09-15T22:32:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:58 smithi077 ceph-mon[30980]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 423 KiB/s wr, 63 op/s 2024-09-15T22:32:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:58 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:58 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:58 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:58 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:58 smithi196 ceph-mon[35109]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 423 KiB/s wr, 63 op/s 2024-09-15T22:32:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:58 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:58 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:58 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:58 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:58 smithi117 ceph-mon[35021]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 423 KiB/s wr, 63 op/s 2024-09-15T22:32:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:58 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:58 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:58 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:58 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:32:58.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 122865 14.25 MB/sec execute 299 sec latency 187.805 ms 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:32:59 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:32:59 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi196 to 176.0M 2024-09-15T22:32:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi196 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-15T22:32:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:32:59 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:59.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 123356 14.28 MB/sec execute 300 sec latency 162.494 ms 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:00.583 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:33:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:33:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:00.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-15T22:33:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:33:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-15T22:33:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:00.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:33:00.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 123838 14.32 MB/sec execute 301 sec latency 149.826 ms 2024-09-15T22:33:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:01 smithi196 ceph-mon[35109]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:33:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:01 smithi196 ceph-mon[35109]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:33:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:01 smithi117 ceph-mon[35021]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:33:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:01 smithi117 ceph-mon[35021]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:33:01.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 124108 14.31 MB/sec execute 302 sec latency 319.223 ms 2024-09-15T22:33:01.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:01 smithi077 ceph-mon[30980]: Adjusting osd_memory_target on smithi077 to 176.0M 2024-09-15T22:33:01.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:01 smithi077 ceph-mon[30980]: Unable to set osd_memory_target on smithi077 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-15T22:33:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:02 smithi196 ceph-mon[35109]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1014 KiB/s wr, 66 op/s 2024-09-15T22:33:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:02 smithi117 ceph-mon[35021]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1014 KiB/s wr, 66 op/s 2024-09-15T22:33:02.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 124422 14.32 MB/sec execute 303 sec latency 501.186 ms 2024-09-15T22:33:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:02 smithi077 ceph-mon[30980]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1014 KiB/s wr, 66 op/s 2024-09-15T22:33:03.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 124645 14.32 MB/sec execute 304 sec latency 339.838 ms 2024-09-15T22:33:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:04 smithi196 ceph-mon[35109]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:33:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:04 smithi117 ceph-mon[35021]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:33:04.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 124863 14.32 MB/sec execute 305 sec latency 554.584 ms 2024-09-15T22:33:04.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:04 smithi077 ceph-mon[30980]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:33:05.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 125311 14.39 MB/sec execute 306 sec latency 166.471 ms 2024-09-15T22:33:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:06 smithi196 ceph-mon[35109]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-15T22:33:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:06 smithi117 ceph-mon[35021]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-15T22:33:06.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 125581 14.38 MB/sec execute 307 sec latency 214.070 ms 2024-09-15T22:33:06.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:06 smithi077 ceph-mon[30980]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-15T22:33:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:07 smithi117 ceph-mon[35021]: osdmap e120: 12 total, 12 up, 12 in 2024-09-15T22:33:07.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 125864 14.37 MB/sec execute 308 sec latency 217.561 ms 2024-09-15T22:33:07.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:07 smithi077 ceph-mon[30980]: osdmap e120: 12 total, 12 up, 12 in 2024-09-15T22:33:07.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:07 smithi196 ceph-mon[35109]: osdmap e120: 12 total, 12 up, 12 in 2024-09-15T22:33:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:08 smithi117 ceph-mon[35021]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:08.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 126236 14.39 MB/sec execute 309 sec latency 201.972 ms 2024-09-15T22:33:08.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:08 smithi077 ceph-mon[30980]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:08 smithi196 ceph-mon[35109]: pgmap v653: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:09.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 126641 14.38 MB/sec execute 310 sec latency 180.466 ms 2024-09-15T22:33:10.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 126979 14.40 MB/sec execute 311 sec latency 1044.178 ms 2024-09-15T22:33:10.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:10 smithi077 ceph-mon[30980]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:10 smithi196 ceph-mon[35109]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:10 smithi117 ceph-mon[35021]: pgmap v654: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 539 op/s 2024-09-15T22:33:11.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 127370 14.44 MB/sec execute 312 sec latency 2044.205 ms 2024-09-15T22:33:12.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 127697 14.45 MB/sec execute 313 sec latency 3044.240 ms 2024-09-15T22:33:12.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:12 smithi077 ceph-mon[30980]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:33:12.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:12 smithi196 ceph-mon[35109]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:33:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:12 smithi117 ceph-mon[35021]: pgmap v655: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:33:13.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 128088 14.46 MB/sec execute 314 sec latency 4044.269 ms 2024-09-15T22:33:14.614 INFO:tasks.workunit.client.0.smithi077.stdout: 10 128406 14.45 MB/sec execute 315 sec latency 5044.305 ms 2024-09-15T22:33:14.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:14 smithi077 ceph-mon[30980]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-15T22:33:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:14 smithi077 ceph-mon[30980]: osdmap e121: 12 total, 12 up, 12 in 2024-09-15T22:33:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:14 smithi196 ceph-mon[35109]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-15T22:33:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:14 smithi196 ceph-mon[35109]: osdmap e121: 12 total, 12 up, 12 in 2024-09-15T22:33:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:14 smithi117 ceph-mon[35021]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 557 op/s 2024-09-15T22:33:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:14 smithi117 ceph-mon[35021]: osdmap e121: 12 total, 12 up, 12 in 2024-09-15T22:33:15.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 128843 14.49 MB/sec execute 316 sec latency 6044.344 ms 2024-09-15T22:33:16.126 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:33:16.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:16 smithi077 ceph-mon[30980]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 247 op/s 2024-09-15T22:33:16.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 129241 14.53 MB/sec execute 317 sec latency 7044.378 ms 2024-09-15T22:33:16.827 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:16.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:16 smithi196 ceph-mon[35109]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 247 op/s 2024-09-15T22:33:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:16 smithi117 ceph-mon[35021]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 247 op/s 2024-09-15T22:33:17.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 129635 14.58 MB/sec execute 318 sec latency 8044.404 ms 2024-09-15T22:33:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 -- 172.21.15.77:0/3864687538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 msgr2=0x7f06a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 --2- 172.21.15.77:0/3864687538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 0x7f06a810da40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009ca0 tx=0x7f06a002a950 comp rx=0 tx=0).stop 2024-09-15T22:33:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 -- 172.21.15.77:0/3864687538 shutdown_connections 2024-09-15T22:33:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 --2- 172.21.15.77:0/3864687538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 0x7f06a810da40 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 --2- 172.21.15.77:0/3864687538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8102570 0x7f06a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 --2- 172.21.15.77:0/3864687538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8100e20 0x7f06a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 -- 172.21.15.77:0/3864687538 >> 172.21.15.77:0/3864687538 conn(0x7f06a80fc7d0 msgr2=0x7f06a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 -- 172.21.15.77:0/3864687538 shutdown_connections 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.947+0000 7f06af7d0640 1 -- 172.21.15.77:0/3864687538 wait complete. 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 Processor -- start 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 -- start start 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102570 0x7f06a8073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 0x7f06a806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8110f80 con 0x7f06a8102ef0 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8110e00 con 0x7f06a8102570 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.948+0000 7f06af7d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8111100 con 0x7f06a8100e20 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:40040/0 (socket says 172.21.15.77:40040) 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 -- 172.21.15.77:0/95310619 learned_addr learned my addr 172.21.15.77:0/95310619 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06acd6b640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102570 0x7f06a8073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 -- 172.21.15.77:0/95310619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102570 msgr2=0x7f06a8073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102570 0x7f06a8073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 -- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 msgr2=0x7f06a806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 0x7f06a806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 -- 172.21.15.77:0/95310619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a806d4c0 con 0x7f06a8100e20 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.949+0000 7f06ad56c640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06a400e9f0 tx=0x7f06a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.950+0000 7f069e7fc640 1 -- 172.21.15.77:0/95310619 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a400cd80 con 0x7f06a8100e20 2024-09-15T22:33:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.950+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a806d7b0 con 0x7f06a8100e20 2024-09-15T22:33:17.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.950+0000 7f069e7fc640 1 -- 172.21.15.77:0/95310619 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a4010910 con 0x7f06a8100e20 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.950+0000 7f069e7fc640 1 -- 172.21.15.77:0/95310619 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a40188f0 con 0x7f06a8100e20 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.950+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a80717d0 con 0x7f06a8100e20 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.952+0000 7f069e7fc640 1 -- 172.21.15.77:0/95310619 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f06a4018b30 con 0x7f06a8100e20 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f069e7fc640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 0x7f06940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f06b1ba9740 1 Processor -- start 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f06b1ba9740 1 -- start start 2024-09-15T22:33:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f06acd6b640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 0x7f06940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f069e7fc640 1 -- 172.21.15.77:0/95310619 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f06a40d0350 con 0x7f06a8100e20 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.953+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc8e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc8e680 0x55f0abc90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0abbe92e0 con 0x55f0abc83dc0 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0abb90ce0 con 0x55f0abc8e680 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0abbe52a0 con 0x55f0abc834a0 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:40044/0 (socket says 172.21.15.77:40044) 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 -- 172.21.15.77:0/207230364 learned_addr learned my addr 172.21.15.77:0/207230364 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06add6d640 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc8e680 0x55f0abc90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06acd6b640 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc8e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06acd6b640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 0x7f06940a58f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f06a806e1e0 tx=0x7f0698005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 -- 172.21.15.77:0/207230364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc8e680 msgr2=0x55f0abc90a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc8e680 0x55f0abc90a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 -- 172.21.15.77:0/207230364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 msgr2=0x55f0abc8e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc8e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 -- 172.21.15.77:0/207230364 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0abb75440 con 0x55f0abc834a0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06ad56c640 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06a4098c70 tx=0x7f06a409ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b499d1416a8dc7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06937fe640 1 -- 172.21.15.77:0/207230364 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a40cde60 con 0x55f0abc834a0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06937fe640 1 -- 172.21.15.77:0/207230364 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a40a4f10 con 0x55f0abc834a0 2024-09-15T22:33:17.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.954+0000 7f06937fe640 1 -- 172.21.15.77:0/207230364 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a40a03d0 con 0x55f0abc834a0 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 -- 172.21.15.77:0/207230364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 msgr2=0x55f0abc83880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f06a4098c70 tx=0x7f06a409ca60 comp rx=0 tx=0).stop 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 -- 172.21.15.77:0/207230364 shutdown_connections 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc8e680 0x55f0abc90a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc8e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/207230364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc834a0 0x55f0abc83880 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 -- 172.21.15.77:0/207230364 >> 172.21.15.77:0/207230364 conn(0x55f0abc7fa30 msgr2=0x55f0abc7fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 -- 172.21.15.77:0/207230364 shutdown_connections 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.955+0000 7f06b1ba9740 1 -- 172.21.15.77:0/207230364 wait complete. 2024-09-15T22:33:17.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.956+0000 7f06b1ba9740 1 -- start start 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.956+0000 7f06b1ba9740 1 Processor -- start 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc8b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f0abc911f0 con 0x55f0abc834a0 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0abba4050 con 0x55f0abc83dc0 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06ad56c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06acd6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06add6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc8b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06acd6b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:38170/0 (socket says 172.21.15.77:38170) 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06acd6b640 1 -- 172.21.15.77:0/2583871002 learned_addr learned my addr 172.21.15.77:0/2583871002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06ad56c640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37892/0 (socket says 172.21.15.77:37892) 2024-09-15T22:33:17.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0abc91370 con 0x55f0abc8e680 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.957+0000 7f06b1ba9740 10 client.0 _ll_register_callbacks cb 0x55f0abbe4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 -- 172.21.15.77:0/2583871002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 msgr2=0x55f0abc8b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc8b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 -- 172.21.15.77:0/2583871002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 msgr2=0x55f0abc8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 -- 172.21.15.77:0/2583871002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0ab95a3c0 con 0x55f0abc83dc0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06ad56c640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06add6d640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc8b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f069801dc00 tx=0x7f0698004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58543a1b22163fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f0690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698004050 con 0x55f0abc83dc0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06a40a4f10 con 0x55f0abc83dc0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.958+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f0698009040 con 0x55f0abc83dc0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.959+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06980041f0 con 0x55f0abc83dc0 2024-09-15T22:33:17.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.959+0000 7f06b1ba9740 10 client.29051 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.959+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f0abc9cc30 con 0x55f0abc83dc0 2024-09-15T22:33:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069800ed90 con 0x55f0abc83dc0 2024-09-15T22:33:17.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0698003cb0 con 0x55f0abc83dc0 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 10 client.29051 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 10 client.29051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 20 client.29051 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abc9e560 0x55f0abca09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 4 client.29051 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.960+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f06a40a4f10 con 0x55f0abc9e560 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.961+0000 7f06ad56c640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abc9e560 0x55f0abca09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.962+0000 7f06ad56c640 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abc9e560 0x55f0abca09a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3a50c1be30bae4 server_cookie=ed3c41eee92c2191 in_seq=0 out_seq=0 2024-09-15T22:33:17.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.962+0000 7f0690ff9640 10 client.29051 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f0690ff9640 1 -- 172.21.15.77:0/2583871002 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f06a40a4f10 con 0x55f0abc9e560 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f0690ff9640 10 client.29051 handle_command_reply: tid=0 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 client.29051 shutdown 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 msgr2=0x55f0abc8b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f069801dc00 tx=0x7f0698004730 comp rx=0 tx=0).stop 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 shutdown_connections 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abc9e560 0x55f0abca09a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc8b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f0690ff9640 0 client.29051 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:17.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2583871002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc834a0 0x55f0abc8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 >> 172.21.15.77:0/2583871002 conn(0x55f0abc7fa30 msgr2=0x55f0abc7eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 shutdown_connections 2024-09-15T22:33:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.969+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2583871002 wait complete. 2024-09-15T22:33:17.973 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.970+0000 7f06b1ba9740 20 client.29051 trim_cache size 0 max 16384 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.993+0000 7f06b1ba9740 1 Processor -- start 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.993+0000 7f06b1ba9740 1 -- start start 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc9a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 0x55f0abc867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:17.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f068c00fe90 con 0x55f0abc8e680 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f068c0119d0 con 0x55f0abc834a0 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f068c010560 con 0x55f0abc83dc0 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06add6d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 0x55f0abc867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06add6d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 0x55f0abc867f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37896/0 (socket says 172.21.15.77:37896) 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06add6d640 1 -- 172.21.15.77:0/2403762064 learned_addr learned my addr 172.21.15.77:0/2403762064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06ad56c640 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc9a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 -- 172.21.15.77:0/2403762064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 msgr2=0x55f0abc9a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc9a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 -- 172.21.15.77:0/2403762064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 msgr2=0x55f0abc867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 0x55f0abc867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.994+0000 7f06acd6b640 1 -- 172.21.15.77:0/2403762064 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0abc8e350 con 0x55f0abc83dc0 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.995+0000 7f06acd6b640 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f069801c430 tx=0x7f0698005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af6d762db93b1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:17.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.995+0000 7f0692ffd640 1 -- 172.21.15.77:0/2403762064 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698004390 con 0x55f0abc83dc0 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.995+0000 7f0692ffd640 1 -- 172.21.15.77:0/2403762064 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069801c6a0 con 0x55f0abc83dc0 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.995+0000 7f0692ffd640 1 -- 172.21.15.77:0/2403762064 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698005420 con 0x55f0abc83dc0 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2403762064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 msgr2=0x55f0abc8b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f069801c430 tx=0x7f0698005d30 comp rx=0 tx=0).stop 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2403762064 shutdown_connections 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc8e680 0x55f0abc867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc83dc0 0x55f0abc8b250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/2403762064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc9a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2403762064 >> 172.21.15.77:0/2403762064 conn(0x55f0abc7fa30 msgr2=0x55f0abc860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:18.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2403762064 shutdown_connections 2024-09-15T22:33:18.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- 172.21.15.77:0/2403762064 wait complete. 2024-09-15T22:33:18.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.996+0000 7f06b1ba9740 1 -- start start 2024-09-15T22:33:18.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 Processor -- start 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f068c011300 con 0x55f0abc83dc0 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f068c010c30 con 0x55f0abc834a0 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f068c00de90 con 0x55f0abc8e680 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06b1ba9740 10 client.0 _ll_register_callbacks cb 0x7f0698011c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06acd6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06add6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06acd6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37904/0 (socket says 172.21.15.77:37904) 2024-09-15T22:33:18.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06acd6b640 1 -- 172.21.15.77:0/79135817 learned_addr learned my addr 172.21.15.77:0/79135817 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.997+0000 7f06ad56c640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 -- 172.21.15.77:0/79135817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 msgr2=0x55f0abc922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc922e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 -- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 msgr2=0x55f0abc96a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc96a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 -- 172.21.15.77:0/79135817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0abc92820 con 0x55f0abc83dc0 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f06acd6b640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f069801c620 tx=0x7f0698005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45924b2678bfba97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.998+0000 7f0690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069801caf0 con 0x55f0abc83dc0 2024-09-15T22:33:18.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f069801cc90 con 0x55f0abc83dc0 2024-09-15T22:33:18.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069801c6a0 con 0x55f0abc83dc0 2024-09-15T22:33:18.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f069801ce90 con 0x55f0abc83dc0 2024-09-15T22:33:18.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f06b1ba9740 10 client.20322 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:18.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:17.999+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f0abd08f40 con 0x55f0abc83dc0 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.000+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069801c6a0 con 0x55f0abc83dc0 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0698004b10 con 0x55f0abc83dc0 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 10 client.20322 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 20 client.20322 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abd09580 0x55f0abd0b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 4 client.20322 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:33:18.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.001+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0abbf6850 con 0x55f0abd09580 2024-09-15T22:33:18.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.002+0000 7f06ad56c640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abd09580 0x55f0abd0b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:18.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.009+0000 7f0690ff9640 10 client.20322 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:18.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.009+0000 7f06ad56c640 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abd09580 0x55f0abd0b9a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2775b8c8a9166e server_cookie=1d6d4c39ef9a169a in_seq=0 out_seq=0 2024-09-15T22:33:18.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.012+0000 7f0690ff9640 1 -- 172.21.15.77:0/79135817 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f0abbf6850 con 0x55f0abd09580 2024-09-15T22:33:18.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.012+0000 7f0690ff9640 10 client.20322 handle_command_reply: tid=0 2024-09-15T22:33:18.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.012+0000 7f06b1ba9740 1 client.20322 shutdown 2024-09-15T22:33:18.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.013+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 msgr2=0x55f0abc91da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:18.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.013+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f069801c620 tx=0x7f0698005020 comp rx=0 tx=0).stop 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 shutdown_connections 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f0abd09580 0x55f0abd0b9a0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0abc8e680 0x55f0abc922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0abc83dc0 0x55f0abc91da0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 --2- 172.21.15.77:0/79135817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0abc834a0 0x55f0abc96a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 >> 172.21.15.77:0/79135817 conn(0x55f0abc7fa30 msgr2=0x55f0abc8d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:18.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.014+0000 7f0690ff9640 0 client.20322 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:18.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.016+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 shutdown_connections 2024-09-15T22:33:18.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.016+0000 7f06b1ba9740 1 -- 172.21.15.77:0/79135817 wait complete. 2024-09-15T22:33:18.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.018+0000 7f06b1ba9740 20 client.20322 trim_cache size 0 max 16384 2024-09-15T22:33:18.021 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:33:18.021 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:33:18.021 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:33:18.021 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 msgr2=0x7f06940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 0x7f06940a58f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f06a806e1e0 tx=0x7f0698005f90 comp rx=0 tx=0).stop 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 msgr2=0x7f06a8073990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06a400e9f0 tx=0x7f06a400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 shutdown_connections 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f06940a3430 0x7f06940a58f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a8102ef0 0x7f06a806cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a8102570 0x7f06a8073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 --2- 172.21.15.77:0/95310619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06a8100e20 0x7f06a8073990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:18.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.019+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 >> 172.21.15.77:0/95310619 conn(0x7f06a80fc7d0 msgr2=0x7f06a8106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:18.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.020+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 shutdown_connections 2024-09-15T22:33:18.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:18.020+0000 7f06af7d0640 1 -- 172.21.15.77:0/95310619 wait complete. 2024-09-15T22:33:18.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 129915 14.57 MB/sec execute 319 sec latency 9044.429 ms 2024-09-15T22:33:18.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:18 smithi077 ceph-mon[30980]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:18.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:18 smithi196 ceph-mon[35109]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:18 smithi117 ceph-mon[35021]: pgmap v659: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:19.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 130145 14.54 MB/sec execute 320 sec latency 10044.459 ms 2024-09-15T22:33:19.693 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:33:19.693 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:33:20.257 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:20.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 130481 14.54 MB/sec execute 321 sec latency 11044.491 ms 2024-09-15T22:33:20.655 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:20 smithi196 ceph-mon[35109]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:20.664 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:20 smithi117 ceph-mon[35021]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:20.698 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:20 smithi077 ceph-mon[30980]: pgmap v660: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 420 op/s 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.821+0000 7f2a66945640 1 -- 172.21.15.77:0/408850085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109450 msgr2=0x7f2a60109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.821+0000 7f2a66945640 1 --2- 172.21.15.77:0/408850085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109450 0x7f2a60109830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009860 tx=0x7f2a5002a340 comp rx=0 tx=0).stop 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 -- 172.21.15.77:0/408850085 shutdown_connections 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 --2- 172.21.15.77:0/408850085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a601154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 --2- 172.21.15.77:0/408850085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109e00 0x7f2a6010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 --2- 172.21.15.77:0/408850085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109450 0x7f2a60109830 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 -- 172.21.15.77:0/408850085 >> 172.21.15.77:0/408850085 conn(0x7f2a6006e2d0 msgr2=0x7f2a6006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 -- 172.21.15.77:0/408850085 shutdown_connections 2024-09-15T22:33:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.822+0000 7f2a66945640 1 -- 172.21.15.77:0/408850085 wait complete. 2024-09-15T22:33:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 Processor -- start 2024-09-15T22:33:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 -- start start 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 0x7f2a60118e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109e00 0x7f2a601193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a60119910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.823+0000 7f2a66945640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60116b80 con 0x7f2a60109450 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a66945640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60116a00 con 0x7f2a60109e00 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a66945640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60116d00 con 0x7f2a6010ea80 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 0x7f2a60118e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 0x7f2a60118e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52120/0 (socket says 172.21.15.77:52120) 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a5ffff640 1 -- 172.21.15.77:0/2970018285 learned_addr learned my addr 172.21.15.77:0/2970018285 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:20.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a60119910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 -- 172.21.15.77:0/2970018285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109e00 msgr2=0x7f2a601193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109e00 0x7f2a601193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 -- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 msgr2=0x7f2a60118e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 0x7f2a60118e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.824+0000 7f2a64ee2640 1 -- 172.21.15.77:0/2970018285 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a601a4e10 con 0x7f2a6010ea80 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.825+0000 7f2a64ee2640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a60119910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009fd0 tx=0x7f2a5800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.825+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800ce20 con 0x7f2a6010ea80 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.825+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a601a5100 con 0x7f2a6010ea80 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.825+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a601a5640 con 0x7f2a6010ea80 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.826+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a58005020 con 0x7f2a6010ea80 2024-09-15T22:33:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.826+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58010720 con 0x7f2a6010ea80 2024-09-15T22:33:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.827+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c005290 con 0x7f2a6010ea80 2024-09-15T22:33:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.829+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2a58004b20 con 0x7f2a6010ea80 2024-09-15T22:33:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.830+0000 7f2a5d7fa640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 0x7f2a480a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.830+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f2a580d0620 con 0x7f2a6010ea80 2024-09-15T22:33:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.831+0000 7f2a5ffff640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 0x7f2a480a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:20.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.832+0000 7f2a5ffff640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 0x7f2a480a5a40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a500095f0 tx=0x7f2a500058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:20.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:20.837+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a5809cc10 con 0x7f2a6010ea80 2024-09-15T22:33:21.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.189+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a4c005850 con 0x7f2a6010ea80 2024-09-15T22:33:21.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.192+0000 7f2a5d7fa640 1 -- 172.21.15.77:0/2970018285 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f2a5809c510 con 0x7f2a6010ea80 2024-09-15T22:33:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:33:21.196 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:33:21.196 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 msgr2=0x7f2a480a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 0x7f2a480a5a40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a500095f0 tx=0x7f2a500058d0 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 msgr2=0x7f2a60119910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a60119910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009fd0 tx=0x7f2a5800ef10 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 shutdown_connections 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2a480a3580 0x7f2a480a5a40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a6010ea80 0x7f2a60119910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a60109e00 0x7f2a601193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 --2- 172.21.15.77:0/2970018285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a60109450 0x7f2a60118e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 >> 172.21.15.77:0/2970018285 conn(0x7f2a6006e2d0 msgr2=0x7f2a6010c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:21.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 shutdown_connections 2024-09-15T22:33:21.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:21.196+0000 7f2a66945640 1 -- 172.21.15.77:0/2970018285 wait complete. 2024-09-15T22:33:21.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:21 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2970018285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:33:21.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 130814 14.55 MB/sec execute 322 sec latency 11889.793 ms 2024-09-15T22:33:21.667 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:21 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2970018285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:33:21.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:21 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2970018285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:33:22.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 131331 14.61 MB/sec execute 323 sec latency 155.724 ms 2024-09-15T22:33:22.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:22 smithi077 ceph-mon[30980]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 402 op/s 2024-09-15T22:33:22.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:22 smithi196 ceph-mon[35109]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 402 op/s 2024-09-15T22:33:22.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:22 smithi117 ceph-mon[35021]: pgmap v661: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 402 op/s 2024-09-15T22:33:22.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:33:23.310 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:23.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 131796 14.66 MB/sec execute 324 sec latency 170.223 ms 2024-09-15T22:33:24.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.837+0000 7f10a64cb640 1 -- 172.21.15.77:0/1740769450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0102d90 msgr2=0x7f10a0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.837+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1740769450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0102d90 0x7f10a0103170 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1090009930 tx=0x7f109002a420 comp rx=0 tx=0).stop 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 -- 172.21.15.77:0/1740769450 shutdown_connections 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1740769450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10a0108290 0x7f10a010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1740769450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 0x7f10a0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1740769450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0102d90 0x7f10a0103170 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 -- 172.21.15.77:0/1740769450 >> 172.21.15.77:0/1740769450 conn(0x7f10a00fc7d0 msgr2=0x7f10a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 -- 172.21.15.77:0/1740769450 shutdown_connections 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 -- 172.21.15.77:0/1740769450 wait complete. 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 Processor -- start 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.838+0000 7f10a64cb640 1 -- start start 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10a0102d90 0x7f10a0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 0x7f10a0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0112150 con 0x7f10a0108290 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0111fd0 con 0x7f10a0103740 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a64cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01122d0 con 0x7f10a0102d90 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52140/0 (socket says 172.21.15.77:52140) 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 -- 172.21.15.77:0/1338548345 learned_addr learned my addr 172.21.15.77:0/1338548345 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f109f7fe640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 0x7f10a0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 -- 172.21.15.77:0/1338548345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10a0102d90 msgr2=0x7f10a0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10a0102d90 0x7f10a0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 -- 172.21.15.77:0/1338548345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 msgr2=0x7f10a0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 0x7f10a0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 -- 172.21.15.77:0/1338548345 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a019d6c0 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.839+0000 7f10a4a68640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f109800b970 tx=0x7f109800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.840+0000 7f109d7fa640 1 -- 172.21.15.77:0/1338548345 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098019070 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.840+0000 7f109d7fa640 1 -- 172.21.15.77:0/1338548345 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1098014440 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.840+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a019d9b0 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.840+0000 7f109d7fa640 1 -- 172.21.15.77:0/1338548345 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980075d0 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.840+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a0100920 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.842+0000 7f109d7fa640 1 -- 172.21.15.77:0/1338548345 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f109801c070 con 0x7f10a0108290 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.842+0000 7f10a88a4740 1 Processor -- start 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.842+0000 7f10a88a4740 1 -- start start 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109d7fa640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 0x7f108c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da3600f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3601a370 0x55da3601c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da35f752f0 con 0x55da3600f5d0 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da35f1ccf0 con 0x55da3601a370 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a88a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da35f712b0 con 0x55da36010b40 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109d7fa640 1 -- 172.21.15.77:0/1338548345 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f10980cf590 con 0x7f10a0108290 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 0x7f108c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52144/0 (socket says 172.21.15.77:52144) 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 -- 172.21.15.77:0/3381848122 learned_addr learned my addr 172.21.15.77:0/3381848122 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f10a4a68640 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3601a370 0x55da3601c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 -- 172.21.15.77:0/3381848122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 msgr2=0x55da3600f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da3600f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 -- 172.21.15.77:0/3381848122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3601a370 msgr2=0x55da3601c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3601a370 0x55da3601c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 -- 172.21.15.77:0/3381848122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da35f013a0 con 0x55da3600f5d0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 0x7f108c0a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10880093f0 tx=0x7f1088009990 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.843+0000 7f109f7fe640 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f10880178e0 tx=0x7f108801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb5ff85e25ec612 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10967fc640 1 -- 172.21.15.77:0/3381848122 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088027070 con 0x55da3600f5d0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10967fc640 1 -- 172.21.15.77:0/3381848122 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088017e60 con 0x55da3600f5d0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10967fc640 1 -- 172.21.15.77:0/3381848122 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088022bf0 con 0x55da3600f5d0 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 -- 172.21.15.77:0/3381848122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 msgr2=0x55da3600fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f10880178e0 tx=0x7f108801a000 comp rx=0 tx=0).stop 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 -- 172.21.15.77:0/3381848122 shutdown_connections 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3601a370 0x55da3601c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da3600fa10 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3381848122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da3600f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 -- 172.21.15.77:0/3381848122 >> 172.21.15.77:0/3381848122 conn(0x55da35fd8310 msgr2=0x55da3600bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 -- 172.21.15.77:0/3381848122 shutdown_connections 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.844+0000 7f10a88a4740 1 -- 172.21.15.77:0/3381848122 wait complete. 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 -- start start 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 Processor -- start 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36019db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da36017060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 0x55da360177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55da3601d010 con 0x55da3601a370 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a4a68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 0x55da360177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da36017060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.845+0000 7f10a88a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da35f30060 con 0x55da3600f5d0 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da36017060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36408/0 (socket says 172.21.15.77:36408) 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f109f7fe640 1 -- 172.21.15.77:0/3230670556 learned_addr learned my addr 172.21.15.77:0/3230670556 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55da3601d190 con 0x55da36010b40 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a88a4740 10 client.0 _ll_register_callbacks cb 0x55da35c838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 -- 172.21.15.77:0/3230670556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 msgr2=0x55da36017060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da36017060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 -- 172.21.15.77:0/3230670556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 msgr2=0x55da36019db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36019db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 -- 172.21.15.77:0/3230670556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da35ce63c0 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a4a68640 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 0x55da360177f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f10980cca60 tx=0x7f109809b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92b0e160eaa1f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f107f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980b3070 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1088017e60 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f10980145f0 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f10980a3dd0 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a88a4740 10 client.20340 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.846+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da36028bb0 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088017e60 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f109809aa40 con 0x55da3601a370 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 10 client.20340 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 10 client.20340 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:33:24.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 20 client.20340 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da3602a010 0x55da3602c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 4 client.20340 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da35f82860 con 0x55da3602a010 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f109ffff640 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da3602a010 0x55da3602c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.847+0000 7f107f7fe640 10 client.20340 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.848+0000 7f109ffff640 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da3602a010 0x55da3602c4a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2fe04cfd1e3c6c8a server_cookie=79507f30425c5a34 in_seq=0 out_seq=0 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.872+0000 7f107f7fe640 1 -- 172.21.15.77:0/3230670556 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da35f82860 con 0x55da3602a010 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.872+0000 7f107f7fe640 10 client.20340 handle_command_reply: tid=0 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.872+0000 7f10a88a4740 1 client.20340 shutdown 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 msgr2=0x55da360177f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 0x55da360177f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f10980cca60 tx=0x7f109809b6b0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 shutdown_connections 2024-09-15T22:33:24.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da3602a010 0x55da3602c4a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3601a370 0x55da360177f0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da36010b40 0x55da36017060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3230670556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36019db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 >> 172.21.15.77:0/3230670556 conn(0x55da35fd8310 msgr2=0x55da36011960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f107f7fe640 0 client.20340 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 shutdown_connections 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 1 -- 172.21.15.77:0/3230670556 wait complete. 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.873+0000 7f10a88a4740 20 client.20340 trim_cache size 0 max 16384 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.886+0000 7f10a88a4740 1 Processor -- start 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.886+0000 7f10a88a4740 1 -- start start 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da36010b40 0x55da36018fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da36012120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01b1330 con 0x55da3600f5d0 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01b2e70 con 0x55da36010b40 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a88a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01b1a00 con 0x55da3601a370 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52156/0 (socket says 172.21.15.77:52156) 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 -- 172.21.15.77:0/3852398044 learned_addr learned my addr 172.21.15.77:0/3852398044 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f10a4a68640 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da36012120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 -- 172.21.15.77:0/3852398044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 msgr2=0x55da36012120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da36012120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 -- 172.21.15.77:0/3852398044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da36010b40 msgr2=0x55da36018fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da36010b40 0x55da36018fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 -- 172.21.15.77:0/3852398044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da36016010 con 0x55da3600f5d0 2024-09-15T22:33:24.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f109ffff640 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f109002a940 tx=0x7f1090031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f7b0c4230e2056 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f1095ffb640 1 -- 172.21.15.77:0/3852398044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090035040 con 0x55da3600f5d0 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f1095ffb640 1 -- 172.21.15.77:0/3852398044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1090005500 con 0x55da3600f5d0 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.887+0000 7f1095ffb640 1 -- 172.21.15.77:0/3852398044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109003b430 con 0x55da3600f5d0 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- 172.21.15.77:0/3852398044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 msgr2=0x55da36016920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f109002a940 tx=0x7f1090031660 comp rx=0 tx=0).stop 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- 172.21.15.77:0/3852398044 shutdown_connections 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da36012120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da36010b40 0x55da36018fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 --2- 172.21.15.77:0/3852398044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da3600f5d0 0x55da36016920 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- 172.21.15.77:0/3852398044 >> 172.21.15.77:0/3852398044 conn(0x55da35fd8310 msgr2=0x55da360188c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- 172.21.15.77:0/3852398044 shutdown_connections 2024-09-15T22:33:24.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- 172.21.15.77:0/3852398044 wait complete. 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.888+0000 7f10a88a4740 1 -- start start 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 Processor -- start 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36022d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da3601e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01b27a0 con 0x55da36010b40 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01b20d0 con 0x55da3600f5d0 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10a01af330 con 0x55da3601a370 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f10a88a4740 10 client.0 _ll_register_callbacks cb 0x7f109809bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52172/0 (socket says 172.21.15.77:52172) 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 -- 172.21.15.77:0/2513523603 learned_addr learned my addr 172.21.15.77:0/2513523603 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 -- 172.21.15.77:0/2513523603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 msgr2=0x55da3601e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:24.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da3601e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 -- 172.21.15.77:0/2513523603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 msgr2=0x55da36022d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36022d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.889+0000 7f109f7fe640 1 -- 172.21.15.77:0/2513523603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da36093d40 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.890+0000 7f109f7fe640 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f10880178e0 tx=0x7f1088015360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4f419ec77e1726 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.890+0000 7f107f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088032020 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f1088018c60 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1090005500 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1088027040 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f10a88a4740 10 client.20346 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.891+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55da36094f00 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.894+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088017e60 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.894+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f108802e480 con 0x55da36010b40 2024-09-15T22:33:24.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 10 client.20346 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 10 client.20346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 20 client.20346 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da36095240 0x55da360976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 4 client.20346 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.895+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1090005500 con 0x55da36095240 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.911+0000 7f109ffff640 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da36095240 0x55da360976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.915+0000 7f109ffff640 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da36095240 0x55da360976e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4022869034d7de02 server_cookie=13d18a88f4767c95 in_seq=0 out_seq=0 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.915+0000 7f107f7fe640 10 client.20346 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.919+0000 7f107f7fe640 1 -- 172.21.15.77:0/2513523603 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1090005500 con 0x55da36095240 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.919+0000 7f107f7fe640 10 client.20346 handle_command_reply: tid=0 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.919+0000 7f10a88a4740 1 client.20346 shutdown 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.922+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 msgr2=0x55da3601e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.922+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f10880178e0 tx=0x7f1088015360 comp rx=0 tx=0).stop 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 shutdown_connections 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55da36095240 0x55da360976e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55da3601a370 0x55da3601e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55da36010b40 0x55da3601e050 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 --2- 172.21.15.77:0/2513523603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da3600f5d0 0x55da36022d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 >> 172.21.15.77:0/2513523603 conn(0x55da35fd8310 msgr2=0x55da360190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.923+0000 7f107f7fe640 0 client.20346 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.924+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 shutdown_connections 2024-09-15T22:33:24.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.924+0000 7f10a88a4740 1 -- 172.21.15.77:0/2513523603 wait complete. 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.924+0000 7f10a88a4740 20 client.20346 trim_cache size 0 max 16384 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 msgr2=0x7f108c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 0x7f108c0a5870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f10880093f0 tx=0x7f1088009990 comp rx=0 tx=0).stop 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 msgr2=0x7f10a019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f109800b970 tx=0x7f109800bd70 comp rx=0 tx=0).stop 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 shutdown_connections 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f108c0a33b0 0x7f108c0a5870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10a0108290 0x7f10a019cfe0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103740 0x7f10a0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 --2- 172.21.15.77:0/1338548345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10a0102d90 0x7f10a0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:24.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.926+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 >> 172.21.15.77:0/1338548345 conn(0x7f10a00fc7d0 msgr2=0x7f10a0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:24.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.927+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 shutdown_connections 2024-09-15T22:33:24.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:23.929+0000 7f10a64cb640 1 -- 172.21.15.77:0/1338548345 wait complete. 2024-09-15T22:33:24.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 132040 14.63 MB/sec execute 325 sec latency 282.476 ms 2024-09-15T22:33:24.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:24 smithi077 ceph-mon[30980]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 421 op/s 2024-09-15T22:33:24.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:24 smithi196 ceph-mon[35109]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 421 op/s 2024-09-15T22:33:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:24 smithi117 ceph-mon[35021]: pgmap v662: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 421 op/s 2024-09-15T22:33:25.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 132359 14.62 MB/sec execute 326 sec latency 249.180 ms 2024-09-15T22:33:25.697 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:33:26.087 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:26.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:26 smithi077 ceph-mon[30980]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 393 op/s 2024-09-15T22:33:26.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 132765 14.65 MB/sec execute 327 sec latency 252.912 ms 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.640+0000 7f97af1a2640 1 -- 172.21.15.77:0/1141301872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109450 msgr2=0x7f97a8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.640+0000 7f97af1a2640 1 --2- 172.21.15.77:0/1141301872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109450 0x7f97a8110250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f979802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.640+0000 7f97af1a2640 1 -- 172.21.15.77:0/1141301872 shutdown_connections 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.640+0000 7f97af1a2640 1 --2- 172.21.15.77:0/1141301872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a811b140 0x7f97a811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.641+0000 7f97af1a2640 1 --2- 172.21.15.77:0/1141301872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a8109830 0x7f97a8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.641+0000 7f97af1a2640 1 --2- 172.21.15.77:0/1141301872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109450 0x7f97a8110250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.641+0000 7f97af1a2640 1 -- 172.21.15.77:0/1141301872 >> 172.21.15.77:0/1141301872 conn(0x7f97a806e140 msgr2=0x7f97a806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.641+0000 7f97af1a2640 1 -- 172.21.15.77:0/1141301872 shutdown_connections 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.641+0000 7f97af1a2640 1 -- 172.21.15.77:0/1141301872 wait complete. 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 Processor -- start 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 -- start start 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a8109450 0x7f97a81a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a811b140 0x7f97a81ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8120da0 con 0x7f97a811b140 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8120c20 con 0x7f97a8109450 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97af1a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8120f20 con 0x7f97a8109830 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36434/0 (socket says 172.21.15.77:36434) 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.642+0000 7f97a7fff640 1 -- 172.21.15.77:0/2487433220 learned_addr learned my addr 172.21.15.77:0/2487433220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:26.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97acf3e640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a8109450 0x7f97a81a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97ad73f640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a811b140 0x7f97a81ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 -- 172.21.15.77:0/2487433220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a8109450 msgr2=0x7f97a81a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a8109450 0x7f97a81a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 -- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a811b140 msgr2=0x7f97a81ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a811b140 0x7f97a81ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 -- 172.21.15.77:0/2487433220 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a81ac010 con 0x7f97a8109830 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f979000d950 tx=0x7f979000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a5ffb640 1 -- 172.21.15.77:0/2487433220 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790017070 con 0x7f97a8109830 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.643+0000 7f97a5ffb640 1 -- 172.21.15.77:0/2487433220 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9790009c30 con 0x7f97a8109830 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.644+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a81ac300 con 0x7f97a8109830 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.644+0000 7f97a5ffb640 1 -- 172.21.15.77:0/2487433220 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000f6b0 con 0x7f97a8109830 2024-09-15T22:33:26.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.644+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a81b3be0 con 0x7f97a8109830 2024-09-15T22:33:26.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.647+0000 7f97a5ffb640 1 -- 172.21.15.77:0/2487433220 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9790010460 con 0x7f97a8109830 2024-09-15T22:33:26.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.647+0000 7f97b157b740 1 Processor -- start 2024-09-15T22:33:26.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.647+0000 7f97b157b740 1 -- start start 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8ef2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8ef2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8e582f0 con 0x55b5e8efd370 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8dffcf0 con 0x55b5e8ef3b40 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97b157b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8e542b0 con 0x55b5e8ef25d0 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97ad73f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8ef2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97ad73f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8ef2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51778/0 (socket says 172.21.15.77:51778) 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97ad73f640 1 -- 172.21.15.77:0/2289708859 learned_addr learned my addr 172.21.15.77:0/2289708859 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a5ffb640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 0x7f97940a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a5ffb640 1 -- 172.21.15.77:0/2487433220 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f97900cfcb0 con 0x7f97a8109830 2024-09-15T22:33:26.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 -- 172.21.15.77:0/2289708859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 msgr2=0x55b5e8ef2a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:33:26.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8ef2a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 -- 172.21.15.77:0/2289708859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 msgr2=0x55b5e8ef2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8ef2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97a7fff640 1 -- 172.21.15.77:0/2289708859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e8de43a0 con 0x55b5e8efd370 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.648+0000 7f97ad73f640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 0x7f97940a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.649+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eff760 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f97900cd010 tx=0x7f9790010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d45e6619b23913 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.649+0000 7f979effd640 1 -- 172.21.15.77:0/2289708859 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790020070 con 0x55b5e8efd370 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.649+0000 7f97ad73f640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 0x7f97940a5970 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f97a00058c0 tx=0x7f97a0004730 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.650+0000 7f979effd640 1 -- 172.21.15.77:0/2289708859 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97900d3320 con 0x55b5e8efd370 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.650+0000 7f979effd640 1 -- 172.21.15.77:0/2289708859 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97900109d0 con 0x55b5e8efd370 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 -- 172.21.15.77:0/2289708859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 msgr2=0x55b5e8eff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eff760 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f97900cd010 tx=0x7f9790010d70 comp rx=0 tx=0).stop 2024-09-15T22:33:26.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 -- 172.21.15.77:0/2289708859 shutdown_connections 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eff760 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8ef2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 --2- 172.21.15.77:0/2289708859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8ef2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 -- 172.21.15.77:0/2289708859 >> 172.21.15.77:0/2289708859 conn(0x55b5e8ebb310 msgr2=0x55b5e8eeec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 -- 172.21.15.77:0/2289708859 shutdown_connections 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.651+0000 7f97b157b740 1 -- 172.21.15.77:0/2289708859 wait complete. 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.652+0000 7f97b157b740 1 -- start start 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.652+0000 7f97b157b740 1 Processor -- start 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.652+0000 7f97b157b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.652+0000 7f97b157b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8efa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97b157b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8efd370 0x55b5e8efa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97b157b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8f00000 con 0x55b5e8ef25d0 2024-09-15T22:33:26.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97b157b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8e13060 con 0x55b5e8efd370 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97b157b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b5e8f00180 con 0x55b5e8ef3b40 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8efa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97acf3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97b157b740 10 client.0 _ll_register_callbacks cb 0x55b5e8b668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97acf3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52210/0 (socket says 172.21.15.77:52210) 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97acf3e640 1 -- 172.21.15.77:0/1296833111 learned_addr learned my addr 172.21.15.77:0/1296833111 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97ad73f640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8efd370 0x55b5e8efa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 -- 172.21.15.77:0/1296833111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8efd370 msgr2=0x55b5e8efa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8efd370 0x55b5e8efa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 -- 172.21.15.77:0/1296833111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 msgr2=0x55b5e8efcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97a7fff640 1 -- 172.21.15.77:0/1296833111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e8bc93c0 con 0x55b5e8ef3b40 2024-09-15T22:33:26.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.653+0000 7f97acf3e640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f97a7fff640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8efa020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97900cd010 tx=0x7f97900a4900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a86babf9ca7c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f9787fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790020050 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f97900cd290 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5e8e65860 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f97900ba3e0 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f97b157b740 10 client.29005 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b5e8f00000 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.654+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97900ba580 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9790097a70 con 0x55b5e8ef3b40 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 10 client.29005 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 10 client.29005 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:33:26.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 20 client.29005 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:26.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f0cc20 0x55b5e8f0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 4 client.29005 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:26.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e8e65860 con 0x55b5e8f0cc20 2024-09-15T22:33:26.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f97acf3e640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f0cc20 0x55b5e8f0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.655+0000 7f9787fff640 10 client.29005 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.656+0000 7f97acf3e640 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f0cc20 0x55b5e8f0f010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3494ccf15818a0d server_cookie=47932c56803c6485 in_seq=0 out_seq=0 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.656+0000 7f9787fff640 1 -- 172.21.15.77:0/1296833111 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5e8e65860 con 0x55b5e8f0cc20 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.656+0000 7f9787fff640 10 client.29005 handle_command_reply: tid=0 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 client.29005 shutdown 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 msgr2=0x55b5e8efa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8efa020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97900cd010 tx=0x7f97900a4900 comp rx=0 tx=0).stop 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 shutdown_connections 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f0cc20 0x55b5e8f0f010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8efd370 0x55b5e8efa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8efa020 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 --2- 172.21.15.77:0/1296833111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8ef25d0 0x55b5e8efcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 >> 172.21.15.77:0/1296833111 conn(0x55b5e8ebb310 msgr2=0x55b5e8ef4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f9787fff640 0 client.29005 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.657+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 shutdown_connections 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.658+0000 7f97b157b740 1 -- 172.21.15.77:0/1296833111 wait complete. 2024-09-15T22:33:26.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.659+0000 7f97b157b740 20 client.29005 trim_cache size 0 max 16384 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 Processor -- start 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 -- start start 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef25d0 0x55b5e8eeed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81bdcb0 con 0x55b5e8efd370 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81bf7f0 con 0x55b5e8ef25d0 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97b157b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81be380 con 0x55b5e8ef3b40 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97ad73f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97ad73f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52222/0 (socket says 172.21.15.77:52222) 2024-09-15T22:33:26.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97ad73f640 1 -- 172.21.15.77:0/435355796 learned_addr learned my addr 172.21.15.77:0/435355796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:26.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97a7fff640 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.684+0000 7f97acf3e640 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef25d0 0x55b5e8eeed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 -- 172.21.15.77:0/435355796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 msgr2=0x55b5e8f092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 -- 172.21.15.77:0/435355796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef25d0 msgr2=0x55b5e8eeed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef25d0 0x55b5e8eeed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 -- 172.21.15.77:0/435355796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e8ef8570 con 0x55b5e8efd370 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f97ad73f640 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f97a00123d0 tx=0x7f97a0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51777f1fb29c8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f979e7fc640 1 -- 172.21.15.77:0/435355796 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0004050 con 0x55b5e8efd370 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f979e7fc640 1 -- 172.21.15.77:0/435355796 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a0026020 con 0x55b5e8efd370 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.685+0000 7f979e7fc640 1 -- 172.21.15.77:0/435355796 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0010690 con 0x55b5e8efd370 2024-09-15T22:33:26.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- 172.21.15.77:0/435355796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 msgr2=0x55b5e8eedbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f97a00123d0 tx=0x7f97a0009820 comp rx=0 tx=0).stop 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- 172.21.15.77:0/435355796 shutdown_connections 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8eedbf0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 --2- 172.21.15.77:0/435355796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef25d0 0x55b5e8eeed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- 172.21.15.77:0/435355796 >> 172.21.15.77:0/435355796 conn(0x55b5e8ebb310 msgr2=0x55b5e8eed4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- 172.21.15.77:0/435355796 shutdown_connections 2024-09-15T22:33:26.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- 172.21.15.77:0/435355796 wait complete. 2024-09-15T22:33:26.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.686+0000 7f97b157b740 1 -- start start 2024-09-15T22:33:26.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 Processor -- start 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8f003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8f00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81bf120 con 0x55b5e8efd370 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81bea50 con 0x55b5e8ef3b40 2024-09-15T22:33:26.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81bbcb0 con 0x55b5e8ef25d0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97b157b740 10 client.0 _ll_register_callbacks cb 0x7f9790084f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97acf3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8f003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51806/0 (socket says 172.21.15.77:51806) 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97a7fff640 1 -- 172.21.15.77:0/2088262976 learned_addr learned my addr 172.21.15.77:0/2088262976 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.687+0000 7f97ad73f640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8f00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 -- 172.21.15.77:0/2088262976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 msgr2=0x55b5e8f00910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 -- 172.21.15.77:0/2088262976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 msgr2=0x55b5e8f00e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8f00e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 -- 172.21.15.77:0/2088262976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e8f75660 con 0x55b5e8ef25d0 2024-09-15T22:33:26.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97acf3e640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8f003d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f9798003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ebe6fc41667247 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f9787fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979804c030 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f9798031450 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.688+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a0026020 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9798031650 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 10 client.29011 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b5e8f76900 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9798036040 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f979802f7d0 con 0x55b5e8ef25d0 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 10 client.29011 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:26.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 10 client.29011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 20 client.29011 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f76fd0 0x55b5e8f793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 4 client.29011 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f97a0026020 con 0x55b5e8f76fd0 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.689+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f76fd0 0x55b5e8f793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f9787fff640 10 client.29011 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97a7fff640 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f76fd0 0x55b5e8f793c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fcf29851c215d34d server_cookie=a6e4a352bdad25ba in_seq=0 out_seq=0 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f9787fff640 1 -- 172.21.15.77:0/2088262976 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f97a0026020 con 0x55b5e8f76fd0 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f9787fff640 10 client.29011 handle_command_reply: tid=0 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97b157b740 1 client.29011 shutdown 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 msgr2=0x55b5e8f003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8f003d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ca0 tx=0x7f9798003bf0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 shutdown_connections 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.690+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55b5e8f76fd0 0x55b5e8f793c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5e8efd370 0x55b5e8f00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5e8ef3b40 0x55b5e8f00910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 --2- 172.21.15.77:0/2088262976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b5e8ef25d0 0x55b5e8f003d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 >> 172.21.15.77:0/2088262976 conn(0x55b5e8ebb310 msgr2=0x55b5e8eedd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f9787fff640 0 client.29011 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 shutdown_connections 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 1 -- 172.21.15.77:0/2088262976 wait complete. 2024-09-15T22:33:26.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97b157b740 20 client.29011 trim_cache size 0 max 16384 2024-09-15T22:33:26.700 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:33:26.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 msgr2=0x7f97940a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 0x7f97940a5970 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f97a00058c0 tx=0x7f97a0004730 comp rx=0 tx=0).stop 2024-09-15T22:33:26.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 msgr2=0x7f97a81a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:26.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.691+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f979000d950 tx=0x7f979000de10 comp rx=0 tx=0).stop 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 shutdown_connections 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f97940a34b0 0x7f97940a5970 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97a811b140 0x7f97a81ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97a8109830 0x7f97a81a75a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 --2- 172.21.15.77:0/2487433220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a8109450 0x7f97a81a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 >> 172.21.15.77:0/2487433220 conn(0x7f97a806e140 msgr2=0x7f97a811c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 shutdown_connections 2024-09-15T22:33:26.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:26.692+0000 7f97af1a2640 1 -- 172.21.15.77:0/2487433220 wait complete. 2024-09-15T22:33:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:26 smithi196 ceph-mon[35109]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 393 op/s 2024-09-15T22:33:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:26 smithi117 ceph-mon[35021]: pgmap v663: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 393 op/s 2024-09-15T22:33:27.555 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:33:27.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 133288 14.70 MB/sec execute 328 sec latency 181.284 ms 2024-09-15T22:33:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:28.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 -- 172.21.15.77:0/1131788252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb0101df0 msgr2=0x7f8cb01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 --2- 172.21.15.77:0/1131788252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb0101df0 0x7f8cb01021d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca002a410 comp rx=0 tx=0).stop 2024-09-15T22:33:28.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 -- 172.21.15.77:0/1131788252 shutdown_connections 2024-09-15T22:33:28.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 --2- 172.21.15.77:0/1131788252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0106f20 0x7f8cb010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 --2- 172.21.15.77:0/1131788252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb01027a0 0x7f8cb01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 --2- 172.21.15.77:0/1131788252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb0101df0 0x7f8cb01021d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 -- 172.21.15.77:0/1131788252 >> 172.21.15.77:0/1131788252 conn(0x7f8cb00fd8f0 msgr2=0x7f8cb00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 -- 172.21.15.77:0/1131788252 shutdown_connections 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 -- 172.21.15.77:0/1131788252 wait complete. 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.377+0000 7f8cb6583640 1 Processor -- start 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 -- start start 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 0x7f8cb01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 0x7f8cb019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0111f80 con 0x7f8cb01027a0 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0111e00 con 0x7f8cb0106f20 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb6583640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0112100 con 0x7f8cb0101df0 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 0x7f8cb019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52266/0 (socket says 172.21.15.77:52266) 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8caf7fe640 1 -- 172.21.15.77:0/337121827 learned_addr learned my addr 172.21.15.77:0/337121827 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.378+0000 7f8caffff640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 0x7f8cb01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 -- 172.21.15.77:0/337121827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 msgr2=0x7f8cb01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 0x7f8cb01984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 -- 172.21.15.77:0/337121827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 msgr2=0x7f8cb019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 0x7f8cb019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 -- 172.21.15.77:0/337121827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb019d450 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caffff640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 0x7f8cb01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 0x7f8cb019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cad7fa640 1 -- 172.21.15.77:0/337121827 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400cd80 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb019d740 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cad7fa640 1 -- 172.21.15.77:0/337121827 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca4010910 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cad7fa640 1 -- 172.21.15.77:0/337121827 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40188f0 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.379+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb01a5020 con 0x7f8cb01027a0 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.383+0000 7f8cb895c740 1 Processor -- start 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.383+0000 7f8cb895c740 1 -- start start 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37a99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37a98560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b379fe2f0 con 0x557b37a98120 2024-09-15T22:33:28.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b379a5cf0 con 0x557b37a99840 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb895c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b379fa2b0 con 0x557b37a98b80 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cad7fa640 1 -- 172.21.15.77:0/337121827 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8ca4010ac0 con 0x7f8cb01027a0 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37a99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37a99c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51818/0 (socket says 172.21.15.77:51818) 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36492/0 (socket says 172.21.15.77:36492) 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8cb4b20640 1 -- 172.21.15.77:0/3382501856 learned_addr learned my addr 172.21.15.77:0/3382501856 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.384+0000 7f8caffff640 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37a98560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 -- 172.21.15.77:0/3382501856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 msgr2=0x557b37aadf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 -- 172.21.15.77:0/3382501856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 msgr2=0x557b37a99c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37a99c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 -- 172.21.15.77:0/3382501856 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3798a3a0 con 0x557b37a98120 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.385+0000 7f8caffff640 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37a98560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005940 tx=0x7f8ca0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d745a402ddc8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8cad7fa640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 0x7f8c980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8c967fc640 1 -- 172.21.15.77:0/3382501856 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0035040 con 0x557b37a98120 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 0x7f8c980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8cad7fa640 1 -- 172.21.15.77:0/337121827 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f8ca40d04b0 con 0x7f8cb01027a0 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8c967fc640 1 -- 172.21.15.77:0/3382501856 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0031830 con 0x557b37a98120 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.386+0000 7f8c967fc640 1 -- 172.21.15.77:0/3382501856 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0003d60 con 0x557b37a98120 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- 172.21.15.77:0/3382501856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 msgr2=0x557b37a98560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 0x7f8c980a5870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009fd0 tx=0x7f8ca8009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37a98560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0005940 tx=0x7f8ca0005d50 comp rx=0 tx=0).stop 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- 172.21.15.77:0/3382501856 shutdown_connections 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98b80 0x557b37aadf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37a98560 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3382501856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37a99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- 172.21.15.77:0/3382501856 >> 172.21.15.77:0/3382501856 conn(0x557b37a61310 msgr2=0x557b37a94c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- 172.21.15.77:0/3382501856 shutdown_connections 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- 172.21.15.77:0/3382501856 wait complete. 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 -- start start 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.387+0000 7f8cb895c740 1 Processor -- start 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b37aae8d0 con 0x557b37a98120 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557b379b9060 con 0x557b37a98b80 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b37aaea50 con 0x557b37a99840 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb895c740 10 client.0 _ll_register_callbacks cb 0x557b3770c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52270/0 (socket says 172.21.15.77:52270) 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51832/0 (socket says 172.21.15.77:51832) 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 -- 172.21.15.77:0/990767189 learned_addr learned my addr 172.21.15.77:0/990767189 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 -- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 msgr2=0x557b37aab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 -- 172.21.15.77:0/990767189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 msgr2=0x557b37aab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.388+0000 7f8caffff640 1 -- 172.21.15.77:0/990767189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3776f3c0 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8caffff640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daddaae4b28a0630 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8c8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca004e030 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f8ca0031630 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.389+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca0031830 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ca0058050 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 10 client.20376 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b37aba2e0 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0031830 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8ca00537b0 con 0x557b37a98120 2024-09-15T22:33:28.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 10 client.20376 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 10 client.20376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 20 client.20376 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37abb630 0x557b37abda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 4 client.20376 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.390+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b37a0b860 con 0x557b37abb630 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.391+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37abb630 0x557b37abda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.391+0000 7f8c8f7fe640 10 client.20376 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.391+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37abb630 0x557b37abda20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b19b4765ccc3bd server_cookie=1e5b1684e2daf23c in_seq=0 out_seq=0 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.392+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/990767189 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b37a0b860 con 0x557b37abb630 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.392+0000 7f8c8f7fe640 10 client.20376 handle_command_reply: tid=0 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.392+0000 7f8cb895c740 1 client.20376 shutdown 2024-09-15T22:33:28.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 msgr2=0x557b37aadda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca00038a0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 shutdown_connections 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37abb630 0x557b37abda20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98b80 0x557b37aab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 --2- 172.21.15.77:0/990767189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98120 0x557b37aadda0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 >> 172.21.15.77:0/990767189 conn(0x557b37a61310 msgr2=0x557b37a9a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.393+0000 7f8c8f7fe640 0 client.20376 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.394+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 shutdown_connections 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.394+0000 7f8cb895c740 1 -- 172.21.15.77:0/990767189 wait complete. 2024-09-15T22:33:28.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.394+0000 7f8cb895c740 20 client.20376 trim_cache size 0 max 16384 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 Processor -- start 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 -- start start 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98120 0x557b37ab7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37a9ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01ace80 con 0x557b37a98b80 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01ae9c0 con 0x557b37a98120 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.411+0000 7f8cb895c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01ad550 con 0x557b37a99840 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37a9ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:36518/0 (socket says 172.21.15.77:36518) 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 -- 172.21.15.77:0/3191456784 learned_addr learned my addr 172.21.15.77:0/3191456784 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8caffff640 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98120 0x557b37ab7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 -- 172.21.15.77:0/3191456784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98120 msgr2=0x557b37ab7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98120 0x557b37ab7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 -- 172.21.15.77:0/3191456784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 msgr2=0x557b37a9ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37a9ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 -- 172.21.15.77:0/3191456784 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b37aac040 con 0x557b37a99840 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37a9ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800f790 tx=0x7f8ca801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1e4ce5f81c9c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.412+0000 7f8c95ffb640 1 -- 172.21.15.77:0/3191456784 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80122d0 con 0x557b37a99840 2024-09-15T22:33:28.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.413+0000 7f8c95ffb640 1 -- 172.21.15.77:0/3191456784 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca8003970 con 0x557b37a99840 2024-09-15T22:33:28.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.413+0000 7f8c95ffb640 1 -- 172.21.15.77:0/3191456784 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004590 con 0x557b37a99840 2024-09-15T22:33:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.413+0000 7f8cb895c740 1 -- 172.21.15.77:0/3191456784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 msgr2=0x557b37aaa9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.413+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800f790 tx=0x7f8ca801a000 comp rx=0 tx=0).stop 2024-09-15T22:33:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 -- 172.21.15.77:0/3191456784 shutdown_connections 2024-09-15T22:33:28.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a99840 0x557b37aaa9a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37a9ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 --2- 172.21.15.77:0/3191456784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a98120 0x557b37ab7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 -- 172.21.15.77:0/3191456784 >> 172.21.15.77:0/3191456784 conn(0x557b37a61310 msgr2=0x557b37a93d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 -- 172.21.15.77:0/3191456784 shutdown_connections 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 -- 172.21.15.77:0/3191456784 wait complete. 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 -- start start 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.414+0000 7f8cb895c740 1 Processor -- start 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 0x557b37ac21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37ac2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01ae2f0 con 0x557b37a98b80 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01adc20 con 0x557b37a99840 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01aae80 con 0x557b37a98120 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb895c740 10 client.0 _ll_register_callbacks cb 0x7f8ca0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8caffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 0x557b37ac21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb4b20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51842/0 (socket says 172.21.15.77:51842) 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8cb4b20640 1 -- 172.21.15.77:0/1586559241 learned_addr learned my addr 172.21.15.77:0/1586559241 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.415+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37ac2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 -- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 msgr2=0x557b37ac21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 0x557b37ac21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 -- 172.21.15.77:0/1586559241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 msgr2=0x557b37ac2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:28.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37ac2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 -- 172.21.15.77:0/1586559241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b37ab3220 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8caf7fe640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37ac2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8caffff640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 0x557b37ac21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb4b20640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8012800 tx=0x7f8ca80122d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aacef6bb483e36c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8c8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004a40 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f8ca8004be0 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.416+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca8003970 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ca803c050 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 10 client.29102 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557b37ab4520 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca8003970 con 0x557b37a99840 2024-09-15T22:33:28.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8ca8031020 con 0x557b37a99840 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 10 client.29102 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 10 client.29102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 20 client.29102 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37b26020 0x557b37b28410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 4 client.29102 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.417+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b37a05d70 con 0x557b37b26020 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.418+0000 7f8caffff640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37b26020 0x557b37b28410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.418+0000 7f8c8f7fe640 10 client.29102 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.418+0000 7f8caffff640 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37b26020 0x557b37b28410 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=272f71b59ae76a75 server_cookie=eeb6293eb7c6ae8f in_seq=0 out_seq=0 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8c8f7fe640 1 -- 172.21.15.77:0/1586559241 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b37a05d70 con 0x557b37b26020 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8c8f7fe640 10 client.29102 handle_command_reply: tid=0 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 client.29102 shutdown 2024-09-15T22:33:28.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 msgr2=0x557b37ab2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8012800 tx=0x7f8ca80122d0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 shutdown_connections 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x557b37b26020 0x557b37b28410 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557b37a99840 0x557b37ab2ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b37a98b80 0x557b37ac2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 --2- 172.21.15.77:0/1586559241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b37a98120 0x557b37ac21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 >> 172.21.15.77:0/1586559241 conn(0x557b37a61310 msgr2=0x557b37a9d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8c8f7fe640 0 client.29102 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 shutdown_connections 2024-09-15T22:33:28.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 1 -- 172.21.15.77:0/1586559241 wait complete. 2024-09-15T22:33:28.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.419+0000 7f8cb895c740 20 client.29102 trim_cache size 0 max 16384 2024-09-15T22:33:28.427 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:33:28.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 msgr2=0x7f8c980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 0x7f8c980a5870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009fd0 tx=0x7f8ca8009500 comp rx=0 tx=0).stop 2024-09-15T22:33:28.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 msgr2=0x7f8cb01989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 shutdown_connections 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c980a33b0 0x7f8c980a5870 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cb0106f20 0x7f8cb019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb01027a0 0x7f8cb01989e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 --2- 172.21.15.77:0/337121827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8cb0101df0 0x7f8cb01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 >> 172.21.15.77:0/337121827 conn(0x7f8cb00fd8f0 msgr2=0x7f8cb00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 shutdown_connections 2024-09-15T22:33:28.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:28.420+0000 7f8cb6583640 1 -- 172.21.15.77:0/337121827 wait complete. 2024-09-15T22:33:28.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:28 smithi077 ceph-mon[30980]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 519 op/s 2024-09-15T22:33:28.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 133779 14.77 MB/sec execute 329 sec latency 226.835 ms 2024-09-15T22:33:28.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:28 smithi196 ceph-mon[35109]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 519 op/s 2024-09-15T22:33:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:28 smithi117 ceph-mon[35021]: pgmap v664: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 519 op/s 2024-09-15T22:33:29.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:33:29.501 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:29.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 134127 14.76 MB/sec execute 330 sec latency 194.347 ms 2024-09-15T22:33:30.571 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:30 smithi077 ceph-mon[30980]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 341 op/s 2024-09-15T22:33:30.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 134340 14.74 MB/sec execute 331 sec latency 495.653 ms 2024-09-15T22:33:30.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:30 smithi196 ceph-mon[35109]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 341 op/s 2024-09-15T22:33:30.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:30 smithi117 ceph-mon[35021]: pgmap v665: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 341 op/s 2024-09-15T22:33:31.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 134638 14.74 MB/sec execute 332 sec latency 542.559 ms 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.448+0000 7fbb46d6e640 1 -- 172.21.15.77:0/1671254429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb40109450 msgr2=0x7fbb40109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.448+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/1671254429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb40109450 0x7fbb40109830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb300092e0 tx=0x7fbb3002a2b0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.451+0000 7fbb46d6e640 1 -- 172.21.15.77:0/1671254429 shutdown_connections 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.451+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/1671254429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb4010ea80 0x7fbb401154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.451+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/1671254429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb40109e00 0x7fbb4010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.451+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/1671254429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb40109450 0x7fbb40109830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.451+0000 7fbb46d6e640 1 -- 172.21.15.77:0/1671254429 >> 172.21.15.77:0/1671254429 conn(0x7fbb4006e2d0 msgr2=0x7fbb4006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.454+0000 7fbb46d6e640 1 -- 172.21.15.77:0/1671254429 shutdown_connections 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.455+0000 7fbb46d6e640 1 -- 172.21.15.77:0/1671254429 wait complete. 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.456+0000 7fbb46d6e640 1 Processor -- start 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.458+0000 7fbb46d6e640 1 -- start start 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb40109450 0x7fbb401a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb4010ea80 0x7fbb4019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401185c0 con 0x7fbb40109e00 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40118440 con 0x7fbb4010ea80 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb46d6e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40118740 con 0x7fbb40109450 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44876/0 (socket says 172.21.15.77:44876) 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.459+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2206765626 learned_addr learned my addr 172.21.15.77:0/2206765626 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2206765626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb40109450 msgr2=0x7fbb401a4770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb40109450 0x7fbb401a4770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2206765626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb4010ea80 msgr2=0x7fbb4019e930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb4010ea80 0x7fbb4019e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2206765626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019f060 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.460+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400fb00 tx=0x7fbb34009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.463+0000 7fbb3dffb640 1 -- 172.21.15.77:0/2206765626 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3401c070 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.463+0000 7fbb3dffb640 1 -- 172.21.15.77:0/2206765626 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3400dc90 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.463+0000 7fbb3dffb640 1 -- 172.21.15.77:0/2206765626 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400c6c0 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.463+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019f350 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.464+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401ab640 con 0x7fbb40109e00 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.471+0000 7fbb49147740 1 Processor -- start 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 -- start start 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37bfde0 0x5580a37c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5580a371d2f0 con 0x5580a37bfde0 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580a36c4cf0 con 0x5580a37cab80 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.479+0000 7fbb49147740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5580a37192b0 con 0x5580a37c1350 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.480+0000 7fbb4530b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.480+0000 7fbb4530b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45044/0 (socket says 172.21.15.77:45044) 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.480+0000 7fbb4530b640 1 -- 172.21.15.77:0/3549424569 learned_addr learned my addr 172.21.15.77:0/3549424569 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.482+0000 7fbb4530b640 1 -- 172.21.15.77:0/3549424569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 msgr2=0x5580a37ccf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.482+0000 7fbb4530b640 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ccf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.482+0000 7fbb4530b640 1 -- 172.21.15.77:0/3549424569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37bfde0 msgr2=0x5580a37c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.482+0000 7fbb4530b640 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37bfde0 0x5580a37c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.482+0000 7fbb4530b640 1 -- 172.21.15.77:0/3549424569 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a36a93a0 con 0x5580a37c1350 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.484+0000 7fbb3dffb640 1 -- 172.21.15.77:0/2206765626 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbb340178f0 con 0x7fbb40109e00 2024-09-15T22:33:31.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.485+0000 7fbb3dffb640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 0x7fbb280a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.486+0000 7fbb3dffb640 1 -- 172.21.15.77:0/2206765626 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7fbb340cffa0 con 0x7fbb40109e00 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.486+0000 7fbb4530b640 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009d00 tx=0x7fbb3800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4161d1bff63258 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.487+0000 7fbb26ffd640 1 -- 172.21.15.77:0/3549424569 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800bd40 con 0x5580a37c1350 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.487+0000 7fbb26ffd640 1 -- 172.21.15.77:0/3549424569 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38017070 con 0x5580a37c1350 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.487+0000 7fbb26ffd640 1 -- 172.21.15.77:0/3549424569 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38011700 con 0x5580a37c1350 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.487+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 0x7fbb280a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.490+0000 7fbb49147740 1 -- 172.21.15.77:0/3549424569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 msgr2=0x5580a37bf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.490+0000 7fbb49147740 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009d00 tx=0x7fbb3800b420 comp rx=0 tx=0).stop 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.490+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 0x7fbb280a59c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbb30035040 tx=0x7fbb30005a90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.494+0000 7fbb49147740 1 -- 172.21.15.77:0/3549424569 shutdown_connections 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.494+0000 7fbb49147740 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.494+0000 7fbb49147740 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37bfde0 0x5580a37c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.494+0000 7fbb49147740 1 --2- 172.21.15.77:0/3549424569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37c1350 0x5580a37bf8a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.494+0000 7fbb49147740 1 -- 172.21.15.77:0/3549424569 >> 172.21.15.77:0/3549424569 conn(0x5580a3780310 msgr2=0x5580a37bc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.502+0000 7fbb49147740 1 -- 172.21.15.77:0/3549424569 shutdown_connections 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.505+0000 7fbb49147740 1 -- 172.21.15.77:0/3549424569 wait complete. 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.506+0000 7fbb49147740 1 -- start start 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.506+0000 7fbb49147740 1 Processor -- start 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37cab80 0x5580a37c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5580a37cd880 con 0x5580a37c1350 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5580a36d8060 con 0x5580a37bfde0 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5580a37cda00 con 0x5580a37cab80 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.507+0000 7fbb49147740 10 client.0 _ll_register_callbacks cb 0x5580a342b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.510+0000 7fbb44b0a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.510+0000 7fbb44b0a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39210/0 (socket says 172.21.15.77:39210) 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.510+0000 7fbb44b0a640 1 -- 172.21.15.77:0/2097075321 learned_addr learned my addr 172.21.15.77:0/2097075321 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb4530b640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37cab80 0x5580a37c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb44b0a640 1 -- 172.21.15.77:0/2097075321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37cab80 msgr2=0x5580a37c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37cab80 0x5580a37c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb44b0a640 1 -- 172.21.15.77:0/2097075321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 msgr2=0x5580a37c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.512+0000 7fbb44b0a640 1 -- 172.21.15.77:0/2097075321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a348e3c0 con 0x5580a37bfde0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.513+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002a8a0 tx=0x7fbb30005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e82075cbb5acef90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.514+0000 7fbb1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.515+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30037040 con 0x5580a37bfde0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.515+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7fbb30030930 con 0x5580a37bfde0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.519+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb38017070 con 0x5580a37bfde0 2024-09-15T22:33:31.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.521+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbb30005050 con 0x5580a37bfde0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.523+0000 7fbb49147740 10 client.29108 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.524+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5580a37d9430 con 0x5580a37bfde0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38017070 con 0x5580a37bfde0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fbb30046070 con 0x5580a37bfde0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 10 client.29108 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 10 client.29108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 20 client.29108 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a37da8f0 0x5580a37dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 4 client.29108 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.526+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580a372a860 con 0x5580a37da8f0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.531+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a37da8f0 0x5580a37dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.534+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a37da8f0 0x5580a37dcd20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8a1abacb99843a6 server_cookie=931842e46f40a5fa in_seq=0 out_seq=0 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.535+0000 7fbb1ffff640 10 client.29108 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:33:31.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.536+0000 7fbb1ffff640 1 -- 172.21.15.77:0/2097075321 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580a372a860 con 0x5580a37da8f0 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.536+0000 7fbb1ffff640 10 client.29108 handle_command_reply: tid=0 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.536+0000 7fbb49147740 1 client.29108 shutdown 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.540+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 msgr2=0x5580a37ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.540+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002a8a0 tx=0x7fbb30005f90 comp rx=0 tx=0).stop 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 shutdown_connections 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a37da8f0 0x5580a37dcd20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37cab80 0x5580a37c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 --2- 172.21.15.77:0/2097075321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37bfde0 0x5580a37ca610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 >> 172.21.15.77:0/2097075321 conn(0x5580a3780310 msgr2=0x5580a37c2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.541+0000 7fbb1ffff640 0 client.29108 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:33:31.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.543+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 shutdown_connections 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.544+0000 7fbb49147740 1 -- 172.21.15.77:0/2097075321 wait complete. 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.544+0000 7fbb49147740 20 client.29108 trim_cache size 0 max 16384 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 Processor -- start 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 -- start start 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b5790 con 0x5580a37c1350 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b72d0 con 0x5580a37cab80 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.570+0000 7fbb49147740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b5e60 con 0x5580a37bfde0 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.571+0000 7fbb44b0a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.571+0000 7fbb4530b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.571+0000 7fbb4530b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:39216/0 (socket says 172.21.15.77:39216) 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.571+0000 7fbb4530b640 1 -- 172.21.15.77:0/2271663609 learned_addr learned my addr 172.21.15.77:0/2271663609 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.571+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2271663609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 msgr2=0x5580a37bc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37bc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2271663609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 msgr2=0x5580a37e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb3ffff640 1 -- 172.21.15.77:0/2271663609 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a37c5e10 con 0x5580a37c1350 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb4530b640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.572+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.573+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c68a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cd360 tx=0x7fbb34017c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0ccde2df10109c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.574+0000 7fbb267fc640 1 -- 172.21.15.77:0/2271663609 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340a4b60 con 0x5580a37c1350 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.574+0000 7fbb267fc640 1 -- 172.21.15.77:0/2271663609 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3409a1f0 con 0x5580a37c1350 2024-09-15T22:33:31.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.574+0000 7fbb267fc640 1 -- 172.21.15.77:0/2271663609 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340bad90 con 0x5580a37c1350 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.575+0000 7fbb49147740 1 -- 172.21.15.77:0/2271663609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 msgr2=0x5580a37c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.575+0000 7fbb49147740 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c68a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cd360 tx=0x7fbb34017c60 comp rx=0 tx=0).stop 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.576+0000 7fbb49147740 1 -- 172.21.15.77:0/2271663609 shutdown_connections 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37c68a0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 --2- 172.21.15.77:0/2271663609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37bc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 -- 172.21.15.77:0/2271663609 >> 172.21.15.77:0/2271663609 conn(0x5580a3780310 msgr2=0x5580a37c6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 -- 172.21.15.77:0/2271663609 shutdown_connections 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 -- 172.21.15.77:0/2271663609 wait complete. 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 -- start start 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.577+0000 7fbb49147740 1 Processor -- start 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b6c00 con 0x5580a37c1350 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b6530 con 0x5580a37cab80 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b3790 con 0x5580a37bfde0 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.578+0000 7fbb49147740 10 client.0 _ll_register_callbacks cb 0x7fbb30035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44900/0 (socket says 172.21.15.77:44900) 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 -- 172.21.15.77:0/1376802807 learned_addr learned my addr 172.21.15.77:0/1376802807 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb4530b640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 -- 172.21.15.77:0/1376802807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 msgr2=0x5580a37d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37d3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 -- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 msgr2=0x5580a37ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.580+0000 7fbb3ffff640 1 -- 172.21.15.77:0/1376802807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a37cf140 con 0x5580a37c1350 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.581+0000 7fbb3ffff640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cd960 tx=0x7fbb3409c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca38a2836fa714d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.581+0000 7fbb1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3409c5f0 con 0x5580a37c1350 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7fbb340bab30 con 0x5580a37c1350 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3409a1f0 con 0x5580a37c1350 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbb340a4b60 con 0x5580a37c1350 2024-09-15T22:33:31.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb49147740 10 client.20391 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.582+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5580a3845410 con 0x5580a37c1350 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.583+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3409a1f0 con 0x5580a37c1350 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.583+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fbb34097c40 con 0x5580a37c1350 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 10 client.20391 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 10 client.20391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 20 client.20391 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a3845a50 0x5580a3847e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 4 client.20391 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.584+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580a3724d70 con 0x5580a3845a50 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.586+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a3845a50 0x5580a3847e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.590+0000 7fbb1ffff640 10 client.20391 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.590+0000 7fbb44b0a640 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a3845a50 0x5580a3847e70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a78ba63ee4769b6 server_cookie=c6ebd7812f76aad2 in_seq=0 out_seq=0 2024-09-15T22:33:31.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb1ffff640 1 -- 172.21.15.77:0/1376802807 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580a3724d70 con 0x5580a3845a50 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb1ffff640 10 client.20391 handle_command_reply: tid=0 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 client.20391 shutdown 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 msgr2=0x5580a37ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbb340cd960 tx=0x7fbb3409c070 comp rx=0 tx=0).stop 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 shutdown_connections 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5580a3845a50 0x5580a3847e70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580a37cab80 0x5580a37ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580a37c1350 0x5580a37ce5c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 --2- 172.21.15.77:0/1376802807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5580a37bfde0 0x5580a37d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb1ffff640 0 client.20391 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 >> 172.21.15.77:0/1376802807 conn(0x5580a3780310 msgr2=0x5580a37c1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 shutdown_connections 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.591+0000 7fbb49147740 1 -- 172.21.15.77:0/1376802807 wait complete. 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.592+0000 7fbb49147740 20 client.20391 trim_cache size 0 max 16384 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 msgr2=0x7fbb280a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 0x7fbb280a59c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbb30035040 tx=0x7fbb30005a90 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 msgr2=0x7fbb401a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400fb00 tx=0x7fbb34009e20 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 shutdown_connections 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbb280a3500 0x7fbb280a59c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb4010ea80 0x7fbb4019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb40109e00 0x7fbb401a4cb0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 --2- 172.21.15.77:0/2206765626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb40109450 0x7fbb401a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 >> 172.21.15.77:0/2206765626 conn(0x7fbb4006e2d0 msgr2=0x7fbb4010c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 shutdown_connections 2024-09-15T22:33:31.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:30.593+0000 7fbb46d6e640 1 -- 172.21.15.77:0/2206765626 wait complete. 2024-09-15T22:33:32.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 135119 14.77 MB/sec execute 333 sec latency 149.136 ms 2024-09-15T22:33:32.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:32 smithi077 ceph-mon[30980]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 392 op/s 2024-09-15T22:33:32.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:32 smithi196 ceph-mon[35109]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 392 op/s 2024-09-15T22:33:32.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:32 smithi117 ceph-mon[35021]: pgmap v666: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 392 op/s 2024-09-15T22:33:33.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:33:33.632 INFO:tasks.workunit.client.0.smithi077.stdout: 10 135693 14.84 MB/sec execute 334 sec latency 135.952 ms 2024-09-15T22:33:33.971 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:34.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 136114 14.86 MB/sec execute 335 sec latency 137.817 ms 2024-09-15T22:33:34.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:34 smithi077 ceph-mon[30980]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 468 op/s 2024-09-15T22:33:34.681 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:34 smithi196 ceph-mon[35109]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 468 op/s 2024-09-15T22:33:34.850 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:34 smithi117 ceph-mon[35021]: pgmap v667: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 468 op/s 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 -- 172.21.15.77:0/2863858648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8100e20 msgr2=0x7f4fa8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2863858648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8100e20 0x7f4fa8101200 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009900 tx=0x7f4fa402a480 comp rx=0 tx=0).stop 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 -- 172.21.15.77:0/2863858648 shutdown_connections 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2863858648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8102ef0 0x7f4fa810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2863858648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8102570 0x7f4fa81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2863858648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8100e20 0x7f4fa8101200 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 -- 172.21.15.77:0/2863858648 >> 172.21.15.77:0/2863858648 conn(0x7f4fa80fc7d0 msgr2=0x7f4fa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 -- 172.21.15.77:0/2863858648 shutdown_connections 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.553+0000 7f4fb0361640 1 -- 172.21.15.77:0/2863858648 wait complete. 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 Processor -- start 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 -- start start 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8100e20 0x7f4fa8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8102ef0 0x7f4fa819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8110f80 con 0x7f4fa8102570 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8110e00 con 0x7f4fa8102ef0 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fb0361640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8111100 con 0x7f4fa8100e20 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44914/0 (socket says 172.21.15.77:44914) 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fae0fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8100e20 0x7f4fa8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2525991141 learned_addr learned my addr 172.21.15.77:0/2525991141 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2525991141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8100e20 msgr2=0x7f4fa8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8100e20 0x7f4fa8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2525991141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8102ef0 msgr2=0x7f4fa819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8102ef0 0x7f4fa819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2525991141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa819d510 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.554+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d770 tx=0x7f4f9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.555+0000 7f4f9f7fe640 1 -- 172.21.15.77:0/2525991141 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98017070 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.555+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa819d800 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.555+0000 7f4f9f7fe640 1 -- 172.21.15.77:0/2525991141 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f98005500 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.555+0000 7f4f9f7fe640 1 -- 172.21.15.77:0/2525991141 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004e00 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.555+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81a50e0 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.557+0000 7f4f9f7fe640 1 -- 172.21.15.77:0/2525991141 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f98004fa0 con 0x7f4fa8102570 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.557+0000 7f4fb273a740 1 Processor -- start 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 -- start start 2024-09-15T22:33:35.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e860048e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e85ffaa2f0 con 0x55e860053780 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e85ff51cf0 con 0x55e8600489e0 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fb273a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e85ffa62b0 con 0x55e860049f50 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44930/0 (socket says 172.21.15.77:44930) 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2423833317 learned_addr learned my addr 172.21.15.77:0/2423833317 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e860048e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4f9f7fe640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 0x7f4f940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2423833317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 msgr2=0x55e8600484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2423833317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 msgr2=0x55e860048e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e860048e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.558+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2423833317 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85ff363a0 con 0x55e860053780 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 0x7f4f940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4f9f7fe640 1 -- 172.21.15.77:0/2525991141 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4f980cfb30 con 0x7f4fa8102570 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4f980cce60 tx=0x7f4f98066470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c82c2cd27a8f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 0x7f4f940a5870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005c40 tx=0x7f4fa0004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4f9cdea640 1 -- 172.21.15.77:0/2423833317 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9809b5c0 con 0x55e860053780 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4f9cdea640 1 -- 172.21.15.77:0/2423833317 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f980046c0 con 0x55e860053780 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.559+0000 7f4f9cdea640 1 -- 172.21.15.77:0/2423833317 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9809b3e0 con 0x55e860053780 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- 172.21.15.77:0/2423833317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 msgr2=0x55e860055b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4f980cce60 tx=0x7f4f98066470 comp rx=0 tx=0).stop 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- 172.21.15.77:0/2423833317 shutdown_connections 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e860055b70 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e860048e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2423833317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- 172.21.15.77:0/2423833317 >> 172.21.15.77:0/2423833317 conn(0x55e86000d310 msgr2=0x55e860045070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- 172.21.15.77:0/2423833317 shutdown_connections 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- 172.21.15.77:0/2423833317 wait complete. 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 -- start start 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.560+0000 7f4fb273a740 1 Processor -- start 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860053780 0x55e8600508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e8600563c0 con 0x55e860049f50 2024-09-15T22:33:35.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e85ff65060 con 0x55e8600489e0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e860056540 con 0x55e860053780 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 10 client.0 _ll_register_callbacks cb 0x55e85fcb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44944/0 (socket says 172.21.15.77:44944) 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2372456779 learned_addr learned my addr 172.21.15.77:0/2372456779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860053780 0x55e8600508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2372456779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860053780 msgr2=0x55e8600508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860053780 0x55e8600508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 msgr2=0x55e8600530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2372456779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85fd1b3c0 con 0x55e860049f50 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4f9809b780 tx=0x7f4f980cfe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64af2d4cbed2ed22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:35.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98020070 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4f9806ca70 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e85ffb7860 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f9809b3e0 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 10 client.20403 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.561+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e860061dd0 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.562+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e85ffb7860 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.562+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4f9806cce0 con 0x55e860049f50 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 10 client.20403 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 10 client.20403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 20 client.20403 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e860063120 0x55e860065510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 4 client.20403 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e85ffb1d70 con 0x55e860063120 2024-09-15T22:33:35.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.563+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e860063120 0x55e860065510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.564+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e860063120 0x55e860065510 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9467cfcf055b629b server_cookie=a4ea9ebd3b0f4ffc in_seq=0 out_seq=0 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.564+0000 7f4f91ffb640 10 client.20403 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2372456779 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e85ffb1d70 con 0x55e860063120 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4f91ffb640 10 client.20403 handle_command_reply: tid=0 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 client.20403 shutdown 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 msgr2=0x55e860050380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4f9809b780 tx=0x7f4f980cfe50 comp rx=0 tx=0).stop 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 shutdown_connections 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e860063120 0x55e860065510 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860053780 0x55e8600508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860049f50 0x55e860050380 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2372456779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 >> 172.21.15.77:0/2372456779 conn(0x55e86000d310 msgr2=0x55e86004ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.566+0000 7f4f91ffb640 0 client.20403 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.567+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 shutdown_connections 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.567+0000 7f4fb273a740 1 -- 172.21.15.77:0/2372456779 wait complete. 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.567+0000 7f4fb273a740 20 client.20403 trim_cache size 0 max 16384 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.585+0000 7f4fb273a740 1 Processor -- start 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 -- start start 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e86004f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e86004b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81acd20 con 0x55e860053780 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81ae860 con 0x55e8600489e0 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.588+0000 7f4fb273a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81ad3f0 con 0x55e860049f50 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.589+0000 7f4fae8fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.589+0000 7f4fae8fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44954/0 (socket says 172.21.15.77:44954) 2024-09-15T22:33:35.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.589+0000 7f4fae8fe640 1 -- 172.21.15.77:0/3437795950 learned_addr learned my addr 172.21.15.77:0/3437795950 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.591+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e86004f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.596+0000 7f4fae8fe640 1 -- 172.21.15.77:0/3437795950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 msgr2=0x55e86004b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.596+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e86004b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.596+0000 7f4fae8fe640 1 -- 172.21.15.77:0/3437795950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 msgr2=0x55e86004f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.596+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e86004f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.596+0000 7f4fae8fe640 1 -- 172.21.15.77:0/3437795950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e860050c50 con 0x55e860053780 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.599+0000 7f4fae8fe640 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0004820 tx=0x7f4fa0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c18d792d5823d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.606+0000 7f4f93fff640 1 -- 172.21.15.77:0/3437795950 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa001f630 con 0x55e860053780 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.606+0000 7f4f93fff640 1 -- 172.21.15.77:0/3437795950 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa0016070 con 0x55e860053780 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.607+0000 7f4fb273a740 1 -- 172.21.15.77:0/3437795950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 msgr2=0x55e86004ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.607+0000 7f4fb273a740 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0004820 tx=0x7f4fa0014000 comp rx=0 tx=0).stop 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.611+0000 7f4fb273a740 1 -- 172.21.15.77:0/3437795950 shutdown_connections 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.611+0000 7f4fb273a740 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e86004ba70 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.611+0000 7f4fb273a740 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e86004b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.611+0000 7f4fb273a740 1 --2- 172.21.15.77:0/3437795950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e86004f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.611+0000 7f4fb273a740 1 -- 172.21.15.77:0/3437795950 >> 172.21.15.77:0/3437795950 conn(0x55e86000d310 msgr2=0x55e860052480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.615+0000 7f4fb273a740 1 -- 172.21.15.77:0/3437795950 shutdown_connections 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.617+0000 7f4fb273a740 1 -- 172.21.15.77:0/3437795950 wait complete. 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.617+0000 7f4fb273a740 1 -- start start 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 Processor -- start 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e8600c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81ae190 con 0x55e860053780 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81adac0 con 0x55e8600489e0 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81aad20 con 0x55e860049f50 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fb273a740 10 client.0 _ll_register_callbacks cb 0x7f4fa4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.618+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:45100/0 (socket says 172.21.15.77:45100) 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2997216654 learned_addr learned my addr 172.21.15.77:0/2997216654 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 msgr2=0x55e8600c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2997216654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 msgr2=0x55e8600c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e8600c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.619+0000 7f4fad8fc640 1 -- 172.21.15.77:0/2997216654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e860043900 con 0x55e860049f50 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.627+0000 7f4fad8fc640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f98097ee0 tx=0x7f4f980d3380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46b9711956a5567 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.627+0000 7f4f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.627+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98020070 con 0x55e860049f50 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.627+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa0016070 con 0x55e860049f50 2024-09-15T22:33:35.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.631+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4f980860b0 con 0x55e860049f50 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.631+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f9809b5c0 con 0x55e860049f50 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.635+0000 7f4fb273a740 10 client.29059 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.635+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e86005ac70 con 0x55e860049f50 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa0016070 con 0x55e860049f50 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4f980980f0 con 0x55e860049f50 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 10 client.29059 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 10 client.29059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 20 client.29059 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e86005b290 0x55e86005b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 4 client.29059 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.639+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e85ff2e170 con 0x55e86005b290 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.640+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e86005b290 0x55e86005b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.658+0000 7f4fae0fd640 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e86005b290 0x55e86005b6a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d81456dade6c24f server_cookie=61ddacc0b97eae5 in_seq=0 out_seq=0 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4f91ffb640 10 client.29059 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:35.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4f91ffb640 1 -- 172.21.15.77:0/2997216654 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e85ff2e170 con 0x55e86005b290 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4f91ffb640 10 client.29059 handle_command_reply: tid=0 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4fb273a740 1 client.29059 shutdown 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 msgr2=0x55e8600c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.659+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f98097ee0 tx=0x7f4f980d3380 comp rx=0 tx=0).stop 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 shutdown_connections 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e86005b290 0x55e86005b6a0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e860053780 0x55e8600c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e860049f50 0x55e8600c75c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 --2- 172.21.15.77:0/2997216654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8600489e0 0x55e8600c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 >> 172.21.15.77:0/2997216654 conn(0x55e86000d310 msgr2=0x55e86003fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.475 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4f91ffb640 0 client.29059 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 shutdown_connections 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 1 -- 172.21.15.77:0/2997216654 wait complete. 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.663+0000 7f4fb273a740 20 client.29059 trim_cache size 0 max 16384 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.664+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 msgr2=0x7f4f940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.664+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 0x7f4f940a5870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005c40 tx=0x7f4fa0004a40 comp rx=0 tx=0).stop 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.664+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 msgr2=0x7f4fa8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.664+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d770 tx=0x7f4f9800dc30 comp rx=0 tx=0).stop 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 shutdown_connections 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f940a33b0 0x7f4f940a5870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa8102ef0 0x7f4fa819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fa8102570 0x7f4fa8198aa0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 --2- 172.21.15.77:0/2525991141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fa8100e20 0x7f4fa8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.666+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 >> 172.21.15.77:0/2525991141 conn(0x7f4fa80fc7d0 msgr2=0x7f4fa806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.667+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 shutdown_connections 2024-09-15T22:33:35.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:34.668+0000 7f4fb0361640 1 -- 172.21.15.77:0/2525991141 wait complete. 2024-09-15T22:33:35.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 136303 14.84 MB/sec execute 336 sec latency 387.482 ms 2024-09-15T22:33:36.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 136648 14.85 MB/sec execute 337 sec latency 224.090 ms 2024-09-15T22:33:36.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:36 smithi077 ceph-mon[30980]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 350 op/s 2024-09-15T22:33:36.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:36 smithi196 ceph-mon[35109]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 350 op/s 2024-09-15T22:33:36.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:36 smithi117 ceph-mon[35021]: pgmap v668: 129 pgs: 129 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 350 op/s 2024-09-15T22:33:37.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 136960 14.84 MB/sec execute 338 sec latency 274.536 ms 2024-09-15T22:33:38.545 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:33:38.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:33:38.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 137447 14.88 MB/sec execute 339 sec latency 252.080 ms 2024-09-15T22:33:38.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:38 smithi077 ceph-mon[30980]: pgmap v669: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 551 op/s 2024-09-15T22:33:38.847 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:38.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:38 smithi196 ceph-mon[35109]: pgmap v669: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 551 op/s 2024-09-15T22:33:38.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:38 smithi117 ceph-mon[35021]: pgmap v669: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 551 op/s 2024-09-15T22:33:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.394+0000 7f5002cfa640 1 -- 172.21.15.77:0/3964117298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc100e20 msgr2=0x7f4ffc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.394+0000 7f5002cfa640 1 --2- 172.21.15.77:0/3964117298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc100e20 0x7f4ffc101200 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002a480 comp rx=0 tx=0).stop 2024-09-15T22:33:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.394+0000 7f5002cfa640 1 -- 172.21.15.77:0/3964117298 shutdown_connections 2024-09-15T22:33:39.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.394+0000 7f5002cfa640 1 --2- 172.21.15.77:0/3964117298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.394+0000 7f5002cfa640 1 --2- 172.21.15.77:0/3964117298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102570 0x7f4ffc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 --2- 172.21.15.77:0/3964117298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc100e20 0x7f4ffc101200 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 -- 172.21.15.77:0/3964117298 >> 172.21.15.77:0/3964117298 conn(0x7f4ffc0fc7d0 msgr2=0x7f4ffc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 -- 172.21.15.77:0/3964117298 shutdown_connections 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 -- 172.21.15.77:0/3964117298 wait complete. 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 Processor -- start 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.395+0000 7f5002cfa640 1 -- start start 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc102570 0x7f4ffc197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc110f50 con 0x7f4ffc102570 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc110dd0 con 0x7f4ffc102ef0 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5002cfa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1110d0 con 0x7f4ffc100e20 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5000a96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41842/0 (socket says 172.21.15.77:41842) 2024-09-15T22:33:39.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 -- 172.21.15.77:0/4047914968 learned_addr learned my addr 172.21.15.77:0/4047914968 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:39.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5000a96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44554/0 (socket says 172.21.15.77:44554) 2024-09-15T22:33:39.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc102570 0x7f4ffc197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 -- 172.21.15.77:0/4047914968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 msgr2=0x7f4ffc197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 -- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc102570 msgr2=0x7f4ffc197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc102570 0x7f4ffc197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5001297640 1 -- 172.21.15.77:0/4047914968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc1a69c0 con 0x7f4ffc102ef0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.396+0000 7f5000a96640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f5001297640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400eb10 tx=0x7f4ff400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f4ff9ffb640 1 -- 172.21.15.77:0/4047914968 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4004b20 con 0x7f4ffc102ef0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f4ff9ffb640 1 -- 172.21.15.77:0/4047914968 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff4004cc0 con 0x7f4ffc102ef0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc1a6cb0 con 0x7f4ffc102ef0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f4ff9ffb640 1 -- 172.21.15.77:0/4047914968 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4010760 con 0x7f4ffc102ef0 2024-09-15T22:33:39.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.397+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc1a71a0 con 0x7f4ffc102ef0 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.400+0000 7f4ff9ffb640 1 -- 172.21.15.77:0/4047914968 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ff4005020 con 0x7f4ffc102ef0 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.400+0000 7f50050d3740 1 Processor -- start 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.400+0000 7f50050d3740 1 -- start start 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aea9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933aea9d40 0x55933aeac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55933ae04280 con 0x55933ae9eb60 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55933adabb10 con 0x55933aea9d40 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f50050d3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55933ae00240 con 0x55933ae9f480 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54964/0 (socket says 172.21.15.77:54964) 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ffbfff640 1 -- 172.21.15.77:0/82726150 learned_addr learned my addr 172.21.15.77:0/82726150 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ff9ffb640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f5000a96640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aea9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933aea9d40 0x55933aeac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f4ff9ffb640 1 -- 172.21.15.77:0/4047914968 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4ff40d0790 con 0x7f4ffc102ef0 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.401+0000 7f5001297640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 -- 172.21.15.77:0/82726150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933aea9d40 msgr2=0x55933aeac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933aea9d40 0x55933aeac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 -- 172.21.15.77:0/82726150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 msgr2=0x55933ae9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 -- 172.21.15.77:0/82726150 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933adb81f0 con 0x55933ae9f480 2024-09-15T22:33:39.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5000a96640 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aea9800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009230 tx=0x7f4ff0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fddf01f283f3da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f5001297640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40cdc00 tx=0x7f4ff40058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.402+0000 7f4fe6ffd640 1 -- 172.21.15.77:0/82726150 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0035030 con 0x55933ae9f480 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f4fe6ffd640 1 -- 172.21.15.77:0/82726150 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0004430 con 0x55933ae9f480 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f4fe6ffd640 1 -- 172.21.15.77:0/82726150 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00035f0 con 0x55933ae9f480 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 -- 172.21.15.77:0/82726150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 msgr2=0x55933aea9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aea9800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009230 tx=0x7f4ff0004040 comp rx=0 tx=0).stop 2024-09-15T22:33:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 -- 172.21.15.77:0/82726150 shutdown_connections 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933aea9d40 0x55933aeac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aea9800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 --2- 172.21.15.77:0/82726150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933ae9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.403+0000 7f50050d3740 1 -- 172.21.15.77:0/82726150 >> 172.21.15.77:0/82726150 conn(0x55933ae9b0f0 msgr2=0x55933ae9b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.404+0000 7f50050d3740 1 -- 172.21.15.77:0/82726150 shutdown_connections 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.404+0000 7f50050d3740 1 -- 172.21.15.77:0/82726150 wait complete. 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.404+0000 7f50050d3740 1 -- start start 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.404+0000 7f50050d3740 1 Processor -- start 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933aea6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9f480 0x55933aea71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55933aeac910 con 0x55933ae9eb60 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55933adbef10 con 0x55933ae9f480 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55933aeaca90 con 0x55933aea9d40 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f50050d3740 10 client.0 _ll_register_callbacks cb 0x55933ad87a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44580/0 (socket says 172.21.15.77:44580) 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.405+0000 7f4ffbfff640 1 -- 172.21.15.77:0/1532216720 learned_addr learned my addr 172.21.15.77:0/1532216720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 -- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9f480 msgr2=0x55933aea71c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9f480 0x55933aea71c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 -- 172.21.15.77:0/1532216720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 msgr2=0x55933aea6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933aea6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 -- 172.21.15.77:0/1532216720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933ab756a0 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8012820 tx=0x7f4fe8012ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82de3fcb8f22ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8010b50 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.406+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4fe8010cf0 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff0004430 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fe801d410 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 10 client.29071 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55933aec0e10 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0004430 con 0x55933aea9d40 2024-09-15T22:33:39.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4fe800e4e0 con 0x55933aea9d40 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 10 client.29071 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 10 client.29071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 20 client.29071 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933aec2320 0x55933aec47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 4 client.29071 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.407+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933ae117f0 con 0x55933aec2320 2024-09-15T22:33:39.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.408+0000 7f5000a96640 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933aec2320 0x55933aec47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.408+0000 7f4fdbfff640 10 client.29071 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:39.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.408+0000 7f5000a96640 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933aec2320 0x55933aec47a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410d02faf64a553b server_cookie=e8f807c143adc1e4 in_seq=0 out_seq=0 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f4fdbfff640 1 -- 172.21.15.77:0/1532216720 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55933ae117f0 con 0x55933aec2320 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f4fdbfff640 10 client.29071 handle_command_reply: tid=0 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f50050d3740 1 client.29071 shutdown 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 msgr2=0x55933aea7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8012820 tx=0x7f4fe8012ce0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.410+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 shutdown_connections 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933aec2320 0x55933aec47a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aea7700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9f480 0x55933aea71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 --2- 172.21.15.77:0/1532216720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9eb60 0x55933aea6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 >> 172.21.15.77:0/1532216720 conn(0x55933ae9b0f0 msgr2=0x55933ae9a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f4fdbfff640 0 client.29071 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:39.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.411+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 shutdown_connections 2024-09-15T22:33:39.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.412+0000 7f50050d3740 1 -- 172.21.15.77:0/1532216720 wait complete. 2024-09-15T22:33:39.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.412+0000 7f50050d3740 20 client.29071 trim_cache size 0 max 16384 2024-09-15T22:33:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 Processor -- start 2024-09-15T22:33:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 -- start start 2024-09-15T22:33:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aea7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aeca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1acda0 con 0x55933ae9f480 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1ae8e0 con 0x55933ae9eb60 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f50050d3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1ad470 con 0x55933aea9d40 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.433+0000 7f5000a96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54984/0 (socket says 172.21.15.77:54984) 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 -- 172.21.15.77:0/116258720 learned_addr learned my addr 172.21.15.77:0/116258720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aea7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aeca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 -- 172.21.15.77:0/116258720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 msgr2=0x55933aeca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aeca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 -- 172.21.15.77:0/116258720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 msgr2=0x55933aea7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aea7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 -- 172.21.15.77:0/116258720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933aec4e20 con 0x55933ae9f480 2024-09-15T22:33:39.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aea7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:39.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f5000a96640 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0004650 tx=0x7f4ff00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131d04a45fc3da4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.434+0000 7f4fe67fc640 1 -- 172.21.15.77:0/116258720 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0035040 con 0x55933ae9f480 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.435+0000 7f4fe67fc640 1 -- 172.21.15.77:0/116258720 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0053030 con 0x55933ae9f480 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.435+0000 7f4fe67fc640 1 -- 172.21.15.77:0/116258720 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff004f420 con 0x55933ae9f480 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.435+0000 7f50050d3740 1 -- 172.21.15.77:0/116258720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 msgr2=0x55933aeca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.435+0000 7f50050d3740 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0004650 tx=0x7f4ff00045e0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 -- 172.21.15.77:0/116258720 shutdown_connections 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933aea9d40 0x55933aeca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933ae9f480 0x55933aeca070 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 --2- 172.21.15.77:0/116258720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aea7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 -- 172.21.15.77:0/116258720 >> 172.21.15.77:0/116258720 conn(0x55933ae9b0f0 msgr2=0x55933aea8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 -- 172.21.15.77:0/116258720 shutdown_connections 2024-09-15T22:33:39.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 -- 172.21.15.77:0/116258720 wait complete. 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.436+0000 7f50050d3740 1 -- start start 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 Processor -- start 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aeb6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933aea9d40 0x55933af2c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1ae210 con 0x55933aea9d40 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1adb40 con 0x55933ae9eb60 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc1aada0 con 0x55933ae9f480 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f50050d3740 10 client.0 _ll_register_callbacks cb 0x7f4fe8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41890/0 (socket says 172.21.15.77:41890) 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f4ffbfff640 1 -- 172.21.15.77:0/2158452237 learned_addr learned my addr 172.21.15.77:0/2158452237 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:39.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.437+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933aea9d40 0x55933af2c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f5000a96640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aeb6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 -- 172.21.15.77:0/2158452237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 msgr2=0x55933aeb6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aeb6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 -- 172.21.15.77:0/2158452237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933aea9d40 msgr2=0x55933af2c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933aea9d40 0x55933af2c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 -- 172.21.15.77:0/2158452237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933af2c580 con 0x55933ae9eb60 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4ffbfff640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800e980 tx=0x7f4fe8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3469fe6d63fe0cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:39.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.438+0000 7f4fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.439+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801d7c0 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.439+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f4fe8005070 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.439+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff0053030 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4fe801d960 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 10 client.29138 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55933af2d750 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0053030 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4fe8004730 con 0x55933ae9eb60 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 10 client.29138 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 10 client.29138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 20 client.29138 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:39.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.440+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933af2ddc0 0x55933af301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.441+0000 7f50050d3740 4 client.29138 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.441+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55933ae0bd00 con 0x55933af2ddc0 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.441+0000 7f5000a96640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933af2ddc0 0x55933af301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.442+0000 7f4fdbfff640 10 client.29138 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.442+0000 7f5000a96640 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933af2ddc0 0x55933af301e0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de871b3675fe3600 server_cookie=e65c84bbeead8be in_seq=0 out_seq=0 2024-09-15T22:33:39.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.443+0000 7f4fdbfff640 1 -- 172.21.15.77:0/2158452237 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55933ae0bd00 con 0x55933af2ddc0 2024-09-15T22:33:39.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.443+0000 7f4fdbfff640 10 client.29138 handle_command_reply: tid=0 2024-09-15T22:33:39.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.444+0000 7f50050d3740 1 client.29138 shutdown 2024-09-15T22:33:39.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.444+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 msgr2=0x55933aeb66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.444+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800e980 tx=0x7f4fe8005f90 comp rx=0 tx=0).stop 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 shutdown_connections 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55933af2ddc0 0x55933af301e0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f4fdbfff640 0 client.29138 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933aea9d40 0x55933af2c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933ae9f480 0x55933aeb6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 --2- 172.21.15.77:0/2158452237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933ae9eb60 0x55933aeb66e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 >> 172.21.15.77:0/2158452237 conn(0x55933ae9b0f0 msgr2=0x55933ae99a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.445+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 shutdown_connections 2024-09-15T22:33:39.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f50050d3740 1 -- 172.21.15.77:0/2158452237 wait complete. 2024-09-15T22:33:39.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f50050d3740 20 client.29138 trim_cache size 0 max 16384 2024-09-15T22:33:39.448 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:33:39.448 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "9df0246a-d0bd-4252-9acb-ee5269aa2299", 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 msgr2=0x7f4fec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40cdc00 tx=0x7f4ff40058d0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 msgr2=0x7f4ffc197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:39.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.446+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400eb10 tx=0x7f4ff400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 shutdown_connections 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ffc102ef0 0x7f4ffc197bd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc102570 0x7f4ffc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 --2- 172.21.15.77:0/4047914968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ffc100e20 0x7f4ffc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 >> 172.21.15.77:0/4047914968 conn(0x7f4ffc0fc7d0 msgr2=0x7f4ffc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 shutdown_connections 2024-09-15T22:33:39.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:39.447+0000 7f5002cfa640 1 -- 172.21.15.77:0/4047914968 wait complete. 2024-09-15T22:33:39.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 137871 14.91 MB/sec execute 340 sec latency 158.673 ms 2024-09-15T22:33:40.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 137995 14.89 MB/sec execute 341 sec latency 974.409 ms 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: pgmap v670: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 383 op/s 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:40.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:40.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:40 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: pgmap v670: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 383 op/s 2024-09-15T22:33:40.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:33:40.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:40.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:40 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: pgmap v670: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 383 op/s 2024-09-15T22:33:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:33:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:40.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:40 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.466 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:33:41.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 138417 14.93 MB/sec execute 342 sec latency 430.090 ms 2024-09-15T22:33:41.806 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:41.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:41 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:41 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:41 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:33:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:41 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:41 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:41 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:41 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:41 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:33:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:41 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:41 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:41 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:41 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:41 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:33:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:41 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:41.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:41 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:42.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3676839213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4112a30 msgr2=0x7f41f4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3676839213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4112a30 0x7f41f4114e20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009be0 tx=0x7f41ec02a6d0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3676839213 shutdown_connections 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3676839213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4112a30 0x7f41f4114e20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3676839213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4077040 0x7f41f41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3676839213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3676839213 >> 172.21.15.77:0/3676839213 conn(0x7f41f40ffab0 msgr2=0x7f41f4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3676839213 shutdown_connections 2024-09-15T22:33:42.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.334+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3676839213 wait complete. 2024-09-15T22:33:42.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.335+0000 7f41f8c5b640 1 Processor -- start 2024-09-15T22:33:42.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.335+0000 7f41f8c5b640 1 -- start start 2024-09-15T22:33:42.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.335+0000 7f41f8c5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.335+0000 7f41f8c5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4077040 0x7f41f41a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f8c5b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4112a30 0x7f41f419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f8c5b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4118570 con 0x7f41f4077040 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f8c5b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41183f0 con 0x7f41f4112a30 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f8c5b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41186f0 con 0x7f41f4076720 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44636/0 (socket says 172.21.15.77:44636) 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 -- 172.21.15.77:0/3135332815 learned_addr learned my addr 172.21.15.77:0/3135332815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:42.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f1d9b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4077040 0x7f41f41a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4112a30 0x7f41f419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 -- 172.21.15.77:0/3135332815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4112a30 msgr2=0x7f41f419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4112a30 0x7f41f419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 -- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4077040 msgr2=0x7f41f41a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4077040 0x7f41f41a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 -- 172.21.15.77:0/3135332815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f419f010 con 0x7f41f4076720 2024-09-15T22:33:42.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.336+0000 7f41f259c640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.337+0000 7f41e37fe640 1 -- 172.21.15.77:0/3135332815 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e400ccc0 con 0x7f41f4076720 2024-09-15T22:33:42.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.337+0000 7f41e37fe640 1 -- 172.21.15.77:0/3135332815 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e4010910 con 0x7f41f4076720 2024-09-15T22:33:42.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.337+0000 7f41e37fe640 1 -- 172.21.15.77:0/3135332815 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e40188a0 con 0x7f41f4076720 2024-09-15T22:33:42.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.337+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f419f300 con 0x7f41f4076720 2024-09-15T22:33:42.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.337+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41ab5f0 con 0x7f41f4076720 2024-09-15T22:33:42.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.340+0000 7f41fb034740 1 Processor -- start 2024-09-15T22:33:42.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 -- start start 2024-09-15T22:33:42.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dcfe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dcfedc0 0x56166dd09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56166dc642e0 con 0x56166dcfedc0 2024-09-15T22:33:42.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56166dc0bce0 con 0x56166dcfe4a0 2024-09-15T22:33:42.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41fb034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56166dc602a0 con 0x56166dd09680 2024-09-15T22:33:42.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41f1d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41f1d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44642/0 (socket says 172.21.15.77:44642) 2024-09-15T22:33:42.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.341+0000 7f41f1d9b640 1 -- 172.21.15.77:0/2485360035 learned_addr learned my addr 172.21.15.77:0/2485360035 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:42.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.342+0000 7f41f1d9b640 1 -- 172.21.15.77:0/2485360035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 msgr2=0x56166dcfe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.342+0000 7f41f1d9b640 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dcfe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.342+0000 7f41f1d9b640 1 -- 172.21.15.77:0/2485360035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dcfedc0 msgr2=0x56166dd09140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:33:42.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.343+0000 7f41f1d9b640 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dcfedc0 0x56166dd09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.343+0000 7f41f1d9b640 1 -- 172.21.15.77:0/2485360035 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56166dbf0440 con 0x56166dd09680 2024-09-15T22:33:42.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.343+0000 7f41f1d9b640 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009be0 tx=0x7f41e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f917aadd1d544cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.343+0000 7f41e37fe640 1 -- 172.21.15.77:0/3135332815 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f41e4010ac0 con 0x7f41f4076720 2024-09-15T22:33:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.344+0000 7f41e0ff9640 1 -- 172.21.15.77:0/2485360035 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e801e070 con 0x56166dd09680 2024-09-15T22:33:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.344+0000 7f41e0ff9640 1 -- 172.21.15.77:0/2485360035 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e80035f0 con 0x56166dd09680 2024-09-15T22:33:42.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.344+0000 7f41e0ff9640 1 -- 172.21.15.77:0/2485360035 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8019690 con 0x56166dd09680 2024-09-15T22:33:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 -- 172.21.15.77:0/2485360035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 msgr2=0x56166dd0ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009be0 tx=0x7f41e8011810 comp rx=0 tx=0).stop 2024-09-15T22:33:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 -- 172.21.15.77:0/2485360035 shutdown_connections 2024-09-15T22:33:42.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dd09680 0x56166dd0ba70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dcfedc0 0x56166dd09140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 --2- 172.21.15.77:0/2485360035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dcfe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 -- 172.21.15.77:0/2485360035 >> 172.21.15.77:0/2485360035 conn(0x56166dcfaa30 msgr2=0x56166dcfae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41e37fe640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 0x7f41dc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41fb034740 1 -- 172.21.15.77:0/2485360035 shutdown_connections 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.345+0000 7f41e37fe640 1 -- 172.21.15.77:0/3135332815 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f41e40d04d0 con 0x7f41f4076720 2024-09-15T22:33:42.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.346+0000 7f41fb034740 1 -- 172.21.15.77:0/2485360035 wait complete. 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.346+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 0x7f41dc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.346+0000 7f41fb034740 1 -- start start 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.346+0000 7f41fb034740 1 Processor -- start 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.347+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 0x7f41dc0a5a80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009b30 tx=0x7f41ec005e30 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.347+0000 7f41fb034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.347+0000 7f41fb034740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.347+0000 7f41fb034740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41fb034740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56166dd1d260 con 0x56166dd09680 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41f2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41f2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55018/0 (socket says 172.21.15.77:55018) 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3185152311 learned_addr learned my addr 172.21.15.77:0/3185152311 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41f259c640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56166dc1f050 con 0x56166dcfedc0 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56166dd1d3e0 con 0x56166dcfe4a0 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.348+0000 7f41fb034740 10 client.0 _ll_register_callbacks cb 0x56166dc5f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3185152311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 msgr2=0x56166dd090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 msgr2=0x56166dd063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166d9d53c0 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f259c640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009be0 tx=0x7f41ec030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd83a672663e284e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.349+0000 7f41d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec036040 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e80035f0 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f41ec003b70 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f41ec056050 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41fb034740 10 client.20430 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.350+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56166dd28d50 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec04d070 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f41ec0484f0 con 0x56166dd09680 2024-09-15T22:33:42.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 10 client.20430 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 10 client.20430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 20 client.20430 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dd2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 4 client.20430 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.351+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f41e80035f0 con 0x56166dd2a210 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.353+0000 7f41f259c640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dd2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.354+0000 7f41d9ffb640 10 client.20430 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.354+0000 7f41f259c640 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dd2c650 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f63d5a6493c756 server_cookie=500ed7983c9e72a7 in_seq=0 out_seq=0 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.356+0000 7f41d9ffb640 1 -- 172.21.15.77:0/3185152311 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f41e80035f0 con 0x56166dd2a210 2024-09-15T22:33:42.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.356+0000 7f41d9ffb640 10 client.20430 handle_command_reply: tid=0 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.356+0000 7f41fb034740 1 client.20430 shutdown 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 msgr2=0x56166dd06ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009be0 tx=0x7f41ec030c60 comp rx=0 tx=0).stop 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 shutdown_connections 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dd2c650 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd06ae0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41d9ffb640 0 client.20430 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:42.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 --2- 172.21.15.77:0/3185152311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 >> 172.21.15.77:0/3185152311 conn(0x56166dcfaa30 msgr2=0x56166dd00760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 shutdown_connections 2024-09-15T22:33:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 1 -- 172.21.15.77:0/3185152311 wait complete. 2024-09-15T22:33:42.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.357+0000 7f41fb034740 20 client.20430 trim_cache size 0 max 16384 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 Processor -- start 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 -- start start 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd31f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41acf00 con 0x56166dd09680 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41aea40 con 0x56166dcfedc0 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41fb034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41ad5d0 con 0x56166dcfe4a0 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41f2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41f2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55028/0 (socket says 172.21.15.77:55028) 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.383+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3612494923 learned_addr learned my addr 172.21.15.77:0/3612494923 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:42.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f259c640 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3612494923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 msgr2=0x56166dd264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3612494923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 msgr2=0x56166dd31f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd31f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 -- 172.21.15.77:0/3612494923 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56166dcfebf0 con 0x56166dd09680 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005cf0 tx=0x7f41ec005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e97989bab229c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.384+0000 7f41dbfff640 1 -- 172.21.15.77:0/3612494923 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec048460 con 0x56166dd09680 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.385+0000 7f41dbfff640 1 -- 172.21.15.77:0/3612494923 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec04d070 con 0x56166dd09680 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.385+0000 7f41dbfff640 1 -- 172.21.15.77:0/3612494923 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec052e60 con 0x56166dd09680 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 -- 172.21.15.77:0/3612494923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 msgr2=0x56166dd32460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005cf0 tx=0x7f41ec005d20 comp rx=0 tx=0).stop 2024-09-15T22:33:42.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 -- 172.21.15.77:0/3612494923 shutdown_connections 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd32460 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfedc0 0x56166dd31f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 --2- 172.21.15.77:0/3612494923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfe4a0 0x56166dd264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 -- 172.21.15.77:0/3612494923 >> 172.21.15.77:0/3612494923 conn(0x56166dcfaa30 msgr2=0x56166dd054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 -- 172.21.15.77:0/3612494923 shutdown_connections 2024-09-15T22:33:42.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.386+0000 7f41fb034740 1 -- 172.21.15.77:0/3612494923 wait complete. 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 -- start start 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 Processor -- start 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dd22d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 0x56166dd20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41ae370 con 0x56166dd09680 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41adca0 con 0x56166dcfe4a0 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41f419fe10 con 0x56166dcfedc0 2024-09-15T22:33:42.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.387+0000 7f41fb034740 10 client.0 _ll_register_callbacks cb 0x7f41ec034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:33:42.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.388+0000 7f41f1d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 0x56166dd20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.388+0000 7f41f259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dd22d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.389+0000 7f41f1d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 0x56166dd20120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:44668/0 (socket says 172.21.15.77:44668) 2024-09-15T22:33:42.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.389+0000 7f41f1d9b640 1 -- 172.21.15.77:0/4101002720 learned_addr learned my addr 172.21.15.77:0/4101002720 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:33:42.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.389+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 -- 172.21.15.77:0/4101002720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 msgr2=0x56166dd20120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 0x56166dd20120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 -- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 msgr2=0x56166dd22d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:33:42.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dd22d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 -- 172.21.15.77:0/4101002720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166dd20e40 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41f2d9d640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd20660 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0099a0 tx=0x7f41ec048670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db3b0cae96a51fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec052860 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.390+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9438+0+0 (secure 0 0 0) 0x7f41ec052a00 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec04d070 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f41ec069050 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41fb034740 10 client.20436 fetch_fsmap learned FSMap version 39 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56166dd94f20 con 0x56166dd09680 2024-09-15T22:33:42.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec031920 con 0x56166dd09680 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f41ec0645d0 con 0x56166dd09680 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41fb034740 10 client.20436 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.391+0000 7f41fb034740 10 client.20436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.392+0000 7f41fb034740 20 client.20436 populate_metadata read hostname 'smithi077' 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.392+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dda8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.392+0000 7f41fb034740 4 client.20436 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.392+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f41ec04d070 con 0x56166dd2a210 2024-09-15T22:33:42.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.392+0000 7f41f259c640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dda8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.395+0000 7f41f259c640 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dda8550 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f513b9a1efb55c5a server_cookie=b1043c8c563828e1 in_seq=0 out_seq=0 2024-09-15T22:33:42.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.395+0000 7f41d9ffb640 10 client.20436 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.396+0000 7f41d9ffb640 1 -- 172.21.15.77:0/4101002720 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f41ec04d070 con 0x56166dd2a210 2024-09-15T22:33:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.396+0000 7f41d9ffb640 10 client.20436 handle_command_reply: tid=0 2024-09-15T22:33:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.396+0000 7f41fb034740 1 client.20436 shutdown 2024-09-15T22:33:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 msgr2=0x56166dd20660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd20660 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0099a0 tx=0x7f41ec048670 comp rx=0 tx=0).stop 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 shutdown_connections 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56166dd2a210 0x56166dda8550 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41d9ffb640 0 client.20436 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56166dd09680 0x56166dd20660 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56166dcfedc0 0x56166dd20120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 --2- 172.21.15.77:0/4101002720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56166dcfe4a0 0x56166dd22d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 >> 172.21.15.77:0/4101002720 conn(0x56166dcfaa30 msgr2=0x56166dcf9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 shutdown_connections 2024-09-15T22:33:42.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.397+0000 7f41fb034740 1 -- 172.21.15.77:0/4101002720 wait complete. 2024-09-15T22:33:42.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.398+0000 7f41fb034740 20 client.20436 trim_cache size 0 max 16384 2024-09-15T22:33:42.401 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:33:42.401 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:33:42.401 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: "9df0246a-d0bd-4252-9acb-ee5269aa2299": { 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "9df0246a-d0bd-4252-9acb-ee5269aa2299", 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:33:42.402 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.400+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 msgr2=0x7f41dc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.400+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 0x7f41dc0a5a80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009b30 tx=0x7f41ec005e30 comp rx=0 tx=0).stop 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.400+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 msgr2=0x7f41f41a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.400+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400edf0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 shutdown_connections 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f41dc0a35c0 0x7f41dc0a5a80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f4112a30 0x7f41f419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41f4077040 0x7f41f41a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 --2- 172.21.15.77:0/3135332815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41f4076720 0x7f41f41a4720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:42.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 >> 172.21.15.77:0/3135332815 conn(0x7f41f40ffab0 msgr2=0x7f41f4101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:42.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 shutdown_connections 2024-09-15T22:33:42.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:33:42.401+0000 7f41f8c5b640 1 -- 172.21.15.77:0/3135332815 wait complete. 2024-09-15T22:33:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:42 smithi077 ceph-mon[30980]: pgmap v671: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 422 op/s 2024-09-15T22:33:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:42 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:42.615 INFO:tasks.workunit.client.0.smithi077.stdout: 10 138775 14.93 MB/sec execute 343 sec latency 192.611 ms 2024-09-15T22:33:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:42 smithi196 ceph-mon[35109]: pgmap v671: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 422 op/s 2024-09-15T22:33:42.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:42 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:42 smithi117 ceph-mon[35021]: pgmap v671: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 422 op/s 2024-09-15T22:33:42.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:42 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:33:43.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 138969 14.92 MB/sec execute 344 sec latency 376.461 ms 2024-09-15T22:33:44.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 139387 14.95 MB/sec execute 345 sec latency 228.597 ms 2024-09-15T22:33:44.768 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:44 smithi077 ceph-mon[30980]: pgmap v672: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-09-15T22:33:44.799 INFO:tasks.cephfs.filesystem:scrub status for tag:9df0246a-d0bd-4252-9acb-ee5269aa2299 - {'path': '/', 'tag': '9df0246a-d0bd-4252-9acb-ee5269aa2299', 'options': 'recursive,force'} 2024-09-15T22:33:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:44 smithi196 ceph-mon[35109]: pgmap v672: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-09-15T22:33:44.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:44 smithi117 ceph-mon[35021]: pgmap v672: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-09-15T22:33:45.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 139768 14.95 MB/sec execute 346 sec latency 158.686 ms 2024-09-15T22:33:46.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 140177 14.98 MB/sec execute 347 sec latency 150.678 ms 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: pgmap v673: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.2 MiB/s wr, 381 op/s 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:33:46.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:33:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: pgmap v673: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.2 MiB/s wr, 381 op/s 2024-09-15T22:33:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:33:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:33:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:33:46.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:33:46.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: pgmap v673: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.2 MiB/s wr, 381 op/s 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:33:46.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:33:47.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 140559 15.00 MB/sec execute 348 sec latency 209.878 ms 2024-09-15T22:33:48.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 140892 14.99 MB/sec execute 349 sec latency 205.539 ms 2024-09-15T22:33:48.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:48 smithi077 ceph-mon[30980]: pgmap v674: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:33:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:48 smithi196 ceph-mon[35109]: pgmap v674: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:33:48.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:48 smithi117 ceph-mon[35021]: pgmap v674: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:33:49.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 141274 14.99 MB/sec execute 350 sec latency 169.595 ms 2024-09-15T22:33:50.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 141463 14.99 MB/sec execute 351 sec latency 536.310 ms 2024-09-15T22:33:50.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:50 smithi077 ceph-mon[30980]: pgmap v675: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 354 op/s 2024-09-15T22:33:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:50 smithi196 ceph-mon[35109]: pgmap v675: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 354 op/s 2024-09-15T22:33:50.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:50 smithi117 ceph-mon[35021]: pgmap v675: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 354 op/s 2024-09-15T22:33:51.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 141942 15.02 MB/sec execute 352 sec latency 141.932 ms 2024-09-15T22:33:52.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 142315 15.05 MB/sec execute 353 sec latency 149.916 ms 2024-09-15T22:33:52.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:52 smithi077 ceph-mon[30980]: pgmap v676: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 420 op/s 2024-09-15T22:33:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:52 smithi196 ceph-mon[35109]: pgmap v676: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 420 op/s 2024-09-15T22:33:52.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:52 smithi117 ceph-mon[35021]: pgmap v676: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.7 MiB/s wr, 420 op/s 2024-09-15T22:33:53.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 142730 15.08 MB/sec execute 354 sec latency 190.434 ms 2024-09-15T22:33:54.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 143103 15.08 MB/sec execute 355 sec latency 139.338 ms 2024-09-15T22:33:54.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:54 smithi077 ceph-mon[30980]: pgmap v677: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-09-15T22:33:54.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:54 smithi196 ceph-mon[35109]: pgmap v677: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-09-15T22:33:54.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:54 smithi117 ceph-mon[35021]: pgmap v677: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 493 op/s 2024-09-15T22:33:55.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 143272 15.06 MB/sec execute 356 sec latency 646.451 ms 2024-09-15T22:33:56.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 143659 15.08 MB/sec execute 357 sec latency 721.794 ms 2024-09-15T22:33:56.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:56 smithi077 ceph-mon[30980]: pgmap v678: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 351 op/s 2024-09-15T22:33:56.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:56 smithi196 ceph-mon[35109]: pgmap v678: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 351 op/s 2024-09-15T22:33:56.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:56 smithi117 ceph-mon[35021]: pgmap v678: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.7 MiB/s wr, 351 op/s 2024-09-15T22:33:57.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 143989 15.09 MB/sec execute 358 sec latency 266.382 ms 2024-09-15T22:33:58.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 144260 15.07 MB/sec execute 359 sec latency 324.128 ms 2024-09-15T22:33:58.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:33:58 smithi077 ceph-mon[30980]: pgmap v679: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-15T22:33:58.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:33:58 smithi196 ceph-mon[35109]: pgmap v679: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-15T22:33:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:33:58 smithi117 ceph-mon[35021]: pgmap v679: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 517 op/s 2024-09-15T22:33:59.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 144788 15.13 MB/sec execute 360 sec latency 154.266 ms 2024-09-15T22:34:00.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 145007 15.12 MB/sec execute 361 sec latency 401.908 ms 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: pgmap v680: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 344 op/s 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:00 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: pgmap v680: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 344 op/s 2024-09-15T22:34:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:00.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:00 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: pgmap v680: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 344 op/s 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:00 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:01.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 145246 15.11 MB/sec execute 362 sec latency 468.883 ms 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: pgmap v681: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 417 op/s 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:02.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:01 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: pgmap v681: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 417 op/s 2024-09-15T22:34:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:02.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:01 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: pgmap v681: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 417 op/s 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:02.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:01 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:34:02.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 145557 15.11 MB/sec execute 363 sec latency 279.859 ms 2024-09-15T22:34:03.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 145938 15.12 MB/sec execute 364 sec latency 257.406 ms 2024-09-15T22:34:04.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:03 smithi077 ceph-mon[30980]: pgmap v682: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 467 op/s 2024-09-15T22:34:04.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:03 smithi196 ceph-mon[35109]: pgmap v682: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 467 op/s 2024-09-15T22:34:04.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:03 smithi117 ceph-mon[35021]: pgmap v682: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 467 op/s 2024-09-15T22:34:04.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 146300 15.15 MB/sec execute 365 sec latency 246.639 ms 2024-09-15T22:34:05.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 146622 15.15 MB/sec execute 366 sec latency 201.535 ms 2024-09-15T22:34:06.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:05 smithi077 ceph-mon[30980]: pgmap v683: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 356 op/s 2024-09-15T22:34:06.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:05 smithi196 ceph-mon[35109]: pgmap v683: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 356 op/s 2024-09-15T22:34:06.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:05 smithi117 ceph-mon[35021]: pgmap v683: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.5 MiB/s wr, 356 op/s 2024-09-15T22:34:06.616 INFO:tasks.workunit.client.0.smithi077.stdout: 10 146970 15.17 MB/sec execute 367 sec latency 187.165 ms 2024-09-15T22:34:07.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:06 smithi077 ceph-mon[30980]: osdmap e122: 12 total, 12 up, 12 in 2024-09-15T22:34:07.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:06 smithi196 ceph-mon[35109]: osdmap e122: 12 total, 12 up, 12 in 2024-09-15T22:34:07.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:06 smithi117 ceph-mon[35021]: osdmap e122: 12 total, 12 up, 12 in 2024-09-15T22:34:07.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 147304 15.16 MB/sec execute 368 sec latency 176.799 ms 2024-09-15T22:34:08.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:07 smithi077 ceph-mon[30980]: pgmap v685: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:08.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:07 smithi196 ceph-mon[35109]: pgmap v685: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:08.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:07 smithi117 ceph-mon[35021]: pgmap v685: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:08.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 147823 15.20 MB/sec execute 369 sec latency 151.252 ms 2024-09-15T22:34:09.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 148212 15.23 MB/sec execute 370 sec latency 914.987 ms 2024-09-15T22:34:10.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:09 smithi077 ceph-mon[30980]: pgmap v686: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:10.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:09 smithi196 ceph-mon[35109]: pgmap v686: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:10.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:09 smithi117 ceph-mon[35021]: pgmap v686: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2024-09-15T22:34:10.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 148620 15.25 MB/sec execute 371 sec latency 1915.019 ms 2024-09-15T22:34:11.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 148893 15.24 MB/sec execute 372 sec latency 2915.048 ms 2024-09-15T22:34:12.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:11 smithi077 ceph-mon[30980]: pgmap v687: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-09-15T22:34:12.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:11 smithi196 ceph-mon[35109]: pgmap v687: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-09-15T22:34:12.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:11 smithi117 ceph-mon[35021]: pgmap v687: 129 pgs: 7 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 95 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-09-15T22:34:12.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 149249 15.24 MB/sec execute 373 sec latency 3915.084 ms 2024-09-15T22:34:13.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 149703 15.26 MB/sec execute 374 sec latency 4915.111 ms 2024-09-15T22:34:14.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:13 smithi077 ceph-mon[30980]: pgmap v688: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-09-15T22:34:14.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:13 smithi196 ceph-mon[35109]: pgmap v688: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-09-15T22:34:14.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:13 smithi117 ceph-mon[35021]: pgmap v688: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.8 MiB/s wr, 398 op/s 2024-09-15T22:34:14.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 150138 15.28 MB/sec execute 375 sec latency 5915.139 ms 2024-09-15T22:34:14.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:34:15.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:15 smithi077 ceph-mon[30980]: osdmap e123: 12 total, 12 up, 12 in 2024-09-15T22:34:15.088 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:15.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:15 smithi196 ceph-mon[35109]: osdmap e123: 12 total, 12 up, 12 in 2024-09-15T22:34:15.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:15 smithi117 ceph-mon[35021]: osdmap e123: 12 total, 12 up, 12 in 2024-09-15T22:34:15.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 150522 15.31 MB/sec execute 376 sec latency 6915.175 ms 2024-09-15T22:34:15.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 -- 172.21.15.77:0/345158877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7181049a0 msgr2=0x7fd7181089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 --2- 172.21.15.77:0/345158877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7181049a0 0x7fd7181089f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009900 tx=0x7fd70c02a410 comp rx=0 tx=0).stop 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 -- 172.21.15.77:0/345158877 shutdown_connections 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 --2- 172.21.15.77:0/345158877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd718109120 0x7fd718110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 --2- 172.21.15.77:0/345158877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7181049a0 0x7fd7181089f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 --2- 172.21.15.77:0/345158877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd7181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 -- 172.21.15.77:0/345158877 >> 172.21.15.77:0/345158877 conn(0x7fd7180ffab0 msgr2=0x7fd718101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 -- 172.21.15.77:0/345158877 shutdown_connections 2024-09-15T22:34:15.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.644+0000 7fd71e306640 1 -- 172.21.15.77:0/345158877 wait complete. 2024-09-15T22:34:15.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 Processor -- start 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 -- start start 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 0x7fd71819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718109120 0x7fd71819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.645+0000 7fd71e306640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd718114120 con 0x7fd7181049a0 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd71e306640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd718113fa0 con 0x7fd718103ff0 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd71e306640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181142a0 con 0x7fd718109120 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd71c8a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718109120 0x7fd71819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48336/0 (socket says 172.21.15.77:48336) 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 -- 172.21.15.77:0/4193713883 learned_addr learned my addr 172.21.15.77:0/4193713883 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd7177fe640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 0x7fd71819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 -- 172.21.15.77:0/4193713883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718109120 msgr2=0x7fd71819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718109120 0x7fd71819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 -- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 msgr2=0x7fd71819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 0x7fd71819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 -- 172.21.15.77:0/4193713883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71819f3b0 con 0x7fd718103ff0 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd7177fe640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 0x7fd71819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.646+0000 7fd717fff640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd70800ea60 tx=0x7fd70800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.647+0000 7fd7157fa640 1 -- 172.21.15.77:0/4193713883 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70800cde0 con 0x7fd718103ff0 2024-09-15T22:34:15.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.647+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71819f6a0 con 0x7fd718103ff0 2024-09-15T22:34:15.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.647+0000 7fd7157fa640 1 -- 172.21.15.77:0/4193713883 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7080035f0 con 0x7fd718103ff0 2024-09-15T22:34:15.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.647+0000 7fd7157fa640 1 -- 172.21.15.77:0/4193713883 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708010670 con 0x7fd718103ff0 2024-09-15T22:34:15.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.647+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7181a6f80 con 0x7fd718103ff0 2024-09-15T22:34:15.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.649+0000 7fd7157fa640 1 -- 172.21.15.77:0/4193713883 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd708010810 con 0x7fd718103ff0 2024-09-15T22:34:15.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 Processor -- start 2024-09-15T22:34:15.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 -- start start 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7157fa640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 0x7fd7000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e85d0 0x564b723f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b723452e0 con 0x564b723e7cb0 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b722ecce0 con 0x564b723f2e90 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7206df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b723412a0 con 0x564b723e85d0 2024-09-15T22:34:15.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.650+0000 7fd7157fa640 1 -- 172.21.15.77:0/4193713883 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd7080d0380 con 0x7fd718103ff0 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39150/0 (socket says 172.21.15.77:39150) 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 -- 172.21.15.77:0/1224968036 learned_addr learned my addr 172.21.15.77:0/1224968036 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 0x7fd7000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd7177fe640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e85d0 0x564b723f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 -- 172.21.15.77:0/1224968036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e85d0 msgr2=0x564b723f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e85d0 0x564b723f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 -- 172.21.15.77:0/1224968036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 msgr2=0x564b723f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 -- 172.21.15.77:0/1224968036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b722d1440 con 0x564b723e7cb0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd717fff640 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a4a10 tx=0x7fd70809bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6224ddad3e0a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.651+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 0x7fd7000a58f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd710005c40 tx=0x7fd710005bd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.652+0000 7fd6fe7fc640 1 -- 172.21.15.77:0/1224968036 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080a0680 con 0x564b723e7cb0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.652+0000 7fd6fe7fc640 1 -- 172.21.15.77:0/1224968036 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7080a4f40 con 0x564b723e7cb0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.652+0000 7fd6fe7fc640 1 -- 172.21.15.77:0/1224968036 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708098940 con 0x564b723e7cb0 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 -- 172.21.15.77:0/1224968036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 msgr2=0x564b723e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a4a10 tx=0x7fd70809bc30 comp rx=0 tx=0).stop 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 -- 172.21.15.77:0/1224968036 shutdown_connections 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e85d0 0x564b723f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 --2- 172.21.15.77:0/1224968036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e7cb0 0x564b723e8090 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 -- 172.21.15.77:0/1224968036 >> 172.21.15.77:0/1224968036 conn(0x564b723e4240 msgr2=0x564b723e4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 -- 172.21.15.77:0/1224968036 shutdown_connections 2024-09-15T22:34:15.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.653+0000 7fd7206df740 1 -- 172.21.15.77:0/1224968036 wait complete. 2024-09-15T22:34:15.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 -- start start 2024-09-15T22:34:15.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 Processor -- start 2024-09-15T22:34:15.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 0x564b723f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723efae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564b723f5a60 con 0x564b723e85d0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564b72300050 con 0x564b723f2e90 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7206df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b723f5be0 con 0x564b723e7cb0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.654+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723efae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39164/0 (socket says 172.21.15.77:39164) 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd7177fe640 1 -- 172.21.15.77:0/2643895552 learned_addr learned my addr 172.21.15.77:0/2643895552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 0x564b723f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd7206df740 10 client.0 _ll_register_callbacks cb 0x564b72340160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 -- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 msgr2=0x564b723f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 -- 172.21.15.77:0/2643895552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 msgr2=0x564b723efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 -- 172.21.15.77:0/2643895552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b720b63c0 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd717fff640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 0x564b723f2830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a4a10 tx=0x7fd708003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6931da82eb3d2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.655+0000 7fd6f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080b4070 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7080a4f40 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd70809a5d0 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd7080a0680 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd7206df740 10 client.29107 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b723f5a60 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7080a4f40 con 0x564b723e7cb0 2024-09-15T22:34:15.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd7080a0160 con 0x564b723e7cb0 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.656+0000 7fd7206df740 10 client.29107 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7206df740 10 client.29107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7206df740 20 client.29107 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b72402a00 0x564b72404e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7206df740 4 client.29107 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b72352850 con 0x564b72402a00 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7177fe640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b72402a00 0x564b72404e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd6f77fe640 10 client.29107 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:15.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.657+0000 7fd7177fe640 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b72402a00 0x564b72404e40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740896d2efab5142 server_cookie=a6551cfd22728ff0 in_seq=0 out_seq=0 2024-09-15T22:34:15.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd6f77fe640 1 -- 172.21.15.77:0/2643895552 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b72352850 con 0x564b72402a00 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd6f77fe640 10 client.29107 handle_command_reply: tid=0 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 client.29107 shutdown 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 msgr2=0x564b723f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 0x564b723f2830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a4a10 tx=0x7fd708003e70 comp rx=0 tx=0).stop 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 shutdown_connections 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b72402a00 0x564b72404e40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 --2- 172.21.15.77:0/2643895552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723e7cb0 0x564b723f2830 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd7080a4a10 tx=0x7fd708003e70 comp rx=0 tx=0).stop 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 >> 172.21.15.77:0/2643895552 conn(0x564b723e4240 msgr2=0x564b723e9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.661+0000 7fd6f77fe640 0 client.29107 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.662+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 shutdown_connections 2024-09-15T22:34:15.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.662+0000 7fd7206df740 1 -- 172.21.15.77:0/2643895552 wait complete. 2024-09-15T22:34:15.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.662+0000 7fd7206df740 20 client.29107 trim_cache size 0 max 16384 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.687+0000 7fd7206df740 1 Processor -- start 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.687+0000 7fd7206df740 1 -- start start 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.687+0000 7fd7206df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723e85d0 0x564b723fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.687+0000 7fd7206df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.687+0000 7fd7206df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b7240a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7206df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ace50 con 0x564b723f2e90 2024-09-15T22:34:15.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7206df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ae990 con 0x564b723e85d0 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7206df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ad520 con 0x564b723daa60 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39166/0 (socket says 172.21.15.77:39166) 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 -- 172.21.15.77:0/4096893248 learned_addr learned my addr 172.21.15.77:0/4096893248 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b7240a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd717fff640 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723e85d0 0x564b723fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 -- 172.21.15.77:0/4096893248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 msgr2=0x564b7240a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b7240a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 -- 172.21.15.77:0/4096893248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723e85d0 msgr2=0x564b723fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723e85d0 0x564b723fecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 -- 172.21.15.77:0/4096893248 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b723eea90 con 0x564b723f2e90 2024-09-15T22:34:15.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd7177fe640 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd70c02a930 tx=0x7fd70c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b390bd8078d24e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd6fdffb640 1 -- 172.21.15.77:0/4096893248 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c035040 con 0x564b723f2e90 2024-09-15T22:34:15.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.688+0000 7fd6fdffb640 1 -- 172.21.15.77:0/4096893248 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd70c031c90 con 0x564b723f2e90 2024-09-15T22:34:15.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd6fdffb640 1 -- 172.21.15.77:0/4096893248 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c03de40 con 0x564b723f2e90 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 -- 172.21.15.77:0/4096893248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 msgr2=0x564b723ee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd70c02a930 tx=0x7fd70c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 -- 172.21.15.77:0/4096893248 shutdown_connections 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b7240a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723f2e90 0x564b723ee3b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 --2- 172.21.15.77:0/4096893248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723e85d0 0x564b723fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 -- 172.21.15.77:0/4096893248 >> 172.21.15.77:0/4096893248 conn(0x564b723e4240 msgr2=0x564b723da350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 -- 172.21.15.77:0/4096893248 shutdown_connections 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.689+0000 7fd7206df740 1 -- 172.21.15.77:0/4096893248 wait complete. 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 -- start start 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 Processor -- start 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b723fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ae2c0 con 0x564b723e85d0 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181adbf0 con 0x564b723f2e90 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181aae70 con 0x564b723daa60 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7206df740 10 client.0 _ll_register_callbacks cb 0x7fd710009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd717fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b723fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7177fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39178/0 (socket says 172.21.15.77:39178) 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd7177fe640 1 -- 172.21.15.77:0/782669560 learned_addr learned my addr 172.21.15.77:0/782669560 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.690+0000 7fd71c8a3640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 -- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 msgr2=0x564b723ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 -- 172.21.15.77:0/782669560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 msgr2=0x564b723ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 -- 172.21.15.77:0/782669560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b723e8400 con 0x564b723daa60 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd7177fe640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd717fff640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b723fb320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd70809c260 tx=0x7fd70809bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90762281b5d09da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd6f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:15.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080b4070 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd70809a5d0 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd70c031c90 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd70809c9d0 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd7206df740 10 client.29113 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.691+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b723f2ba0 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd70c031c90 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd708098940 con 0x564b723daa60 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 10 client.29113 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 10 client.29113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 20 client.29113 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b7246d690 0x564b7246fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:15.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 4 client.29113 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b7234cd60 con 0x564b7246d690 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.692+0000 7fd7177fe640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b7246d690 0x564b7246fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.693+0000 7fd7177fe640 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b7246d690 0x564b7246fab0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b2c040540ea628 server_cookie=707c5e602842f59 in_seq=0 out_seq=0 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.693+0000 7fd6f77fe640 10 client.29113 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.693+0000 7fd6f77fe640 1 -- 172.21.15.77:0/782669560 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564b7234cd60 con 0x564b7246d690 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.693+0000 7fd6f77fe640 10 client.29113 handle_command_reply: tid=0 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.693+0000 7fd7206df740 1 client.29113 shutdown 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 msgr2=0x564b723fb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b723fb320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd70809c260 tx=0x7fd70809bc30 comp rx=0 tx=0).stop 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 shutdown_connections 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x564b7246d690 0x564b7246fab0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b723f2e90 0x564b723ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b723e85d0 0x564b723ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 --2- 172.21.15.77:0/782669560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b723daa60 0x564b723fb320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 >> 172.21.15.77:0/782669560 conn(0x564b723e4240 msgr2=0x564b723ee370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd6f77fe640 0 client.29113 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 shutdown_connections 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 1 -- 172.21.15.77:0/782669560 wait complete. 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.694+0000 7fd7206df740 20 client.29113 trim_cache size 0 max 16384 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:34:15.701 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:34:15.702 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:34:15.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 msgr2=0x7fd7000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 0x7fd7000a58f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd710005c40 tx=0x7fd710005bd0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 msgr2=0x7fd71819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd70800ea60 tx=0x7fd70800ef20 comp rx=0 tx=0).stop 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 shutdown_connections 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd7000a3430 0x7fd7000a58f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd718109120 0x7fd71819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7181049a0 0x7fd71819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 --2- 172.21.15.77:0/4193713883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718103ff0 0x7fd71819a400 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:15.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 >> 172.21.15.77:0/4193713883 conn(0x7fd7180ffab0 msgr2=0x7fd718100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:15.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 shutdown_connections 2024-09-15T22:34:15.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:15.695+0000 7fd71e306640 1 -- 172.21.15.77:0/4193713883 wait complete. 2024-09-15T22:34:16.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:16 smithi077 ceph-mon[30980]: pgmap v690: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 330 op/s 2024-09-15T22:34:16.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:16 smithi196 ceph-mon[35109]: pgmap v690: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 330 op/s 2024-09-15T22:34:16.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:16 smithi117 ceph-mon[35021]: pgmap v690: 129 pgs: 129 active+clean; 1.7 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.5 MiB/s wr, 330 op/s 2024-09-15T22:34:16.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:34:16.526 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:34:16.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 150923 15.32 MB/sec execute 377 sec latency 7915.213 ms 2024-09-15T22:34:16.844 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:17.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/3139370529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102ef0 msgr2=0x7f4cd810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:17.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/3139370529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102ef0 0x7f4cd810da40 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009ca0 tx=0x7f4cd002a950 comp rx=0 tx=0).stop 2024-09-15T22:34:17.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/3139370529 shutdown_connections 2024-09-15T22:34:17.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/3139370529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102ef0 0x7f4cd810da40 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/3139370529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8102570 0x7f4cd81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/3139370529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8100e20 0x7f4cd8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/3139370529 >> 172.21.15.77:0/3139370529 conn(0x7f4cd80fc7d0 msgr2=0x7f4cd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/3139370529 shutdown_connections 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/3139370529 wait complete. 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.363+0000 7f4cdf3c9640 1 Processor -- start 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 -- start start 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8100e20 0x7f4cd81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8102ef0 0x7f4cd819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8110f80 con 0x7f4cd8102570 2024-09-15T22:34:17.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8110e00 con 0x7f4cd8102ef0 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdf3c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111100 con 0x7f4cd8100e20 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39186/0 (socket says 172.21.15.77:39186) 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 -- 172.21.15.77:0/470598749 learned_addr learned my addr 172.21.15.77:0/470598749 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdd165640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8100e20 0x7f4cd81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdd966640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8102ef0 0x7f4cd819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 -- 172.21.15.77:0/470598749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8100e20 msgr2=0x7f4cd81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8100e20 0x7f4cd81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 -- 172.21.15.77:0/470598749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8102ef0 msgr2=0x7f4cd819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8102ef0 0x7f4cd819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.364+0000 7f4cdc964640 1 -- 172.21.15.77:0/470598749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd819d460 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cdc964640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d770 tx=0x7f4cc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8017070 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd819d750 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a5030 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc8005500 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.365+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8020060 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.366+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cbc005290 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.367+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4cc8009c30 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.368+0000 7f4cce7fc640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 0x7f4cc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.368+0000 7f4cdd165640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 0x7f4cc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.368+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f4cc80cfdd0 con 0x7f4cd8102570 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.369+0000 7f4cdd165640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 0x7f4cc40a5870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009bd0 tx=0x7f4cd4005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:17.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.372+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cc80046c0 con 0x7f4cd8102570 2024-09-15T22:34:17.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 151287 15.33 MB/sec execute 378 sec latency 8915.247 ms 2024-09-15T22:34:17.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.636+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cbc005580 con 0x7f4cd8102570 2024-09-15T22:34:17.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.636+0000 7f4cce7fc640 1 -- 172.21.15.77:0/470598749 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f4cc809c420 con 0x7f4cd8102570 2024-09-15T22:34:17.638 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:34:17.638 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:34:17.638 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:34:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 msgr2=0x7f4cc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:17.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 0x7f4cc40a5870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009bd0 tx=0x7f4cd4005f90 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 msgr2=0x7f4cd81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d770 tx=0x7f4cc800dc30 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 shutdown_connections 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4cc40a33b0 0x7f4cc40a5870 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cd8102ef0 0x7f4cd819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4cd8102570 0x7f4cd81989f0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 --2- 172.21.15.77:0/470598749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cd8100e20 0x7f4cd81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 >> 172.21.15.77:0/470598749 conn(0x7f4cd80fc7d0 msgr2=0x7f4cd806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 shutdown_connections 2024-09-15T22:34:17.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:17.639+0000 7f4cdf3c9640 1 -- 172.21.15.77:0/470598749 wait complete. 2024-09-15T22:34:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:18 smithi077 ceph-mon[30980]: pgmap v691: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:18.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:18 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/470598749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:18.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:18 smithi196 ceph-mon[35109]: pgmap v691: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:18.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:18 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/470598749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:18 smithi117 ceph-mon[35021]: pgmap v691: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:18.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:18 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/470598749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:18.390 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:34:18.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 151704 15.36 MB/sec execute 379 sec latency 9915.272 ms 2024-09-15T22:34:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 -- 172.21.15.77:0/3917280898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109450 msgr2=0x7fda30110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 --2- 172.21.15.77:0/3917280898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109450 0x7fda30110250 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fda28009ca0 tx=0x7fda2802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 -- 172.21.15.77:0/3917280898 shutdown_connections 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 --2- 172.21.15.77:0/3917280898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda3011b140 0x7fda3011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 --2- 172.21.15.77:0/3917280898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda30109830 0x7fda30109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 --2- 172.21.15.77:0/3917280898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109450 0x7fda30110250 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 -- 172.21.15.77:0/3917280898 >> 172.21.15.77:0/3917280898 conn(0x7fda3006e140 msgr2=0x7fda3006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 -- 172.21.15.77:0/3917280898 shutdown_connections 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.243+0000 7fda36151640 1 -- 172.21.15.77:0/3917280898 wait complete. 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 Processor -- start 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 -- start start 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda30109450 0x7fda301a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3011b140 0x7fda301abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda30120df0 con 0x7fda30109830 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda30120c70 con 0x7fda30109450 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda36151640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda30120f70 con 0x7fda3011b140 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.244+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3011b140 0x7fda301abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47234/0 (socket says 172.21.15.77:47234) 2024-09-15T22:34:19.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 -- 172.21.15.77:0/3854568061 learned_addr learned my addr 172.21.15.77:0/3854568061 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 -- 172.21.15.77:0/3854568061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3011b140 msgr2=0x7fda301abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3011b140 0x7fda301abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 -- 172.21.15.77:0/3854568061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda30109450 msgr2=0x7fda301a7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda30109450 0x7fda301a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 -- 172.21.15.77:0/3854568061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301ac1e0 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.245+0000 7fda2effd640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.246+0000 7fda2cff9640 1 -- 172.21.15.77:0/3854568061 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400cd80 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.246+0000 7fda2cff9640 1 -- 172.21.15.77:0/3854568061 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24005500 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.246+0000 7fda2cff9640 1 -- 172.21.15.77:0/3854568061 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24010690 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.246+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301ac4d0 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.246+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301b3db0 con 0x7fda30109830 2024-09-15T22:34:19.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.249+0000 7fda2cff9640 1 -- 172.21.15.77:0/3854568061 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fda240046c0 con 0x7fda30109830 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 Processor -- start 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 -- start start 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2cff9640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 0x7fda180a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 0x7fda180a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2cff9640 1 -- 172.21.15.77:0/3854568061 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fda240d03d0 con 0x7fda30109830 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e465342f0 con 0x562e465d9370 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e464dbcf0 con 0x562e465ce5d0 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda3852a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e465302b0 con 0x562e465cfb40 2024-09-15T22:34:19.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47502/0 (socket says 172.21.15.77:47502) 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2f7fe640 1 -- 172.21.15.77:0/3078890555 learned_addr learned my addr 172.21.15.77:0/3078890555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2effd640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.250+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 -- 172.21.15.77:0/3078890555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 msgr2=0x562e465cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 -- 172.21.15.77:0/3078890555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 msgr2=0x562e465db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 -- 172.21.15.77:0/3078890555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e464c03a0 con 0x562e465cfb40 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2effd640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda28004af0 tx=0x7fda28005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958fb383c468d51d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 0x7fda180a59c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fda20013820 tx=0x7fda20009830 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda15ffb640 1 -- 172.21.15.77:0/3078890555 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28031db0 con 0x562e465cfb40 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda15ffb640 1 -- 172.21.15.77:0/3078890555 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2803f070 con 0x562e465cfb40 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.251+0000 7fda15ffb640 1 -- 172.21.15.77:0/3078890555 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda280042a0 con 0x562e465cfb40 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- 172.21.15.77:0/3078890555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 msgr2=0x562e465ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda28004af0 tx=0x7fda28005f70 comp rx=0 tx=0).stop 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- 172.21.15.77:0/3078890555 shutdown_connections 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 --2- 172.21.15.77:0/3078890555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465ce090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- 172.21.15.77:0/3078890555 >> 172.21.15.77:0/3078890555 conn(0x562e46597310 msgr2=0x562e465cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- 172.21.15.77:0/3078890555 shutdown_connections 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- 172.21.15.77:0/3078890555 wait complete. 2024-09-15T22:34:19.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.252+0000 7fda3852a740 1 -- start start 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 Processor -- start 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465ce5d0 0x562e465d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465d9370 0x562e465d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e465dc010 con 0x562e465cfb40 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e464ef060 con 0x562e465d9370 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562e465dc190 con 0x562e465ce5d0 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda3852a740 10 client.0 _ll_register_callbacks cb 0x562e462428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda2effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda2effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47252/0 (socket says 172.21.15.77:47252) 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda2effd640 1 -- 172.21.15.77:0/602128092 learned_addr learned my addr 172.21.15.77:0/602128092 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:19.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465ce5d0 0x562e465d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.253+0000 7fda2ffff640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465d9370 0x562e465d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 -- 172.21.15.77:0/602128092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465ce5d0 msgr2=0x562e465d8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465ce5d0 0x562e465d8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 -- 172.21.15.77:0/602128092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465d9370 msgr2=0x562e465d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465d9370 0x562e465d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 -- 172.21.15.77:0/602128092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e462a53c0 con 0x562e465cfb40 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda2effd640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d010 tx=0x7fda2409bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5cd5d8f24740eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda0effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b4070 con 0x562e465cfb40 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fda24098990 con 0x562e465cfb40 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2803f070 con 0x562e465cfb40 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda240a06d0 con 0x562e465cfb40 2024-09-15T22:34:19.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda3852a740 10 client.20472 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562e465e7ab0 con 0x562e465cfb40 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.254+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2803f070 con 0x562e465cfb40 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fda2409b770 con 0x562e465cfb40 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 10 client.20472 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 10 client.20472 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:34:19.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 20 client.20472 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e465e8f10 0x562e465eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 4 client.20472 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e46541860 con 0x562e465e8f10 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.255+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e465e8f10 0x562e465eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.256+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e465e8f10 0x562e465eb300 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4a18f4f630be1f4 server_cookie=3cf6e8ca787fabde in_seq=0 out_seq=0 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.256+0000 7fda0effd640 10 client.20472 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.257+0000 7fda0effd640 1 -- 172.21.15.77:0/602128092 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e46541860 con 0x562e465e8f10 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.257+0000 7fda0effd640 10 client.20472 handle_command_reply: tid=0 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.257+0000 7fda3852a740 1 client.20472 shutdown 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.257+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 msgr2=0x562e465d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.257+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d010 tx=0x7fda2409bd70 comp rx=0 tx=0).stop 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 shutdown_connections 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e465e8f10 0x562e465eb300 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465d9370 0x562e465d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d6010 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda0effd640 0 client.20472 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 --2- 172.21.15.77:0/602128092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465ce5d0 0x562e465d8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 >> 172.21.15.77:0/602128092 conn(0x562e46597310 msgr2=0x562e465d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 shutdown_connections 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 1 -- 172.21.15.77:0/602128092 wait complete. 2024-09-15T22:34:19.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.258+0000 7fda3852a740 20 client.20472 trim_cache size 0 max 16384 2024-09-15T22:34:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.274+0000 7fda3852a740 1 Processor -- start 2024-09-15T22:34:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 -- start start 2024-09-15T22:34:19.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda301bdd50 con 0x562e465cfb40 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda301bf890 con 0x562e465ce5d0 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda3852a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda301be420 con 0x562e465d9370 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47522/0 (socket says 172.21.15.77:47522) 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda2ffff640 1 -- 172.21.15.77:0/3011389247 learned_addr learned my addr 172.21.15.77:0/3011389247 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.275+0000 7fda2effd640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 -- 172.21.15.77:0/3011389247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 msgr2=0x562e465e52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 -- 172.21.15.77:0/3011389247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 msgr2=0x562e465d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 -- 172.21.15.77:0/3011389247 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e465cebe0 con 0x562e465d9370 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2effd640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda2ffff640 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda20009c20 tx=0x7fda20005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=834da0ccadbb72d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda157fa640 1 -- 172.21.15.77:0/3011389247 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20013e00 con 0x562e465d9370 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda157fa640 1 -- 172.21.15.77:0/3011389247 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda200130c0 con 0x562e465d9370 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.276+0000 7fda157fa640 1 -- 172.21.15.77:0/3011389247 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20010dd0 con 0x562e465d9370 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 -- 172.21.15.77:0/3011389247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 msgr2=0x562e465f0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda20009c20 tx=0x7fda20005f90 comp rx=0 tx=0).stop 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 -- 172.21.15.77:0/3011389247 shutdown_connections 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465d9370 0x562e465f0bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465cfb40 0x562e465d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 --2- 172.21.15.77:0/3011389247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 -- 172.21.15.77:0/3011389247 >> 172.21.15.77:0/3011389247 conn(0x562e46597310 msgr2=0x562e465d7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 -- 172.21.15.77:0/3011389247 shutdown_connections 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.277+0000 7fda3852a740 1 -- 172.21.15.77:0/3011389247 wait complete. 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.278+0000 7fda3852a740 1 -- start start 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.278+0000 7fda3852a740 1 Processor -- start 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda301bf1c0 con 0x562e465d9370 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda301beaf0 con 0x562e465ce5d0 2024-09-15T22:34:19.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda301bbd50 con 0x562e465cfb40 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40734/0 (socket says 172.21.15.77:40734) 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2f7fe640 1 -- 172.21.15.77:0/4080248280 learned_addr learned my addr 172.21.15.77:0/4080248280 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda3852a740 10 client.0 _ll_register_callbacks cb 0x7fda2409c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 -- 172.21.15.77:0/4080248280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 msgr2=0x562e465e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 -- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 msgr2=0x562e465de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 -- 172.21.15.77:0/4080248280 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e465de930 con 0x562e465cfb40 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda2effd640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465e1960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562e465d75c0 tx=0x7fda240095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1fc6c379e5a724d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.279+0000 7fda0effd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240ae030 con 0x562e465cfb40 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda200130c0 con 0x562e465cfb40 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fda240b5260 con 0x562e465cfb40 2024-09-15T22:34:19.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda24082050 con 0x562e465cfb40 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 10 client.29137 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562e46654260 con 0x562e465cfb40 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda200130c0 con 0x562e465cfb40 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fda240a4b00 con 0x562e465cfb40 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 10 client.29137 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 10 client.29137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 20 client.29137 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e466548c0 0x562e46656cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 4 client.29137 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e4653bd70 con 0x562e466548c0 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.280+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e466548c0 0x562e46656cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda0effd640 10 client.29137 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:34:19.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda2f7fe640 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e466548c0 0x562e46656cb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bef96987d51bea65 server_cookie=f93d3864df91cdfd in_seq=0 out_seq=0 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda0effd640 1 -- 172.21.15.77:0/4080248280 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e4653bd70 con 0x562e466548c0 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda0effd640 10 client.29137 handle_command_reply: tid=0 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 client.29137 shutdown 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 msgr2=0x562e465e1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465e1960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562e465d75c0 tx=0x7fda240095b0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 shutdown_connections 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x562e466548c0 0x562e46656cb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e465d9370 0x562e465de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda0effd640 0 client.29137 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562e465cfb40 0x562e465e1960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 --2- 172.21.15.77:0/4080248280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e465ce5d0 0x562e465e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 >> 172.21.15.77:0/4080248280 conn(0x562e46597310 msgr2=0x562e465d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 shutdown_connections 2024-09-15T22:34:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.281+0000 7fda3852a740 1 -- 172.21.15.77:0/4080248280 wait complete. 2024-09-15T22:34:19.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda3852a740 20 client.29137 trim_cache size 0 max 16384 2024-09-15T22:34:19.294 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:34:19.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 msgr2=0x7fda180a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 0x7fda180a59c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fda20013820 tx=0x7fda20009830 comp rx=0 tx=0).stop 2024-09-15T22:34:19.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 msgr2=0x7fda301a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400edf0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 shutdown_connections 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fda180a3500 0x7fda180a59c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda3011b140 0x7fda301abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda30109830 0x7fda301a7770 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 --2- 172.21.15.77:0/3854568061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda30109450 0x7fda301a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.282+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 >> 172.21.15.77:0/3854568061 conn(0x7fda3006e140 msgr2=0x7fda3011be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.283+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 shutdown_connections 2024-09-15T22:34:19.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:19.283+0000 7fda36151640 1 -- 172.21.15.77:0/3854568061 wait complete. 2024-09-15T22:34:19.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 152039 15.38 MB/sec execute 380 sec latency 10279.309 ms 2024-09-15T22:34:20.181 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:20 smithi077 ceph-mon[30980]: pgmap v692: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:20.220 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:34:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:20 smithi196 ceph-mon[35109]: pgmap v692: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:20 smithi117 ceph-mon[35021]: pgmap v692: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-09-15T22:34:20.592 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:20.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 152380 15.38 MB/sec execute 381 sec latency 1308.516 ms 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 -- 172.21.15.77:0/1061794153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81011e0 msgr2=0x7f4ea8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 --2- 172.21.15.77:0/1061794153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81011e0 0x7f4ea8105980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e9801a940 tx=0x7f4e9803b450 comp rx=0 tx=0).stop 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 -- 172.21.15.77:0/1061794153 shutdown_connections 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 --2- 172.21.15.77:0/1061794153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 --2- 172.21.15.77:0/1061794153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81011e0 0x7f4ea8105980 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 --2- 172.21.15.77:0/1061794153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea81008c0 0x7f4ea8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 -- 172.21.15.77:0/1061794153 >> 172.21.15.77:0/1061794153 conn(0x7f4ea80fc7d0 msgr2=0x7f4ea80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 -- 172.21.15.77:0/1061794153 shutdown_connections 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.449+0000 7f4eae241640 1 -- 172.21.15.77:0/1061794153 wait complete. 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.450+0000 7f4eae241640 1 Processor -- start 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 -- start start 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81008c0 0x7f4ea8114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea81011e0 0x7f4ea8110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81098c0 con 0x7f4ea8105ec0 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8109740 con 0x7f4ea81011e0 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4eae241640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8109a40 con 0x7f4ea81008c0 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47292/0 (socket says 172.21.15.77:47292) 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4ea7fff640 1 -- 172.21.15.77:0/653566894 learned_addr learned my addr 172.21.15.77:0/653566894 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.451+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea81011e0 0x7f4ea8110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 -- 172.21.15.77:0/653566894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81008c0 msgr2=0x7f4ea8114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81008c0 0x7f4ea8114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 -- 172.21.15.77:0/653566894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea81011e0 msgr2=0x7f4ea8110f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea81011e0 0x7f4ea8110f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 -- 172.21.15.77:0/653566894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea8111b70 con 0x7f4ea8105ec0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000eb10 tx=0x7f4ea000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea4ff9640 1 -- 172.21.15.77:0/653566894 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0004b20 con 0x7f4ea8105ec0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81a2a60 con 0x7f4ea8105ec0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a2eb0 con 0x7f4ea8105ec0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.452+0000 7f4ea4ff9640 1 -- 172.21.15.77:0/653566894 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0004cc0 con 0x7f4ea8105ec0 2024-09-15T22:34:21.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.453+0000 7f4ea4ff9640 1 -- 172.21.15.77:0/653566894 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0010790 con 0x7f4ea8105ec0 2024-09-15T22:34:21.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.457+0000 7f4eb061a740 1 Processor -- start 2024-09-15T22:34:21.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 -- start start 2024-09-15T22:34:21.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4ea4ff9640 1 -- 172.21.15.77:0/653566894 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ea0005020 con 0x7f4ea8105ec0 2024-09-15T22:34:21.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b18c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b199760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f94b0f22f0 con 0x55f94b18c5d0 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f94b099cf0 con 0x55f94b197370 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.458+0000 7f4eb061a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f94b0ee2b0 con 0x55f94b18db40 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.459+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.459+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47294/0 (socket says 172.21.15.77:47294) 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.459+0000 7f4ea77fe640 1 -- 172.21.15.77:0/2751724590 learned_addr learned my addr 172.21.15.77:0/2751724590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:21.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.459+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.459+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b18c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea77fe640 1 -- 172.21.15.77:0/2751724590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 msgr2=0x55f94b18c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b18c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea77fe640 1 -- 172.21.15.77:0/2751724590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 msgr2=0x55f94b199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea77fe640 1 -- 172.21.15.77:0/2751724590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94b07e3a0 con 0x55f94b18c5d0 2024-09-15T22:34:21.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea4ff9640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 0x7f4e900a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.460+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 0x7f4e900a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00bb10 tx=0x7f4e9c00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e643043b5a8d2861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 0x7f4e900a59c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98052040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4e8dffb640 1 -- 172.21.15.77:0/2751724590 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c016070 con 0x55f94b18c5d0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4e8dffb640 1 -- 172.21.15.77:0/2751724590 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c005500 con 0x55f94b18c5d0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4e8dffb640 1 -- 172.21.15.77:0/2751724590 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c010460 con 0x55f94b18c5d0 2024-09-15T22:34:21.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.461+0000 7f4ea4ff9640 1 -- 172.21.15.77:0/653566894 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f4ea00d0560 con 0x7f4ea8105ec0 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 -- 172.21.15.77:0/2751724590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 msgr2=0x55f94b18ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00bb10 tx=0x7f4e9c00d2e0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 -- 172.21.15.77:0/2751724590 shutdown_connections 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18c5d0 0x55f94b18ca10 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 --2- 172.21.15.77:0/2751724590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b18c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.462+0000 7f4eb061a740 1 -- 172.21.15.77:0/2751724590 >> 172.21.15.77:0/2751724590 conn(0x55f94b155310 msgr2=0x55f94b188c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.463+0000 7f4eb061a740 1 -- 172.21.15.77:0/2751724590 shutdown_connections 2024-09-15T22:34:21.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.463+0000 7f4eb061a740 1 -- 172.21.15.77:0/2751724590 wait complete. 2024-09-15T22:34:21.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.463+0000 7f4eb061a740 1 -- start start 2024-09-15T22:34:21.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.463+0000 7f4eb061a740 1 Processor -- start 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.464+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18db40 0x55f94b1941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b194710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 0x55f94b1a3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b194710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40772/0 (socket says 172.21.15.77:40772) 2024-09-15T22:34:21.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4ea6ffd640 1 -- 172.21.15.77:0/4048680515 learned_addr learned my addr 172.21.15.77:0/4048680515 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 0x55f94b1a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f94b19a070 con 0x55f94b18db40 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f94b0ad060 con 0x55f94b197370 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f94b19a1f0 con 0x55f94b194c50 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.465+0000 7f4eb061a740 10 client.0 _ll_register_callbacks cb 0x55f94ae008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:21.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18db40 0x55f94b1941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea7fff640 1 -- 172.21.15.77:0/4048680515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 msgr2=0x55f94b194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea7fff640 1 -- 172.21.15.77:0/4048680515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18db40 msgr2=0x55f94b1941d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18db40 0x55f94b1941d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.466+0000 7f4ea7fff640 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94ae633c0 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.467+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 0x55f94b1a3580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00cd8c0 tx=0x7f4ea009c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf2a3de9400f65b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.467+0000 7f4e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00b4020 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c005500 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f4ea00d0130 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ea00cdd50 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4eb061a740 10 client.29146 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.468+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f94b1a6120 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c005500 con 0x55f94b194c50 2024-09-15T22:34:21.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4ea009b8a0 con 0x55f94b194c50 2024-09-15T22:34:21.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 10 client.29146 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 10 client.29146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 20 client.29146 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b1a7a70 0x55f94b1a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 4 client.29146 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.469+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f94b0ff860 con 0x55f94b1a7a70 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.471+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b1a7a70 0x55f94b1a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.471+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b1a7a70 0x55f94b1a9f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ddc21e97efc813e5 server_cookie=c67d090ddbc9e27d in_seq=0 out_seq=0 2024-09-15T22:34:21.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.471+0000 7f4e86ffd640 10 client.29146 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:34:21.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.472+0000 7f4e86ffd640 1 -- 172.21.15.77:0/4048680515 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f94b0ff860 con 0x55f94b1a7a70 2024-09-15T22:34:21.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.472+0000 7f4e86ffd640 10 client.29146 handle_command_reply: tid=0 2024-09-15T22:34:21.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.472+0000 7f4eb061a740 1 client.29146 shutdown 2024-09-15T22:34:21.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 msgr2=0x55f94b1a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 0x55f94b1a3580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00cd8c0 tx=0x7f4ea009c310 comp rx=0 tx=0).stop 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 shutdown_connections 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b1a7a70 0x55f94b1a9f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b194c50 0x55f94b1a3580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b197370 0x55f94b194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 --2- 172.21.15.77:0/4048680515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b18db40 0x55f94b1941d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 >> 172.21.15.77:0/4048680515 conn(0x55f94b155310 msgr2=0x55f94b18e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.473+0000 7f4e86ffd640 0 client.29146 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:34:21.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.474+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 shutdown_connections 2024-09-15T22:34:21.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.474+0000 7f4eb061a740 1 -- 172.21.15.77:0/4048680515 wait complete. 2024-09-15T22:34:21.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.474+0000 7f4eb061a740 20 client.29146 trim_cache size 0 max 16384 2024-09-15T22:34:21.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.497+0000 7f4eb061a740 1 Processor -- start 2024-09-15T22:34:21.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.497+0000 7f4eb061a740 1 -- start start 2024-09-15T22:34:21.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.497+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b194c50 0x55f94b1a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b197370 0x55f94b188700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4eb061a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9400ffb0 con 0x55f94b194c50 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4eb061a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94011af0 con 0x55f94b18db40 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4eb061a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94010680 con 0x55f94b197370 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40774/0 (socket says 172.21.15.77:40774) 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4ea77fe640 1 -- 172.21.15.77:0/1957738237 learned_addr learned my addr 172.21.15.77:0/1957738237 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b194c50 0x55f94b1a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.498+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b197370 0x55f94b188700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 -- 172.21.15.77:0/1957738237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b197370 msgr2=0x55f94b188700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b197370 0x55f94b188700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 -- 172.21.15.77:0/1957738237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b194c50 msgr2=0x55f94b1a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b194c50 0x55f94b1a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 -- 172.21.15.77:0/1957738237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94b196fe0 con 0x55f94b18db40 2024-09-15T22:34:21.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001eb0 tx=0x7f4e9c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f5f3762cb17c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4e8d7fa640 1 -- 172.21.15.77:0/1957738237 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c016040 con 0x55f94b18db40 2024-09-15T22:34:21.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4e8d7fa640 1 -- 172.21.15.77:0/1957738237 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c0049b0 con 0x55f94b18db40 2024-09-15T22:34:21.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.499+0000 7f4e8d7fa640 1 -- 172.21.15.77:0/1957738237 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00dea0 con 0x55f94b18db40 2024-09-15T22:34:21.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 -- 172.21.15.77:0/1957738237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 msgr2=0x55f94b1a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001eb0 tx=0x7f4e9c0046c0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 -- 172.21.15.77:0/1957738237 shutdown_connections 2024-09-15T22:34:21.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b197370 0x55f94b188700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b194c50 0x55f94b1a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1957738237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b18db40 0x55f94b1a3370 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 -- 172.21.15.77:0/1957738237 >> 172.21.15.77:0/1957738237 conn(0x55f94b155310 msgr2=0x55f94b194190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 -- 172.21.15.77:0/1957738237 shutdown_connections 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.500+0000 7f4eb061a740 1 -- 172.21.15.77:0/1957738237 wait complete. 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.501+0000 7f4eb061a740 1 -- start start 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.501+0000 7f4eb061a740 1 Processor -- start 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b1a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b197370 0x55f94b19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94011420 con 0x55f94b197370 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94010d50 con 0x55f94b194c50 2024-09-15T22:34:21.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9400dfb0 con 0x55f94b18db40 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4eb061a740 10 client.0 _ll_register_callbacks cb 0x7f4ea0098760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b1a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b1a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47602/0 (socket says 172.21.15.77:47602) 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea77fe640 1 -- 172.21.15.77:0/1639489550 learned_addr learned my addr 172.21.15.77:0/1639489550 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40786/0 (socket says 172.21.15.77:40786) 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea7fff640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b197370 0x55f94b19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 -- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 msgr2=0x55f94b1a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b1a2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 -- 172.21.15.77:0/1639489550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b197370 msgr2=0x55f94b19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b197370 0x55f94b19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.502+0000 7f4ea6ffd640 1 -- 172.21.15.77:0/1639489550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94b19ea20 con 0x55f94b194c50 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4ea6ffd640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e980057c0 tx=0x7f4e9801a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=151be5f4e0bd2f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:21.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98046040 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c0049b0 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f4e98004c60 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4e98040510 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4eb061a740 10 client.29204 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.503+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f94b19fd70 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c0049b0 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4e98066070 con 0x55f94b194c50 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 10 client.29204 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 10 client.29204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 20 client.29204 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b2128b0 0x55f94b214ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 4 client.29204 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:34:21.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f94b0f9d70 con 0x55f94b2128b0 2024-09-15T22:34:21.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b2128b0 0x55f94b214ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.504+0000 7f4e86ffd640 10 client.29204 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4ea77fe640 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b2128b0 0x55f94b214ca0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1dc4a7783cc66db server_cookie=4493848b4f7a481 in_seq=0 out_seq=0 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4e86ffd640 1 -- 172.21.15.77:0/1639489550 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f94b0f9d70 con 0x55f94b2128b0 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4e86ffd640 10 client.29204 handle_command_reply: tid=0 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 client.29204 shutdown 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 msgr2=0x55f94b1a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e980057c0 tx=0x7f4e9801a270 comp rx=0 tx=0).stop 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 shutdown_connections 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55f94b2128b0 0x55f94b214ca0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f94b197370 0x55f94b19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f94b194c50 0x55f94b1a2e40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 --2- 172.21.15.77:0/1639489550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f94b18db40 0x55f94b1a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4e86ffd640 0 client.29204 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 >> 172.21.15.77:0/1639489550 conn(0x55f94b155310 msgr2=0x55f94b188700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 shutdown_connections 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 1 -- 172.21.15.77:0/1639489550 wait complete. 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.505+0000 7f4eb061a740 20 client.29204 trim_cache size 0 max 16384 2024-09-15T22:34:21.511 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 msgr2=0x7f4e900a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 0x7f4e900a59c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98052040 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 msgr2=0x7f4ea8111470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000eb10 tx=0x7f4ea000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 shutdown_connections 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4e900a3500 0x7f4e900a59c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ea8105ec0 0x7f4ea8111470 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ea81011e0 0x7f4ea8110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 --2- 172.21.15.77:0/653566894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ea81008c0 0x7f4ea8114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:21.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 >> 172.21.15.77:0/653566894 conn(0x7f4ea80fc7d0 msgr2=0x7f4ea8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:21.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 shutdown_connections 2024-09-15T22:34:21.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:21.506+0000 7f4eae241640 1 -- 172.21.15.77:0/653566894 wait complete. 2024-09-15T22:34:21.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 152780 15.40 MB/sec execute 382 sec latency 2308.547 ms 2024-09-15T22:34:22.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:22 smithi077 ceph-mon[30980]: pgmap v693: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-09-15T22:34:22.342 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:34:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:22 smithi196 ceph-mon[35109]: pgmap v693: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-09-15T22:34:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:22 smithi117 ceph-mon[35021]: pgmap v693: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-09-15T22:34:22.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 153179 15.40 MB/sec execute 383 sec latency 3308.580 ms 2024-09-15T22:34:22.666 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 -- 172.21.15.77:0/3171525519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 msgr2=0x7f583c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 --2- 172.21.15.77:0/3171525519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c103110 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5830009930 tx=0x7f583002a420 comp rx=0 tx=0).stop 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 -- 172.21.15.77:0/3171525519 shutdown_connections 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 --2- 172.21.15.77:0/3171525519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1082a0 0x7f583c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 --2- 172.21.15.77:0/3171525519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1036e0 0x7f583c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 --2- 172.21.15.77:0/3171525519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c103110 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 -- 172.21.15.77:0/3171525519 >> 172.21.15.77:0/3171525519 conn(0x7f583c0fc7d0 msgr2=0x7f583c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 -- 172.21.15.77:0/3171525519 shutdown_connections 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.210+0000 7f58431d7640 1 -- 172.21.15.77:0/3171525519 wait complete. 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 Processor -- start 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 -- start start 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1036e0 0x7f583c1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 0x7f583c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1120e0 con 0x7f583c102d30 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f583c111f60 con 0x7f583c1082a0 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f58431d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f583c112260 con 0x7f583c1036e0 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f5840f73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f5840f73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47348/0 (socket says 172.21.15.77:47348) 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f5840f73640 1 -- 172.21.15.77:0/1264701224 learned_addr learned my addr 172.21.15.77:0/1264701224 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:23.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f5841774640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 0x7f583c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.211+0000 7f583bfff640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1036e0 0x7f583c1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 -- 172.21.15.77:0/1264701224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1036e0 msgr2=0x7f583c1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1036e0 0x7f583c1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 -- 172.21.15.77:0/1264701224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 msgr2=0x7f583c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 0x7f583c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 -- 172.21.15.77:0/1264701224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c19ad20 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5841774640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 0x7f583c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5840f73640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f583002a940 tx=0x7f5830003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5839ffb640 1 -- 172.21.15.77:0/1264701224 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583003f070 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c19afb0 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5839ffb640 1 -- 172.21.15.77:0/1264701224 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5830031c90 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f5839ffb640 1 -- 172.21.15.77:0/1264701224 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830035030 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.212+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1a7300 con 0x7f583c102d30 2024-09-15T22:34:23.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.215+0000 7f5839ffb640 1 -- 172.21.15.77:0/1264701224 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5830042070 con 0x7f583c102d30 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.215+0000 7f58455b0740 1 Processor -- start 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.215+0000 7f5839ffb640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 0x7f582c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.215+0000 7f58455b0740 1 -- start start 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95ab0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95abd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5841774640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 0x7f582c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a95a162f0 con 0x561a95ab1b40 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5839ffb640 1 -- 172.21.15.77:0/1264701224 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f58300f06c0 con 0x7f583c102d30 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a959bdcf0 con 0x561a95abb370 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f58455b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a95a122b0 con 0x561a95ab05d0 2024-09-15T22:34:23.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5841774640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95abd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47364/0 (socket says 172.21.15.77:47364) 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 -- 172.21.15.77:0/1544580023 learned_addr learned my addr 172.21.15.77:0/1544580023 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 -- 172.21.15.77:0/1544580023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 msgr2=0x561a95ab0a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95ab0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 -- 172.21.15.77:0/1544580023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 msgr2=0x561a95abd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95abd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 -- 172.21.15.77:0/1544580023 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a959a23a0 con 0x561a95ab1b40 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5841774640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 0x7f582c0a58f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58340099e0 tx=0x7f5834009ef0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.216+0000 7f5840f73640 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f58300c4d50 tx=0x7f5830005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76bc271a9aa5ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.217+0000 7f5826ffd640 1 -- 172.21.15.77:0/1544580023 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830049070 con 0x561a95ab1b40 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.217+0000 7f5826ffd640 1 -- 172.21.15.77:0/1544580023 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58300c5280 con 0x561a95ab1b40 2024-09-15T22:34:23.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.217+0000 7f5826ffd640 1 -- 172.21.15.77:0/1544580023 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300f0310 con 0x561a95ab1b40 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 -- 172.21.15.77:0/1544580023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 msgr2=0x561a95ab0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f58300c4d50 tx=0x7f5830005dc0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 -- 172.21.15.77:0/1544580023 shutdown_connections 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95abd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95ab0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 --2- 172.21.15.77:0/1544580023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab0090 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 -- 172.21.15.77:0/1544580023 >> 172.21.15.77:0/1544580023 conn(0x561a95a79310 msgr2=0x561a95aacc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.218+0000 7f58455b0740 1 -- 172.21.15.77:0/1544580023 shutdown_connections 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.219+0000 7f58455b0740 1 -- 172.21.15.77:0/1544580023 wait complete. 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.219+0000 7f58455b0740 1 -- start start 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.219+0000 7f58455b0740 1 Processor -- start 2024-09-15T22:34:23.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95ab8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a95abde30 con 0x561a95ab1b40 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a959d1060 con 0x561a95abb370 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a95abdfb0 con 0x561a95ab05d0 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f58455b0740 10 client.0 _ll_register_callbacks cb 0x561a957248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47628/0 (socket says 172.21.15.77:47628) 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 -- 172.21.15.77:0/813624886 learned_addr learned my addr 172.21.15.77:0/813624886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5841774640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95ab8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f583bfff640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 -- 172.21.15.77:0/813624886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 msgr2=0x561a95ab8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95ab8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 -- 172.21.15.77:0/813624886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 msgr2=0x561a95ab80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.220+0000 7f5840f73640 1 -- 172.21.15.77:0/813624886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a957873c0 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f5840f73640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58300edb30 tx=0x7f5830005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89dc6ceba5a8176 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f581bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58300da060 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58300c5280 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f58300b8d30 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f58300d4070 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f58455b0740 10 client.29161 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a95abde30 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.221+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58300c5280 con 0x561a95ab05d0 2024-09-15T22:34:23.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f583003bce0 con 0x561a95ab05d0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 10 client.29161 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 10 client.29161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 20 client.29161 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95acaab0 0x561a95accea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 4 client.29161 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a95a23860 con 0x561a95acaab0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f583bfff640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95acaab0 0x561a95accea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f581bfff640 10 client.29161 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.222+0000 7f583bfff640 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95acaab0 0x561a95accea0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99077b8a1063bc67 server_cookie=b01dd45bac4cf6ca in_seq=0 out_seq=0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.223+0000 7f581bfff640 1 -- 172.21.15.77:0/813624886 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a95a23860 con 0x561a95acaab0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.223+0000 7f581bfff640 10 client.29161 handle_command_reply: tid=0 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.223+0000 7f58455b0740 1 client.29161 shutdown 2024-09-15T22:34:23.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 msgr2=0x561a95abae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58300edb30 tx=0x7f5830005e50 comp rx=0 tx=0).stop 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 shutdown_connections 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95acaab0 0x561a95accea0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95abb370 0x561a95ab8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab1b40 0x561a95ab80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f581bfff640 0 client.29161 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 --2- 172.21.15.77:0/813624886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab05d0 0x561a95abae10 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f58300edb30 tx=0x7f5830005e50 comp rx=0 tx=0).stop 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 >> 172.21.15.77:0/813624886 conn(0x561a95a79310 msgr2=0x561a95ab2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 shutdown_connections 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 1 -- 172.21.15.77:0/813624886 wait complete. 2024-09-15T22:34:23.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.224+0000 7f58455b0740 20 client.29161 trim_cache size 0 max 16384 2024-09-15T22:34:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.248+0000 7f58455b0740 1 Processor -- start 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.249+0000 7f58455b0740 1 -- start start 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95ab6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95abb370 0x561a95ad2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1b1340 con 0x561a95ab74b0 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1b2e80 con 0x561a95ab1b40 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f5841774640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f58455b0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1b1a10 con 0x561a95abb370 2024-09-15T22:34:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f5841774640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47376/0 (socket says 172.21.15.77:47376) 2024-09-15T22:34:23.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f5841774640 1 -- 172.21.15.77:0/4005295833 learned_addr learned my addr 172.21.15.77:0/4005295833 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:23.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.250+0000 7f583bfff640 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95abb370 0x561a95ad2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 -- 172.21.15.77:0/4005295833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95abb370 msgr2=0x561a95ad2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95abb370 0x561a95ad2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 -- 172.21.15.77:0/4005295833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 msgr2=0x561a95ab6f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95ab6f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 -- 172.21.15.77:0/4005295833 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a95aad960 con 0x561a95ab74b0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.251+0000 7f5841774640 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5834013db0 tx=0x7f5834009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ee467ea6f3d39b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.252+0000 7f58267fc640 1 -- 172.21.15.77:0/4005295833 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834003d40 con 0x561a95ab74b0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.252+0000 7f58267fc640 1 -- 172.21.15.77:0/4005295833 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834013130 con 0x561a95ab74b0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.252+0000 7f58267fc640 1 -- 172.21.15.77:0/4005295833 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834010be0 con 0x561a95ab74b0 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- 172.21.15.77:0/4005295833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 msgr2=0x561a95ad2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5834013db0 tx=0x7f5834009500 comp rx=0 tx=0).stop 2024-09-15T22:34:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- 172.21.15.77:0/4005295833 shutdown_connections 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95ab74b0 0x561a95ad2c60 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95abb370 0x561a95ad2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 --2- 172.21.15.77:0/4005295833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95ab6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- 172.21.15.77:0/4005295833 >> 172.21.15.77:0/4005295833 conn(0x561a95a79310 msgr2=0x561a95ab6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- 172.21.15.77:0/4005295833 shutdown_connections 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- 172.21.15.77:0/4005295833 wait complete. 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 -- start start 2024-09-15T22:34:23.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.253+0000 7f58455b0740 1 Processor -- start 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95abf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 0x561a95abfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1b27b0 con 0x561a95abb370 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f583bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 0x561a95abfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f5841774640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f5840f73640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95abf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f5841774640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47382/0 (socket says 172.21.15.77:47382) 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f5840f73640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95abf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40830/0 (socket says 172.21.15.77:40830) 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f5841774640 1 -- 172.21.15.77:0/2916213456 learned_addr learned my addr 172.21.15.77:0/2916213456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1b20e0 con 0x561a95ab1b40 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1af340 con 0x561a95ab74b0 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.254+0000 7f58455b0740 10 client.0 _ll_register_callbacks cb 0x7f5834013a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 -- 172.21.15.77:0/2916213456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 msgr2=0x561a95abf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95abf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 -- 172.21.15.77:0/2916213456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 msgr2=0x561a95b3cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a95ac7000 con 0x561a95ab74b0 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f5841774640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f583bfff640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 0x561a95abfb30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f583c19b730 tx=0x7f5828011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc264102f656220d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f581bfff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828004220 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.255+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5834013130 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f58280043c0 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5828005020 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 10 client.29167 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561a95b3e5b0 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834013130 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f5828021030 con 0x561a95ab74b0 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 10 client.29167 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 10 client.29167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 20 client.29167 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95b3edb0 0x561a95b411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 4 client.29167 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:34:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.256+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a95a1dd70 con 0x561a95b3edb0 2024-09-15T22:34:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.257+0000 7f5840f73640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95b3edb0 0x561a95b411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.258+0000 7f5840f73640 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95b3edb0 0x561a95b411d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d75b2039e1e9590 server_cookie=b950c6aee707c6dd in_seq=0 out_seq=0 2024-09-15T22:34:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.258+0000 7f581bfff640 10 client.29167 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:34:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f581bfff640 1 -- 172.21.15.77:0/2916213456 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a95a1dd70 con 0x561a95b3edb0 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f581bfff640 10 client.29167 handle_command_reply: tid=0 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 client.29167 shutdown 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 msgr2=0x561a95abfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 0x561a95abfb30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f583c19b730 tx=0x7f5828011810 comp rx=0 tx=0).stop 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 shutdown_connections 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561a95b3edb0 0x561a95b411d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a95abb370 0x561a95b3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f581bfff640 0 client.29167 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a95ab74b0 0x561a95abfb30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 --2- 172.21.15.77:0/2916213456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a95ab1b40 0x561a95abf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.259+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 >> 172.21.15.77:0/2916213456 conn(0x561a95a79310 msgr2=0x561a95aab500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 shutdown_connections 2024-09-15T22:34:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58455b0740 1 -- 172.21.15.77:0/2916213456 wait complete. 2024-09-15T22:34:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58455b0740 20 client.29167 trim_cache size 0 max 16384 2024-09-15T22:34:23.265 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:34:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 msgr2=0x7f582c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 0x7f582c0a58f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58340099e0 tx=0x7f5834009ef0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 msgr2=0x7f583c1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f583002a940 tx=0x7f5830003b90 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 shutdown_connections 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f582c0a3430 0x7f582c0a58f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f583c1082a0 0x7f583c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f583c1036e0 0x7f583c1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 --2- 172.21.15.77:0/1264701224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f583c102d30 0x7f583c1a0430 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.260+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 >> 172.21.15.77:0/1264701224 conn(0x7f583c0fc7d0 msgr2=0x7f583c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.261+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 shutdown_connections 2024-09-15T22:34:23.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:23.261+0000 7f58431d7640 1 -- 172.21.15.77:0/1264701224 wait complete. 2024-09-15T22:34:23.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 153620 15.41 MB/sec execute 384 sec latency 4308.614 ms 2024-09-15T22:34:24.005 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:34:24.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:24 smithi077 ceph-mon[30980]: pgmap v694: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 303 op/s 2024-09-15T22:34:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:24 smithi196 ceph-mon[35109]: pgmap v694: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 303 op/s 2024-09-15T22:34:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:24 smithi117 ceph-mon[35021]: pgmap v694: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 303 op/s 2024-09-15T22:34:24.366 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:24.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 154172 15.45 MB/sec execute 385 sec latency 5308.644 ms 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 -- 172.21.15.77:0/3096781814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e941009b0 msgr2=0x7f9e94100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 --2- 172.21.15.77:0/3096781814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e941009b0 0x7f9e94100d90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009ca0 tx=0x7f9e9002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 -- 172.21.15.77:0/3096781814 shutdown_connections 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 --2- 172.21.15.77:0/3096781814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e9410e4d0 0x7f9e941108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 --2- 172.21.15.77:0/3096781814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 0x7f9e9410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 --2- 172.21.15.77:0/3096781814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e941009b0 0x7f9e94100d90 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 -- 172.21.15.77:0/3096781814 >> 172.21.15.77:0/3096781814 conn(0x7f9e940fc7d0 msgr2=0x7f9e940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.926+0000 7f9e9b923640 1 -- 172.21.15.77:0/3096781814 shutdown_connections 2024-09-15T22:34:24.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.927+0000 7f9e9b923640 1 -- 172.21.15.77:0/3096781814 wait complete. 2024-09-15T22:34:24.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.927+0000 7f9e9b923640 1 Processor -- start 2024-09-15T22:34:24.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.927+0000 7f9e9b923640 1 -- start start 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 0x7f9e9419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9410e4d0 0x7f9e9419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94114280 con 0x7f9e9410e4d0 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47658/0 (socket says 172.21.15.77:47658) 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94114100 con 0x7f9e94101360 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 -- 172.21.15.77:0/2982318501 learned_addr learned my addr 172.21.15.77:0/2982318501 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94114400 con 0x7f9e941009b0 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 0x7f9e9419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9410e4d0 0x7f9e9419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 -- 172.21.15.77:0/2982318501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 msgr2=0x7f9e9419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 0x7f9e9419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 -- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9410e4d0 msgr2=0x7f9e9419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9410e4d0 0x7f9e9419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.928+0000 7f9e996bf640 1 -- 172.21.15.77:0/2982318501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9419f570 con 0x7f9e941009b0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e996bf640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e90004af0 tx=0x7f9e90003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e867fc640 1 -- 172.21.15.77:0/2982318501 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90031750 con 0x7f9e941009b0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9419f800 con 0x7f9e941009b0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e867fc640 1 -- 172.21.15.77:0/2982318501 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e900318f0 con 0x7f9e941009b0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941a7140 con 0x7f9e941009b0 2024-09-15T22:34:24.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.929+0000 7f9e867fc640 1 -- 172.21.15.77:0/2982318501 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9002fc10 con 0x7f9e941009b0 2024-09-15T22:34:24.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.932+0000 7f9e9dcfc740 1 Processor -- start 2024-09-15T22:34:24.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.932+0000 7f9e9dcfc740 1 -- start start 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.932+0000 7f9e867fc640 1 -- 172.21.15.77:0/2982318501 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9e900305c0 con 0x7f9e941009b0 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.932+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.932+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371075a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c371082760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c370fdb2f0 con 0x55c371080370 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c370f82cf0 con 0x55c3710755d0 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c370fd72b0 con 0x55c371076b40 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47660/0 (socket says 172.21.15.77:47660) 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3698417079 learned_addr learned my addr 172.21.15.77:0/3698417079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c371082760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e996bf640 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3698417079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 msgr2=0x55c371075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3698417079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 msgr2=0x55c371082760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c371082760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3698417079 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c370f673a0 con 0x55c371076b40 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e867fc640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 0x7f9e800a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009c20 tx=0x7f9e8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16183188c4927e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.933+0000 7f9e996bf640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 0x7f9e800a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.934+0000 7f9e7f7fe640 1 -- 172.21.15.77:0/3698417079 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c01e070 con 0x55c371076b40 2024-09-15T22:34:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.934+0000 7f9e7f7fe640 1 -- 172.21.15.77:0/3698417079 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c019b10 con 0x55c371076b40 2024-09-15T22:34:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.934+0000 7f9e867fc640 1 -- 172.21.15.77:0/2982318501 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f9e900f1460 con 0x7f9e941009b0 2024-09-15T22:34:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.934+0000 7f9e7f7fe640 1 -- 172.21.15.77:0/3698417079 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00e9e0 con 0x55c371076b40 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3698417079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 msgr2=0x55c371075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009c20 tx=0x7f9e8c011810 comp rx=0 tx=0).stop 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e996bf640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 0x7f9e800a5940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55c371072ac0 tx=0x7f9e900b9040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3698417079 shutdown_connections 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c371082760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3698417079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371075090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3698417079 >> 172.21.15.77:0/3698417079 conn(0x55c37103e310 msgr2=0x55c371071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3698417079 shutdown_connections 2024-09-15T22:34:24.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3698417079 wait complete. 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 -- start start 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.935+0000 7f9e9dcfc740 1 Processor -- start 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371080370 0x55c37107d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47424/0 (socket says 172.21.15.77:47424) 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e98ebe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e98ebe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40858/0 (socket says 172.21.15.77:40858) 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3260037127 learned_addr learned my addr 172.21.15.77:0/3260037127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c371083070 con 0x55c3710755d0 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c370f96060 con 0x55c371076b40 2024-09-15T22:34:24.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3710831f0 con 0x55c371080370 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.936+0000 7f9e9dcfc740 10 client.0 _ll_register_callbacks cb 0x55c370ce98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371080370 0x55c37107d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3260037127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371080370 msgr2=0x55c37107d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371080370 0x55c37107d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 msgr2=0x55c37107d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c370d4c3c0 con 0x55c3710755d0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009bf0 tx=0x7f9e8c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e361e3f6d5b9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.937+0000 7f9e7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02c060 con 0x55c3710755d0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f9e8c004230 con 0x55c3710755d0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8c019b10 con 0x55c3710755d0 2024-09-15T22:34:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c033050 con 0x55c3710755d0 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e9dcfc740 10 client.20523 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c37108eab0 con 0x55c3710755d0 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.938+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c004430 con 0x55c3710755d0 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9e8c024040 con 0x55c3710755d0 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 10 client.20523 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 10 client.20523 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:34:24.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 20 client.20523 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c3710922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 4 client.20523 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e8c019b10 con 0x55c37108feb0 2024-09-15T22:34:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.939+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c3710922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.940+0000 7f9e7cff9640 10 client.20523 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:34:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.940+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c3710922a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a45a182d9c76787 server_cookie=79d00c042c4b8bff in_seq=0 out_seq=0 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.941+0000 7f9e7cff9640 1 -- 172.21.15.77:0/3260037127 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9e8c019b10 con 0x55c37108feb0 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.941+0000 7f9e7cff9640 10 client.20523 handle_command_reply: tid=0 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.941+0000 7f9e9dcfc740 1 client.20523 shutdown 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 msgr2=0x55c37107fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009bf0 tx=0x7f9e8c009940 comp rx=0 tx=0).stop 2024-09-15T22:34:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 shutdown_connections 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c3710922a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371080370 0x55c37107d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e7cff9640 0 client.20523 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371076b40 0x55c37107d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/3260037127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3710755d0 0x55c37107fd50 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 >> 172.21.15.77:0/3260037127 conn(0x55c37103e310 msgr2=0x55c371077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.942+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 shutdown_connections 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.943+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/3260037127 wait complete. 2024-09-15T22:34:24.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.943+0000 7f9e9dcfc740 20 client.20523 trim_cache size 0 max 16384 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 Processor -- start 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 -- start start 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371078660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c3710979c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b0fa0 con 0x55c371080370 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b2ae0 con 0x55c3710755d0 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b1670 con 0x55c371076b40 2024-09-15T22:34:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:40870/0 (socket says 172.21.15.77:40870) 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1486454189 learned_addr learned my addr 172.21.15.77:0/1486454189 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c3710979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371078660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1486454189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 msgr2=0x55c371078660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371078660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1486454189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 msgr2=0x55c3710979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c3710979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.957+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1486454189 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37107b550 con 0x55c3710755d0 2024-09-15T22:34:24.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.958+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009910 tx=0x7f9e8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16681c07c0708af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.958+0000 7f9e7effd640 1 -- 172.21.15.77:0/1486454189 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02c070 con 0x55c3710755d0 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.958+0000 7f9e7effd640 1 -- 172.21.15.77:0/1486454189 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c00e5b0 con 0x55c3710755d0 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.958+0000 7f9e7effd640 1 -- 172.21.15.77:0/1486454189 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c01e040 con 0x55c3710755d0 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1486454189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 msgr2=0x55c371078120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009910 tx=0x7f9e8c011810 comp rx=0 tx=0).stop 2024-09-15T22:34:24.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1486454189 shutdown_connections 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371080370 0x55c3710979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c371076b40 0x55c371078660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1486454189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3710755d0 0x55c371078120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1486454189 >> 172.21.15.77:0/1486454189 conn(0x55c37103e310 msgr2=0x55c371070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1486454189 shutdown_connections 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1486454189 wait complete. 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.959+0000 7f9e9dcfc740 1 -- start start 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 Processor -- start 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371076b40 0x55c3710889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371080370 0x55c371085450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b2410 con 0x55c371076b40 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b1d40 con 0x55c371080370 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e98ebe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371076b40 0x55c3710889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e99ec0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47688/0 (socket says 172.21.15.77:47688) 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1828471933 learned_addr learned my addr 172.21.15.77:0/1828471933 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.960+0000 7f9e9dcfc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941aefa0 con 0x55c3710755d0 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371080370 0x55c371085450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371080370 msgr2=0x55c371085450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371080370 0x55c371085450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1828471933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371076b40 msgr2=0x55c3710889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371076b40 0x55c3710889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 -- 172.21.15.77:0/1828471933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c371085990 con 0x55c3710755d0 2024-09-15T22:34:24.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e9dcfc740 10 client.0 _ll_register_callbacks cb 0x7f9e8c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8c00e5b0 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e99ec0640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0059f0 tx=0x7f9e8c005220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e218a7bf78fb35e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e7cff9640 10 client.29185 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e7cff9640 1 -- 172.21.15.77:0/1828471933 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c01e070 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e7cff9640 1 -- 172.21.15.77:0/1828471933 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f9e8c004810 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e7cff9640 1 -- 172.21.15.77:0/1828471933 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c03c040 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e9dcfc740 10 client.29185 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.961+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55c371103540 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e7cff9640 1 -- 172.21.15.77:0/1828471933 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f9e8c024040 con 0x55c3710755d0 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 10 client.29185 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 10 client.29185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 20 client.29185 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:24.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c371105bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 4 client.29185 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c370fe8860 con 0x55c37108feb0 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c371105bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e7cff9640 10 client.29185 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.962+0000 7f9e98ebe640 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c371105bf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=544a15f646a8d9d0 server_cookie=9e7abadf1d1cd136 in_seq=0 out_seq=0 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e7cff9640 1 -- 172.21.15.77:0/1828471933 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c370fe8860 con 0x55c37108feb0 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e7cff9640 10 client.29185 handle_command_reply: tid=0 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 client.29185 shutdown 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 msgr2=0x55c371088480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0059f0 tx=0x7f9e8c005220 comp rx=0 tx=0).stop 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 shutdown_connections 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55c37108feb0 0x55c371105bf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c371080370 0x55c371085450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c371076b40 0x55c3710889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 --2- 172.21.15.77:0/1828471933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c3710755d0 0x55c371088480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 >> 172.21.15.77:0/1828471933 conn(0x55c37103e310 msgr2=0x55c37107e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e7cff9640 0 client.29185 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 shutdown_connections 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 1 -- 172.21.15.77:0/1828471933 wait complete. 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.963+0000 7f9e9dcfc740 20 client.29185 trim_cache size 0 max 16384 2024-09-15T22:34:24.970 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:34:24.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 msgr2=0x7f9e800a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 0x7f9e800a5940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55c371072ac0 tx=0x7f9e900b9040 comp rx=0 tx=0).stop 2024-09-15T22:34:24.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 msgr2=0x7f9e9419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e90004af0 tx=0x7f9e90003f30 comp rx=0 tx=0).stop 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 shutdown_connections 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f9e800a3480 0x7f9e800a5940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9410e4d0 0x7f9e9419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e94101360 0x7f9e9419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 --2- 172.21.15.77:0/2982318501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e941009b0 0x7f9e9419a5c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 >> 172.21.15.77:0/2982318501 conn(0x7f9e940fc7d0 msgr2=0x7f9e94102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 shutdown_connections 2024-09-15T22:34:24.972 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:24.964+0000 7f9e9b923640 1 -- 172.21.15.77:0/2982318501 wait complete. 2024-09-15T22:34:25.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 154654 15.50 MB/sec execute 386 sec latency 6308.673 ms 2024-09-15T22:34:26.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:26 smithi077 ceph-mon[30980]: pgmap v695: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.1 MiB/s wr, 283 op/s 2024-09-15T22:34:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:26 smithi196 ceph-mon[35109]: pgmap v695: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.1 MiB/s wr, 283 op/s 2024-09-15T22:34:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:26 smithi117 ceph-mon[35021]: pgmap v695: 129 pgs: 129 active+clean; 1.7 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.1 MiB/s wr, 283 op/s 2024-09-15T22:34:26.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 154904 15.49 MB/sec execute 387 sec latency 6377.724 ms 2024-09-15T22:34:27.165 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:34:27.506 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:27.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 155209 15.48 MB/sec execute 388 sec latency 178.236 ms 2024-09-15T22:34:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:28 smithi196 ceph-mon[35109]: pgmap v696: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.4 MiB/s wr, 402 op/s 2024-09-15T22:34:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:28 smithi117 ceph-mon[35021]: pgmap v696: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.4 MiB/s wr, 402 op/s 2024-09-15T22:34:28.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:28 smithi077 ceph-mon[30980]: pgmap v696: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.4 MiB/s wr, 402 op/s 2024-09-15T22:34:28.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 155601 15.49 MB/sec execute 389 sec latency 159.466 ms 2024-09-15T22:34:29.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 155992 15.51 MB/sec execute 390 sec latency 243.972 ms 2024-09-15T22:34:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:30 smithi196 ceph-mon[35109]: pgmap v697: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.5 MiB/s wr, 269 op/s 2024-09-15T22:34:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:30 smithi117 ceph-mon[35021]: pgmap v697: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.5 MiB/s wr, 269 op/s 2024-09-15T22:34:30.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:30 smithi077 ceph-mon[30980]: pgmap v697: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.5 MiB/s wr, 269 op/s 2024-09-15T22:34:30.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 156034 15.48 MB/sec execute 391 sec latency 1139.142 ms 2024-09-15T22:34:31.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 156185 15.46 MB/sec execute 392 sec latency 991.391 ms 2024-09-15T22:34:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:32 smithi196 ceph-mon[35109]: pgmap v698: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 333 op/s 2024-09-15T22:34:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:32 smithi117 ceph-mon[35021]: pgmap v698: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 333 op/s 2024-09-15T22:34:32.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 -- 172.21.15.77:0/3019389549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0102ef0 msgr2=0x7f38e010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 --2- 172.21.15.77:0/3019389549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0102ef0 0x7f38e010da40 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009ca0 tx=0x7f38d802a950 comp rx=0 tx=0).stop 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 -- 172.21.15.77:0/3019389549 shutdown_connections 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 --2- 172.21.15.77:0/3019389549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0102ef0 0x7f38e010da40 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 --2- 172.21.15.77:0/3019389549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 0x7f38e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 --2- 172.21.15.77:0/3019389549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0100e20 0x7f38e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.328+0000 7f38e8646640 1 -- 172.21.15.77:0/3019389549 >> 172.21.15.77:0/3019389549 conn(0x7f38e00fc7d0 msgr2=0x7f38e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 -- 172.21.15.77:0/3019389549 shutdown_connections 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 -- 172.21.15.77:0/3019389549 wait complete. 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 Processor -- start 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 -- start start 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.329+0000 7f38e8646640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 0x7f38e010ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e8646640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0102ef0 0x7f38e010f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e8646640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38e0112060 con 0x7f38e0100e20 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e8646640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38e0111ee0 con 0x7f38e0102570 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e8646640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01121e0 con 0x7f38e0102ef0 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44768/0 (socket says 172.21.15.77:44768) 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0102ef0 0x7f38e010f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 -- 172.21.15.77:0/3731564446 learned_addr learned my addr 172.21.15.77:0/3731564446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e5be1640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 0x7f38e010ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 -- 172.21.15.77:0/3731564446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0102ef0 msgr2=0x7f38e010f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0102ef0 0x7f38e010f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 -- 172.21.15.77:0/3731564446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 msgr2=0x7f38e010ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 0x7f38e010ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 -- 172.21.15.77:0/3731564446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e01a6920 con 0x7f38e0100e20 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38e63e2640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e9f0 tx=0x7f38dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.330+0000 7f38d77fe640 1 -- 172.21.15.77:0/3731564446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00cd80 con 0x7f38e0100e20 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.331+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e01a6c10 con 0x7f38e0100e20 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.331+0000 7f38d77fe640 1 -- 172.21.15.77:0/3731564446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc005500 con 0x7f38e0100e20 2024-09-15T22:34:32.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.331+0000 7f38d77fe640 1 -- 172.21.15.77:0/3731564446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc010690 con 0x7f38e0100e20 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.331+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e01a7150 con 0x7f38e0100e20 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.334+0000 7f38eaa1f740 1 Processor -- start 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 -- start start 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c8482f0 con 0x55ea0c8e25d0 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c7efcf0 con 0x55ea0c8f5b80 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c8442b0 con 0x55ea0c8e3b40 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37064/0 (socket says 172.21.15.77:37064) 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.339+0000 7f38e6be3640 1 -- 172.21.15.77:0/2146608938 learned_addr learned my addr 172.21.15.77:0/2146608938 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:32.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.340+0000 7f38e5be1640 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 -- 172.21.15.77:0/2146608938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8f5b80 msgr2=0x55ea0c8f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 -- 172.21.15.77:0/2146608938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 msgr2=0x55ea0c8e2a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8e2a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 -- 172.21.15.77:0/2146608938 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c7d43a0 con 0x55ea0c8e3b40 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.341+0000 7f38e6be3640 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38d8035000 tx=0x7f38d80093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f59887060c8acb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.342+0000 7f38d4ff9640 1 -- 172.21.15.77:0/2146608938 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8048070 con 0x55ea0c8e3b40 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.344+0000 7f38d4ff9640 1 -- 172.21.15.77:0/2146608938 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d802f930 con 0x55ea0c8e3b40 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.344+0000 7f38d77fe640 1 -- 172.21.15.77:0/3731564446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f38dc0046c0 con 0x7f38e0100e20 2024-09-15T22:34:32.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.345+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2146608938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 msgr2=0x55ea0c8e2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.345+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38d8035000 tx=0x7f38d80093e0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.345+0000 7f38d77fe640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 0x7f38cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.345+0000 7f38d77fe640 1 -- 172.21.15.77:0/3731564446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f38dc0d0530 con 0x7f38e0100e20 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.346+0000 7f38e6be3640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 0x7f38cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.350+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2146608938 shutdown_connections 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.350+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.350+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.350+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2146608938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8e2090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.350+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2146608938 >> 172.21.15.77:0/2146608938 conn(0x55ea0c8ab310 msgr2=0x55ea0c8dec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.351+0000 7f38e6be3640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 0x7f38cc0a58f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38d8030040 tx=0x7f38d8003cd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.352+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2146608938 shutdown_connections 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.352+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2146608938 wait complete. 2024-09-15T22:34:32.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.352+0000 7f38eaa1f740 1 -- start start 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.352+0000 7f38eaa1f740 1 Processor -- start 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.352+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c8e5720 con 0x55ea0c8e25d0 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c803060 con 0x55ea0c8e3b40 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c8e58a0 con 0x55ea0c8f5b80 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38eaa1f740 10 client.0 _ll_register_callbacks cb 0x55ea0c5568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44782/0 (socket says 172.21.15.77:44782) 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 -- 172.21.15.77:0/1483043999 learned_addr learned my addr 172.21.15.77:0/1483043999 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e5be1640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 -- 172.21.15.77:0/1483043999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 msgr2=0x55ea0c8f5210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:34:32.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f5210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 -- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 msgr2=0x55ea0c8efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.353+0000 7f38e63e2640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.354+0000 7f38e63e2640 1 -- 172.21.15.77:0/1483043999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c5b93c0 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.354+0000 7f38e63e2640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e9c0 tx=0x7f38dc09bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda3be027b93fa1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.354+0000 7f38c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.357+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0cdd10 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.357+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d802f930 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.357+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f38dc0b5210 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.358+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38dc0b5410 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.361+0000 7f38eaa1f740 10 client.20538 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.361+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0c8f9b50 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.364+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d802f930 con 0x55ea0c8e25d0 2024-09-15T22:34:32.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.364+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f38dc09b870 con 0x55ea0c8e25d0 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.365+0000 7f38eaa1f740 10 client.20538 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.365+0000 7f38eaa1f740 10 client.20538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.366+0000 7f38eaa1f740 20 client.20538 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.366+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c8fb510 0x55ea0c8fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.366+0000 7f38eaa1f740 4 client.20538 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.366+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0c855860 con 0x55ea0c8fb510 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.367+0000 7f38e5be1640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c8fb510 0x55ea0c8fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.372+0000 7f38e5be1640 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c8fb510 0x55ea0c8fd990 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b881bd2bb9678e4 server_cookie=ff47a0b5f1f56e99 in_seq=0 out_seq=0 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.372+0000 7f38c9ffb640 10 client.20538 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.379+0000 7f38c9ffb640 1 -- 172.21.15.77:0/1483043999 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0c855860 con 0x55ea0c8fb510 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.379+0000 7f38c9ffb640 10 client.20538 handle_command_reply: tid=0 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.379+0000 7f38eaa1f740 1 client.20538 shutdown 2024-09-15T22:34:32.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.382+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 msgr2=0x55ea0c8ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.382+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e9c0 tx=0x7f38dc09bd80 comp rx=0 tx=0).stop 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 shutdown_connections 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c8fb510 0x55ea0c8fd990 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/1483043999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ef500 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.383+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 >> 172.21.15.77:0/1483043999 conn(0x55ea0c8ab310 msgr2=0x55ea0c8e4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.384+0000 7f38c9ffb640 0 client.20538 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.386+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 shutdown_connections 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.386+0000 7f38eaa1f740 1 -- 172.21.15.77:0/1483043999 wait complete. 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.386+0000 7f38eaa1f740 20 client.20538 trim_cache size 0 max 16384 2024-09-15T22:34:32.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.411+0000 7f38eaa1f740 1 Processor -- start 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.412+0000 7f38eaa1f740 1 -- start start 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.412+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.412+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.412+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.412+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01ace60 con 0x55ea0c8e25d0 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01ae9a0 con 0x55ea0c8e3b40 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37070/0 (socket says 172.21.15.77:37070) 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e6be3640 1 -- 172.21.15.77:0/247671491 learned_addr learned my addr 172.21.15.77:0/247671491 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01ad530 con 0x55ea0c8f5b80 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e63e2640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e5be1640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e63e2640 1 -- 172.21.15.77:0/247671491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 msgr2=0x55ea0c8de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e63e2640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e63e2640 1 -- 172.21.15.77:0/247671491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 msgr2=0x55ea0c8f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.413+0000 7f38e63e2640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.414+0000 7f38e63e2640 1 -- 172.21.15.77:0/247671491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c8e23d0 con 0x55ea0c8e25d0 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.414+0000 7f38e5be1640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.414+0000 7f38e6be3640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.414+0000 7f38e63e2640 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ea7f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f38dc098da0 tx=0x7f38dc09beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a0111d0707244b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.414+0000 7f38cbfff640 1 -- 172.21.15.77:0/247671491 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc09c1f0 con 0x55ea0c8e25d0 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.415+0000 7f38cbfff640 1 -- 172.21.15.77:0/247671491 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc0a5090 con 0x55ea0c8e25d0 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.415+0000 7f38cbfff640 1 -- 172.21.15.77:0/247671491 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0cdb10 con 0x55ea0c8e25d0 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 msgr2=0x55ea0c8ea7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ea7f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f38dc098da0 tx=0x7f38dc09beb0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 shutdown_connections 2024-09-15T22:34:32.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/247671491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e25d0 0x55ea0c8ea7f0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 >> 172.21.15.77:0/247671491 conn(0x55ea0c8ab310 msgr2=0x55ea0c8ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 shutdown_connections 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.416+0000 7f38eaa1f740 1 -- 172.21.15.77:0/247671491 wait complete. 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.417+0000 7f38eaa1f740 1 -- start start 2024-09-15T22:34:32.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.417+0000 7f38eaa1f740 1 Processor -- start 2024-09-15T22:34:32.439 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:32 smithi077 ceph-mon[30980]: pgmap v698: 129 pgs: 129 active+clean; 1.7 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.3 MiB/s wr, 333 op/s 2024-09-15T22:34:32.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01ae2d0 con 0x55ea0c8e3b40 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01adc00 con 0x55ea0c8e25d0 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38e01aaee0 con 0x55ea0c8f5b80 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38e6be3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38eaa1f740 10 client.0 _ll_register_callbacks cb 0x7f38dc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38e63e2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58678/0 (socket says 172.21.15.77:58678) 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38e63e2640 1 -- 172.21.15.77:0/2672261283 learned_addr learned my addr 172.21.15.77:0/2672261283 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.418+0000 7f38e5be1640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 -- 172.21.15.77:0/2672261283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 msgr2=0x55ea0c8ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 -- 172.21.15.77:0/2672261283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e3b40 msgr2=0x55ea0c8fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:32.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 -- 172.21.15.77:0/2672261283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c8ffc30 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38e63e2640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0049d0 tx=0x7f38dc09b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b5d35ee9ba7358 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0ac070 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f38dc0a4c00 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.419+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38dc0a5090 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.420+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38dc0ac070 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.420+0000 7f38eaa1f740 10 client.29252 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.420+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ea0c989690 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.420+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc0a5090 con 0x55ea0c8e25d0 2024-09-15T22:34:32.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f38dc0af030 con 0x55ea0c8e25d0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 10 client.29252 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 10 client.29252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 20 client.29252 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c989ac0 0x55ea0c98beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 4 client.29252 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea0c8e5930 con 0x55ea0c989ac0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.421+0000 7f38e5be1640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c989ac0 0x55ea0c98beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.422+0000 7f38e5be1640 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c989ac0 0x55ea0c98beb0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8127831c8db7085 server_cookie=3aa1c29cb95957d3 in_seq=0 out_seq=0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.422+0000 7f38c9ffb640 10 client.29252 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.425+0000 7f38c9ffb640 1 -- 172.21.15.77:0/2672261283 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea0c8e5930 con 0x55ea0c989ac0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.425+0000 7f38c9ffb640 10 client.29252 handle_command_reply: tid=0 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.425+0000 7f38eaa1f740 1 client.29252 shutdown 2024-09-15T22:34:32.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 msgr2=0x55ea0c903ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0049d0 tx=0x7f38dc09b980 comp rx=0 tx=0).stop 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 shutdown_connections 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ea0c989ac0 0x55ea0c98beb0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea0c8f5b80 0x55ea0c8ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ea0c8e3b40 0x55ea0c8fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.444 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 --2- 172.21.15.77:0/2672261283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea0c8e25d0 0x55ea0c903ae0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 >> 172.21.15.77:0/2672261283 conn(0x55ea0c8ab310 msgr2=0x55ea0c8ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.427+0000 7f38c9ffb640 0 client.29252 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.428+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 shutdown_connections 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.428+0000 7f38eaa1f740 1 -- 172.21.15.77:0/2672261283 wait complete. 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.428+0000 7f38eaa1f740 20 client.29252 trim_cache size 0 max 16384 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 msgr2=0x7f38cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 0x7f38cc0a58f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38d8030040 tx=0x7f38d8003cd0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 msgr2=0x7f38e010e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e9f0 tx=0x7f38dc00edf0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 shutdown_connections 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f38cc0a3430 0x7f38cc0a58f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38e0102ef0 0x7f38e010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e0102570 0x7f38e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 --2- 172.21.15.77:0/3731564446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38e0100e20 0x7f38e010e700 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 >> 172.21.15.77:0/3731564446 conn(0x7f38e00fc7d0 msgr2=0x7f38e0106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 shutdown_connections 2024-09-15T22:34:32.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:31.429+0000 7f38e8646640 1 -- 172.21.15.77:0/3731564446 wait complete. 2024-09-15T22:34:32.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 156393 15.45 MB/sec execute 393 sec latency 195.444 ms 2024-09-15T22:34:33.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 156668 15.46 MB/sec execute 394 sec latency 352.661 ms 2024-09-15T22:34:34.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:34 smithi077 ceph-mon[30980]: pgmap v699: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 447 op/s 2024-09-15T22:34:34.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:34 smithi196 ceph-mon[35109]: pgmap v699: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 447 op/s 2024-09-15T22:34:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:34 smithi117 ceph-mon[35021]: pgmap v699: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 447 op/s 2024-09-15T22:34:34.617 INFO:tasks.workunit.client.0.smithi077.stdout: 10 156865 15.44 MB/sec execute 395 sec latency 269.321 ms 2024-09-15T22:34:35.504 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:34:35.505 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:34:35.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 157052 15.43 MB/sec execute 396 sec latency 421.885 ms 2024-09-15T22:34:35.785 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:36.180 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:36 smithi077 ceph-mon[30980]: pgmap v700: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 369 op/s 2024-09-15T22:34:36.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 -- 172.21.15.77:0/1188211364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe390102d30 msgr2=0x7fe390103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1188211364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe390102d30 0x7fe390103110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009930 tx=0x7fe38c02a420 comp rx=0 tx=0).stop 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 -- 172.21.15.77:0/1188211364 shutdown_connections 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1188211364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901082a0 0x7fe39010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1188211364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3901036e0 0x7fe390107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1188211364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe390102d30 0x7fe390103110 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.281+0000 7fe3977b0640 1 -- 172.21.15.77:0/1188211364 >> 172.21.15.77:0/1188211364 conn(0x7fe3900fc7d0 msgr2=0x7fe3900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.282+0000 7fe3977b0640 1 -- 172.21.15.77:0/1188211364 shutdown_connections 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.282+0000 7fe3977b0640 1 -- 172.21.15.77:0/1188211364 wait complete. 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.282+0000 7fe3977b0640 1 Processor -- start 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.282+0000 7fe3977b0640 1 -- start start 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 0x7fe3901a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901036e0 0x7fe3901a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.290 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901120e0 con 0x7fe3901082a0 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe390111f60 con 0x7fe390102d30 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe3977b0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe390112260 con 0x7fe3901036e0 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 0x7fe3901a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44814/0 (socket says 172.21.15.77:44814) 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe395d4d640 1 -- 172.21.15.77:0/1568891436 learned_addr learned my addr 172.21.15.77:0/1568891436 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe394d4b640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901036e0 0x7fe3901a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 -- 172.21.15.77:0/1568891436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901036e0 msgr2=0x7fe3901a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901036e0 0x7fe3901a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 -- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 msgr2=0x7fe39010ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe39554c640 1 -- 172.21.15.77:0/1568891436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39010e4e0 con 0x7fe390102d30 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.283+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.284+0000 7fe39554c640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 0x7fe3901a4830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a940 tx=0x7fe38c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.284+0000 7fe3867fc640 1 -- 172.21.15.77:0/1568891436 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c03f070 con 0x7fe390102d30 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.284+0000 7fe3867fc640 1 -- 172.21.15.77:0/1568891436 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38c031c90 con 0x7fe390102d30 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.285+0000 7fe3867fc640 1 -- 172.21.15.77:0/1568891436 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c035030 con 0x7fe390102d30 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.285+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39010e770 con 0x7fe390102d30 2024-09-15T22:34:36.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.285+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39010eb90 con 0x7fe390102d30 2024-09-15T22:34:36.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.289+0000 7fe3867fc640 1 -- 172.21.15.77:0/1568891436 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe38c042070 con 0x7fe390102d30 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.290+0000 7fe399b89740 1 Processor -- start 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.290+0000 7fe399b89740 1 -- start start 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.290+0000 7fe3867fc640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 0x7fe37c0a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.291+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 0x7fe37c0a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.291+0000 7fe3867fc640 1 -- 172.21.15.77:0/1568891436 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe38c0f0b30 con 0x7fe390102d30 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 0x560184d05f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d064a0 0x560184d10820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 0x7fe37c0a57d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe3880049d0 tx=0x7fe388009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560184c6b280 con 0x560184d05b80 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560184c12b10 con 0x560184d10d60 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe399b89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560184c67240 con 0x560184d064a0 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58706/0 (socket says 172.21.15.77:58706) 2024-09-15T22:34:36.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.292+0000 7fe395d4d640 1 -- 172.21.15.77:0/1745344415 learned_addr learned my addr 172.21.15.77:0/1745344415 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe39554c640 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 0x560184d05f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 -- 172.21.15.77:0/1745344415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d064a0 msgr2=0x560184d10820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d064a0 0x560184d10820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 -- 172.21.15.77:0/1745344415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 msgr2=0x560184d05f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 0x560184d05f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 -- 172.21.15.77:0/1745344415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560184c1f1f0 con 0x560184d10d60 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe39554c640 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 0x560184d05f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.293+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe38800fe80 tx=0x7fe388014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b0f4ad9c5eef5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.294+0000 7fe37b7fe640 1 -- 172.21.15.77:0/1745344415 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388029070 con 0x560184d10d60 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.294+0000 7fe37b7fe640 1 -- 172.21.15.77:0/1745344415 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38801ce10 con 0x560184d10d60 2024-09-15T22:34:36.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.294+0000 7fe37b7fe640 1 -- 172.21.15.77:0/1745344415 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388024650 con 0x560184d10d60 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.295+0000 7fe399b89740 1 -- 172.21.15.77:0/1745344415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 msgr2=0x560184d13150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.295+0000 7fe399b89740 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe38800fe80 tx=0x7fe388014000 comp rx=0 tx=0).stop 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 -- 172.21.15.77:0/1745344415 shutdown_connections 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d10d60 0x560184d13150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d064a0 0x560184d10820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 --2- 172.21.15.77:0/1745344415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d05b80 0x560184d05f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 -- 172.21.15.77:0/1745344415 >> 172.21.15.77:0/1745344415 conn(0x560184d02110 msgr2=0x560184d024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 -- 172.21.15.77:0/1745344415 shutdown_connections 2024-09-15T22:34:36.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.296+0000 7fe399b89740 1 -- 172.21.15.77:0/1745344415 wait complete. 2024-09-15T22:34:36.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 -- start start 2024-09-15T22:34:36.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 Processor -- start 2024-09-15T22:34:36.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d0e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560184d13930 con 0x560184d10d60 2024-09-15T22:34:36.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560184c25f10 con 0x560184d064a0 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560184d13ab0 con 0x560184d05b80 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.297+0000 7fe399b89740 10 client.0 _ll_register_callbacks cb 0x560184beea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44840/0 (socket says 172.21.15.77:44840) 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 -- 172.21.15.77:0/1510318704 learned_addr learned my addr 172.21.15.77:0/1510318704 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 -- 172.21.15.77:0/1510318704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 msgr2=0x560184d0dc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe39554c640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 -- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 msgr2=0x560184d0e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d0e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe395d4d640 1 -- 172.21.15.77:0/1510318704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601849dc6a0 con 0x560184d10d60 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.298+0000 7fe39554c640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:34:36.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fe38801cc60 tx=0x7fe38801ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68989d5954a3ba42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe378ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388029070 con 0x560184d10d60 2024-09-15T22:34:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38801ce10 con 0x560184d10d60 2024-09-15T22:34:36.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe388009040 con 0x560184d10d60 2024-09-15T22:34:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe388038050 con 0x560184d10d60 2024-09-15T22:34:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe399b89740 10 client.20556 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.299+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560184d1f510 con 0x560184d10d60 2024-09-15T22:34:36.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38801ce10 con 0x560184d10d60 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe388018040 con 0x560184d10d60 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 10 client.20556 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 10 client.20556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 20 client.20556 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d20860 0x560184d22c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 4 client.20556 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560184c787f0 con 0x560184d20860 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.300+0000 7fe39554c640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d20860 0x560184d22c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.301+0000 7fe378ff9640 10 client.20556 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:36.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.301+0000 7fe39554c640 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d20860 0x560184d22c50 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba65c123f37970e server_cookie=ed1f944eb408cf8e in_seq=0 out_seq=0 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe378ff9640 1 -- 172.21.15.77:0/1510318704 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560184c787f0 con 0x560184d20860 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe378ff9640 10 client.20556 handle_command_reply: tid=0 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 client.20556 shutdown 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 msgr2=0x560184d0e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fe38801cc60 tx=0x7fe38801ab00 comp rx=0 tx=0).stop 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 shutdown_connections 2024-09-15T22:34:36.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d20860 0x560184d22c50 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d0e700 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d0e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 --2- 172.21.15.77:0/1510318704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 >> 172.21.15.77:0/1510318704 conn(0x560184d02110 msgr2=0x560184d07e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.303+0000 7fe378ff9640 0 client.20556 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.304+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 shutdown_connections 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.304+0000 7fe399b89740 1 -- 172.21.15.77:0/1510318704 wait complete. 2024-09-15T22:34:36.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.304+0000 7fe399b89740 20 client.20556 trim_cache size 0 max 16384 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 Processor -- start 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 -- start start 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d28310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b53d0 con 0x560184d10d60 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b6f10 con 0x560184d064a0 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.327+0000 7fe399b89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b5aa0 con 0x560184d05b80 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe394d4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d28310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe39554c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe394d4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d28310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58712/0 (socket says 172.21.15.77:58712) 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe394d4b640 1 -- 172.21.15.77:0/1473794532 learned_addr learned my addr 172.21.15.77:0/1473794532 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 -- 172.21.15.77:0/1473794532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 msgr2=0x560184d0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 -- 172.21.15.77:0/1473794532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 msgr2=0x560184d28310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d28310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 -- 172.21.15.77:0/1473794532 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560184d062d0 con 0x560184d10d60 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe39554c640 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d28850 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe3880243e0 tx=0x7fe388005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b646d6d1706a1538 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.328+0000 7fe37affd640 1 -- 172.21.15.77:0/1473794532 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388020070 con 0x560184d10d60 2024-09-15T22:34:36.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe37affd640 1 -- 172.21.15.77:0/1473794532 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388004c50 con 0x560184d10d60 2024-09-15T22:34:36.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe37affd640 1 -- 172.21.15.77:0/1473794532 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388029040 con 0x560184d10d60 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 -- 172.21.15.77:0/1473794532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 msgr2=0x560184d28850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d28850 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fe3880243e0 tx=0x7fe388005430 comp rx=0 tx=0).stop 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 -- 172.21.15.77:0/1473794532 shutdown_connections 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d28850 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d28310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 --2- 172.21.15.77:0/1473794532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.329+0000 7fe399b89740 1 -- 172.21.15.77:0/1473794532 >> 172.21.15.77:0/1473794532 conn(0x560184d02110 msgr2=0x560184d0f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.330+0000 7fe399b89740 1 -- 172.21.15.77:0/1473794532 shutdown_connections 2024-09-15T22:34:36.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.330+0000 7fe399b89740 1 -- 172.21.15.77:0/1473794532 wait complete. 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.330+0000 7fe399b89740 1 -- start start 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.331+0000 7fe399b89740 1 Processor -- start 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.331+0000 7fe399b89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.331+0000 7fe399b89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.331+0000 7fe399b89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe399b89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b6840 con 0x560184d10d60 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe399b89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b6170 con 0x560184d064a0 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe399b89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901b33d0 con 0x560184d05b80 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe39554c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe39554c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37132/0 (socket says 172.21.15.77:37132) 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe39554c640 1 -- 172.21.15.77:0/1861993690 learned_addr learned my addr 172.21.15.77:0/1861993690 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe399b89740 10 client.0 _ll_register_callbacks cb 0x7fe38801a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:36.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe395d4d640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.332+0000 7fe394d4b640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 -- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 msgr2=0x560184d84bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d84bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 -- 172.21.15.77:0/1861993690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 msgr2=0x560184d152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 -- 172.21.15.77:0/1861993690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560184d15b20 con 0x560184d05b80 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe39554c640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0edfa0 tx=0x7fe38c031f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d465596943ca18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:36.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe378ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0c0e30 con 0x560184d05b80 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe388004c50 con 0x560184d05b80 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.333+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fe38c0ee170 con 0x560184d05b80 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.334+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe38c0d9070 con 0x560184d05b80 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.334+0000 7fe399b89740 10 client.29212 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:36.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.334+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560184d8bf10 con 0x560184d05b80 2024-09-15T22:34:36.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.334+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388004c50 con 0x560184d05b80 2024-09-15T22:34:36.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe38c04c060 con 0x560184d05b80 2024-09-15T22:34:36.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 10 client.29212 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:36.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 10 client.29212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 20 client.29212 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d8c6d0 0x560184d8eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 4 client.29212 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560184c72d00 con 0x560184d8c6d0 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.335+0000 7fe394d4b640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d8c6d0 0x560184d8eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.337+0000 7fe394d4b640 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d8c6d0 0x560184d8eac0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f083d783012d19 server_cookie=3fb785ad5475314f in_seq=0 out_seq=0 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.338+0000 7fe378ff9640 10 client.29212 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:36.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.339+0000 7fe378ff9640 1 -- 172.21.15.77:0/1861993690 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560184c72d00 con 0x560184d8c6d0 2024-09-15T22:34:36.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.339+0000 7fe378ff9640 10 client.29212 handle_command_reply: tid=0 2024-09-15T22:34:36.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.339+0000 7fe399b89740 1 client.29212 shutdown 2024-09-15T22:34:36.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.340+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 msgr2=0x560184d84670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.340+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0edfa0 tx=0x7fe38c031f00 comp rx=0 tx=0).stop 2024-09-15T22:34:36.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 shutdown_connections 2024-09-15T22:34:36.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x560184d8c6d0 0x560184d8eac0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560184d10d60 0x560184d152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560184d064a0 0x560184d84bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 --2- 172.21.15.77:0/1861993690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560184d05b80 0x560184d84670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 >> 172.21.15.77:0/1861993690 conn(0x560184d02110 msgr2=0x560184d07df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe378ff9640 0 client.29212 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 shutdown_connections 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 1 -- 172.21.15.77:0/1861993690 wait complete. 2024-09-15T22:34:36.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.341+0000 7fe399b89740 20 client.29212 trim_cache size 0 max 16384 2024-09-15T22:34:36.344 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:34:36.344 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:34:36.344 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "3241fdb2-3502-4c3d-a6c0-eb7484c06bd3", 2024-09-15T22:34:36.344 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 msgr2=0x7fe37c0a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 0x7fe37c0a57d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe3880049d0 tx=0x7fe388009500 comp rx=0 tx=0).stop 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 msgr2=0x7fe3901a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 0x7fe3901a4830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a940 tx=0x7fe38c003b20 comp rx=0 tx=0).stop 2024-09-15T22:34:36.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 shutdown_connections 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe37c0a3310 0x7fe37c0a57d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3901082a0 0x7fe39010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3901036e0 0x7fe3901a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 --2- 172.21.15.77:0/1568891436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe390102d30 0x7fe3901a4830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 >> 172.21.15.77:0/1568891436 conn(0x7fe3900fc7d0 msgr2=0x7fe390101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 shutdown_connections 2024-09-15T22:34:36.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:36.342+0000 7fe3977b0640 1 -- 172.21.15.77:0/1568891436 wait complete. 2024-09-15T22:34:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:36 smithi196 ceph-mon[35109]: pgmap v700: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 369 op/s 2024-09-15T22:34:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:36 smithi117 ceph-mon[35021]: pgmap v700: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 369 op/s 2024-09-15T22:34:36.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 157229 15.40 MB/sec execute 397 sec latency 803.984 ms 2024-09-15T22:34:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:34:37.189 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.190 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:37 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.191 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:34:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:34:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:37.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:37.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:37.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:37 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.532 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:34:37.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:37 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:37.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 157712 15.43 MB/sec execute 398 sec latency 189.376 ms 2024-09-15T22:34:38.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.067+0000 7f1e591c4640 1 -- 172.21.15.77:0/2592891731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e5411b140 msgr2=0x7f1e5411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.067+0000 7f1e591c4640 1 --2- 172.21.15.77:0/2592891731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e5411b140 0x7f1e5411d530 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009ca0 tx=0x7f1e4c02a8b0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 -- 172.21.15.77:0/2592891731 shutdown_connections 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 --2- 172.21.15.77:0/2592891731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e5411b140 0x7f1e5411d530 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 --2- 172.21.15.77:0/2592891731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109830 0x7f1e54109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 --2- 172.21.15.77:0/2592891731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e54109450 0x7f1e54110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 -- 172.21.15.77:0/2592891731 >> 172.21.15.77:0/2592891731 conn(0x7f1e5406e140 msgr2=0x7f1e5406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 -- 172.21.15.77:0/2592891731 shutdown_connections 2024-09-15T22:34:38.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.068+0000 7f1e591c4640 1 -- 172.21.15.77:0/2592891731 wait complete. 2024-09-15T22:34:38.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 Processor -- start 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 -- start start 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 0x7f1e541117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e5411b140 0x7f1e54111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54120da0 con 0x7f1e54109830 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54120c20 con 0x7f1e5411b140 2024-09-15T22:34:38.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.069+0000 7f1e591c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54120f20 con 0x7f1e54109450 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37140/0 (socket says 172.21.15.77:37140) 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 -- 172.21.15.77:0/1566390184 learned_addr learned my addr 172.21.15.77:0/1566390184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e5259c640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 0x7f1e541117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e5359e640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e5411b140 0x7f1e54111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 -- 172.21.15.77:0/1566390184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e5411b140 msgr2=0x7f1e54111ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e5411b140 0x7f1e54111ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 -- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 msgr2=0x7f1e541117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 0x7f1e541117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e52d9d640 1 -- 172.21.15.77:0/1566390184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e541122a0 con 0x7f1e54109450 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.070+0000 7f1e5259c640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 0x7f1e541117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:38.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.071+0000 7f1e52d9d640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d950 tx=0x7f1e4800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.071+0000 7f1e3bfff640 1 -- 172.21.15.77:0/1566390184 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48017070 con 0x7f1e54109450 2024-09-15T22:34:38.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.071+0000 7f1e3bfff640 1 -- 172.21.15.77:0/1566390184 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48009c30 con 0x7f1e54109450 2024-09-15T22:34:38.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.071+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e54112590 con 0x7f1e54109450 2024-09-15T22:34:38.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.071+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e54116560 con 0x7f1e54109450 2024-09-15T22:34:38.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.072+0000 7f1e3bfff640 1 -- 172.21.15.77:0/1566390184 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48017070 con 0x7f1e54109450 2024-09-15T22:34:38.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.075+0000 7f1e5b59d740 1 Processor -- start 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.075+0000 7f1e5b59d740 1 -- start start 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.075+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.075+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b16dc0 0x555ef2b21140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b23a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ef2a7c2e0 con 0x555ef2b21680 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ef2a23ce0 con 0x555ef2b16dc0 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ef2a782a0 con 0x555ef2b164a0 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37156/0 (socket says 172.21.15.77:37156) 2024-09-15T22:34:38.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 -- 172.21.15.77:0/3705550246 learned_addr learned my addr 172.21.15.77:0/3705550246 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e3bfff640 1 -- 172.21.15.77:0/1566390184 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1e48010460 con 0x7f1e54109450 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5259c640 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b23a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 -- 172.21.15.77:0/3705550246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b16dc0 msgr2=0x555ef2b21140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b16dc0 0x555ef2b21140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 -- 172.21.15.77:0/3705550246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 msgr2=0x555ef2b23a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b23a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5359e640 1 -- 172.21.15.77:0/3705550246 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef2a08440 con 0x555ef2b164a0 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.076+0000 7f1e5259c640 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b23a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:34:38.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009ca0 tx=0x7f1e4c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea95b246e983a049 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e3bfff640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 0x7f1e3c0a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e397fa640 1 -- 172.21.15.77:0/3705550246 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c0303a0 con 0x555ef2b164a0 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e397fa640 1 -- 172.21.15.77:0/3705550246 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c036030 con 0x555ef2b164a0 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e397fa640 1 -- 172.21.15.77:0/3705550246 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c031880 con 0x555ef2b164a0 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.077+0000 7f1e3bfff640 1 -- 172.21.15.77:0/1566390184 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f1e48009a30 con 0x7f1e54109450 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e52d9d640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 0x7f1e3c0a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3705550246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 msgr2=0x555ef2b16880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009ca0 tx=0x7f1e4c009a20 comp rx=0 tx=0).stop 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e52d9d640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 0x7f1e3c0a5cb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x555ef2b13890 tx=0x7f1e4800b510 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3705550246 shutdown_connections 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b23a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b16dc0 0x555ef2b21140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3705550246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b16880 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.078+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3705550246 >> 172.21.15.77:0/3705550246 conn(0x555ef2b12a30 msgr2=0x555ef2b12e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3705550246 shutdown_connections 2024-09-15T22:34:38.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3705550246 wait complete. 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 -- start start 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 Processor -- start 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b164a0 0x555ef2b20fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.079+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b1e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ef2b24250 con 0x555ef2b21680 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ef2a37050 con 0x555ef2b164a0 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ef2b243d0 con 0x555ef2b16dc0 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5b59d740 10 client.0 _ll_register_callbacks cb 0x555ef2a77160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:38.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37168/0 (socket says 172.21.15.77:37168) 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b164a0 0x555ef2b20fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 -- 172.21.15.77:0/3568893415 learned_addr learned my addr 172.21.15.77:0/3568893415 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b1e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 -- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b164a0 msgr2=0x555ef2b20fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b164a0 0x555ef2b20fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 -- 172.21.15.77:0/3568893415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 msgr2=0x555ef2b1e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b1e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 -- 172.21.15.77:0/3568893415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef27ed3c0 con 0x555ef2b16dc0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e5259c640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e44012a10 tx=0x7f1e44012ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55867244855838a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.080+0000 7f1e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400fa60 con 0x555ef2b16dc0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f1e4400fc00 con 0x555ef2b16dc0 2024-09-15T22:34:38.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4c036030 con 0x555ef2b16dc0 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1e4400fe00 con 0x555ef2b16dc0 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 10 client.29224 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555ef2b2fc60 con 0x555ef2b16dc0 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c036030 con 0x555ef2b16dc0 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1e4401c5e0 con 0x555ef2b16dc0 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 10 client.29224 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 10 client.29224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:38.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 20 client.29224 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b311c0 0x555ef2b335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 4 client.29224 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.081+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef2a89850 con 0x555ef2b311c0 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.082+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b311c0 0x555ef2b335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.082+0000 7f1e5359e640 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b311c0 0x555ef2b335e0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147a6e5411d936f0 server_cookie=44219970e34326 in_seq=0 out_seq=0 2024-09-15T22:34:38.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.082+0000 7f1e36ff5640 10 client.29224 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:38.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e36ff5640 1 -- 172.21.15.77:0/3568893415 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ef2a89850 con 0x555ef2b311c0 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e36ff5640 10 client.29224 handle_command_reply: tid=0 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 client.29224 shutdown 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 msgr2=0x555ef2b1e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e44012a10 tx=0x7f1e44012ed0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 shutdown_connections 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b311c0 0x555ef2b335e0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b21680 0x555ef2b1e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b1e270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/3568893415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b164a0 0x555ef2b20fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e36ff5640 0 client.29224 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 >> 172.21.15.77:0/3568893415 conn(0x555ef2b12a30 msgr2=0x555ef2b18710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 shutdown_connections 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 1 -- 172.21.15.77:0/3568893415 wait complete. 2024-09-15T22:34:38.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.086+0000 7f1e5b59d740 20 client.29224 trim_cache size 0 max 16384 2024-09-15T22:34:38.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 Processor -- start 2024-09-15T22:34:38.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 -- start start 2024-09-15T22:34:38.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b2d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c65a0 con 0x555ef2b164a0 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c80e0 con 0x555ef2b12a30 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c6c70 con 0x555ef2b16dc0 2024-09-15T22:34:38.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44904/0 (socket says 172.21.15.77:44904) 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 -- 172.21.15.77:0/878640557 learned_addr learned my addr 172.21.15.77:0/878640557 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5359e640 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b2d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5359e640 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 -- 172.21.15.77:0/878640557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 msgr2=0x555ef2b38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 -- 172.21.15.77:0/878640557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 msgr2=0x555ef2b2d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b2d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.112+0000 7f1e5259c640 1 -- 172.21.15.77:0/878640557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef2b33c40 con 0x555ef2b164a0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.113+0000 7f1e5259c640 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009730 tx=0x7f1e44009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eec4956d9d726b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.113+0000 7f1e38ff9640 1 -- 172.21.15.77:0/878640557 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400fb20 con 0x555ef2b164a0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.113+0000 7f1e38ff9640 1 -- 172.21.15.77:0/878640557 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44005030 con 0x555ef2b164a0 2024-09-15T22:34:38.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.113+0000 7f1e38ff9640 1 -- 172.21.15.77:0/878640557 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004880 con 0x555ef2b164a0 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- 172.21.15.77:0/878640557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 msgr2=0x555ef2b197f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009730 tx=0x7f1e44009240 comp rx=0 tx=0).stop 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- 172.21.15.77:0/878640557 shutdown_connections 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b16dc0 0x555ef2b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b164a0 0x555ef2b197f0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/878640557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b2d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- 172.21.15.77:0/878640557 >> 172.21.15.77:0/878640557 conn(0x555ef2b1e910 msgr2=0x555ef2b190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- 172.21.15.77:0/878640557 shutdown_connections 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- 172.21.15.77:0/878640557 wait complete. 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 -- start start 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.114+0000 7f1e5b59d740 1 Processor -- start 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b25900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 0x555ef2b25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c7a10 con 0x555ef2b16dc0 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c7340 con 0x555ef2b12a30 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541c45a0 con 0x555ef2b164a0 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5b59d740 10 client.0 _ll_register_callbacks cb 0x7f1e44009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b25900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58772/0 (socket says 172.21.15.77:58772) 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5359e640 1 -- 172.21.15.77:0/1785694130 learned_addr learned my addr 172.21.15.77:0/1785694130 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5359e640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 0x555ef2b25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 -- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 msgr2=0x555ef2b253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 -- 172.21.15.77:0/1785694130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 msgr2=0x555ef2b25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 0x555ef2b25e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 -- 172.21.15.77:0/1785694130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef2b13ae0 con 0x555ef2b164a0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5359e640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 0x555ef2b25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e5259c640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b25900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x555ef2b1c9a0 tx=0x7f1e4400fd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e23cca7981326dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.115+0000 7f1e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44037060 con 0x555ef2b164a0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e44005030 con 0x555ef2b164a0 2024-09-15T22:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f1e44018040 con 0x555ef2b164a0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1e44005c50 con 0x555ef2b164a0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 10 client.29230 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555ef2b9bfb0 con 0x555ef2b164a0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44005030 con 0x555ef2b164a0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1e44003bf0 con 0x555ef2b164a0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 10 client.29230 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 10 client.29230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 20 client.29230 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b21680 0x555ef2b9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 4 client.29230 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ef2a83d60 con 0x555ef2b21680 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.116+0000 7f1e5359e640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b21680 0x555ef2b9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:38.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.117+0000 7f1e36ff5640 10 client.29230 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:38.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.117+0000 7f1e5359e640 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b21680 0x555ef2b9e490 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da08e3e79334c78e server_cookie=9541980fb5c88945 in_seq=0 out_seq=0 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e36ff5640 1 -- 172.21.15.77:0/1785694130 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555ef2a83d60 con 0x555ef2b21680 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e36ff5640 10 client.29230 handle_command_reply: tid=0 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 client.29230 shutdown 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 msgr2=0x555ef2b25900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b25900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x555ef2b1c9a0 tx=0x7f1e4400fd50 comp rx=0 tx=0).stop 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 shutdown_connections 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x555ef2b21680 0x555ef2b9e490 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ef2b16dc0 0x555ef2b25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ef2b164a0 0x555ef2b25900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 --2- 172.21.15.77:0/1785694130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ef2b12a30 0x555ef2b253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 >> 172.21.15.77:0/1785694130 conn(0x555ef2b1e910 msgr2=0x555ef2b202f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e36ff5640 0 client.29230 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.120+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 shutdown_connections 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.121+0000 7f1e5b59d740 1 -- 172.21.15.77:0/1785694130 wait complete. 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.121+0000 7f1e5b59d740 20 client.29230 trim_cache size 0 max 16384 2024-09-15T22:34:38.125 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "3241fdb2-3502-4c3d-a6c0-eb7484c06bd3": { 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "3241fdb2-3502-4c3d-a6c0-eb7484c06bd3", 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:38.126 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 msgr2=0x7f1e3c0a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 0x7f1e3c0a5cb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x555ef2b13890 tx=0x7f1e4800b510 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 msgr2=0x7f1e54118710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d950 tx=0x7f1e4800de10 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 shutdown_connections 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1e3c0a37f0 0x7f1e3c0a5cb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e5411b140 0x7f1e54111ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e54109830 0x7f1e541117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 --2- 172.21.15.77:0/1566390184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e54109450 0x7f1e54118710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 >> 172.21.15.77:0/1566390184 conn(0x7f1e5406e140 msgr2=0x7f1e5411bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 shutdown_connections 2024-09-15T22:34:38.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:38.122+0000 7f1e591c4640 1 -- 172.21.15.77:0/1566390184 wait complete. 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:34:38.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:38 smithi077 ceph-mon[30980]: pgmap v701: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 591 op/s 2024-09-15T22:34:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:38 smithi117 ceph-mon[35021]: pgmap v701: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 591 op/s 2024-09-15T22:34:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:34:38.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:38 smithi196 ceph-mon[35109]: pgmap v701: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 591 op/s 2024-09-15T22:34:38.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 158119 15.45 MB/sec execute 399 sec latency 168.674 ms 2024-09-15T22:34:38.868 INFO:tasks.cephfs.filesystem:scrub status for tag:3241fdb2-3502-4c3d-a6c0-eb7484c06bd3 - {'path': '/', 'tag': '3241fdb2-3502-4c3d-a6c0-eb7484c06bd3', 'options': 'recursive,force'} 2024-09-15T22:34:39.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 158486 15.46 MB/sec execute 400 sec latency 174.755 ms 2024-09-15T22:34:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:40 smithi117 ceph-mon[35021]: pgmap v702: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 441 op/s 2024-09-15T22:34:40.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:40 smithi077 ceph-mon[30980]: pgmap v702: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 441 op/s 2024-09-15T22:34:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:40 smithi196 ceph-mon[35109]: pgmap v702: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 13 MiB/s wr, 441 op/s 2024-09-15T22:34:40.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 158896 15.48 MB/sec execute 401 sec latency 194.274 ms 2024-09-15T22:34:41.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 159273 15.48 MB/sec execute 402 sec latency 164.275 ms 2024-09-15T22:34:42.472 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.4 2024-09-15T22:34:42.472 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:34:42.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:42 smithi077 ceph-mon[30980]: pgmap v703: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 475 op/s 2024-09-15T22:34:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:42 smithi196 ceph-mon[35109]: pgmap v703: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 475 op/s 2024-09-15T22:34:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:42 smithi117 ceph-mon[35021]: pgmap v703: 129 pgs: 129 active+clean; 1.7 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 475 op/s 2024-09-15T22:34:42.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 159702 15.50 MB/sec execute 403 sec latency 150.712 ms 2024-09-15T22:34:42.771 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:43.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 -- 172.21.15.77:0/2398031147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 msgr2=0x7fc4381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:43.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 --2- 172.21.15.77:0/2398031147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc4381067e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc428009860 tx=0x7fc42802a340 comp rx=0 tx=0).stop 2024-09-15T22:34:43.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 -- 172.21.15.77:0/2398031147 shutdown_connections 2024-09-15T22:34:43.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 --2- 172.21.15.77:0/2398031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438106f10 0x7fc43810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 --2- 172.21.15.77:0/2398031147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc4381067e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 --2- 172.21.15.77:0/2398031147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc438101de0 0x7fc4381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 -- 172.21.15.77:0/2398031147 >> 172.21.15.77:0/2398031147 conn(0x7fc4380fd8e0 msgr2=0x7fc4380ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:43.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 -- 172.21.15.77:0/2398031147 shutdown_connections 2024-09-15T22:34:43.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.308+0000 7fc440688640 1 -- 172.21.15.77:0/2398031147 wait complete. 2024-09-15T22:34:43.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 Processor -- start 2024-09-15T22:34:43.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 -- start start 2024-09-15T22:34:43.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:43.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc438074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:43.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc438106f10 0x7fc438074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc438111f70 con 0x7fc438101de0 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc438111df0 con 0x7fc438102790 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.309+0000 7fc440688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381120f0 con 0x7fc438106f10 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52246/0 (socket says 172.21.15.77:52246) 2024-09-15T22:34:43.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43dc23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc438074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:43.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 -- 172.21.15.77:0/372422702 learned_addr learned my addr 172.21.15.77:0/372422702 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:43.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 -- 172.21.15.77:0/372422702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc438106f10 msgr2=0x7fc438074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:34:43.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc438106f10 0x7fc438074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 -- 172.21.15.77:0/372422702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 msgr2=0x7fc438074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:43.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc438074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 -- 172.21.15.77:0/372422702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4381a6710 con 0x7fc438101de0 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43dc23640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc438074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc43e424640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e9f0 tx=0x7fc43400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.310+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43400cd80 con 0x7fc438101de0 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.311+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4381a6a00 con 0x7fc438101de0 2024-09-15T22:34:43.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.311+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc434010910 con 0x7fc438101de0 2024-09-15T22:34:43.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.311+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4381a6f60 con 0x7fc438101de0 2024-09-15T22:34:43.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.311+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434018960 con 0x7fc438101de0 2024-09-15T22:34:43.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.312+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c005290 con 0x7fc438101de0 2024-09-15T22:34:43.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.314+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc434010430 con 0x7fc438101de0 2024-09-15T22:34:43.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.314+0000 7fc42f7fe640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 0x7fc4240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:43.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.315+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fc4340d06f0 con 0x7fc438101de0 2024-09-15T22:34:43.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.315+0000 7fc43dc23640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 0x7fc4240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:43.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.316+0000 7fc43dc23640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 0x7fc4240a5870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc428009830 tx=0x7fc42800b040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:43.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.319+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc434026080 con 0x7fc438101de0 2024-09-15T22:34:43.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160058 15.51 MB/sec execute 404 sec latency 150.785 ms 2024-09-15T22:34:43.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.622+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc41c005580 con 0x7fc438101de0 2024-09-15T22:34:43.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.623+0000 7fc42f7fe640 1 -- 172.21.15.77:0/372422702 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fc434098070 con 0x7fc438101de0 2024-09-15T22:34:43.625 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:34:43.625 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:34:43.626 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:34:43.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 msgr2=0x7fc4240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 0x7fc4240a5870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc428009830 tx=0x7fc42800b040 comp rx=0 tx=0).stop 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 msgr2=0x7fc4380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e9f0 tx=0x7fc43400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 shutdown_connections 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4240a33b0 0x7fc4240a5870 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc438106f10 0x7fc438074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc438102790 0x7fc438074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 --2- 172.21.15.77:0/372422702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc438101de0 0x7fc4380744f0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:43.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 >> 172.21.15.77:0/372422702 conn(0x7fc4380fd8e0 msgr2=0x7fc4380fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:43.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 shutdown_connections 2024-09-15T22:34:43.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:43.627+0000 7fc440688640 1 -- 172.21.15.77:0/372422702 wait complete. 2024-09-15T22:34:44.269 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:44 smithi077 ceph-mon[30980]: pgmap v704: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-09-15T22:34:44.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:44 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/372422702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:44.302 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0' 2024-09-15T22:34:44.303 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0 2024-09-15T22:34:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:44 smithi196 ceph-mon[35109]: pgmap v704: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-09-15T22:34:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:44 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/372422702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:44 smithi117 ceph-mon[35021]: pgmap v704: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 549 op/s 2024-09-15T22:34:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:44 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/372422702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:34:44.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160452 15.53 MB/sec execute 405 sec latency 142.058 ms 2024-09-15T22:34:44.635 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:45.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 -- 172.21.15.77:0/5833037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 msgr2=0x7fd090102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 --2- 172.21.15.77:0/5833037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd090102f70 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd080009900 tx=0x7fd08002a480 comp rx=0 tx=0).stop 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 -- 172.21.15.77:0/5833037 shutdown_connections 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 --2- 172.21.15.77:0/5833037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd09010bfc0 0x7fd09010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 --2- 172.21.15.77:0/5833037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0901034b0 0x7fd09010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 --2- 172.21.15.77:0/5833037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd090102f70 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 -- 172.21.15.77:0/5833037 >> 172.21.15.77:0/5833037 conn(0x7fd0900fc610 msgr2=0x7fd0900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 -- 172.21.15.77:0/5833037 shutdown_connections 2024-09-15T22:34:45.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.400+0000 7fd094d9f640 1 -- 172.21.15.77:0/5833037 wait complete. 2024-09-15T22:34:45.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.401+0000 7fd094d9f640 1 Processor -- start 2024-09-15T22:34:45.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.401+0000 7fd094d9f640 1 -- start start 2024-09-15T22:34:45.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd09019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 0x7fd09019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd09010bfc0 0x7fd09019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd090113f80 con 0x7fd090102b90 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd090113e00 con 0x7fd09010bfc0 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd094d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd090114100 con 0x7fd0901034b0 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd08effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 0x7fd09019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd08effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 0x7fd09019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:54752/0 (socket says 172.21.15.77:54752) 2024-09-15T22:34:45.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd08effd640 1 -- 172.21.15.77:0/2246601255 learned_addr learned my addr 172.21.15.77:0/2246601255 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:45.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.402+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd09019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd09010bfc0 0x7fd09019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 -- 172.21.15.77:0/2246601255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 msgr2=0x7fd09019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 0x7fd09019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 -- 172.21.15.77:0/2246601255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd09010bfc0 msgr2=0x7fd09019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd09010bfc0 0x7fd09019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 -- 172.21.15.77:0/2246601255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09019f490 con 0x7fd090102b90 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.403+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd09019a4e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd080005940 tx=0x7fd0800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.404+0000 7fd08cff9640 1 -- 172.21.15.77:0/2246601255 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08003f030 con 0x7fd090102b90 2024-09-15T22:34:45.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.404+0000 7fd08cff9640 1 -- 172.21.15.77:0/2246601255 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0800044c0 con 0x7fd090102b90 2024-09-15T22:34:45.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.404+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09019f720 con 0x7fd090102b90 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.404+0000 7fd08cff9640 1 -- 172.21.15.77:0/2246601255 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080035070 con 0x7fd090102b90 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.404+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a7060 con 0x7fd090102b90 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.408+0000 7fd0983da740 1 Processor -- start 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 -- start start 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc373500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc37ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc3806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572dc2bf2a0 con 0x5572dc37e300 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5572dc266b10 con 0x5572dc373120 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.411+0000 7fd0983da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572dc2bb260 con 0x5572dc373a40 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc373500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc373500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:53316/0 (socket says 172.21.15.77:53316) 2024-09-15T22:34:45.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08ffff640 1 -- 172.21.15.77:0/483410244 learned_addr learned my addr 172.21.15.77:0/483410244 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc37ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08effd640 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc3806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08effd640 1 -- 172.21.15.77:0/483410244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 msgr2=0x5572dc37ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08effd640 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc37ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08effd640 1 -- 172.21.15.77:0/483410244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 msgr2=0x5572dc373500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.412+0000 7fd08effd640 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc373500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.413+0000 7fd08effd640 1 -- 172.21.15.77:0/483410244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dc030b90 con 0x5572dc37e300 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.413+0000 7fd08effd640 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc3806f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd084009bf0 tx=0x7fd0840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b735ac416cac188d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.413+0000 7fd08cff9640 1 -- 172.21.15.77:0/2246601255 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd080042030 con 0x7fd090102b90 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.413+0000 7fd075ffb640 1 -- 172.21.15.77:0/483410244 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084012450 con 0x5572dc37e300 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.413+0000 7fd075ffb640 1 -- 172.21.15.77:0/483410244 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084012ad0 con 0x5572dc37e300 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.414+0000 7fd08cff9640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 0x7fd0780a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.414+0000 7fd08cff9640 1 -- 172.21.15.77:0/2246601255 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd0800f04a0 con 0x7fd090102b90 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.414+0000 7fd075ffb640 1 -- 172.21.15.77:0/483410244 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08400fc50 con 0x5572dc37e300 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.414+0000 7fd0983da740 1 -- 172.21.15.77:0/483410244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 msgr2=0x5572dc3806f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.414+0000 7fd0983da740 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc3806f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd084009bf0 tx=0x7fd0840117b0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 -- 172.21.15.77:0/483410244 shutdown_connections 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc3806f0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc37ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 --2- 172.21.15.77:0/483410244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc373500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 -- 172.21.15.77:0/483410244 >> 172.21.15.77:0/483410244 conn(0x5572dc36f6b0 msgr2=0x5572dc36fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:45.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 0x7fd0780a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 -- 172.21.15.77:0/483410244 shutdown_connections 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.415+0000 7fd0983da740 1 -- 172.21.15.77:0/483410244 wait complete. 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.416+0000 7fd0983da740 1 -- start start 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.416+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 0x7fd0780a5970 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd088005fd0 tx=0x7fd088009500 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.416+0000 7fd0983da740 1 Processor -- start 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373120 0x5572dc37b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc37e300 0x5572dc37bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5572dc380f20 con 0x5572dc373a40 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5572dc2476e0 con 0x5572dc37e300 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572dc3810a0 con 0x5572dc373120 2024-09-15T22:34:45.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd0983da740 10 client.0 _ll_register_callbacks cb 0x5572dc26ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd08effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd08effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52264/0 (socket says 172.21.15.77:52264) 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.417+0000 7fd08effd640 1 -- 172.21.15.77:0/3562222228 learned_addr learned my addr 172.21.15.77:0/3562222228 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 -- 172.21.15.77:0/3562222228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373120 msgr2=0x5572dc37b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373120 0x5572dc37b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 -- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc37e300 msgr2=0x5572dc37bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc37e300 0x5572dc37bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 -- 172.21.15.77:0/3562222228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dc38a470 con 0x5572dc373a40 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd08effd640 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd084012010 tx=0x7fd084004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dac2836ac7eddf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.418+0000 7fd06effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.419+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084012650 con 0x5572dc373a40 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.419+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd0840127f0 con 0x5572dc373a40 2024-09-15T22:34:45.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.419+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd084012ad0 con 0x5572dc373a40 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.420+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd08400f8d0 con 0x5572dc373a40 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.420+0000 7fd0983da740 10 client.20598 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.420+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572dc38cde0 con 0x5572dc373a40 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.420+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084029070 con 0x5572dc373a40 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.420+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd0840046c0 con 0x5572dc373a40 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 10 client.20598 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 10 client.20598 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 20 client.20598 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc390580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 4 client.20598 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:34:45.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.421+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd084012ad0 con 0x5572dc38e190 2024-09-15T22:34:45.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.424+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc390580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.425+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc390580 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec96ee379f9e016 server_cookie=d7ee1ead448de6d in_seq=0 out_seq=0 2024-09-15T22:34:45.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.425+0000 7fd06effd640 10 client.20598 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:45.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.429+0000 7fd06effd640 1 -- 172.21.15.77:0/3562222228 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd084012ad0 con 0x5572dc38e190 2024-09-15T22:34:45.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.429+0000 7fd06effd640 10 client.20598 handle_command_reply: tid=0 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 client.20598 shutdown 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 msgr2=0x5572dc37b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd084012010 tx=0x7fd084004270 comp rx=0 tx=0).stop 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 shutdown_connections 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc390580 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd06effd640 0 client.20598 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc37e300 0x5572dc37bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc37b6d0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 --2- 172.21.15.77:0/3562222228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373120 0x5572dc37b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 >> 172.21.15.77:0/3562222228 conn(0x5572dc36f6b0 msgr2=0x5572dc366010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 shutdown_connections 2024-09-15T22:34:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.430+0000 7fd0983da740 1 -- 172.21.15.77:0/3562222228 wait complete. 2024-09-15T22:34:45.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.431+0000 7fd0983da740 20 client.20598 trim_cache size 0 max 16384 2024-09-15T22:34:45.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 Processor -- start 2024-09-15T22:34:45.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 -- start start 2024-09-15T22:34:45.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc37ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc395d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0900716d0 con 0x5572dc37e300 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd090073210 con 0x5572dc373120 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.460+0000 7fd0983da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd090071da0 con 0x5572dc373a40 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52270/0 (socket says 172.21.15.77:52270) 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 -- 172.21.15.77:0/2686470367 learned_addr learned my addr 172.21.15.77:0/2686470367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08effd640 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc395d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc37ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 -- 172.21.15.77:0/2686470367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 msgr2=0x5572dc395d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc395d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 -- 172.21.15.77:0/2686470367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 msgr2=0x5572dc37ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.461+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc37ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.462+0000 7fd08ffff640 1 -- 172.21.15.77:0/2686470367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dc37d8a0 con 0x5572dc37e300 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.462+0000 7fd08ffff640 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd0880049a0 tx=0x7fd088004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f23c92e50885ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.462+0000 7fd0757fa640 1 -- 172.21.15.77:0/2686470367 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08802c070 con 0x5572dc37e300 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.463+0000 7fd0757fa640 1 -- 172.21.15.77:0/2686470367 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08801fe50 con 0x5572dc37e300 2024-09-15T22:34:45.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.463+0000 7fd0757fa640 1 -- 172.21.15.77:0/2686470367 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088011740 con 0x5572dc37e300 2024-09-15T22:34:45.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 -- 172.21.15.77:0/2686470367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 msgr2=0x5572dc396280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:45.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd0880049a0 tx=0x7fd088004a40 comp rx=0 tx=0).stop 2024-09-15T22:34:45.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 -- 172.21.15.77:0/2686470367 shutdown_connections 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc37e300 0x5572dc396280 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc373a40 0x5572dc395d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 --2- 172.21.15.77:0/2686470367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc37ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 -- 172.21.15.77:0/2686470367 >> 172.21.15.77:0/2686470367 conn(0x5572dc36f6b0 msgr2=0x5572dc366010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 -- 172.21.15.77:0/2686470367 shutdown_connections 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.464+0000 7fd0983da740 1 -- 172.21.15.77:0/2686470367 wait complete. 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.465+0000 7fd0983da740 1 -- start start 2024-09-15T22:34:45.467 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.465+0000 7fd0983da740 1 Processor -- start 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc3863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc3868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 0x5572dc383370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd090072b40 con 0x5572dc373a40 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd090072470 con 0x5572dc373120 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd09006f6d0 con 0x5572dc37e300 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.466+0000 7fd0983da740 10 client.0 _ll_register_callbacks cb 0x7fd084012eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 0x5572dc383370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 0x5572dc383370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:54790/0 (socket says 172.21.15.77:54790) 2024-09-15T22:34:45.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08ffff640 1 -- 172.21.15.77:0/4155825183 learned_addr learned my addr 172.21.15.77:0/4155825183 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc3868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 -- 172.21.15.77:0/4155825183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 msgr2=0x5572dc383370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 0x5572dc383370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 -- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 msgr2=0x5572dc3863a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc3863a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 -- 172.21.15.77:0/4155825183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dc3838b0 con 0x5572dc373a40 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd08effd640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc3868e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd084011cb0 tx=0x7fd0840092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16907388711c998 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:45.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.467+0000 7fd06effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.468+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084005290 con 0x5572dc373a40 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.469+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7fd084005430 con 0x5572dc373a40 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.469+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08801fe50 con 0x5572dc373a40 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.469+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd084009810 con 0x5572dc373a40 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.469+0000 7fd0983da740 10 client.20604 fetch_fsmap learned FSMap version 39 2024-09-15T22:34:45.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.469+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572dc3f93b0 con 0x5572dc373a40 2024-09-15T22:34:45.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.470+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08403c070 con 0x5572dc373a40 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fd084036040 con 0x5572dc373a40 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 10 client.20604 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 10 client.20604 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 20 client.20604 populate_metadata read hostname 'smithi077' 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc3fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 4 client.20604 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}] 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}) -- 0x7fd08801fe50 con 0x5572dc38e190 2024-09-15T22:34:45.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.471+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc3fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:45.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.472+0000 7fd08f7fe640 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc3fbac0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d668b7d1905fb00 server_cookie=9e2c1afe4a958ab in_seq=0 out_seq=0 2024-09-15T22:34:45.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:45.472+0000 7fd06effd640 10 client.20604 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:45.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160836 15.54 MB/sec execute 406 sec latency 171.317 ms 2024-09-15T22:34:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:46 smithi077 ceph-mon[30980]: pgmap v705: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-09-15T22:34:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:34:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:34:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:34:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:46 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:34:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:46 smithi196 ceph-mon[35109]: pgmap v705: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-09-15T22:34:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:34:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:34:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:34:46.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:46 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:34:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:46 smithi117 ceph-mon[35021]: pgmap v705: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-09-15T22:34:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:34:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:34:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:34:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:46 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:34:46.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160848 15.51 MB/sec execute 407 sec latency 1030.948 ms 2024-09-15T22:34:46.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.638+0000 7fd06effd640 1 -- 172.21.15.77:0/4155825183 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fd08801fe50 con 0x5572dc38e190 2024-09-15T22:34:46.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.638+0000 7fd06effd640 10 client.20604 handle_command_reply: tid=0 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 client.20604 shutdown 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 msgr2=0x5572dc3868e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc3868e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fd084011cb0 tx=0x7fd0840092a0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 shutdown_connections 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5572dc38e190 0x5572dc3fbac0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572dc37e300 0x5572dc383370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5572dc373a40 0x5572dc3868e0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd06effd640 0 client.20604 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 --2- 172.21.15.77:0/4155825183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572dc373120 0x5572dc3863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 >> 172.21.15.77:0/4155825183 conn(0x5572dc36f6b0 msgr2=0x5572dc379930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 shutdown_connections 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 1 -- 172.21.15.77:0/4155825183 wait complete. 2024-09-15T22:34:46.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.639+0000 7fd0983da740 20 client.20604 trim_cache size 0 max 16384 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 78, 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "f7de3897": { 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 78, 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:34:46.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 154.4, 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:34:46.643 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 msgr2=0x7fd0780a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 0x7fd0780a5970 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd088005fd0 tx=0x7fd088009500 comp rx=0 tx=0).stop 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 msgr2=0x7fd09019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd09019a4e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd080005940 tx=0x7fd0800040c0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 shutdown_connections 2024-09-15T22:34:46.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fd0780a34b0 0x7fd0780a5970 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd09010bfc0 0x7fd09019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0901034b0 0x7fd09019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 --2- 172.21.15.77:0/2246601255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd090102b90 0x7fd09019a4e0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 >> 172.21.15.77:0/2246601255 conn(0x7fd0900fc610 msgr2=0x7fd0900fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 shutdown_connections 2024-09-15T22:34:46.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:34:46.640+0000 7fd094d9f640 1 -- 172.21.15.77:0/2246601255 wait complete. 2024-09-15T22:34:47.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:47 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:34:47.323 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f7de3897', quiesce took 4.9 seconds. Will release after: 29.5 2024-09-15T22:34:47.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:47 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:34:47.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:47 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:34:47.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.47 MB/sec execute 408 sec latency 2031.045 ms 2024-09-15T22:34:48.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:48 smithi077 ceph-mon[30980]: pgmap v706: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-09-15T22:34:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:48 smithi196 ceph-mon[35109]: pgmap v706: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-09-15T22:34:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:48 smithi117 ceph-mon[35021]: pgmap v706: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 536 op/s 2024-09-15T22:34:48.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.43 MB/sec execute 409 sec latency 3031.128 ms 2024-09-15T22:34:49.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.39 MB/sec execute 410 sec latency 4031.229 ms 2024-09-15T22:34:50.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:50 smithi196 ceph-mon[35109]: pgmap v707: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-15T22:34:50.414 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:50 smithi117 ceph-mon[35021]: pgmap v707: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-15T22:34:50.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:50 smithi077 ceph-mon[30980]: pgmap v707: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-15T22:34:50.618 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.36 MB/sec execute 411 sec latency 5031.287 ms 2024-09-15T22:34:51.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.32 MB/sec execute 412 sec latency 6031.339 ms 2024-09-15T22:34:52.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:52 smithi077 ceph-mon[30980]: pgmap v708: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:52 smithi196 ceph-mon[35109]: pgmap v708: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:52 smithi117 ceph-mon[35021]: pgmap v708: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:52.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.28 MB/sec execute 413 sec latency 7031.436 ms 2024-09-15T22:34:53.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.24 MB/sec execute 414 sec latency 8031.569 ms 2024-09-15T22:34:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:54 smithi077 ceph-mon[30980]: pgmap v709: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:54 smithi196 ceph-mon[35109]: pgmap v709: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:54 smithi117 ceph-mon[35021]: pgmap v709: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-09-15T22:34:54.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.21 MB/sec execute 415 sec latency 9031.667 ms 2024-09-15T22:34:55.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.17 MB/sec execute 416 sec latency 10031.754 ms 2024-09-15T22:34:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:56 smithi077 ceph-mon[30980]: pgmap v710: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-15T22:34:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:56 smithi196 ceph-mon[35109]: pgmap v710: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-15T22:34:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:56 smithi117 ceph-mon[35021]: pgmap v710: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-09-15T22:34:56.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.13 MB/sec execute 417 sec latency 11031.826 ms 2024-09-15T22:34:57.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.10 MB/sec execute 418 sec latency 12031.874 ms 2024-09-15T22:34:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:34:58 smithi077 ceph-mon[30980]: pgmap v711: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2024-09-15T22:34:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:34:58 smithi196 ceph-mon[35109]: pgmap v711: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2024-09-15T22:34:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:34:58 smithi117 ceph-mon[35021]: pgmap v711: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 193 op/s 2024-09-15T22:34:58.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.06 MB/sec execute 419 sec latency 13031.974 ms 2024-09-15T22:34:59.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 15.03 MB/sec execute 420 sec latency 14032.024 ms 2024-09-15T22:35:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:00 smithi077 ceph-mon[30980]: pgmap v712: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 507 KiB/s wr, 50 op/s 2024-09-15T22:35:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:00 smithi196 ceph-mon[35109]: pgmap v712: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 507 KiB/s wr, 50 op/s 2024-09-15T22:35:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:00 smithi117 ceph-mon[35021]: pgmap v712: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 507 KiB/s wr, 50 op/s 2024-09-15T22:35:00.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.99 MB/sec execute 421 sec latency 15032.063 ms 2024-09-15T22:35:01.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.95 MB/sec execute 422 sec latency 16032.094 ms 2024-09-15T22:35:02.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:02 smithi077 ceph-mon[30980]: pgmap v713: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 507 KiB/s wr, 53 op/s 2024-09-15T22:35:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:02 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:02 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:02 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:02 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:35:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:02 smithi196 ceph-mon[35109]: pgmap v713: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 507 KiB/s wr, 53 op/s 2024-09-15T22:35:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:02 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:02 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:02 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:02.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:02 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:35:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:02 smithi117 ceph-mon[35021]: pgmap v713: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 507 KiB/s wr, 53 op/s 2024-09-15T22:35:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:02 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:02 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:02 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:02 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:35:02.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.92 MB/sec execute 423 sec latency 17032.130 ms 2024-09-15T22:35:03.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.88 MB/sec execute 424 sec latency 18032.218 ms 2024-09-15T22:35:04.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:04 smithi077 ceph-mon[30980]: pgmap v714: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 404 KiB/s wr, 62 op/s 2024-09-15T22:35:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:04 smithi196 ceph-mon[35109]: pgmap v714: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 404 KiB/s wr, 62 op/s 2024-09-15T22:35:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:04 smithi117 ceph-mon[35021]: pgmap v714: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 404 KiB/s wr, 62 op/s 2024-09-15T22:35:04.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.85 MB/sec execute 425 sec latency 19032.312 ms 2024-09-15T22:35:05.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.81 MB/sec execute 426 sec latency 20032.362 ms 2024-09-15T22:35:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:06 smithi077 ceph-mon[30980]: pgmap v715: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-09-15T22:35:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:06 smithi196 ceph-mon[35109]: pgmap v715: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-09-15T22:35:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:06 smithi117 ceph-mon[35021]: pgmap v715: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-09-15T22:35:06.619 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.78 MB/sec execute 427 sec latency 21032.431 ms 2024-09-15T22:35:07.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.75 MB/sec execute 428 sec latency 22032.513 ms 2024-09-15T22:35:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:08 smithi077 ceph-mon[30980]: pgmap v716: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2024-09-15T22:35:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:08 smithi196 ceph-mon[35109]: pgmap v716: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2024-09-15T22:35:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:08 smithi117 ceph-mon[35021]: pgmap v716: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2024-09-15T22:35:08.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.71 MB/sec execute 429 sec latency 23032.655 ms 2024-09-15T22:35:08.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:35:09.174 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:09.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.68 MB/sec execute 430 sec latency 24032.721 ms 2024-09-15T22:35:09.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 -- 172.21.15.77:0/1107020681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc100a90 msgr2=0x7f7bdc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 --2- 172.21.15.77:0/1107020681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc100a90 0x7f7bdc100ef0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009930 tx=0x7f7bd002a420 comp rx=0 tx=0).stop 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 -- 172.21.15.77:0/1107020681 shutdown_connections 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 --2- 172.21.15.77:0/1107020681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc101430 0x7f7bdc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 --2- 172.21.15.77:0/1107020681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc100a90 0x7f7bdc100ef0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 --2- 172.21.15.77:0/1107020681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc106a90 0x7f7bdc106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 -- 172.21.15.77:0/1107020681 >> 172.21.15.77:0/1107020681 conn(0x7f7bdc0fc7d0 msgr2=0x7f7bdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 -- 172.21.15.77:0/1107020681 shutdown_connections 2024-09-15T22:35:09.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.663+0000 7f7be221d640 1 -- 172.21.15.77:0/1107020681 wait complete. 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 Processor -- start 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 -- start start 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc100a90 0x7f7bdc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 0x7f7bdc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc110eb0 con 0x7f7bdc101430 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc110d30 con 0x7f7bdc100a90 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7be221d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc111030 con 0x7f7bdc106a90 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56052/0 (socket says 172.21.15.77:56052) 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdaffd640 1 -- 172.21.15.77:0/1214495404 learned_addr learned my addr 172.21.15.77:0/1214495404 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc100a90 0x7f7bdc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 0x7f7bdc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdaffd640 1 -- 172.21.15.77:0/1214495404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 msgr2=0x7f7bdc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.664+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 0x7f7bdc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bdaffd640 1 -- 172.21.15.77:0/1214495404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc100a90 msgr2=0x7f7bdc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc100a90 0x7f7bdc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bdaffd640 1 -- 172.21.15.77:0/1214495404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc19d480 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 0x7f7bdc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009230 tx=0x7f7bd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bd8ff9640 1 -- 172.21.15.77:0/1214495404 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd003f030 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc19d710 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bd8ff9640 1 -- 172.21.15.77:0/1214495404 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0003ad0 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7bd8ff9640 1 -- 172.21.15.77:0/1214495404 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0035070 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.665+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1a5050 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.667+0000 7f7bd8ff9640 1 -- 172.21.15.77:0/1214495404 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7bd0042030 con 0x7f7bdc101430 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.667+0000 7f7be45f6740 1 Processor -- start 2024-09-15T22:35:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.667+0000 7f7be45f6740 1 -- start start 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 0x55d44bdc8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdd3680 0x55d44bdd5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d44bd2e2e0 con 0x55d44bdc84a0 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d44bcd5ce0 con 0x55d44bdd3680 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7be45f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d44bd2a2a0 con 0x55d44bdc8dc0 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bd8ff9640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 0x7f7bc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 0x55d44bdc8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 0x55d44bdc8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56058/0 (socket says 172.21.15.77:56058) 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdbfff640 1 -- 172.21.15.77:0/2595106658 learned_addr learned my addr 172.21.15.77:0/2595106658 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 0x7f7bc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bd8ff9640 1 -- 172.21.15.77:0/1214495404 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f7bd00f0650 con 0x7f7bdc101430 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdd3680 0x55d44bdd5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/2595106658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdd3680 msgr2=0x55d44bdd5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdd3680 0x55d44bdd5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/2595106658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 msgr2=0x55d44bdc8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 0x55d44bdc8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/2595106658 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44bcba440 con 0x55d44bdc8dc0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd3140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00bbe0 tx=0x7f7bcc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79832b77bf2abd93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.668+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 0x7f7bc40a5870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400eca0 tx=0x7f7bd4009ad0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.669+0000 7f7bc1ffb640 1 -- 172.21.15.77:0/2595106658 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc015070 con 0x55d44bdc8dc0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.669+0000 7f7bc1ffb640 1 -- 172.21.15.77:0/2595106658 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc0035f0 con 0x55d44bdc8dc0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.669+0000 7f7bc1ffb640 1 -- 172.21.15.77:0/2595106658 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0116a0 con 0x55d44bdc8dc0 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.669+0000 7f7be45f6740 1 -- 172.21.15.77:0/2595106658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 msgr2=0x55d44bdd3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.669+0000 7f7be45f6740 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd3140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00bbe0 tx=0x7f7bcc00f730 comp rx=0 tx=0).stop 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- 172.21.15.77:0/2595106658 shutdown_connections 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdd3680 0x55d44bdd5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd3140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- 172.21.15.77:0/2595106658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc84a0 0x55d44bdc8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- 172.21.15.77:0/2595106658 >> 172.21.15.77:0/2595106658 conn(0x55d44bdc4a30 msgr2=0x55d44bdc4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- 172.21.15.77:0/2595106658 shutdown_connections 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- 172.21.15.77:0/2595106658 wait complete. 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- start start 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 Processor -- start 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdd2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d44bdd6250 con 0x55d44bdd3680 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d44bce9050 con 0x55d44bdc84a0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.670+0000 7f7be45f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d44bdd63d0 con 0x55d44bdc8dc0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7be45f6740 10 client.0 _ll_register_callbacks cb 0x55d44bd29160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56074/0 (socket says 172.21.15.77:56074) 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 -- 172.21.15.77:0/1868507037 learned_addr learned my addr 172.21.15.77:0/1868507037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdd2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 -- 172.21.15.77:0/1868507037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 msgr2=0x55d44bdd0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 -- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 msgr2=0x55d44bdd2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdd2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 -- 172.21.15.77:0/1868507037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44ba9f3c0 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400efd0 tx=0x7f7bd40097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a7b3c75da4ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd401ac10 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bcc0035f0 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f7bd401adb0 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7bd4031050 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7be45f6740 10 client.20616 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.671+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d44bde1bf0 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc0035f0 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7bd401b070 con 0x55d44bdd3680 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 10 client.20616 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:09.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 10 client.20616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 20 client.20616 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bde30b0 0x55d44bde5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 4 client.20616 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d44bd3b850 con 0x55d44bde30b0 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bde30b0 0x55d44bde5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bde30b0 0x55d44bde5500 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c8670b10abc171 server_cookie=a7fc856647447afc in_seq=0 out_seq=0 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.672+0000 7f7bbaffd640 10 client.20616 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7bbaffd640 1 -- 172.21.15.77:0/1868507037 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d44bd3b850 con 0x55d44bde30b0 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7bbaffd640 10 client.20616 handle_command_reply: tid=0 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 client.20616 shutdown 2024-09-15T22:35:09.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 msgr2=0x55d44bdd0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400efd0 tx=0x7f7bd40097b0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 shutdown_connections 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bde30b0 0x55d44bde5500 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdd3680 0x55d44bdd0740 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdd0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7bbaffd640 0 client.20616 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 --2- 172.21.15.77:0/1868507037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdd2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 >> 172.21.15.77:0/1868507037 conn(0x55d44bdc4a30 msgr2=0x55d44bdca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 shutdown_connections 2024-09-15T22:35:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 1 -- 172.21.15.77:0/1868507037 wait complete. 2024-09-15T22:35:09.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.673+0000 7f7be45f6740 20 client.20616 trim_cache size 0 max 16384 2024-09-15T22:35:09.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.690+0000 7f7be45f6740 1 Processor -- start 2024-09-15T22:35:09.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.690+0000 7f7be45f6740 1 -- start start 2024-09-15T22:35:09.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc84a0 0x55d44bdd01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1aef70 con 0x55d44bdc4a30 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1b0ab0 con 0x55d44bdc8dc0 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7be45f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1af640 con 0x55d44bdc84a0 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56088/0 (socket says 172.21.15.77:56088) 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/262322399 learned_addr learned my addr 172.21.15.77:0/262322399 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/262322399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc84a0 msgr2=0x55d44bdd01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc84a0 0x55d44bdd01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/262322399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 msgr2=0x55d44bdcb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/262322399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44bdc4860 con 0x55d44bdc4a30 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0058c0 tx=0x7f7bcc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21225ba7a8d5da34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bc17fa640 1 -- 172.21.15.77:0/262322399 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc02c070 con 0x55d44bdc4a30 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bc17fa640 1 -- 172.21.15.77:0/262322399 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc00b3a0 con 0x55d44bdc4a30 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.691+0000 7f7bc17fa640 1 -- 172.21.15.77:0/262322399 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc015050 con 0x55d44bdc4a30 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- 172.21.15.77:0/262322399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 msgr2=0x55d44bddf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0058c0 tx=0x7f7bcc005850 comp rx=0 tx=0).stop 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- 172.21.15.77:0/262322399 shutdown_connections 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc84a0 0x55d44bdd01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 --2- 172.21.15.77:0/262322399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bddf4b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- 172.21.15.77:0/262322399 >> 172.21.15.77:0/262322399 conn(0x55d44bdd08a0 msgr2=0x55d44bdcb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- 172.21.15.77:0/262322399 shutdown_connections 2024-09-15T22:35:09.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- 172.21.15.77:0/262322399 wait complete. 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.692+0000 7f7be45f6740 1 -- start start 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 Processor -- start 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdcd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1b03e0 con 0x55d44bdc4a30 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1afd10 con 0x55d44bdc84a0 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1acf70 con 0x55d44bdc8dc0 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7be45f6740 10 client.0 _ll_register_callbacks cb 0x7f7bd400ecd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56104/0 (socket says 172.21.15.77:56104) 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/605025886 learned_addr learned my addr 172.21.15.77:0/605025886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdcd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdbfff640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/605025886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 msgr2=0x55d44bdcdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 msgr2=0x55d44bdcd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdcd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdb7fe640 1 -- 172.21.15.77:0/605025886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d44bdc56c0 con 0x55d44bdc4a30 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.693+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdcd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bdb7fe640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc00fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1962ef04f727d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:09.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc035060 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9438+0+0 (secure 0 0 0) 0x7f7bcc0035f0 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d44bd35d60 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7bcc015020 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 10 client.20622 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d44be4de70 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d44bd35d60 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f7bcc02f070 con 0x55d44bdc4a30 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 10 client.20622 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 10 client.20622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 20 client.20622 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bdd3680 0x55d44be50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 4 client.20622 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:35:09.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.694+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d44bcb2210 con 0x55d44bdd3680 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bdd3680 0x55d44be50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bdaffd640 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bdd3680 0x55d44be50340 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d453d876485e20 server_cookie=6c64c0c5e3c6c2aa in_seq=0 out_seq=0 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bbaffd640 10 client.20622 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bbaffd640 1 -- 172.21.15.77:0/605025886 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d44bcb2210 con 0x55d44bdd3680 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bbaffd640 10 client.20622 handle_command_reply: tid=0 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 client.20622 shutdown 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 msgr2=0x55d44bdcd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc00fdd0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 shutdown_connections 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d44bdd3680 0x55d44be50340 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7bbaffd640 0 client.20622 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d44bdc8dc0 0x55d44bdcdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d44bdc84a0 0x55d44bdcd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 --2- 172.21.15.77:0/605025886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d44bdc4a30 0x55d44bdcd1b0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 >> 172.21.15.77:0/605025886 conn(0x55d44bdd08a0 msgr2=0x55d44bdcb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 shutdown_connections 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.695+0000 7f7be45f6740 1 -- 172.21.15.77:0/605025886 wait complete. 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be45f6740 20 client.20622 trim_cache size 0 max 16384 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:35:09.707 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:35:09.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 msgr2=0x7f7bc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 0x7f7bc40a5870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400eca0 tx=0x7f7bd4009ad0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 msgr2=0x7f7bdc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009230 tx=0x7f7bd0004270 comp rx=0 tx=0).stop 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 shutdown_connections 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7bc40a33b0 0x7f7bc40a5870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc106a90 0x7f7bdc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bdc101430 0x7f7bdc198a10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 --2- 172.21.15.77:0/1214495404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bdc100a90 0x7f7bdc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 >> 172.21.15.77:0/1214495404 conn(0x7f7bdc0fc7d0 msgr2=0x7f7bdc1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 shutdown_connections 2024-09-15T22:35:09.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:09.696+0000 7f7be221d640 1 -- 172.21.15.77:0/1214495404 wait complete. 2024-09-15T22:35:10.566 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:10 smithi077 ceph-mon[30980]: pgmap v717: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 79 op/s 2024-09-15T22:35:10.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:35:10.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:10 smithi196 ceph-mon[35109]: pgmap v717: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 79 op/s 2024-09-15T22:35:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:10 smithi117 ceph-mon[35021]: pgmap v717: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 79 op/s 2024-09-15T22:35:10.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.64 MB/sec execute 431 sec latency 25032.754 ms 2024-09-15T22:35:10.847 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:11.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 -- 172.21.15.77:0/3212249708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462010e480 msgr2=0x7f4620110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:11.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 --2- 172.21.15.77:0/3212249708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462010e480 0x7f4620110870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4618009be0 tx=0x7f461802a710 comp rx=0 tx=0).stop 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 -- 172.21.15.77:0/3212249708 shutdown_connections 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 --2- 172.21.15.77:0/3212249708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462010e480 0x7f4620110870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 --2- 172.21.15.77:0/3212249708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4620101280 0x7f462010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 --2- 172.21.15.77:0/3212249708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46201008d0 0x7f4620100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.412+0000 7f4625c4c640 1 -- 172.21.15.77:0/3212249708 >> 172.21.15.77:0/3212249708 conn(0x7f46200fc7a0 msgr2=0x7f46200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:11.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.413+0000 7f4625c4c640 1 -- 172.21.15.77:0/3212249708 shutdown_connections 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.415+0000 7f4625c4c640 1 -- 172.21.15.77:0/3212249708 wait complete. 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 Processor -- start 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 -- start start 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46201008d0 0x7f4620102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4620101280 0x7f46201030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:11.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46201141c0 con 0x7f46201008d0 2024-09-15T22:35:11.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4620114040 con 0x7f462010e480 2024-09-15T22:35:11.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.416+0000 7f4625c4c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4620114340 con 0x7f4620101280 2024-09-15T22:35:11.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:11.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55518/0 (socket says 172.21.15.77:55518) 2024-09-15T22:35:11.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 -- 172.21.15.77:0/108683014 learned_addr learned my addr 172.21.15.77:0/108683014 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:11.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461effd640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4620101280 0x7f46201030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 -- 172.21.15.77:0/108683014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4620101280 msgr2=0x7f46201030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4620101280 0x7f46201030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 -- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46201008d0 msgr2=0x7f4620102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.418+0000 7f461ffff640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46201008d0 0x7f4620102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f461ffff640 1 -- 172.21.15.77:0/108683014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620072af0 con 0x7f462010e480 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f461ffff640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f461802ad00 tx=0x7f4618003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618036070 con 0x7f462010e480 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4618031e50 con 0x7f462010e480 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4620072d80 con 0x7f462010e480 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618004850 con 0x7f462010e480 2024-09-15T22:35:11.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.419+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46200732b0 con 0x7f462010e480 2024-09-15T22:35:11.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.424+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4618031440 con 0x7f462010e480 2024-09-15T22:35:11.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.424+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462006b070 con 0x7f462010e480 2024-09-15T22:35:11.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.425+0000 7f461cff9640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 0x7f46080a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:11.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.425+0000 7f461f7fe640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 0x7f46080a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:11.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.425+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f46180f11e0 con 0x7f462010e480 2024-09-15T22:35:11.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.427+0000 7f461f7fe640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 0x7f46080a5b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4610005fd0 tx=0x7f4610005a00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:11.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.431+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46180bd770 con 0x7f462010e480 2024-09-15T22:35:11.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.61 MB/sec execute 432 sec latency 26032.790 ms 2024-09-15T22:35:11.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.801+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4620073df0 con 0x7f462010e480 2024-09-15T22:35:11.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.802+0000 7f461cff9640 1 -- 172.21.15.77:0/108683014 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f46180bd070 con 0x7f462010e480 2024-09-15T22:35:11.804 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:35:11.804 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:35:11.804 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:35:11.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.804+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 msgr2=0x7f46080a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:11.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.804+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 0x7f46080a5b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4610005fd0 tx=0x7f4610005a00 comp rx=0 tx=0).stop 2024-09-15T22:35:11.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.804+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 msgr2=0x7f4620103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.804+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f461802ad00 tx=0x7f4618003f10 comp rx=0 tx=0).stop 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 shutdown_connections 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f46080a36a0 0x7f46080a5b60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f462010e480 0x7f4620103630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4620101280 0x7f46201030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 --2- 172.21.15.77:0/108683014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46201008d0 0x7f4620102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 >> 172.21.15.77:0/108683014 conn(0x7f46200fc7a0 msgr2=0x7f46200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 shutdown_connections 2024-09-15T22:35:11.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:11.805+0000 7f4625c4c640 1 -- 172.21.15.77:0/108683014 wait complete. 2024-09-15T22:35:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:12 smithi077 ceph-mon[30980]: pgmap v718: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-15T22:35:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:12 smithi077 ceph-mon[30980]: osdmap e124: 12 total, 12 up, 12 in 2024-09-15T22:35:12.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:12 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/108683014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:12 smithi196 ceph-mon[35109]: pgmap v718: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-15T22:35:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:12 smithi196 ceph-mon[35109]: osdmap e124: 12 total, 12 up, 12 in 2024-09-15T22:35:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:12 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/108683014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:12 smithi117 ceph-mon[35021]: pgmap v718: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-15T22:35:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:12 smithi117 ceph-mon[35021]: osdmap e124: 12 total, 12 up, 12 in 2024-09-15T22:35:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:12 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/108683014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:12.617 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:35:12.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.58 MB/sec execute 433 sec latency 27032.844 ms 2024-09-15T22:35:12.871 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 -- 172.21.15.77:0/2353307069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4100a80 msgr2=0x7f54c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 --2- 172.21.15.77:0/2353307069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4100a80 0x7f54c4100ee0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009930 tx=0x7f54b802a420 comp rx=0 tx=0).stop 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 -- 172.21.15.77:0/2353307069 shutdown_connections 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 --2- 172.21.15.77:0/2353307069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54c4101420 0x7f54c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 --2- 172.21.15.77:0/2353307069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4100a80 0x7f54c4100ee0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 --2- 172.21.15.77:0/2353307069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4106a80 0x7f54c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 -- 172.21.15.77:0/2353307069 >> 172.21.15.77:0/2353307069 conn(0x7f54c40fc7a0 msgr2=0x7f54c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 -- 172.21.15.77:0/2353307069 shutdown_connections 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.371+0000 7f54c9118640 1 -- 172.21.15.77:0/2353307069 wait complete. 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 Processor -- start 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 -- start start 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 0x7f54c41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54c4106a80 0x7f54c419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4110e80 con 0x7f54c4101420 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4110d00 con 0x7f54c4100a80 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c9118640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4111000 con 0x7f54c4106a80 2024-09-15T22:35:13.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56130/0 (socket says 172.21.15.77:56130) 2024-09-15T22:35:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.372+0000 7f54c259c640 1 -- 172.21.15.77:0/3879122208 learned_addr learned my addr 172.21.15.77:0/3879122208 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 0x7f54c41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 -- 172.21.15.77:0/3879122208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54c4106a80 msgr2=0x7f54c419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54c4106a80 0x7f54c419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 -- 172.21.15.77:0/3879122208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 msgr2=0x7f54c41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 0x7f54c41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 -- 172.21.15.77:0/3879122208 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c419d390 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 0x7f54c41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c259c640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009230 tx=0x7f54b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54abfff640 1 -- 172.21.15.77:0/3879122208 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b803f030 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c419d620 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54abfff640 1 -- 172.21.15.77:0/3879122208 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8003ad0 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54abfff640 1 -- 172.21.15.77:0/3879122208 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8035070 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.373+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41a4f60 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54abfff640 1 -- 172.21.15.77:0/3879122208 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f54b8042030 con 0x7f54c4101420 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 Processor -- start 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 -- start start 2024-09-15T22:35:13.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e5b40 0x5616346e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56163464a2f0 con 0x5616346ef370 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5616345f1cf0 con 0x5616346e5b40 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616346462b0 con 0x5616346e45d0 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56144/0 (socket says 172.21.15.77:56144) 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c259c640 1 -- 172.21.15.77:0/1159587409 learned_addr learned my addr 172.21.15.77:0/1159587409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c359e640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e5b40 0x5616346e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54abfff640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 0x7f54ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.376+0000 7f54c359e640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 0x7f54ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 -- 172.21.15.77:0/1159587409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 msgr2=0x5616346e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 -- 172.21.15.77:0/1159587409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e5b40 msgr2=0x5616346e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e5b40 0x5616346e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 -- 172.21.15.77:0/1159587409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616345d63a0 con 0x5616346ef370 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54abfff640 1 -- 172.21.15.77:0/3879122208 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f54b80f06b0 con 0x7f54c4101420 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c259c640 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f54b80bc2a0 tx=0x7f54b804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc90835b7500998f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54c359e640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 0x7f54ac0a5870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009270 tx=0x7f54bc009200 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54a97fa640 1 -- 172.21.15.77:0/1159587409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8049040 con 0x5616346ef370 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54a97fa640 1 -- 172.21.15.77:0/1159587409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bcca0 con 0x5616346ef370 2024-09-15T22:35:13.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.377+0000 7f54a97fa640 1 -- 172.21.15.77:0/1159587409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80bc5a0 con 0x5616346ef370 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1159587409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 msgr2=0x5616346f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f54b80bc2a0 tx=0x7f54b804b000 comp rx=0 tx=0).stop 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1159587409 shutdown_connections 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346f1760 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1159587409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e5b40 0x5616346e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1159587409 >> 172.21.15.77:0/1159587409 conn(0x5616346ad310 msgr2=0x5616346e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1159587409 shutdown_connections 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1159587409 wait complete. 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- start start 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 Processor -- start 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e5b40 0x5616346ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5616346f1e30 con 0x5616346ef370 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561634605060 con 0x5616346e45d0 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.378+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5616346f1fb0 con 0x5616346e5b40 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54cb4f1740 10 client.0 _ll_register_callbacks cb 0x5616343588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56152/0 (socket says 172.21.15.77:56152) 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 -- 172.21.15.77:0/1028838441 learned_addr learned my addr 172.21.15.77:0/1028838441 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:13.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 -- 172.21.15.77:0/1028838441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e5b40 msgr2=0x5616346ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e5b40 0x5616346ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 -- 172.21.15.77:0/1028838441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 msgr2=0x5616346eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346eeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 -- 172.21.15.77:0/1028838441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616343bb3c0 con 0x5616346ef370 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54c359e640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003db0 tx=0x7f54bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c962d485c95f1e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01c050 con 0x5616346ef370 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f54bc0047b0 con 0x5616346ef370 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b80bcca0 con 0x5616346ef370 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f54bc02f050 con 0x5616346ef370 2024-09-15T22:35:13.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54cb4f1740 10 client.20634 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.379+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5616346fd860 con 0x5616346ef370 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bcca0 con 0x5616346ef370 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f54bc00f420 con 0x5616346ef370 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 10 client.20634 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 10 client.20634 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 20 client.20634 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5616346fed70 0x561634701200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 4 client.20634 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561634657860 con 0x5616346fed70 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5616346fed70 0x561634701200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54a6ff5640 10 client.20634 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.380+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5616346fed70 0x561634701200 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcf0bd0a8e56b94d server_cookie=25c0de089b5f31fb in_seq=0 out_seq=0 2024-09-15T22:35:13.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54a6ff5640 1 -- 172.21.15.77:0/1028838441 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561634657860 con 0x5616346fed70 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54a6ff5640 10 client.20634 handle_command_reply: tid=0 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 client.20634 shutdown 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 msgr2=0x5616346ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003db0 tx=0x7f54bc004a40 comp rx=0 tx=0).stop 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 shutdown_connections 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x5616346fed70 0x561634701200 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346ef370 0x5616346ec7e0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54a6ff5640 0 client.20634 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e5b40 0x5616346ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/1028838441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 >> 172.21.15.77:0/1028838441 conn(0x5616346ad310 msgr2=0x5616346e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 shutdown_connections 2024-09-15T22:35:13.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 1 -- 172.21.15.77:0/1028838441 wait complete. 2024-09-15T22:35:13.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.381+0000 7f54cb4f1740 20 client.20634 trim_cache size 0 max 16384 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 Processor -- start 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 -- start start 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346ef370 0x561634706fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41af0d0 con 0x5616346e5b40 2024-09-15T22:35:13.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41b0c10 con 0x5616346e45d0 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41af7a0 con 0x5616346ef370 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56154/0 (socket says 172.21.15.77:56154) 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 -- 172.21.15.77:0/2453258403 learned_addr learned my addr 172.21.15.77:0/2453258403 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c359e640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346ef370 0x561634706fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 -- 172.21.15.77:0/2453258403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346ef370 msgr2=0x561634706fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346ef370 0x561634706fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 -- 172.21.15.77:0/2453258403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 msgr2=0x5616346e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c259c640 1 -- 172.21.15.77:0/2453258403 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616346fb000 con 0x5616346e5b40 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.395+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54c259c640 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f54b80b8a00 tx=0x7f54b80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4570b5c73f9aae59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54a8ff9640 1 -- 172.21.15.77:0/2453258403 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8049070 con 0x5616346e5b40 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54a8ff9640 1 -- 172.21.15.77:0/2453258403 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bcca0 con 0x5616346e5b40 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54a8ff9640 1 -- 172.21.15.77:0/2453258403 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80edc90 con 0x5616346e5b40 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 -- 172.21.15.77:0/2453258403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 msgr2=0x561634706a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f54b80b8a00 tx=0x7f54b80045f0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 -- 172.21.15.77:0/2453258403 shutdown_connections 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346ef370 0x561634706fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x561634706a80 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.396+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/2453258403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346e45d0 0x5616346e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- 172.21.15.77:0/2453258403 >> 172.21.15.77:0/2453258403 conn(0x5616346ad310 msgr2=0x5616346e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- 172.21.15.77:0/2453258403 shutdown_connections 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- 172.21.15.77:0/2453258403 wait complete. 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- start start 2024-09-15T22:35:13.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 Processor -- start 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346ef370 0x561634768a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41b0540 con 0x5616346e5b40 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41afe70 con 0x5616346ef370 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41ad0d0 con 0x5616346e45d0 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54cb4f1740 10 client.0 _ll_register_callbacks cb 0x7f54bc009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54c259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56170/0 (socket says 172.21.15.77:56170) 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.397+0000 7f54c259c640 1 -- 172.21.15.77:0/3682567508 learned_addr learned my addr 172.21.15.77:0/3682567508 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c359e640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346ef370 0x561634768a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 -- 172.21.15.77:0/3682567508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 msgr2=0x5616346f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 -- 172.21.15.77:0/3682567508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346ef370 msgr2=0x561634768a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346ef370 0x561634768a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 -- 172.21.15.77:0/3682567508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616346f4150 con 0x5616346e5b40 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54c259c640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f54b80c5180 tx=0x7f54b80b8220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae29699d09069e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8049050 con 0x5616346e5b40 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f54b80b8c10 con 0x5616346e5b40 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b80bcca0 con 0x5616346e5b40 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f54b80b8840 con 0x5616346e5b40 2024-09-15T22:35:13.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54cb4f1740 10 client.20640 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56163476a030 con 0x5616346e5b40 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bcca0 con 0x5616346e5b40 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.398+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f54b8031bd0 con 0x5616346e5b40 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 10 client.20640 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 10 client.20640 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 20 client.20640 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56163476a3c0 0x56163476c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 4 client.20640 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:35:13.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561634651d70 con 0x56163476a3c0 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56163476a3c0 0x56163476c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54a6ff5640 10 client.20640 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54c2d9d640 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56163476a3c0 0x56163476c7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4789adf22204be4 server_cookie=e4982b35f21cdc3a in_seq=0 out_seq=0 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54a6ff5640 1 -- 172.21.15.77:0/3682567508 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561634651d70 con 0x56163476a3c0 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54a6ff5640 10 client.20640 handle_command_reply: tid=0 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 client.20640 shutdown 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 msgr2=0x5616346f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f54b80c5180 tx=0x7f54b80b8220 comp rx=0 tx=0).stop 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 shutdown_connections 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x56163476a3c0 0x56163476c7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5616346ef370 0x561634768a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54a6ff5640 0 client.20640 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5616346e5b40 0x5616346f3850 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 --2- 172.21.15.77:0/3682567508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5616346e45d0 0x5616346f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.399+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 >> 172.21.15.77:0/3682567508 conn(0x5616346ad310 msgr2=0x5616346fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 shutdown_connections 2024-09-15T22:35:13.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54cb4f1740 1 -- 172.21.15.77:0/3682567508 wait complete. 2024-09-15T22:35:13.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54cb4f1740 20 client.20640 trim_cache size 0 max 16384 2024-09-15T22:35:13.416 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 msgr2=0x7f54ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 0x7f54ac0a5870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009270 tx=0x7f54bc009200 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 msgr2=0x7f54c4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009230 tx=0x7f54b8004270 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 shutdown_connections 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f54ac0a33b0 0x7f54ac0a5870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54c4106a80 0x7f54c419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c4101420 0x7f54c4198920 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 --2- 172.21.15.77:0/3879122208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c4100a80 0x7f54c41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 >> 172.21.15.77:0/3879122208 conn(0x7f54c40fc7a0 msgr2=0x7f54c406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 shutdown_connections 2024-09-15T22:35:13.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:13.400+0000 7f54c9118640 1 -- 172.21.15.77:0/3879122208 wait complete. 2024-09-15T22:35:13.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.54 MB/sec execute 434 sec latency 28032.901 ms 2024-09-15T22:35:14.150 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:35:14.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:14 smithi077 ceph-mon[30980]: pgmap v720: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:14 smithi196 ceph-mon[35109]: pgmap v720: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:14 smithi117 ceph-mon[35021]: pgmap v720: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:14.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.51 MB/sec execute 435 sec latency 29032.951 ms 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 -- 172.21.15.77:0/763167881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b01027a0 msgr2=0x7f79b01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/763167881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b01027a0 0x7f79b01067f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009900 tx=0x7f79ac02a410 comp rx=0 tx=0).stop 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 -- 172.21.15.77:0/763167881 shutdown_connections 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/763167881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 0x7f79b010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/763167881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b01027a0 0x7f79b01067f0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/763167881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b0101df0 0x7f79b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 -- 172.21.15.77:0/763167881 >> 172.21.15.77:0/763167881 conn(0x7f79b00fd8b0 msgr2=0x7f79b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 -- 172.21.15.77:0/763167881 shutdown_connections 2024-09-15T22:35:14.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.872+0000 7f79b7c5a640 1 -- 172.21.15.77:0/763167881 wait complete. 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 Processor -- start 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 -- start start 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b01027a0 0x7f79b019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 0x7f79b019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0111f80 con 0x7f79b0101df0 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0111e00 con 0x7f79b0106f20 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b7c5a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0112100 con 0x7f79b01027a0 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b59f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b59f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56198/0 (socket says 172.21.15.77:56198) 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b59f6640 1 -- 172.21.15.77:0/71793103 learned_addr learned my addr 172.21.15.77:0/71793103 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b61f7640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 0x7f79b019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b59f6640 1 -- 172.21.15.77:0/71793103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b01027a0 msgr2=0x7f79b019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.873+0000 7f79b51f5640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b01027a0 0x7f79b019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b59f6640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b01027a0 0x7f79b019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b59f6640 1 -- 172.21.15.77:0/71793103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 msgr2=0x7f79b019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b59f6640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 0x7f79b019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b59f6640 1 -- 172.21.15.77:0/71793103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b019f5d0 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b59f6640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79a2ffd640 1 -- 172.21.15.77:0/71793103 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4017070 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b019f8c0 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79a2ffd640 1 -- 172.21.15.77:0/71793103 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a4005500 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79a2ffd640 1 -- 172.21.15.77:0/71793103 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4004e00 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.874+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b01a71a0 con 0x7f79b0101df0 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 Processor -- start 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 -- start start 2024-09-15T22:35:14.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79a2ffd640 1 -- 172.21.15.77:0/71793103 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f79a4004fa0 con 0x7f79b0101df0 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d39ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56005d3022f0 con 0x56005d39db40 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56005d2a9cf0 con 0x56005d39c5d0 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79ba033740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56005d2fe2b0 con 0x56005d3a7370 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.877+0000 7f79b61f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56210/0 (socket says 172.21.15.77:56210) 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 -- 172.21.15.77:0/1490937146 learned_addr learned my addr 172.21.15.77:0/1490937146 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79a2ffd640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 0x7f799c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 -- 172.21.15.77:0/1490937146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 msgr2=0x56005d3a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 -- 172.21.15.77:0/1490937146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 msgr2=0x56005d39ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d39ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d39ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 -- 172.21.15.77:0/1490937146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005d28e3a0 con 0x56005d39db40 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d39ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79a2ffd640 1 -- 172.21.15.77:0/71793103 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f79a40cfbf0 con 0x7f79b0101df0 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 0x7f799c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f70 tx=0x7f79a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418aa56664b84b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f799bfff640 1 -- 172.21.15.77:0/1490937146 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8022070 con 0x56005d39db40 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f799bfff640 1 -- 172.21.15.77:0/1490937146 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a801ce30 con 0x56005d39db40 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.878+0000 7f799bfff640 1 -- 172.21.15.77:0/1490937146 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8016070 con 0x56005d39db40 2024-09-15T22:35:14.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 -- 172.21.15.77:0/1490937146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 msgr2=0x56005d39c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79b61f7640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 0x7f799c0a5870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f79a801f000 tx=0x7f79a8022000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f70 tx=0x7f79a8010800 comp rx=0 tx=0).stop 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 -- 172.21.15.77:0/1490937146 shutdown_connections 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d39ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 --2- 172.21.15.77:0/1490937146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d39c090 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 -- 172.21.15.77:0/1490937146 >> 172.21.15.77:0/1490937146 conn(0x56005d365310 msgr2=0x56005d398c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 -- 172.21.15.77:0/1490937146 shutdown_connections 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.879+0000 7f79ba033740 1 -- 172.21.15.77:0/1490937146 wait complete. 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 -- start start 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 Processor -- start 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 0x56005d3a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56005d3aa070 con 0x56005d39c5d0 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56005d2bd060 con 0x56005d39db40 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56005d3aa1f0 con 0x56005d3a7370 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79ba033740 10 client.0 _ll_register_callbacks cb 0x56005d0108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79b59f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79b59f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56220/0 (socket says 172.21.15.77:56220) 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79b59f6640 1 -- 172.21.15.77:0/1220215425 learned_addr learned my addr 172.21.15.77:0/1220215425 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 0x56005d3a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.880+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 -- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 msgr2=0x56005d3a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 -- 172.21.15.77:0/1220215425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 msgr2=0x56005d3a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 0x56005d3a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 -- 172.21.15.77:0/1220215425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005d0733c0 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 0x56005d3a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f79a409ba10 tx=0x7f79a409b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f393f9372169578 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4020040 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f79a40860b0 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a801ce30 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79a409c180 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 10 client.20652 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56005d3b5a10 con 0x56005d39c5d0 2024-09-15T22:35:14.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a801ce30 con 0x56005d39c5d0 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f79a40980f0 con 0x56005d39c5d0 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 10 client.20652 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 10 client.20652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 20 client.20652 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.881+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d3b6e20 0x56005d3b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.882+0000 7f79ba033740 4 client.20652 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.882+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005d30f860 con 0x56005d3b6e20 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.882+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d3b6e20 0x56005d3b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.882+0000 7f79997fa640 10 client.20652 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.882+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d3b6e20 0x56005d3b9250 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fc2ac452c6e735b server_cookie=aa099dfc12a25a8 in_seq=0 out_seq=0 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79997fa640 1 -- 172.21.15.77:0/1220215425 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56005d30f860 con 0x56005d3b6e20 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79997fa640 10 client.20652 handle_command_reply: tid=0 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 client.20652 shutdown 2024-09-15T22:35:14.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 msgr2=0x56005d3a6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f79a409ba10 tx=0x7f79a409b3e0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 shutdown_connections 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d3b6e20 0x56005d3b9250 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d3a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39db40 0x56005d3a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 --2- 172.21.15.77:0/1220215425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39c5d0 0x56005d3a6d00 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79997fa640 0 client.20652 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 >> 172.21.15.77:0/1220215425 conn(0x56005d365310 msgr2=0x56005d39e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 shutdown_connections 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 1 -- 172.21.15.77:0/1220215425 wait complete. 2024-09-15T22:35:14.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.883+0000 7f79ba033740 20 client.20652 trim_cache size 0 max 16384 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 Processor -- start 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 -- start start 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d398e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d39f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01b1320 con 0x56005d39db40 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01b2e60 con 0x56005d39c5d0 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79ba033740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01b19f0 con 0x56005d3a7370 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79b51f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79b51f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56222/0 (socket says 172.21.15.77:56222) 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.896+0000 7f79b51f5640 1 -- 172.21.15.77:0/1714596438 learned_addr learned my addr 172.21.15.77:0/1714596438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b61f7640 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d39f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b59f6640 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d398e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 -- 172.21.15.77:0/1714596438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 msgr2=0x56005d39f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d39f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 -- 172.21.15.77:0/1714596438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 msgr2=0x56005d398e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d398e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 -- 172.21.15.77:0/1714596438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005d3a48a0 con 0x56005d39db40 2024-09-15T22:35:14.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f79b51f5640 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f79ac005ab0 tx=0x7f79ac004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ebe8944e3906d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f799b7fe640 1 -- 172.21.15.77:0/1714596438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac049070 con 0x56005d39db40 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f799b7fe640 1 -- 172.21.15.77:0/1714596438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ac005520 con 0x56005d39db40 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.897+0000 7f799b7fe640 1 -- 172.21.15.77:0/1714596438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac035040 con 0x56005d39db40 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- 172.21.15.77:0/1714596438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 msgr2=0x56005d3b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f79ac005ab0 tx=0x7f79ac004310 comp rx=0 tx=0).stop 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- 172.21.15.77:0/1714596438 shutdown_connections 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d3a7370 0x56005d39f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d39db40 0x56005d3b3310 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 --2- 172.21.15.77:0/1714596438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d398e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- 172.21.15.77:0/1714596438 >> 172.21.15.77:0/1714596438 conn(0x56005d365310 msgr2=0x56005d398700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- 172.21.15.77:0/1714596438 shutdown_connections 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- 172.21.15.77:0/1714596438 wait complete. 2024-09-15T22:35:14.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 -- start start 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.898+0000 7f79ba033740 1 Processor -- start 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d3afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 0x56005d3aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01b2790 con 0x56005d3a7370 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01b20c0 con 0x56005d39c5d0 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01af320 con 0x56005d39db40 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 10 client.0 _ll_register_callbacks cb 0x7f79a40056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56238/0 (socket says 172.21.15.77:56238) 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 -- 172.21.15.77:0/3035238552 learned_addr learned my addr 172.21.15.77:0/3035238552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:14.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b59f6640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d3afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b51f5640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 0x56005d3aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 -- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 msgr2=0x56005d3aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 0x56005d3aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 -- 172.21.15.77:0/3035238552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 msgr2=0x56005d3afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d3afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 -- 172.21.15.77:0/3035238552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005d3abc00 con 0x56005d3a7370 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b59f6640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d3afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b51f5640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 0x56005d3aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79b61f7640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f70 tx=0x7f79a8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f59ff275dab051 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79997fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8035060 con 0x56005d3a7370 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f79a8003b80 con 0x56005d3a7370 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.899+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ac005520 con 0x56005d3a7370 2024-09-15T22:35:14.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79a803c050 con 0x56005d3a7370 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 10 client.20658 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56005d421b10 con 0x56005d3a7370 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ac005520 con 0x56005d3a7370 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f79a800f040 con 0x56005d3a7370 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 10 client.20658 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 10 client.20658 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 20 client.20658 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d421f80 0x56005d4243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 4 client.20658 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56005d309d70 con 0x56005d421f80 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.900+0000 7f79b59f6640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d421f80 0x56005d4243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79997fa640 10 client.20658 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:35:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79b59f6640 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d421f80 0x56005d4243c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68437e10ad09260 server_cookie=30aca23c68ec9037 in_seq=0 out_seq=0 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79997fa640 1 -- 172.21.15.77:0/3035238552 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56005d309d70 con 0x56005d421f80 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79997fa640 10 client.20658 handle_command_reply: tid=0 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 client.20658 shutdown 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 msgr2=0x56005d3ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009f70 tx=0x7f79a8009ea0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 shutdown_connections 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56005d421f80 0x56005d4243c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56005d3a7370 0x56005d3ab400 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56005d39db40 0x56005d3aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 --2- 172.21.15.77:0/3035238552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56005d39c5d0 0x56005d3afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79997fa640 0 client.20658 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:35:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 >> 172.21.15.77:0/3035238552 conn(0x56005d365310 msgr2=0x56005d397310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 shutdown_connections 2024-09-15T22:35:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 1 -- 172.21.15.77:0/3035238552 wait complete. 2024-09-15T22:35:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.901+0000 7f79ba033740 20 client.20658 trim_cache size 0 max 16384 2024-09-15T22:35:14.914 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 msgr2=0x7f799c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 0x7f799c0a5870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f79a801f000 tx=0x7f79a8022000 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 msgr2=0x7f79b019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 shutdown_connections 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f799c0a33b0 0x7f799c0a5870 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79b0106f20 0x7f79b019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79b01027a0 0x7f79b019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 --2- 172.21.15.77:0/71793103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b0101df0 0x7f79b019a620 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 >> 172.21.15.77:0/71793103 conn(0x7f79b00fd8b0 msgr2=0x7f79b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 shutdown_connections 2024-09-15T22:35:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:14.902+0000 7f79b7c5a640 1 -- 172.21.15.77:0/71793103 wait complete. 2024-09-15T22:35:15.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.47 MB/sec execute 436 sec latency 30032.980 ms 2024-09-15T22:35:15.635 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:35:15.880 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:16.310 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:16 smithi077 ceph-mon[30980]: pgmap v721: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 -- 172.21.15.77:0/3376148176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18501036f0 msgr2=0x7f185010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 --2- 172.21.15.77:0/3376148176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18501036f0 0x7f185010bbe0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1840009900 tx=0x7f184002a480 comp rx=0 tx=0).stop 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 -- 172.21.15.77:0/3376148176 shutdown_connections 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 --2- 172.21.15.77:0/3376148176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185010c1b0 0x7f185010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 --2- 172.21.15.77:0/3376148176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18501036f0 0x7f185010bbe0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 --2- 172.21.15.77:0/3376148176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1850102dd0 0x7f18501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 -- 172.21.15.77:0/3376148176 >> 172.21.15.77:0/3376148176 conn(0x7f18500fc7d0 msgr2=0x7f18500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 -- 172.21.15.77:0/3376148176 shutdown_connections 2024-09-15T22:35:16.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.392+0000 7f1857b62640 1 -- 172.21.15.77:0/3376148176 wait complete. 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 Processor -- start 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 -- start start 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 0x7f1850100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 0x7f1850100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1850111fc0 con 0x7f185010c1b0 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1850111e40 con 0x7f18501036f0 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f1857b62640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1850112140 con 0x7f1850102dd0 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56260/0 (socket says 172.21.15.77:56260) 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 -- 172.21.15.77:0/198177120 learned_addr learned my addr 172.21.15.77:0/198177120 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18550fd640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 0x7f1850100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18558fe640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 0x7f1850100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 -- 172.21.15.77:0/198177120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 msgr2=0x7f1850100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 0x7f1850100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.393+0000 7f18560ff640 1 -- 172.21.15.77:0/198177120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 msgr2=0x7f1850100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f18560ff640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 0x7f1850100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f18560ff640 1 -- 172.21.15.77:0/198177120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501a6a00 con 0x7f185010c1b0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f18558fe640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 0x7f1850100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f18550fd640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 0x7f1850100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f18560ff640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f184800cad0 tx=0x7f184800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f1846ffd640 1 -- 172.21.15.77:0/198177120 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800ae20 con 0x7f185010c1b0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a6cf0 con 0x7f185010c1b0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f1846ffd640 1 -- 172.21.15.77:0/198177120 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848005020 con 0x7f185010c1b0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f1846ffd640 1 -- 172.21.15.77:0/198177120 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800f6f0 con 0x7f185010c1b0 2024-09-15T22:35:16.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.394+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a71e0 con 0x7f185010c1b0 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.396+0000 7f1846ffd640 1 -- 172.21.15.77:0/198177120 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f18480051d0 con 0x7f185010c1b0 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.396+0000 7f1859f3b740 1 Processor -- start 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 -- start start 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60252d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c602538370 0x55c60253a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6024932f0 con 0x55c60252d5d0 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c60243acf0 con 0x55c602538370 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1859f3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c60248f2b0 con 0x55c60252eb40 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56276/0 (socket says 172.21.15.77:56276) 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 -- 172.21.15.77:0/3634651062 learned_addr learned my addr 172.21.15.77:0/3634651062 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:16.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18550fd640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c602538370 0x55c60253a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18550fd640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60252d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 -- 172.21.15.77:0/3634651062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 msgr2=0x55c60252d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60252d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f1846ffd640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 0x7f183c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 -- 172.21.15.77:0/3634651062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c602538370 msgr2=0x55c60253a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c602538370 0x55c60253a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18558fe640 1 -- 172.21.15.77:0/3634651062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60241f3a0 con 0x55c60252d5d0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.397+0000 7f18550fd640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60252d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f18560ff640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 0x7f183c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f18558fe640 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f184c00aa10 tx=0x7f184c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f0214a4c9d694 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f1846ffd640 1 -- 172.21.15.77:0/198177120 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f18480d02d0 con 0x7f185010c1b0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f183bfff640 1 -- 172.21.15.77:0/3634651062 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c015070 con 0x55c60252d5d0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f183bfff640 1 -- 172.21.15.77:0/3634651062 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c00cc60 con 0x55c60252d5d0 2024-09-15T22:35:16.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.398+0000 7f183bfff640 1 -- 172.21.15.77:0/3634651062 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c018690 con 0x55c60252d5d0 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- 172.21.15.77:0/3634651062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 msgr2=0x55c60252da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f184c00aa10 tx=0x7f184c00aed0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f18560ff640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 0x7f183c0a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1848061e00 tx=0x7f18480bc040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- 172.21.15.77:0/3634651062 shutdown_connections 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c602538370 0x55c60253a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252d5d0 0x55c60252da10 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3634651062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60252d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- 172.21.15.77:0/3634651062 >> 172.21.15.77:0/3634651062 conn(0x55c6024f6310 msgr2=0x55c602529c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- 172.21.15.77:0/3634651062 shutdown_connections 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- 172.21.15.77:0/3634651062 wait complete. 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 -- start start 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.399+0000 7f1859f3b740 1 Processor -- start 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c602537d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c602534fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c60253b070 con 0x55c602538370 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c60244e060 con 0x55c60252d5d0 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c60253b1f0 con 0x55c60252eb40 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 10 client.0 _ll_register_callbacks cb 0x55c6021a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56290/0 (socket says 172.21.15.77:56290) 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 -- 172.21.15.77:0/1244885913 learned_addr learned my addr 172.21.15.77:0/1244885913 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c602537d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18558fe640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c602534fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 -- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 msgr2=0x55c602534fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c602534fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 -- 172.21.15.77:0/1244885913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 msgr2=0x55c602537d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c602537d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 -- 172.21.15.77:0/1244885913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6022043c0 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18558fe640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c602534fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18550fd640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1840005ec0 tx=0x7f184002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfd3c31c10470a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840035040 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f1840003ac0 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184c00cc60 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1840003cc0 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 10 client.20670 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.400+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c602546a30 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c00cc60 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f184003a040 con 0x55c602538370 2024-09-15T22:35:16.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 10 client.20670 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 10 client.20670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 20 client.20670 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602547ef0 0x55c60254a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 4 client.20670 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6024a0860 con 0x55c602547ef0 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.401+0000 7f18558fe640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602547ef0 0x55c60254a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f18558fe640 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602547ef0 0x55c60254a2e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a1df067fead6490 server_cookie=90f432785c6ca090 in_seq=0 out_seq=0 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f18397fa640 10 client.20670 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f18397fa640 1 -- 172.21.15.77:0/1244885913 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6024a0860 con 0x55c602547ef0 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f18397fa640 10 client.20670 handle_command_reply: tid=0 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f1859f3b740 1 client.20670 shutdown 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 msgr2=0x55c602535560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.402+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1840005ec0 tx=0x7f184002a480 comp rx=0 tx=0).stop 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 shutdown_connections 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602547ef0 0x55c60254a2e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c602538370 0x55c602535560 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c602534fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1244885913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c602537d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 >> 172.21.15.77:0/1244885913 conn(0x55c6024f6310 msgr2=0x55c60252f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f18397fa640 0 client.20670 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 shutdown_connections 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 1 -- 172.21.15.77:0/1244885913 wait complete. 2024-09-15T22:35:16.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.403+0000 7f1859f3b740 20 client.20670 trim_cache size 0 max 16384 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 Processor -- start 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 -- start start 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6024f6310 0x55c602544310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252d5d0 0x55c60253a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b1400 con 0x55c60252eb40 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b2f40 con 0x55c6024f6310 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f1859f3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b1ad0 con 0x55c60252d5d0 2024-09-15T22:35:16.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56306/0 (socket says 172.21.15.77:56306) 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 -- 172.21.15.77:0/3705778245 learned_addr learned my addr 172.21.15.77:0/3705778245 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18558fe640 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252d5d0 0x55c60253a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6024f6310 0x55c602544310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 -- 172.21.15.77:0/3705778245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252d5d0 msgr2=0x55c60253a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252d5d0 0x55c60253a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 -- 172.21.15.77:0/3705778245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6024f6310 msgr2=0x55c602544310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6024f6310 0x55c602544310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 -- 172.21.15.77:0/3705778245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c602533510 con 0x55c60252eb40 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.416+0000 7f18550fd640 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1840001eb0 tx=0x7f1840040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd5da6f2e99a9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f183b7fe640 1 -- 172.21.15.77:0/3705778245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840035040 con 0x55c60252eb40 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f183b7fe640 1 -- 172.21.15.77:0/3705778245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1840003e60 con 0x55c60252eb40 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f183b7fe640 1 -- 172.21.15.77:0/3705778245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184003d480 con 0x55c60252eb40 2024-09-15T22:35:16.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 -- 172.21.15.77:0/3705778245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 msgr2=0x55c602539760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1840001eb0 tx=0x7f1840040040 comp rx=0 tx=0).stop 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 -- 172.21.15.77:0/3705778245 shutdown_connections 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c60252eb40 0x55c602539760 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252d5d0 0x55c60253a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 --2- 172.21.15.77:0/3705778245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6024f6310 0x55c602544310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 -- 172.21.15.77:0/3705778245 >> 172.21.15.77:0/3705778245 conn(0x55c6025309d0 msgr2=0x55c602533890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 -- 172.21.15.77:0/3705778245 shutdown_connections 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.417+0000 7f1859f3b740 1 -- 172.21.15.77:0/3705778245 wait complete. 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 -- start start 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 Processor -- start 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c6025409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60253b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b2870 con 0x55c6024f6310 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18501b21a0 con 0x55c60252d5d0 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18501af400 con 0x55c60252eb40 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f1859f3b740 10 client.0 _ll_register_callbacks cb 0x7f1840009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f18550fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56312/0 (socket says 172.21.15.77:56312) 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f18550fd640 1 -- 172.21.15.77:0/1137637662 learned_addr learned my addr 172.21.15.77:0/1137637662 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.418+0000 7f18550fd640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60253b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18558fe640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c6025409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 -- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 msgr2=0x55c60253b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60253b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 -- 172.21.15.77:0/1137637662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 msgr2=0x55c6025409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:16.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c6025409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 -- 172.21.15.77:0/1137637662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c602541110 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18550fd640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1840009850 tx=0x7f1840004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bccf934ccc73b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840035040 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f1840004d30 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840003e60 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1840004f30 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f1859f3b740 10 client.20676 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c6025b31c0 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1840003e60 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.419+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1840055070 con 0x55c6024f6310 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 10 client.20676 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 10 client.20676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:35:16.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 20 client.20676 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602538370 0x55c6025b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 4 client.20676 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c60253ab40 con 0x55c602538370 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f18558fe640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602538370 0x55c6025b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f18397fa640 10 client.20676 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f18558fe640 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602538370 0x55c6025b57e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47acb38c17cdda96 server_cookie=751541d438d68545 in_seq=0 out_seq=0 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f18397fa640 1 -- 172.21.15.77:0/1137637662 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c60253ab40 con 0x55c602538370 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f18397fa640 10 client.20676 handle_command_reply: tid=0 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.420+0000 7f1859f3b740 1 client.20676 shutdown 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 msgr2=0x55c602540490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1840009850 tx=0x7f1840004280 comp rx=0 tx=0).stop 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 shutdown_connections 2024-09-15T22:35:16.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55c602538370 0x55c6025b57e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c60252eb40 0x55c60253b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f18397fa640 0 client.20676 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c60252d5d0 0x55c6025409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 --2- 172.21.15.77:0/1137637662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6024f6310 0x55c602540490 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 >> 172.21.15.77:0/1137637662 conn(0x55c6025309d0 msgr2=0x55c6025284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 shutdown_connections 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 1 -- 172.21.15.77:0/1137637662 wait complete. 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1859f3b740 20 client.20676 trim_cache size 0 max 16384 2024-09-15T22:35:16.431 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:35:16.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 msgr2=0x7f183c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 0x7f183c0a5870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1848061e00 tx=0x7f18480bc040 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 msgr2=0x7f18501014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f184800cad0 tx=0x7f184800cf90 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 shutdown_connections 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.421+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f183c0a33b0 0x7f183c0a5870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f185010c1b0 0x7f18501014b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18501036f0 0x7f1850100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 --2- 172.21.15.77:0/198177120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1850102dd0 0x7f1850100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 >> 172.21.15.77:0/198177120 conn(0x7f18500fc7d0 msgr2=0x7f18500fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 shutdown_connections 2024-09-15T22:35:16.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:16.422+0000 7f1857b62640 1 -- 172.21.15.77:0/198177120 wait complete. 2024-09-15T22:35:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:16 smithi196 ceph-mon[35109]: pgmap v721: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:16 smithi117 ceph-mon[35021]: pgmap v721: 129 pgs: 129 active+clean; 1.8 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 4.3 MiB/s wr, 115 op/s 2024-09-15T22:35:16.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.44 MB/sec execute 437 sec latency 31033.013 ms 2024-09-15T22:35:16.824 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:17.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:35:17.410 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:17.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.41 MB/sec execute 438 sec latency 32033.100 ms 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 -- 172.21.15.77:0/3209714016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 msgr2=0x7f1a341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 --2- 172.21.15.77:0/3209714016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a341029b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c02a480 comp rx=0 tx=0).stop 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 -- 172.21.15.77:0/3209714016 shutdown_connections 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 --2- 172.21.15.77:0/3209714016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34102ef0 0x7f1a3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 --2- 172.21.15.77:0/3209714016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a341029b0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 --2- 172.21.15.77:0/3209714016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34100e20 0x7f1a34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.883+0000 7f1a39b35640 1 -- 172.21.15.77:0/3209714016 >> 172.21.15.77:0/3209714016 conn(0x7f1a340fc7d0 msgr2=0x7f1a340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:17.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- 172.21.15.77:0/3209714016 shutdown_connections 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- 172.21.15.77:0/3209714016 wait complete. 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 Processor -- start 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- start start 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34100e20 0x7f1a341986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34111020 con 0x7f1a34102570 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34110ea0 con 0x7f1a34102ef0 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.884+0000 7f1a39b35640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341111a0 con 0x7f1a34100e20 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56338/0 (socket says 172.21.15.77:56338) 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 -- 172.21.15.77:0/4132524726 learned_addr learned my addr 172.21.15.77:0/4132524726 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:17.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a337fe640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34100e20 0x7f1a341986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a33fff640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 -- 172.21.15.77:0/4132524726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34100e20 msgr2=0x7f1a341986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34100e20 0x7f1a341986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 -- 172.21.15.77:0/4132524726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 msgr2=0x7f1a3419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 -- 172.21.15.77:0/4132524726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3419d6a0 con 0x7f1a34102570 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a33fff640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a32ffd640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c006010 tx=0x7f1a1c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c03f070 con 0x7f1a34102570 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3419d930 con 0x7f1a34102570 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c031e30 con 0x7f1a34102570 2024-09-15T22:35:17.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c035030 con 0x7f1a34102570 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.885+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a341a5270 con 0x7f1a34102570 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.887+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34101280 con 0x7f1a34102570 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.888+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1a1c042070 con 0x7f1a34102570 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.889+0000 7f1a30ff9640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 0x7f1a200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.889+0000 7f1a337fe640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 0x7f1a200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.889+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7f1a1c0f1050 con 0x7f1a34102570 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.889+0000 7f1a337fe640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 0x7f1a200a5870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400a400 tx=0x7f1a2400d040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:17.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:17.892+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a1c0bd6d0 con 0x7f1a34102570 2024-09-15T22:35:17.978 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:18.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.104+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a3410a0a0 con 0x7f1a34102570 2024-09-15T22:35:18.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.104+0000 7f1a30ff9640 1 -- 172.21.15.77:0/4132524726 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f1a1c0bcfd0 con 0x7f1a34102570 2024-09-15T22:35:18.106 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:35:18.106 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:35:18.106 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 msgr2=0x7f1a200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 0x7f1a200a5870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400a400 tx=0x7f1a2400d040 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 msgr2=0x7f1a34198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c006010 tx=0x7f1a1c003c80 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 shutdown_connections 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f1a200a33b0 0x7f1a200a5870 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a34102ef0 0x7f1a3419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a34102570 0x7f1a34198c30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 --2- 172.21.15.77:0/4132524726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a34100e20 0x7f1a341986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.106+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 >> 172.21.15.77:0/4132524726 conn(0x7f1a340fc7d0 msgr2=0x7f1a3406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.107+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 shutdown_connections 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.107+0000 7f1a39b35640 1 -- 172.21.15.77:0/4132524726 wait complete. 2024-09-15T22:35:18.569 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:18 smithi077 ceph-mon[30980]: pgmap v722: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:18.569 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:18 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4132524726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:18 smithi196 ceph-mon[35109]: pgmap v722: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:18 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4132524726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:18 smithi117 ceph-mon[35021]: pgmap v722: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:18 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4132524726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:18.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.38 MB/sec execute 439 sec latency 33033.141 ms 2024-09-15T22:35:18.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 -- 172.21.15.77:0/813328747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 msgr2=0x7fb0601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 --2- 172.21.15.77:0/813328747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb0601067e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb048009900 tx=0x7fb04802a410 comp rx=0 tx=0).stop 2024-09-15T22:35:18.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 -- 172.21.15.77:0/813328747 shutdown_connections 2024-09-15T22:35:18.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 --2- 172.21.15.77:0/813328747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 0x7fb06010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 --2- 172.21.15.77:0/813328747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb0601067e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.681 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 --2- 172.21.15.77:0/813328747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 0x7fb0601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 -- 172.21.15.77:0/813328747 >> 172.21.15.77:0/813328747 conn(0x7fb0600fd8c0 msgr2=0x7fb0600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 -- 172.21.15.77:0/813328747 shutdown_connections 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.649+0000 7fb0660b1640 1 -- 172.21.15.77:0/813328747 wait complete. 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 Processor -- start 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 -- start start 2024-09-15T22:35:18.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 0x7fb060198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 0x7fb06019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111f30 con 0x7fb060102790 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111db0 con 0x7fb060106f10 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb0660b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601120b0 con 0x7fb060101de0 2024-09-15T22:35:18.683 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56354/0 (socket says 172.21.15.77:56354) 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 -- 172.21.15.77:0/272665494 learned_addr learned my addr 172.21.15.77:0/272665494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05ffff640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 0x7fb06019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 -- 172.21.15.77:0/272665494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 msgr2=0x7fb060198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 0x7fb060198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 0x7fb060198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 -- 172.21.15.77:0/272665494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 msgr2=0x7fb06019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.684 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 0x7fb06019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.650+0000 7fb05effd640 1 -- 172.21.15.77:0/272665494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019d3c0 con 0x7fb060102790 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb05effd640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb048005ab0 tx=0x7fb0480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb05cff9640 1 -- 172.21.15.77:0/272665494 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04803f030 con 0x7fb060102790 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb05cff9640 1 -- 172.21.15.77:0/272665494 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb048003ce0 con 0x7fb060102790 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb05cff9640 1 -- 172.21.15.77:0/272665494 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048035070 con 0x7fb060102790 2024-09-15T22:35:18.685 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06019d650 con 0x7fb060102790 2024-09-15T22:35:18.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.651+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a4f90 con 0x7fb060102790 2024-09-15T22:35:18.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.653+0000 7fb05cff9640 1 -- 172.21.15.77:0/272665494 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb048042030 con 0x7fb060102790 2024-09-15T22:35:18.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.653+0000 7fb06848a740 1 Processor -- start 2024-09-15T22:35:18.686 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.653+0000 7fb06848a740 1 -- start start 2024-09-15T22:35:18.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ac915432f0 con 0x55ac915dd5d0 2024-09-15T22:35:18.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac914eacf0 con 0x55ac915deb40 2024-09-15T22:35:18.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb06848a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac9153f2b0 con 0x55ac915e8370 2024-09-15T22:35:18.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56368/0 (socket says 172.21.15.77:56368) 2024-09-15T22:35:18.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2026131602 learned_addr learned my addr 172.21.15.77:0/2026131602 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:18.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05ffff640 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05effd640 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2026131602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 msgr2=0x55ac915ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2026131602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 msgr2=0x55ac915dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2026131602 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac914cf3a0 con 0x55ac915dd5d0 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05cff9640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 0x7fb04c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb05000bb10 tx=0x7fb05000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e186a40f16e2d2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05ffff640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 0x7fb04c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.654+0000 7fb05cff9640 1 -- 172.21.15.77:0/272665494 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7fb0480f0b10 con 0x7fb060102790 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb055ffb640 1 -- 172.21.15.77:0/2026131602 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050017420 con 0x55ac915dd5d0 2024-09-15T22:35:18.690 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb055ffb640 1 -- 172.21.15.77:0/2026131602 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050017aa0 con 0x55ac915dd5d0 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb055ffb640 1 -- 172.21.15.77:0/2026131602 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000fb20 con 0x55ac915dd5d0 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 -- 172.21.15.77:0/2026131602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 msgr2=0x55ac915dda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb05ffff640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 0x7fb04c0a5870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb058008dc0 tx=0x7fb058008570 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb05000bb10 tx=0x7fb05000d340 comp rx=0 tx=0).stop 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 -- 172.21.15.77:0/2026131602 shutdown_connections 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915dda10 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.691 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 --2- 172.21.15.77:0/2026131602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 -- 172.21.15.77:0/2026131602 >> 172.21.15.77:0/2026131602 conn(0x55ac915a6310 msgr2=0x55ac915d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 -- 172.21.15.77:0/2026131602 shutdown_connections 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.655+0000 7fb06848a740 1 -- 172.21.15.77:0/2026131602 wait complete. 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 -- start start 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 Processor -- start 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 0x55ac915e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ac915eb070 con 0x55ac915deb40 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ac914fe060 con 0x55ac915dd5d0 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ac915eb1f0 con 0x55ac915e8370 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb06848a740 10 client.0 _ll_register_callbacks cb 0x55ac912518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb05effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb05effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56380/0 (socket says 172.21.15.77:56380) 2024-09-15T22:35:18.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.656+0000 7fb05effd640 1 -- 172.21.15.77:0/73936439 learned_addr learned my addr 172.21.15.77:0/73936439 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05ffff640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 0x55ac915e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 -- 172.21.15.77:0/73936439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 msgr2=0x55ac915e5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 -- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 msgr2=0x55ac915e7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 0x55ac915e7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 -- 172.21.15.77:0/73936439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac912b43c0 con 0x55ac915deb40 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 0x55ac915e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb05effd640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb04803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12a8bc39bd4c7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048049030 con 0x55ac915deb40 2024-09-15T22:35:18.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7fb0480b9010 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb050017aa0 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb0480bc4e0 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 10 client.20694 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac915f6aa0 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050017aa0 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb0480ee090 con 0x55ac915deb40 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 10 client.20694 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 10 client.20694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.657+0000 7fb06848a740 20 client.20694 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac915f7eb0 0x55ac915fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb06848a740 4 client.20694 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac91550860 con 0x55ac915f7eb0 2024-09-15T22:35:18.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac915f7eb0 0x55ac915fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb03effd640 10 client.20694 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:35:18.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.658+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac915f7eb0 0x55ac915fa2e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=866ef576a3b0ed31 server_cookie=9bccedba66e592c4 in_seq=0 out_seq=0 2024-09-15T22:35:18.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb03effd640 1 -- 172.21.15.77:0/73936439 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac91550860 con 0x55ac915f7eb0 2024-09-15T22:35:18.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb03effd640 10 client.20694 handle_command_reply: tid=0 2024-09-15T22:35:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 client.20694 shutdown 2024-09-15T22:35:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 msgr2=0x55ac915e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb048006010 tx=0x7fb04803b800 comp rx=0 tx=0).stop 2024-09-15T22:35:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 shutdown_connections 2024-09-15T22:35:18.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac915f7eb0 0x55ac915fa2e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915deb40 0x55ac915e4fb0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb03effd640 0 client.20694 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 --2- 172.21.15.77:0/73936439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915dd5d0 0x55ac915e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 >> 172.21.15.77:0/73936439 conn(0x55ac915a6310 msgr2=0x55ac915df930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 shutdown_connections 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 1 -- 172.21.15.77:0/73936439 wait complete. 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.659+0000 7fb06848a740 20 client.20694 trim_cache size 0 max 16384 2024-09-15T22:35:18.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 Processor -- start 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 -- start start 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601af080 con 0x55ac915dd5d0 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b0bc0 con 0x55ac915deb40 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb06848a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601af750 con 0x55ac915e8370 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56386/0 (socket says 172.21.15.77:56386) 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2698302898 learned_addr learned my addr 172.21.15.77:0/2698302898 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05effd640 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2698302898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 msgr2=0x55ac915e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2698302898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 msgr2=0x55ac915e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.672+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2698302898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac915e7920 con 0x55ac915dd5d0 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb050000c00 tx=0x7fb05000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0523fbe6e719a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb0557fa640 1 -- 172.21.15.77:0/2698302898 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05002e070 con 0x55ac915dd5d0 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb0557fa640 1 -- 172.21.15.77:0/2698302898 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05000f830 con 0x55ac915dd5d0 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb0557fa640 1 -- 172.21.15.77:0/2698302898 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500048a0 con 0x55ac915dd5d0 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 -- 172.21.15.77:0/2698302898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 msgr2=0x55ac915e6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb050000c00 tx=0x7fb05000f700 comp rx=0 tx=0).stop 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 -- 172.21.15.77:0/2698302898 shutdown_connections 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 --2- 172.21.15.77:0/2698302898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915e6f20 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 -- 172.21.15.77:0/2698302898 >> 172.21.15.77:0/2698302898 conn(0x55ac915a6310 msgr2=0x55ac915e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 -- 172.21.15.77:0/2698302898 shutdown_connections 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.673+0000 7fb06848a740 1 -- 172.21.15.77:0/2698302898 wait complete. 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 -- start start 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 Processor -- start 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601b04f0 con 0x55ac915dd5d0 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601afe20 con 0x55ac915deb40 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ad080 con 0x55ac915e8370 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb06848a740 10 client.0 _ll_register_callbacks cb 0x7fb048003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:18.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56392/0 (socket says 172.21.15.77:56392) 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2864409522 learned_addr learned my addr 172.21.15.77:0/2864409522 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb05ffff640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.674+0000 7fb05effd640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2864409522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 msgr2=0x55ac915ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 msgr2=0x55ac915f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 -- 172.21.15.77:0/2864409522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac915e2e80 con 0x55ac915dd5d0 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05effd640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05ffff640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb05f7fe640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb05000f530 tx=0x7fb05000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611dfe786c19b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:18.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05002e070 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9526+0+0 (secure 0 0 0) 0x7fb05000fdf0 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05000f830 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb050005040 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 10 client.20700 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac91663060 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05000f830 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb050004b80 con 0x55ac915dd5d0 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 10 client.20700 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:18.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 10 client.20700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 20 client.20700 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac916634e0 0x55ac916658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 4 client.20700 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.675+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac9154ad70 con 0x55ac916634e0 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb05effd640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac916634e0 0x55ac916658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb05effd640 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac916634e0 0x55ac916658d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bd170cb57436d92 server_cookie=28fc71bb872c800a in_seq=0 out_seq=0 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb03effd640 10 client.20700 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb03effd640 1 -- 172.21.15.77:0/2864409522 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac9154ad70 con 0x55ac916634e0 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb03effd640 10 client.20700 handle_command_reply: tid=0 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.676+0000 7fb06848a740 1 client.20700 shutdown 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 msgr2=0x55ac915f0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb05000f530 tx=0x7fb05000f5e0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 shutdown_connections 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55ac916634e0 0x55ac916658d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ac915e8370 0x55ac915ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ac915deb40 0x55ac915f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb03effd640 0 client.20700 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:35:18.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 --2- 172.21.15.77:0/2864409522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ac915dd5d0 0x55ac915f0480 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 >> 172.21.15.77:0/2864409522 conn(0x55ac915a6310 msgr2=0x55ac915e7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 shutdown_connections 2024-09-15T22:35:18.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 1 -- 172.21.15.77:0/2864409522 wait complete. 2024-09-15T22:35:18.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb06848a740 20 client.20700 trim_cache size 0 max 16384 2024-09-15T22:35:18.707 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:35:18.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 msgr2=0x7fb04c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 0x7fb04c0a5870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb058008dc0 tx=0x7fb058008570 comp rx=0 tx=0).stop 2024-09-15T22:35:18.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 msgr2=0x7fb060198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.677+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb048005ab0 tx=0x7fb0480042f0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 shutdown_connections 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fb04c0a33b0 0x7fb04c0a5870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb060106f10 0x7fb06019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb060102790 0x7fb060198950 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 --2- 172.21.15.77:0/272665494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060101de0 0x7fb060198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 >> 172.21.15.77:0/272665494 conn(0x7fb0600fd8c0 msgr2=0x7fb0600fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 shutdown_connections 2024-09-15T22:35:18.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:18.678+0000 7fb0660b1640 1 -- 172.21.15.77:0/272665494 wait complete. 2024-09-15T22:35:19.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id f7de3897 --release --await' 2024-09-15T22:35:19.178 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id f7de3897 --release --await 2024-09-15T22:35:19.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160852 14.34 MB/sec execute 440 sec latency 34033.174 ms 2024-09-15T22:35:19.898 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:35:19.933 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:20.195 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:20 smithi077 ceph-mon[30980]: pgmap v723: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:20.195 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:20 smithi077 ceph-mon[30980]: osdmap e125: 12 total, 12 up, 12 in 2024-09-15T22:35:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:20 smithi196 ceph-mon[35109]: pgmap v723: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:20.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:20 smithi196 ceph-mon[35109]: osdmap e125: 12 total, 12 up, 12 in 2024-09-15T22:35:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:20 smithi117 ceph-mon[35021]: pgmap v723: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 752 KiB/s wr, 67 op/s 2024-09-15T22:35:20.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:20 smithi117 ceph-mon[35021]: osdmap e125: 12 total, 12 up, 12 in 2024-09-15T22:35:20.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- 172.21.15.77:0/3912051647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc102790 msgr2=0x7ffbdc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3912051647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc102790 0x7ffbdc1067e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd002a410 comp rx=0 tx=0).stop 2024-09-15T22:35:20.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- 172.21.15.77:0/3912051647 shutdown_connections 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3912051647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3912051647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc102790 0x7ffbdc1067e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3912051647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc101de0 0x7ffbdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- 172.21.15.77:0/3912051647 >> 172.21.15.77:0/3912051647 conn(0x7ffbdc0fd880 msgr2=0x7ffbdc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- 172.21.15.77:0/3912051647 shutdown_connections 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- 172.21.15.77:0/3912051647 wait complete. 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 Processor -- start 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.435+0000 7ffbe465f640 1 -- start start 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 0x7ffbdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc111f30 con 0x7ffbdc101de0 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc111db0 con 0x7ffbdc106f10 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1120b0 con 0x7ffbdc102790 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41998/0 (socket says 172.21.15.77:41998) 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 -- 172.21.15.77:0/3524908563 learned_addr learned my addr 172.21.15.77:0/3524908563 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 0x7ffbdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 -- 172.21.15.77:0/3524908563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 msgr2=0x7ffbdc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 0x7ffbdc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 -- 172.21.15.77:0/3524908563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 msgr2=0x7ffbdc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 -- 172.21.15.77:0/3524908563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc1a6970 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 0x7ffbdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c890 tx=0x7ffbd400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbdb7fe640 1 -- 172.21.15.77:0/3524908563 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4004220 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbdb7fe640 1 -- 172.21.15.77:0/3524908563 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd40043c0 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc1a6c60 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbdb7fe640 1 -- 172.21.15.77:0/3524908563 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40049e0 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.436+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1a71a0 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbdb7fe640 1 -- 172.21.15.77:0/3524908563 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffbd40079b0 con 0x7ffbdc101de0 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 Processor -- start 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 -- start start 2024-09-15T22:35:20.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x55987703ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x559877041630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559876f99620 con 0x559877034060 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559876f40e50 con 0x559877034980 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559876f955e0 con 0x55987703f240 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbdb7fe640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 0x7ffbc40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe2bfc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.439+0000 7ffbe2bfc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42012/0 (socket says 172.21.15.77:42012) 2024-09-15T22:35:20.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/3298089527 learned_addr learned my addr 172.21.15.77:0/3298089527 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 0x7ffbc40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbdb7fe640 1 -- 172.21.15.77:0/3524908563 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7ffbd40d0220 con 0x7ffbdc101de0 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x55987703ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x559877041630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/3298089527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 msgr2=0x559877041630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x559877041630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/3298089527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 msgr2=0x55987703ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x55987703ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/3298089527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559876f31d40 con 0x559877034060 2024-09-15T22:35:20.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0092f0 tx=0x7ffbcc014d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc3a1a7b650131 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbd8ff9640 1 -- 172.21.15.77:0/3298089527 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00f840 con 0x559877034060 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbd8ff9640 1 -- 172.21.15.77:0/3298089527 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc022070 con 0x559877034060 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 0x7ffbc40a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00f000 tx=0x7ffbcc01b000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.440+0000 7ffbd8ff9640 1 -- 172.21.15.77:0/3298089527 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01dc30 con 0x559877034060 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- 172.21.15.77:0/3298089527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 msgr2=0x559877034440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0092f0 tx=0x7ffbcc014d50 comp rx=0 tx=0).stop 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- 172.21.15.77:0/3298089527 shutdown_connections 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x559877041630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x55987703ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/3298089527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877034440 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- 172.21.15.77:0/3298089527 >> 172.21.15.77:0/3298089527 conn(0x5598770305f0 msgr2=0x5598770309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- 172.21.15.77:0/3298089527 shutdown_connections 2024-09-15T22:35:20.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- 172.21.15.77:0/3298089527 wait complete. 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.441+0000 7ffbe6a38740 1 -- start start 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 Processor -- start 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034060 0x55987703c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x55987703cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559877041e10 con 0x559877034980 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559876f21f10 con 0x559877034060 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559877041f90 con 0x55987703f240 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe6a38740 10 client.0 _ll_register_callbacks cb 0x559876f45db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42016/0 (socket says 172.21.15.77:42016) 2024-09-15T22:35:20.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 -- 172.21.15.77:0/2979982500 learned_addr learned my addr 172.21.15.77:0/2979982500 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x55987703cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034060 0x55987703c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 -- 172.21.15.77:0/2979982500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 msgr2=0x55987703cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x55987703cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 -- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034060 msgr2=0x55987703c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034060 0x55987703c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.442+0000 7ffbe1bfa640 1 -- 172.21.15.77:0/2979982500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559876ca8580 con 0x559877034980 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd0031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeade167789ee561 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0035040 con 0x559877034980 2024-09-15T22:35:20.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7ffbd0004240 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc022070 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffbd004e050 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 10 client.20712 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559876d72960 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc022070 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffbd0004a90 con 0x559877034980 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 10 client.20712 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 10 client.20712 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:35:20.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 20 client.20712 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55987704ee10 0x559877051270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 4 client.20712 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.443+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559876fa6b90 con 0x55987704ee10 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.444+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55987704ee10 0x559877051270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.444+0000 7ffbc1ffb640 10 client.20712 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.444+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55987704ee10 0x559877051270 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4887eef2069238f3 server_cookie=a428baf14b72123d in_seq=0 out_seq=0 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/2979982500 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559876fa6b90 con 0x55987704ee10 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbc1ffb640 10 client.20712 handle_command_reply: tid=0 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 client.20712 shutdown 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 msgr2=0x55987703c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd0031920 comp rx=0 tx=0).stop 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 shutdown_connections 2024-09-15T22:35:20.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55987704ee10 0x559877051270 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987703f240 0x55987703cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034980 0x55987703c680 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2979982500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034060 0x55987703c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 >> 172.21.15.77:0/2979982500 conn(0x5598770305f0 msgr2=0x55987702f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbc1ffb640 0 client.20712 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 shutdown_connections 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2979982500 wait complete. 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.445+0000 7ffbe6a38740 20 client.20712 trim_cache size 0 max 16384 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 Processor -- start 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 -- start start 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 0x55987703b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x559877056b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1b1280 con 0x55987703f240 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1b2dc0 con 0x559877034980 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1b1950 con 0x559877034060 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42022/0 (socket says 172.21.15.77:42022) 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/2649440087 learned_addr learned my addr 172.21.15.77:0/2649440087 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 0x55987703b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x559877056b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/2649440087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 msgr2=0x55987703b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 0x55987703b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/2649440087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 msgr2=0x559877056b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x559877056b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 -- 172.21.15.77:0/2649440087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987702ebb0 con 0x55987703f240 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 0x55987703b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x559877056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.462+0000 7ffbe2bfc640 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009fd0 tx=0x7ffbcc01d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87690d4f662fddec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbc3fff640 1 -- 172.21.15.77:0/2649440087 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01d420 con 0x55987703f240 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbc3fff640 1 -- 172.21.15.77:0/2649440087 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc022070 con 0x55987703f240 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbc3fff640 1 -- 172.21.15.77:0/2649440087 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00faa0 con 0x55987703f240 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2649440087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 msgr2=0x559877057080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009fd0 tx=0x7ffbcc01d8b0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2649440087 shutdown_connections 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55987703f240 0x559877057080 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559877034980 0x559877056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/2649440087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034060 0x55987703b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2649440087 >> 172.21.15.77:0/2649440087 conn(0x5598770305f0 msgr2=0x55987703e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2649440087 shutdown_connections 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.463+0000 7ffbe6a38740 1 -- 172.21.15.77:0/2649440087 wait complete. 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 -- start start 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 Processor -- start 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034980 0x559877043430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55987703f240 0x559877043970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1b26f0 con 0x559877034060 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1b2020 con 0x55987703f240 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1af280 con 0x559877034980 2024-09-15T22:35:20.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe6a38740 10 client.0 _ll_register_callbacks cb 0x7ffbd0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42030/0 (socket says 172.21.15.77:42030) 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 -- 172.21.15.77:0/563528324 learned_addr learned my addr 172.21.15.77:0/563528324 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 -- 172.21.15.77:0/563528324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034980 msgr2=0x559877043430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034980 0x559877043430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 -- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55987703f240 msgr2=0x559877043970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55987703f240 0x559877043970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 -- 172.21.15.77:0/563528324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598770b8b10 con 0x559877034060 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbe23fb640 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005450 tx=0x7ffbd40cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ba001debd72fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.464+0000 7ffbc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd401d040 con 0x559877034060 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7ffbd40bc0b0 con 0x559877034060 2024-09-15T22:35:20.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc022070 con 0x559877034060 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffbd40a04c0 con 0x559877034060 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 10 client.20718 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5598770b9d60 con 0x559877034060 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc022070 con 0x559877034060 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffbd4007ca0 con 0x559877034060 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 10 client.20718 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 10 client.20718 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 20 client.20718 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598770ba4d0 0x5598770bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 4 client.20718 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f7de3897", "release": true, "await": true}] 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f7de3897", "release": true, "await": true}) -- 0x559876fa10a0 con 0x5598770ba4d0 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.465+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598770ba4d0 0x5598770bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:20.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.466+0000 7ffbc1ffb640 10 client.20718 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:20.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.466+0000 7ffbe1bfa640 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598770ba4d0 0x5598770bc970 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597778d97f619dbd server_cookie=c55031c792aad50d in_seq=0 out_seq=0 2024-09-15T22:35:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbc1ffb640 1 -- 172.21.15.77:0/563528324 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559876fa10a0 con 0x5598770ba4d0 2024-09-15T22:35:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbc1ffb640 10 client.20718 handle_command_reply: tid=0 2024-09-15T22:35:20.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 client.20718 shutdown 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 msgr2=0x559877042ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005450 tx=0x7ffbd40cf040 comp rx=0 tx=0).stop 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 shutdown_connections 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5598770ba4d0 0x5598770bc970 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55987703f240 0x559877043970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbc1ffb640 0 client.20718 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559877034980 0x559877043430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 --2- 172.21.15.77:0/563528324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559877034060 0x559877042ef0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.529+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 >> 172.21.15.77:0/563528324 conn(0x5598770305f0 msgr2=0x55987703c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 shutdown_connections 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe6a38740 1 -- 172.21.15.77:0/563528324 wait complete. 2024-09-15T22:35:20.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe6a38740 20 client.20718 trim_cache size 0 max 16384 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 80, 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "f7de3897": { 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 80, 2024-09-15T22:35:20.533 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 154.4, 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:35:20.534 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:20.535 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 msgr2=0x7ffbc40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 0x7ffbc40a5870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00f000 tx=0x7ffbcc01b000 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 msgr2=0x7ffbdc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.530+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c890 tx=0x7ffbd400cd50 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 shutdown_connections 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffbc40a33b0 0x7ffbc40a5870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbdc106f10 0x7ffbdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbdc102790 0x7ffbdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 --2- 172.21.15.77:0/3524908563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbdc101de0 0x7ffbdc0744f0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:20.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 >> 172.21.15.77:0/3524908563 conn(0x7ffbdc0fd880 msgr2=0x7ffbdc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 shutdown_connections 2024-09-15T22:35:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:20.531+0000 7ffbe465f640 1 -- 172.21.15.77:0/3524908563 wait complete. 2024-09-15T22:35:20.537 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:20.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 160878 14.31 MB/sec execute 441 sec latency 34938.573 ms 2024-09-15T22:35:21.519 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f7de3897', total seconds elapsed: 39.0 2024-09-15T22:35:21.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 161302 14.34 MB/sec execute 442 sec latency 144.800 ms 2024-09-15T22:35:21.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 -- 172.21.15.77:0/1240918671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc101df0 msgr2=0x7f5ecc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/1240918671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc101df0 0x7f5ecc1021d0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009900 tx=0x7f5ebc02a410 comp rx=0 tx=0).stop 2024-09-15T22:35:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 -- 172.21.15.77:0/1240918671 shutdown_connections 2024-09-15T22:35:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/1240918671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc106f20 0x7f5ecc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/1240918671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/1240918671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc101df0 0x7f5ecc1021d0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 -- 172.21.15.77:0/1240918671 >> 172.21.15.77:0/1240918671 conn(0x7f5ecc0fd8f0 msgr2=0x7f5ecc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 -- 172.21.15.77:0/1240918671 shutdown_connections 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.720+0000 7f5ed1a31640 1 -- 172.21.15.77:0/1240918671 wait complete. 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 Processor -- start 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 -- start start 2024-09-15T22:35:21.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 0x7f5ecc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc106f20 0x7f5ecc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111f80 con 0x7f5ecc1027a0 2024-09-15T22:35:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111e00 con 0x7f5ecc106f20 2024-09-15T22:35:21.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ed1a31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc112100 con 0x7f5ecc101df0 2024-09-15T22:35:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.721+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 0x7f5ecc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42052/0 (socket says 172.21.15.77:42052) 2024-09-15T22:35:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3094010316 learned_addr learned my addr 172.21.15.77:0/3094010316 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:21.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 0x7f5ecc198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34432/0 (socket says 172.21.15.77:34432) 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc106f20 0x7f5ecc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3094010316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 msgr2=0x7f5ecc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 0x7f5ecc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3094010316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc106f20 msgr2=0x7f5ecc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc106f20 0x7f5ecc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3094010316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19d3f0 con 0x7f5ecc1027a0 2024-09-15T22:35:21.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000e9f0 tx=0x7f5ec000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ed0a2f640 1 -- 172.21.15.77:0/3094010316 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000cd80 con 0x7f5ecc1027a0 2024-09-15T22:35:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19d6e0 con 0x7f5ecc1027a0 2024-09-15T22:35:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ed0a2f640 1 -- 172.21.15.77:0/3094010316 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0010910 con 0x7f5ecc1027a0 2024-09-15T22:35:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ed0a2f640 1 -- 172.21.15.77:0/3094010316 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00188f0 con 0x7f5ecc1027a0 2024-09-15T22:35:21.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.722+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a4fc0 con 0x7f5ecc1027a0 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed0a2f640 1 -- 172.21.15.77:0/3094010316 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5ec0010ac0 con 0x7f5ecc1027a0 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 Processor -- start 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 -- start start 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x55938924aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559389259780 0x55938925bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5593891b02f0 con 0x55938924a5d0 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559389157cf0 con 0x55938924bb40 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5593891ac2b0 con 0x559389259780 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559389259780 0x55938925bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.725+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x55938924aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ed0a2f640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 0x7f5eb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55458/0 (socket says 172.21.15.77:55458) 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/1045607445 learned_addr learned my addr 172.21.15.77:0/1045607445 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 0x7f5eb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/1045607445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559389259780 msgr2=0x55938925bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559389259780 0x55938925bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/1045607445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 msgr2=0x55938924aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x55938924aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/1045607445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938913c3a0 con 0x55938924bb40 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ed0a2f640 1 -- 172.21.15.77:0/3094010316 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f5ec00d0440 con 0x7f5ecc1027a0 2024-09-15T22:35:21.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x55938924aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4014a60 tx=0x7f5ec4014e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b007b709d4e7c8a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 0x7f5eb40a5870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4012760 tx=0x7f5ec401e000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5eb1ffb640 1 -- 172.21.15.77:0/1045607445 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4004d20 con 0x55938924bb40 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5eb1ffb640 1 -- 172.21.15.77:0/1045607445 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4022030 con 0x55938924bb40 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.726+0000 7f5eb1ffb640 1 -- 172.21.15.77:0/1045607445 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4005520 con 0x55938924bb40 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/1045607445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 msgr2=0x55938924a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4014a60 tx=0x7f5ec4014e60 comp rx=0 tx=0).stop 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/1045607445 shutdown_connections 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559389259780 0x55938925bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x55938924aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/1045607445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x55938924a090 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/1045607445 >> 172.21.15.77:0/1045607445 conn(0x559389213310 msgr2=0x559389246c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/1045607445 shutdown_connections 2024-09-15T22:35:21.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.727+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/1045607445 wait complete. 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 -- start start 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 Processor -- start 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x559389259160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559389259780 0x5593892569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55938925c480 con 0x55938924a5d0 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55938916b060 con 0x559389259780 2024-09-15T22:35:21.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55938925c600 con 0x55938924bb40 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ed3e0a740 10 client.0 _ll_register_callbacks cb 0x559388ebe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559389259780 0x5593892569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x559389259160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34458/0 (socket says 172.21.15.77:34458) 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.728+0000 7f5eca7fc640 1 -- 172.21.15.77:0/2366386372 learned_addr learned my addr 172.21.15.77:0/2366386372 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 -- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559389259780 msgr2=0x5593892569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559389259780 0x5593892569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 -- 172.21.15.77:0/2366386372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 msgr2=0x559389259160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x559389259160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 -- 172.21.15.77:0/2366386372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559388f213c0 con 0x55938924bb40 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x559389259160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:21.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00988a0 tx=0x7f5ec00d0640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8832bd45597a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00a5000 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f5ec0010430 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec4022030 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.729+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ec009c440 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 10 client.29323 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559389267f50 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4022030 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f5ec009b890 con 0x55938924bb40 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 10 client.29323 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:21.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 10 client.29323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 20 client.29323 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559389269830 0x55938926bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 4 client.29323 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593891bd860 con 0x559389269830 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.730+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559389269830 0x55938926bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.731+0000 7f5eaaffd640 10 client.29323 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.731+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559389269830 0x55938926bc20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b84bb7360531e3 server_cookie=c0b664c60a4dc966 in_seq=0 out_seq=0 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.732+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2366386372 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593891bd860 con 0x559389269830 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.732+0000 7f5eaaffd640 10 client.29323 handle_command_reply: tid=0 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.732+0000 7f5ed3e0a740 1 client.29323 shutdown 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 msgr2=0x559389256410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00988a0 tx=0x7f5ec00d0640 comp rx=0 tx=0).stop 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 shutdown_connections 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x559389269830 0x55938926bc20 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559389259780 0x5593892569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x559389256410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5eaaffd640 0 client.29323 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2366386372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55938924a5d0 0x559389259160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 >> 172.21.15.77:0/2366386372 conn(0x559389213310 msgr2=0x55938924c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 shutdown_connections 2024-09-15T22:35:21.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2366386372 wait complete. 2024-09-15T22:35:21.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.733+0000 7f5ed3e0a740 20 client.29323 trim_cache size 0 max 16384 2024-09-15T22:35:21.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.748+0000 7f5ed3e0a740 1 Processor -- start 2024-09-15T22:35:21.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.748+0000 7f5ed3e0a740 1 -- start start 2024-09-15T22:35:21.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924a5d0 0x5593892712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x559389253670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1acd50 con 0x559389259780 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ae890 con 0x55938924a5d0 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ad420 con 0x55938924bb40 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34462/0 (socket says 172.21.15.77:34462) 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3691343848 learned_addr learned my addr 172.21.15.77:0/3691343848 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924a5d0 0x5593892712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3691343848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924a5d0 msgr2=0x5593892712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924a5d0 0x5593892712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3691343848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 msgr2=0x559389253670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x559389253670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 -- 172.21.15.77:0/3691343848 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593892594d0 con 0x55938924bb40 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.749+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c6a0 tx=0x7f5ec009a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec252e18c959a0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.750+0000 7f5eb17fa640 1 -- 172.21.15.77:0/3691343848 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec009bda0 con 0x55938924bb40 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.750+0000 7f5eb17fa640 1 -- 172.21.15.77:0/3691343848 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec00a0740 con 0x55938924bb40 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.750+0000 7f5eb17fa640 1 -- 172.21.15.77:0/3691343848 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0010690 con 0x55938924bb40 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.750+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/3691343848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 msgr2=0x5593892717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.750+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c6a0 tx=0x7f5ec009a950 comp rx=0 tx=0).stop 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/3691343848 shutdown_connections 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x559389253670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924bb40 0x5593892717f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/3691343848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924a5d0 0x5593892712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/3691343848 >> 172.21.15.77:0/3691343848 conn(0x559389213310 msgr2=0x559389254ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/3691343848 shutdown_connections 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/3691343848 wait complete. 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.751+0000 7f5ed3e0a740 1 -- start start 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.752+0000 7f5ed3e0a740 1 Processor -- start 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.752+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 0x5593892716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.752+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x559389271c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.752+0000 7f5ed3e0a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x55938926ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 0x5593892716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ae1c0 con 0x559389259780 2024-09-15T22:35:21.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1adaf0 con 0x55938924bb40 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ed3e0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1aad50 con 0x55938924a5d0 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 0x5593892716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34466/0 (socket says 172.21.15.77:34466) 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x55938926ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecaffd640 1 -- 172.21.15.77:0/2720032706 learned_addr learned my addr 172.21.15.77:0/2720032706 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5eca7fc640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x559389271c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ed3e0a740 10 client.0 _ll_register_callbacks cb 0x7f5ec000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/2720032706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 msgr2=0x5593892716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 0x5593892716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 msgr2=0x559389271c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x559389271c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 -- 172.21.15.77:0/2720032706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938926f490 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5ecb7fe640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x55938926ec60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4006010 tx=0x7f5ec4014590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad737a583aff248b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.753+0000 7f5eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4014220 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f5ec40049a0 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec00a0740 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ec4004bc0 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5ed3e0a740 10 client.20733 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.754+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5593892d3f40 con 0x559389259780 2024-09-15T22:35:21.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.755+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec00a0740 con 0x559389259780 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.755+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f5ec4031070 con 0x559389259780 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 10 client.20733 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 10 client.20733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 20 client.20733 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5593892d4330 0x5593892d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 4 client.20733 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593891b7d70 con 0x5593892d4330 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.756+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5593892d4330 0x5593892d6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.757+0000 7f5eaaffd640 10 client.20733 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:21.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.757+0000 7f5ecaffd640 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5593892d4330 0x5593892d6720 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57082d69d79bbe83 server_cookie=250e28aaf4e7ebbd in_seq=0 out_seq=0 2024-09-15T22:35:21.806 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.804+0000 7f5eaaffd640 1 -- 172.21.15.77:0/2720032706 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593891b7d70 con 0x5593892d4330 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.804+0000 7f5eaaffd640 10 client.20733 handle_command_reply: tid=0 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.804+0000 7f5ed3e0a740 1 client.20733 shutdown 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 msgr2=0x55938926ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x55938926ec60 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4006010 tx=0x7f5ec4014590 comp rx=0 tx=0).stop 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 shutdown_connections 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5593892d4330 0x5593892d6720 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559389259780 0x55938926ec60 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5eaaffd640 0 client.20733 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55938924bb40 0x559389271c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 --2- 172.21.15.77:0/2720032706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55938924a5d0 0x5593892716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 >> 172.21.15.77:0/2720032706 conn(0x559389213310 msgr2=0x5593892454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 shutdown_connections 2024-09-15T22:35:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 1 -- 172.21.15.77:0/2720032706 wait complete. 2024-09-15T22:35:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.805+0000 7f5ed3e0a740 20 client.20733 trim_cache size 0 max 16384 2024-09-15T22:35:21.808 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:35:21.808 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 msgr2=0x7f5eb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 0x7f5eb40a5870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4012760 tx=0x7f5ec401e000 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 msgr2=0x7f5ecc198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000e9f0 tx=0x7f5ec000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 shutdown_connections 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5eb40a33b0 0x7f5eb40a5870 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ecc106f20 0x7f5ecc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ecc1027a0 0x7f5ecc198980 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 --2- 172.21.15.77:0/3094010316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ecc101df0 0x7f5ecc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.809 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 >> 172.21.15.77:0/3094010316 conn(0x7f5ecc0fd8f0 msgr2=0x7f5ecc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 shutdown_connections 2024-09-15T22:35:21.810 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:21.806+0000 7f5ed1a31640 1 -- 172.21.15.77:0/3094010316 wait complete. 2024-09-15T22:35:22.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:22 smithi077 ceph-mon[30980]: pgmap v725: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-09-15T22:35:22.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:22 smithi196 ceph-mon[35109]: pgmap v725: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-09-15T22:35:22.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:22 smithi117 ceph-mon[35021]: pgmap v725: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-09-15T22:35:22.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 161744 14.37 MB/sec execute 443 sec latency 151.953 ms 2024-09-15T22:35:23.562 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:35:23.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:35:23.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 161976 14.37 MB/sec execute 444 sec latency 198.522 ms 2024-09-15T22:35:23.893 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:24.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:24 smithi077 ceph-mon[30980]: pgmap v726: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:24.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:24 smithi196 ceph-mon[35109]: pgmap v726: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:24.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:24 smithi117 ceph-mon[35021]: pgmap v726: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:24.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 -- 172.21.15.77:0/115985639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4106a80 msgr2=0x7f6df4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/115985639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4106a80 0x7f6df4106e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009900 tx=0x7f6df002a410 comp rx=0 tx=0).stop 2024-09-15T22:35:24.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 -- 172.21.15.77:0/115985639 shutdown_connections 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/115985639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4101420 0x7f6df410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/115985639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df4100a80 0x7f6df4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/115985639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4106a80 0x7f6df4106e60 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009900 tx=0x7f6df002a410 comp rx=0 tx=0).stop 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 -- 172.21.15.77:0/115985639 >> 172.21.15.77:0/115985639 conn(0x7f6df40fc7a0 msgr2=0x7f6df40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 -- 172.21.15.77:0/115985639 shutdown_connections 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.415+0000 7f6dfbc04640 1 -- 172.21.15.77:0/115985639 wait complete. 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.416+0000 7f6dfbc04640 1 Processor -- start 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.417+0000 7f6dfbc04640 1 -- start start 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.417+0000 7f6dfbc04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 0x7f6df4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.417+0000 7f6dfbc04640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4101420 0x7f6df4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfbc04640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 0x7f6df419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfbc04640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4110e80 con 0x7f6df4100a80 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfbc04640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4110d00 con 0x7f6df4101420 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfbc04640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111000 con 0x7f6df419be50 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6df99a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 0x7f6df4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6df99a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 0x7f6df4197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42100/0 (socket says 172.21.15.77:42100) 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6df99a0640 1 -- 172.21.15.77:0/748538071 learned_addr learned my addr 172.21.15.77:0/748538071 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 0x7f6df419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6df919f640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4101420 0x7f6df4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/748538071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4101420 msgr2=0x7f6df4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4101420 0x7f6df4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 msgr2=0x7f6df4197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 0x7f6df4197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.418+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/748538071 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df419c910 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 0x7f6df419c300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00dba0 tx=0x7f6dec007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6deaffd640 1 -- 172.21.15.77:0/748538071 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00ddb0 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6deaffd640 1 -- 172.21.15.77:0/748538071 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec003e50 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df419cc00 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6deaffd640 1 -- 172.21.15.77:0/748538071 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec019470 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.419+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df406aa70 con 0x7f6df419be50 2024-09-15T22:35:24.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.422+0000 7f6deaffd640 1 -- 172.21.15.77:0/748538071 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6dec019610 con 0x7f6df419be50 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.422+0000 7f6dfdfdd740 1 Processor -- start 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 -- start start 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6deaffd640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 0x7f6de00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6f04280 con 0x55dbf6f9f480 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6eabb10 con 0x55dbf6f9eb60 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6f00240 con 0x55dbf6fa9d40 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6dfa1a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6df919f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6df919f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55510/0 (socket says 172.21.15.77:55510) 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6df919f640 1 -- 172.21.15.77:0/4252004483 learned_addr learned my addr 172.21.15.77:0/4252004483 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6df99a0640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 0x7f6de00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6df919f640 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.423+0000 7f6deaffd640 1 -- 172.21.15.77:0/748538071 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f6dec0d1b70 con 0x7f6df419be50 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 -- 172.21.15.77:0/4252004483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 msgr2=0x55dbf6fac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 -- 172.21.15.77:0/4252004483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9f480 msgr2=0x55dbf6fa9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 -- 172.21.15.77:0/4252004483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf6eb81f0 con 0x55dbf6f9eb60 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6df919f640 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009fd0 tx=0x7f6de4009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5487702f3d901989 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6ddffff640 1 -- 172.21.15.77:0/4252004483 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de400f790 con 0x55dbf6f9eb60 2024-09-15T22:35:24.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6ddffff640 1 -- 172.21.15.77:0/4252004483 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de400fe10 con 0x55dbf6f9eb60 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.424+0000 7f6ddffff640 1 -- 172.21.15.77:0/4252004483 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de401fe10 con 0x55dbf6f9eb60 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6df99a0640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 0x7f6de00a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6df0000c00 tx=0x7f6df00058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/4252004483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 msgr2=0x55dbf6f9ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009fd0 tx=0x7f6de4009370 comp rx=0 tx=0).stop 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/4252004483 shutdown_connections 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/4252004483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6f9ef40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.425+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/4252004483 >> 172.21.15.77:0/4252004483 conn(0x55dbf6f9b0f0 msgr2=0x55dbf6f9b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.426+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/4252004483 shutdown_connections 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.426+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/4252004483 wait complete. 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.426+0000 7f6dfdfdd740 1 -- start start 2024-09-15T22:35:24.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.427+0000 7f6dfdfdd740 1 Processor -- start 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.427+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df99a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df99a0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42126/0 (socket says 172.21.15.77:42126) 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df99a0640 1 -- 172.21.15.77:0/563189469 learned_addr learned my addr 172.21.15.77:0/563189469 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6fb0d20 con 0x55dbf6f9eb60 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6ebef10 con 0x55dbf6f9f480 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dbf6fb0ea0 con 0x55dbf6fa9d40 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6dfdfdd740 10 client.0 _ll_register_callbacks cb 0x55dbf6e87a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:24.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 -- 172.21.15.77:0/563189469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 msgr2=0x55dbf6fa76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 -- 172.21.15.77:0/563189469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 msgr2=0x55dbf6fa6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df919f640 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf6c756a0 con 0x55dbf6f9f480 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.428+0000 7f6df99a0640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6df919f640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa7160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6de401f990 tx=0x7f6de401f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d04f5092cd37323 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6ddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004240 con 0x55dbf6f9f480 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de400fe10 con 0x55dbf6f9f480 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f6de40043e0 con 0x55dbf6f9f480 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6de400f790 con 0x55dbf6f9f480 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6dfdfdd740 10 client.29405 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:24.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbf6fadf70 con 0x55dbf6f9f480 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.429+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de400fe10 con 0x55dbf6f9f480 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6de401d020 con 0x55dbf6f9f480 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 10 client.29405 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 10 client.29405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 20 client.29405 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf6faf320 0x55dbf6faf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 4 client.29405 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf6f117f0 con 0x55dbf6faf320 2024-09-15T22:35:24.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.430+0000 7f6df99a0640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf6faf320 0x55dbf6faf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.434+0000 7f6df99a0640 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf6faf320 0x55dbf6faf7a0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfa1f1d503939e19 server_cookie=9b47d490248bce18 in_seq=0 out_seq=0 2024-09-15T22:35:24.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.435+0000 7f6ddd7fa640 10 client.29405 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:24.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.437+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/563189469 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbf6f117f0 con 0x55dbf6faf320 2024-09-15T22:35:24.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.437+0000 7f6ddd7fa640 10 client.29405 handle_command_reply: tid=0 2024-09-15T22:35:24.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.437+0000 7f6dfdfdd740 1 client.29405 shutdown 2024-09-15T22:35:24.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 msgr2=0x55dbf6fa7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa7160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6de401f990 tx=0x7f6de401f6a0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 shutdown_connections 2024-09-15T22:35:24.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf6faf320 0x55dbf6faf7a0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6ddd7fa640 0 client.29405 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9f480 0x55dbf6fa7160 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6de401f990 tx=0x7f6de401f6a0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/563189469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.438+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 >> 172.21.15.77:0/563189469 conn(0x55dbf6f9b0f0 msgr2=0x55dbf6fa0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.439+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 shutdown_connections 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.439+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/563189469 wait complete. 2024-09-15T22:35:24.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.440+0000 7f6dfdfdd740 20 client.29405 trim_cache size 0 max 16384 2024-09-15T22:35:24.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.472+0000 7f6dfdfdd740 1 Processor -- start 2024-09-15T22:35:24.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.473+0000 7f6dfdfdd740 1 -- start start 2024-09-15T22:35:24.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ad3d0 con 0x55dbf6fa9d40 2024-09-15T22:35:24.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41aef10 con 0x55dbf6f9eb60 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41adaa0 con 0x55dbf6faf320 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfa1a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfa1a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:34520/0 (socket says 172.21.15.77:34520) 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.474+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/768209648 learned_addr learned my addr 172.21.15.77:0/768209648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.475+0000 7f6df919f640 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.475+0000 7f6df99a0640 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.476+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/768209648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 msgr2=0x55dbf6fa1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.476+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.476+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/768209648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 msgr2=0x55dbf6fa3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.476+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.476+0000 7f6dfa1a1640 1 -- 172.21.15.77:0/768209648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf6fb0590 con 0x55dbf6faf320 2024-09-15T22:35:24.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.477+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6dec007fd0 tx=0x7f6dec09d660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f64cee1310145945 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.478+0000 7f6ddf7fe640 1 -- 172.21.15.77:0/768209648 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec004000 con 0x55dbf6faf320 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.478+0000 7f6ddf7fe640 1 -- 172.21.15.77:0/768209648 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec09da60 con 0x55dbf6faf320 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.478+0000 7f6ddf7fe640 1 -- 172.21.15.77:0/768209648 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a66d0 con 0x55dbf6faf320 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.479+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/768209648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 msgr2=0x55dbf6fa43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.479+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6dec007fd0 tx=0x7f6dec09d660 comp rx=0 tx=0).stop 2024-09-15T22:35:24.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/768209648 shutdown_connections 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fa43c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fa3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/768209648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fa1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/768209648 >> 172.21.15.77:0/768209648 conn(0x55dbf6f9b0f0 msgr2=0x55dbf6fb9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.480+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/768209648 shutdown_connections 2024-09-15T22:35:24.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.481+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/768209648 wait complete. 2024-09-15T22:35:24.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.481+0000 7f6dfdfdd740 1 -- start start 2024-09-15T22:35:24.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.482+0000 7f6dfdfdd740 1 Processor -- start 2024-09-15T22:35:24.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fcab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fc5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ae840 con 0x55dbf6fa9d40 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ae170 con 0x55dbf6f9eb60 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41ab3d0 con 0x55dbf6faf320 2024-09-15T22:35:24.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.483+0000 7f6dfdfdd740 10 client.0 _ll_register_callbacks cb 0x7f6de4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:24.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42142/0 (socket says 172.21.15.77:42142) 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 -- 172.21.15.77:0/1268438227 learned_addr learned my addr 172.21.15.77:0/1268438227 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6dfa1a1640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fc5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df99a0640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fcab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 -- 172.21.15.77:0/1268438227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 msgr2=0x55dbf6fc5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fc5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 -- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 msgr2=0x55dbf6fcab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:24.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fcab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.484+0000 7f6df919f640 1 -- 172.21.15.77:0/1268438227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf6fc6730 con 0x55dbf6fa9d40 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.485+0000 7f6df919f640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f6de401f670 tx=0x7f6de4009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5625bb52b5c4fdf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.485+0000 7f6ddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.485+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4005c50 con 0x55dbf6fa9d40 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.485+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dec09da60 con 0x55dbf6fa9d40 2024-09-15T22:35:24.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.485+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f6de4003800 con 0x55dbf6fa9d40 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.486+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6de4043050 con 0x55dbf6fa9d40 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.486+0000 7f6dfdfdd740 10 client.20751 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.486+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dbf702d3f0 con 0x55dbf6fa9d40 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec09da60 con 0x55dbf6fa9d40 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f6de401d040 con 0x55dbf6fa9d40 2024-09-15T22:35:24.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 10 client.20751 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 10 client.20751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 20 client.20751 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf702da30 0x55dbf702fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 4 client.20751 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.487+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbf6f0bd00 con 0x55dbf702da30 2024-09-15T22:35:24.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.489+0000 7f6df99a0640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf702da30 0x55dbf702fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:24.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.497+0000 7f6df99a0640 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf702da30 0x55dbf702fe50 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf0ad19722986f0 server_cookie=2b790eeb83a34e1 in_seq=0 out_seq=0 2024-09-15T22:35:24.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.497+0000 7f6ddd7fa640 10 client.20751 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.500+0000 7f6ddd7fa640 1 -- 172.21.15.77:0/1268438227 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbf6f0bd00 con 0x55dbf702da30 2024-09-15T22:35:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.500+0000 7f6ddd7fa640 10 client.20751 handle_command_reply: tid=0 2024-09-15T22:35:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.500+0000 7f6dfdfdd740 1 client.20751 shutdown 2024-09-15T22:35:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 msgr2=0x55dbf6fcb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f6de401f670 tx=0x7f6de4009370 comp rx=0 tx=0).stop 2024-09-15T22:35:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 shutdown_connections 2024-09-15T22:35:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55dbf702da30 0x55dbf702fe50 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf6faf320 0x55dbf6fc5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dbf6fa9d40 0x55dbf6fcb060 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 --2- 172.21.15.77:0/1268438227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dbf6f9eb60 0x55dbf6fcab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.501+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 >> 172.21.15.77:0/1268438227 conn(0x55dbf6f9b0f0 msgr2=0x55dbf6fa53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.502+0000 7f6ddd7fa640 0 client.20751 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:24.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.507+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 shutdown_connections 2024-09-15T22:35:24.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.507+0000 7f6dfdfdd740 1 -- 172.21.15.77:0/1268438227 wait complete. 2024-09-15T22:35:24.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.507+0000 7f6dfdfdd740 20 client.20751 trim_cache size 0 max 16384 2024-09-15T22:35:24.508 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:35:24.509 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:35:24.509 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "c3beda0e-d5be-42a0-87a6-7c4eb6e253d7", 2024-09-15T22:35:24.509 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:35:24.509 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 msgr2=0x7f6de00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 0x7f6de00a58f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6df0000c00 tx=0x7f6df00058d0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 msgr2=0x7f6df419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 0x7f6df419c300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00dba0 tx=0x7f6dec007590 comp rx=0 tx=0).stop 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 shutdown_connections 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6de00a3430 0x7f6de00a58f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6df419be50 0x7f6df419c300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6df4101420 0x7f6df4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 --2- 172.21.15.77:0/748538071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df4100a80 0x7f6df4197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:24.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 >> 172.21.15.77:0/748538071 conn(0x7f6df40fc7a0 msgr2=0x7f6df40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:24.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 shutdown_connections 2024-09-15T22:35:24.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:24.508+0000 7f6dfbc04640 1 -- 172.21.15.77:0/748538071 wait complete. 2024-09-15T22:35:24.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 162317 14.37 MB/sec execute 445 sec latency 165.851 ms 2024-09-15T22:35:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:35:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:35:25.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:25.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:25.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:25.379 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:35:25.620 INFO:tasks.workunit.client.0.smithi077.stdout: 10 162693 14.38 MB/sec execute 446 sec latency 239.093 ms 2024-09-15T22:35:25.698 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:26.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 -- 172.21.15.77:0/3969643468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 msgr2=0x7f8c5c114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 --2- 172.21.15.77:0/3969643468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c114570 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02a400 comp rx=0 tx=0).stop 2024-09-15T22:35:26.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 -- 172.21.15.77:0/3969643468 shutdown_connections 2024-09-15T22:35:26.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 --2- 172.21.15.77:0/3969643468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 --2- 172.21.15.77:0/3969643468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c114570 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 --2- 172.21.15.77:0/3969643468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 0x7f8c5c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.240+0000 7f8c6395a640 1 -- 172.21.15.77:0/3969643468 >> 172.21.15.77:0/3969643468 conn(0x7f8c5c06e2d0 msgr2=0x7f8c5c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.241+0000 7f8c6395a640 1 -- 172.21.15.77:0/3969643468 shutdown_connections 2024-09-15T22:35:26.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.241+0000 7f8c6395a640 1 -- 172.21.15.77:0/3969643468 wait complete. 2024-09-15T22:35:26.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.242+0000 7f8c6395a640 1 Processor -- start 2024-09-15T22:35:26.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.242+0000 7f8c6395a640 1 -- start start 2024-09-15T22:35:26.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.242+0000 7f8c6395a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 0x7f8c5c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c6395a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c6395a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c6395a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c118560 con 0x7f8c5c109ba0 2024-09-15T22:35:26.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c6395a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c1183e0 con 0x7f8c5c114ab0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c6395a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c1186e0 con 0x7f8c5c1091f0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c61ef7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c61ef7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55564/0 (socket says 172.21.15.77:55564) 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c61ef7640 1 -- 172.21.15.77:0/2322494668 learned_addr learned my addr 172.21.15.77:0/2322494668 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.243+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c61ef7640 1 -- 172.21.15.77:0/2322494668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 msgr2=0x7f8c5c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c616f6640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 0x7f8c5c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c61ef7640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 0x7f8c5c1a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c61ef7640 1 -- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 msgr2=0x7f8c5c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c61ef7640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c61ef7640 1 -- 172.21.15.77:0/2322494668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c19efc0 con 0x7f8c5c114ab0 2024-09-15T22:35:26.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.244+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c61ef7640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9d0 tx=0x7f8c5400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c527fc640 1 -- 172.21.15.77:0/2322494668 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400cd80 con 0x7f8c5c114ab0 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c19f2b0 con 0x7f8c5c114ab0 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c1ab5e0 con 0x7f8c5c114ab0 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c527fc640 1 -- 172.21.15.77:0/2322494668 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c54005020 con 0x7f8c5c114ab0 2024-09-15T22:35:26.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.245+0000 7f8c527fc640 1 -- 172.21.15.77:0/2322494668 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c540116d0 con 0x7f8c5c114ab0 2024-09-15T22:35:26.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.249+0000 7f8c65d33740 1 Processor -- start 2024-09-15T22:35:26.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c527fc640 1 -- 172.21.15.77:0/2322494668 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8c54011930 con 0x7f8c5c114ab0 2024-09-15T22:35:26.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c65d33740 1 -- start start 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1feb7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb75d0 0x55db1fec1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 0x55db1fec4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.250+0000 7f8c65d33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db1fe142e0 con 0x55db1feb6cb0 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c65d33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db1fdbbce0 con 0x55db1feb75d0 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c65d33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db1fe102a0 con 0x55db1fec1e90 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c616f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb75d0 0x55db1fec1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c527fc640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 0x7f8c480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 0x55db1fec4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1feb7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1feb7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42172/0 (socket says 172.21.15.77:42172) 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c60ef5640 1 -- 172.21.15.77:0/4007483722 learned_addr learned my addr 172.21.15.77:0/4007483722 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.251+0000 7f8c527fc640 1 -- 172.21.15.77:0/2322494668 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f8c540d0410 con 0x7f8c5c114ab0 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 -- 172.21.15.77:0/4007483722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb75d0 msgr2=0x55db1fec1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb75d0 0x55db1fec1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 -- 172.21.15.77:0/4007483722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 msgr2=0x55db1feb7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1feb7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 -- 172.21.15.77:0/4007483722 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1fda0440 con 0x55db1fec1e90 2024-09-15T22:35:26.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c61ef7640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 0x7f8c480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.252+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 0x55db1fec4280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ec0 tx=0x7f8c4c005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d0a5ff91457b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.253+0000 7f8c477fe640 1 -- 172.21.15.77:0/4007483722 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035020 con 0x55db1fec1e90 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.253+0000 7f8c477fe640 1 -- 172.21.15.77:0/4007483722 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4c004c20 con 0x55db1fec1e90 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.253+0000 7f8c477fe640 1 -- 172.21.15.77:0/4007483722 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003770 con 0x55db1fec1e90 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c61ef7640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 0x7f8c480a58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c54099b90 tx=0x7f8c54013000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- 172.21.15.77:0/4007483722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 msgr2=0x55db1fec4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 0x55db1fec4280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005ec0 tx=0x7f8c4c005b70 comp rx=0 tx=0).stop 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- 172.21.15.77:0/4007483722 shutdown_connections 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1fec1e90 0x55db1fec4280 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb75d0 0x55db1fec1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 --2- 172.21.15.77:0/4007483722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1feb7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- 172.21.15.77:0/4007483722 >> 172.21.15.77:0/4007483722 conn(0x55db1feb3240 msgr2=0x55db1feb3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- 172.21.15.77:0/4007483722 shutdown_connections 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- 172.21.15.77:0/4007483722 wait complete. 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.254+0000 7f8c65d33740 1 -- start start 2024-09-15T22:35:26.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.255+0000 7f8c65d33740 1 Processor -- start 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.255+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1febea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1fec1e90 0x55db1febef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db1fec4a60 con 0x55db1fec1e90 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db1fdcf050 con 0x55db1feb6cb0 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55db1fec4be0 con 0x55db1feb75d0 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c65d33740 10 client.0 _ll_register_callbacks cb 0x55db1fe0f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.256+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.257+0000 7f8c616f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1febea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.257+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:55590/0 (socket says 172.21.15.77:55590) 2024-09-15T22:35:26.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.257+0000 7f8c60ef5640 1 -- 172.21.15.77:0/2330956569 learned_addr learned my addr 172.21.15.77:0/2330956569 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.257+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1fec1e90 0x55db1febef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 -- 172.21.15.77:0/2330956569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 msgr2=0x55db1febea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1febea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 -- 172.21.15.77:0/2330956569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1fec1e90 msgr2=0x55db1febef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1fec1e90 0x55db1febef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 -- 172.21.15.77:0/2330956569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1fb853c0 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c003650 tx=0x7f8c4c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6360acf532750c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035030 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c004c20 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.258+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f8c4c031760 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c005080 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 10 client.29423 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55db1fec4a60 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4c004c20 con 0x55db1feb6cb0 2024-09-15T22:35:26.262 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8c4c03a040 con 0x55db1feb6cb0 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 10 client.29423 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 10 client.29423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 20 client.29423 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1fed15e0 0x55db1fed39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 4 client.29423 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.259+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db1fe21850 con 0x55db1fed15e0 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.260+0000 7f8c616f6640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1fed15e0 0x55db1fed39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.260+0000 7f8c616f6640 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1fed15e0 0x55db1fed39d0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24894640cedcb27 server_cookie=d504058121fe0547 in_seq=0 out_seq=0 2024-09-15T22:35:26.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.260+0000 7f8c44ff9640 10 client.29423 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.263+0000 7f8c44ff9640 1 -- 172.21.15.77:0/2330956569 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db1fe21850 con 0x55db1fed15e0 2024-09-15T22:35:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.263+0000 7f8c44ff9640 10 client.29423 handle_command_reply: tid=0 2024-09-15T22:35:26.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.263+0000 7f8c65d33740 1 client.29423 shutdown 2024-09-15T22:35:26.265 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 msgr2=0x55db1fec1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c003650 tx=0x7f8c4c0045e0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 shutdown_connections 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1fed15e0 0x55db1fed39d0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1fec1e90 0x55db1febef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1febea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 --2- 172.21.15.77:0/2330956569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1feb6cb0 0x55db1fec1780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 >> 172.21.15.77:0/2330956569 conn(0x55db1feb3240 msgr2=0x55db1fea9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c44ff9640 0 client.29423 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 shutdown_connections 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 1 -- 172.21.15.77:0/2330956569 wait complete. 2024-09-15T22:35:26.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.264+0000 7f8c65d33740 20 client.29423 trim_cache size 0 max 16384 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.289+0000 7f8c65d33740 1 Processor -- start 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.289+0000 7f8c65d33740 1 -- start start 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.289+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.289+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1fed90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.289+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fed9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c65d33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4000fe90 con 0x55db1feb6cb0 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c65d33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400119d0 con 0x55db1fec1e90 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c65d33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40010560 con 0x55db1feb75d0 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42194/0 (socket says 172.21.15.77:42194) 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 -- 172.21.15.77:0/1687537714 learned_addr learned my addr 172.21.15.77:0/1687537714 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c616f6640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1fed90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fed9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 -- 172.21.15.77:0/1687537714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 msgr2=0x55db1fed90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1fed90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 -- 172.21.15.77:0/1687537714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 msgr2=0x55db1fed9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fed9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 -- 172.21.15.77:0/1687537714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1fec1af0 con 0x55db1feb6cb0 2024-09-15T22:35:26.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c616f6640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1fed90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:26.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005bb0 tx=0x7f8c4c0314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac615c912685b2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c46ffd640 1 -- 172.21.15.77:0/1687537714 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c035040 con 0x55db1feb6cb0 2024-09-15T22:35:26.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c46ffd640 1 -- 172.21.15.77:0/1687537714 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4c003b60 con 0x55db1feb6cb0 2024-09-15T22:35:26.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.290+0000 7f8c46ffd640 1 -- 172.21.15.77:0/1687537714 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c03daa0 con 0x55db1feb6cb0 2024-09-15T22:35:26.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 -- 172.21.15.77:0/1687537714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 msgr2=0x55db1fecdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005bb0 tx=0x7f8c4c0314d0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 -- 172.21.15.77:0/1687537714 shutdown_connections 2024-09-15T22:35:26.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fed9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1fed90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 --2- 172.21.15.77:0/1687537714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1fecdd50 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 -- 172.21.15.77:0/1687537714 >> 172.21.15.77:0/1687537714 conn(0x55db1feb3240 msgr2=0x55db1fea9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 -- 172.21.15.77:0/1687537714 shutdown_connections 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.291+0000 7f8c65d33740 1 -- 172.21.15.77:0/1687537714 wait complete. 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 -- start start 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 Processor -- start 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1ff348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fec5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40011300 con 0x55db1feb6cb0 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40010c30 con 0x55db1fec1e90 2024-09-15T22:35:26.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4000de90 con 0x55db1feb75d0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.292+0000 7f8c65d33740 10 client.0 _ll_register_callbacks cb 0x7f8c4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c60ef5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42200/0 (socket says 172.21.15.77:42200) 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c60ef5640 1 -- 172.21.15.77:0/3274408606 learned_addr learned my addr 172.21.15.77:0/3274408606 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1ff348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fec5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 -- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 msgr2=0x55db1fec5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fec5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 -- 172.21.15.77:0/3274408606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 msgr2=0x55db1ff343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 -- 172.21.15.77:0/3274408606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1fec5c50 con 0x55db1feb75d0 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:35:26.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c616f6640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1ff348f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55db1fed4060 tx=0x7f8c58012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e382165f55240ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.293+0000 7f8c44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58004020 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f8c580041c0 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c003b60 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c5802d050 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 10 client.29359 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55db1ff3bc30 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c580043c0 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8c5800f040 con 0x55db1feb75d0 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 10 client.29359 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 10 client.29359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 20 client.29359 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:26.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1ff3c3d0 0x55db1ff3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 4 client.29359 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.294+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8c4c003b60 con 0x55db1ff3c3d0 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.295+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1ff3c3d0 0x55db1ff3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.295+0000 7f8c44ff9640 10 client.29359 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.295+0000 7f8c60ef5640 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1ff3c3d0 0x55db1ff3e7f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220280ae9c932420 server_cookie=17dd662131430451 in_seq=0 out_seq=0 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.296+0000 7f8c44ff9640 1 -- 172.21.15.77:0/3274408606 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f8c4c003b60 con 0x55db1ff3c3d0 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.296+0000 7f8c44ff9640 10 client.29359 handle_command_reply: tid=0 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 client.29359 shutdown 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 msgr2=0x55db1ff348f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1ff348f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55db1fed4060 tx=0x7f8c58012820 comp rx=0 tx=0).stop 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 shutdown_connections 2024-09-15T22:35:26.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55db1ff3c3d0 0x55db1ff3e7f0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c44ff9640 0 client.29359 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db1fec1e90 0x55db1fec5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.297+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55db1feb75d0 0x55db1ff348f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.298+0000 7f8c65d33740 1 --2- 172.21.15.77:0/3274408606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db1feb6cb0 0x55db1ff343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.298+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 >> 172.21.15.77:0/3274408606 conn(0x55db1feb3240 msgr2=0x55db1feb8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.298+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 shutdown_connections 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.298+0000 7f8c65d33740 1 -- 172.21.15.77:0/3274408606 wait complete. 2024-09-15T22:35:26.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.298+0000 7f8c65d33740 20 client.29359 trim_cache size 0 max 16384 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "c3beda0e-d5be-42a0-87a6-7c4eb6e253d7": { 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "c3beda0e-d5be-42a0-87a6-7c4eb6e253d7", 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:35:26.305 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 msgr2=0x7f8c480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 0x7f8c480a58f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c54099b90 tx=0x7f8c54013000 comp rx=0 tx=0).stop 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 msgr2=0x7f8c5c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9d0 tx=0x7f8c5400ee90 comp rx=0 tx=0).stop 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 shutdown_connections 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c480a3430 0x7f8c480a58f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c5c114ab0 0x7f8c5c19e730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c5c109ba0 0x7f8c5c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 --2- 172.21.15.77:0/2322494668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c5c1091f0 0x7f8c5c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:26.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.300+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 >> 172.21.15.77:0/2322494668 conn(0x7f8c5c06e2d0 msgr2=0x7f8c5c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:26.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.301+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 shutdown_connections 2024-09-15T22:35:26.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:26.301+0000 7f8c6395a640 1 -- 172.21.15.77:0/2322494668 wait complete. 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:35:26.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: pgmap v727: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:26.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:26.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: pgmap v727: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:26.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:35:26.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: pgmap v727: 129 pgs: 129 active+clean; 1.7 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 312 KiB/s wr, 42 op/s 2024-09-15T22:35:26.363 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:35:26.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 163054 14.39 MB/sec execute 447 sec latency 222.972 ms 2024-09-15T22:35:27.502 INFO:tasks.cephfs.filesystem:scrub status for tag:c3beda0e-d5be-42a0-87a6-7c4eb6e253d7 - {'path': '/', 'tag': 'c3beda0e-d5be-42a0-87a6-7c4eb6e253d7', 'options': 'recursive,force'} 2024-09-15T22:35:27.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 163396 14.39 MB/sec execute 448 sec latency 157.314 ms 2024-09-15T22:35:28.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:28 smithi077 ceph-mon[30980]: pgmap v728: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:28.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:28 smithi196 ceph-mon[35109]: pgmap v728: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:28.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:28 smithi117 ceph-mon[35021]: pgmap v728: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:28.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 163843 14.40 MB/sec execute 449 sec latency 128.375 ms 2024-09-15T22:35:29.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 164306 14.44 MB/sec execute 450 sec latency 151.748 ms 2024-09-15T22:35:30.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:30 smithi196 ceph-mon[35109]: pgmap v729: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:30.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:30 smithi117 ceph-mon[35021]: pgmap v729: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:30.584 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:30 smithi077 ceph-mon[30980]: pgmap v729: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-09-15T22:35:30.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 164690 14.46 MB/sec execute 451 sec latency 386.212 ms 2024-09-15T22:35:31.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:35:31.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:35:31.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:35:31.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:35:31.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:35:31.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:35:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:35:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:35:31.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:35:31.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 165068 14.47 MB/sec execute 452 sec latency 1386.243 ms 2024-09-15T22:35:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:32 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:35:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:32 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:35:32.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:32 smithi196 ceph-mon[35109]: pgmap v730: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-09-15T22:35:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:32 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:35:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:32 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:35:32.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:32 smithi117 ceph-mon[35021]: pgmap v730: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-09-15T22:35:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:32 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:35:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:32 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:35:32.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:32 smithi077 ceph-mon[30980]: pgmap v730: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-09-15T22:35:32.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 165456 14.49 MB/sec execute 453 sec latency 2386.275 ms 2024-09-15T22:35:33.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 165890 14.51 MB/sec execute 454 sec latency 3386.310 ms 2024-09-15T22:35:34.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:34 smithi196 ceph-mon[35109]: pgmap v731: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-09-15T22:35:34.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:34 smithi117 ceph-mon[35021]: pgmap v731: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-09-15T22:35:34.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:34 smithi077 ceph-mon[30980]: pgmap v731: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-09-15T22:35:34.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 166332 14.54 MB/sec execute 455 sec latency 4386.343 ms 2024-09-15T22:35:35.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 166622 14.54 MB/sec execute 456 sec latency 5386.392 ms 2024-09-15T22:35:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:36 smithi196 ceph-mon[35109]: pgmap v732: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-09-15T22:35:36.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:36 smithi117 ceph-mon[35021]: pgmap v732: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-09-15T22:35:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:36 smithi077 ceph-mon[30980]: pgmap v732: 129 pgs: 129 active+clean; 1.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 327 op/s 2024-09-15T22:35:36.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 166881 14.53 MB/sec execute 457 sec latency 5702.163 ms 2024-09-15T22:35:37.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 167247 14.54 MB/sec execute 458 sec latency 140.663 ms 2024-09-15T22:35:38.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:38 smithi196 ceph-mon[35109]: pgmap v733: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2024-09-15T22:35:38.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:38 smithi117 ceph-mon[35021]: pgmap v733: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2024-09-15T22:35:38.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:38 smithi077 ceph-mon[30980]: pgmap v733: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2024-09-15T22:35:38.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 167582 14.55 MB/sec execute 459 sec latency 149.818 ms 2024-09-15T22:35:39.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 167897 14.55 MB/sec execute 460 sec latency 137.103 ms 2024-09-15T22:35:40.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:40 smithi117 ceph-mon[35021]: pgmap v734: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-09-15T22:35:40.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:40 smithi077 ceph-mon[30980]: pgmap v734: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-09-15T22:35:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:40 smithi196 ceph-mon[35109]: pgmap v734: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 291 op/s 2024-09-15T22:35:40.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 168249 14.55 MB/sec execute 461 sec latency 149.516 ms 2024-09-15T22:35:41.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 168469 14.55 MB/sec execute 462 sec latency 505.987 ms 2024-09-15T22:35:42.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:42 smithi077 ceph-mon[30980]: pgmap v735: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 341 op/s 2024-09-15T22:35:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:42 smithi196 ceph-mon[35109]: pgmap v735: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 341 op/s 2024-09-15T22:35:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:42 smithi117 ceph-mon[35021]: pgmap v735: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 341 op/s 2024-09-15T22:35:42.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 168576 14.53 MB/sec execute 463 sec latency 1066.239 ms 2024-09-15T22:35:43.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 168841 14.53 MB/sec execute 464 sec latency 249.198 ms 2024-09-15T22:35:44.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:44 smithi077 ceph-mon[30980]: pgmap v736: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 362 op/s 2024-09-15T22:35:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:44 smithi196 ceph-mon[35109]: pgmap v736: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 362 op/s 2024-09-15T22:35:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:44 smithi117 ceph-mon[35021]: pgmap v736: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 362 op/s 2024-09-15T22:35:44.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 169105 14.53 MB/sec execute 465 sec latency 157.060 ms 2024-09-15T22:35:45.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 169451 14.54 MB/sec execute 466 sec latency 147.369 ms 2024-09-15T22:35:46.393 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:46 smithi077 ceph-mon[30980]: pgmap v737: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.9 MiB/s wr, 280 op/s 2024-09-15T22:35:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:46 smithi196 ceph-mon[35109]: pgmap v737: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.9 MiB/s wr, 280 op/s 2024-09-15T22:35:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:46 smithi117 ceph-mon[35021]: pgmap v737: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.9 MiB/s wr, 280 op/s 2024-09-15T22:35:46.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 169809 14.55 MB/sec execute 467 sec latency 142.318 ms 2024-09-15T22:35:47.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 170180 14.56 MB/sec execute 468 sec latency 149.295 ms 2024-09-15T22:35:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:48 smithi077 ceph-mon[30980]: pgmap v738: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-15T22:35:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:48 smithi196 ceph-mon[35109]: pgmap v738: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-15T22:35:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:48 smithi117 ceph-mon[35021]: pgmap v738: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-09-15T22:35:48.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 170604 14.57 MB/sec execute 469 sec latency 151.515 ms 2024-09-15T22:35:49.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 171119 14.60 MB/sec execute 470 sec latency 138.746 ms 2024-09-15T22:35:50.405 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:50 smithi196 ceph-mon[35109]: pgmap v739: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 410 op/s 2024-09-15T22:35:50.413 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:50 smithi117 ceph-mon[35021]: pgmap v739: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 410 op/s 2024-09-15T22:35:50.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:50 smithi077 ceph-mon[30980]: pgmap v739: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 410 op/s 2024-09-15T22:35:50.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 171565 14.64 MB/sec execute 471 sec latency 149.348 ms 2024-09-15T22:35:51.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 171897 14.65 MB/sec execute 472 sec latency 155.480 ms 2024-09-15T22:35:52.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:52 smithi077 ceph-mon[30980]: pgmap v740: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 444 op/s 2024-09-15T22:35:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:52 smithi196 ceph-mon[35109]: pgmap v740: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 444 op/s 2024-09-15T22:35:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:52 smithi117 ceph-mon[35021]: pgmap v740: 129 pgs: 129 active+clean; 1.8 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 11 MiB/s wr, 444 op/s 2024-09-15T22:35:52.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 172223 14.66 MB/sec execute 473 sec latency 192.510 ms 2024-09-15T22:35:53.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 172565 14.66 MB/sec execute 474 sec latency 143.343 ms 2024-09-15T22:35:54.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:54 smithi077 ceph-mon[30980]: pgmap v741: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-15T22:35:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:54 smithi196 ceph-mon[35109]: pgmap v741: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-15T22:35:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:54 smithi117 ceph-mon[35021]: pgmap v741: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 505 op/s 2024-09-15T22:35:54.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 172894 14.66 MB/sec execute 475 sec latency 161.526 ms 2024-09-15T22:35:55.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 173278 14.66 MB/sec execute 476 sec latency 190.767 ms 2024-09-15T22:35:56.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:56 smithi077 ceph-mon[30980]: pgmap v742: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:35:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:56 smithi196 ceph-mon[35109]: pgmap v742: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:35:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:56 smithi117 ceph-mon[35021]: pgmap v742: 129 pgs: 129 active+clean; 1.8 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:35:56.621 INFO:tasks.workunit.client.0.smithi077.stdout: 10 173681 14.68 MB/sec execute 477 sec latency 214.660 ms 2024-09-15T22:35:57.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:35:57.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 174054 14.69 MB/sec execute 478 sec latency 150.935 ms 2024-09-15T22:35:57.851 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:58.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:35:58 smithi077 ceph-mon[30980]: pgmap v743: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:35:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:35:58 smithi196 ceph-mon[35109]: pgmap v743: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:35:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:35:58 smithi117 ceph-mon[35021]: pgmap v743: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 557 op/s 2024-09-15T22:35:58.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 174433 14.71 MB/sec execute 479 sec latency 195.641 ms 2024-09-15T22:35:58.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 -- 172.21.15.77:0/1228304920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102ef0 msgr2=0x7f89f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 --2- 172.21.15.77:0/1228304920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102ef0 0x7f89f010da40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009c10 tx=0x7f89e802a770 comp rx=0 tx=0).stop 2024-09-15T22:35:58.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 -- 172.21.15.77:0/1228304920 shutdown_connections 2024-09-15T22:35:58.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 --2- 172.21.15.77:0/1228304920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102ef0 0x7f89f010da40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 --2- 172.21.15.77:0/1228304920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102570 0x7f89f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 --2- 172.21.15.77:0/1228304920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 0x7f89f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 -- 172.21.15.77:0/1228304920 >> 172.21.15.77:0/1228304920 conn(0x7f89f00fc7d0 msgr2=0x7f89f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.721+0000 7f89f6253640 1 -- 172.21.15.77:0/1228304920 shutdown_connections 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.722+0000 7f89f6253640 1 -- 172.21.15.77:0/1228304920 wait complete. 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.722+0000 7f89f6253640 1 Processor -- start 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.722+0000 7f89f6253640 1 -- start start 2024-09-15T22:35:58.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 0x7f89f0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102ef0 0x7f89f019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0110f80 con 0x7f89f0100e20 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0110e00 con 0x7f89f0102ef0 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89f6253640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0111100 con 0x7f89f0102570 2024-09-15T22:35:58.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 0x7f89f0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55660/0 (socket says 172.21.15.77:55660) 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 -- 172.21.15.77:0/685334772 learned_addr learned my addr 172.21.15.77:0/685334772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89effff640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102ef0 0x7f89f019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 -- 172.21.15.77:0/685334772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102ef0 msgr2=0x7f89f019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102ef0 0x7f89f019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 -- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 msgr2=0x7f89f0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 0x7f89f0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.723+0000 7f89eeffd640 1 -- 172.21.15.77:0/685334772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019d4b0 con 0x7f89f0102570 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89eeffd640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e9f0 tx=0x7f89e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89ecff9640 1 -- 172.21.15.77:0/685334772 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400ccc0 con 0x7f89f0102570 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89ecff9640 1 -- 172.21.15.77:0/685334772 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e400ce60 con 0x7f89f0102570 2024-09-15T22:35:58.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89ecff9640 1 -- 172.21.15.77:0/685334772 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4010690 con 0x7f89f0102570 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019d7a0 con 0x7f89f0102570 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.724+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f01a5080 con 0x7f89f0102570 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89ecff9640 1 -- 172.21.15.77:0/685334772 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f89e40046c0 con 0x7f89f0102570 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 Processor -- start 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 -- start start 2024-09-15T22:35:58.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 0x55c2e05af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05ba680 0x55c2e05bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2e05152e0 con 0x55c2e05ba680 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2e04bcce0 con 0x55c2e05afdc0 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.727+0000 7f89f862c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2e05112a0 con 0x55c2e05af4a0 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89ecff9640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 0x7f89d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89effff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 0x55c2e05af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89effff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 0x55c2e05af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55668/0 (socket says 172.21.15.77:55668) 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89effff640 1 -- 172.21.15.77:0/473414441 learned_addr learned my addr 172.21.15.77:0/473414441 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34338/0 (socket says 172.21.15.77:34338) 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89ecff9640 1 -- 172.21.15.77:0/685334772 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f89e40d0570 con 0x7f89f0102570 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.728+0000 7f89eeffd640 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05ba680 0x55c2e05bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 -- 172.21.15.77:0/473414441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 msgr2=0x55c2e05af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 0x55c2e05af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89effff640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 0x7f89d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 -- 172.21.15.77:0/473414441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05ba680 msgr2=0x55c2e05bca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05ba680 0x55c2e05bca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 -- 172.21.15.77:0/473414441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e04a1440 con 0x55c2e05afdc0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009bb0 tx=0x7f89e0013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dba4338a83f62b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89d5ffb640 1 -- 172.21.15.77:0/473414441 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0009d60 con 0x55c2e05afdc0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89d5ffb640 1 -- 172.21.15.77:0/473414441 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e000e990 con 0x55c2e05afdc0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.729+0000 7f89d5ffb640 1 -- 172.21.15.77:0/473414441 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0003ab0 con 0x55c2e05afdc0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 -- 172.21.15.77:0/473414441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 msgr2=0x55c2e05ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89effff640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 0x7f89d80a5870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009370 tx=0x7f89e8005fd0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009bb0 tx=0x7f89e0013ba0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 -- 172.21.15.77:0/473414441 shutdown_connections 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05ba680 0x55c2e05bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05afdc0 0x55c2e05ba140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 --2- 172.21.15.77:0/473414441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05af4a0 0x55c2e05af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 -- 172.21.15.77:0/473414441 >> 172.21.15.77:0/473414441 conn(0x55c2e05aba30 msgr2=0x55c2e05abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 -- 172.21.15.77:0/473414441 shutdown_connections 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.730+0000 7f89f862c740 1 -- 172.21.15.77:0/473414441 wait complete. 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 -- start start 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 Processor -- start 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05af4a0 0x55c2e05b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05ba680 0x55c2e05b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2e05bd250 con 0x55c2e05af4a0 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2e04d0050 con 0x55c2e05ba680 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c2e05bd3d0 con 0x55c2e05afdc0 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.731+0000 7f89f862c740 10 client.0 _ll_register_callbacks cb 0x55c2e0510160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55670/0 (socket says 172.21.15.77:55670) 2024-09-15T22:35:58.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 -- 172.21.15.77:0/4153983486 learned_addr learned my addr 172.21.15.77:0/4153983486 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05af4a0 0x55c2e05b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89effff640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05ba680 0x55c2e05b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 -- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05ba680 msgr2=0x55c2e05b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05ba680 0x55c2e05b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 -- 172.21.15.77:0/4153983486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05af4a0 msgr2=0x55c2e05b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05af4a0 0x55c2e05b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 -- 172.21.15.77:0/4153983486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e02863c0 con 0x55c2e05afdc0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89eeffd640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89e409c450 tx=0x7f89e409cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc40ce9634b90c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40b4020 con 0x55c2e05afdc0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e000e990 con 0x55c2e05afdc0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f89e40d0360 con 0x55c2e05afdc0 2024-09-15T22:35:58.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89e40a5130 con 0x55c2e05afdc0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89f862c740 10 client.29371 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.732+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2e05c8c80 con 0x55c2e05afdc0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.733+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e000e990 con 0x55c2e05afdc0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.733+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f89e40b5220 con 0x55c2e05afdc0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 10 client.29371 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 10 client.29371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 20 client.29371 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ca090 0x55c2e05cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 4 client.29371 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e0522850 con 0x55c2e05ca090 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ca090 0x55c2e05cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ca090 0x55c2e05cc4a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b1abcba44828a6 server_cookie=8fa0b628c1ae01b9 in_seq=0 out_seq=0 2024-09-15T22:35:58.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.734+0000 7f89ceffd640 10 client.29371 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.736+0000 7f89ceffd640 1 -- 172.21.15.77:0/4153983486 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2e0522850 con 0x55c2e05ca090 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.736+0000 7f89ceffd640 10 client.29371 handle_command_reply: tid=0 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.736+0000 7f89f862c740 1 client.29371 shutdown 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 msgr2=0x55c2e05b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89e409c450 tx=0x7f89e409cc80 comp rx=0 tx=0).stop 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 shutdown_connections 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ca090 0x55c2e05cc4a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05ba680 0x55c2e05b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05b7240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 --2- 172.21.15.77:0/4153983486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05af4a0 0x55c2e05b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89ceffd640 0 client.29371 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 >> 172.21.15.77:0/4153983486 conn(0x55c2e05aba30 msgr2=0x55c2e05aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 shutdown_connections 2024-09-15T22:35:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 1 -- 172.21.15.77:0/4153983486 wait complete. 2024-09-15T22:35:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.737+0000 7f89f862c740 20 client.29371 trim_cache size 0 max 16384 2024-09-15T22:35:58.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 Processor -- start 2024-09-15T22:35:58.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 -- start start 2024-09-15T22:35:58.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01acdb0 con 0x55c2e05aba30 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01ae8f0 con 0x55c2e05af4a0 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89f862c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01ad480 con 0x55c2e05afdc0 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39432/0 (socket says 172.21.15.77:39432) 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89ef7fe640 1 -- 172.21.15.77:0/2715736677 learned_addr learned my addr 172.21.15.77:0/2715736677 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89eeffd640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.757+0000 7f89effff640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 -- 172.21.15.77:0/2715736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 msgr2=0x55c2e05d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 -- 172.21.15.77:0/2715736677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 msgr2=0x55c2e05b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 -- 172.21.15.77:0/2715736677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e05af2d0 con 0x55c2e05aba30 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89effff640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f89e00057c0 tx=0x7f89e0013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fba571de672e10d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89d57fa640 1 -- 172.21.15.77:0/2715736677 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0022040 con 0x55c2e05aba30 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89d57fa640 1 -- 172.21.15.77:0/2715736677 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e0005b30 con 0x55c2e05aba30 2024-09-15T22:35:58.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.758+0000 7f89d57fa640 1 -- 172.21.15.77:0/2715736677 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0013310 con 0x55c2e05aba30 2024-09-15T22:35:58.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 -- 172.21.15.77:0/2715736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 msgr2=0x55c2e05b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f89e00057c0 tx=0x7f89e0013ba0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 -- 172.21.15.77:0/2715736677 shutdown_connections 2024-09-15T22:35:58.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 --2- 172.21.15.77:0/2715736677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05b7200 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 -- 172.21.15.77:0/2715736677 >> 172.21.15.77:0/2715736677 conn(0x55c2e05b78e0 msgr2=0x55c2e05b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 -- 172.21.15.77:0/2715736677 shutdown_connections 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.759+0000 7f89f862c740 1 -- 172.21.15.77:0/2715736677 wait complete. 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.760+0000 7f89f862c740 1 -- start start 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.760+0000 7f89f862c740 1 Processor -- start 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01ae220 con 0x55c2e05aba30 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01adb50 con 0x55c2e05af4a0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01aadb0 con 0x55c2e05afdc0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89f862c740 10 client.0 _ll_register_callbacks cb 0x7f89e4098760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89effff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:34362/0 (socket says 172.21.15.77:34362) 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 -- 172.21.15.77:0/1046627811 learned_addr learned my addr 172.21.15.77:0/1046627811 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89effff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05c1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55692/0 (socket says 172.21.15.77:55692) 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 -- 172.21.15.77:0/1046627811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 msgr2=0x55c2e05c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 -- 172.21.15.77:0/1046627811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 msgr2=0x55c2e05c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.761+0000 7f89eeffd640 1 -- 172.21.15.77:0/1046627811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e0633f20 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89eeffd640 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89e4098e90 tx=0x7f89e40b5450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2df31a54af9423 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40aa070 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e0005b30 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f89e40d0160 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89e40b4040 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89f862c740 10 client.29447 fetch_fsmap learned FSMap version 39 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2e06351f0 con 0x55c2e05af4a0 2024-09-15T22:35:58.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.762+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e0005b30 con 0x55c2e05af4a0 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f89e40ad020 con 0x55c2e05af4a0 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 10 client.29447 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 10 client.29447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 20 client.29447 populate_metadata read hostname 'smithi077' 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ba680 0x55c2e0637870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 4 client.29447 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2e051cd60 con 0x55c2e05ba680 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ba680 0x55c2e0637870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89ef7fe640 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ba680 0x55c2e0637870 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13fa591061f1866a server_cookie=ff674123aa3153a in_seq=0 out_seq=0 2024-09-15T22:35:58.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.763+0000 7f89ceffd640 10 client.29447 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89ceffd640 1 -- 172.21.15.77:0/1046627811 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2e051cd60 con 0x55c2e05ba680 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89ceffd640 10 client.29447 handle_command_reply: tid=0 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 client.29447 shutdown 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 msgr2=0x55c2e05c0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89e4098e90 tx=0x7f89e40b5450 comp rx=0 tx=0).stop 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 shutdown_connections 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55c2e05ba680 0x55c2e0637870 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c2e05afdc0 0x55c2e05c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89ceffd640 0 client.29447 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2e05af4a0 0x55c2e05c0b20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 --2- 172.21.15.77:0/1046627811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2e05aba30 0x55c2e05c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 >> 172.21.15.77:0/1046627811 conn(0x55c2e05b78e0 msgr2=0x55c2e05b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 shutdown_connections 2024-09-15T22:35:58.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 1 -- 172.21.15.77:0/1046627811 wait complete. 2024-09-15T22:35:58.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.765+0000 7f89f862c740 20 client.29447 trim_cache size 0 max 16384 2024-09-15T22:35:58.770 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:35:58.770 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:35:58.770 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:35:58.770 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 msgr2=0x7f89d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 0x7f89d80a5870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009370 tx=0x7f89e8005fd0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 msgr2=0x7f89f0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f89e400e9f0 tx=0x7f89e400edf0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 shutdown_connections 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89d80a33b0 0x7f89d80a5870 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89f0102ef0 0x7f89f019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f0102570 0x7f89f0198a40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 --2- 172.21.15.77:0/685334772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0100e20 0x7f89f0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 >> 172.21.15.77:0/685334772 conn(0x7f89f00fc7d0 msgr2=0x7f89f006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:58.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 shutdown_connections 2024-09-15T22:35:58.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:35:58.766+0000 7f89f6253640 1 -- 172.21.15.77:0/685334772 wait complete. 2024-09-15T22:35:59.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 174750 14.71 MB/sec execute 480 sec latency 153.423 ms 2024-09-15T22:35:59.841 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:35:59.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:00.151 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:00.421 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:00 smithi077 ceph-mon[30980]: pgmap v744: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-15T22:36:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:00 smithi196 ceph-mon[35109]: pgmap v744: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-15T22:36:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:00 smithi117 ceph-mon[35021]: pgmap v744: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-15T22:36:00.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 175192 14.72 MB/sec execute 481 sec latency 166.495 ms 2024-09-15T22:36:01.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:01 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3421158997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.566 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8afdffb640 1 -- 172.21.15.77:0/2046821262 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4037070 con 0x7f8b00100de0 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 -- 172.21.15.77:0/2046821262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100de0 msgr2=0x7f8b00104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 --2- 172.21.15.77:0/2046821262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100de0 0x7f8b00104e30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009860 tx=0x7f8af402a380 comp rx=0 tx=0).stop 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 -- 172.21.15.77:0/2046821262 shutdown_connections 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 --2- 172.21.15.77:0/2046821262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b00105560 0x7f8b0010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 --2- 172.21.15.77:0/2046821262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100de0 0x7f8b00104e30 secure :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009860 tx=0x7f8af402a380 comp rx=0 tx=0).stop 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 --2- 172.21.15.77:0/2046821262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00100430 0x7f8b00100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.715+0000 7f8b0576a640 1 -- 172.21.15.77:0/2046821262 >> 172.21.15.77:0/2046821262 conn(0x7f8b00077230 msgr2=0x7f8b000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.716+0000 7f8b0576a640 1 -- 172.21.15.77:0/2046821262 shutdown_connections 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.716+0000 7f8b0576a640 1 -- 172.21.15.77:0/2046821262 wait complete. 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.716+0000 7f8b0576a640 1 Processor -- start 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 -- start start 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105560 0x7f8b00108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b00112c60 0x7f8b001150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0006d170 con 0x7f8b00100430 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0006cff0 con 0x7f8b00112c60 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.717+0000 7f8b0576a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0006d2f0 con 0x7f8b00105560 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.721+0000 7f8afeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:01.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.721+0000 7f8afeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50170/0 (socket says 172.21.15.77:50170) 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.721+0000 7f8afeffd640 1 -- 172.21.15.77:0/3421158997 learned_addr learned my addr 172.21.15.77:0/3421158997 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.729+0000 7f8afeffd640 1 -- 172.21.15.77:0/3421158997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105560 msgr2=0x7f8b00108ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.729+0000 7f8afeffd640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105560 0x7f8b00108ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.729+0000 7f8afeffd640 1 -- 172.21.15.77:0/3421158997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b00112c60 msgr2=0x7f8b001150b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.729+0000 7f8afeffd640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b00112c60 0x7f8b001150b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.729+0000 7f8afeffd640 1 -- 172.21.15.77:0/3421158997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b001155f0 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.735+0000 7f8afeffd640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b450 tx=0x7f8af000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.739+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0015020 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.739+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af0011d70 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.740+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000f340 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.740+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b001158e0 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.740+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b00115da0 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.745+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b00101910 con 0x7f8b00100430 2024-09-15T22:36:01.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.751+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8af0018020 con 0x7f8b00100430 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.752+0000 7f8ae7fff640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 0x7f8ae80a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.752+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9438+0+0 (secure 0 0 0) 0x7f8af00cfff0 con 0x7f8b00100430 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.757+0000 7f8afe7fc640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 0x7f8ae80a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.762+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8af000d040 con 0x7f8b00100430 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:00.767+0000 7f8afe7fc640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 0x7f8ae80a57d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b00109f50 tx=0x7f8af4005d60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.069+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b001160c0 con 0x7f8b00100430 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.069+0000 7f8ae7fff640 1 -- 172.21.15.77:0/3421158997 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f8af009c750 con 0x7f8b00100430 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 msgr2=0x7f8ae80a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 0x7f8ae80a57d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b00109f50 tx=0x7f8af4005d60 comp rx=0 tx=0).stop 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 msgr2=0x7f8b00108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b450 tx=0x7f8af000b910 comp rx=0 tx=0).stop 2024-09-15T22:36:01.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 shutdown_connections 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ae80a3310 0x7f8ae80a57d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b00112c60 0x7f8b001150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105560 0x7f8b00108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 --2- 172.21.15.77:0/3421158997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b00100430 0x7f8b00108960 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 >> 172.21.15.77:0/3421158997 conn(0x7f8b00077230 msgr2=0x7f8b00102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 shutdown_connections 2024-09-15T22:36:01.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:01.072+0000 7f8b0576a640 1 -- 172.21.15.77:0/3421158997 wait complete. 2024-09-15T22:36:01.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:01 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3421158997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:01 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3421158997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 175236 14.69 MB/sec execute 482 sec latency 777.204 ms 2024-09-15T22:36:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:02 smithi077 ceph-mon[30980]: pgmap v745: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.4 MiB/s wr, 305 op/s 2024-09-15T22:36:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:02 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:02 smithi196 ceph-mon[35109]: pgmap v745: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.4 MiB/s wr, 305 op/s 2024-09-15T22:36:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:02 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:02 smithi117 ceph-mon[35021]: pgmap v745: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.4 MiB/s wr, 305 op/s 2024-09-15T22:36:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:02 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 175364 14.68 MB/sec execute 483 sec latency 1249.262 ms 2024-09-15T22:36:03.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 175586 14.67 MB/sec execute 484 sec latency 284.467 ms 2024-09-15T22:36:04.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:04 smithi077 ceph-mon[30980]: pgmap v746: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-15T22:36:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:04 smithi196 ceph-mon[35109]: pgmap v746: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-15T22:36:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:04 smithi117 ceph-mon[35021]: pgmap v746: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-15T22:36:04.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 175868 14.69 MB/sec execute 485 sec latency 373.027 ms 2024-09-15T22:36:05.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:05.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:05.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:36:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:36:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:36:05.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 176178 14.70 MB/sec execute 486 sec latency 215.580 ms 2024-09-15T22:36:06.056 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:36:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:06.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 176332 14.68 MB/sec execute 487 sec latency 320.191 ms 2024-09-15T22:36:06.795 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:06 smithi077 ceph-mon[30980]: pgmap v747: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-15T22:36:06.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:06 smithi196 ceph-mon[35109]: pgmap v747: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-15T22:36:06.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:06 smithi117 ceph-mon[35021]: pgmap v747: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.0 MiB/s wr, 290 op/s 2024-09-15T22:36:07.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.121+0000 7f3c23b2f640 1 -- 172.21.15.77:0/2795958171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 msgr2=0x7f3c1c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.121+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/2795958171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c106340 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009900 tx=0x7f3c0c02a440 comp rx=0 tx=0).stop 2024-09-15T22:36:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.123+0000 7f3c23b2f640 1 -- 172.21.15.77:0/2795958171 shutdown_connections 2024-09-15T22:36:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.123+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/2795958171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c106340 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.123+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/2795958171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 0x7f3c1c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.123+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/2795958171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c1c1008c0 0x7f3c1c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.123+0000 7f3c23b2f640 1 -- 172.21.15.77:0/2795958171 >> 172.21.15.77:0/2795958171 conn(0x7f3c1c0fc7d0 msgr2=0x7f3c1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.125+0000 7f3c23b2f640 1 -- 172.21.15.77:0/2795958171 shutdown_connections 2024-09-15T22:36:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.125+0000 7f3c23b2f640 1 -- 172.21.15.77:0/2795958171 wait complete. 2024-09-15T22:36:07.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 Processor -- start 2024-09-15T22:36:07.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 -- start start 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c1c1008c0 0x7f3c1c114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 0x7f3c1c1151e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1187a0 con 0x7f3c1c105ec0 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c118620 con 0x7f3c1c1008c0 2024-09-15T22:36:07.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.126+0000 7f3c23b2f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c118920 con 0x7f3c1c1011e0 2024-09-15T22:36:07.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.128+0000 7f3c220cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.128+0000 7f3c220cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50186/0 (socket says 172.21.15.77:50186) 2024-09-15T22:36:07.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.128+0000 7f3c220cc640 1 -- 172.21.15.77:0/661878037 learned_addr learned my addr 172.21.15.77:0/661878037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:07.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.128+0000 7f3c210ca640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 0x7f3c1c1151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.131+0000 7f3c220cc640 1 -- 172.21.15.77:0/661878037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 msgr2=0x7f3c1c1151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.131+0000 7f3c220cc640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 0x7f3c1c1151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.131+0000 7f3c220cc640 1 -- 172.21.15.77:0/661878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c1c1008c0 msgr2=0x7f3c1c114ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:36:07.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.131+0000 7f3c220cc640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c1c1008c0 0x7f3c1c114ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.131+0000 7f3c220cc640 1 -- 172.21.15.77:0/661878037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c10e7d0 con 0x7f3c1c105ec0 2024-09-15T22:36:07.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.133+0000 7f3c220cc640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006010 tx=0x7f3c0c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.136+0000 7f3c12ffd640 1 -- 172.21.15.77:0/661878037 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c03f070 con 0x7f3c1c105ec0 2024-09-15T22:36:07.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.136+0000 7f3c12ffd640 1 -- 172.21.15.77:0/661878037 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c031c20 con 0x7f3c1c105ec0 2024-09-15T22:36:07.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.136+0000 7f3c12ffd640 1 -- 172.21.15.77:0/661878037 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c035030 con 0x7f3c1c105ec0 2024-09-15T22:36:07.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.136+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c10ea60 con 0x7f3c1c105ec0 2024-09-15T22:36:07.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.136+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1129a0 con 0x7f3c1c105ec0 2024-09-15T22:36:07.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.141+0000 7f3c25f08740 1 Processor -- start 2024-09-15T22:36:07.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.142+0000 7f3c12ffd640 1 -- 172.21.15.77:0/661878037 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3c0c042070 con 0x7f3c1c105ec0 2024-09-15T22:36:07.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.143+0000 7f3c12ffd640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 0x7f3c080a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.143+0000 7f3c12ffd640 1 -- 172.21.15.77:0/661878037 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f3c0c0f03b0 con 0x7f3c1c105ec0 2024-09-15T22:36:07.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.143+0000 7f3c210ca640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 0x7f3c080a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 -- start start 2024-09-15T22:36:07.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea781090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea78e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea6e72f0 con 0x55d7ea7815d0 2024-09-15T22:36:07.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea68ecf0 con 0x55d7ea782b40 2024-09-15T22:36:07.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c25f08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea6e32b0 con 0x55d7ea78c370 2024-09-15T22:36:07.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50192/0 (socket says 172.21.15.77:50192) 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.144+0000 7f3c210ca640 1 -- 172.21.15.77:0/3717555851 learned_addr learned my addr 172.21.15.77:0/3717555851 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.145+0000 7f3c210ca640 1 -- 172.21.15.77:0/3717555851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 msgr2=0x55d7ea78e760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.145+0000 7f3c210ca640 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea78e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.145+0000 7f3c210ca640 1 -- 172.21.15.77:0/3717555851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 msgr2=0x55d7ea781090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.145+0000 7f3c210ca640 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea781090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.145+0000 7f3c210ca640 1 -- 172.21.15.77:0/3717555851 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ea6733a0 con 0x55d7ea7815d0 2024-09-15T22:36:07.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.146+0000 7f3c210ca640 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3c180091b0 tx=0x7f3c18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a97fa26151d0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.146+0000 7f3c210ca640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 0x7f3c080a5a80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009d90 tx=0x7f3c18014660 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.147+0000 7f3c07fff640 1 -- 172.21.15.77:0/3717555851 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1801d070 con 0x55d7ea7815d0 2024-09-15T22:36:07.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.147+0000 7f3c07fff640 1 -- 172.21.15.77:0/3717555851 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c18011aa0 con 0x55d7ea7815d0 2024-09-15T22:36:07.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.147+0000 7f3c07fff640 1 -- 172.21.15.77:0/3717555851 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18021b90 con 0x55d7ea7815d0 2024-09-15T22:36:07.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.148+0000 7f3c25f08740 1 -- 172.21.15.77:0/3717555851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 msgr2=0x55d7ea781a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.148+0000 7f3c25f08740 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3c180091b0 tx=0x7f3c18017000 comp rx=0 tx=0).stop 2024-09-15T22:36:07.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.152+0000 7f3c25f08740 1 -- 172.21.15.77:0/3717555851 shutdown_connections 2024-09-15T22:36:07.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.152+0000 7f3c25f08740 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea78e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.152+0000 7f3c25f08740 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea781a10 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.152+0000 7f3c25f08740 1 --2- 172.21.15.77:0/3717555851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea781090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.152+0000 7f3c25f08740 1 -- 172.21.15.77:0/3717555851 >> 172.21.15.77:0/3717555851 conn(0x55d7ea74a310 msgr2=0x55d7ea77dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.160+0000 7f3c25f08740 1 -- 172.21.15.77:0/3717555851 shutdown_connections 2024-09-15T22:36:07.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.161+0000 7f3c25f08740 1 -- 172.21.15.77:0/3717555851 wait complete. 2024-09-15T22:36:07.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.162+0000 7f3c25f08740 1 -- start start 2024-09-15T22:36:07.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.162+0000 7f3c25f08740 1 Processor -- start 2024-09-15T22:36:07.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea78bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea7894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea78efb0 con 0x55d7ea7815d0 2024-09-15T22:36:07.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea6a2060 con 0x55d7ea782b40 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.165+0000 7f3c25f08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d7ea78f130 con 0x55d7ea78c370 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.166+0000 7f3c25f08740 10 client.0 _ll_register_callbacks cb 0x55d7ea3f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.166+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.166+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:58096/0 (socket says 172.21.15.77:58096) 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.166+0000 7f3c210ca640 1 -- 172.21.15.77:0/512832216 learned_addr learned my addr 172.21.15.77:0/512832216 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.167+0000 7f3c210ca640 1 -- 172.21.15.77:0/512832216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 msgr2=0x55d7ea7894f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.167+0000 7f3c210ca640 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea7894f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.167+0000 7f3c210ca640 1 -- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 msgr2=0x55d7ea78bce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.167+0000 7f3c210ca640 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea78bce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.167+0000 7f3c210ca640 1 -- 172.21.15.77:0/512832216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ea4583c0 con 0x55d7ea782b40 2024-09-15T22:36:07.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.168+0000 7f3c210ca640 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c625284c4a5fe5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.168+0000 7f3c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:36:07.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.170+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800f040 con 0x55d7ea782b40 2024-09-15T22:36:07.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.170+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f3c18005990 con 0x55d7ea782b40 2024-09-15T22:36:07.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.171+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c18011aa0 con 0x55d7ea782b40 2024-09-15T22:36:07.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.172+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c1801d040 con 0x55d7ea782b40 2024-09-15T22:36:07.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.172+0000 7f3c25f08740 10 client.29453 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:07.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.172+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d7ea79aa90 con 0x55d7ea782b40 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.174+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c180118c0 con 0x55d7ea782b40 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3c18003a00 con 0x55d7ea782b40 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 10 client.29453 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 10 client.29453 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 20 client.29453 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea79bef0 0x55d7ea79e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 4 client.29453 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:07.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.175+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3c18011aa0 con 0x55d7ea79bef0 2024-09-15T22:36:07.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.180+0000 7f3c218cb640 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea79bef0 0x55d7ea79e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.186+0000 7f3c218cb640 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea79bef0 0x55d7ea79e2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9d37640aca6e1b7 server_cookie=e64b461416a23c24 in_seq=0 out_seq=0 2024-09-15T22:36:07.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.186+0000 7f3c057fa640 10 client.29453 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:36:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.194+0000 7f3c057fa640 1 -- 172.21.15.77:0/512832216 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3c18011aa0 con 0x55d7ea79bef0 2024-09-15T22:36:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.194+0000 7f3c057fa640 10 client.29453 handle_command_reply: tid=0 2024-09-15T22:36:07.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.194+0000 7f3c25f08740 1 client.29453 shutdown 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 msgr2=0x55d7ea788f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18009710 comp rx=0 tx=0).stop 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 shutdown_connections 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea79bef0 0x55d7ea79e2e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea78c370 0x55d7ea7894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea782b40 0x55d7ea788f90 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18009710 comp rx=0 tx=0).stop 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 --2- 172.21.15.77:0/512832216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea7815d0 0x55d7ea78bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 >> 172.21.15.77:0/512832216 conn(0x55d7ea74a310 msgr2=0x55d7ea783930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.196+0000 7f3c057fa640 0 client.29453 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:36:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.197+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 shutdown_connections 2024-09-15T22:36:07.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.198+0000 7f3c25f08740 1 -- 172.21.15.77:0/512832216 wait complete. 2024-09-15T22:36:07.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.201+0000 7f3c25f08740 20 client.29453 trim_cache size 0 max 16384 2024-09-15T22:36:07.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.227+0000 7f3c25f08740 1 Processor -- start 2024-09-15T22:36:07.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 -- start start 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 0x55d7ea798320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea7a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 0x55d7ea7a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00010080 con 0x55d7ea78c370 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00011bc0 con 0x55d7ea7815d0 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.228+0000 7f3c25f08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00010750 con 0x55d7ea7895c0 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.229+0000 7f3c220cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 0x55d7ea7a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.229+0000 7f3c218cb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 0x55d7ea798320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.229+0000 7f3c220cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 0x55d7ea7a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:55430/0 (socket says 172.21.15.77:55430) 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.229+0000 7f3c220cc640 1 -- 172.21.15.77:0/4251596046 learned_addr learned my addr 172.21.15.77:0/4251596046 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:07.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.230+0000 7f3c218cb640 1 -- 172.21.15.77:0/4251596046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 msgr2=0x55d7ea7a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:07.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.230+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 0x55d7ea7a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.230+0000 7f3c218cb640 1 -- 172.21.15.77:0/4251596046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 msgr2=0x55d7ea7a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:07.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.230+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea7a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.230+0000 7f3c218cb640 1 -- 172.21.15.77:0/4251596046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ea78c050 con 0x55d7ea7815d0 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.233+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 0x55d7ea798320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c14000f80 tx=0x7f3c14009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7375a83b951ad260 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.235+0000 7f3c077fe640 1 -- 172.21.15.77:0/4251596046 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14017460 con 0x55d7ea7815d0 2024-09-15T22:36:07.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.235+0000 7f3c077fe640 1 -- 172.21.15.77:0/4251596046 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1401c070 con 0x55d7ea7815d0 2024-09-15T22:36:07.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.235+0000 7f3c077fe640 1 -- 172.21.15.77:0/4251596046 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14016bc0 con 0x55d7ea7815d0 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.236+0000 7f3c25f08740 1 -- 172.21.15.77:0/4251596046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 msgr2=0x55d7ea798320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.236+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 0x55d7ea798320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c14000f80 tx=0x7f3c14009230 comp rx=0 tx=0).stop 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.237+0000 7f3c25f08740 1 -- 172.21.15.77:0/4251596046 shutdown_connections 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.237+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea7895c0 0x55d7ea7a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.237+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea7a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.237+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4251596046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7815d0 0x55d7ea798320 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c14000f80 tx=0x7f3c14009230 comp rx=0 tx=0).stop 2024-09-15T22:36:07.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.237+0000 7f3c25f08740 1 -- 172.21.15.77:0/4251596046 >> 172.21.15.77:0/4251596046 conn(0x55d7ea74a310 msgr2=0x55d7ea788080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.241+0000 7f3c25f08740 1 -- 172.21.15.77:0/4251596046 shutdown_connections 2024-09-15T22:36:07.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.244+0000 7f3c25f08740 1 -- 172.21.15.77:0/4251596046 wait complete. 2024-09-15T22:36:07.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.245+0000 7f3c25f08740 1 -- start start 2024-09-15T22:36:07.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.245+0000 7f3c25f08740 1 Processor -- start 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.248+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7895c0 0x55d7ea794b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.248+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.248+0000 7f3c25f08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea786160 0x55d7ea7865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.248+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c25f08740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c000114f0 con 0x55d7ea78c370 2024-09-15T22:36:07.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50208/0 (socket says 172.21.15.77:50208) 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 -- 172.21.15.77:0/4195720025 learned_addr learned my addr 172.21.15.77:0/4195720025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7895c0 0x55d7ea794b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c220cc640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea786160 0x55d7ea7865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00010e20 con 0x55d7ea7895c0 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0000e080 con 0x55d7ea786160 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 -- 172.21.15.77:0/4195720025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea786160 msgr2=0x55d7ea7865e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea786160 0x55d7ea7865e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 -- 172.21.15.77:0/4195720025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7895c0 msgr2=0x55d7ea794b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7895c0 0x55d7ea794b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.249+0000 7f3c210ca640 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ea8059b0 con 0x55d7ea78c370 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.250+0000 7f3c25f08740 10 client.0 _ll_register_callbacks cb 0x7f3c18009100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.250+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1401c070 con 0x55d7ea78c370 2024-09-15T22:36:07.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.250+0000 7f3c210ca640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3c18003e50 tx=0x7f3c18011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa536815ea7bb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:07.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.250+0000 7f3c057fa640 10 client.20805 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:07.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.254+0000 7f3c057fa640 1 -- 172.21.15.77:0/4195720025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18021610 con 0x55d7ea78c370 2024-09-15T22:36:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.254+0000 7f3c057fa640 1 -- 172.21.15.77:0/4195720025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f3c18005990 con 0x55d7ea78c370 2024-09-15T22:36:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.254+0000 7f3c057fa640 1 -- 172.21.15.77:0/4195720025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c18005bf0 con 0x55d7ea78c370 2024-09-15T22:36:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.254+0000 7f3c25f08740 10 client.20805 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:07.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.254+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55d7ea806ab0 con 0x55d7ea78c370 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.257+0000 7f3c057fa640 1 -- 172.21.15.77:0/4195720025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3c1803b030 con 0x55d7ea78c370 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 10 client.20805 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 10 client.20805 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 20 client.20805 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea807070 0x55d7ea809490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:07.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 4 client.20805 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:36:07.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.258+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7ea6f4860 con 0x55d7ea807070 2024-09-15T22:36:07.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.260+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea807070 0x55d7ea809490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:07.268 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.267+0000 7f3c218cb640 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea807070 0x55d7ea809490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7ec281502c91eeb server_cookie=e324f8c49400316e in_seq=0 out_seq=0 2024-09-15T22:36:07.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.267+0000 7f3c057fa640 10 client.20805 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:36:07.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.274+0000 7f3c057fa640 1 -- 172.21.15.77:0/4195720025 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7ea6f4860 con 0x55d7ea807070 2024-09-15T22:36:07.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.274+0000 7f3c057fa640 10 client.20805 handle_command_reply: tid=0 2024-09-15T22:36:07.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.274+0000 7f3c25f08740 1 client.20805 shutdown 2024-09-15T22:36:07.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 msgr2=0x55d7ea785c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3c18003e50 tx=0x7f3c18011a60 comp rx=0 tx=0).stop 2024-09-15T22:36:07.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 shutdown_connections 2024-09-15T22:36:07.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55d7ea807070 0x55d7ea809490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d7ea786160 0x55d7ea7865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ea78c370 0x55d7ea785c20 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 --2- 172.21.15.77:0/4195720025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d7ea7895c0 0x55d7ea794b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c057fa640 0 client.20805 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:36:07.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.275+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 >> 172.21.15.77:0/4195720025 conn(0x55d7ea74a310 msgr2=0x55d7ea783110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.276+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 shutdown_connections 2024-09-15T22:36:07.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.276+0000 7f3c25f08740 1 -- 172.21.15.77:0/4195720025 wait complete. 2024-09-15T22:36:07.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.277+0000 7f3c25f08740 20 client.20805 trim_cache size 0 max 16384 2024-09-15T22:36:07.281 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 msgr2=0x7f3c080a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 0x7f3c080a5a80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009d90 tx=0x7f3c18014660 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 msgr2=0x7f3c1c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006010 tx=0x7f3c0c003b20 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 shutdown_connections 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f3c080a35c0 0x7f3c080a5a80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c1c105ec0 0x7f3c1c10df10 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c1c1011e0 0x7f3c1c1151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 --2- 172.21.15.77:0/661878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c1c1008c0 0x7f3c1c114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.280+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 >> 172.21.15.77:0/661878037 conn(0x7f3c1c0fc7d0 msgr2=0x7f3c1c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.281+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 shutdown_connections 2024-09-15T22:36:07.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:07.281+0000 7f3c23b2f640 1 -- 172.21.15.77:0/661878037 wait complete. 2024-09-15T22:36:07.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 176499 14.66 MB/sec execute 488 sec latency 836.193 ms 2024-09-15T22:36:07.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:07 smithi077 ceph-mon[30980]: osdmap e126: 12 total, 12 up, 12 in 2024-09-15T22:36:07.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:07 smithi196 ceph-mon[35109]: osdmap e126: 12 total, 12 up, 12 in 2024-09-15T22:36:07.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:07 smithi117 ceph-mon[35021]: osdmap e126: 12 total, 12 up, 12 in 2024-09-15T22:36:08.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 176764 14.66 MB/sec execute 489 sec latency 289.350 ms 2024-09-15T22:36:08.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:08 smithi077 ceph-mon[30980]: pgmap v749: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:08.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:08 smithi196 ceph-mon[35109]: pgmap v749: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:08.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:08 smithi117 ceph-mon[35021]: pgmap v749: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:09.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 177113 14.68 MB/sec execute 490 sec latency 271.336 ms 2024-09-15T22:36:10.141 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:36:10.448 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:10.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 177436 14.68 MB/sec execute 491 sec latency 150.520 ms 2024-09-15T22:36:10.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:10 smithi077 ceph-mon[30980]: pgmap v750: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:10.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:10 smithi196 ceph-mon[35109]: pgmap v750: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:10.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:10 smithi117 ceph-mon[35021]: pgmap v750: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 483 op/s 2024-09-15T22:36:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 -- 172.21.15.77:0/2760723279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 msgr2=0x7f0fb410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2760723279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb410eaa0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009ca0 tx=0x7f0fac02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 -- 172.21.15.77:0/2760723279 shutdown_connections 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2760723279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb410eaa0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2760723279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 0x7f0fb41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2760723279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 -- 172.21.15.77:0/2760723279 >> 172.21.15.77:0/2760723279 conn(0x7f0fb40fd8b0 msgr2=0x7f0fb40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 -- 172.21.15.77:0/2760723279 shutdown_connections 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.015+0000 7f0fbc181640 1 -- 172.21.15.77:0/2760723279 wait complete. 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.016+0000 7f0fbc181640 1 Processor -- start 2024-09-15T22:36:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.016+0000 7f0fbc181640 1 -- start start 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.016+0000 7f0fbc181640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.016+0000 7f0fbc181640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 0x7f0fb41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.016+0000 7f0fbc181640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fbc181640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4111f80 con 0x7f0fb4106f20 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fbc181640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4111e00 con 0x7f0fb41027a0 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fbc181640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4112100 con 0x7f0fb4101df0 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39056/0 (socket says 172.21.15.77:39056) 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/2976506344 learned_addr learned my addr 172.21.15.77:0/2976506344 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fb971c640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 0x7f0fb41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 -- 172.21.15.77:0/2976506344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 msgr2=0x7f0fb4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 -- 172.21.15.77:0/2976506344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 msgr2=0x7f0fb41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 0x7f0fb41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 -- 172.21.15.77:0/2976506344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb419d310 con 0x7f0fb4106f20 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.017+0000 7f0fba71e640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb419cc30 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f0fac02ade0 tx=0x7f0fac0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.018+0000 7f0faaffd640 1 -- 172.21.15.77:0/2976506344 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac003b70 con 0x7f0fb4106f20 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.018+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb419d5a0 con 0x7f0fb4106f20 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.018+0000 7f0faaffd640 1 -- 172.21.15.77:0/2976506344 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fac003d10 con 0x7f0fb4106f20 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.018+0000 7f0faaffd640 1 -- 172.21.15.77:0/2976506344 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac02fc40 con 0x7f0fb4106f20 2024-09-15T22:36:11.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.018+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb4074420 con 0x7f0fb4106f20 2024-09-15T22:36:11.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.022+0000 7f0faaffd640 1 -- 172.21.15.77:0/2976506344 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0fac004050 con 0x7f0fb4106f20 2024-09-15T22:36:11.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.022+0000 7f0fbe55a740 1 Processor -- start 2024-09-15T22:36:11.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.022+0000 7f0fbe55a740 1 -- start start 2024-09-15T22:36:11.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.022+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b6eeaa2f0 con 0x561b6ef45b40 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b6ee51cf0 con 0x561b6ef445d0 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b6eea62b0 con 0x561b6ef57b80 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36090/0 (socket says 172.21.15.77:36090) 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/3138700101 learned_addr learned my addr 172.21.15.77:0/3138700101 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb971c640 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/3138700101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 msgr2=0x561b6ef59f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef59f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/3138700101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 msgr2=0x561b6ef44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/3138700101 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6ee363a0 con 0x561b6ef445d0 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.023+0000 7f0faaffd640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 0x7f0fa00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0faaffd640 1 -- 172.21.15.77:0/2976506344 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f0fac0f0f00 con 0x7f0fb4106f20 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009e10 tx=0x7f0fb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a157ef04a36409 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0fb971c640 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef59f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0f9ffff640 1 -- 172.21.15.77:0/3138700101 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb001e070 con 0x561b6ef445d0 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0f9ffff640 1 -- 172.21.15.77:0/3138700101 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb0019b10 con 0x561b6ef445d0 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0f9ffff640 1 -- 172.21.15.77:0/3138700101 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000fc10 con 0x561b6ef445d0 2024-09-15T22:36:11.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.024+0000 7f0fb971c640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 0x7f0fa00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 -- 172.21.15.77:0/3138700101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 msgr2=0x561b6ef44a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fb971c640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 0x7f0fa00a58f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561b6ef45070 tx=0x7f0fa4009970 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009e10 tx=0x7f0fb0011810 comp rx=0 tx=0).stop 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 -- 172.21.15.77:0/3138700101 shutdown_connections 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef59f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef44a10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/3138700101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 -- 172.21.15.77:0/3138700101 >> 172.21.15.77:0/3138700101 conn(0x561b6ef0d310 msgr2=0x561b6ef40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 -- 172.21.15.77:0/3138700101 shutdown_connections 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.025+0000 7f0fbe55a740 1 -- 172.21.15.77:0/3138700101 wait complete. 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.026+0000 7f0fbe55a740 1 -- start start 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.026+0000 7f0fbe55a740 1 Processor -- start 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.026+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef57550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42908/0 (socket says 172.21.15.77:42908) 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef54df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/48156182 learned_addr learned my addr 172.21.15.77:0/48156182 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b6ef5a880 con 0x561b6ef445d0 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b6ee65060 con 0x561b6ef45b40 2024-09-15T22:36:11.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561b6ef5aa00 con 0x561b6ef57b80 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef54df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fb971c640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fbe55a740 10 client.0 _ll_register_callbacks cb 0x561b6ebb88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 -- 172.21.15.77:0/48156182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 msgr2=0x561b6ef54800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef54800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 -- 172.21.15.77:0/48156182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 msgr2=0x561b6ef57550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef57550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6ec1b3c0 con 0x561b6ef57b80 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0fba71e640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef54df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0ee310 tx=0x7f0fac0bce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78bd963feab1f298 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.027+0000 7f0f9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0bcfc0 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f0fac0ee480 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb0019b10 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0fac0c1370 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 10 client.29395 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b6ef5a880 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb0019b10 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0fac0bf030 con 0x561b6ef57b80 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 10 client.29395 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 10 client.29395 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 20 client.29395 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:11.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6ef677e0 0x561b6ef69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 4 client.29395 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b6eeb7860 con 0x561b6ef677e0 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.028+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6ef677e0 0x561b6ef69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.029+0000 7f0f9d7fa640 10 client.29395 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.029+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6ef677e0 0x561b6ef69c50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b33c7e03c3fb3e33 server_cookie=ee14aa52e5257b57 in_seq=0 out_seq=0 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/48156182 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b6eeb7860 con 0x561b6ef677e0 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0f9d7fa640 10 client.29395 handle_command_reply: tid=0 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 client.29395 shutdown 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 msgr2=0x561b6ef54df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef54df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0ee310 tx=0x7f0fac0bce50 comp rx=0 tx=0).stop 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 shutdown_connections 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6ef677e0 0x561b6ef69c50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef54df0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef54800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/48156182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef57550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0f9d7fa640 0 client.29395 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 >> 172.21.15.77:0/48156182 conn(0x561b6ef0d310 msgr2=0x561b6ef46930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 shutdown_connections 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 1 -- 172.21.15.77:0/48156182 wait complete. 2024-09-15T22:36:11.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.030+0000 7f0fbe55a740 20 client.29395 trim_cache size 0 max 16384 2024-09-15T22:36:11.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 Processor -- start 2024-09-15T22:36:11.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 -- start start 2024-09-15T22:36:11.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef63b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41acde0 con 0x561b6ef45b40 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ae920 con 0x561b6ef445d0 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ad4b0 con 0x561b6ef57b80 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fb971c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef63b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39080/0 (socket says 172.21.15.77:39080) 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 -- 172.21.15.77:0/1321442819 learned_addr learned my addr 172.21.15.77:0/1321442819 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fb971c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef63b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42910/0 (socket says 172.21.15.77:42910) 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 -- 172.21.15.77:0/1321442819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 msgr2=0x561b6ef40e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef40e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 -- 172.21.15.77:0/1321442819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 msgr2=0x561b6ef63b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef63b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.045+0000 7f0fba71e640 1 -- 172.21.15.77:0/1321442819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6ef537b0 con 0x561b6ef57b80 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.046+0000 7f0fba71e640 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0bcf50 tx=0x7f0fac0bcec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c10f513ea50c448 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.046+0000 7f0f9f7fe640 1 -- 172.21.15.77:0/1321442819 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0bc920 con 0x561b6ef57b80 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.046+0000 7f0f9f7fe640 1 -- 172.21.15.77:0/1321442819 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fac036040 con 0x561b6ef57b80 2024-09-15T22:36:11.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.046+0000 7f0f9f7fe640 1 -- 172.21.15.77:0/1321442819 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0b9630 con 0x561b6ef57b80 2024-09-15T22:36:11.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 -- 172.21.15.77:0/1321442819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 msgr2=0x561b6ef540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0bcf50 tx=0x7f0fac0bcec0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 -- 172.21.15.77:0/1321442819 shutdown_connections 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef540c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef45b40 0x561b6ef63b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/1321442819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef445d0 0x561b6ef40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 -- 172.21.15.77:0/1321442819 >> 172.21.15.77:0/1321442819 conn(0x561b6ef0d310 msgr2=0x561b6ef40700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 -- 172.21.15.77:0/1321442819 shutdown_connections 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.047+0000 7f0fbe55a740 1 -- 172.21.15.77:0/1321442819 wait complete. 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.048+0000 7f0fbe55a740 1 -- start start 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.048+0000 7f0fbe55a740 1 Processor -- start 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.048+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef51730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41ae250 con 0x561b6ef445d0 2024-09-15T22:36:11.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41adb80 con 0x561b6ef45b40 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41aade0 con 0x561b6ef57b80 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fbe55a740 10 client.0 _ll_register_callbacks cb 0x7f0fac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb9f1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42924/0 (socket says 172.21.15.77:42924) 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb9f1d640 1 -- 172.21.15.77:0/980873378 learned_addr learned my addr 172.21.15.77:0/980873378 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fba71e640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef51730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 -- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 msgr2=0x561b6ef51730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef51730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 -- 172.21.15.77:0/980873378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 msgr2=0x561b6ef600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb971c640 1 -- 172.21.15.77:0/980873378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6ef51c70 con 0x561b6ef45b40 2024-09-15T22:36:11.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.049+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0fb971c640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef511f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x561b6ef55e40 tx=0x7f0fa4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7738f2d29c2340 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0f9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4005020 con 0x561b6ef45b40 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f0fa40051c0 con 0x561b6ef45b40 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fac036040 con 0x561b6ef45b40 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0fa40053c0 con 0x561b6ef45b40 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0fbe55a740 10 client.29477 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.050+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b6efd21d0 con 0x561b6ef45b40 2024-09-15T22:36:11.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.051+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa4005560 con 0x561b6ef45b40 2024-09-15T22:36:11.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0fa4028a70 con 0x561b6ef45b40 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 10 client.29477 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 10 client.29477 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 20 client.29477 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6efd2940 0x561b6efd4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6efd2940 0x561b6efd4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 4 client.29477 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.052+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0fac036040 con 0x561b6efd2940 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.053+0000 7f0f9d7fa640 10 client.29477 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.053+0000 7f0fb9f1d640 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6efd2940 0x561b6efd4d60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f519fc7325d6313 server_cookie=9b207dce77c773db in_seq=0 out_seq=0 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.053+0000 7f0f9d7fa640 1 -- 172.21.15.77:0/980873378 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0fac036040 con 0x561b6efd2940 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.053+0000 7f0f9d7fa640 10 client.29477 handle_command_reply: tid=0 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 client.29477 shutdown 2024-09-15T22:36:11.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 msgr2=0x561b6ef511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef511f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x561b6ef55e40 tx=0x7f0fa4004730 comp rx=0 tx=0).stop 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 shutdown_connections 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x561b6efd2940 0x561b6efd4d60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561b6ef57b80 0x561b6ef51730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0f9d7fa640 0 client.29477 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b6ef45b40 0x561b6ef511f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 --2- 172.21.15.77:0/980873378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b6ef445d0 0x561b6ef600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 >> 172.21.15.77:0/980873378 conn(0x561b6ef0d310 msgr2=0x561b6ef56060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.054+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 shutdown_connections 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbe55a740 1 -- 172.21.15.77:0/980873378 wait complete. 2024-09-15T22:36:11.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbe55a740 20 client.29477 trim_cache size 0 max 16384 2024-09-15T22:36:11.060 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:36:11.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 msgr2=0x7f0fa00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 0x7f0fa00a58f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561b6ef45070 tx=0x7f0fa4009970 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 msgr2=0x7f0fb419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.055+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb419cc30 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f0fac02ade0 tx=0x7f0fac0319f0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 shutdown_connections 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0fa00a3430 0x7f0fa00a58f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fb4106f20 0x7f0fb419cc30 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fb41027a0 0x7f0fb41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 --2- 172.21.15.77:0/2976506344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb4101df0 0x7f0fb4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 >> 172.21.15.77:0/2976506344 conn(0x7f0fb40fd8b0 msgr2=0x7f0fb40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 shutdown_connections 2024-09-15T22:36:11.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:11.056+0000 7f0fbc181640 1 -- 172.21.15.77:0/2976506344 wait complete. 2024-09-15T22:36:11.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 177808 14.68 MB/sec execute 492 sec latency 161.221 ms 2024-09-15T22:36:12.137 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:36:12.500 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:12.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 178182 14.71 MB/sec execute 493 sec latency 141.642 ms 2024-09-15T22:36:12.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:12 smithi077 ceph-mon[30980]: pgmap v751: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-09-15T22:36:12.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:12 smithi196 ceph-mon[35109]: pgmap v751: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-09-15T22:36:12.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:12 smithi117 ceph-mon[35021]: pgmap v751: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 530 op/s 2024-09-15T22:36:13.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 178480 14.71 MB/sec execute 494 sec latency 169.074 ms 2024-09-15T22:36:14.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.516+0000 7f854e223640 1 -- 172.21.15.77:0/3274986051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 msgr2=0x7f854806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.516+0000 7f853bfff640 1 -- 172.21.15.77:0/3274986051 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400307f0 con 0x7f8548110190 2024-09-15T22:36:14.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.516+0000 7f854e223640 1 --2- 172.21.15.77:0/3274986051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f854806e4f0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8540009ca0 tx=0x7f854002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.520+0000 7f854e223640 1 -- 172.21.15.77:0/3274986051 shutdown_connections 2024-09-15T22:36:14.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.520+0000 7f854e223640 1 --2- 172.21.15.77:0/3274986051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854811b140 0x7f854811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.520+0000 7f854e223640 1 --2- 172.21.15.77:0/3274986051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854806ea30 0x7f854806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.520+0000 7f854e223640 1 --2- 172.21.15.77:0/3274986051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f854806e4f0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.520+0000 7f854e223640 1 -- 172.21.15.77:0/3274986051 >> 172.21.15.77:0/3274986051 conn(0x7f8548073180 msgr2=0x7f8548073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.522+0000 7f854e223640 1 -- 172.21.15.77:0/3274986051 shutdown_connections 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.524+0000 7f854e223640 1 -- 172.21.15.77:0/3274986051 wait complete. 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.525+0000 7f854e223640 1 Processor -- start 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.529+0000 7f854e223640 1 -- start start 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.529+0000 7f854e223640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 0x7f85481a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.530+0000 7f854e223640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f85481a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.530+0000 7f854e223640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854811b140 0x7f85481aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.530+0000 7f854e223640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8548120c60 con 0x7f8548110190 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.530+0000 7f854e223640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8548120ae0 con 0x7f854806ea30 2024-09-15T22:36:14.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.530+0000 7f854e223640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8548120de0 con 0x7f854811b140 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.532+0000 7f854d221640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 0x7f85481a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.532+0000 7f854d221640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 0x7f85481a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36144/0 (socket says 172.21.15.77:36144) 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.532+0000 7f854d221640 1 -- 172.21.15.77:0/4210908922 learned_addr learned my addr 172.21.15.77:0/4210908922 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.532+0000 7f854ca20640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f85481a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.532+0000 7f854da22640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854811b140 0x7f85481aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 -- 172.21.15.77:0/4210908922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854811b140 msgr2=0x7f85481aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854811b140 0x7f85481aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 -- 172.21.15.77:0/4210908922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 msgr2=0x7f85481a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 0x7f85481a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 -- 172.21.15.77:0/4210908922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85481ac150 con 0x7f8548110190 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.533+0000 7f854ca20640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f85481a7760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f853c009fd0 tx=0x7f853c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.534+0000 7f853a7fc640 1 -- 172.21.15.77:0/4210908922 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c015020 con 0x7f8548110190 2024-09-15T22:36:14.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.534+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85481ac440 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.535+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8548110920 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.535+0000 7f853a7fc640 1 -- 172.21.15.77:0/4210908922 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853c011d10 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.535+0000 7f853a7fc640 1 -- 172.21.15.77:0/4210908922 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00f2e0 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.539+0000 7f853a7fc640 1 -- 172.21.15.77:0/4210908922 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f853c018020 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.542+0000 7f855185e740 1 Processor -- start 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.545+0000 7f853a7fc640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 0x7f85340a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.545+0000 7f855185e740 1 -- start start 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.545+0000 7f853a7fc640 1 -- 172.21.15.77:0/4210908922 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f853c0cf660 con 0x7f8548110190 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a76b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a76b780 0x561c9a76bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c9a6d12f0 con 0x561c9a776520 2024-09-15T22:36:14.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c9a678cf0 con 0x561c9a76b780 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f855185e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c9a6cd2b0 con 0x561c9a76ccf0 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f854da22640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f854da22640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42940/0 (socket says 172.21.15.77:42940) 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f854da22640 1 -- 172.21.15.77:0/216635307 learned_addr learned my addr 172.21.15.77:0/216635307 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.546+0000 7f854da22640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 0x7f85340a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854ca20640 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a76b780 0x561c9a76bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854da22640 1 -- 172.21.15.77:0/216635307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 msgr2=0x561c9a76b240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854da22640 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a76b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854da22640 1 -- 172.21.15.77:0/216635307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a76b780 msgr2=0x561c9a76bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854da22640 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a76b780 0x561c9a76bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.547+0000 7f854da22640 1 -- 172.21.15.77:0/216635307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9a65d3a0 con 0x561c9a776520 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.548+0000 7f854da22640 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8544011670 tx=0x7f8544011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bee2ecb54ce0467 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.548+0000 7f85337fe640 1 -- 172.21.15.77:0/216635307 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544004220 con 0x561c9a776520 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.548+0000 7f85337fe640 1 -- 172.21.15.77:0/216635307 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544020030 con 0x561c9a776520 2024-09-15T22:36:14.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.549+0000 7f855185e740 1 -- 172.21.15.77:0/216635307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 msgr2=0x561c9a778910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.549+0000 7f855185e740 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8544011670 tx=0x7f8544011b30 comp rx=0 tx=0).stop 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f855185e740 1 -- 172.21.15.77:0/216635307 shutdown_connections 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f855185e740 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a776520 0x561c9a778910 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f855185e740 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a76b780 0x561c9a76bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f855185e740 1 --2- 172.21.15.77:0/216635307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a76b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f855185e740 1 -- 172.21.15.77:0/216635307 >> 172.21.15.77:0/216635307 conn(0x561c9a734310 msgr2=0x561c9a767e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.550+0000 7f854da22640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 0x7f85340a5a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f854401b000 tx=0x7f854401d000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.551+0000 7f855185e740 1 -- 172.21.15.77:0/216635307 shutdown_connections 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.551+0000 7f855185e740 1 -- 172.21.15.77:0/216635307 wait complete. 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.551+0000 7f855185e740 1 -- start start 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.551+0000 7f855185e740 1 Processor -- start 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a775880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a776240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c9a779210 con 0x561c9a76b780 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c9a68c060 con 0x561c9a775dc0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c9a779390 con 0x561c9a76ccf0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.552+0000 7f855185e740 10 client.0 _ll_register_callbacks cb 0x561c9a3df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.554+0000 7f854d221640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.554+0000 7f854d221640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42954/0 (socket says 172.21.15.77:42954) 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.554+0000 7f854d221640 1 -- 172.21.15.77:0/2981784741 learned_addr learned my addr 172.21.15.77:0/2981784741 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854ca20640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a775880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854da22640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a776240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854d221640 1 -- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 msgr2=0x561c9a775880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854d221640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a775880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854d221640 1 -- 172.21.15.77:0/2981784741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 msgr2=0x561c9a776240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854d221640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a776240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854d221640 1 -- 172.21.15.77:0/2981784741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9a4423c0 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.555+0000 7f854ca20640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a775880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.557+0000 7f854d221640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8540003bf0 tx=0x7f8540003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a73e33952da682 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.559+0000 7f8530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.559+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540031680 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.559+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c9a6de860 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.559+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f8540031820 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.563+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8540003e40 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.565+0000 7f855185e740 10 client.20832 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.565+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c9a7a6f60 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.570+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c9a6de860 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.570+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f85400309e0 con 0x561c9a76b780 2024-09-15T22:36:14.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 10 client.20832 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 10 client.20832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 20 client.20832 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a7a8310 0x561c9a7aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 4 client.20832 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.573+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c9a6d8d70 con 0x561c9a7a8310 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.575+0000 7f854ca20640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a7a8310 0x561c9a7aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.578+0000 7f854ca20640 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a7a8310 0x561c9a7aa760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6692e0c4cfb2189 server_cookie=506a371d98e9601c in_seq=0 out_seq=0 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.580+0000 7f8530ff9640 10 client.20832 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.582+0000 7f8530ff9640 1 -- 172.21.15.77:0/2981784741 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c9a6d8d70 con 0x561c9a7a8310 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.582+0000 7f8530ff9640 10 client.20832 handle_command_reply: tid=0 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.582+0000 7f855185e740 1 client.20832 shutdown 2024-09-15T22:36:14.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 msgr2=0x561c9a7785d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8540003bf0 tx=0x7f8540003c20 comp rx=0 tx=0).stop 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 shutdown_connections 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a7a8310 0x561c9a7aa760 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a776240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a775880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 --2- 172.21.15.77:0/2981784741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a7785d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f8530ff9640 0 client.20832 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 >> 172.21.15.77:0/2981784741 conn(0x561c9a734310 msgr2=0x561c9a76db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 shutdown_connections 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.584+0000 7f855185e740 1 -- 172.21.15.77:0/2981784741 wait complete. 2024-09-15T22:36:14.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.585+0000 7f855185e740 20 client.20832 trim_cache size 0 max 16384 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 Processor -- start 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 -- start start 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a766da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a7b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f852c010110 con 0x561c9a76b780 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f852c011c50 con 0x561c9a775dc0 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f855185e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0107e0 con 0x561c9a76ccf0 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.601+0000 7f854d221640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42958/0 (socket says 172.21.15.77:42958) 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 -- 172.21.15.77:0/2922863106 learned_addr learned my addr 172.21.15.77:0/2922863106 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854da22640 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a7b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854ca20640 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a766da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 -- 172.21.15.77:0/2922863106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 msgr2=0x561c9a766da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a766da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 -- 172.21.15.77:0/2922863106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 msgr2=0x561c9a7b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a7b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 -- 172.21.15.77:0/2922863106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9a768b10 con 0x561c9a76b780 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f854d221640 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8540002230 tx=0x7f854002aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474d794041721c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f8532ffd640 1 -- 172.21.15.77:0/2922863106 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540031680 con 0x561c9a76b780 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f8532ffd640 1 -- 172.21.15.77:0/2922863106 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8540036040 con 0x561c9a76b780 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.602+0000 7f8532ffd640 1 -- 172.21.15.77:0/2922863106 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854004fc50 con 0x561c9a76b780 2024-09-15T22:36:14.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 -- 172.21.15.77:0/2922863106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 msgr2=0x561c9a767f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8540002230 tx=0x7f854002aea0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 -- 172.21.15.77:0/2922863106 shutdown_connections 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a7b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76ccf0 0x561c9a766da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 --2- 172.21.15.77:0/2922863106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76b780 0x561c9a767f20 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 -- 172.21.15.77:0/2922863106 >> 172.21.15.77:0/2922863106 conn(0x561c9a734310 msgr2=0x561c9a766690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 -- 172.21.15.77:0/2922863106 shutdown_connections 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.603+0000 7f855185e740 1 -- 172.21.15.77:0/2922863106 wait complete. 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 -- start start 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 Processor -- start 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76b780 0x561c9a77ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a77c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f852c011580 con 0x561c9a76ccf0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f852c010eb0 con 0x561c9a775dc0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f852c00e110 con 0x561c9a76b780 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f855185e740 10 client.0 _ll_register_callbacks cb 0x7f8540009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.604+0000 7f854ca20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42970/0 (socket says 172.21.15.77:42970) 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 -- 172.21.15.77:0/1351727168 learned_addr learned my addr 172.21.15.77:0/1351727168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854da22640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a77c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854d221640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76b780 0x561c9a77ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 -- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76b780 msgr2=0x561c9a77ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76b780 0x561c9a77ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 -- 172.21.15.77:0/1351727168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 msgr2=0x561c9a77c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a77c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 -- 172.21.15.77:0/1351727168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9a77cbe0 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f854ca20640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f853c09b360 tx=0x7f853c0972c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6308dbb3a8fe0a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f8530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c010810 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9526+0+0 (secure 0 0 0) 0x7f853c0ccc40 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.605+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8540036040 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f853c097cd0 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 10 client.20838 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c9a812f00 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8540036040 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f853c0cf1f0 con 0x561c9a76ccf0 2024-09-15T22:36:14.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 10 client.20838 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 10 client.20838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 20 client.20838 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a813680 0x561c9a815aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 4 client.20838 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c9a655170 con 0x561c9a813680 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.606+0000 7f854d221640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a813680 0x561c9a815aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.607+0000 7f8530ff9640 10 client.20838 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.607+0000 7f854d221640 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a813680 0x561c9a815aa0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c93451dfef1a9d1 server_cookie=bb597177ecee020d in_seq=0 out_seq=0 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.607+0000 7f8530ff9640 1 -- 172.21.15.77:0/1351727168 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c9a655170 con 0x561c9a813680 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.607+0000 7f8530ff9640 10 client.20838 handle_command_reply: tid=0 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.607+0000 7f855185e740 1 client.20838 shutdown 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 msgr2=0x561c9a77c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f853c09b360 tx=0x7f853c0972c0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 shutdown_connections 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x561c9a813680 0x561c9a815aa0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c9a775dc0 0x561c9a77c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c9a76ccf0 0x561c9a77c000 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f8530ff9640 0 client.20838 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 --2- 172.21.15.77:0/1351727168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c9a76b780 0x561c9a77ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 >> 172.21.15.77:0/1351727168 conn(0x561c9a734310 msgr2=0x561c9a766eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 shutdown_connections 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 1 -- 172.21.15.77:0/1351727168 wait complete. 2024-09-15T22:36:14.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f855185e740 20 client.20838 trim_cache size 0 max 16384 2024-09-15T22:36:14.052 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:36:14.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 msgr2=0x7f85340a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 0x7f85340a5a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f854401b000 tx=0x7f854401d000 comp rx=0 tx=0).stop 2024-09-15T22:36:14.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 msgr2=0x7f85481a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f85481a7760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f853c009fd0 tx=0x7f853c00b870 comp rx=0 tx=0).stop 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 shutdown_connections 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85340a3540 0x7f85340a5a00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f854811b140 0x7f85481aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8548110190 0x7f85481a7760 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 --2- 172.21.15.77:0/4210908922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f854806ea30 0x7f85481a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.608+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 >> 172.21.15.77:0/4210908922 conn(0x7f8548073180 msgr2=0x7f854810e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.609+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 shutdown_connections 2024-09-15T22:36:14.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:13.609+0000 7f854e223640 1 -- 172.21.15.77:0/4210908922 wait complete. 2024-09-15T22:36:14.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 178655 14.70 MB/sec execute 495 sec latency 1169.114 ms 2024-09-15T22:36:14.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:14 smithi077 ceph-mon[30980]: pgmap v752: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 511 op/s 2024-09-15T22:36:14.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:14 smithi077 ceph-mon[30980]: osdmap e127: 12 total, 12 up, 12 in 2024-09-15T22:36:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:14 smithi196 ceph-mon[35109]: pgmap v752: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 511 op/s 2024-09-15T22:36:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:14 smithi196 ceph-mon[35109]: osdmap e127: 12 total, 12 up, 12 in 2024-09-15T22:36:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:14 smithi117 ceph-mon[35021]: pgmap v752: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.4 MiB/s wr, 511 op/s 2024-09-15T22:36:14.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:14 smithi117 ceph-mon[35021]: osdmap e127: 12 total, 12 up, 12 in 2024-09-15T22:36:15.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 178857 14.69 MB/sec execute 496 sec latency 1203.444 ms 2024-09-15T22:36:15.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:36:16.289 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:16.587 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:16 smithi077 ceph-mon[30980]: pgmap v754: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 281 op/s 2024-09-15T22:36:16.622 INFO:tasks.workunit.client.0.smithi077.stdout: 10 179188 14.69 MB/sec execute 497 sec latency 165.561 ms 2024-09-15T22:36:16.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:16 smithi196 ceph-mon[35109]: pgmap v754: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 281 op/s 2024-09-15T22:36:16.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:16 smithi117 ceph-mon[35021]: pgmap v754: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.0 MiB/s wr, 281 op/s 2024-09-15T22:36:16.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.859+0000 7fc4c362b640 1 -- 172.21.15.77:0/862809495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1036e0 msgr2=0x7fc4bc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.859+0000 7fc4c362b640 1 --2- 172.21.15.77:0/862809495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1036e0 0x7fc4bc107b90 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009930 tx=0x7fc4ac02a420 comp rx=0 tx=0).stop 2024-09-15T22:36:16.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 -- 172.21.15.77:0/862809495 shutdown_connections 2024-09-15T22:36:16.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 --2- 172.21.15.77:0/862809495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 --2- 172.21.15.77:0/862809495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1036e0 0x7fc4bc107b90 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 --2- 172.21.15.77:0/862809495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 -- 172.21.15.77:0/862809495 >> 172.21.15.77:0/862809495 conn(0x7fc4bc0fc7d0 msgr2=0x7fc4bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 -- 172.21.15.77:0/862809495 shutdown_connections 2024-09-15T22:36:16.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.860+0000 7fc4c362b640 1 -- 172.21.15.77:0/862809495 wait complete. 2024-09-15T22:36:16.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.861+0000 7fc4c362b640 1 Processor -- start 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.861+0000 7fc4c362b640 1 -- start start 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.861+0000 7fc4c362b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c362b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4bc1036e0 0x7fc4bc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c362b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c362b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1120b0 con 0x7fc4bc1082a0 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c362b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc111f30 con 0x7fc4bc102d30 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c362b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc112230 con 0x7fc4bc1036e0 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c13c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42990/0 (socket says 172.21.15.77:42990) 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 -- 172.21.15.77:0/3520884752 learned_addr learned my addr 172.21.15.77:0/3520884752 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 -- 172.21.15.77:0/3520884752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4bc1036e0 msgr2=0x7fc4bc198bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:36:16.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4bc1036e0 0x7fc4bc198bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 -- 172.21.15.77:0/3520884752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 msgr2=0x7fc4bc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c1bc8640 1 -- 172.21.15.77:0/3520884752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc19d660 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.862+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.863+0000 7fc4c1bc8640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e970 tx=0x7fc4b400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.863+0000 7fc4b27fc640 1 -- 172.21.15.77:0/3520884752 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400ccc0 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.863+0000 7fc4b27fc640 1 -- 172.21.15.77:0/3520884752 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b400ce60 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.864+0000 7fc4b27fc640 1 -- 172.21.15.77:0/3520884752 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4010490 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.864+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc19d950 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.864+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1a5230 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.868+0000 7fc4c5a04740 1 Processor -- start 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.868+0000 7fc4b27fc640 1 -- 172.21.15.77:0/3520884752 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc4b4005020 con 0x7fc4bc1082a0 2024-09-15T22:36:16.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.869+0000 7fc4c5a04740 1 -- start start 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2877b40 0x55f1c2876090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c2876a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1c27dc2f0 con 0x55f1c2877b40 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2783cf0 con 0x55f1c28765d0 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f1c27d82b0 con 0x55f1c2881370 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4b27fc640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 0x7fc4a80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39160/0 (socket says 172.21.15.77:39160) 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/3941335373 learned_addr learned my addr 172.21.15.77:0/3941335373 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c2876a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.870+0000 7fc4b27fc640 1 -- 172.21.15.77:0/3520884752 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fc4b40d04e0 con 0x7fc4bc1082a0 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c1bc8640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 0x7fc4a80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/3941335373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 msgr2=0x55f1c2876a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c2876a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/3941335373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2877b40 msgr2=0x55f1c2876090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2877b40 0x55f1c2876090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/3941335373 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c27683a0 con 0x55f1c2881370 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.871+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009930 tx=0x7fc4ac031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1c90411e15510d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.872+0000 7fc4c1bc8640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 0x7fc4a80a59c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc4b409c360 tx=0x7fc4b4009310 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.872+0000 7fc4a77fe640 1 -- 172.21.15.77:0/3941335373 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac040020 con 0x55f1c2881370 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.872+0000 7fc4a77fe640 1 -- 172.21.15.77:0/3941335373 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0046c0 con 0x55f1c2881370 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.872+0000 7fc4a77fe640 1 -- 172.21.15.77:0/3941335373 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac035070 con 0x55f1c2881370 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 -- 172.21.15.77:0/3941335373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 msgr2=0x55f1c2883760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009930 tx=0x7fc4ac031490 comp rx=0 tx=0).stop 2024-09-15T22:36:16.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 -- 172.21.15.77:0/3941335373 shutdown_connections 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2881370 0x55f1c2883760 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c2876a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/3941335373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2877b40 0x55f1c2876090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 -- 172.21.15.77:0/3941335373 >> 172.21.15.77:0/3941335373 conn(0x55f1c283f310 msgr2=0x55f1c2872c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 -- 172.21.15.77:0/3941335373 shutdown_connections 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.873+0000 7fc4c5a04740 1 -- 172.21.15.77:0/3941335373 wait complete. 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 -- start start 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 Processor -- start 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c28765d0 0x55f1c287a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c287a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2883f20 con 0x55f1c28765d0 2024-09-15T22:36:16.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2797060 con 0x55f1c2881370 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.874+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f1c28840a0 con 0x55f1c2877b40 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c5a04740 10 client.0 _ll_register_callbacks cb 0x55f1c24ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36198/0 (socket says 172.21.15.77:36198) 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/702404530 learned_addr learned my addr 172.21.15.77:0/702404530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c287a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 -- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 msgr2=0x55f1c287ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 -- 172.21.15.77:0/702404530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c28765d0 msgr2=0x55f1c287a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c28765d0 0x55f1c287a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c13c7640 1 -- 172.21.15.77:0/702404530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c254d3c0 con 0x55f1c2877b40 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.875+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:16.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.876+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c287a7f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f1c2873ac0 tx=0x7fc4b80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e3b38dc0b76382 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.876+0000 7fc4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.876+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8003e40 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.876+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fc4b8004050 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.876+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4ac0046c0 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8004250 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 10 client.29422 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f1c28983e0 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac0046c0 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc4b801a070 con 0x55f1c2877b40 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 10 client.29422 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 10 client.29422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 20 client.29422 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.877+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c28998a0 0x55f1c289bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.878+0000 7fc4c5a04740 4 client.29422 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.878+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c27e9860 con 0x55f1c28998a0 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.878+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c28998a0 0x55f1c289bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.878+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c28998a0 0x55f1c289bcf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c773cadee84c1e8a server_cookie=f71eb451a529e553 in_seq=0 out_seq=0 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.878+0000 7fc4a4ff9640 10 client.29422 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/702404530 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1c27e9860 con 0x55f1c28998a0 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4a4ff9640 10 client.29422 handle_command_reply: tid=0 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 client.29422 shutdown 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 msgr2=0x55f1c287a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c287a7f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f1c2873ac0 tx=0x7fc4b80117b0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 shutdown_connections 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c28998a0 0x55f1c289bcf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2881370 0x55f1c287ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c287a7f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4a4ff9640 0 client.29422 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/702404530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c28765d0 0x55f1c287a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.879+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 >> 172.21.15.77:0/702404530 conn(0x55f1c283f310 msgr2=0x55f1c2878930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.880+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 shutdown_connections 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.880+0000 7fc4c5a04740 1 -- 172.21.15.77:0/702404530 wait complete. 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.880+0000 7fc4c5a04740 20 client.29422 trim_cache size 0 max 16384 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.894+0000 7fc4c5a04740 1 Processor -- start 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.894+0000 7fc4c5a04740 1 -- start start 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.894+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c28a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1acde0 con 0x55f1c2881370 2024-09-15T22:36:16.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ae920 con 0x55f1c28765d0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ad4b0 con 0x55f1c2877b40 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36204/0 (socket says 172.21.15.77:36204) 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/1483245993 learned_addr learned my addr 172.21.15.77:0/1483245993 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c28a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/1483245993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 msgr2=0x55f1c28a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c28a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/1483245993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 msgr2=0x55f1c28a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.895+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/1483245993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c289c370 con 0x55f1c28765d0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.896+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac005a80 tx=0x7fc4ac005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94600340856d55e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.896+0000 7fc4a6ffd640 1 -- 172.21.15.77:0/1483245993 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac054070 con 0x55f1c28765d0 2024-09-15T22:36:16.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.896+0000 7fc4a6ffd640 1 -- 172.21.15.77:0/1483245993 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac004cf0 con 0x55f1c28765d0 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.896+0000 7fc4a6ffd640 1 -- 172.21.15.77:0/1483245993 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac035070 con 0x55f1c28765d0 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1483245993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 msgr2=0x55f1c287ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac005a80 tx=0x7fc4ac005fd0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1483245993 shutdown_connections 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c2877b40 0x55f1c28a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1483245993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c28765d0 0x55f1c287ab80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1483245993 >> 172.21.15.77:0/1483245993 conn(0x55f1c283f310 msgr2=0x55f1c287e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1483245993 shutdown_connections 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.897+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1483245993 wait complete. 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.898+0000 7fc4c5a04740 1 -- start start 2024-09-15T22:36:16.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.898+0000 7fc4c5a04740 1 Processor -- start 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c28765d0 0x55f1c2896040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 0x55f1c2885c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ae250 con 0x55f1c2881370 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1adb80 con 0x55f1c2877b40 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1aade0 con 0x55f1c28765d0 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c5a04740 10 client.0 _ll_register_callbacks cb 0x7fc4b8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c28765d0 0x55f1c2896040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 0x55f1c2885c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c0bc6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43032/0 (socket says 172.21.15.77:43032) 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c0bc6640 1 -- 172.21.15.77:0/1984374437 learned_addr learned my addr 172.21.15.77:0/1984374437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 -- 172.21.15.77:0/1984374437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c28765d0 msgr2=0x55f1c2896040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c28765d0 0x55f1c2896040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 -- 172.21.15.77:0/1984374437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 msgr2=0x55f1c28861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.899+0000 7fc4c13c7640 1 -- 172.21.15.77:0/1984374437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c28866f0 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4c13c7640 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 0x55f1c2885c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc4b801ccf0 tx=0x7fc4b8009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6258035379535b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b801c850 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4ac004cf0 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fc4b801c9f0 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8039050 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4c5a04740 10 client.29507 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f1c2904370 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.900+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ac004cf0 con 0x55f1c2877b40 2024-09-15T22:36:16.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc4b801a070 con 0x55f1c2877b40 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 10 client.29507 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 10 client.29507 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 20 client.29507 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c2904b70 0x55f1c2906f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c2904b70 0x55f1c2906f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 4 client.29507 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.901+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1c27e3d70 con 0x55f1c2904b70 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.902+0000 7fc4c0bc6640 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c2904b70 0x55f1c2906f90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fb142fe5e88086a server_cookie=cf77cb01c58619e1 in_seq=0 out_seq=0 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.902+0000 7fc4a4ff9640 10 client.29507 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.903+0000 7fc4a4ff9640 1 -- 172.21.15.77:0/1984374437 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1c27e3d70 con 0x55f1c2904b70 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.903+0000 7fc4a4ff9640 10 client.29507 handle_command_reply: tid=0 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.903+0000 7fc4c5a04740 1 client.29507 shutdown 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 msgr2=0x55f1c2885c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 0x55f1c2885c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc4b801ccf0 tx=0x7fc4b8009280 comp rx=0 tx=0).stop 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 shutdown_connections 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55f1c2904b70 0x55f1c2906f90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4a4ff9640 0 client.29507 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1c2881370 0x55f1c28861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c2877b40 0x55f1c2885c70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 --2- 172.21.15.77:0/1984374437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f1c28765d0 0x55f1c2896040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 >> 172.21.15.77:0/1984374437 conn(0x55f1c283f310 msgr2=0x55f1c2878110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.923 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 shutdown_connections 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.904+0000 7fc4c5a04740 1 -- 172.21.15.77:0/1984374437 wait complete. 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.905+0000 7fc4c5a04740 20 client.29507 trim_cache size 0 max 16384 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.905+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 msgr2=0x7fc4a80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.905+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 0x7fc4a80a59c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc4b409c360 tx=0x7fc4b4009310 comp rx=0 tx=0).stop 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.905+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 msgr2=0x7fc4bc19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.905+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e970 tx=0x7fc4b400ed70 comp rx=0 tx=0).stop 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 shutdown_connections 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc4a80a3500 0x7fc4a80a59c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4bc1082a0 0x7fc4bc19cf80 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4bc1036e0 0x7fc4bc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 --2- 172.21.15.77:0/3520884752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4bc102d30 0x7fc4bc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 >> 172.21.15.77:0/3520884752 conn(0x7fc4bc0fc7d0 msgr2=0x7fc4bc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 shutdown_connections 2024-09-15T22:36:16.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:16.906+0000 7fc4c362b640 1 -- 172.21.15.77:0/3520884752 wait complete. 2024-09-15T22:36:17.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 179628 14.72 MB/sec execute 498 sec latency 171.589 ms 2024-09-15T22:36:17.768 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:36:18.136 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:18.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 179897 14.71 MB/sec execute 499 sec latency 159.454 ms 2024-09-15T22:36:18.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:18 smithi077 ceph-mon[30980]: pgmap v755: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:18.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:18 smithi196 ceph-mon[35109]: pgmap v755: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:18.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:18 smithi117 ceph-mon[35021]: pgmap v755: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:19.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 180014 14.68 MB/sec execute 500 sec latency 523.743 ms 2024-09-15T22:36:20.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 -- 172.21.15.77:0/3447097289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83741027c0 msgr2=0x7f8374106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 --2- 172.21.15.77:0/3447097289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83741027c0 0x7f8374106810 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8364009900 tx=0x7f836402a410 comp rx=0 tx=0).stop 2024-09-15T22:36:20.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 -- 172.21.15.77:0/3447097289 shutdown_connections 2024-09-15T22:36:20.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 --2- 172.21.15.77:0/3447097289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8374106f40 0x7f837410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 --2- 172.21.15.77:0/3447097289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83741027c0 0x7f8374106810 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 --2- 172.21.15.77:0/3447097289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f83741021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 -- 172.21.15.77:0/3447097289 >> 172.21.15.77:0/3447097289 conn(0x7f83740fd8d0 msgr2=0x7f83740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 -- 172.21.15.77:0/3447097289 shutdown_connections 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 -- 172.21.15.77:0/3447097289 wait complete. 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.765+0000 7f837bbb2640 1 Processor -- start 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 -- start start 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f8374198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 0x7f8374198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 0x7f837419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837914d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 0x7f8374198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f8374198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837914d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 0x7f8374198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39214/0 (socket says 172.21.15.77:39214) 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f8374198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36240/0 (socket says 172.21.15.77:36240) 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837914d640 1 -- 172.21.15.77:0/1472504321 learned_addr learned my addr 172.21.15.77:0/1472504321 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 0x7f837419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8374111fa0 con 0x7f8374106f40 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8374111e20 con 0x7f8374101e10 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8374112120 con 0x7f83741027c0 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 -- 172.21.15.77:0/1472504321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 msgr2=0x7f8374198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 0x7f8374198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 -- 172.21.15.77:0/1472504321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 msgr2=0x7f8374198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f8374198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.766+0000 7f837a14f640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837419d400 con 0x7f8374106f40 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.767+0000 7f837a14f640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 0x7f837419cd20 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f836c00da10 tx=0x7f836c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.767+0000 7f836affd640 1 -- 172.21.15.77:0/1472504321 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c00bd70 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.767+0000 7f836affd640 1 -- 172.21.15.77:0/1472504321 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c003e50 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.767+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837419d6f0 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.767+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83741a4fd0 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.768+0000 7f836affd640 1 -- 172.21.15.77:0/1472504321 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0106a0 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.769+0000 7f836affd640 1 -- 172.21.15.77:0/1472504321 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f836c003970 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.770+0000 7f836affd640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 0x7f83600a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.770+0000 7f837df8b740 1 Processor -- start 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.770+0000 7f837994e640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 0x7f83600a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.770+0000 7f836affd640 1 -- 172.21.15.77:0/1472504321 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f836c0cfb50 con 0x7f8374106f40 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.770+0000 7f837df8b740 1 -- start start 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 0x56379b688a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56379b5ee2f0 con 0x56379b693370 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56379b595cf0 con 0x56379b689b40 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837df8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56379b5ea2b0 con 0x56379b6885d0 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837994e640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 0x7f83600a5750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8370009bd0 tx=0x7f8370005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837914d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 0x56379b688a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43040/0 (socket says 172.21.15.77:43040) 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36256/0 (socket says 172.21.15.77:36256) 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 -- 172.21.15.77:0/177207058 learned_addr learned my addr 172.21.15.77:0/177207058 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837914d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 0x56379b688a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:39222/0 (socket says 172.21.15.77:39222) 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 -- 172.21.15.77:0/177207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 msgr2=0x56379b688a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 0x56379b688a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 -- 172.21.15.77:0/177207058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 msgr2=0x56379b688090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 -- 172.21.15.77:0/177207058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56379b57a3a0 con 0x56379b693370 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837994e640 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f837a14f640 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f836c098410 tx=0x7f836c09b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2ff39299831eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.771+0000 7f835ffff640 1 -- 172.21.15.77:0/177207058 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b4070 con 0x56379b693370 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f835ffff640 1 -- 172.21.15.77:0/177207058 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c09fdf0 con 0x56379b693370 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f835ffff640 1 -- 172.21.15.77:0/177207058 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0a4220 con 0x56379b693370 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 -- 172.21.15.77:0/177207058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 msgr2=0x56379b695760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f836c098410 tx=0x7f836c09b3a0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 -- 172.21.15.77:0/177207058 shutdown_connections 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b695760 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b6885d0 0x56379b688a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 --2- 172.21.15.77:0/177207058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b689b40 0x56379b688090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 -- 172.21.15.77:0/177207058 >> 172.21.15.77:0/177207058 conn(0x56379b651310 msgr2=0x56379b684c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.772+0000 7f837df8b740 1 -- 172.21.15.77:0/177207058 shutdown_connections 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 -- 172.21.15.77:0/177207058 wait complete. 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 -- start start 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 Processor -- start 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b692de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 0x56379b690090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 0x56379b6907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56379b69a480 con 0x56379b689b40 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56379b5a9060 con 0x56379b6885d0 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56379b69a600 con 0x56379b693370 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.773+0000 7f837df8b740 10 client.0 _ll_register_callbacks cb 0x56379b2fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:20.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b692de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837914d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 0x56379b690090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b692de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:36264/0 (socket says 172.21.15.77:36264) 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837994e640 1 -- 172.21.15.77:0/858389203 learned_addr learned my addr 172.21.15.77:0/858389203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 0x56379b6907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 -- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 msgr2=0x56379b692de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b692de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 -- 172.21.15.77:0/858389203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 msgr2=0x56379b690090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 0x56379b690090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 -- 172.21.15.77:0/858389203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379b35f3c0 con 0x56379b693370 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837914d640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 0x56379b690090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837a14f640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 0x56379b6907d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f836c09b740 tx=0x7f836c0cd3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4573a1943f5a4a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f835d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b4070 con 0x56379b693370 2024-09-15T22:36:20.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c09fdf0 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.774+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f836c0a4430 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f836c003c60 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f837df8b740 10 client.29440 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56379b69a480 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836c09fdf0 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f836c09ae90 con 0x56379b693370 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f837df8b740 10 client.29440 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f837df8b740 10 client.29440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.775+0000 7f837df8b740 20 client.29440 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b6a7130 0x56379b6a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f837df8b740 4 client.29440 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56379b5fb860 con 0x56379b6a7130 2024-09-15T22:36:20.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f837994e640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b6a7130 0x56379b6a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f835d7fa640 10 client.29440 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.776+0000 7f837994e640 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b6a7130 0x56379b6a9520 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d53eec6ed78de9 server_cookie=532e10a21dd510bf in_seq=0 out_seq=0 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f835d7fa640 1 -- 172.21.15.77:0/858389203 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56379b5fb860 con 0x56379b6a7130 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f835d7fa640 10 client.29440 handle_command_reply: tid=0 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 client.29440 shutdown 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 msgr2=0x56379b6907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 0x56379b6907d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f836c09b740 tx=0x7f836c0cd3f0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 shutdown_connections 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b6a7130 0x56379b6a9520 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b693370 0x56379b6907d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b689b40 0x56379b690090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 --2- 172.21.15.77:0/858389203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b692de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f835d7fa640 0 client.29440 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 >> 172.21.15.77:0/858389203 conn(0x56379b651310 msgr2=0x56379b68a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 shutdown_connections 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 1 -- 172.21.15.77:0/858389203 wait complete. 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.777+0000 7f837df8b740 20 client.29440 trim_cache size 0 max 16384 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 Processor -- start 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 -- start start 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b684e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b6a3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83741ac940 con 0x56379b693370 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83741ae480 con 0x56379b6885d0 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837df8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83741ad010 con 0x56379b689b40 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837a14f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837994e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b684e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b6a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837a14f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43062/0 (socket says 172.21.15.77:43062) 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837a14f640 1 -- 172.21.15.77:0/26447402 learned_addr learned my addr 172.21.15.77:0/26447402 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 -- 172.21.15.77:0/26447402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 msgr2=0x56379b684e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b684e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 -- 172.21.15.77:0/26447402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 msgr2=0x56379b6aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837914d640 1 -- 172.21.15.77:0/26447402 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56379b68b120 con 0x56379b689b40 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.791+0000 7f837a14f640 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f837914d640 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b6a3720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8364009230 tx=0x7f8364032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814d03c1493128f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f835f7fe640 1 -- 172.21.15.77:0/26447402 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364035040 con 0x56379b689b40 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f835f7fe640 1 -- 172.21.15.77:0/26447402 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8364031870 con 0x56379b689b40 2024-09-15T22:36:20.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f835f7fe640 1 -- 172.21.15.77:0/26447402 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364004740 con 0x56379b689b40 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f837df8b740 1 -- 172.21.15.77:0/26447402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 msgr2=0x56379b6a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.792+0000 7f837df8b740 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b6a3720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8364009230 tx=0x7f8364032e70 comp rx=0 tx=0).stop 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 -- 172.21.15.77:0/26447402 shutdown_connections 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b693370 0x56379b6aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b6a3720 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 --2- 172.21.15.77:0/26447402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b6885d0 0x56379b684e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 -- 172.21.15.77:0/26447402 >> 172.21.15.77:0/26447402 conn(0x56379b651310 msgr2=0x56379b684700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 -- 172.21.15.77:0/26447402 shutdown_connections 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 -- 172.21.15.77:0/26447402 wait complete. 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 -- start start 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 Processor -- start 2024-09-15T22:36:20.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.793+0000 7f837df8b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b69ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b693370 0x56379b69b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83741addb0 con 0x56379b6885d0 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83741ad6e0 con 0x56379b693370 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83741aa9d0 con 0x56379b689b40 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 10 client.0 _ll_register_callbacks cb 0x7f836c0b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43064/0 (socket says 172.21.15.77:43064) 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 -- 172.21.15.77:0/4099511213 learned_addr learned my addr 172.21.15.77:0/4099511213 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837a14f640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b693370 0x56379b69b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837914d640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b69ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 -- 172.21.15.77:0/4099511213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 msgr2=0x56379b69ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b69ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 -- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b693370 msgr2=0x56379b69b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b693370 0x56379b69b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 -- 172.21.15.77:0/4099511213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379b70fd10 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837994e640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x56379b6a9b20 tx=0x7f8370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8528bc2f5614c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f835d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837001d820 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f837001d9c0 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.794+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8364031870 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f837003f050 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 10 client.20865 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56379b710ff0 con 0x56379b6885d0 2024-09-15T22:36:20.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8364031870 con 0x56379b6885d0 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8370031c80 con 0x56379b6885d0 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 10 client.20865 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 10 client.20865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 20 client.20865 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b711710 0x56379b713b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 4 client.20865 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56379b5f5d70 con 0x56379b711710 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.795+0000 7f837914d640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b711710 0x56379b713b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837914d640 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b711710 0x56379b713b00 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4b0befe0cf9cc6 server_cookie=d95033a75ba11451 in_seq=0 out_seq=0 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f835d7fa640 10 client.20865 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f835d7fa640 1 -- 172.21.15.77:0/4099511213 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56379b5f5d70 con 0x56379b711710 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f835d7fa640 10 client.20865 handle_command_reply: tid=0 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 client.20865 shutdown 2024-09-15T22:36:20.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 msgr2=0x56379b69a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x56379b6a9b20 tx=0x7f8370004730 comp rx=0 tx=0).stop 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 shutdown_connections 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56379b711710 0x56379b713b00 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56379b693370 0x56379b69b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f835d7fa640 0 client.20865 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56379b689b40 0x56379b69ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.224 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 --2- 172.21.15.77:0/4099511213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56379b6885d0 0x56379b69a850 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 >> 172.21.15.77:0/4099511213 conn(0x56379b651310 msgr2=0x56379b68a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 shutdown_connections 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 1 -- 172.21.15.77:0/4099511213 wait complete. 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.796+0000 7f837df8b740 20 client.20865 trim_cache size 0 max 16384 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 msgr2=0x7f83600a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 0x7f83600a5750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8370009bd0 tx=0x7f8370005f90 comp rx=0 tx=0).stop 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 msgr2=0x7f837419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 0x7f837419cd20 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f836c00da10 tx=0x7f836c00ded0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 shutdown_connections 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f83600a3290 0x7f83600a5750 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8374106f40 0x7f837419cd20 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83741027c0 0x7f8374198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 --2- 172.21.15.77:0/1472504321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8374101e10 0x7f8374198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:20.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 >> 172.21.15.77:0/1472504321 conn(0x7f83740fd8d0 msgr2=0x7f83740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:20.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 shutdown_connections 2024-09-15T22:36:20.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:18.797+0000 7f837bbb2640 1 -- 172.21.15.77:0/1472504321 wait complete. 2024-09-15T22:36:20.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 180320 14.68 MB/sec execute 501 sec latency 244.335 ms 2024-09-15T22:36:20.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:20 smithi077 ceph-mon[30980]: pgmap v756: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:20.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:20 smithi196 ceph-mon[35109]: pgmap v756: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:20.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:20 smithi117 ceph-mon[35021]: pgmap v756: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 425 op/s 2024-09-15T22:36:21.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 180739 14.70 MB/sec execute 502 sec latency 241.294 ms 2024-09-15T22:36:22.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 181081 14.70 MB/sec execute 503 sec latency 226.374 ms 2024-09-15T22:36:22.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:22 smithi077 ceph-mon[30980]: pgmap v757: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 405 op/s 2024-09-15T22:36:22.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:22 smithi196 ceph-mon[35109]: pgmap v757: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 405 op/s 2024-09-15T22:36:22.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:22 smithi117 ceph-mon[35021]: pgmap v757: 129 pgs: 129 active+clean; 1.9 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 405 op/s 2024-09-15T22:36:23.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 181533 14.74 MB/sec execute 504 sec latency 193.392 ms 2024-09-15T22:36:23.961 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:36:23.961 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:36:24.290 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:24.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 181896 14.75 MB/sec execute 505 sec latency 180.645 ms 2024-09-15T22:36:24.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:24 smithi077 ceph-mon[30980]: pgmap v758: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 408 op/s 2024-09-15T22:36:24.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:24 smithi196 ceph-mon[35109]: pgmap v758: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 408 op/s 2024-09-15T22:36:24.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:24 smithi117 ceph-mon[35021]: pgmap v758: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 408 op/s 2024-09-15T22:36:24.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 -- 172.21.15.77:0/3875536533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c102ef0 msgr2=0x7f880c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 --2- 172.21.15.77:0/3875536533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c102ef0 0x7f880c10da40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402a950 comp rx=0 tx=0).stop 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 -- 172.21.15.77:0/3875536533 shutdown_connections 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 --2- 172.21.15.77:0/3875536533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c102ef0 0x7f880c10da40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 --2- 172.21.15.77:0/3875536533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102570 0x7f880c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 --2- 172.21.15.77:0/3875536533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c100e20 0x7f880c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 -- 172.21.15.77:0/3875536533 >> 172.21.15.77:0/3875536533 conn(0x7f880c0fc7d0 msgr2=0x7f880c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 -- 172.21.15.77:0/3875536533 shutdown_connections 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.842+0000 7f88111cf640 1 -- 172.21.15.77:0/3875536533 wait complete. 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.843+0000 7f88111cf640 1 Processor -- start 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.843+0000 7f88111cf640 1 -- start start 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 0x7f880c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 0x7f880c1a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10ff30 con 0x7f880c100e20 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f880c06ae40 con 0x7f880c102570 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f88111cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1100b0 con 0x7f880c102ef0 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 0x7f880c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56616/0 (socket says 172.21.15.77:56616) 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 0x7f880c1a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880ad9d640 1 -- 172.21.15.77:0/3546020450 learned_addr learned my addr 172.21.15.77:0/3546020450 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 -- 172.21.15.77:0/3546020450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 msgr2=0x7f880c1a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 0x7f880c1a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 -- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 msgr2=0x7f880c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 -- 172.21.15.77:0/3546020450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1a69e0 con 0x7f880c102570 2024-09-15T22:36:24.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880ad9d640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880b59e640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 0x7f880c1a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.844+0000 7f880a59c640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 0x7f880c19b970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f880000e9f0 tx=0x7f880000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.845+0000 7f87f3fff640 1 -- 172.21.15.77:0/3546020450 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000ccc0 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.845+0000 7f87f3fff640 1 -- 172.21.15.77:0/3546020450 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800005020 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.845+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1a6cd0 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.845+0000 7f87f3fff640 1 -- 172.21.15.77:0/3546020450 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000049e0 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.845+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a71c0 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 Processor -- start 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 -- start start 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f87f3fff640 1 -- 172.21.15.77:0/3546020450 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8800005560 con 0x7f880c102570 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5618f431c280 con 0x5618f43b7480 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618f42c3b10 con 0x5618f43c1d40 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.848+0000 7f88135a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5618f4318240 con 0x5618f43b6b60 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:37902/0 (socket says 172.21.15.77:37902) 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 -- 172.21.15.77:0/2206888100 learned_addr learned my addr 172.21.15.77:0/2206888100 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880a59c640 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 -- 172.21.15.77:0/2206888100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 msgr2=0x5618f43c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 -- 172.21.15.77:0/2206888100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 msgr2=0x5618f43c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43c1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 -- 172.21.15.77:0/2206888100 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f42d01f0 con 0x5618f43b6b60 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880ad9d640 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.849+0000 7f880b59e640 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f8804009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27d23202129521e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f87f17fa640 1 -- 172.21.15.77:0/2206888100 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804048070 con 0x5618f43b6b60 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f87f17fa640 1 -- 172.21.15.77:0/2206888100 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804009820 con 0x5618f43b6b60 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f87f17fa640 1 -- 172.21.15.77:0/2206888100 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804034040 con 0x5618f43b6b60 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f87f3fff640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 0x7f87f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f880ad9d640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 0x7f87f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f87f3fff640 1 -- 172.21.15.77:0/3546020450 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f88000cff80 con 0x7f880c102570 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f88135a8740 1 -- 172.21.15.77:0/2206888100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 msgr2=0x5618f43b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.850+0000 7f88135a8740 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f8804009360 comp rx=0 tx=0).stop 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- 172.21.15.77:0/2206888100 shutdown_connections 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- 172.21.15.77:0/2206888100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43b6f40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- 172.21.15.77:0/2206888100 >> 172.21.15.77:0/2206888100 conn(0x5618f43b30f0 msgr2=0x5618f43b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- 172.21.15.77:0/2206888100 shutdown_connections 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f880ad9d640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 0x7f87f40a5870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009620 tx=0x7f87fc005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- 172.21.15.77:0/2206888100 wait complete. 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- start start 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 Processor -- start 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b7480 0x5618f43bf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 0x5618f43bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5618f43c4910 con 0x5618f43b6b60 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618f42d6f10 con 0x5618f43b7480 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.851+0000 7f88135a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5618f43c4a90 con 0x5618f43c1d40 2024-09-15T22:36:24.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f88135a8740 10 client.0 _ll_register_callbacks cb 0x5618f429fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56638/0 (socket says 172.21.15.77:56638) 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 -- 172.21.15.77:0/1201833760 learned_addr learned my addr 172.21.15.77:0/1201833760 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880a59c640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b7480 0x5618f43bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880b59e640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 0x5618f43bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 -- 172.21.15.77:0/1201833760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 msgr2=0x5618f43bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 0x5618f43bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 -- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b7480 msgr2=0x5618f43bf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b7480 0x5618f43bf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 -- 172.21.15.77:0/1201833760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f408d6a0 con 0x5618f43b6b60 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880b59e640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 0x5618f43bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f880ad9d640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004600 tx=0x7f87fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b643384f15417bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f87eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc003b70 con 0x5618f43b6b60 2024-09-15T22:36:24.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f87fc003d10 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8804009820 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87fc01e470 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f88135a8740 10 client.20877 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.852+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5618f43d0570 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804009820 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f87fc027070 con 0x5618f43b6b60 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 10 client.20877 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 10 client.20877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 20 client.20877 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f43d1a30 0x5618f43d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 4 client.20877 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.853+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f43297f0 con 0x5618f43d1a30 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.854+0000 7f880a59c640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f43d1a30 0x5618f43d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.854+0000 7f87eeff5640 10 client.20877 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.854+0000 7f880a59c640 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f43d1a30 0x5618f43d3e20 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e2bc2771d35063 server_cookie=5c3730166590cf31 in_seq=0 out_seq=0 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f87eeff5640 1 -- 172.21.15.77:0/1201833760 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618f43297f0 con 0x5618f43d1a30 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f87eeff5640 10 client.20877 handle_command_reply: tid=0 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 client.20877 shutdown 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 msgr2=0x5618f43bec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004600 tx=0x7f87fc004730 comp rx=0 tx=0).stop 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 shutdown_connections 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f43d1a30 0x5618f43d3e20 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43c1d40 0x5618f43bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b7480 0x5618f43bf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 --2- 172.21.15.77:0/1201833760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b6b60 0x5618f43bec20 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 >> 172.21.15.77:0/1201833760 conn(0x5618f43b30f0 msgr2=0x5618f43b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.856+0000 7f87eeff5640 0 client.20877 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.857+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 shutdown_connections 2024-09-15T22:36:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.857+0000 7f88135a8740 1 -- 172.21.15.77:0/1201833760 wait complete. 2024-09-15T22:36:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.857+0000 7f88135a8740 20 client.20877 trim_cache size 0 max 16384 2024-09-15T22:36:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.876+0000 7f88135a8740 1 Processor -- start 2024-09-15T22:36:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.876+0000 7f88135a8740 1 -- start start 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.876+0000 7f88135a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f88135a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f88135a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f88135a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b1160 con 0x5618f43b7480 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f88135a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b2ca0 con 0x5618f43c1d40 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f88135a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b1830 con 0x5618f43b6b60 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56654/0 (socket says 172.21.15.77:56654) 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 -- 172.21.15.77:0/344522911 learned_addr learned my addr 172.21.15.77:0/344522911 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880b59e640 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880ad9d640 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 -- 172.21.15.77:0/344522911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 msgr2=0x5618f43bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 -- 172.21.15.77:0/344522911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 msgr2=0x5618f43d9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43d9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 -- 172.21.15.77:0/344522911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f43ae830 con 0x5618f43b7480 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f880a59c640 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f8800005480 tx=0x7f88000bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3cc1b0beae0245 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f87f0ff9640 1 -- 172.21.15.77:0/344522911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880001c040 con 0x5618f43b7480 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f87f0ff9640 1 -- 172.21.15.77:0/344522911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88000051d0 con 0x5618f43b7480 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.877+0000 7f87f0ff9640 1 -- 172.21.15.77:0/344522911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000cd5c0 con 0x5618f43b7480 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 -- 172.21.15.77:0/344522911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 msgr2=0x5618f43d96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f8800005480 tx=0x7f88000bc040 comp rx=0 tx=0).stop 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 -- 172.21.15.77:0/344522911 shutdown_connections 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43c1d40 0x5618f43d9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43b7480 0x5618f43d96f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 --2- 172.21.15.77:0/344522911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b6b60 0x5618f43bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 -- 172.21.15.77:0/344522911 >> 172.21.15.77:0/344522911 conn(0x5618f43b30f0 msgr2=0x5618f43bd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.878+0000 7f88135a8740 1 -- 172.21.15.77:0/344522911 shutdown_connections 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 -- 172.21.15.77:0/344522911 wait complete. 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 -- start start 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 Processor -- start 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 0x5618f43c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 0x5618f43ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 0x5618f43bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b25d0 con 0x5618f43c1d40 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1b1f00 con 0x5618f43b6b60 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1af160 con 0x5618f43b7480 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.879+0000 7f88135a8740 10 client.0 _ll_register_callbacks cb 0x7f87fc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 0x5618f43ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 0x5618f43c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 0x5618f43c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49512/0 (socket says 172.21.15.77:49512) 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880ad9d640 1 -- 172.21.15.77:0/3561634094 learned_addr learned my addr 172.21.15.77:0/3561634094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880b59e640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 0x5618f43bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 -- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 msgr2=0x5618f43c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 0x5618f43c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 -- 172.21.15.77:0/3561634094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 msgr2=0x5618f43bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 0x5618f43bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 -- 172.21.15.77:0/3561634094 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f43b72b0 con 0x5618f43b7480 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880b59e640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 0x5618f43bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f880a59c640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 0x5618f43ca220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8800005380 tx=0x7f880000c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f04422aa51a137 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f87eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000b7020 con 0x5618f43b7480 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88000051d0 con 0x5618f43b7480 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.880+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f88000ba740 con 0x5618f43b7480 2024-09-15T22:36:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f880001c040 con 0x5618f43b7480 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 10 client.29464 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5618f443bde0 con 0x5618f43b7480 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f880009bed0 con 0x5618f43b7480 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f88000ba350 con 0x5618f43b7480 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 10 client.29464 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 10 client.29464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 20 client.29464 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f443c560 0x5618f443e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 4 client.29464 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f88000051d0 con 0x5618f443c560 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.881+0000 7f880ad9d640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f443c560 0x5618f443e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.882+0000 7f87eeff5640 10 client.29464 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.882+0000 7f880ad9d640 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f443c560 0x5618f443e980 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705525d50afaaa0a server_cookie=5649724206b42c2f in_seq=0 out_seq=0 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.884+0000 7f87eeff5640 1 -- 172.21.15.77:0/3561634094 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f88000051d0 con 0x5618f443c560 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.884+0000 7f87eeff5640 10 client.29464 handle_command_reply: tid=0 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.884+0000 7f88135a8740 1 client.29464 shutdown 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 msgr2=0x5618f43ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 0x5618f43ca220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8800005380 tx=0x7f880000c840 comp rx=0 tx=0).stop 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 shutdown_connections 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5618f443c560 0x5618f443e980 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5618f43c1d40 0x5618f43bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618f43b7480 0x5618f43ca220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f87eeff5640 0 client.29464 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 --2- 172.21.15.77:0/3561634094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618f43b6b60 0x5618f43c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 >> 172.21.15.77:0/3561634094 conn(0x5618f43b30f0 msgr2=0x5618f43bdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 shutdown_connections 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 1 -- 172.21.15.77:0/3561634094 wait complete. 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.885+0000 7f88135a8740 20 client.29464 trim_cache size 0 max 16384 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:36:24.893 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "73e472d6-f5eb-42c8-80ef-71ea99eb8df6", 2024-09-15T22:36:24.894 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:36:24.894 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:36:24.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 msgr2=0x7f87f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 0x7f87f40a5870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009620 tx=0x7f87fc005f90 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 msgr2=0x7f880c19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 0x7f880c19b970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f880000e9f0 tx=0x7f880000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 shutdown_connections 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f87f40a33b0 0x7f87f40a5870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f880c102ef0 0x7f880c1a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880c102570 0x7f880c19b970 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 --2- 172.21.15.77:0/3546020450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f880c100e20 0x7f880c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 >> 172.21.15.77:0/3546020450 conn(0x7f880c0fc7d0 msgr2=0x7f880c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 shutdown_connections 2024-09-15T22:36:24.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:24.886+0000 7f88111cf640 1 -- 172.21.15.77:0/3546020450 wait complete. 2024-09-15T22:36:25.406 INFO:tasks.quiescer.fs.[cephfs]:Total duration (54.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:36:25.406 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-15T22:36:25.407 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:25.617 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:36:25.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 182156 14.74 MB/sec execute 506 sec latency 187.666 ms 2024-09-15T22:36:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:25.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.833 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:25.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:25 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:36:25.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:25.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:25 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:25.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:25 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2846456598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109450 msgr2=0x7f17ac109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2846456598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109450 0x7f17ac109830 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009be0 tx=0x7f17a402a6c0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2846456598 shutdown_connections 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2846456598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac10ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2846456598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109d70 0x7f17ac10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2846456598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109450 0x7f17ac109830 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2846456598 >> 172.21.15.77:0/2846456598 conn(0x7f17ac073660 msgr2=0x7f17ac073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2846456598 shutdown_connections 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.373+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2846456598 wait complete. 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.374+0000 7f17b1b2e640 1 Processor -- start 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.374+0000 7f17b1b2e640 1 -- start start 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 0x7f17ac1a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 0x7f17ac1a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac11a5f0 con 0x7f17ac109d70 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac11a470 con 0x7f17ac109450 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b1b2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac11a770 con 0x7f17ac10e7f0 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 0x7f17ac1a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b0b2c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 0x7f17ac1a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17abfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 0x7f17ac1a8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56674/0 (socket says 172.21.15.77:56674) 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17abfff640 1 -- 172.21.15.77:0/2770071268 learned_addr learned my addr 172.21.15.77:0/2770071268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.375+0000 7f17b132d640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b0b2c640 1 -- 172.21.15.77:0/2770071268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 msgr2=0x7f17ac115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b0b2c640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac115300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b0b2c640 1 -- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 msgr2=0x7f17ac1a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b0b2c640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 0x7f17ac1a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b0b2c640 1 -- 172.21.15.77:0/2770071268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac115b30 con 0x7f17ac109450 2024-09-15T22:36:26.380 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.376+0000 7f17b132d640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.377+0000 7f17b0b2c640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 0x7f17ac1a89c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17a402abe0 tx=0x7f17a4030620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.377+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4041020 con 0x7f17ac109450 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.377+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a4031920 con 0x7f17ac109450 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.377+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac115dc0 con 0x7f17ac109450 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.377+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac19fa80 con 0x7f17ac109450 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.378+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4041020 con 0x7f17ac109450 2024-09-15T22:36:26.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.379+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ac06eef0 con 0x7f17ac109450 2024-09-15T22:36:26.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.382+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17a4044020 con 0x7f17ac109450 2024-09-15T22:36:26.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.383+0000 7f17a9ffb640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 0x7f17940a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.383+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f17a40b8c40 con 0x7f17ac109450 2024-09-15T22:36:26.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.384+0000 7f17abfff640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 0x7f17940a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.387 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.385+0000 7f17abfff640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 0x7f17940a5a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17ac1a4290 tx=0x7f179c00b2c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:26.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.388+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17a404b070 con 0x7f17ac109450 2024-09-15T22:36:26.443 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:26.623 INFO:tasks.workunit.client.0.smithi077.stdout: 10 182475 14.76 MB/sec execute 507 sec latency 252.065 ms 2024-09-15T22:36:26.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.764+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17ac1a3fd0 con 0x7f17ac109450 2024-09-15T22:36:26.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.766+0000 7f17a9ffb640 1 -- 172.21.15.77:0/2770071268 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f17a404b070 con 0x7f17ac109450 2024-09-15T22:36:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:36:26.769 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:36:26.769 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 msgr2=0x7f17940a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 0x7f17940a5a70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17ac1a4290 tx=0x7f179c00b2c0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 msgr2=0x7f17ac1a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 0x7f17ac1a89c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17a402abe0 tx=0x7f17a4030620 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 shutdown_connections 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f17940a35b0 0x7f17940a5a70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ac10e7f0 0x7f17ac115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ac109d70 0x7f17ac1a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 --2- 172.21.15.77:0/2770071268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17ac109450 0x7f17ac1a89c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 >> 172.21.15.77:0/2770071268 conn(0x7f17ac073660 msgr2=0x7f17ac10a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:26.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 shutdown_connections 2024-09-15T22:36:26.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:26.772+0000 7f17b1b2e640 1 -- 172.21.15.77:0/2770071268 wait complete. 2024-09-15T22:36:26.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:26 smithi077 ceph-mon[30980]: pgmap v759: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 381 op/s 2024-09-15T22:36:26.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:26 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:36:26.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:26 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:26 smithi196 ceph-mon[35109]: pgmap v759: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 381 op/s 2024-09-15T22:36:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:26 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:36:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:26 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:26 smithi117 ceph-mon[35021]: pgmap v759: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 11 MiB/s wr, 381 op/s 2024-09-15T22:36:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:26 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:36:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:26.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:26 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:36:27.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 182659 14.74 MB/sec execute 508 sec latency 720.093 ms 2024-09-15T22:36:27.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:27 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2770071268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:27.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:27 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2770071268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:27.892 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:27 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2770071268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:28.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 182986 14.76 MB/sec execute 509 sec latency 212.951 ms 2024-09-15T22:36:28.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:28 smithi077 ceph-mon[30980]: pgmap v760: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 479 op/s 2024-09-15T22:36:28.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:28 smithi117 ceph-mon[35021]: pgmap v760: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 479 op/s 2024-09-15T22:36:29.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:28 smithi196 ceph-mon[35109]: pgmap v760: 129 pgs: 129 active+clean; 1.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 479 op/s 2024-09-15T22:36:29.235 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-15T22:36:29.236 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 -- 172.21.15.77:0/813500258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 msgr2=0x7f5594103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 --2- 172.21.15.77:0/813500258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f5594103170 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5584009930 tx=0x7f558402a420 comp rx=0 tx=0).stop 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 -- 172.21.15.77:0/813500258 shutdown_connections 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 --2- 172.21.15.77:0/813500258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 0x7f559410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 --2- 172.21.15.77:0/813500258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f5594107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 --2- 172.21.15.77:0/813500258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f5594103170 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 -- 172.21.15.77:0/813500258 >> 172.21.15.77:0/813500258 conn(0x7f55940fc7d0 msgr2=0x7f55940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 -- 172.21.15.77:0/813500258 shutdown_connections 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.474+0000 7f5599198640 1 -- 172.21.15.77:0/813500258 wait complete. 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.475+0000 7f5599198640 1 Processor -- start 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 -- start start 2024-09-15T22:36:29.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f55941a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 0x7f559410de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5594112090 con 0x7f5594102d90 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5594111f10 con 0x7f5594108290 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5599198640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5594112210 con 0x7f5594103740 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f559259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f559259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47850/0 (socket says 172.21.15.77:47850) 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f559259c640 1 -- 172.21.15.77:0/2186569406 learned_addr learned my addr 172.21.15.77:0/2186569406 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f5592d9d640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f55941a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.476+0000 7f559359e640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 0x7f559410de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 -- 172.21.15.77:0/2186569406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 msgr2=0x7f559410de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 0x7f559410de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 -- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 msgr2=0x7f55941a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f55941a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 -- 172.21.15.77:0/2186569406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559410e5c0 con 0x7f5594103740 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f559259c640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9c0 tx=0x7f558800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f557bfff640 1 -- 172.21.15.77:0/2186569406 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558800cd80 con 0x7f5594103740 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f557bfff640 1 -- 172.21.15.77:0/2186569406 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5588005500 con 0x7f5594103740 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f557bfff640 1 -- 172.21.15.77:0/2186569406 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588010690 con 0x7f5594103740 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559410e8b0 con 0x7f5594103740 2024-09-15T22:36:29.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.477+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941ab710 con 0x7f5594103740 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 Processor -- start 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 -- start start 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0332144a0 0x55f033214880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f03321f680 0x55f033221a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f03317a2e0 con 0x55f033214dc0 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f033121ce0 con 0x55f0332144a0 2024-09-15T22:36:29.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.482+0000 7f559b571740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0331762a0 con 0x55f03321f680 2024-09-15T22:36:29.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.483+0000 7f557bfff640 1 -- 172.21.15.77:0/2186569406 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5588005020 con 0x7f5594103740 2024-09-15T22:36:29.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.484+0000 7f557bfff640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 0x7f557c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f557bfff640 1 -- 172.21.15.77:0/2186569406 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f55880d05c0 con 0x7f5594103740 2024-09-15T22:36:29.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47728/0 (socket says 172.21.15.77:47728) 2024-09-15T22:36:29.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 -- 172.21.15.77:0/570406752 learned_addr learned my addr 172.21.15.77:0/570406752 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:29.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 -- 172.21.15.77:0/570406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f03321f680 msgr2=0x55f033221a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f03321f680 0x55f033221a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 -- 172.21.15.77:0/570406752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0332144a0 msgr2=0x55f033214880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0332144a0 0x55f033214880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.485+0000 7f5592d9d640 1 -- 172.21.15.77:0/570406752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f033106440 con 0x55f033214dc0 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.486+0000 7f5592d9d640 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f558402a940 tx=0x7f5584005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48efd323ce87c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.486+0000 7f55797fa640 1 -- 172.21.15.77:0/570406752 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558403f070 con 0x55f033214dc0 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.486+0000 7f55797fa640 1 -- 172.21.15.77:0/570406752 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584004030 con 0x55f033214dc0 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.486+0000 7f55797fa640 1 -- 172.21.15.77:0/570406752 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584035070 con 0x55f033214dc0 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.487+0000 7f559b571740 1 -- 172.21.15.77:0/570406752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 msgr2=0x55f03321f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.487+0000 7f559b571740 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f558402a940 tx=0x7f5584005dd0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559359e640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 0x7f557c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559b571740 1 -- 172.21.15.77:0/570406752 shutdown_connections 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559b571740 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f03321f680 0x55f033221a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559b571740 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f033214dc0 0x55f03321f140 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559b571740 1 --2- 172.21.15.77:0/570406752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0332144a0 0x55f033214880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.490+0000 7f559b571740 1 -- 172.21.15.77:0/570406752 >> 172.21.15.77:0/570406752 conn(0x55f033210a30 msgr2=0x55f033210e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.491+0000 7f559b571740 1 -- 172.21.15.77:0/570406752 shutdown_connections 2024-09-15T22:36:29.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.492+0000 7f559359e640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 0x7f557c0a5970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f558c005c40 tx=0x7f558c004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.492+0000 7f559b571740 1 -- 172.21.15.77:0/570406752 wait complete. 2024-09-15T22:36:29.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.493+0000 7f559b571740 1 -- start start 2024-09-15T22:36:29.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.493+0000 7f559b571740 1 Processor -- start 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0332144a0 0x55f03321efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f033214dc0 0x55f03321c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f033222250 con 0x55f0332144a0 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f033135050 con 0x55f03321f680 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f0332223d0 con 0x55f033214dc0 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559b571740 10 client.0 _ll_register_callbacks cb 0x55f033175160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:51112/0 (socket says 172.21.15.77:51112) 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559359e640 1 -- 172.21.15.77:0/3874733160 learned_addr learned my addr 172.21.15.77:0/3874733160 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:29.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.494+0000 7f559259c640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f033214dc0 0x55f03321c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 -- 172.21.15.77:0/3874733160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f033214dc0 msgr2=0x55f03321c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f033214dc0 0x55f03321c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 -- 172.21.15.77:0/3874733160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0332144a0 msgr2=0x55f03321efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0332144a0 0x55f03321efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 -- 172.21.15.77:0/3874733160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f032eeb3c0 con 0x55f03321f680 2024-09-15T22:36:29.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f559359e640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f558c011a10 tx=0x7f558c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a18ea76eb60967 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.495+0000 7f5576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c01c620 con 0x55f03321f680 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584004030 con 0x55f03321f680 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f558c01c7c0 con 0x55f03321f680 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f558c02e050 con 0x55f03321f680 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f559b571740 10 client.29558 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:29.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.496+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f03322dcc0 con 0x55f03321f680 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5584004030 con 0x55f03321f680 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f558c026c20 con 0x55f03321f680 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 10 client.29558 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 10 client.29558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 20 client.29558 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03322f500 0x55f0332319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 4 client.29558 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:29.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f033187850 con 0x55f03322f500 2024-09-15T22:36:29.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.497+0000 7f5592d9d640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03322f500 0x55f0332319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.498+0000 7f5576ff5640 10 client.29558 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:29.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.498+0000 7f5592d9d640 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03322f500 0x55f0332319a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd464b4dcede1199 server_cookie=ecd672d6b2c4ba93 in_seq=0 out_seq=0 2024-09-15T22:36:29.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f5576ff5640 1 -- 172.21.15.77:0/3874733160 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f033187850 con 0x55f03322f500 2024-09-15T22:36:29.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f5576ff5640 10 client.29558 handle_command_reply: tid=0 2024-09-15T22:36:29.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 client.29558 shutdown 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 msgr2=0x55f03321c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f558c011a10 tx=0x7f558c009500 comp rx=0 tx=0).stop 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 shutdown_connections 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03322f500 0x55f0332319a0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f03321f680 0x55f03321c790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f033214dc0 0x55f03321c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 --2- 172.21.15.77:0/3874733160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f0332144a0 0x55f03321efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.501+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 >> 172.21.15.77:0/3874733160 conn(0x55f033210a30 msgr2=0x55f033216760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.502+0000 7f5576ff5640 0 client.29558 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.502+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 shutdown_connections 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.502+0000 7f559b571740 1 -- 172.21.15.77:0/3874733160 wait complete. 2024-09-15T22:36:29.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.502+0000 7f559b571740 20 client.29558 trim_cache size 0 max 16384 2024-09-15T22:36:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.524+0000 7f559b571740 1 Processor -- start 2024-09-15T22:36:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.524+0000 7f559b571740 1 -- start start 2024-09-15T22:36:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f03322b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033231510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5650 con 0x55f03321f680 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b7190 con 0x55f033214dc0 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559b571740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5d20 con 0x55f0332144a0 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033231510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47754/0 (socket says 172.21.15.77:47754) 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 -- 172.21.15.77:0/960997645 learned_addr learned my addr 172.21.15.77:0/960997645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:29.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f5592d9d640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f03322b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 -- 172.21.15.77:0/960997645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 msgr2=0x55f03322b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f03322b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 -- 172.21.15.77:0/960997645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 msgr2=0x55f033231510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033231510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.525+0000 7f559359e640 1 -- 172.21.15.77:0/960997645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03321b200 con 0x55f03321f680 2024-09-15T22:36:29.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f5592d9d640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f03322b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f559359e640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f558c0092f0 tx=0x7f558c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64c6ab7ccb6cff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f559259c640 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033231510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f5578ff9640 1 -- 172.21.15.77:0/960997645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c01d8d0 con 0x55f03321f680 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f5578ff9640 1 -- 172.21.15.77:0/960997645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c021030 con 0x55f03321f680 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.526+0000 7f5578ff9640 1 -- 172.21.15.77:0/960997645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c032400 con 0x55f03321f680 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 -- 172.21.15.77:0/960997645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 msgr2=0x55f0332177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f558c0092f0 tx=0x7f558c009e40 comp rx=0 tx=0).stop 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 -- 172.21.15.77:0/960997645 shutdown_connections 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f0332177a0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033231510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 --2- 172.21.15.77:0/960997645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f03322b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.527+0000 7f559b571740 1 -- 172.21.15.77:0/960997645 >> 172.21.15.77:0/960997645 conn(0x55f033210a30 msgr2=0x55f033217090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.528+0000 7f559b571740 1 -- 172.21.15.77:0/960997645 shutdown_connections 2024-09-15T22:36:29.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.528+0000 7f559b571740 1 -- 172.21.15.77:0/960997645 wait complete. 2024-09-15T22:36:29.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.528+0000 7f559b571740 1 -- start start 2024-09-15T22:36:29.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.528+0000 7f559b571740 1 Processor -- start 2024-09-15T22:36:29.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033223340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f033227480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f5592d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f5592d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47898/0 (socket says 172.21.15.77:47898) 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033223340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f5592d9d640 1 -- 172.21.15.77:0/3581320951 learned_addr learned my addr 172.21.15.77:0/3581320951 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b6ac0 con 0x55f03321f680 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b63f0 con 0x55f033214dc0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b3650 con 0x55f0332144a0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559359e640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f033227480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.529+0000 7f559b571740 10 client.0 _ll_register_callbacks cb 0x7f55941a5580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 -- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 msgr2=0x55f033223340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033223340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 -- 172.21.15.77:0/3581320951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 msgr2=0x55f033227480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f033227480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f033227b70 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5592d9d640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5584005c40 tx=0x7f558402a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff3e574caee04e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5576ff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584035040 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c021030 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.530+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f55840317b0 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f558405c050 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 10 client.29482 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f033299d80 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c021030 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f55840038f0 con 0x55f0332144a0 2024-09-15T22:36:29.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 10 client.29482 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 10 client.29482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 20 client.29482 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03329a0c0 0x55f03329c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 4 client.29482 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f033181d60 con 0x55f03329a0c0 2024-09-15T22:36:29.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.531+0000 7f559259c640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03329a0c0 0x55f03329c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.532+0000 7f5576ff5640 10 client.29482 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.532+0000 7f559259c640 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03329a0c0 0x55f03329c4b0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f589c7947b1c634 server_cookie=e52bdd0530cb2fa7 in_seq=0 out_seq=0 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f5576ff5640 1 -- 172.21.15.77:0/3581320951 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f033181d60 con 0x55f03329a0c0 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f5576ff5640 10 client.29482 handle_command_reply: tid=0 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f559b571740 1 client.29482 shutdown 2024-09-15T22:36:29.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 msgr2=0x55f033222e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5584005c40 tx=0x7f558402a420 comp rx=0 tx=0).stop 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.533+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 shutdown_connections 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f03329a0c0 0x55f03329c4b0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f03321f680 0x55f033227480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f033214dc0 0x55f033223340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f5576ff5640 0 client.29482 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 --2- 172.21.15.77:0/3581320951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f0332144a0 0x55f033222e00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 >> 172.21.15.77:0/3581320951 conn(0x55f033210a30 msgr2=0x55f0332178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 shutdown_connections 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 1 -- 172.21.15.77:0/3581320951 wait complete. 2024-09-15T22:36:29.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.534+0000 7f559b571740 20 client.29482 trim_cache size 0 max 16384 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "73e472d6-f5eb-42c8-80ef-71ea99eb8df6": { 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "73e472d6-f5eb-42c8-80ef-71ea99eb8df6", 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:29.539 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 msgr2=0x7f557c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 0x7f557c0a5970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f558c005c40 tx=0x7f558c004a40 comp rx=0 tx=0).stop 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 msgr2=0x7f55941a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9c0 tx=0x7f558800ee80 comp rx=0 tx=0).stop 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 shutdown_connections 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f557c0a34b0 0x7f557c0a5970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5594108290 0x7f559410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5594103740 0x7f55941a4e10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 --2- 172.21.15.77:0/2186569406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5594102d90 0x7f55941a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 >> 172.21.15.77:0/2186569406 conn(0x7f55940fc7d0 msgr2=0x7f5594104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 shutdown_connections 2024-09-15T22:36:29.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:29.536+0000 7f5599198640 1 -- 172.21.15.77:0/2186569406 wait complete. 2024-09-15T22:36:29.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183388 14.77 MB/sec execute 510 sec latency 163.595 ms 2024-09-15T22:36:30.343 INFO:tasks.cephfs.filesystem:scrub status for tag:73e472d6-f5eb-42c8-80ef-71ea99eb8df6 - {'path': '/', 'tag': '73e472d6-f5eb-42c8-80ef-71ea99eb8df6', 'options': 'recursive,force'} 2024-09-15T22:36:30.400 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:30.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183630 14.76 MB/sec execute 511 sec latency 182.386 ms 2024-09-15T22:36:30.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:30 smithi117 ceph-mon[35021]: pgmap v761: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.1 MiB/s wr, 329 op/s 2024-09-15T22:36:30.919 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:30 smithi077 ceph-mon[30980]: pgmap v761: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.1 MiB/s wr, 329 op/s 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/2345102677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4101de0 msgr2=0x7fa3c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/2345102677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4101de0 0x7fa3c41021c0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009900 tx=0x7fa3b802a410 comp rx=0 tx=0).stop 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/2345102677 shutdown_connections 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/2345102677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4106f10 0x7fa3c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/2345102677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4102790 0x7fa3c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/2345102677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4101de0 0x7fa3c41021c0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/2345102677 >> 172.21.15.77:0/2345102677 conn(0x7fa3c40fd8e0 msgr2=0x7fa3c40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/2345102677 shutdown_connections 2024-09-15T22:36:31.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.036+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/2345102677 wait complete. 2024-09-15T22:36:31.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3cb1ea640 1 Processor -- start 2024-09-15T22:36:31.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3cb1ea640 1 -- start start 2024-09-15T22:36:31.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3cb1ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3cb1ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 0x7fa3c41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3cb1ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 0x7fa3c419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 0x7fa3c41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.037+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47922/0 (socket says 172.21.15.77:47922) 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 -- 172.21.15.77:0/3367692389 learned_addr learned my addr 172.21.15.77:0/3367692389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4111f70 con 0x7fa3c4102790 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4111df0 con 0x7fa3c4106f10 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41120f0 con 0x7fa3c4101de0 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c9787640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 0x7fa3c419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 -- 172.21.15.77:0/3367692389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 msgr2=0x7fa3c419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 0x7fa3c419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 -- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 msgr2=0x7fa3c41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 0x7fa3c41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419d460 con 0x7fa3c4101de0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 0x7fa3c41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c9787640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 0x7fa3c419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006010 tx=0x7fa3b80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.038+0000 7fa3c1ffb640 1 -- 172.21.15.77:0/3367692389 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b803f030 con 0x7fa3c4101de0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.039+0000 7fa3c1ffb640 1 -- 172.21.15.77:0/3367692389 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8003ce0 con 0x7fa3c4101de0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.039+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419d6f0 con 0x7fa3c4101de0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.039+0000 7fa3c1ffb640 1 -- 172.21.15.77:0/3367692389 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8035070 con 0x7fa3c4101de0 2024-09-15T22:36:31.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.039+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a5030 con 0x7fa3c4101de0 2024-09-15T22:36:31.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.041+0000 7fa3c1ffb640 1 -- 172.21.15.77:0/3367692389 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa3b8042030 con 0x7fa3c4101de0 2024-09-15T22:36:31.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.042+0000 7fa3cd5c3740 1 Processor -- start 2024-09-15T22:36:31.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.042+0000 7fa3cd5c3740 1 -- start start 2024-09-15T22:36:31.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.042+0000 7fa3c1ffb640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 0x7fa3b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.042+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec982400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c9787640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 0x7fa3b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c1ffb640 1 -- 172.21.15.77:0/3367692389 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fa3b80f0590 con 0x7fa3c4101de0 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec984d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47798/0 (socket says 172.21.15.77:47798) 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 -- 172.21.15.77:0/1583403238 learned_addr learned my addr 172.21.15.77:0/1583403238 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec8dd2a0 con 0x55d8ec977760 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec884b10 con 0x55d8ec982940 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec8d9260 con 0x55d8ec978080 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec982400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c9787640 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec984d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 -- 172.21.15.77:0/1583403238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 msgr2=0x55d8ec982400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec982400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 -- 172.21.15.77:0/1583403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 msgr2=0x55d8ec984d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec984d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 -- 172.21.15.77:0/1583403238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ec64eb90 con 0x55d8ec977760 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.043+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8004650 tx=0x7fa3b803b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9ff27fcb54586f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.044+0000 7fa3c9787640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 0x7fa3b40a58f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55d8ec975070 tx=0x7fa3bc009d40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.044+0000 7fa3aeffd640 1 -- 172.21.15.77:0/1583403238 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8049040 con 0x55d8ec977760 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.044+0000 7fa3aeffd640 1 -- 172.21.15.77:0/1583403238 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b80bcca0 con 0x55d8ec977760 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.044+0000 7fa3aeffd640 1 -- 172.21.15.77:0/1583403238 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80c0950 con 0x55d8ec977760 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 msgr2=0x55d8ec977b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8004650 tx=0x7fa3b803b510 comp rx=0 tx=0).stop 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 shutdown_connections 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec984d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec982400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1583403238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec977b40 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 >> 172.21.15.77:0/1583403238 conn(0x55d8ec973cf0 msgr2=0x55d8ec9740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 shutdown_connections 2024-09-15T22:36:31.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1583403238 wait complete. 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 -- start start 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.045+0000 7fa3cd5c3740 1 Processor -- start 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec97f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec978080 0x55d8ec97fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec9802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec9852d0 con 0x55d8ec977760 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec8656e0 con 0x55d8ec978080 2024-09-15T22:36:31.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8ec985450 con 0x55d8ec982940 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3cd5c3740 10 client.0 _ll_register_callbacks cb 0x55d8ec889a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec97f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec9802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec97f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47806/0 (socket says 172.21.15.77:47806) 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c8f86640 1 -- 172.21.15.77:0/2175832538 learned_addr learned my addr 172.21.15.77:0/2175832538 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec978080 0x55d8ec97fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 -- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec978080 msgr2=0x55d8ec97fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec978080 0x55d8ec97fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 -- 172.21.15.77:0/2175832538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 msgr2=0x55d8ec97f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec97f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.046+0000 7fa3c9787640 1 -- 172.21.15.77:0/2175832538 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ec98e760 con 0x55d8ec982940 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3c9787640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec9802d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005fd0 tx=0x7fa3bc009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e55c94c42ce70a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc018de0 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fa3bc026470 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b80bcca0 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc034050 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 10 client.29494 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d8ec9910f0 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b80bcca0 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa3bc011640 con 0x55d8ec982940 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 10 client.29494 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 10 client.29494 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 20 client.29494 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:31.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8ec994ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 4 client.29494 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:31.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.047+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ec8ea9d0 con 0x55d8ec9926e0 2024-09-15T22:36:31.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.049+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8ec994ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.049+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8ec994ba0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9801a968ca674a7 server_cookie=cc9ad5a7e01a1ced in_seq=0 out_seq=0 2024-09-15T22:36:31.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.049+0000 7fa3a3fff640 10 client.29494 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.050+0000 7fa3a3fff640 1 -- 172.21.15.77:0/2175832538 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8ec8ea9d0 con 0x55d8ec9926e0 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.050+0000 7fa3a3fff640 10 client.29494 handle_command_reply: tid=0 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.050+0000 7fa3cd5c3740 1 client.29494 shutdown 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 msgr2=0x55d8ec9802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec9802d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005fd0 tx=0x7fa3bc009300 comp rx=0 tx=0).stop 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 shutdown_connections 2024-09-15T22:36:31.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8ec994ba0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec9802d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3a3fff640 0 client.29494 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec978080 0x55d8ec97fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/2175832538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec97f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 >> 172.21.15.77:0/2175832538 conn(0x55d8ec973cf0 msgr2=0x55d8ec979a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 shutdown_connections 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/2175832538 wait complete. 2024-09-15T22:36:31.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.051+0000 7fa3cd5c3740 20 client.29494 trim_cache size 0 max 16384 2024-09-15T22:36:31.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.075+0000 7fa3cd5c3740 1 Processor -- start 2024-09-15T22:36:31.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.075+0000 7fa3cd5c3740 1 -- start start 2024-09-15T22:36:31.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec977760 0x55d8ec99a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec97cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ace80 con 0x55d8ec978080 2024-09-15T22:36:31.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ae9c0 con 0x55d8ec977760 2024-09-15T22:36:31.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3cd5c3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ad550 con 0x55d8ec982940 2024-09-15T22:36:31.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47814/0 (socket says 172.21.15.77:47814) 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c3fff640 1 -- 172.21.15.77:0/1398250036 learned_addr learned my addr 172.21.15.77:0/1398250036 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec97cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec977760 0x55d8ec99a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 -- 172.21.15.77:0/1398250036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec977760 msgr2=0x55d8ec99a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec977760 0x55d8ec99a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 -- 172.21.15.77:0/1398250036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 msgr2=0x55d8ec99a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c9787640 1 -- 172.21.15.77:0/1398250036 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ec985070 con 0x55d8ec982940 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.076+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.077+0000 7fa3c9787640 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec97cab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00fc30 tx=0x7fa3bc009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f3af2103123c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.077+0000 7fa3ae7fc640 1 -- 172.21.15.77:0/1398250036 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc013a70 con 0x55d8ec982940 2024-09-15T22:36:31.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.077+0000 7fa3ae7fc640 1 -- 172.21.15.77:0/1398250036 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc02b070 con 0x55d8ec982940 2024-09-15T22:36:31.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.077+0000 7fa3ae7fc640 1 -- 172.21.15.77:0/1398250036 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc026c80 con 0x55d8ec982940 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1398250036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 msgr2=0x55d8ec97cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec97cab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00fc30 tx=0x7fa3bc009640 comp rx=0 tx=0).stop 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1398250036 shutdown_connections 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec982940 0x55d8ec97cab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec978080 0x55d8ec99a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/1398250036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec977760 0x55d8ec99a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1398250036 >> 172.21.15.77:0/1398250036 conn(0x55d8ec973cf0 msgr2=0x55d8ec9818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1398250036 shutdown_connections 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/1398250036 wait complete. 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 -- start start 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.078+0000 7fa3cd5c3740 1 Processor -- start 2024-09-15T22:36:31.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec986570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec986ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3cd5c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec986ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec986570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3c8f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec986570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47822/0 (socket says 172.21.15.77:47822) 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3c8f86640 1 -- 172.21.15.77:0/965908434 learned_addr learned my addr 172.21.15.77:0/965908434 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.079+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ae2f0 con 0x55d8ec977760 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41adc20 con 0x55d8ec982940 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec986ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41aae80 con 0x55d8ec978080 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3cd5c3740 10 client.0 _ll_register_callbacks cb 0x7fa3bc0139c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c9787640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec986ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 -- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 msgr2=0x55d8ec986ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec986ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 -- 172.21.15.77:0/965908434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 msgr2=0x55d8ec986570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec986570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ec9fc770 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3c3fff640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec986ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55d8ec98ed10 tx=0x7fa3b000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0a44a69c1a0e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0023060 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.080+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fa3b0004250 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3bc02b070 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3b0004450 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 10 client.29500 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d8ec9fd9f0 con 0x55d8ec978080 2024-09-15T22:36:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b0003ad0 con 0x55d8ec978080 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa3b001e750 con 0x55d8ec978080 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 10 client.29500 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 10 client.29500 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 20 client.29500 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8eca00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 4 client.29500 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.081+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x7fa3bc02b070 con 0x55d8ec9926e0 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.082+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8eca00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.083+0000 7fa3c8f86640 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8eca00230 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d0af2cf9f96e9e server_cookie=492b645b5ae6e4f4 in_seq=0 out_seq=0 2024-09-15T22:36:31.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.083+0000 7fa3a3fff640 10 client.29500 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:30 smithi196 ceph-mon[35109]: pgmap v761: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.1 MiB/s wr, 329 op/s 2024-09-15T22:36:31.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183769 14.75 MB/sec execute 512 sec latency 556.314 ms 2024-09-15T22:36:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.936+0000 7fa3a3fff640 1 -- 172.21.15.77:0/965908434 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa3bc02b070 con 0x55d8ec9926e0 2024-09-15T22:36:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.936+0000 7fa3a3fff640 10 client.29500 handle_command_reply: tid=0 2024-09-15T22:36:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.936+0000 7fa3cd5c3740 1 client.29500 shutdown 2024-09-15T22:36:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 msgr2=0x55d8ec986ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec986ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55d8ec98ed10 tx=0x7fa3b000f790 comp rx=0 tx=0).stop 2024-09-15T22:36:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:36:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:36:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:36:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:36:31.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:31 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 shutdown_connections 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d8ec9926e0 0x55d8eca00230 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d8ec982940 0x55d8ec986ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8ec978080 0x55d8ec986ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3a3fff640 0 client.29500 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 --2- 172.21.15.77:0/965908434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8ec977760 0x55d8ec986570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 >> 172.21.15.77:0/965908434 conn(0x55d8ec973cf0 msgr2=0x55d8ec9810b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 shutdown_connections 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 1 -- 172.21.15.77:0/965908434 wait complete. 2024-09-15T22:36:31.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.937+0000 7fa3cd5c3740 20 client.29500 trim_cache size 0 max 16384 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 82, 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "e47c9c22": { 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 82, 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:31.942 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 147.2, 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:31.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:31.944 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:36:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 msgr2=0x7fa3b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 0x7fa3b40a58f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55d8ec975070 tx=0x7fa3bc009d40 comp rx=0 tx=0).stop 2024-09-15T22:36:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 msgr2=0x7fa3c41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:31.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006010 tx=0x7fa3b80042f0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 shutdown_connections 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa3b40a3430 0x7fa3b40a58f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3c4106f10 0x7fa3c419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3c4102790 0x7fa3c41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 --2- 172.21.15.77:0/3367692389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101de0 0x7fa3c41984b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 >> 172.21.15.77:0/3367692389 conn(0x7fa3c40fd8e0 msgr2=0x7fa3c40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 shutdown_connections 2024-09-15T22:36:31.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:31.938+0000 7fa3cb1ea640 1 -- 172.21.15.77:0/3367692389 wait complete. 2024-09-15T22:36:32.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:36:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:36:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:36:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:36:32.104 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:31 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:36:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:36:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:36:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:36:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:36:32.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:31 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:36:32.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.72 MB/sec execute 513 sec latency 1556.347 ms 2024-09-15T22:36:32.760 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e47c9c22', quiesce took 7.4 seconds. Will release after: 19.799999999999997 2024-09-15T22:36:32.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:32 smithi117 ceph-mon[35021]: pgmap v762: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.7 MiB/s wr, 383 op/s 2024-09-15T22:36:33.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:32 smithi077 ceph-mon[30980]: pgmap v762: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.7 MiB/s wr, 383 op/s 2024-09-15T22:36:33.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:32 smithi196 ceph-mon[35109]: pgmap v762: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.7 MiB/s wr, 383 op/s 2024-09-15T22:36:33.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.69 MB/sec execute 514 sec latency 2556.393 ms 2024-09-15T22:36:34.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:33 smithi077 ceph-mon[30980]: pgmap v763: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 472 op/s 2024-09-15T22:36:34.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:33 smithi196 ceph-mon[35109]: pgmap v763: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 472 op/s 2024-09-15T22:36:34.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:33 smithi117 ceph-mon[35021]: pgmap v763: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 472 op/s 2024-09-15T22:36:34.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.66 MB/sec execute 515 sec latency 3556.446 ms 2024-09-15T22:36:35.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.63 MB/sec execute 516 sec latency 4556.531 ms 2024-09-15T22:36:36.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:35 smithi196 ceph-mon[35109]: pgmap v764: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.5 MiB/s wr, 356 op/s 2024-09-15T22:36:36.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:35 smithi117 ceph-mon[35021]: pgmap v764: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.5 MiB/s wr, 356 op/s 2024-09-15T22:36:36.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:35 smithi077 ceph-mon[30980]: pgmap v764: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.5 MiB/s wr, 356 op/s 2024-09-15T22:36:36.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.60 MB/sec execute 517 sec latency 5556.632 ms 2024-09-15T22:36:37.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.58 MB/sec execute 518 sec latency 6556.689 ms 2024-09-15T22:36:38.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:37 smithi196 ceph-mon[35109]: pgmap v765: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-09-15T22:36:38.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:37 smithi117 ceph-mon[35021]: pgmap v765: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-09-15T22:36:38.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:37 smithi077 ceph-mon[30980]: pgmap v765: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-09-15T22:36:38.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.55 MB/sec execute 519 sec latency 7556.727 ms 2024-09-15T22:36:39.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.52 MB/sec execute 520 sec latency 8556.780 ms 2024-09-15T22:36:40.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:39 smithi077 ceph-mon[30980]: pgmap v766: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 264 op/s 2024-09-15T22:36:40.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:39 smithi196 ceph-mon[35109]: pgmap v766: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 264 op/s 2024-09-15T22:36:40.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:39 smithi117 ceph-mon[35021]: pgmap v766: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 264 op/s 2024-09-15T22:36:40.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.49 MB/sec execute 521 sec latency 9556.867 ms 2024-09-15T22:36:41.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.46 MB/sec execute 522 sec latency 10556.966 ms 2024-09-15T22:36:42.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:41 smithi117 ceph-mon[35021]: pgmap v767: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-15T22:36:42.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:41 smithi077 ceph-mon[30980]: pgmap v767: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-15T22:36:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:41 smithi196 ceph-mon[35109]: pgmap v767: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 247 op/s 2024-09-15T22:36:42.624 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.44 MB/sec execute 523 sec latency 11557.065 ms 2024-09-15T22:36:43.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.41 MB/sec execute 524 sec latency 12557.160 ms 2024-09-15T22:36:44.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:43 smithi117 ceph-mon[35021]: pgmap v768: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-15T22:36:44.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:43 smithi077 ceph-mon[30980]: pgmap v768: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-15T22:36:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:43 smithi196 ceph-mon[35109]: pgmap v768: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-15T22:36:44.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.38 MB/sec execute 525 sec latency 13557.267 ms 2024-09-15T22:36:45.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.35 MB/sec execute 526 sec latency 14557.397 ms 2024-09-15T22:36:46.134 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:45 smithi077 ceph-mon[30980]: pgmap v769: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 490 KiB/s wr, 59 op/s 2024-09-15T22:36:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:45 smithi196 ceph-mon[35109]: pgmap v769: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 490 KiB/s wr, 59 op/s 2024-09-15T22:36:46.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:45 smithi117 ceph-mon[35021]: pgmap v769: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 490 KiB/s wr, 59 op/s 2024-09-15T22:36:46.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.33 MB/sec execute 527 sec latency 15557.479 ms 2024-09-15T22:36:47.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.30 MB/sec execute 528 sec latency 16557.531 ms 2024-09-15T22:36:48.151 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:47 smithi077 ceph-mon[30980]: pgmap v770: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 66 op/s 2024-09-15T22:36:48.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:47 smithi196 ceph-mon[35109]: pgmap v770: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 66 op/s 2024-09-15T22:36:48.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:47 smithi117 ceph-mon[35021]: pgmap v770: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 66 op/s 2024-09-15T22:36:48.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.27 MB/sec execute 529 sec latency 17557.595 ms 2024-09-15T22:36:49.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.25 MB/sec execute 530 sec latency 18557.725 ms 2024-09-15T22:36:50.156 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:49 smithi196 ceph-mon[35109]: pgmap v771: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-15T22:36:50.164 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:49 smithi117 ceph-mon[35021]: pgmap v771: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-15T22:36:50.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:49 smithi077 ceph-mon[30980]: pgmap v771: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 75 KiB/s wr, 20 op/s 2024-09-15T22:36:50.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.22 MB/sec execute 531 sec latency 19557.828 ms 2024-09-15T22:36:51.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.19 MB/sec execute 532 sec latency 20557.932 ms 2024-09-15T22:36:52.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:51 smithi077 ceph-mon[30980]: pgmap v772: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-09-15T22:36:52.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:51 smithi196 ceph-mon[35109]: pgmap v772: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-09-15T22:36:52.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:51 smithi117 ceph-mon[35021]: pgmap v772: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 75 KiB/s wr, 22 op/s 2024-09-15T22:36:52.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:52.625 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.17 MB/sec execute 533 sec latency 21557.979 ms 2024-09-15T22:36:52.813 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:53.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- 172.21.15.77:0/1661097057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578112b90 msgr2=0x7f7578114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 --2- 172.21.15.77:0/1661097057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578112b90 0x7f7578114f80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7570009900 tx=0x7f757002a480 comp rx=0 tx=0).stop 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- 172.21.15.77:0/1661097057 shutdown_connections 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 --2- 172.21.15.77:0/1661097057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578112b90 0x7f7578114f80 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 --2- 172.21.15.77:0/1661097057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074eb0 0x7f7578112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 --2- 172.21.15.77:0/1661097057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578074590 0x7f7578074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- 172.21.15.77:0/1661097057 >> 172.21.15.77:0/1661097057 conn(0x7f75780fd8c0 msgr2=0x7f75780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:53.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- 172.21.15.77:0/1661097057 shutdown_connections 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- 172.21.15.77:0/1661097057 wait complete. 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 Processor -- start 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.337+0000 7f757f878640 1 -- start start 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074590 0x7f75781a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578112b90 0x7f757819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7578118550 con 0x7f7578074eb0 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75781183d0 con 0x7f7578112b90 2024-09-15T22:36:53.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757f878640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75781186d0 con 0x7f7578074590 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60656/0 (socket says 172.21.15.77:60656) 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 -- 172.21.15.77:0/897808250 learned_addr learned my addr 172.21.15.77:0/897808250 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757d614640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074590 0x7f75781a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757de15640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578112b90 0x7f757819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 -- 172.21.15.77:0/897808250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074590 msgr2=0x7f75781a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074590 0x7f75781a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 -- 172.21.15.77:0/897808250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578112b90 msgr2=0x7f757819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578112b90 0x7f757819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 -- 172.21.15.77:0/897808250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757819efb0 con 0x7f7578074eb0 2024-09-15T22:36:53.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f757ce13640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f756c00db80 tx=0x7f756c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c017070 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.338+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756c003e50 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.339+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757819f2a0 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.339+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0104c0 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.339+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781ab590 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.340+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578074970 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.341+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f756c003970 con 0x7f7578074eb0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.341+0000 7f75767fc640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.341+0000 7f757d614640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:53.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.341+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f756c0d0750 con 0x7f7578074eb0 2024-09-15T22:36:53.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.342+0000 7f757d614640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 0x7f75600a58f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f757819f9c0 tx=0x7f756800a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:53.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.344+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f756c09ce20 con 0x7f7578074eb0 2024-09-15T22:36:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.581+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f757819fdb0 con 0x7f7578074eb0 2024-09-15T22:36:53.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.582+0000 7f75767fc640 1 -- 172.21.15.77:0/897808250 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f756c09c720 con 0x7f7578074eb0 2024-09-15T22:36:53.584 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:36:53.584 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:36:53.584 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 msgr2=0x7f75600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 0x7f75600a58f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f757819f9c0 tx=0x7f756800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 msgr2=0x7f75781a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f756c00db80 tx=0x7f756c00df80 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 shutdown_connections 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7578112b90 0x7f757819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7578074eb0 0x7f75781a4c00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 --2- 172.21.15.77:0/897808250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7578074590 0x7f75781a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 >> 172.21.15.77:0/897808250 conn(0x7f75780fd8c0 msgr2=0x7f75780fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:53.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 shutdown_connections 2024-09-15T22:36:53.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:53.584+0000 7f757f878640 1 -- 172.21.15.77:0/897808250 wait complete. 2024-09-15T22:36:53.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.14 MB/sec execute 534 sec latency 22558.045 ms 2024-09-15T22:36:54.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:53 smithi077 ceph-mon[30980]: pgmap v773: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-15T22:36:54.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:53 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/897808250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:53 smithi196 ceph-mon[35109]: pgmap v773: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-15T22:36:54.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:53 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/897808250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:53 smithi117 ceph-mon[35021]: pgmap v773: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 21 op/s 2024-09-15T22:36:54.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:53 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/897808250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:54.426 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id e47c9c22 --release --await' 2024-09-15T22:36:54.427 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id e47c9c22 --release --await 2024-09-15T22:36:54.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183770 14.11 MB/sec execute 535 sec latency 23558.116 ms 2024-09-15T22:36:54.673 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 -- 172.21.15.77:0/1487900061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a4103660 msgr2=0x7f40a410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 --2- 172.21.15.77:0/1487900061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a4103660 0x7f40a410bb50 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c02a480 comp rx=0 tx=0).stop 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 -- 172.21.15.77:0/1487900061 shutdown_connections 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 --2- 172.21.15.77:0/1487900061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a410c170 0x7f40a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 --2- 172.21.15.77:0/1487900061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a4103660 0x7f40a410bb50 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 --2- 172.21.15.77:0/1487900061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4102d40 0x7f40a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 -- 172.21.15.77:0/1487900061 >> 172.21.15.77:0/1487900061 conn(0x7f40a40fc7a0 msgr2=0x7f40a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 -- 172.21.15.77:0/1487900061 shutdown_connections 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.384+0000 7f40a9385640 1 -- 172.21.15.77:0/1487900061 wait complete. 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 Processor -- start 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 -- start start 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 0x7f40a41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 0x7f40a4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4111f70 con 0x7f40a410c170 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4111df0 con 0x7f40a4102d40 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a9385640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41120f0 con 0x7f40a4103660 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.385+0000 7f40a37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60680/0 (socket says 172.21.15.77:60680) 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 -- 172.21.15.77:0/3022454562 learned_addr learned my addr 172.21.15.77:0/3022454562 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f409bfff640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 0x7f40a4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 0x7f40a41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 -- 172.21.15.77:0/3022454562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 msgr2=0x7f40a4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 0x7f40a4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 -- 172.21.15.77:0/3022454562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 msgr2=0x7f40a41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 0x7f40a41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 -- 172.21.15.77:0/3022454562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a419d4a0 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f409bfff640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 0x7f40a4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 0x7f40a41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a37fe640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f409c00cad0 tx=0x7f409c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a0ff9640 1 -- 172.21.15.77:0/3022454562 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00ae20 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a0ff9640 1 -- 172.21.15.77:0/3022454562 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c005020 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a0ff9640 1 -- 172.21.15.77:0/3022454562 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00f6f0 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a419d790 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.386+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a5070 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.389+0000 7f40ab75e740 1 Processor -- start 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.389+0000 7f40a0ff9640 1 -- 172.21.15.77:0/3022454562 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f409c0051d0 con 0x7f40a410c170 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.389+0000 7f40ab75e740 1 -- start start 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181aff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e181a57620 con 0x55e181af2980 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e1819fee50 con 0x55e181afd240 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40ab75e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e181a535e0 con 0x55e181af2060 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f409bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60686/0 (socket says 172.21.15.77:60686) 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 -- 172.21.15.77:0/4146878456 learned_addr learned my addr 172.21.15.77:0/4146878456 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f409bfff640 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181aff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 -- 172.21.15.77:0/4146878456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 msgr2=0x55e181af2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 -- 172.21.15.77:0/4146878456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 msgr2=0x55e181aff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181aff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 -- 172.21.15.77:0/4146878456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1819efd40 con 0x55e181af2980 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f409400bd40 tx=0x7f409400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a0720424bc6dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.390+0000 7f40a0ff9640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 0x7f40900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f409bfff640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 0x7f40900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40a0ff9640 1 -- 172.21.15.77:0/3022454562 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f409c0d0140 con 0x7f40a410c170 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40997fa640 1 -- 172.21.15.77:0/4146878456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094015070 con 0x55e181af2980 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40997fa640 1 -- 172.21.15.77:0/4146878456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409400be30 con 0x55e181af2980 2024-09-15T22:36:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40997fa640 1 -- 172.21.15.77:0/4146878456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094011a70 con 0x55e181af2980 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40ab75e740 1 -- 172.21.15.77:0/4146878456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 msgr2=0x55e181afcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f40ab75e740 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f409400bd40 tx=0x7f409400f730 comp rx=0 tx=0).stop 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.391+0000 7f409bfff640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 0x7f40900a58f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f40a410d640 tx=0x7f408c009260 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- 172.21.15.77:0/4146878456 shutdown_connections 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181aff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afcd00 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- 172.21.15.77:0/4146878456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- 172.21.15.77:0/4146878456 >> 172.21.15.77:0/4146878456 conn(0x55e181aee5f0 msgr2=0x55e181aee9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- 172.21.15.77:0/4146878456 shutdown_connections 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- 172.21.15.77:0/4146878456 wait complete. 2024-09-15T22:36:55.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- start start 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 Processor -- start 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181af2060 0x55e181afa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 0x55e181afabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e181affe10 con 0x55e181af2980 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e1819dff10 con 0x55e181af2060 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e181afff90 con 0x55e181afd240 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f40ab75e740 10 client.0 _ll_register_callbacks cb 0x55e181a03db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.392+0000 7f409bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60688/0 (socket says 172.21.15.77:60688) 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 -- 172.21.15.77:0/1203202157 learned_addr learned my addr 172.21.15.77:0/1203202157 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f40a37fe640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 0x55e181afabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 -- 172.21.15.77:0/1203202157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 msgr2=0x55e181afabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 0x55e181afabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 -- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181af2060 msgr2=0x55e181afa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181af2060 0x55e181afa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 -- 172.21.15.77:0/1203202157 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e181766580 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f40a37fe640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 0x55e181afabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f409bfff640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f408c033000 tx=0x7f408c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade563ffa67d65eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f408a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c04e030 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f408c003b70 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409400be30 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f408c05b050 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f40ab75e740 10 client.20940 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e181830960 con 0x55e181af2980 2024-09-15T22:36:55.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.393+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409400be30 con 0x55e181af2980 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f408c03a040 con 0x55e181af2980 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 10 client.20940 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 10 client.20940 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 20 client.20940 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b0cdc0 0x55e181b0f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 4 client.20940 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e181a64b90 con 0x55e181b0cdc0 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b0cdc0 0x55e181b0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b0cdc0 0x55e181b0f260 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13a06590c08b76b server_cookie=fd8feb06f0e25b40 in_seq=0 out_seq=0 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.394+0000 7f408a7fc640 10 client.20940 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.395+0000 7f408a7fc640 1 -- 172.21.15.77:0/1203202157 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e181a64b90 con 0x55e181b0cdc0 2024-09-15T22:36:55.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.395+0000 7f408a7fc640 10 client.20940 handle_command_reply: tid=0 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.395+0000 7f40ab75e740 1 client.20940 shutdown 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 msgr2=0x55e181afa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f408c033000 tx=0x7f408c0056d0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 shutdown_connections 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b0cdc0 0x55e181b0f260 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181afd240 0x55e181afabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181afa670 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1203202157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181af2060 0x55e181afa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f408a7fc640 0 client.20940 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 >> 172.21.15.77:0/1203202157 conn(0x55e181aee5f0 msgr2=0x55e181af42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 shutdown_connections 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 1 -- 172.21.15.77:0/1203202157 wait complete. 2024-09-15T22:36:55.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.396+0000 7f40ab75e740 20 client.20940 trim_cache size 0 max 16384 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 Processor -- start 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 -- start start 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b14b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41af070 con 0x55e181af2980 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b0bb0 con 0x55e181afd240 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40ab75e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41af740 con 0x55e181af2060 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60694/0 (socket says 172.21.15.77:60694) 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 -- 172.21.15.77:0/2507656671 learned_addr learned my addr 172.21.15.77:0/2507656671 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f40a37fe640 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b14b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 -- 172.21.15.77:0/2507656671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 msgr2=0x55e181af4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 -- 172.21.15.77:0/2507656671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 msgr2=0x55e181b14b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b14b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.413+0000 7f409bfff640 1 -- 172.21.15.77:0/2507656671 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e181af5000 con 0x55e181af2980 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f409bfff640 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344c24231481140f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f4098ff9640 1 -- 172.21.15.77:0/2507656671 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c04e040 con 0x55e181af2980 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f4098ff9640 1 -- 172.21.15.77:0/2507656671 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c045070 con 0x55e181af2980 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f4098ff9640 1 -- 172.21.15.77:0/2507656671 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c035040 con 0x55e181af2980 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f40ab75e740 1 -- 172.21.15.77:0/2507656671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 msgr2=0x55e181aed640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.414+0000 7f40ab75e740 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c02a480 comp rx=0 tx=0).stop 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- 172.21.15.77:0/2507656671 shutdown_connections 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b14b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2980 0x55e181aed640 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- 172.21.15.77:0/2507656671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2060 0x55e181af4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- 172.21.15.77:0/2507656671 >> 172.21.15.77:0/2507656671 conn(0x55e181aee5f0 msgr2=0x55e181aecf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- 172.21.15.77:0/2507656671 shutdown_connections 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- 172.21.15.77:0/2507656671 wait complete. 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- start start 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 Processor -- start 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 0x55e181b05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b04e0 con 0x55e181af2060 2024-09-15T22:36:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41afe10 con 0x55e181afd240 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41ad070 con 0x55e181af2980 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40ab75e740 10 client.0 _ll_register_callbacks cb 0x7f408c032e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40a2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40a2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60700/0 (socket says 172.21.15.77:60700) 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40a2ffd640 1 -- 172.21.15.77:0/1415609518 learned_addr learned my addr 172.21.15.77:0/1415609518 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.415+0000 7f40a37fe640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f409bfff640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 0x55e181b05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 -- 172.21.15.77:0/1415609518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 msgr2=0x55e181b05720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 0x55e181b05720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 -- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 msgr2=0x55e181b021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 -- 172.21.15.77:0/1415609518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e181b026f0 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f409bfff640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 0x55e181b05720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40a2ffd640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4094010010 tx=0x7f409400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7440fca4dc4ffce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f408a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094015040 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f4094005240 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c045070 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4094035050 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40ab75e740 10 client.20946 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e181b78220 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.416+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c045070 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4094004680 con 0x55e181af2060 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 10 client.20946 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 10 client.20946 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 20 client.20946 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b786c0 0x55e181b7aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 4 client.20946 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e47c9c22", "release": true, "await": true}] 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e47c9c22", "release": true, "await": true}) -- 0x55e181ae9950 con 0x55e181b786c0 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f409bfff640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b786c0 0x55e181b7aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f408a7fc640 10 client.20946 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.417+0000 7f409bfff640 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b786c0 0x55e181b7aab0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b744d66933a8300 server_cookie=7a96bd529275b756 in_seq=0 out_seq=0 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f408a7fc640 1 -- 172.21.15.77:0/1415609518 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e181ae9950 con 0x55e181b786c0 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f408a7fc640 10 client.20946 handle_command_reply: tid=0 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 client.20946 shutdown 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 msgr2=0x55e181b051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4094010010 tx=0x7f409400b480 comp rx=0 tx=0).stop 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 shutdown_connections 2024-09-15T22:36:55.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e181b786c0 0x55e181b7aab0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e181afd240 0x55e181b021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e181af2980 0x55e181b05720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 --2- 172.21.15.77:0/1415609518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e181af2060 0x55e181b051e0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 >> 172.21.15.77:0/1415609518 conn(0x55e181aee5f0 msgr2=0x55e181aed750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f408a7fc640 0 client.20946 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 shutdown_connections 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 1 -- 172.21.15.77:0/1415609518 wait complete. 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.476+0000 7f40ab75e740 20 client.20946 trim_cache size 0 max 16384 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:36:55.479 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 84, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "e47c9c22": { 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 84, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:36:55.480 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 147.2, 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:55.481 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:36:55.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 msgr2=0x7f40900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 0x7f40900a58f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f40a410d640 tx=0x7f408c009260 comp rx=0 tx=0).stop 2024-09-15T22:36:55.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 msgr2=0x7f40a419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:55.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f409c00cad0 tx=0x7f409c00cf90 comp rx=0 tx=0).stop 2024-09-15T22:36:55.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 shutdown_connections 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f40900a3430 0x7f40900a58f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40a410c170 0x7f40a419cdc0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40a4103660 0x7f40a4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 --2- 172.21.15.77:0/3022454562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a4102d40 0x7f40a41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 >> 172.21.15.77:0/3022454562 conn(0x7f40a40fc7a0 msgr2=0x7f40a40fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 shutdown_connections 2024-09-15T22:36:55.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:55.477+0000 7f40a9385640 1 -- 172.21.15.77:0/3022454562 wait complete. 2024-09-15T22:36:55.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 183816 14.09 MB/sec execute 536 sec latency 24408.223 ms 2024-09-15T22:36:56.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:55 smithi077 ceph-mon[30980]: pgmap v774: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-15T22:36:56.295 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e47c9c22', total seconds elapsed: 30.9 2024-09-15T22:36:56.295 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-15T22:36:56.295 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:56.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:55 smithi196 ceph-mon[35109]: pgmap v774: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-15T22:36:56.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:55 smithi117 ceph-mon[35021]: pgmap v774: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-15T22:36:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:56.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 184197 14.10 MB/sec execute 537 sec latency 149.177 ms 2024-09-15T22:36:57.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 -- 172.21.15.77:0/2504131140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1082a0 msgr2=0x7f4b0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 --2- 172.21.15.77:0/2504131140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c10ec40 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009c10 tx=0x7f4b0402a6f0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 -- 172.21.15.77:0/2504131140 shutdown_connections 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 --2- 172.21.15.77:0/2504131140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c10ec40 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 --2- 172.21.15.77:0/2504131140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 --2- 172.21.15.77:0/2504131140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 0x7f4b0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.123+0000 7f4b1437a640 1 -- 172.21.15.77:0/2504131140 >> 172.21.15.77:0/2504131140 conn(0x7f4b0c0fc7d0 msgr2=0x7f4b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.124+0000 7f4b1437a640 1 -- 172.21.15.77:0/2504131140 shutdown_connections 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.124+0000 7f4b1437a640 1 -- 172.21.15.77:0/2504131140 wait complete. 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.124+0000 7f4b1437a640 1 Processor -- start 2024-09-15T22:36:57.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.124+0000 7f4b1437a640 1 -- start start 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 0x7f4b0c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c1120a0 con 0x7f4b0c1036e0 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c111f20 con 0x7f4b0c1082a0 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b1437a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c112220 con 0x7f4b0c102d30 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60730/0 (socket says 172.21.15.77:60730) 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 -- 172.21.15.77:0/649104076 learned_addr learned my addr 172.21.15.77:0/649104076 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b12116640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 0x7f4b0c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b12917640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 -- 172.21.15.77:0/649104076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 msgr2=0x7f4b0c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 0x7f4b0c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 -- 172.21.15.77:0/649104076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1082a0 msgr2=0x7f4b0c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 -- 172.21.15.77:0/649104076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c19d3f0 con 0x7f4b0c1036e0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.125+0000 7f4b11915640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009e20 tx=0x7f4afc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.126+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc020070 con 0x7f4b0c1036e0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.126+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c19d6e0 con 0x7f4b0c1036e0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.126+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c1008c0 con 0x7f4b0c1036e0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.126+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4afc00ccc0 con 0x7f4b0c1036e0 2024-09-15T22:36:57.129 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.126+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc01ba60 con 0x7f4b0c1036e0 2024-09-15T22:36:57.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.127+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0005290 con 0x7f4b0c1036e0 2024-09-15T22:36:57.130 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.129+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4afc023070 con 0x7f4b0c1036e0 2024-09-15T22:36:57.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.129+0000 7f4b037fe640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 0x7f4af80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.130+0000 7f4b12116640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 0x7f4af80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.130+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f4afc0d4140 con 0x7f4b0c1036e0 2024-09-15T22:36:57.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.130+0000 7f4b12116640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 0x7f4af80a58f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009bd0 tx=0x7f4b08005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:57.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.135+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4afc0a0250 con 0x7f4b0c1036e0 2024-09-15T22:36:57.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.436+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4af0005580 con 0x7f4b0c1036e0 2024-09-15T22:36:57.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.437+0000 7f4b037fe640 1 -- 172.21.15.77:0/649104076 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f4afc0a4550 con 0x7f4b0c1036e0 2024-09-15T22:36:57.439 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:36:57.439 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:36:57.439 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:36:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 msgr2=0x7f4af80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 0x7f4af80a58f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009bd0 tx=0x7f4b08005f90 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 msgr2=0x7f4b0c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009e20 tx=0x7f4afc00ed90 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 shutdown_connections 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4af80a3430 0x7f4af80a58f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b0c1082a0 0x7f4b0c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b0c1036e0 0x7f4b0c198980 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 --2- 172.21.15.77:0/649104076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c102d30 0x7f4b0c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 >> 172.21.15.77:0/649104076 conn(0x7f4b0c0fc7d0 msgr2=0x7f4b0c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 shutdown_connections 2024-09-15T22:36:57.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:57.440+0000 7f4b1437a640 1 -- 172.21.15.77:0/649104076 wait complete. 2024-09-15T22:36:57.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 184523 14.10 MB/sec execute 538 sec latency 196.234 ms 2024-09-15T22:36:58.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:57 smithi077 ceph-mon[30980]: pgmap v775: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 59 op/s 2024-09-15T22:36:58.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:57 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/649104076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:57 smithi196 ceph-mon[35109]: pgmap v775: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 59 op/s 2024-09-15T22:36:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:57 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/649104076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:57 smithi117 ceph-mon[35021]: pgmap v775: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 59 op/s 2024-09-15T22:36:58.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:57 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/649104076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.429 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-15T22:36:58.430 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-15T22:36:58.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 184921 14.11 MB/sec execute 539 sec latency 153.674 ms 2024-09-15T22:36:58.750 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 -- 172.21.15.77:0/2733075301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8100100 msgr2=0x7fdce8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2733075301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8100100 0x7fdce8100560 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002a410 comp rx=0 tx=0).stop 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 -- 172.21.15.77:0/2733075301 shutdown_connections 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2733075301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100aa0 0x7fdce810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2733075301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8100100 0x7fdce8100560 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2733075301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8106b80 0x7fdce8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 -- 172.21.15.77:0/2733075301 >> 172.21.15.77:0/2733075301 conn(0x7fdce8077230 msgr2=0x7fdce80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 -- 172.21.15.77:0/2733075301 shutdown_connections 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.330+0000 7fdced8d8640 1 -- 172.21.15.77:0/2733075301 wait complete. 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 Processor -- start 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 -- start start 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100100 0x7fdce81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 0x7fdce8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdce8112050 con 0x7fdce8106b80 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdce8111ed0 con 0x7fdce8100100 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdced8d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81121d0 con 0x7fdce8100aa0 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100100 0x7fdce81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47548/0 (socket says 172.21.15.77:47548) 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.331+0000 7fdcdffff640 1 -- 172.21.15.77:0/2803985002 learned_addr learned my addr 172.21.15.77:0/2803985002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdce77fe640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 0x7fdce8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 -- 172.21.15.77:0/2803985002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100100 msgr2=0x7fdce81012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100100 0x7fdce81012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 -- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 msgr2=0x7fdce8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 0x7fdce8101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 -- 172.21.15.77:0/2803985002 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce81a69d0 con 0x7fdce8100aa0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdce77fe640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 0x7fdce8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcec8d6640 1 -- 172.21.15.77:0/2803985002 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd003f030 con 0x7fdce8100aa0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce81a6c60 con 0x7fdce8100aa0 2024-09-15T22:36:59.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcec8d6640 1 -- 172.21.15.77:0/2803985002 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd0004440 con 0x7fdce8100aa0 2024-09-15T22:36:59.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.332+0000 7fdcec8d6640 1 -- 172.21.15.77:0/2803985002 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0035070 con 0x7fdce8100aa0 2024-09-15T22:36:59.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.333+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce81a7140 con 0x7fdce8100aa0 2024-09-15T22:36:59.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.335+0000 7fdcec8d6640 1 -- 172.21.15.77:0/2803985002 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdcd0042030 con 0x7fdce8100aa0 2024-09-15T22:36:59.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 Processor -- start 2024-09-15T22:36:59.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 -- start start 2024-09-15T22:36:59.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6015400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b6015940 0x5592b6017d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592b5f702a0 con 0x5592b600a760 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592b5f17b10 con 0x5592b600b080 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592b5f6c260 con 0x5592b6015940 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35588/0 (socket says 172.21.15.77:35588) 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdce77fe640 1 -- 172.21.15.77:0/464848772 learned_addr learned my addr 172.21.15.77:0/464848772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcec8d6640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 0x7fdcd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6015400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.336+0000 7fdcdffff640 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b6015940 0x5592b6017d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdcec8d6640 1 -- 172.21.15.77:0/2803985002 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fdcd00f05c0 con 0x7fdce8100aa0 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 -- 172.21.15.77:0/464848772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b6015940 msgr2=0x5592b6017d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b6015940 0x5592b6017d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 -- 172.21.15.77:0/464848772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 msgr2=0x5592b6015400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6015400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 -- 172.21.15.77:0/464848772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b5ce1b90 con 0x5592b600a760 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 0x7fdcd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdce77fe640 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011480 tx=0x7fdce0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90643d5f8863fbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdcddffb640 1 -- 172.21.15.77:0/464848772 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce001c400 con 0x5592b600a760 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdcddffb640 1 -- 172.21.15.77:0/464848772 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce001ca80 con 0x5592b600a760 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.337+0000 7fdcddffb640 1 -- 172.21.15.77:0/464848772 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce001b8a0 con 0x5592b600a760 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 -- 172.21.15.77:0/464848772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 msgr2=0x5592b600ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdce77fe640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 0x7fdcd40a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdce0020000 tx=0x7fdce001d000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fdce0011480 tx=0x7fdce0011940 comp rx=0 tx=0).stop 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 -- 172.21.15.77:0/464848772 shutdown_connections 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b6015940 0x5592b6017d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6015400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/464848772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600a760 0x5592b600ab40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 -- 172.21.15.77:0/464848772 >> 172.21.15.77:0/464848772 conn(0x5592b6006cf0 msgr2=0x5592b60070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 -- 172.21.15.77:0/464848772 shutdown_connections 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.338+0000 7fdcefcb1740 1 -- 172.21.15.77:0/464848772 wait complete. 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 -- start start 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 Processor -- start 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b60127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6012d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592b6018510 con 0x5592b6015940 2024-09-15T22:36:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592b5ef86e0 con 0x5592b600b080 2024-09-15T22:36:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5592b6018690 con 0x5592b600a760 2024-09-15T22:36:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b60127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35602/0 (socket says 172.21.15.77:35602) 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdce77fe640 1 -- 172.21.15.77:0/4029271815 learned_addr learned my addr 172.21.15.77:0/4029271815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.339+0000 7fdcdffff640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6012d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 -- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 msgr2=0x5592b6012d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6012d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 -- 172.21.15.77:0/4029271815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 msgr2=0x5592b6013240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 -- 172.21.15.77:0/4029271815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b60219b0 con 0x5592b600a760 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce77fe640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcefcb1740 10 client.0 _ll_register_callbacks cb 0x5592b5f1ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce001ca80 con 0x5592b600a760 2024-09-15T22:36:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b60127c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8012700 tx=0x7fdcd8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1502b368f496ee8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcc6ffd640 10 client.29536 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/4029271815 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8004050 con 0x5592b600a760 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/4029271815 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fdcd80041f0 con 0x5592b600a760 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/4029271815 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcd80043f0 con 0x5592b600a760 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcefcb1740 10 client.29536 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.340+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5592b6024160 con 0x5592b600a760 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/4029271815 <== mon.1 v2:172.21.15.196:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdcd800f920 con 0x5592b600a760 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 10 client.29536 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 10 client.29536 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 20 client.29536 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6025560 0x5592b60279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 4 client.29536 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b5f7d9d0 con 0x5592b6025560 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcdffff640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6025560 0x5592b60279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcdffff640 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6025560 0x5592b60279f0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417cbdc14329e68f server_cookie=14789a33a1c54409 in_seq=0 out_seq=0 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.341+0000 7fdcc6ffd640 10 client.29536 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.342+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/4029271815 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592b5f7d9d0 con 0x5592b6025560 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.342+0000 7fdcc6ffd640 10 client.29536 handle_command_reply: tid=0 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 client.29536 shutdown 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 msgr2=0x5592b60127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b60127c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8012700 tx=0x7fdcd8012bc0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 shutdown_connections 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6025560 0x5592b60279f0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b6013240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600b080 0x5592b6012d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/4029271815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b60127c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 >> 172.21.15.77:0/4029271815 conn(0x5592b6006cf0 msgr2=0x5592b6005e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcc6ffd640 0 client.29536 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 shutdown_connections 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 1 -- 172.21.15.77:0/4029271815 wait complete. 2024-09-15T22:36:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.343+0000 7fdcefcb1740 20 client.29536 trim_cache size 0 max 16384 2024-09-15T22:36:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 Processor -- start 2024-09-15T22:36:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 -- start start 2024-09-15T22:36:59.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b6005e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 0x5592b6013250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81b1150 con 0x5592b600b080 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81b2c90 con 0x5592b6015940 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81b1820 con 0x5592b600a760 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35604/0 (socket says 172.21.15.77:35604) 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdcdffff640 1 -- 172.21.15.77:0/573507267 learned_addr learned my addr 172.21.15.77:0/573507267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b6005e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdce77fe640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 0x5592b6013250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.363+0000 7fdce6ffd640 1 -- 172.21.15.77:0/573507267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 msgr2=0x5592b6013250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 0x5592b6013250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce6ffd640 1 -- 172.21.15.77:0/573507267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 msgr2=0x5592b6012d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce6ffd640 1 -- 172.21.15.77:0/573507267 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b6006790 con 0x5592b600a760 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdcdffff640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce77fe640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 0x5592b6013250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b6005e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009fb0 tx=0x7fdcd8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a570bb14825a2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdcdd7fa640 1 -- 172.21.15.77:0/573507267 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd800f920 con 0x5592b600a760 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdcdd7fa640 1 -- 172.21.15.77:0/573507267 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce001ca80 con 0x5592b600a760 2024-09-15T22:36:59.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.364+0000 7fdcdd7fa640 1 -- 172.21.15.77:0/573507267 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd801c420 con 0x5592b600a760 2024-09-15T22:36:59.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- 172.21.15.77:0/573507267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 msgr2=0x5592b6005e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b6005e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009fb0 tx=0x7fdcd8013000 comp rx=0 tx=0).stop 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- 172.21.15.77:0/573507267 shutdown_connections 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b6015940 0x5592b6013250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b600b080 0x5592b6012d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/573507267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600a760 0x5592b6005e70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- 172.21.15.77:0/573507267 >> 172.21.15.77:0/573507267 conn(0x5592b6006cf0 msgr2=0x5592b60148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- 172.21.15.77:0/573507267 shutdown_connections 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- 172.21.15.77:0/573507267 wait complete. 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.365+0000 7fdcefcb1740 1 -- start start 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 Processor -- start 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 0x5592b601def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b600f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81b25c0 con 0x5592b6015940 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81b1ef0 con 0x5592b600a760 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdce81af150 con 0x5592b600b080 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcefcb1740 10 client.0 _ll_register_callbacks cb 0x7fdcd8009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 0x5592b601def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdce77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b600f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47574/0 (socket says 172.21.15.77:47574) 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 0x5592b601def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41156/0 (socket says 172.21.15.77:41156) 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.366+0000 7fdcdffff640 1 -- 172.21.15.77:0/2316683678 learned_addr learned my addr 172.21.15.77:0/2316683678 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 -- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 msgr2=0x5592b601def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 0x5592b601def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 -- 172.21.15.77:0/2316683678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 msgr2=0x5592b600f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b600f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 -- 172.21.15.77:0/2316683678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b6010080 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdce77fe640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b600f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcdffff640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005ab0 tx=0x7fdcd003b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1911b185ccf53e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0049040 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fdcd00b8b20 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce001ca80 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdcd00c0860 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.367+0000 7fdcefcb1740 10 client.29542 fetch_fsmap learned FSMap version 39 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592b6090250 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce001ca80 con 0x5592b600b080 2024-09-15T22:36:59.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdcd00bc7a0 con 0x5592b600b080 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 10 client.29542 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 10 client.29542 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 20 client.29542 populate_metadata read hostname 'smithi077' 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6090a40 0x5592b6092e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 4 client.29542 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x5592b5f77d20 con 0x5592b6090a40 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.368+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6090a40 0x5592b6092e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.369+0000 7fdcc6ffd640 10 client.29542 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:59.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.369+0000 7fdce6ffd640 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6090a40 0x5592b6092e60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b68060e672cf285 server_cookie=2c7979cb0470c699 in_seq=0 out_seq=0 2024-09-15T22:36:59.626 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185229 14.10 MB/sec execute 540 sec latency 231.346 ms 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcc6ffd640 1 -- 172.21.15.77:0/2316683678 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5592b5f77d20 con 0x5592b6090a40 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcc6ffd640 10 client.29542 handle_command_reply: tid=0 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 client.29542 shutdown 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 msgr2=0x5592b600f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005ab0 tx=0x7fdcd003b4f0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 shutdown_connections 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5592b6090a40 0x5592b6092e60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592b6015940 0x5592b600f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5592b600b080 0x5592b600f430 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcc6ffd640 0 client.29542 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 --2- 172.21.15.77:0/2316683678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592b600a760 0x5592b601def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 >> 172.21.15.77:0/2316683678 conn(0x5592b6006cf0 msgr2=0x5592b6005e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 shutdown_connections 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.940+0000 7fdcefcb1740 1 -- 172.21.15.77:0/2316683678 wait complete. 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdcefcb1740 20 client.29542 trim_cache size 0 max 16384 2024-09-15T22:36:59.943 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 86, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "11c120d8": { 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 86, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:36:59.944 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 147.2, 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:36:59.945 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 msgr2=0x7fdcd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 0x7fdcd40a5870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdce0020000 tx=0x7fdce001d000 comp rx=0 tx=0).stop 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 msgr2=0x7fdce8101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd0004040 comp rx=0 tx=0).stop 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 shutdown_connections 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdcd40a33b0 0x7fdcd40a5870 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdce8106b80 0x7fdce8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdce8100aa0 0x7fdce8101820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 --2- 172.21.15.77:0/2803985002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce8100100 0x7fdce81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 >> 172.21.15.77:0/2803985002 conn(0x7fdce8077230 msgr2=0x7fdce8104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 shutdown_connections 2024-09-15T22:36:59.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:36:59.941+0000 7fdced8d8640 1 -- 172.21.15.77:0/2803985002 wait complete. 2024-09-15T22:37:00.283 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:36:59 smithi077 ceph-mon[30980]: pgmap v776: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-15T22:37:00.344 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:37:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:36:59 smithi196 ceph-mon[35109]: pgmap v776: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-15T22:37:00.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:36:59 smithi117 ceph-mon[35021]: pgmap v776: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-15T22:37:00.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 14.08 MB/sec execute 541 sec latency 1138.308 ms 2024-09-15T22:37:01.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 14.05 MB/sec execute 542 sec latency 2138.350 ms 2024-09-15T22:37:02.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:01 smithi077 ceph-mon[30980]: pgmap v777: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 107 op/s 2024-09-15T22:37:02.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:01 smithi196 ceph-mon[35109]: pgmap v777: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 107 op/s 2024-09-15T22:37:02.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:01 smithi117 ceph-mon[35021]: pgmap v777: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 107 op/s 2024-09-15T22:37:02.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 14.02 MB/sec execute 543 sec latency 3138.432 ms 2024-09-15T22:37:02.664 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '11c120d8', quiesce took 6.4 seconds. Will release after: 20.799999999999997 2024-09-15T22:37:02.697 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:03.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- 172.21.15.77:0/1599906464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78102790 msgr2=0x7fed781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/1599906464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78102790 0x7fed781067e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fed64009900 tx=0x7fed6402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- 172.21.15.77:0/1599906464 shutdown_connections 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/1599906464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 0x7fed7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/1599906464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78102790 0x7fed781067e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/1599906464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed78101de0 0x7fed781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- 172.21.15.77:0/1599906464 >> 172.21.15.77:0/1599906464 conn(0x7fed780fd8c0 msgr2=0x7fed780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- 172.21.15.77:0/1599906464 shutdown_connections 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- 172.21.15.77:0/1599906464 wait complete. 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 Processor -- start 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.187+0000 7fed7ebb0640 1 -- start start 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed78102790 0x7fed78198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 0x7fed7819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed78112010 con 0x7fed78101de0 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed78111e90 con 0x7fed78106f10 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed78112190 con 0x7fed78102790 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35616/0 (socket says 172.21.15.77:35616) 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 -- 172.21.15.77:0/3557485143 learned_addr learned my addr 172.21.15.77:0/3557485143 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 0x7fed7819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 -- 172.21.15.77:0/3557485143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed78102790 msgr2=0x7fed78198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed78102790 0x7fed78198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 -- 172.21.15.77:0/3557485143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 msgr2=0x7fed7819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.219 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 0x7fed7819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 -- 172.21.15.77:0/3557485143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7819d560 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b4a0 tx=0x7fed6c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed75ffb640 1 -- 172.21.15.77:0/3557485143 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c014070 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7819d850 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed75ffb640 1 -- 172.21.15.77:0/3557485143 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6c010af0 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.188+0000 7fed75ffb640 1 -- 172.21.15.77:0/3557485143 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c017d10 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.189+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781a5130 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.191+0000 7fed75ffb640 1 -- 172.21.15.77:0/3557485143 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fed6c017070 con 0x7fed78101de0 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.191+0000 7fed80f89740 1 Processor -- start 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 -- start start 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 0x55d529190550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.220 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d529192e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5290e72e0 con 0x55d529181380 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d52908ece0 con 0x55d529181ca0 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed80f89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5290e32a0 con 0x55d529190a90 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35626/0 (socket says 172.21.15.77:35626) 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 -- 172.21.15.77:0/3538145752 learned_addr learned my addr 172.21.15.77:0/3538145752 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed77fff640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d529192e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 0x55d529190550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 -- 172.21.15.77:0/3538145752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 msgr2=0x55d529192e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d529192e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 -- 172.21.15.77:0/3538145752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 msgr2=0x55d529190550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.221 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 0x55d529190550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 -- 172.21.15.77:0/3538145752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d529073440 con 0x55d529181380 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed75ffb640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 0x55d529190550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fed70009f20 tx=0x7fed7000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a3000dfe27f79a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.192+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed75ffb640 1 -- 172.21.15.77:0/3557485143 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fed6c0cf420 con 0x7fed78101de0 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed62ffd640 1 -- 172.21.15.77:0/3538145752 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7001a070 con 0x55d529181380 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed62ffd640 1 -- 172.21.15.77:0/3538145752 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed700158f0 con 0x55d529181380 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed62ffd640 1 -- 172.21.15.77:0/3538145752 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000a960 con 0x55d529181380 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 -- 172.21.15.77:0/3538145752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 msgr2=0x55d529181760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed7c94c640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 0x7fed680a5870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0cc9a0 tx=0x7fed6c0046c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fed70009f20 tx=0x7fed7000c8b0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.222 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 -- 172.21.15.77:0/3538145752 shutdown_connections 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d529192e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181ca0 0x55d529190550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 --2- 172.21.15.77:0/3538145752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529181760 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 -- 172.21.15.77:0/3538145752 >> 172.21.15.77:0/3538145752 conn(0x55d52917da30 msgr2=0x55d52917de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 -- 172.21.15.77:0/3538145752 shutdown_connections 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.193+0000 7fed80f89740 1 -- 172.21.15.77:0/3538145752 wait complete. 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 -- start start 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 Processor -- start 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 0x55d52918d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d52918dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d529193660 con 0x55d529181380 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5290a2050 con 0x55d529190a90 2024-09-15T22:37:03.223 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d5291937e0 con 0x55d529181ca0 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed80f89740 10 client.0 _ll_register_callbacks cb 0x55d5290e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed7d14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed7d14d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35642/0 (socket says 172.21.15.77:35642) 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.194+0000 7fed7d14d640 1 -- 172.21.15.77:0/2962058979 learned_addr learned my addr 172.21.15.77:0/2962058979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed77fff640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 0x55d52918d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d52918dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 -- 172.21.15.77:0/2962058979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 msgr2=0x55d52918d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 0x55d52918d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 -- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 msgr2=0x55d52918dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d52918dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 -- 172.21.15.77:0/2962058979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d528e583c0 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed77fff640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 0x55d52918d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed7d14d640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fed7000a010 tx=0x7fed700049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e608df9188fbd2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70027060 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed700158f0 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fed7000a9b0 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed7002e050 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed80f89740 10 client.20979 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d52919f030 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed700158f0 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.195+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fed70022c20 con 0x55d529181380 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 10 client.20979 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:03.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 10 client.20979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 20 client.20979 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5291a0440 0x55d5291a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 4 client.20979 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5290f4850 con 0x55d5291a0440 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed77fff640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5291a0440 0x55d5291a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed77fff640 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5291a0440 0x55d5291a2830 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0239a0da22338dc server_cookie=c37e7c89bc4db543 in_seq=0 out_seq=0 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.196+0000 7fed57fff640 10 client.20979 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed57fff640 1 -- 172.21.15.77:0/2962058979 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5290f4850 con 0x55d5291a0440 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed57fff640 10 client.20979 handle_command_reply: tid=0 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 client.20979 shutdown 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 msgr2=0x55d529190400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fed7000a010 tx=0x7fed700049c0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 shutdown_connections 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d5291a0440 0x55d5291a2830 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d52918dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181ca0 0x55d52918d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed57fff640 0 client.20979 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 --2- 172.21.15.77:0/2962058979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181380 0x55d529190400 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 >> 172.21.15.77:0/2962058979 conn(0x55d52917da30 msgr2=0x55d529183790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 shutdown_connections 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 1 -- 172.21.15.77:0/2962058979 wait complete. 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.197+0000 7fed80f89740 20 client.20979 trim_cache size 0 max 16384 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 Processor -- start 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 -- start start 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181380 0x55d52919c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d52917ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed781acfd0 con 0x55d529181ca0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed781aeb10 con 0x55d529181380 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed80f89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed781ad6a0 con 0x55d529190a90 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35646/0 (socket says 172.21.15.77:35646) 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.214+0000 7fed77fff640 1 -- 172.21.15.77:0/3552698662 learned_addr learned my addr 172.21.15.77:0/3552698662 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181380 0x55d52919c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d52917ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 -- 172.21.15.77:0/3552698662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 msgr2=0x55d52917ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d52917ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 -- 172.21.15.77:0/3552698662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181380 msgr2=0x55d52919c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181380 0x55d52919c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 -- 172.21.15.77:0/3552698662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52918bc00 con 0x55d529181ca0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed77fff640 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2be729e1fb7c6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed627fc640 1 -- 172.21.15.77:0/3552698662 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6404f020 con 0x55d529181ca0 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed627fc640 1 -- 172.21.15.77:0/3552698662 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64031a20 con 0x55d529181ca0 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.215+0000 7fed627fc640 1 -- 172.21.15.77:0/3552698662 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64035040 con 0x55d529181ca0 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- 172.21.15.77:0/3552698662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 msgr2=0x55d52918cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed64009250 comp rx=0 tx=0).stop 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- 172.21.15.77:0/3552698662 shutdown_connections 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529190a90 0x55d52917ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d52918cf90 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 --2- 172.21.15.77:0/3552698662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529181380 0x55d52919c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- 172.21.15.77:0/3552698662 >> 172.21.15.77:0/3552698662 conn(0x55d52917da30 msgr2=0x55d52917c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- 172.21.15.77:0/3552698662 shutdown_connections 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- 172.21.15.77:0/3552698662 wait complete. 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 -- start start 2024-09-15T22:37:03.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.216+0000 7fed80f89740 1 Processor -- start 2024-09-15T22:37:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 0x55d529184830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d529197a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fed781ae440 con 0x55d529181ca0 2024-09-15T22:37:03.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed781add70 con 0x55d529190a90 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed781aafd0 con 0x55d529181380 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed80f89740 10 client.0 _ll_register_callbacks cb 0x7fed70018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35654/0 (socket says 172.21.15.77:35654) 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 -- 172.21.15.77:0/3401397084 learned_addr learned my addr 172.21.15.77:0/3401397084 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d529197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 0x55d529184830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 -- 172.21.15.77:0/3401397084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 msgr2=0x55d529184830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 0x55d529184830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 -- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 msgr2=0x55d529197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d529197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed77fff640 1 -- 172.21.15.77:0/3401397084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d529197fa0 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.217+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 0x55d529184830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed77fff640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fed64009850 tx=0x7fed64005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196473a43e36f6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64058070 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fed64004030 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed64031a20 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fed64064050 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 10 client.20985 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d529199220 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64031a20 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fed64003bf0 con 0x55d529181ca0 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 10 client.20985 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:03.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 10 client.20985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 20 client.20985 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d529206ed0 0x55d5292092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 4 client.20985 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.218+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5290eed60 con 0x55d529206ed0 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d529206ed0 0x55d5292092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed7d14d640 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d529206ed0 0x55d5292092c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f73d8291f448b4 server_cookie=7f7c37066a120664 in_seq=0 out_seq=0 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed57fff640 10 client.20985 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed57fff640 1 -- 172.21.15.77:0/3401397084 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d5290eed60 con 0x55d529206ed0 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed57fff640 10 client.20985 handle_command_reply: tid=0 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 client.20985 shutdown 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 msgr2=0x55d529184d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fed64009850 tx=0x7fed64005780 comp rx=0 tx=0).stop 2024-09-15T22:37:03.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 shutdown_connections 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55d529206ed0 0x55d5292092c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d529190a90 0x55d529197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d529181ca0 0x55d529184d70 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 --2- 172.21.15.77:0/3401397084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d529181380 0x55d529184830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed57fff640 0 client.20985 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.219+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 >> 172.21.15.77:0/3401397084 conn(0x55d52917da30 msgr2=0x55d5291836b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 shutdown_connections 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed80f89740 1 -- 172.21.15.77:0/3401397084 wait complete. 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed80f89740 20 client.20985 trim_cache size 0 max 16384 2024-09-15T22:37:03.236 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:37:03.237 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:37:03.237 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:37:03.237 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:37:03.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 msgr2=0x7fed680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 0x7fed680a5870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0cc9a0 tx=0x7fed6c0046c0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 msgr2=0x7fed781985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b4a0 tx=0x7fed6c00b960 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 shutdown_connections 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fed680a33b0 0x7fed680a5870 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed78106f10 0x7fed7819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed78102790 0x7fed78198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 --2- 172.21.15.77:0/3557485143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed78101de0 0x7fed781985b0 secure :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b4a0 tx=0x7fed6c00b960 comp rx=0 tx=0).stop 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.220+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 >> 172.21.15.77:0/3557485143 conn(0x7fed780fd8c0 msgr2=0x7fed780fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.221+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 shutdown_connections 2024-09-15T22:37:03.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:03.221+0000 7fed7ebb0640 1 -- 172.21.15.77:0/3557485143 wait complete. 2024-09-15T22:37:03.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 14.00 MB/sec execute 544 sec latency 4138.544 ms 2024-09-15T22:37:03.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:37:03.916 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:04.325 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:03 smithi077 ceph-mon[30980]: pgmap v778: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.5 MiB/s wr, 226 op/s 2024-09-15T22:37:04.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:03 smithi196 ceph-mon[35109]: pgmap v778: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.5 MiB/s wr, 226 op/s 2024-09-15T22:37:04.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:03 smithi117 ceph-mon[35021]: pgmap v778: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.5 MiB/s wr, 226 op/s 2024-09-15T22:37:04.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.97 MB/sec execute 545 sec latency 5138.597 ms 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 -- 172.21.15.77:0/1136198411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 msgr2=0x7fc63810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/1136198411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63810da40 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ca0 tx=0x7fc63002a950 comp rx=0 tx=0).stop 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 -- 172.21.15.77:0/1136198411 shutdown_connections 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/1136198411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63810da40 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/1136198411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 0x7fc6381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/1136198411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 -- 172.21.15.77:0/1136198411 >> 172.21.15.77:0/1136198411 conn(0x7fc6380fc7d0 msgr2=0x7fc6380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 -- 172.21.15.77:0/1136198411 shutdown_connections 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 -- 172.21.15.77:0/1136198411 wait complete. 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.726+0000 7fc63dbb5640 1 Processor -- start 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 -- start start 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 0x7fc638198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110f80 con 0x7fc638102ef0 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110e00 con 0x7fc638100e20 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc63dbb5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc638111100 con 0x7fc638102570 2024-09-15T22:37:04.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35680/0 (socket says 172.21.15.77:35680) 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 -- 172.21.15.77:0/3480409331 learned_addr learned my addr 172.21.15.77:0/3480409331 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc636ffd640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 0x7fc638198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc6377fe640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 -- 172.21.15.77:0/3480409331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 msgr2=0x7fc638198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 0x7fc638198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 -- 172.21.15.77:0/3480409331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 msgr2=0x7fc638198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc637fff640 1 -- 172.21.15.77:0/3480409331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63819d4e0 con 0x7fc638102ef0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.727+0000 7fc6377fe640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc637fff640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc63002ae70 tx=0x7fc63002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630034040 con 0x7fc638102ef0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63819d770 con 0x7fc638102ef0 2024-09-15T22:37:04.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630030920 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630045480 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.728+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6381a50b0 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.729+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6381021b0 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.730+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc630037020 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.730+0000 7fc634ff9640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 0x7fc6200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.731+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7fc6300f0e00 con 0x7fc638102ef0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.731+0000 7fc6377fe640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 0x7fc6200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.731+0000 7fc6377fe640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 0x7fc6200a5870 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005d00 tx=0x7fc62c00b410 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:04.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.735+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6300bd560 con 0x7fc638102ef0 2024-09-15T22:37:04.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.946+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc63810a0a0 con 0x7fc638102ef0 2024-09-15T22:37:04.948 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:04 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:04.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.947+0000 7fc634ff9640 1 -- 172.21.15.77:0/3480409331 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fc6300bce60 con 0x7fc638102ef0 2024-09-15T22:37:04.949 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:37:04.949 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:37:04.950 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 msgr2=0x7fc6200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 0x7fc6200a5870 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc62c005d00 tx=0x7fc62c00b410 comp rx=0 tx=0).stop 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 msgr2=0x7fc63819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc63002ae70 tx=0x7fc63002f5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 shutdown_connections 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc6200a33b0 0x7fc6200a5870 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc638102ef0 0x7fc63819ce00 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc638102570 0x7fc638198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.949+0000 7fc63dbb5640 1 --2- 172.21.15.77:0/3480409331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc638100e20 0x7fc638198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.950+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 >> 172.21.15.77:0/3480409331 conn(0x7fc6380fc7d0 msgr2=0x7fc63806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:04.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.950+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 shutdown_connections 2024-09-15T22:37:04.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:04.950+0000 7fc63dbb5640 1 -- 172.21.15.77:0/3480409331 wait complete. 2024-09-15T22:37:05.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:04 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:05.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:04 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:05.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.95 MB/sec execute 546 sec latency 6138.692 ms 2024-09-15T22:37:05.673 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:37:05.919 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:06.040 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:05 smithi077 ceph-mon[30980]: pgmap v779: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.4 MiB/s wr, 221 op/s 2024-09-15T22:37:06.040 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:05 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3480409331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:06.040 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:06.040 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:06.041 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:37:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:05 smithi196 ceph-mon[35109]: pgmap v779: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.4 MiB/s wr, 221 op/s 2024-09-15T22:37:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:05 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3480409331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:06.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:37:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:05 smithi117 ceph-mon[35021]: pgmap v779: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.4 MiB/s wr, 221 op/s 2024-09-15T22:37:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:05 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3480409331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:06.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:37:06.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.457+0000 7f242f59e640 1 -- 172.21.15.77:0/2771471950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428098f80 msgr2=0x7f24280a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.457+0000 7f242f59e640 1 --2- 172.21.15.77:0/2771471950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428098f80 0x7f24280a0b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2430067960 tx=0x7f2430098d90 comp rx=0 tx=0).stop 2024-09-15T22:37:06.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 -- 172.21.15.77:0/2771471950 shutdown_connections 2024-09-15T22:37:06.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 --2- 172.21.15.77:0/2771471950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428098f80 0x7f24280a0b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 --2- 172.21.15.77:0/2771471950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428094cb0 0x7f2428098850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 --2- 172.21.15.77:0/2771471950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f24280946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 -- 172.21.15.77:0/2771471950 >> 172.21.15.77:0/2771471950 conn(0x7f2428009960 msgr2=0x7f2428009d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.458+0000 7f242f59e640 1 -- 172.21.15.77:0/2771471950 shutdown_connections 2024-09-15T22:37:06.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.459+0000 7f242f59e640 1 -- 172.21.15.77:0/2771471950 wait complete. 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.459+0000 7f242f59e640 1 Processor -- start 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 -- start start 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f242809f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428094cb0 0x7f242809fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24280a2cd0 con 0x7f2428094300 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24280a2b50 con 0x7f2428098f80 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242f59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24280a2e50 con 0x7f2428094cb0 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41256/0 (socket says 172.21.15.77:41256) 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.460+0000 7f242ed9d640 1 -- 172.21.15.77:0/2578301268 learned_addr learned my addr 172.21.15.77:0/2578301268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242e59c640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f242809f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.462 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242dd9b640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428094cb0 0x7f242809fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 -- 172.21.15.77:0/2578301268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428094cb0 msgr2=0x7f242809fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428094cb0 0x7f242809fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 -- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 msgr2=0x7f242809f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f242809f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 -- 172.21.15.77:0/2578301268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242813e620 con 0x7f2428098f80 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242e59c640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f242809f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f242ed9d640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2430051790 tx=0x7f24300a29c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.461+0000 7f241f7fe640 1 -- 172.21.15.77:0/2578301268 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24300b0020 con 0x7f2428098f80 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.462+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242813e8b0 con 0x7f2428098f80 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.462+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242813ee10 con 0x7f2428098f80 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.462+0000 7f241f7fe640 1 -- 172.21.15.77:0/2578301268 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24300ac950 con 0x7f2428098f80 2024-09-15T22:37:06.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.462+0000 7f241f7fe640 1 -- 172.21.15.77:0/2578301268 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24300aa800 con 0x7f2428098f80 2024-09-15T22:37:06.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.465+0000 7f24371c6740 1 Processor -- start 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.467+0000 7f24371c6740 1 -- start start 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.467+0000 7f24371c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f24371c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f41bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f24371c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f4e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f24371c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e050ea72f0 con 0x55e050f4c520 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f24371c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e050e4ecf0 con 0x55e050f42cf0 2024-09-15T22:37:06.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f24371c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e050ea32b0 con 0x55e050f41780 2024-09-15T22:37:06.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.468+0000 7f241f7fe640 1 -- 172.21.15.77:0/2578301268 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f24300b3020 con 0x7f2428098f80 2024-09-15T22:37:06.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.469+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.469+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41262/0 (socket says 172.21.15.77:41262) 2024-09-15T22:37:06.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.469+0000 7f242dd9b640 1 -- 172.21.15.77:0/1830501366 learned_addr learned my addr 172.21.15.77:0/1830501366 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:06.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.469+0000 7f242dd9b640 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f4e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.470+0000 7f242e59c640 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f41bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f241f7fe640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 0x7f24180a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 -- 172.21.15.77:0/1830501366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 msgr2=0x55e050f41bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f41bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 -- 172.21.15.77:0/1830501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 msgr2=0x55e050f4e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f4e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 -- 172.21.15.77:0/1830501366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e050e333a0 con 0x55e050f42cf0 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f241f7fe640 1 -- 172.21.15.77:0/2578301268 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f243015d900 con 0x7f2428098f80 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242ed9d640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 0x7f24180a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.471+0000 7f242dd9b640 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f24240058c0 tx=0x7f2424009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5fcac30c736a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.473 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.472+0000 7f241cff9640 1 -- 172.21.15.77:0/1830501366 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424012c60 con 0x55e050f42cf0 2024-09-15T22:37:06.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.472+0000 7f241cff9640 1 -- 172.21.15.77:0/1830501366 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2424012e00 con 0x55e050f42cf0 2024-09-15T22:37:06.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.472+0000 7f241cff9640 1 -- 172.21.15.77:0/1830501366 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424003850 con 0x55e050f42cf0 2024-09-15T22:37:06.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.472+0000 7f242ed9d640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 0x7f24180a5980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2430000b60 tx=0x7f2430067a00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.473+0000 7f24371c6740 1 -- 172.21.15.77:0/1830501366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 msgr2=0x55e050f41240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.473+0000 7f24371c6740 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f24240058c0 tx=0x7f2424009680 comp rx=0 tx=0).stop 2024-09-15T22:37:06.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 -- 172.21.15.77:0/1830501366 shutdown_connections 2024-09-15T22:37:06.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f4e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f41bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 --2- 172.21.15.77:0/1830501366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f41240 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 -- 172.21.15.77:0/1830501366 >> 172.21.15.77:0/1830501366 conn(0x55e050f0a310 msgr2=0x55e050f3de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 -- 172.21.15.77:0/1830501366 shutdown_connections 2024-09-15T22:37:06.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.474+0000 7f24371c6740 1 -- 172.21.15.77:0/1830501366 wait complete. 2024-09-15T22:37:06.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.475+0000 7f24371c6740 1 -- start start 2024-09-15T22:37:06.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.475+0000 7f24371c6740 1 Processor -- start 2024-09-15T22:37:06.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.475+0000 7f24371c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f4bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:47678/0 (socket says 172.21.15.77:47678) 2024-09-15T22:37:06.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41272/0 (socket says 172.21.15.77:41272) 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f49780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e050f60230 con 0x55e050f4c520 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e050e62060 con 0x55e050f42cf0 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e050f603b0 con 0x55e050f41780 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f24371c6740 10 client.0 _ll_register_callbacks cb 0x55e050bb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242dd9b640 1 -- 172.21.15.77:0/2972720258 learned_addr learned my addr 172.21.15.77:0/2972720258 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 -- 172.21.15.77:0/2972720258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 msgr2=0x55e050f4bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f4bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 -- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 msgr2=0x55e050f49780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f49780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.476+0000 7f242e59c640 1 -- 172.21.15.77:0/2972720258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e050c183c0 con 0x55e050f42cf0 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f242e59c640 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2420005fd0 tx=0x7f2420012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b5aa641ab6938 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f2415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004230 con 0x55e050f42cf0 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f24200043d0 con 0x55e050f42cf0 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2424012e00 con 0x55e050f42cf0 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2420010760 con 0x55e050f42cf0 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f24371c6740 10 client.29648 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:06.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e050f6bdd0 con 0x55e050f42cf0 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.477+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2420010de0 con 0x55e050f42cf0 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2420019380 con 0x55e050f42cf0 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 10 client.29648 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 10 client.29648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 20 client.29648 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050f6d360 0x55e050f6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 4 client.29648 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:06.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2424012e00 con 0x55e050f6d360 2024-09-15T22:37:06.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f242dd9b640 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050f6d360 0x55e050f6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.478+0000 7f242dd9b640 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050f6d360 0x55e050f6f780 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79aa0a6e2e7f547c server_cookie=e58647620f5b85e6 in_seq=0 out_seq=0 2024-09-15T22:37:06.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.479+0000 7f2415ffb640 10 client.29648 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:06.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.480+0000 7f2415ffb640 1 -- 172.21.15.77:0/2972720258 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2424012e00 con 0x55e050f6d360 2024-09-15T22:37:06.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.480+0000 7f2415ffb640 10 client.29648 handle_command_reply: tid=0 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.480+0000 7f24371c6740 1 client.29648 shutdown 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 msgr2=0x55e050f491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2420005fd0 tx=0x7f2420012820 comp rx=0 tx=0).stop 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 shutdown_connections 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050f6d360 0x55e050f6f780 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f4c520 0x55e050f49780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f42cf0 0x55e050f491b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 --2- 172.21.15.77:0/2972720258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f4bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 >> 172.21.15.77:0/2972720258 conn(0x55e050f0a310 msgr2=0x55e050f43ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.481+0000 7f2415ffb640 0 client.29648 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:06.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.482+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 shutdown_connections 2024-09-15T22:37:06.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.482+0000 7f24371c6740 1 -- 172.21.15.77:0/2972720258 wait complete. 2024-09-15T22:37:06.484 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.482+0000 7f24371c6740 20 client.29648 trim_cache size 0 max 16384 2024-09-15T22:37:06.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.499+0000 7f24371c6740 1 Processor -- start 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.499+0000 7f24371c6740 1 -- start start 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.499+0000 7f24371c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.499+0000 7f24371c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f24371c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 0x55e050f442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f24371c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f241000ffb0 con 0x55e050f42cf0 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f24371c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2410011af0 con 0x55e050f4c520 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f24371c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2410010680 con 0x55e050f41780 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35732/0 (socket says 172.21.15.77:35732) 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 -- 172.21.15.77:0/112861115 learned_addr learned my addr 172.21.15.77:0/112861115 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242dd9b640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242dd9b640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 0x55e050f442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 -- 172.21.15.77:0/112861115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 msgr2=0x55e050f3d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f3d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 -- 172.21.15.77:0/112861115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 msgr2=0x55e050f442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 0x55e050f442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 -- 172.21.15.77:0/112861115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e050f3eb10 con 0x55e050f42cf0 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242dd9b640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 0x55e050f442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f242e59c640 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f2420019990 tx=0x7f24200059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3570b0089ca7f612 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f2417fff640 1 -- 172.21.15.77:0/112861115 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420019580 con 0x55e050f42cf0 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f2417fff640 1 -- 172.21.15.77:0/112861115 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2420012d20 con 0x55e050f42cf0 2024-09-15T22:37:06.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.500+0000 7f2417fff640 1 -- 172.21.15.77:0/112861115 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420005290 con 0x55e050f42cf0 2024-09-15T22:37:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 -- 172.21.15.77:0/112861115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 msgr2=0x55e050f3cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f2420019990 tx=0x7f24200059b0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 -- 172.21.15.77:0/112861115 shutdown_connections 2024-09-15T22:37:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f4c520 0x55e050f442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f3cda0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 --2- 172.21.15.77:0/112861115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f41780 0x55e050f3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 -- 172.21.15.77:0/112861115 >> 172.21.15.77:0/112861115 conn(0x55e050f0a310 msgr2=0x55e050f3c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 -- 172.21.15.77:0/112861115 shutdown_connections 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.501+0000 7f24371c6740 1 -- 172.21.15.77:0/112861115 wait complete. 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 -- start start 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 Processor -- start 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f61050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f4c520 0x55e050f61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2410011420 con 0x55e050f42cf0 2024-09-15T22:37:06.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2410010d50 con 0x55e050f41780 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f241000dfb0 con 0x55e050f4c520 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:41292/0 (socket says 172.21.15.77:41292) 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f242dd9b640 1 -- 172.21.15.77:0/730136956 learned_addr learned my addr 172.21.15.77:0/730136956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f24371c6740 10 client.0 _ll_register_callbacks cb 0x7f242000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.502+0000 7f242dd9b640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f4c520 0x55e050f61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242e59c640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f61050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 -- 172.21.15.77:0/730136956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f4c520 msgr2=0x55e050f61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f4c520 0x55e050f61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 -- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 msgr2=0x55e050f61050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f61050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 -- 172.21.15.77:0/730136956 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e050f61dc0 con 0x55e050f41780 2024-09-15T22:37:06.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f242dd9b640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2424010de0 tx=0x7f2424005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd5b6f330690363 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f2415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424017040 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2420012d20 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f2424012250 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f242403a050 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f24371c6740 10 client.29654 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.503+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e050fd7f40 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2420012d20 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f2424003830 con 0x55e050f41780 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 10 client.29654 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 10 client.29654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 20 client.29654 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050fd84d0 0x55e050fda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 4 client.29654 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e050eb4860 con 0x55e050fd84d0 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f242e59c640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050fd84d0 0x55e050fda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f2415ffb640 10 client.29654 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.504+0000 7f242e59c640 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050fd84d0 0x55e050fda8c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91f74d55be11be21 server_cookie=f36a2355acc9539 in_seq=0 out_seq=0 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f2415ffb640 1 -- 172.21.15.77:0/730136956 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e050eb4860 con 0x55e050fd84d0 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f2415ffb640 10 client.29654 handle_command_reply: tid=0 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 client.29654 shutdown 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 msgr2=0x55e050f65d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2424010de0 tx=0x7f2424005040 comp rx=0 tx=0).stop 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 shutdown_connections 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55e050fd84d0 0x55e050fda8c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e050f4c520 0x55e050f61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e050f42cf0 0x55e050f61050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f2415ffb640 0 client.29654 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 --2- 172.21.15.77:0/730136956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e050f41780 0x55e050f65d20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 >> 172.21.15.77:0/730136956 conn(0x55e050f0a310 msgr2=0x55e050f3ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 shutdown_connections 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 1 -- 172.21.15.77:0/730136956 wait complete. 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.505+0000 7f24371c6740 20 client.29654 trim_cache size 0 max 16384 2024-09-15T22:37:06.512 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 msgr2=0x7f24180a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 0x7f24180a5980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2430000b60 tx=0x7f2430067a00 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 msgr2=0x7f242813ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2430051790 tx=0x7f24300a29c0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 shutdown_connections 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f24180a34c0 0x7f24180a5980 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2428098f80 0x7f242813ded0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2428094cb0 0x7f242809fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 --2- 172.21.15.77:0/2578301268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2428094300 0x7f242809f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 >> 172.21.15.77:0/2578301268 conn(0x7f2428009960 msgr2=0x7f2428093740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 shutdown_connections 2024-09-15T22:37:06.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:06.506+0000 7f242f59e640 1 -- 172.21.15.77:0/2578301268 wait complete. 2024-09-15T22:37:06.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.92 MB/sec execute 547 sec latency 7138.737 ms 2024-09-15T22:37:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:07 smithi077 ceph-mon[30980]: osdmap e128: 12 total, 12 up, 12 in 2024-09-15T22:37:07.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:07 smithi196 ceph-mon[35109]: osdmap e128: 12 total, 12 up, 12 in 2024-09-15T22:37:07.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:07 smithi117 ceph-mon[35021]: osdmap e128: 12 total, 12 up, 12 in 2024-09-15T22:37:07.537 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:37:07.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.90 MB/sec execute 548 sec latency 8138.782 ms 2024-09-15T22:37:07.786 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:08.200 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:08 smithi077 ceph-mon[30980]: pgmap v781: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-09-15T22:37:08.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 -- 172.21.15.77:0/3382468176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 msgr2=0x7fc74c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3382468176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c10da40 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc744009ca0 tx=0x7fc74402a950 comp rx=0 tx=0).stop 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 -- 172.21.15.77:0/3382468176 shutdown_connections 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3382468176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c10da40 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3382468176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c102570 0x7fc74c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3382468176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c100e20 0x7fc74c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 -- 172.21.15.77:0/3382468176 >> 172.21.15.77:0/3382468176 conn(0x7fc74c0fc7d0 msgr2=0x7fc74c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 -- 172.21.15.77:0/3382468176 shutdown_connections 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.280+0000 7fc7531e6640 1 -- 172.21.15.77:0/3382468176 wait complete. 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 Processor -- start 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 -- start start 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c100e20 0x7fc74c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c102570 0x7fc74c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c110f80 con 0x7fc74c102ef0 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c110e00 con 0x7fc74c102570 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc7531e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c111100 con 0x7fc74c100e20 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc751783640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc751783640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35758/0 (socket says 172.21.15.77:35758) 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc751783640 1 -- 172.21.15.77:0/3450676638 learned_addr learned my addr 172.21.15.77:0/3450676638 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc74bfff640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c102570 0x7fc74c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.281+0000 7fc750f82640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c100e20 0x7fc74c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 -- 172.21.15.77:0/3450676638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c100e20 msgr2=0x7fc74c197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c100e20 0x7fc74c197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 -- 172.21.15.77:0/3450676638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c102570 msgr2=0x7fc74c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c102570 0x7fc74c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 -- 172.21.15.77:0/3450676638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c19c3b0 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc751783640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc74402ae70 tx=0x7fc74402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc749ffb640 1 -- 172.21.15.77:0/3450676638 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744034040 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c19c640 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc749ffb640 1 -- 172.21.15.77:0/3450676638 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc744030920 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc749ffb640 1 -- 172.21.15.77:0/3450676638 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744045480 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.282+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c1a3f80 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.284+0000 7fc749ffb640 1 -- 172.21.15.77:0/3450676638 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc744037020 con 0x7fc74c102ef0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 Processor -- start 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc749ffb640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 0x7fc73c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 -- start start 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc74bfff640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 0x7fc73c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.304 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b6905970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc749ffb640 1 -- 172.21.15.77:0/3450676638 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7fc7440f0ec0 con 0x7fc74c102ef0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9b685c2f0 con 0x55d9b68f87e0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6803cf0 con 0x55d9b6903580 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc7555bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9b68582b0 con 0x55d9b68f9d50 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35772/0 (socket says 172.21.15.77:35772) 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.285+0000 7fc74bfff640 1 -- 172.21.15.77:0/769888932 learned_addr learned my addr 172.21.15.77:0/769888932 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc750f82640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc751783640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b6905970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 0x7fc73c0a5870 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc73800bb00 tx=0x7fc73800bf00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 -- 172.21.15.77:0/769888932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 msgr2=0x55d9b68f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 -- 172.21.15.77:0/769888932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 msgr2=0x55d9b6905970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b6905970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 -- 172.21.15.77:0/769888932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b67e83a0 con 0x55d9b68f87e0 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc751783640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b6905970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc74bfff640 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc738003a80 tx=0x7fc73801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e520143da5be310 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc736ffd640 1 -- 172.21.15.77:0/769888932 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738023a60 con 0x55d9b68f87e0 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc736ffd640 1 -- 172.21.15.77:0/769888932 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738027030 con 0x55d9b68f87e0 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.286+0000 7fc736ffd640 1 -- 172.21.15.77:0/769888932 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738004b00 con 0x55d9b68f87e0 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- 172.21.15.77:0/769888932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 msgr2=0x55d9b68f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc738003a80 tx=0x7fc73801e000 comp rx=0 tx=0).stop 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- 172.21.15.77:0/769888932 shutdown_connections 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b6905970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f8c20 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 --2- 172.21.15.77:0/769888932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- 172.21.15.77:0/769888932 >> 172.21.15.77:0/769888932 conn(0x55d9b68bf310 msgr2=0x55d9b68f4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- 172.21.15.77:0/769888932 shutdown_connections 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- 172.21.15.77:0/769888932 wait complete. 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 -- start start 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.287+0000 7fc7555bf740 1 Processor -- start 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68feb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9b68f27d0 con 0x55d9b68f87e0 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d9b6817060 con 0x55d9b6903580 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9b68f2950 con 0x55d9b68f9d50 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 10 client.0 _ll_register_callbacks cb 0x55d9b656a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:08.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35786/0 (socket says 172.21.15.77:35786) 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 -- 172.21.15.77:0/4261934913 learned_addr learned my addr 172.21.15.77:0/4261934913 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc74bfff640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc751783640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 -- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 msgr2=0x55d9b68feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68feb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 -- 172.21.15.77:0/4261934913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 msgr2=0x55d9b68ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 -- 172.21.15.77:0/4261934913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b65cd3c0 con 0x55d9b68f87e0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc750f82640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc74000ba00 tx=0x7fc74000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d30fdb4420f693 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc72bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:08.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74001b640 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7fc74001b7e0 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc738027030 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc740024050 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 10 client.21018 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.288+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d9b690fb20 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738027030 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc740019020 con 0x55d9b68f87e0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 10 client.21018 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 10 client.21018 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 20 client.21018 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b6910fe0 0x55d9b6913460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 4 client.21018 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b6869860 con 0x55d9b6910fe0 2024-09-15T22:37:08.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.289+0000 7fc74bfff640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b6910fe0 0x55d9b6913460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc72bfff640 10 client.21018 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc74bfff640 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b6910fe0 0x55d9b6913460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17d2f587d69feb67 server_cookie=3968b74d2e126317 in_seq=0 out_seq=0 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc72bfff640 1 -- 172.21.15.77:0/4261934913 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9b6869860 con 0x55d9b6910fe0 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc72bfff640 10 client.21018 handle_command_reply: tid=0 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc7555bf740 1 client.21018 shutdown 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 msgr2=0x55d9b68fe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.290+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc74000ba00 tx=0x7fc74000bec0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 shutdown_connections 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b6910fe0 0x55d9b6913460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68feb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc72bfff640 0 client.21018 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 --2- 172.21.15.77:0/4261934913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68fe600 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 >> 172.21.15.77:0/4261934913 conn(0x55d9b68bf310 msgr2=0x55d9b68fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 shutdown_connections 2024-09-15T22:37:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 1 -- 172.21.15.77:0/4261934913 wait complete. 2024-09-15T22:37:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.291+0000 7fc7555bf740 20 client.21018 trim_cache size 0 max 16384 2024-09-15T22:37:08.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 Processor -- start 2024-09-15T22:37:08.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 -- start start 2024-09-15T22:37:08.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1acdb0 con 0x55d9b68f87e0 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1ae8f0 con 0x55d9b6903580 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc7555bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1ad480 con 0x55d9b68f9d50 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.304+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35790/0 (socket says 172.21.15.77:35790) 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 -- 172.21.15.77:0/3865400490 learned_addr learned my addr 172.21.15.77:0/3865400490 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc751783640 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 -- 172.21.15.77:0/3865400490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 msgr2=0x55d9b68f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 -- 172.21.15.77:0/3865400490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 msgr2=0x55d9b68f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 -- 172.21.15.77:0/3865400490 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b68ff5d0 con 0x55d9b68f87e0 2024-09-15T22:37:08.316 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc751783640 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc750f82640 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc740000c00 tx=0x7fc740003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6bd61dd9c0a2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc7367fc640 1 -- 172.21.15.77:0/3865400490 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740010440 con 0x55d9b68f87e0 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc7367fc640 1 -- 172.21.15.77:0/3865400490 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74002d070 con 0x55d9b68f87e0 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.305+0000 7fc7367fc640 1 -- 172.21.15.77:0/3865400490 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740010970 con 0x55d9b68f87e0 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- 172.21.15.77:0/3865400490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 msgr2=0x55d9b68f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc740000c00 tx=0x7fc740003fb0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- 172.21.15.77:0/3865400490 shutdown_connections 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b6903580 0x55d9b68f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b68f9d50 0x55d9b68f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 --2- 172.21.15.77:0/3865400490 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b68f4f80 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- 172.21.15.77:0/3865400490 >> 172.21.15.77:0/3865400490 conn(0x55d9b68bf310 msgr2=0x55d9b68f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- 172.21.15.77:0/3865400490 shutdown_connections 2024-09-15T22:37:08.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- 172.21.15.77:0/3865400490 wait complete. 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 -- start start 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.306+0000 7fc7555bf740 1 Processor -- start 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b68f9d50 0x55d9b6908530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b6903580 0x55d9b6908a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1ae220 con 0x55d9b68f87e0 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1adb50 con 0x55d9b68f9d50 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c1aadb0 con 0x55d9b6903580 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc7555bf740 10 client.0 _ll_register_callbacks cb 0x7fc740004140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35804/0 (socket says 172.21.15.77:35804) 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 -- 172.21.15.77:0/698382048 learned_addr learned my addr 172.21.15.77:0/698382048 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:08.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc751783640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b6903580 0x55d9b6908a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc74bfff640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b68f9d50 0x55d9b6908530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 -- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b6903580 msgr2=0x55d9b6908a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b6903580 0x55d9b6908a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 -- 172.21.15.77:0/698382048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b68f9d50 msgr2=0x55d9b6908530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b68f9d50 0x55d9b6908530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 -- 172.21.15.77:0/698382048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b697ad50 con 0x55d9b68f87e0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc750f82640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc740010690 tx=0x7fc74000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9dbe0b11db4012 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc72bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740011440 con 0x55d9b68f87e0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.307+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7fc7400115e0 con 0x55d9b68f87e0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74002d070 con 0x55d9b68f87e0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc740011820 con 0x55d9b68f87e0 2024-09-15T22:37:08.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 10 client.21024 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d9b697bfc0 con 0x55d9b68f87e0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74002d070 con 0x55d9b68f87e0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fc740019070 con 0x55d9b68f87e0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 10 client.21024 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 10 client.21024 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 20 client.21024 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b697c6f0 0x55d9b697eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 4 client.21024 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9b6863d70 con 0x55d9b697c6f0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.308+0000 7fc74bfff640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b697c6f0 0x55d9b697eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc74bfff640 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b697c6f0 0x55d9b697eb10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=355dbea065a5f5ef server_cookie=e96a8a3a7d8ba5f9 in_seq=0 out_seq=0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc72bfff640 10 client.21024 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc72bfff640 1 -- 172.21.15.77:0/698382048 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9b6863d70 con 0x55d9b697c6f0 2024-09-15T22:37:08.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc72bfff640 10 client.21024 handle_command_reply: tid=0 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 client.21024 shutdown 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 msgr2=0x55d9b6907ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc740010690 tx=0x7fc74000b390 comp rx=0 tx=0).stop 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 shutdown_connections 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55d9b697c6f0 0x55d9b697eb10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9b6903580 0x55d9b6908a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d9b68f9d50 0x55d9b6908530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 --2- 172.21.15.77:0/698382048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9b68f87e0 0x55d9b6907ff0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 >> 172.21.15.77:0/698382048 conn(0x55d9b68bf310 msgr2=0x55d9b68f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc72bfff640 0 client.21024 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 shutdown_connections 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 1 -- 172.21.15.77:0/698382048 wait complete. 2024-09-15T22:37:08.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.309+0000 7fc7555bf740 20 client.21024 trim_cache size 0 max 16384 2024-09-15T22:37:08.322 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:08.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 msgr2=0x7fc73c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 0x7fc73c0a5870 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc73800bb00 tx=0x7fc73800bf00 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 msgr2=0x7fc74c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc74402ae70 tx=0x7fc74402f5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 shutdown_connections 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fc73c0a33b0 0x7fc73c0a5870 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc74c102ef0 0x7fc74c19bcd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc74c102570 0x7fc74c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 --2- 172.21.15.77:0/3450676638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c100e20 0x7fc74c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 >> 172.21.15.77:0/3450676638 conn(0x7fc74c0fc7d0 msgr2=0x7fc74c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 shutdown_connections 2024-09-15T22:37:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:08.310+0000 7fc7531e6640 1 -- 172.21.15.77:0/3450676638 wait complete. 2024-09-15T22:37:08.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:08 smithi196 ceph-mon[35109]: pgmap v781: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-09-15T22:37:08.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:08 smithi117 ceph-mon[35021]: pgmap v781: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 240 op/s 2024-09-15T22:37:08.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.87 MB/sec execute 549 sec latency 9138.840 ms 2024-09-15T22:37:09.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:37:09.305 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:09.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.84 MB/sec execute 550 sec latency 10138.866 ms 2024-09-15T22:37:09.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 -- 172.21.15.77:0/456603359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214105f40 msgr2=0x7f42141063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 --2- 172.21.15.77:0/456603359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214105f40 0x7f42141063d0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f420c009ca0 tx=0x7f420c02a940 comp rx=0 tx=0).stop 2024-09-15T22:37:09.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 -- 172.21.15.77:0/456603359 shutdown_connections 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 --2- 172.21.15.77:0/456603359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214105f40 0x7f42141063d0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 --2- 172.21.15.77:0/456603359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214101450 0x7f4214105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 --2- 172.21.15.77:0/456603359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f4214100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 -- 172.21.15.77:0/456603359 >> 172.21.15.77:0/456603359 conn(0x7f42140fc7a0 msgr2=0x7f42140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 -- 172.21.15.77:0/456603359 shutdown_connections 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.791+0000 7f421c08f640 1 -- 172.21.15.77:0/456603359 wait complete. 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 Processor -- start 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 -- start start 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f42141a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214105f40 0x7f421419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4214118560 con 0x7f4214101450 2024-09-15T22:37:09.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42141183e0 con 0x7f4214105f40 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421c08f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42141186e0 con 0x7f4214100b30 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56526/0 (socket says 172.21.15.77:56526) 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f421962a640 1 -- 172.21.15.77:0/1414971590 learned_addr learned my addr 172.21.15.77:0/1414971590 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.792+0000 7f4219e2b640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f42141a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421a62c640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214105f40 0x7f421419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 -- 172.21.15.77:0/1414971590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 msgr2=0x7f42141a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f42141a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 -- 172.21.15.77:0/1414971590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214105f40 msgr2=0x7f421419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214105f40 0x7f421419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 -- 172.21.15.77:0/1414971590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421419efd0 con 0x7f4214101450 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f4219e2b640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f42141a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:09.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421962a640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f420400d740 tx=0x7f420400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f420affd640 1 -- 172.21.15.77:0/1414971590 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204017070 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f420affd640 1 -- 172.21.15.77:0/1414971590 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420400de50 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421419f2c0 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f420affd640 1 -- 172.21.15.77:0/1414971590 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400f620 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.793+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42141ab510 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.795+0000 7f420affd640 1 -- 172.21.15.77:0/1414971590 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4204009c30 con 0x7f4214101450 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.795+0000 7f421e468740 1 Processor -- start 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 -- start start 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e7f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e7f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55933e75e2f0 con 0x55933e803370 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55933e705cf0 con 0x55933e7f9b40 2024-09-15T22:37:09.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421e468740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55933e75a2b0 con 0x55933e7f85d0 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56536/0 (socket says 172.21.15.77:56536) 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421962a640 1 -- 172.21.15.77:0/905715381 learned_addr learned my addr 172.21.15.77:0/905715381 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421a62c640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e7f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f4219e2b640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e7f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f420affd640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 0x7f42000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421962a640 1 -- 172.21.15.77:0/905715381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 msgr2=0x55933e7f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.796+0000 7f421962a640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e7f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421962a640 1 -- 172.21.15.77:0/905715381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 msgr2=0x55933e7f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421962a640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e7f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421962a640 1 -- 172.21.15.77:0/905715381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933e6ea3a0 con 0x55933e803370 2024-09-15T22:37:09.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421a62c640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e7f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421a62c640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 0x7f42000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f420affd640 1 -- 172.21.15.77:0/1414971590 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f42040cfd60 con 0x7f4214101450 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421962a640 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f42040cd030 tx=0x7f4204066590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489d2ed96c6a2874 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f41fffff640 1 -- 172.21.15.77:0/905715381 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420409b790 con 0x55933e803370 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f421a62c640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 0x7f42000a5870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f420c006010 tx=0x7f420c0095c0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f41fffff640 1 -- 172.21.15.77:0/905715381 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42040ba890 con 0x55933e803370 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.797+0000 7f41fffff640 1 -- 172.21.15.77:0/905715381 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420409b5b0 con 0x55933e803370 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- 172.21.15.77:0/905715381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 msgr2=0x55933e805760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f42040cd030 tx=0x7f4204066590 comp rx=0 tx=0).stop 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- 172.21.15.77:0/905715381 shutdown_connections 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e805760 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e7f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 --2- 172.21.15.77:0/905715381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e7f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- 172.21.15.77:0/905715381 >> 172.21.15.77:0/905715381 conn(0x55933e7c1310 msgr2=0x55933e7f4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- 172.21.15.77:0/905715381 shutdown_connections 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- 172.21.15.77:0/905715381 wait complete. 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 -- start start 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.798+0000 7f421e468740 1 Processor -- start 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e802d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e803370 0x55933e8005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55933e805fb0 con 0x55933e7f9b40 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55933e719060 con 0x55933e803370 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55933e806130 con 0x55933e7f85d0 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421e468740 10 client.0 _ll_register_callbacks cb 0x55933e46c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56550/0 (socket says 172.21.15.77:56550) 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 -- 172.21.15.77:0/1597449405 learned_addr learned my addr 172.21.15.77:0/1597449405 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421a62c640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e803370 0x55933e8005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 -- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 msgr2=0x55933e802d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e802d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 -- 172.21.15.77:0/1597449405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e803370 msgr2=0x55933e8005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e803370 0x55933e8005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 -- 172.21.15.77:0/1597449405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933e4cf3c0 con 0x55933e7f9b40 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f421962a640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f42040a4390 tx=0x7f42040d0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbecc2f8fa76e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.799+0000 7f41fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204020040 con 0x55933e7f9b40 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42040ba890 con 0x55933e7f9b40 2024-09-15T22:37:09.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f42040afe30 con 0x55933e7f9b40 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f420409b5b0 con 0x55933e7f9b40 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 10 client.21036 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55933e811a10 con 0x55933e7f9b40 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42040ba890 con 0x55933e7f9b40 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f42040982c0 con 0x55933e7f9b40 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 10 client.21036 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 10 client.21036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 20 client.21036 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e812d60 0x55933e815150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 4 client.21036 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.800+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933e76b860 con 0x55933e812d60 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.801+0000 7f4219e2b640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e812d60 0x55933e815150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.801+0000 7f4219e2b640 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e812d60 0x55933e815150 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ae1727c02c1df40 server_cookie=54446e15601e2fc0 in_seq=0 out_seq=0 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.801+0000 7f41fd7fa640 10 client.21036 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f41fd7fa640 1 -- 172.21.15.77:0/1597449405 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55933e76b860 con 0x55933e812d60 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f41fd7fa640 10 client.21036 handle_command_reply: tid=0 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 client.21036 shutdown 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 msgr2=0x55933e7fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f42040a4390 tx=0x7f42040d0060 comp rx=0 tx=0).stop 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 shutdown_connections 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e812d60 0x55933e815150 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e803370 0x55933e8005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e7f9b40 0x55933e7fffe0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 --2- 172.21.15.77:0/1597449405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e802d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f41fd7fa640 0 client.21036 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:09.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 >> 172.21.15.77:0/1597449405 conn(0x55933e7c1310 msgr2=0x55933e7fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 shutdown_connections 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 1 -- 172.21.15.77:0/1597449405 wait complete. 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.802+0000 7f421e468740 20 client.21036 trim_cache size 0 max 16384 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 Processor -- start 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 -- start start 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f85d0 0x55933e80f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f9b40 0x55933e7ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b5810 con 0x55933e803370 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b7350 con 0x55933e7f85d0 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421e468740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b5ee0 con 0x55933e7f9b40 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421a62c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421a62c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56562/0 (socket says 172.21.15.77:56562) 2024-09-15T22:37:09.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.815+0000 7f421a62c640 1 -- 172.21.15.77:0/1191512455 learned_addr learned my addr 172.21.15.77:0/1191512455 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421962a640 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f9b40 0x55933e7ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f4219e2b640 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f85d0 0x55933e80f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 -- 172.21.15.77:0/1191512455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f9b40 msgr2=0x55933e7ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f9b40 0x55933e7ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 -- 172.21.15.77:0/1191512455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f85d0 msgr2=0x55933e80f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f85d0 0x55933e80f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 -- 172.21.15.77:0/1191512455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933e7f4de0 con 0x55933e803370 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f421a62c640 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f420c02f6d0 tx=0x7f420c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f8cc942a9e8c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f41ff7fe640 1 -- 172.21.15.77:0/1191512455 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c02f700 con 0x55933e803370 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f41ff7fe640 1 -- 172.21.15.77:0/1191512455 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c030b20 con 0x55933e803370 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.816+0000 7f41ff7fe640 1 -- 172.21.15.77:0/1191512455 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c004560 con 0x55933e803370 2024-09-15T22:37:09.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- 172.21.15.77:0/1191512455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 msgr2=0x55933e81a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f420c02f6d0 tx=0x7f420c041000 comp rx=0 tx=0).stop 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- 172.21.15.77:0/1191512455 shutdown_connections 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e81a7b0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f9b40 0x55933e7ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- 172.21.15.77:0/1191512455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f85d0 0x55933e80f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- 172.21.15.77:0/1191512455 >> 172.21.15.77:0/1191512455 conn(0x55933e7c1310 msgr2=0x55933e7f3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- 172.21.15.77:0/1191512455 shutdown_connections 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- 172.21.15.77:0/1191512455 wait complete. 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- start start 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 Processor -- start 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e807390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e8078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b6c80 con 0x55933e803370 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b65b0 con 0x55933e7f9b40 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b3810 con 0x55933e7f85d0 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.817+0000 7f421e468740 10 client.0 _ll_register_callbacks cb 0x7f4204084f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56572/0 (socket says 172.21.15.77:56572) 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 -- 172.21.15.77:0/2036801025 learned_addr learned my addr 172.21.15.77:0/2036801025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f4219e2b640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e807390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421962a640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e8078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 -- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 msgr2=0x55933e807390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e807390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 -- 172.21.15.77:0/2036801025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 msgr2=0x55933e8078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e8078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 -- 172.21.15.77:0/2036801025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933e8030a0 con 0x55933e803370 2024-09-15T22:37:09.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421a62c640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f420c030020 tx=0x7f420c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0d710d26beb0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f41fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c02f700 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f420c004510 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c030b20 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f420c004710 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421e468740 10 client.21042 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55933e800940 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.818+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c030b20 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f420c02f070 con 0x55933e803370 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 10 client.21042 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:09.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 10 client.21042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 20 client.21042 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e87def0 0x55933e8802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 4 client.21042 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55933e765d70 con 0x55933e87def0 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f4219e2b640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e87def0 0x55933e8802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f41fd7fa640 10 client.21042 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:09.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f4219e2b640 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e87def0 0x55933e8802e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea417d1939f5b6a6 server_cookie=aa71da188786509b in_seq=0 out_seq=0 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f41fd7fa640 1 -- 172.21.15.77:0/2036801025 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55933e765d70 con 0x55933e87def0 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f41fd7fa640 10 client.21042 handle_command_reply: tid=0 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.819+0000 7f421e468740 1 client.21042 shutdown 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 msgr2=0x55933e807e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f420c030020 tx=0x7f420c005bf0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 shutdown_connections 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55933e87def0 0x55933e8802e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55933e803370 0x55933e807e10 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933e7f9b40 0x55933e8078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f41fd7fa640 0 client.21042 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 --2- 172.21.15.77:0/2036801025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55933e7f85d0 0x55933e807390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 >> 172.21.15.77:0/2036801025 conn(0x55933e7c1310 msgr2=0x55933e7fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 shutdown_connections 2024-09-15T22:37:09.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 1 -- 172.21.15.77:0/2036801025 wait complete. 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421e468740 20 client.21042 trim_cache size 0 max 16384 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 msgr2=0x7f42000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 0x7f42000a5870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f420c006010 tx=0x7f420c0095c0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 msgr2=0x7f42141a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f420400d740 tx=0x7f420400dc00 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 shutdown_connections 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f42000a33b0 0x7f42000a5870 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4214105f40 0x7f421419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4214101450 0x7f42141a4bf0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 --2- 172.21.15.77:0/1414971590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4214100b30 0x7f42141a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.820+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 >> 172.21.15.77:0/1414971590 conn(0x7f42140fc7a0 msgr2=0x7f42141038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.821+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 shutdown_connections 2024-09-15T22:37:09.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:09.821+0000 7f421c08f640 1 -- 172.21.15.77:0/1414971590 wait complete. 2024-09-15T22:37:10.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:10 smithi117 ceph-mon[35021]: pgmap v782: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-09-15T22:37:10.429 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:10 smithi077 ceph-mon[30980]: pgmap v782: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-09-15T22:37:10.584 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:37:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:10 smithi196 ceph-mon[35109]: pgmap v782: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-09-15T22:37:10.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.82 MB/sec execute 551 sec latency 11138.917 ms 2024-09-15T22:37:10.827 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:11.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 -- 172.21.15.77:0/3824172874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0100e20 msgr2=0x7f69d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3824172874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0100e20 0x7f69d0101200 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009ca0 tx=0x7f69c802a950 comp rx=0 tx=0).stop 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 -- 172.21.15.77:0/3824172874 shutdown_connections 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3824172874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0102ef0 0x7f69d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3824172874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102570 0x7f69d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3824172874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0100e20 0x7f69d0101200 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 -- 172.21.15.77:0/3824172874 >> 172.21.15.77:0/3824172874 conn(0x7f69d00fc7d0 msgr2=0x7f69d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 -- 172.21.15.77:0/3824172874 shutdown_connections 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.299+0000 7f69d51f5640 1 -- 172.21.15.77:0/3824172874 wait complete. 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 Processor -- start 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 -- start start 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0100e20 0x7f69d01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102ef0 0x7f69d019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0110fa0 con 0x7f69d0102570 2024-09-15T22:37:11.319 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0110e20 con 0x7f69d0102ef0 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69d51f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111120 con 0x7f69d0100e20 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56590/0 (socket says 172.21.15.77:56590) 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69ced9d640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0100e20 0x7f69d01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69ce59c640 1 -- 172.21.15.77:0/3266349126 learned_addr learned my addr 172.21.15.77:0/3266349126 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.300+0000 7f69cf59e640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102ef0 0x7f69d019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 -- 172.21.15.77:0/3266349126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0100e20 msgr2=0x7f69d01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0100e20 0x7f69d01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 -- 172.21.15.77:0/3266349126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102ef0 msgr2=0x7f69d019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102ef0 0x7f69d019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 -- 172.21.15.77:0/3266349126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d019d490 con 0x7f69d0102570 2024-09-15T22:37:11.320 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69ce59c640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b4a0 tx=0x7f69b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69c7fff640 1 -- 172.21.15.77:0/3266349126 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8014070 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d019d780 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69c7fff640 1 -- 172.21.15.77:0/3266349126 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b8010af0 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69c7fff640 1 -- 172.21.15.77:0/3266349126 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8017d10 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.301+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01a5060 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.303+0000 7f69c7fff640 1 -- 172.21.15.77:0/3266349126 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f69b8017070 con 0x7f69d0102570 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.303+0000 7f69d75ce740 1 Processor -- start 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 -- start start 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c758a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7597760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574c74f02f0 con 0x5574c758a5d0 2024-09-15T22:37:11.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c7497cf0 con 0x5574c7595370 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69d75ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574c74ec2b0 con 0x5574c758bb40 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69c7fff640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 0x7f69bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56592/0 (socket says 172.21.15.77:56592) 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69ced9d640 1 -- 172.21.15.77:0/1771260958 learned_addr learned my addr 172.21.15.77:0/1771260958 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69cf59e640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 0x7f69bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69ce59c640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7597760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69cf59e640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c758a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.304+0000 7f69c7fff640 1 -- 172.21.15.77:0/3266349126 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f69b80cf440 con 0x7f69d0102570 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 -- 172.21.15.77:0/1771260958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 msgr2=0x5574c758a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c758a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 -- 172.21.15.77:0/1771260958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 msgr2=0x5574c7597760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7597760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 -- 172.21.15.77:0/1771260958 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c747c3a0 con 0x5574c758a5d0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69cf59e640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c758a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69cf59e640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 0x7f69bc0a5870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f69c000bfd0 tx=0x7f69c0005ed0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f69c8035000 tx=0x7f69c802f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cec65ca9496174 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69c57fa640 1 -- 172.21.15.77:0/1771260958 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8048070 con 0x5574c758a5d0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69c57fa640 1 -- 172.21.15.77:0/1771260958 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c802fb90 con 0x5574c758a5d0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.305+0000 7f69c57fa640 1 -- 172.21.15.77:0/1771260958 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8034040 con 0x5574c758a5d0 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- 172.21.15.77:0/1771260958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 msgr2=0x5574c758aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f69c8035000 tx=0x7f69c802f930 comp rx=0 tx=0).stop 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- 172.21.15.77:0/1771260958 shutdown_connections 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7597760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c758aa10 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1771260958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c758a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- 172.21.15.77:0/1771260958 >> 172.21.15.77:0/1771260958 conn(0x5574c7553310 msgr2=0x5574c7586c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- 172.21.15.77:0/1771260958 shutdown_connections 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- 172.21.15.77:0/1771260958 wait complete. 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 -- start start 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 Processor -- start 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 0x5574c7594d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.306+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c7595370 0x5574c75925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5574c7598010 con 0x5574c758bb40 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574c74ab060 con 0x5574c758a5d0 2024-09-15T22:37:11.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5574c7598190 con 0x5574c7595370 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 10 client.0 _ll_register_callbacks cb 0x5574c71fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56602/0 (socket says 172.21.15.77:56602) 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 -- 172.21.15.77:0/1179859031 learned_addr learned my addr 172.21.15.77:0/1179859031 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69cf59e640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c7595370 0x5574c75925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 -- 172.21.15.77:0/1179859031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c7595370 msgr2=0x5574c75925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c7595370 0x5574c75925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 0x5574c7594d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 -- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 msgr2=0x5574c7594d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 0x5574c7594d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 -- 172.21.15.77:0/1179859031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c72613c0 con 0x5574c758bb40 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 0x5574c7594d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:11.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69ce59c640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f69b809b140 tx=0x7f69b800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e675e317f14cab78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b801e070 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c802fb90 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f69b80cca20 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69b809bb40 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 10 client.21054 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574c75a3a90 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.307+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c802fb90 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f69b8097ab0 con 0x5574c758bb40 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 10 client.21054 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 10 client.21054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 20 client.21054 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:11.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c75a4f50 0x5574c75a7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 4 client.21054 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c74fd860 con 0x5574c75a4f50 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c75a4f50 0x5574c75a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69ced9d640 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c75a4f50 0x5574c75a7360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=250a168ede765bb server_cookie=e3d653af08ca796d in_seq=0 out_seq=0 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.308+0000 7f69b67fc640 10 client.21054 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69b67fc640 1 -- 172.21.15.77:0/1179859031 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c74fd860 con 0x5574c75a4f50 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69b67fc640 10 client.21054 handle_command_reply: tid=0 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 client.21054 shutdown 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 msgr2=0x5574c7592000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f69b809b140 tx=0x7f69b800c9e0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 shutdown_connections 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c75a4f50 0x5574c75a7360 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c7595370 0x5574c75925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c7592000 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69b67fc640 0 client.21054 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 --2- 172.21.15.77:0/1179859031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c758a5d0 0x5574c7594d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.309+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 >> 172.21.15.77:0/1179859031 conn(0x5574c7553310 msgr2=0x5574c758c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.310+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 shutdown_connections 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.310+0000 7f69d75ce740 1 -- 172.21.15.77:0/1179859031 wait complete. 2024-09-15T22:37:11.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.310+0000 7f69d75ce740 20 client.21054 trim_cache size 0 max 16384 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.322+0000 7f69d75ce740 1 Processor -- start 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.322+0000 7f69d75ce740 1 -- start start 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c759d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c759dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01af010 con 0x5574c758a5d0 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b0b50 con 0x5574c7595370 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69d75ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01af6e0 con 0x5574c758bb40 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56612/0 (socket says 172.21.15.77:56612) 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 -- 172.21.15.77:0/612173506 learned_addr learned my addr 172.21.15.77:0/612173506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:11.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69cf59e640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c759dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ce59c640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c759d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 -- 172.21.15.77:0/612173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 msgr2=0x5574c759d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c759d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 -- 172.21.15.77:0/612173506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 msgr2=0x5574c759dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c759dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 -- 172.21.15.77:0/612173506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c75a79e0 con 0x5574c758a5d0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69cf59e640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c759dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.323+0000 7f69ced9d640 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f69c8006010 tx=0x7f69c8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd69470d9fcd8387 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69c4ff9640 1 -- 172.21.15.77:0/612173506 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8048040 con 0x5574c758a5d0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69c4ff9640 1 -- 172.21.15.77:0/612173506 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c8003af0 con 0x5574c758a5d0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69c4ff9640 1 -- 172.21.15.77:0/612173506 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8034040 con 0x5574c758a5d0 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 -- 172.21.15.77:0/612173506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 msgr2=0x5574c7593f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f69c8006010 tx=0x7f69c8005fa0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 -- 172.21.15.77:0/612173506 shutdown_connections 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c759dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758bb40 0x5574c759d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 --2- 172.21.15.77:0/612173506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758a5d0 0x5574c7593f70 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 -- 172.21.15.77:0/612173506 >> 172.21.15.77:0/612173506 conn(0x5574c7553310 msgr2=0x5574c7593860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 -- 172.21.15.77:0/612173506 shutdown_connections 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.324+0000 7f69d75ce740 1 -- 172.21.15.77:0/612173506 wait complete. 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 -- start start 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 Processor -- start 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758a5d0 0x5574c7607da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7612c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b0480 con 0x5574c758bb40 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01afdb0 con 0x5574c7595370 2024-09-15T22:37:11.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01ad010 con 0x5574c758a5d0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69d75ce740 10 client.0 _ll_register_callbacks cb 0x7f69b80bb040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69ce59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56626/0 (socket says 172.21.15.77:56626) 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.325+0000 7f69ce59c640 1 -- 172.21.15.77:0/2042591646 learned_addr learned my addr 172.21.15.77:0/2042591646 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ced9d640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758a5d0 0x5574c7607da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 -- 172.21.15.77:0/2042591646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758a5d0 msgr2=0x5574c7607da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758a5d0 0x5574c7607da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 -- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 msgr2=0x5574c7612c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7612c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 -- 172.21.15.77:0/2042591646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c76132e0 con 0x5574c758bb40 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69ce59c640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f69b80ccda0 tx=0x7f69b80cf7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cca76422d0af861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b801e050 con 0x5574c758bb40 2024-09-15T22:37:11.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f69b8013170 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c8003af0 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f69b809ada0 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 10 client.21060 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5574c76145c0 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c8003af0 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f69b80b4240 con 0x5574c758bb40 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 10 client.21060 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 10 client.21060 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.326+0000 7f69d75ce740 20 client.21060 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c7614cb0 0x5574c76170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 4 client.21060 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574c74f7d70 con 0x5574c7614cb0 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69ced9d640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c7614cb0 0x5574c76170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69ced9d640 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c7614cb0 0x5574c76170f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26fa17d9be7fdca6 server_cookie=d31968d88602811b in_seq=0 out_seq=0 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69b67fc640 10 client.21060 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:37:11.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69b67fc640 1 -- 172.21.15.77:0/2042591646 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574c74f7d70 con 0x5574c7614cb0 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69b67fc640 10 client.21060 handle_command_reply: tid=0 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 1 client.21060 shutdown 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 msgr2=0x5574c76082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.327+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f69b80ccda0 tx=0x7f69b80cf7b0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 shutdown_connections 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x5574c7614cb0 0x5574c76170f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574c7595370 0x5574c7612c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5574c758bb40 0x5574c76082e0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 --2- 172.21.15.77:0/2042591646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5574c758a5d0 0x5574c7607da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 >> 172.21.15.77:0/2042591646 conn(0x5574c7553310 msgr2=0x5574c75a02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69b67fc640 0 client.21060 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 shutdown_connections 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 1 -- 172.21.15.77:0/2042591646 wait complete. 2024-09-15T22:37:11.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d75ce740 20 client.21060 trim_cache size 0 max 16384 2024-09-15T22:37:11.340 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:11.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 msgr2=0x7f69bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 0x7f69bc0a5870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f69c000bfd0 tx=0x7f69c0005ed0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 msgr2=0x7f69d0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b4a0 tx=0x7f69b800b960 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 shutdown_connections 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f69bc0a33b0 0x7f69bc0a5870 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69d0102ef0 0x7f69d019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f69d0102570 0x7f69d0198a20 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 --2- 172.21.15.77:0/3266349126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69d0100e20 0x7f69d01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 >> 172.21.15.77:0/3266349126 conn(0x7f69d00fc7d0 msgr2=0x7f69d0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 shutdown_connections 2024-09-15T22:37:11.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:11.328+0000 7f69d51f5640 1 -- 172.21.15.77:0/3266349126 wait complete. 2024-09-15T22:37:11.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.79 MB/sec execute 552 sec latency 12138.996 ms 2024-09-15T22:37:12.075 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:37:12.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:12 smithi077 ceph-mon[30980]: pgmap v783: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:37:12.359 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.362 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:12 smithi117 ceph-mon[35021]: pgmap v783: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:37:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:12 smithi196 ceph-mon[35109]: pgmap v783: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-15T22:37:12.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.77 MB/sec execute 553 sec latency 13139.049 ms 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 -- 172.21.15.77:0/1238409444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec102570 msgr2=0x7f61ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 --2- 172.21.15.77:0/1238409444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec102570 0x7f61ec1029b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009900 tx=0x7f61e802a480 comp rx=0 tx=0).stop 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 -- 172.21.15.77:0/1238409444 shutdown_connections 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 --2- 172.21.15.77:0/1238409444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 --2- 172.21.15.77:0/1238409444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec102570 0x7f61ec1029b0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 --2- 172.21.15.77:0/1238409444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec100e20 0x7f61ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 -- 172.21.15.77:0/1238409444 >> 172.21.15.77:0/1238409444 conn(0x7f61ec0fc7d0 msgr2=0x7f61ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 -- 172.21.15.77:0/1238409444 shutdown_connections 2024-09-15T22:37:12.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.849+0000 7f61f41ed640 1 -- 172.21.15.77:0/1238409444 wait complete. 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 Processor -- start 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 -- start start 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec102570 0x7f61ec1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec110f50 con 0x7f61ec100e20 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec110dd0 con 0x7f61ec102ef0 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f41ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1110d0 con 0x7f61ec102570 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56642/0 (socket says 172.21.15.77:56642) 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 -- 172.21.15.77:0/3551411184 learned_addr learned my addr 172.21.15.77:0/3551411184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f278a640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1788640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec102570 0x7f61ec1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 -- 172.21.15.77:0/3551411184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec102570 msgr2=0x7f61ec1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec102570 0x7f61ec1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 -- 172.21.15.77:0/3551411184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 msgr2=0x7f61ec19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f1f89640 1 -- 172.21.15.77:0/3551411184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec19c320 con 0x7f61ec100e20 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.850+0000 7f61f278a640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61f1f89640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c920 tx=0x7f61e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61deffd640 1 -- 172.21.15.77:0/3551411184 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0007c70 con 0x7f61ec100e20 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61deffd640 1 -- 172.21.15.77:0/3551411184 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0007e10 con 0x7f61ec100e20 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec19c610 con 0x7f61ec100e20 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61deffd640 1 -- 172.21.15.77:0/3551411184 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0018890 con 0x7f61ec100e20 2024-09-15T22:37:12.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.851+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a3ef0 con 0x7f61ec100e20 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 Processor -- start 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 -- start start 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61deffd640 1 -- 172.21.15.77:0/3551411184 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61e0018a70 con 0x7f61ec100e20 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435be05370 0x56435be07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56435bd602f0 con 0x56435bdfa5d0 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56435bd07cf0 con 0x56435be05370 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f65c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56435bd5c2b0 con 0x56435bdfbb40 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56646/0 (socket says 172.21.15.77:56646) 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.854+0000 7f61f1f89640 1 -- 172.21.15.77:0/2617980575 learned_addr learned my addr 172.21.15.77:0/2617980575 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f278a640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1788640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435be05370 0x56435be07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 -- 172.21.15.77:0/2617980575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 msgr2=0x56435bdfa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 -- 172.21.15.77:0/2617980575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435be05370 msgr2=0x56435be07760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435be05370 0x56435be07760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 -- 172.21.15.77:0/2617980575 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435bcec3a0 con 0x56435bdfa5d0 2024-09-15T22:37:12.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f278a640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61deffd640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f1f89640 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cd540 tx=0x7f61e00d85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f11502a372aa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61f278a640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61d7fff640 1 -- 172.21.15.77:0/2617980575 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0098400 con 0x56435bdfa5d0 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61deffd640 1 -- 172.21.15.77:0/3551411184 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f61e00d0270 con 0x7f61ec100e20 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61d7fff640 1 -- 172.21.15.77:0/2617980575 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e009c860 con 0x56435bdfa5d0 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.855+0000 7f61d7fff640 1 -- 172.21.15.77:0/2617980575 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00a4540 con 0x56435bdfa5d0 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 -- 172.21.15.77:0/2617980575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 msgr2=0x56435bdfaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cd540 tx=0x7f61e00d85a0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f278a640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f61e40095f0 tx=0x7f61e4009580 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 -- 172.21.15.77:0/2617980575 shutdown_connections 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435be05370 0x56435be07760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435bdfaa10 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2617980575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 -- 172.21.15.77:0/2617980575 >> 172.21.15.77:0/2617980575 conn(0x56435bdc3310 msgr2=0x56435bdf6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 -- 172.21.15.77:0/2617980575 shutdown_connections 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.856+0000 7f61f65c6740 1 -- 172.21.15.77:0/2617980575 wait complete. 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 -- start start 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 Processor -- start 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 0x56435be0d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be04240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56435be08070 con 0x56435bdfa5d0 2024-09-15T22:37:12.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56435bd1b060 con 0x56435bdfbb40 2024-09-15T22:37:12.906 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:12.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56435be081f0 con 0x56435be05370 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f65c6740 10 client.0 _ll_register_callbacks cb 0x56435ba6e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f1f89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56660/0 (socket says 172.21.15.77:56660) 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f1f89640 1 -- 172.21.15.77:0/1368843910 learned_addr learned my addr 172.21.15.77:0/1368843910 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f1788640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 0x56435be0d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f1f89640 1 -- 172.21.15.77:0/1368843910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 msgr2=0x56435be04240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.857+0000 7f61f278a640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be04240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1f89640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be04240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1f89640 1 -- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 msgr2=0x56435be0d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1f89640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 0x56435be0d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1f89640 1 -- 172.21.15.77:0/1368843910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435bad13c0 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1788640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 0x56435be0d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f1f89640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cd680 tx=0x7f61e00cd540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d6c5725920062b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00a4730 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f61e00a4ae0 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e009c860 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61e00a4940 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 10 client.21072 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56435be08070 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e009c860 con 0x56435bdfa5d0 2024-09-15T22:37:12.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f61e00987d0 con 0x56435bdfa5d0 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 10 client.21072 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 10 client.21072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.858+0000 7f61f65c6740 20 client.21072 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be16e60 0x56435be19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61f65c6740 4 client.21072 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56435bd6d860 con 0x56435be16e60 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61f1788640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be16e60 0x56435be19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61f1788640 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be16e60 0x56435be19250 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f278cc7e6793a2eb server_cookie=1d07cba56176edc5 in_seq=0 out_seq=0 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.859+0000 7f61d57fa640 10 client.21072 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61d57fa640 1 -- 172.21.15.77:0/1368843910 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56435bd6d860 con 0x56435be16e60 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61d57fa640 10 client.21072 handle_command_reply: tid=0 2024-09-15T22:37:12.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 client.21072 shutdown 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 msgr2=0x56435be0d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f61e00cd680 tx=0x7f61e00cd540 comp rx=0 tx=0).stop 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 shutdown_connections 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be16e60 0x56435be19250 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be04240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfbb40 0x56435be0d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 --2- 172.21.15.77:0/1368843910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfa5d0 0x56435be0d040 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61d57fa640 0 client.21072 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 >> 172.21.15.77:0/1368843910 conn(0x56435bdc3310 msgr2=0x56435bdfc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 shutdown_connections 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.860+0000 7f61f65c6740 1 -- 172.21.15.77:0/1368843910 wait complete. 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.861+0000 7f61f65c6740 20 client.21072 trim_cache size 0 max 16384 2024-09-15T22:37:12.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.873+0000 7f61f65c6740 1 Processor -- start 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.873+0000 7f61f65c6740 1 -- start start 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435bdf5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ace60 con 0x56435be05370 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ae9a0 con 0x56435bdfa5d0 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f65c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ad530 con 0x56435bdfbb40 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56668/0 (socket says 172.21.15.77:56668) 2024-09-15T22:37:12.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 -- 172.21.15.77:0/2821015108 learned_addr learned my addr 172.21.15.77:0/2821015108 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f1788640 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f1f89640 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435bdf5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 -- 172.21.15.77:0/2821015108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 msgr2=0x56435bdfd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 -- 172.21.15.77:0/2821015108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 msgr2=0x56435bdf5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435bdf5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 -- 172.21.15.77:0/2821015108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435be0d8f0 con 0x56435be05370 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61f278a640 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f61e401c500 tx=0x7f61e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ce6238b827400 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61d77fe640 1 -- 172.21.15.77:0/2821015108 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e402c070 con 0x56435be05370 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61d77fe640 1 -- 172.21.15.77:0/2821015108 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4027900 con 0x56435be05370 2024-09-15T22:37:12.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.874+0000 7f61d77fe640 1 -- 172.21.15.77:0/2821015108 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4010b40 con 0x56435be05370 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- 172.21.15.77:0/2821015108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 msgr2=0x56435bdfd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f61e401c500 tx=0x7f61e4015000 comp rx=0 tx=0).stop 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- 172.21.15.77:0/2821015108 shutdown_connections 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435be05370 0x56435bdfd660 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435bdfbb40 0x56435bdfd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 --2- 172.21.15.77:0/2821015108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435bdf5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- 172.21.15.77:0/2821015108 >> 172.21.15.77:0/2821015108 conn(0x56435bdc3310 msgr2=0x56435bdf54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- 172.21.15.77:0/2821015108 shutdown_connections 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- 172.21.15.77:0/2821015108 wait complete. 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 -- start start 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.875+0000 7f61f65c6740 1 Processor -- start 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435be11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be80920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ae2d0 con 0x56435bdfbb40 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1adc00 con 0x56435bdfa5d0 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1aae60 con 0x56435be05370 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f65c6740 10 client.0 _ll_register_callbacks cb 0x7f61e00d8360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56676/0 (socket says 172.21.15.77:56676) 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 -- 172.21.15.77:0/88783926 learned_addr learned my addr 172.21.15.77:0/88783926 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1f89640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435be11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f278a640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be80920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 -- 172.21.15.77:0/88783926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 msgr2=0x56435be80920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be80920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 -- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 msgr2=0x56435be11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435be11530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 -- 172.21.15.77:0/88783926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56435be80e60 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61f1788640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c00 tx=0x7f61e8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e1786eb3d108f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.876+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035040 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9526+0+0 (secure 0 0 0) 0x7f61e8005510 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e4027900 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f61e8052050 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 10 client.21078 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56435be82080 con 0x56435bdfbb40 2024-09-15T22:37:12.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4027900 con 0x56435bdfbb40 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f61e8003a00 con 0x56435bdfbb40 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 10 client.21078 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 10 client.21078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 20 client.21078 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be82800 0x56435be84c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 4 client.21078 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56435bd67d70 con 0x56435be82800 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.877+0000 7f61f1f89640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be82800 0x56435be84c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61d57fa640 10 client.21078 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f1f89640 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be82800 0x56435be84c20 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccde2dd8e9b3b50 server_cookie=32c67f1540697419 in_seq=0 out_seq=0 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61d57fa640 1 -- 172.21.15.77:0/88783926 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56435bd67d70 con 0x56435be82800 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61d57fa640 10 client.21078 handle_command_reply: tid=0 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 client.21078 shutdown 2024-09-15T22:37:12.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 msgr2=0x56435be11a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c00 tx=0x7f61e8004570 comp rx=0 tx=0).stop 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 shutdown_connections 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x56435be82800 0x56435be84c20 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56435be05370 0x56435be80920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56435bdfbb40 0x56435be11a70 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 --2- 172.21.15.77:0/88783926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56435bdfa5d0 0x56435be11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61d57fa640 0 client.21078 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 >> 172.21.15.77:0/88783926 conn(0x56435bdc3310 msgr2=0x56435bdf5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.878+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 shutdown_connections 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f65c6740 1 -- 172.21.15.77:0/88783926 wait complete. 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f65c6740 20 client.21078 trim_cache size 0 max 16384 2024-09-15T22:37:12.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 msgr2=0x7f61d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f61e40095f0 tx=0x7f61e4009580 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 msgr2=0x7f61ec197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c920 tx=0x7f61e000cde0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 shutdown_connections 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f61d80a3430 0x7f61d80a58f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61ec102ef0 0x7f61ec19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61ec102570 0x7f61ec1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 --2- 172.21.15.77:0/3551411184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61ec100e20 0x7f61ec197370 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 >> 172.21.15.77:0/3551411184 conn(0x7f61ec0fc7d0 msgr2=0x7f61ec106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 shutdown_connections 2024-09-15T22:37:12.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:12.879+0000 7f61f41ed640 1 -- 172.21.15.77:0/3551411184 wait complete. 2024-09-15T22:37:13.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.74 MB/sec execute 554 sec latency 14139.169 ms 2024-09-15T22:37:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:14 smithi077 ceph-mon[30980]: pgmap v784: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 139 KiB/s wr, 39 op/s 2024-09-15T22:37:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:14 smithi077 ceph-mon[30980]: osdmap e129: 12 total, 12 up, 12 in 2024-09-15T22:37:14.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:37:14.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:37:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:14 smithi196 ceph-mon[35109]: pgmap v784: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 139 KiB/s wr, 39 op/s 2024-09-15T22:37:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:14 smithi196 ceph-mon[35109]: osdmap e129: 12 total, 12 up, 12 in 2024-09-15T22:37:14.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:14 smithi117 ceph-mon[35021]: pgmap v784: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 139 KiB/s wr, 39 op/s 2024-09-15T22:37:14.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:14 smithi117 ceph-mon[35021]: osdmap e129: 12 total, 12 up, 12 in 2024-09-15T22:37:14.627 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.72 MB/sec execute 555 sec latency 15139.238 ms 2024-09-15T22:37:14.863 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.343+0000 7ffab361a640 1 -- 172.21.15.77:0/3144701074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac106a80 msgr2=0x7ffaac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.343+0000 7ffab361a640 1 --2- 172.21.15.77:0/3144701074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac106a80 0x7ffaac106e60 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009930 tx=0x7ffaa802a420 comp rx=0 tx=0).stop 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- 172.21.15.77:0/3144701074 shutdown_connections 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- 172.21.15.77:0/3144701074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac101420 0x7ffaac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- 172.21.15.77:0/3144701074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac100a80 0x7ffaac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- 172.21.15.77:0/3144701074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac106a80 0x7ffaac106e60 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- 172.21.15.77:0/3144701074 >> 172.21.15.77:0/3144701074 conn(0x7ffaac0fc7a0 msgr2=0x7ffaac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- 172.21.15.77:0/3144701074 shutdown_connections 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- 172.21.15.77:0/3144701074 wait complete. 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 Processor -- start 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- start start 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac100a80 0x7ffaac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac106a80 0x7ffaac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110e40 con 0x7ffaac101420 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110cc0 con 0x7ffaac100a80 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.344+0000 7ffab361a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110fc0 con 0x7ffaac106a80 2024-09-15T22:37:15.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56682/0 (socket says 172.21.15.77:56682) 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 -- 172.21.15.77:0/2490164415 learned_addr learned my addr 172.21.15.77:0/2490164415 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac106a80 0x7ffaac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab13b6640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac100a80 0x7ffaac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 -- 172.21.15.77:0/2490164415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac106a80 msgr2=0x7ffaac19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac106a80 0x7ffaac19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 -- 172.21.15.77:0/2490164415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac100a80 msgr2=0x7ffaac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac100a80 0x7ffaac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 -- 172.21.15.77:0/2490164415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19d400 con 0x7ffaac101420 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d950 tx=0x7ffa9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffaa27fc640 1 -- 172.21.15.77:0/2490164415 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c017030 con 0x7ffaac101420 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19d6f0 con 0x7ffaac101420 2024-09-15T22:37:15.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffaa27fc640 1 -- 172.21.15.77:0/2490164415 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c004020 con 0x7ffaac101420 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffaa27fc640 1 -- 172.21.15.77:0/2490164415 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0047a0 con 0x7ffaac101420 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.345+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1a4fd0 con 0x7ffaac101420 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffaa27fc640 1 -- 172.21.15.77:0/2490164415 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa9c0041d0 con 0x7ffaac101420 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 Processor -- start 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 -- start start 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee83d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee840130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55edee798280 con 0x55edee832b60 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55edee73fb10 con 0x55edee83dd40 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab59f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55edee794240 con 0x55edee833480 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab1bb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab1bb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56688/0 (socket says 172.21.15.77:56688) 2024-09-15T22:37:15.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffab1bb7640 1 -- 172.21.15.77:0/2716442268 learned_addr learned my addr 172.21.15.77:0/2716442268 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.348+0000 7ffaa27fc640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 0x7ffa980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab13b6640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee83d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee840130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 0x7ffa980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 -- 172.21.15.77:0/2716442268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 msgr2=0x55edee83d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee83d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffaa27fc640 1 -- 172.21.15.77:0/2490164415 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7ffa9c0cfb50 con 0x7ffaac101420 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 -- 172.21.15.77:0/2716442268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 msgr2=0x55edee840130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee840130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 -- 172.21.15.77:0/2716442268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edee74c1f0 con 0x55edee832b60 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee840130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40129f0 tx=0x7ffaa4012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c697351747e379d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffa977fe640 1 -- 172.21.15.77:0/2716442268 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4023070 con 0x55edee832b60 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffa977fe640 1 -- 172.21.15.77:0/2716442268 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4004710 con 0x55edee832b60 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.349+0000 7ffa977fe640 1 -- 172.21.15.77:0/2716442268 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa401fa70 con 0x55edee832b60 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- 172.21.15.77:0/2716442268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 msgr2=0x55edee832f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 0x7ffa980a5870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4022000 tx=0x7ffaa4021000 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40129f0 tx=0x7ffaa4012eb0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- 172.21.15.77:0/2716442268 shutdown_connections 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee840130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee83d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2716442268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee832f40 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- 172.21.15.77:0/2716442268 >> 172.21.15.77:0/2716442268 conn(0x55edee82f0f0 msgr2=0x55edee82f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- 172.21.15.77:0/2716442268 shutdown_connections 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- 172.21.15.77:0/2716442268 wait complete. 2024-09-15T22:37:15.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.350+0000 7ffab59f3740 1 -- start start 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 Processor -- start 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee83abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee83b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55edee840910 con 0x55edee833480 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55edee752f10 con 0x55edee83dd40 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55edee840a90 con 0x55edee832b60 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab59f3740 10 client.0 _ll_register_callbacks cb 0x55edee71ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab0bb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab0bb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56690/0 (socket says 172.21.15.77:56690) 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab0bb5640 1 -- 172.21.15.77:0/1342469681 learned_addr learned my addr 172.21.15.77:0/1342469681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab13b6640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee83abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.351+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee83b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 -- 172.21.15.77:0/1342469681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 msgr2=0x55edee83abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee83abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 -- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 msgr2=0x55edee83b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee83b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 -- 172.21.15.77:0/1342469681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edee5096a0 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee83b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c09b7a0 tx=0x7ffa9c06b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e562bcb3551de22b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0668f0 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7ffa9c086070 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa4004710 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa9c09c1a0 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab59f3740 10 client.21090 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55edee84c520 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4004710 con 0x55edee833480 2024-09-15T22:37:15.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.352+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffa9c0a4280 con 0x55edee833480 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 10 client.21090 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 10 client.21090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 20 client.21090 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee84db40 0x55edee84ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 4 client.21090 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edee7a57f0 con 0x55edee84db40 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab13b6640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee84db40 0x55edee84ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffab13b6640 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee84db40 0x55edee84ff70 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44637f83af047342 server_cookie=d71813b77f1006aa in_seq=0 out_seq=0 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.353+0000 7ffa94ff9640 10 client.21090 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffa94ff9640 1 -- 172.21.15.77:0/1342469681 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edee7a57f0 con 0x55edee84db40 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffa94ff9640 10 client.21090 handle_command_reply: tid=0 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 client.21090 shutdown 2024-09-15T22:37:15.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 msgr2=0x55edee83b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c09b7a0 tx=0x7ffa9c06b6c0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 shutdown_connections 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee84db40 0x55edee84ff70 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee83b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffa94ff9640 0 client.21090 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee833480 0x55edee83b100 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 --2- 172.21.15.77:0/1342469681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee83abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 >> 172.21.15.77:0/1342469681 conn(0x55edee82f0f0 msgr2=0x55edee82e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 shutdown_connections 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 1 -- 172.21.15.77:0/1342469681 wait complete. 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.354+0000 7ffab59f3740 20 client.21090 trim_cache size 0 max 16384 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 Processor -- start 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 -- start start 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee8556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee855bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1aefe0 con 0x55edee832b60 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b0b20 con 0x55edee83dd40 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab59f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1af6b0 con 0x55edee833480 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56702/0 (socket says 172.21.15.77:56702) 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 -- 172.21.15.77:0/2728579779 learned_addr learned my addr 172.21.15.77:0/2728579779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee8556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee855bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 -- 172.21.15.77:0/2728579779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 msgr2=0x55edee8556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee8556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 -- 172.21.15.77:0/2728579779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 msgr2=0x55edee855bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee855bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 -- 172.21.15.77:0/2728579779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edee83d1c0 con 0x55edee832b60 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee855bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.372+0000 7ffab13b6640 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005940 tx=0x7ffaa80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6dbcbad34e9cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffa96ffd640 1 -- 172.21.15.77:0/2728579779 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa804a070 con 0x55edee832b60 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffa96ffd640 1 -- 172.21.15.77:0/2728579779 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8005520 con 0x55edee832b60 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffa96ffd640 1 -- 172.21.15.77:0/2728579779 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8035040 con 0x55edee832b60 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 -- 172.21.15.77:0/2728579779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 msgr2=0x55edee8355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005940 tx=0x7ffaa80058b0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 -- 172.21.15.77:0/2728579779 shutdown_connections 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee83dd40 0x55edee855bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee833480 0x55edee8556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 --2- 172.21.15.77:0/2728579779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee832b60 0x55edee8355c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 -- 172.21.15.77:0/2728579779 >> 172.21.15.77:0/2728579779 conn(0x55edee82f0f0 msgr2=0x55edee839b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 -- 172.21.15.77:0/2728579779 shutdown_connections 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.373+0000 7ffab59f3740 1 -- 172.21.15.77:0/2728579779 wait complete. 2024-09-15T22:37:15.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 -- start start 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 Processor -- start 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee846000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee833480 0x55edee846540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b0450 con 0x55edee83dd40 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1afd80 con 0x55edee833480 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1acfe0 con 0x55edee832b60 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab59f3740 10 client.0 _ll_register_callbacks cb 0x7ffa9c097e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab1bb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab1bb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56706/0 (socket says 172.21.15.77:56706) 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab1bb7640 1 -- 172.21.15.77:0/4201471018 learned_addr learned my addr 172.21.15.77:0/4201471018 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.374+0000 7ffab0bb5640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee833480 0x55edee846540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab13b6640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee846000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 -- 172.21.15.77:0/4201471018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 msgr2=0x55edee846000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee846000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 -- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee833480 msgr2=0x55edee846540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee833480 0x55edee846540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 -- 172.21.15.77:0/4201471018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edee82a830 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab1bb7640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40094e0 tx=0x7ffaa40126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc19f76fa955ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffa94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4023070 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7ffaa4005250 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa8005520 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffaa401fa50 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab59f3740 10 client.21096 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55edee8332b0 con 0x55edee83dd40 2024-09-15T22:37:15.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.375+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8005520 con 0x55edee83dd40 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "8dc79d46-1fd6-4c3e-ba6d-7e38dbbc8145", 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7ffaa401e070 con 0x55edee83dd40 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 10 client.21096 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:15.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 10 client.21096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 20 client.21096 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee8b9590 0x55edee8bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 4 client.21096 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55edee79fd00 con 0x55edee8b9590 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab13b6640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee8b9590 0x55edee8bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffa94ff9640 10 client.21096 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.376+0000 7ffab13b6640 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee8b9590 0x55edee8bb9b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bfe24b204400c98 server_cookie=6ef1967531d516dd in_seq=0 out_seq=0 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.377+0000 7ffa94ff9640 1 -- 172.21.15.77:0/4201471018 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55edee79fd00 con 0x55edee8b9590 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.377+0000 7ffa94ff9640 10 client.21096 handle_command_reply: tid=0 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.377+0000 7ffab59f3740 1 client.21096 shutdown 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 msgr2=0x55edee842df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40094e0 tx=0x7ffaa40126d0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 shutdown_connections 2024-09-15T22:37:15.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55edee8b9590 0x55edee8bb9b0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55edee83dd40 0x55edee842df0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edee833480 0x55edee846540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 --2- 172.21.15.77:0/4201471018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55edee832b60 0x55edee846000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 >> 172.21.15.77:0/4201471018 conn(0x55edee82f0f0 msgr2=0x55edee839360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffa94ff9640 0 client.21096 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 shutdown_connections 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 1 -- 172.21.15.77:0/4201471018 wait complete. 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.378+0000 7ffab59f3740 20 client.21096 trim_cache size 0 max 16384 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 msgr2=0x7ffa980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 0x7ffa980a5870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4022000 tx=0x7ffaa4021000 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 msgr2=0x7ffaac198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d950 tx=0x7ffa9c00de10 comp rx=0 tx=0).stop 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 shutdown_connections 2024-09-15T22:37:15.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7ffa980a33b0 0x7ffa980a5870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaac106a80 0x7ffaac19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaac101420 0x7ffaac198990 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 --2- 172.21.15.77:0/2490164415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaac100a80 0x7ffaac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 >> 172.21.15.77:0/2490164415 conn(0x7ffaac0fc7a0 msgr2=0x7ffaac06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 shutdown_connections 2024-09-15T22:37:15.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:15.379+0000 7ffab361a640 1 -- 172.21.15.77:0/2490164415 wait complete. 2024-09-15T22:37:15.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.70 MB/sec execute 556 sec latency 16139.367 ms 2024-09-15T22:37:16.206 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:37:16.269 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: pgmap v786: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.270 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:16 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.491 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: pgmap v786: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-15T22:37:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:37:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:16 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: pgmap v786: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:16 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:16.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.67 MB/sec execute 557 sec latency 17139.461 ms 2024-09-15T22:37:17.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 -- 172.21.15.77:0/3263532211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf0102d30 msgr2=0x7f0cf0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3263532211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf0102d30 0x7f0cf0103110 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009cd0 tx=0x7f0ce002a910 comp rx=0 tx=0).stop 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 -- 172.21.15.77:0/3263532211 shutdown_connections 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3263532211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01082a0 0x7f0cf010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3263532211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf01036e0 0x7f0cf0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3263532211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf0102d30 0x7f0cf0103110 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.994+0000 7f0cf7384640 1 -- 172.21.15.77:0/3263532211 >> 172.21.15.77:0/3263532211 conn(0x7f0cf00fc7d0 msgr2=0x7f0cf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- 172.21.15.77:0/3263532211 shutdown_connections 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- 172.21.15.77:0/3263532211 wait complete. 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 Processor -- start 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- start start 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 0x7f0cf0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 0x7f0cf0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0112100 con 0x7f0cf01082a0 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0111f80 con 0x7f0cf0102d30 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf7384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0112280 con 0x7f0cf01036e0 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf5921640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf5921640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56726/0 (socket says 172.21.15.77:56726) 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.995+0000 7f0cf5921640 1 -- 172.21.15.77:0/3215399151 learned_addr learned my addr 172.21.15.77:0/3215399151 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 0x7f0cf0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 -- 172.21.15.77:0/3215399151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 msgr2=0x7f0cf0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 0x7f0cf0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 0x7f0cf0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 -- 172.21.15.77:0/3215399151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 msgr2=0x7f0cf0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 0x7f0cf0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 -- 172.21.15.77:0/3215399151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf01a6ae0 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 0x7f0cf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 0x7f0cf0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf5921640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c970 tx=0x7f0ce800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cee7fc640 1 -- 172.21.15.77:0/3215399151 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8007bb0 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cee7fc640 1 -- 172.21.15.77:0/3215399151 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8007d50 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf01a6dd0 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cee7fc640 1 -- 172.21.15.77:0/3215399151 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800f450 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.996+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01a72e0 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 Processor -- start 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cee7fc640 1 -- 172.21.15.77:0/3215399151 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ce8004f20 con 0x7f0cf01082a0 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 -- start start 2024-09-15T22:37:17.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d5b9e02e0 con 0x557d5ba7adc0 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d5b987ce0 con 0x557d5ba7a4a0 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:16.999+0000 7f0cf975d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d5b9dc2a0 con 0x557d5ba85680 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56728/0 (socket says 172.21.15.77:56728) 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 -- 172.21.15.77:0/361181396 learned_addr learned my addr 172.21.15.77:0/361181396 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf491f640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf491f640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 -- 172.21.15.77:0/361181396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 msgr2=0x557d5ba87a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba87a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 -- 172.21.15.77:0/361181396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 msgr2=0x557d5ba7a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba7a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 -- 172.21.15.77:0/361181396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5b96c440 con 0x557d5ba7adc0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf491f640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cee7fc640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 0x7f0cd80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf5120640 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009830 tx=0x7f0ce0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae7176ac7c945a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 0x7f0cd80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cee7fc640 1 -- 172.21.15.77:0/3215399151 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f0ce80cfde0 con 0x7f0cf01082a0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cd77fe640 1 -- 172.21.15.77:0/361181396 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003970 con 0x557d5ba7adc0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cd77fe640 1 -- 172.21.15.77:0/361181396 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce0036070 con 0x557d5ba7adc0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.000+0000 7f0cd77fe640 1 -- 172.21.15.77:0/361181396 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0004460 con 0x557d5ba7adc0 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- 172.21.15.77:0/361181396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 msgr2=0x557d5ba85140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009830 tx=0x7f0ce0009860 comp rx=0 tx=0).stop 2024-09-15T22:37:17.020 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 0x7f0cd80a5870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557d5ba77db0 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- 172.21.15.77:0/361181396 shutdown_connections 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba85140 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 --2- 172.21.15.77:0/361181396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- 172.21.15.77:0/361181396 >> 172.21.15.77:0/361181396 conn(0x557d5ba76a30 msgr2=0x557d5ba76e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- 172.21.15.77:0/361181396 shutdown_connections 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- 172.21.15.77:0/361181396 wait complete. 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.001+0000 7f0cf975d740 1 -- start start 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 Processor -- start 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba84fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba82790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557d5ba88250 con 0x557d5ba7adc0 2024-09-15T22:37:17.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557d5b99b050 con 0x557d5ba7a4a0 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557d5ba883d0 con 0x557d5ba85680 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf975d740 10 client.0 _ll_register_callbacks cb 0x557d5b9db160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56736/0 (socket says 172.21.15.77:56736) 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 -- 172.21.15.77:0/3305790517 learned_addr learned my addr 172.21.15.77:0/3305790517 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba84fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf5921640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba82790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 -- 172.21.15.77:0/3305790517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 msgr2=0x557d5ba82790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba82790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 -- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 msgr2=0x557d5ba84fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba84fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf491f640 1 -- 172.21.15.77:0/3305790517 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d5b7513c0 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.002+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba84fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf491f640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006c90 tx=0x7f0ce400c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666044c75b0767c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4023070 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce0036070 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f0ce401f800 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ce402f050 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 10 client.21108 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d5ba93c60 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce0036070 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0ce401e070 con 0x557d5ba7adc0 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 10 client.21108 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 10 client.21108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:17.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 20 client.21108 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5ba95110 0x557d5ba97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 4 client.21108 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.003+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d5b9ed850 con 0x557d5ba95110 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.004+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5ba95110 0x557d5ba97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.004+0000 7f0cd4ff9640 10 client.21108 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.004+0000 7f0cf5120640 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5ba95110 0x557d5ba97590 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5542045170d54c server_cookie=cf38c4a1f30b21e0 in_seq=0 out_seq=0 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/3305790517 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d5b9ed850 con 0x557d5ba95110 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cd4ff9640 10 client.21108 handle_command_reply: tid=0 2024-09-15T22:37:17.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 client.21108 shutdown 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 msgr2=0x557d5ba82250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006c90 tx=0x7f0ce400c190 comp rx=0 tx=0).stop 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 shutdown_connections 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5ba95110 0x557d5ba97590 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5ba82790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba82250 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 --2- 172.21.15.77:0/3305790517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba84fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 >> 172.21.15.77:0/3305790517 conn(0x557d5ba76a30 msgr2=0x557d5ba75b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cd4ff9640 0 client.21108 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 shutdown_connections 2024-09-15T22:37:17.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 1 -- 172.21.15.77:0/3305790517 wait complete. 2024-09-15T22:37:17.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.005+0000 7f0cf975d740 20 client.21108 trim_cache size 0 max 16384 2024-09-15T22:37:17.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.022+0000 7f0cf975d740 1 Processor -- start 2024-09-15T22:37:17.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.022+0000 7f0cf975d740 1 -- start start 2024-09-15T22:37:17.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.022+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba7a4a0 0x557d5ba914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.022+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.022+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba85680 0x557d5ba9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf975d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01b13b0 con 0x557d5ba7adc0 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf975d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01b2ef0 con 0x557d5ba85680 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf975d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01b1a80 con 0x557d5ba7a4a0 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56752/0 (socket says 172.21.15.77:56752) 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 -- 172.21.15.77:0/849140645 learned_addr learned my addr 172.21.15.77:0/849140645 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf5120640 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba7a4a0 0x557d5ba914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 -- 172.21.15.77:0/849140645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba7a4a0 msgr2=0x557d5ba914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba7a4a0 0x557d5ba914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 -- 172.21.15.77:0/849140645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba85680 msgr2=0x557d5ba9d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba85680 0x557d5ba9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 -- 172.21.15.77:0/849140645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5ba978f0 con 0x557d5ba7adc0 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cf491f640 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c160 tx=0x7f0ce400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aeb3d8a08a8c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cd6ffd640 1 -- 172.21.15.77:0/849140645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4023040 con 0x557d5ba7adc0 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cd6ffd640 1 -- 172.21.15.77:0/849140645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4019070 con 0x557d5ba7adc0 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.023+0000 7f0cd6ffd640 1 -- 172.21.15.77:0/849140645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400f040 con 0x557d5ba7adc0 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- 172.21.15.77:0/849140645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 msgr2=0x557d5ba9ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c160 tx=0x7f0ce400c850 comp rx=0 tx=0).stop 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- 172.21.15.77:0/849140645 shutdown_connections 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba85680 0x557d5ba9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba9ce60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 --2- 172.21.15.77:0/849140645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba7a4a0 0x557d5ba914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- 172.21.15.77:0/849140645 >> 172.21.15.77:0/849140645 conn(0x557d5ba76a30 msgr2=0x557d5ba81340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- 172.21.15.77:0/849140645 shutdown_connections 2024-09-15T22:37:17.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- 172.21.15.77:0/849140645 wait complete. 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 -- start start 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.024+0000 7f0cf975d740 1 Processor -- start 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba88db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5baf7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01b2820 con 0x557d5ba7adc0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01b2150 con 0x557d5ba7a4a0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf01af3b0 con 0x557d5ba85680 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf975d740 10 client.0 _ll_register_callbacks cb 0x7f0ce400c2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56760/0 (socket says 172.21.15.77:56760) 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 -- 172.21.15.77:0/4040848782 learned_addr learned my addr 172.21.15.77:0/4040848782 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf5120640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba88db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf5921640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5baf7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 -- 172.21.15.77:0/4040848782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 msgr2=0x557d5baf7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5baf7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 -- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 msgr2=0x557d5ba88db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba88db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 -- 172.21.15.77:0/4040848782 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d5baf8490 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf5921640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5baf7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cf491f640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401fb80 tx=0x7f0ce401f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a612d9f7a707d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.025+0000 7f0cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4040020 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f0ce4005bc0 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4019070 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ce404b050 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 10 client.21114 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d5baff870 con 0x557d5ba7adc0 2024-09-15T22:37:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4019070 con 0x557d5ba7adc0 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0ce401fce0 con 0x557d5ba7adc0 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 10 client.21114 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 10 client.21114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 20 client.21114 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5bafff00 0x557d5bb02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 4 client.21114 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.026+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d5b9e7d60 con 0x557d5bafff00 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf5120640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5bafff00 0x557d5bb02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf5120640 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5bafff00 0x557d5bb02320 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34fce5063a4d4108 server_cookie=9b33d405627c84d5 in_seq=0 out_seq=0 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cd4ff9640 10 client.21114 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cd4ff9640 1 -- 172.21.15.77:0/4040848782 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557d5b9e7d60 con 0x557d5bafff00 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cd4ff9640 10 client.21114 handle_command_reply: tid=0 2024-09-15T22:37:17.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 client.21114 shutdown 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 msgr2=0x557d5ba892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401fb80 tx=0x7f0ce401f600 comp rx=0 tx=0).stop 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 shutdown_connections 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x557d5bafff00 0x557d5bb02320 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557d5ba85680 0x557d5baf7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557d5ba7adc0 0x557d5ba892f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cd4ff9640 0 client.21114 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 --2- 172.21.15.77:0/4040848782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d5ba7a4a0 0x557d5ba88db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 >> 172.21.15.77:0/4040848782 conn(0x557d5ba76a30 msgr2=0x557d5ba83ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 shutdown_connections 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.027+0000 7f0cf975d740 1 -- 172.21.15.77:0/4040848782 wait complete. 2024-09-15T22:37:17.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf975d740 20 client.21114 trim_cache size 0 max 16384 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "8dc79d46-1fd6-4c3e-ba6d-7e38dbbc8145": { 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "8dc79d46-1fd6-4c3e-ba6d-7e38dbbc8145", 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:17.037 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:37:17.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 msgr2=0x7f0cd80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 0x7f0cd80a5870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557d5ba77db0 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-09-15T22:37:17.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 msgr2=0x7f0cf0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:17.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c970 tx=0x7f0ce800ce30 comp rx=0 tx=0).stop 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 shutdown_connections 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0cd80a33b0 0x7f0cd80a5870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cf01082a0 0x7f0cf0101410 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cf01036e0 0x7f0cf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 --2- 172.21.15.77:0/3215399151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf0102d30 0x7f0cf0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 >> 172.21.15.77:0/3215399151 conn(0x7f0cf00fc7d0 msgr2=0x7f0cf0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 shutdown_connections 2024-09-15T22:37:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:17.028+0000 7f0cf7384640 1 -- 172.21.15.77:0/3215399151 wait complete. 2024-09-15T22:37:17.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:37:17.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:17 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:37:17.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:17 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:37:17.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:17 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:37:17.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.65 MB/sec execute 558 sec latency 18139.550 ms 2024-09-15T22:37:17.776 INFO:tasks.cephfs.filesystem:scrub status for tag:8dc79d46-1fd6-4c3e-ba6d-7e38dbbc8145 - {'path': '/', 'tag': '8dc79d46-1fd6-4c3e-ba6d-7e38dbbc8145', 'options': 'recursive,force'} 2024-09-15T22:37:18.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:18 smithi077 ceph-mon[30980]: pgmap v787: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 112 KiB/s wr, 63 op/s 2024-09-15T22:37:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:18 smithi196 ceph-mon[35109]: pgmap v787: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 112 KiB/s wr, 63 op/s 2024-09-15T22:37:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:18 smithi117 ceph-mon[35021]: pgmap v787: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 112 KiB/s wr, 63 op/s 2024-09-15T22:37:18.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.62 MB/sec execute 559 sec latency 19139.633 ms 2024-09-15T22:37:19.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.60 MB/sec execute 560 sec latency 20139.677 ms 2024-09-15T22:37:20.406 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:20 smithi196 ceph-mon[35109]: pgmap v788: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 111 KiB/s wr, 65 op/s 2024-09-15T22:37:20.414 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:20 smithi117 ceph-mon[35021]: pgmap v788: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 111 KiB/s wr, 65 op/s 2024-09-15T22:37:20.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:20 smithi077 ceph-mon[30980]: pgmap v788: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 111 KiB/s wr, 65 op/s 2024-09-15T22:37:20.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.57 MB/sec execute 561 sec latency 21139.774 ms 2024-09-15T22:37:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:21 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:37:21.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:21 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:37:21.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:21 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:37:21.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:21 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:37:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:21 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:37:21.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:21 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:37:21.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.55 MB/sec execute 562 sec latency 22139.824 ms 2024-09-15T22:37:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:22 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:37:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:22 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:37:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:22 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:37:22.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:22 smithi077 ceph-mon[30980]: pgmap v789: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 110 KiB/s wr, 64 op/s 2024-09-15T22:37:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:22 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:37:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:22 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:37:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:22 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:37:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:22 smithi196 ceph-mon[35109]: pgmap v789: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 110 KiB/s wr, 64 op/s 2024-09-15T22:37:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:22 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:37:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:22 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:37:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:22 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:37:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:22 smithi117 ceph-mon[35021]: pgmap v789: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 110 KiB/s wr, 64 op/s 2024-09-15T22:37:22.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.53 MB/sec execute 563 sec latency 23139.910 ms 2024-09-15T22:37:23.466 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:23.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.50 MB/sec execute 564 sec latency 24139.961 ms 2024-09-15T22:37:23.728 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:24.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 -- 172.21.15.77:0/1648656216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 msgr2=0x7f4b541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:24.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 --2- 172.21.15.77:0/1648656216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b541021c0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009900 tx=0x7f4b5002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:24.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 -- 172.21.15.77:0/1648656216 shutdown_connections 2024-09-15T22:37:24.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 --2- 172.21.15.77:0/1648656216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54106f10 0x7f4b5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 --2- 172.21.15.77:0/1648656216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b54102790 0x7f4b541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 --2- 172.21.15.77:0/1648656216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b541021c0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 -- 172.21.15.77:0/1648656216 >> 172.21.15.77:0/1648656216 conn(0x7f4b540fd8c0 msgr2=0x7f4b540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 -- 172.21.15.77:0/1648656216 shutdown_connections 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.196+0000 7f4b5b070640 1 -- 172.21.15.77:0/1648656216 wait complete. 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 Processor -- start 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 -- start start 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54102790 0x7f4b541988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b54106f10 0x7f4b5419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:24.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b54111f70 con 0x7f4b54101de0 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b54111df0 con 0x7f4b54106f10 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b5b070640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b541120f0 con 0x7f4b54102790 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b58e0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b58e0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52078/0 (socket says 172.21.15.77:52078) 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.197+0000 7f4b58e0c640 1 -- 172.21.15.77:0/2126625224 learned_addr learned my addr 172.21.15.77:0/2126625224 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 -- 172.21.15.77:0/2126625224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54102790 msgr2=0x7f4b541988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b4bfff640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54102790 0x7f4b541988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54102790 0x7f4b541988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 -- 172.21.15.77:0/2126625224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b54106f10 msgr2=0x7f4b5419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b54106f10 0x7f4b5419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 -- 172.21.15.77:0/2126625224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5419d320 con 0x7f4b54101de0 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b58e0c640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:24.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5003f030 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b50003ce0 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5419d5b0 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50035070 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.198+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541a4ef0 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.199+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b541032c0 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.200+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4b50042030 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.201+0000 7f4b49ffb640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.201+0000 7f4b4bfff640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.201+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4b500f0670 con 0x7f4b54101de0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.202+0000 7f4b4bfff640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4b541088e0 tx=0x7f4b44005cc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:24.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.205+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b500bcd80 con 0x7f4b54101de0 2024-09-15T22:37:24.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.407+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b5410a0f0 con 0x7f4b54101de0 2024-09-15T22:37:24.410 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:24 smithi077 ceph-mon[30980]: pgmap v790: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 54 KiB/s wr, 62 op/s 2024-09-15T22:37:24.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.408+0000 7f4b49ffb640 1 -- 172.21.15.77:0/2126625224 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7f4b500bc680 con 0x7f4b54101de0 2024-09-15T22:37:24.411 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:37:24.411 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:37:24.412 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:37:24.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 msgr2=0x7f4b3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:24.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4b541088e0 tx=0x7f4b44005cc0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 msgr2=0x7f4b54198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b500042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 shutdown_connections 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4b3c0a33b0 0x7f4b3c0a5870 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b54106f10 0x7f4b5419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b54102790 0x7f4b541988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 --2- 172.21.15.77:0/2126625224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b54101de0 0x7f4b54198370 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:24.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 >> 172.21.15.77:0/2126625224 conn(0x7f4b540fd8c0 msgr2=0x7f4b540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:24.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 shutdown_connections 2024-09-15T22:37:24.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:24.411+0000 7f4b5b070640 1 -- 172.21.15.77:0/2126625224 wait complete. 2024-09-15T22:37:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:24 smithi196 ceph-mon[35109]: pgmap v790: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 54 KiB/s wr, 62 op/s 2024-09-15T22:37:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:24 smithi117 ceph-mon[35021]: pgmap v790: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 54 KiB/s wr, 62 op/s 2024-09-15T22:37:24.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.48 MB/sec execute 565 sec latency 25140.035 ms 2024-09-15T22:37:25.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --set-id 11c120d8 --release --await' 2024-09-15T22:37:25.211 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --set-id 11c120d8 --release --await 2024-09-15T22:37:25.455 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:25.576 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2126625224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:25.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2126625224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:25.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2126625224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:25.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185236 13.45 MB/sec execute 566 sec latency 26140.130 ms 2024-09-15T22:37:25.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 -- 172.21.15.77:0/3741352685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 msgr2=0x7f4f4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/3741352685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4810da40 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009ca0 tx=0x7f4f4002a950 comp rx=0 tx=0).stop 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 -- 172.21.15.77:0/3741352685 shutdown_connections 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/3741352685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4810da40 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/3741352685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 0x7f4f481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/3741352685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 -- 172.21.15.77:0/3741352685 >> 172.21.15.77:0/3741352685 conn(0x7f4f480fc7d0 msgr2=0x7f4f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 -- 172.21.15.77:0/3741352685 shutdown_connections 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.979+0000 7f4f4fc29640 1 -- 172.21.15.77:0/3741352685 wait complete. 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 Processor -- start 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 -- start start 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f481975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 0x7f4f48197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48111020 con 0x7f4f48102ef0 2024-09-15T22:37:25.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48110ea0 con 0x7f4f48100e20 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4fc29640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481111a0 con 0x7f4f48102570 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52094/0 (socket says 172.21.15.77:52094) 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 -- 172.21.15.77:0/1038237519 learned_addr learned my addr 172.21.15.77:0/1038237519 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f481975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 0x7f4f48197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 -- 172.21.15.77:0/1038237519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 msgr2=0x7f4f48197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 0x7f4f48197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 -- 172.21.15.77:0/1038237519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 msgr2=0x7f4f481975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f481975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.980+0000 7f4f4e1c6640 1 -- 172.21.15.77:0/1038237519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4819c590 con 0x7f4f48102ef0 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f481975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f4e1c6640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f4f40006010 tx=0x7f4f4002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f3effd640 1 -- 172.21.15.77:0/1038237519 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40034040 con 0x7f4f48102ef0 2024-09-15T22:37:25.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f3effd640 1 -- 172.21.15.77:0/1038237519 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f40003e00 con 0x7f4f48102ef0 2024-09-15T22:37:25.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4819c820 con 0x7f4f48102ef0 2024-09-15T22:37:25.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f3effd640 1 -- 172.21.15.77:0/1038237519 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40045480 con 0x7f4f48102ef0 2024-09-15T22:37:25.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.981+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f48069f70 con 0x7f4f48102ef0 2024-09-15T22:37:25.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.983+0000 7f4f3effd640 1 -- 172.21.15.77:0/1038237519 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f40037020 con 0x7f4f48102ef0 2024-09-15T22:37:25.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.983+0000 7f4f52002740 1 Processor -- start 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 -- start start 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47980 0x55b54af51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b54aeac620 con 0x55b54af52240 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b54ae53e50 con 0x55b54af47980 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f52002740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b54aea85e0 con 0x55b54af47060 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52104/0 (socket says 172.21.15.77:52104) 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/4061496809 learned_addr learned my addr 172.21.15.77:0/4061496809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f3effd640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 0x7f4f340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47980 0x55b54af51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4e1c6640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 0x7f4f340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/4061496809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 msgr2=0x55b54af47440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af47440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/4061496809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47980 msgr2=0x55b54af51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47980 0x55b54af51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/4061496809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54ae44d40 con 0x55b54af52240 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.984+0000 7f4f3effd640 1 -- 172.21.15.77:0/1038237519 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4f400f1150 con 0x7f4f48102ef0 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf60 tx=0x7f4f3800c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18ab7332ddbc73a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f4e1c6640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 0x7f4f340a58f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f400ee6d0 tx=0x7f4f400dd040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f33fff640 1 -- 172.21.15.77:0/4061496809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801d070 con 0x55b54af52240 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f33fff640 1 -- 172.21.15.77:0/4061496809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38018b50 con 0x55b54af52240 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f33fff640 1 -- 172.21.15.77:0/4061496809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800f040 con 0x55b54af52240 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f52002740 1 -- 172.21.15.77:0/4061496809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 msgr2=0x55b54af54630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.985+0000 7f4f52002740 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf60 tx=0x7f4f3800c170 comp rx=0 tx=0).stop 2024-09-15T22:37:25.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- 172.21.15.77:0/4061496809 shutdown_connections 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af54630 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47980 0x55b54af51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- 172.21.15.77:0/4061496809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af47440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- 172.21.15.77:0/4061496809 >> 172.21.15.77:0/4061496809 conn(0x55b54af435f0 msgr2=0x55b54af439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- 172.21.15.77:0/4061496809 shutdown_connections 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- 172.21.15.77:0/4061496809 wait complete. 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- start start 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 Processor -- start 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 0x55b54af4fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b54af54e10 con 0x55b54af47980 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b54ae34f10 con 0x55b54af52240 2024-09-15T22:37:25.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.986+0000 7f4f52002740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b54af54f90 con 0x55b54af47060 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f52002740 10 client.0 _ll_register_callbacks cb 0x55b54ae58db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52112/0 (socket says 172.21.15.77:52112) 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/1437325999 learned_addr learned my addr 172.21.15.77:0/1437325999 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 0x55b54af4fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/1437325999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 msgr2=0x55b54af4f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af4f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 msgr2=0x55b54af4fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 0x55b54af4fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/1437325999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54abbb580 con 0x55b54af47980 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 0x55b54af4fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400f530 tx=0x7f4f4400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b158e68041a925c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44013850 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4f440139f0 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f38018b50 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f44024050 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f52002740 10 client.21132 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.987+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b54ac85960 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38018b50 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4f440113e0 con 0x55b54af47980 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 10 client.21132 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 10 client.21132 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 20 client.21132 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54af61f30 0x55b54af64370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 4 client.21132 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:25.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b54aeb9b90 con 0x55b54af61f30 2024-09-15T22:37:25.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.988+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54af61f30 0x55b54af64370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.989+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54af61f30 0x55b54af64370 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ba0d6a74a8d850 server_cookie=a94cec8b4d4a3c07 in_seq=0 out_seq=0 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.989+0000 7f4f317fa640 10 client.21132 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.989+0000 7f4f317fa640 1 -- 172.21.15.77:0/1437325999 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b54aeb9b90 con 0x55b54af61f30 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.989+0000 7f4f317fa640 10 client.21132 handle_command_reply: tid=0 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 client.21132 shutdown 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 msgr2=0x55b54af4f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400f530 tx=0x7f4f4400f9f0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 shutdown_connections 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54af61f30 0x55b54af64370 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af52240 0x55b54af4fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af4f750 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 --2- 172.21.15.77:0/1437325999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47060 0x55b54af4f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f317fa640 0 client.21132 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 >> 172.21.15.77:0/1437325999 conn(0x55b54af435f0 msgr2=0x55b54af492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 shutdown_connections 2024-09-15T22:37:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 1 -- 172.21.15.77:0/1437325999 wait complete. 2024-09-15T22:37:25.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:25.990+0000 7f4f52002740 20 client.21132 trim_cache size 0 max 16384 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 Processor -- start 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 -- start start 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af4e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47980 0x55b54af69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481af040 con 0x55b54af52240 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481b0b80 con 0x55b54af47060 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f52002740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481af710 con 0x55b54af47980 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52122/0 (socket says 172.21.15.77:52122) 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/3331419042 learned_addr learned my addr 172.21.15.77:0/3331419042 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:26.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af4e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47980 0x55b54af69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/3331419042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47980 msgr2=0x55b54af69c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47980 0x55b54af69c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/3331419042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 msgr2=0x55b54af4e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af4e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 -- 172.21.15.77:0/3331419042 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54af46e90 con 0x55b54af52240 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.007+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf60 tx=0x7f4f3800c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540ca9a1467809ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f337fe640 1 -- 172.21.15.77:0/3331419042 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38033070 con 0x55b54af52240 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f337fe640 1 -- 172.21.15.77:0/3331419042 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38018950 con 0x55b54af52240 2024-09-15T22:37:26.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f337fe640 1 -- 172.21.15.77:0/3331419042 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3801d040 con 0x55b54af52240 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 -- 172.21.15.77:0/3331419042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 msgr2=0x55b54af6a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800cf60 tx=0x7f4f3800c170 comp rx=0 tx=0).stop 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 -- 172.21.15.77:0/3331419042 shutdown_connections 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af52240 0x55b54af6a180 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af47980 0x55b54af69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 --2- 172.21.15.77:0/3331419042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af4e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 -- 172.21.15.77:0/3331419042 >> 172.21.15.77:0/3331419042 conn(0x55b54af435f0 msgr2=0x55b54af4d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 -- 172.21.15.77:0/3331419042 shutdown_connections 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.008+0000 7f4f52002740 1 -- 172.21.15.77:0/3331419042 wait complete. 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 -- start start 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 Processor -- start 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af5a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af52240 0x55b54af57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481b04b0 con 0x55b54af47980 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481afde0 con 0x55b54af47060 2024-09-15T22:37:26.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481ad040 con 0x55b54af52240 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f52002740 10 client.0 _ll_register_callbacks cb 0x7f4f4400aed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f4d9c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f4d9c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52126/0 (socket says 172.21.15.77:52126) 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/612935619 learned_addr learned my addr 172.21.15.77:0/612935619 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.009+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af5a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/612935619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af52240 msgr2=0x55b54af57730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af52240 0x55b54af57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 msgr2=0x55b54af5a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af5a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 -- 172.21.15.77:0/612935619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54af57f60 con 0x55b54af47980 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af5a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f4d9c5640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4f44004810 tx=0x7f4f44011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0de3bf6d9410a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f317fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:26.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4402c070 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f38018950 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4f44005bd0 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4f44005dd0 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f52002740 10 client.21138 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b54afcdd00 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.010+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38018950 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4f44013cc0 con 0x55b54af47980 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 10 client.21138 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 10 client.21138 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 20 client.21138 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54afce450 0x55b54afd0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 4 client.21138 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "11c120d8", "release": true, "await": true}] 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "set_id": "11c120d8", "release": true, "await": true}) -- 0x55b54aeb40a0 con 0x55b54afce450 2024-09-15T22:37:26.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54afce450 0x55b54afd0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:26.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f4d1c4640 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54afce450 0x55b54afd0870 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54184069b7fa3980 server_cookie=6b4a3b387fc4be21 in_seq=0 out_seq=0 2024-09-15T22:37:26.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.011+0000 7f4f317fa640 10 client.21138 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f317fa640 1 -- 172.21.15.77:0/612935619 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b54aeb40a0 con 0x55b54afce450 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f317fa640 10 client.21138 handle_command_reply: tid=0 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 client.21138 shutdown 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 msgr2=0x55b54af571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f4f44004810 tx=0x7f4f44011690 comp rx=0 tx=0).stop 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 shutdown_connections 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55b54afce450 0x55b54afd0870 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b54af52240 0x55b54af57730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b54af47980 0x55b54af571b0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 --2- 172.21.15.77:0/612935619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b54af47060 0x55b54af5a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 >> 172.21.15.77:0/612935619 conn(0x55b54af435f0 msgr2=0x55b54af4e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f317fa640 0 client.21138 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 shutdown_connections 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.092+0000 7f4f52002740 1 -- 172.21.15.77:0/612935619 wait complete. 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f52002740 20 client.21138 trim_cache size 0 max 16384 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:37:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 88, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "11c120d8": { 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 88, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 147.2, 2024-09-15T22:37:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:37:26.098 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 msgr2=0x7f4f340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 0x7f4f340a58f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f400ee6d0 tx=0x7f4f400dd040 comp rx=0 tx=0).stop 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 msgr2=0x7f4f4819beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f4f40006010 tx=0x7f4f4002f6f0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 shutdown_connections 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f340a3430 0x7f4f340a58f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48102ef0 0x7f4f4819beb0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f48102570 0x7f4f48197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 --2- 172.21.15.77:0/1038237519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f48100e20 0x7f4f481975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:26.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 >> 172.21.15.77:0/1038237519 conn(0x7f4f480fc7d0 msgr2=0x7f4f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:26.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 shutdown_connections 2024-09-15T22:37:26.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:26.093+0000 7f4f4fc29640 1 -- 172.21.15.77:0/1038237519 wait complete. 2024-09-15T22:37:26.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:26 smithi077 ceph-mon[30980]: pgmap v791: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 51 KiB/s wr, 58 op/s 2024-09-15T22:37:26.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:26 smithi196 ceph-mon[35109]: pgmap v791: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 51 KiB/s wr, 58 op/s 2024-09-15T22:37:26.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:26 smithi117 ceph-mon[35021]: pgmap v791: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 51 KiB/s wr, 58 op/s 2024-09-15T22:37:26.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185468 13.46 MB/sec execute 567 sec latency 26563.970 ms 2024-09-15T22:37:26.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '11c120d8', total seconds elapsed: 30.7 2024-09-15T22:37:27.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 185835 13.48 MB/sec execute 568 sec latency 141.204 ms 2024-09-15T22:37:28.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:28 smithi077 ceph-mon[30980]: pgmap v792: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 74 KiB/s wr, 56 op/s 2024-09-15T22:37:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:28 smithi196 ceph-mon[35109]: pgmap v792: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 74 KiB/s wr, 56 op/s 2024-09-15T22:37:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:28 smithi117 ceph-mon[35021]: pgmap v792: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 74 KiB/s wr, 56 op/s 2024-09-15T22:37:28.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 186340 13.52 MB/sec execute 569 sec latency 145.758 ms 2024-09-15T22:37:29.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 186640 13.52 MB/sec execute 570 sec latency 132.330 ms 2024-09-15T22:37:30.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:30 smithi077 ceph-mon[30980]: pgmap v793: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-15T22:37:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:30 smithi196 ceph-mon[35109]: pgmap v793: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-15T22:37:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:30 smithi117 ceph-mon[35021]: pgmap v793: 129 pgs: 129 active+clean; 2.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-15T22:37:30.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 186993 13.53 MB/sec execute 571 sec latency 145.541 ms 2024-09-15T22:37:31.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 187404 13.54 MB/sec execute 572 sec latency 155.826 ms 2024-09-15T22:37:32.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:32 smithi077 ceph-mon[30980]: pgmap v794: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-15T22:37:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:32 smithi196 ceph-mon[35109]: pgmap v794: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-15T22:37:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:32 smithi117 ceph-mon[35021]: pgmap v794: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-09-15T22:37:32.628 INFO:tasks.workunit.client.0.smithi077.stdout: 10 187570 13.53 MB/sec execute 573 sec latency 744.304 ms 2024-09-15T22:37:33.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 187897 13.55 MB/sec execute 574 sec latency 748.816 ms 2024-09-15T22:37:34.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:34 smithi077 ceph-mon[30980]: pgmap v795: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-15T22:37:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:34 smithi196 ceph-mon[35109]: pgmap v795: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-15T22:37:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:34 smithi117 ceph-mon[35021]: pgmap v795: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-09-15T22:37:34.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 188174 13.55 MB/sec execute 575 sec latency 196.381 ms 2024-09-15T22:37:35.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 188429 13.55 MB/sec execute 576 sec latency 182.335 ms 2024-09-15T22:37:36.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:36 smithi077 ceph-mon[30980]: pgmap v796: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-15T22:37:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:36 smithi196 ceph-mon[35109]: pgmap v796: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-15T22:37:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:36 smithi117 ceph-mon[35021]: pgmap v796: 129 pgs: 129 active+clean; 1.8 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 143 op/s 2024-09-15T22:37:36.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 188791 13.55 MB/sec execute 577 sec latency 158.744 ms 2024-09-15T22:37:37.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 189054 13.55 MB/sec execute 578 sec latency 356.364 ms 2024-09-15T22:37:38.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:38 smithi077 ceph-mon[30980]: pgmap v797: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 311 op/s 2024-09-15T22:37:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:38 smithi196 ceph-mon[35109]: pgmap v797: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 311 op/s 2024-09-15T22:37:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:38 smithi117 ceph-mon[35021]: pgmap v797: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 311 op/s 2024-09-15T22:37:38.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 189471 13.56 MB/sec execute 579 sec latency 277.147 ms 2024-09-15T22:37:39.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 189874 13.58 MB/sec execute 580 sec latency 173.933 ms 2024-09-15T22:37:40.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:40 smithi077 ceph-mon[30980]: pgmap v798: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 322 op/s 2024-09-15T22:37:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:40 smithi196 ceph-mon[35109]: pgmap v798: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 322 op/s 2024-09-15T22:37:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:40 smithi117 ceph-mon[35021]: pgmap v798: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 322 op/s 2024-09-15T22:37:40.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 190168 13.58 MB/sec execute 581 sec latency 209.803 ms 2024-09-15T22:37:41.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 190505 13.59 MB/sec execute 582 sec latency 273.691 ms 2024-09-15T22:37:42.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:42 smithi077 ceph-mon[30980]: pgmap v799: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 375 op/s 2024-09-15T22:37:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:42 smithi196 ceph-mon[35109]: pgmap v799: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 375 op/s 2024-09-15T22:37:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:42 smithi117 ceph-mon[35021]: pgmap v799: 129 pgs: 129 active+clean; 1.9 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 375 op/s 2024-09-15T22:37:42.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 190824 13.59 MB/sec execute 583 sec latency 212.056 ms 2024-09-15T22:37:43.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 191250 13.61 MB/sec execute 584 sec latency 245.970 ms 2024-09-15T22:37:44.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:44 smithi077 ceph-mon[30980]: pgmap v800: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:37:44.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:44 smithi196 ceph-mon[35109]: pgmap v800: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:37:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:44 smithi117 ceph-mon[35021]: pgmap v800: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-15T22:37:44.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 191535 13.63 MB/sec execute 585 sec latency 358.284 ms 2024-09-15T22:37:45.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 191884 13.64 MB/sec execute 586 sec latency 169.937 ms 2024-09-15T22:37:46.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:46 smithi077 ceph-mon[30980]: pgmap v801: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:46 smithi196 ceph-mon[35109]: pgmap v801: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:46 smithi117 ceph-mon[35021]: pgmap v801: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:46.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 192176 13.64 MB/sec execute 587 sec latency 184.835 ms 2024-09-15T22:37:47.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 192508 13.66 MB/sec execute 588 sec latency 167.644 ms 2024-09-15T22:37:47.778 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:37:48.314 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:48.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:48 smithi077 ceph-mon[30980]: pgmap v802: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 501 op/s 2024-09-15T22:37:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:48 smithi196 ceph-mon[35109]: pgmap v802: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 501 op/s 2024-09-15T22:37:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:48 smithi117 ceph-mon[35021]: pgmap v802: 129 pgs: 129 active+clean; 1.9 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 501 op/s 2024-09-15T22:37:48.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 192675 13.64 MB/sec execute 589 sec latency 381.362 ms 2024-09-15T22:37:49.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 192955 13.64 MB/sec execute 590 sec latency 284.755 ms 2024-09-15T22:37:50.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:50 smithi077 ceph-mon[30980]: pgmap v803: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:50 smithi196 ceph-mon[35109]: pgmap v803: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:50 smithi117 ceph-mon[35021]: pgmap v803: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.4 MiB/s wr, 351 op/s 2024-09-15T22:37:50.630 INFO:tasks.workunit.client.0.smithi077.stdout: 10 193167 13.63 MB/sec execute 591 sec latency 275.175 ms 2024-09-15T22:37:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 -- 172.21.15.77:0/594112789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4103740 msgr2=0x7fa9e4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 --2- 172.21.15.77:0/594112789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4103740 0x7fa9e4107b80 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009930 tx=0x7fa9cc02a420 comp rx=0 tx=0).stop 2024-09-15T22:37:50.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 -- 172.21.15.77:0/594112789 shutdown_connections 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 --2- 172.21.15.77:0/594112789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4108290 0x7fa9e410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 --2- 172.21.15.77:0/594112789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4103740 0x7fa9e4107b80 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 --2- 172.21.15.77:0/594112789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4102d90 0x7fa9e4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 -- 172.21.15.77:0/594112789 >> 172.21.15.77:0/594112789 conn(0x7fa9e40fc7d0 msgr2=0x7fa9e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 -- 172.21.15.77:0/594112789 shutdown_connections 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.821+0000 7fa9e9668640 1 -- 172.21.15.77:0/594112789 wait complete. 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 Processor -- start 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 -- start start 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 0x7fa9e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4103740 0x7fa9e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4112090 con 0x7fa9e4108290 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4111f10 con 0x7fa9e4103740 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e9668640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4112210 con 0x7fa9e4102d90 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60446/0 (socket says 172.21.15.77:60446) 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1281896285 learned_addr learned my addr 172.21.15.77:0/1281896285 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4103740 0x7fa9e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.822+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 0x7fa9e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1281896285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 msgr2=0x7fa9e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 0x7fa9e41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1281896285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4103740 msgr2=0x7fa9e4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4103740 0x7fa9e4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1281896285 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e419d560 con 0x7fa9e4108290 2024-09-15T22:37:50.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 0x7fa9e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00eb10 tx=0x7fa9dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9dbfff640 1 -- 172.21.15.77:0/1281896285 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004b20 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9dbfff640 1 -- 172.21.15.77:0/1281896285 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc004cc0 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e419d850 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9dbfff640 1 -- 172.21.15.77:0/1281896285 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc010760 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.823+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e4100920 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.825+0000 7fa9dbfff640 1 -- 172.21.15.77:0/1281896285 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa9dc005020 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.826+0000 7fa9dbfff640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 0x7fa9d00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.826+0000 7fa9dbfff640 1 -- 172.21.15.77:0/1281896285 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fa9dc0d0660 con 0x7fa9e4108290 2024-09-15T22:37:50.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.826+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 0x7fa9d00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.827+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 0x7fa9d00a58f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005d00 tx=0x7fa9d400b410 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.827+0000 7fa9eba41740 1 Processor -- start 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.827+0000 7fa9eba41740 1 -- start start 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db5ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f92dab42e0 con 0x55f92db59680 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f92da5bce0 con 0x55f92db4edc0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9eba41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f92dab02a0 con 0x55f92db4e4a0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db5ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48910/0 (socket says 172.21.15.77:48910) 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:41880/0 (socket says 172.21.15.77:41880) 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 -- 172.21.15.77:0/1546660934 learned_addr learned my addr 172.21.15.77:0/1546660934 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 -- 172.21.15.77:0/1546660934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 msgr2=0x55f92db4e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 -- 172.21.15.77:0/1546660934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 msgr2=0x55f92db5ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db5ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 -- 172.21.15.77:0/1546660934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92da40440 con 0x55f92db4edc0 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009900 tx=0x7fa9cc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83bffc1553a6a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.828+0000 7fa9d97fa640 1 -- 172.21.15.77:0/1546660934 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc040070 con 0x55f92db4edc0 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db5ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9d97fa640 1 -- 172.21.15.77:0/1546660934 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc0318f0 con 0x55f92db4edc0 2024-09-15T22:37:50.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9d97fa640 1 -- 172.21.15.77:0/1546660934 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc035040 con 0x55f92db4edc0 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 -- 172.21.15.77:0/1546660934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 msgr2=0x55f92db59140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009900 tx=0x7fa9cc005c00 comp rx=0 tx=0).stop 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 -- 172.21.15.77:0/1546660934 shutdown_connections 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db5ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db59140 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1546660934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4e4a0 0x55f92db4e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.829+0000 7fa9eba41740 1 -- 172.21.15.77:0/1546660934 >> 172.21.15.77:0/1546660934 conn(0x55f92db4aa30 msgr2=0x55f92db4ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- 172.21.15.77:0/1546660934 shutdown_connections 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- 172.21.15.77:0/1546660934 wait complete. 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- start start 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 Processor -- start 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db56710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f92db5c250 con 0x55f92db59680 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f92da6f050 con 0x55f92db4e4a0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.830+0000 7fa9eba41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f92db5c3d0 con 0x55f92db4edc0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48926/0 (socket says 172.21.15.77:48926) 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.77:0/62270217 learned_addr learned my addr 172.21.15.77:0/62270217 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db56710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9eba41740 10 client.0 _ll_register_callbacks cb 0x55f92daaf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.77:0/62270217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 msgr2=0x55f92db561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.77:0/62270217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 msgr2=0x55f92db56710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db56710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 -- 172.21.15.77:0/62270217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92d8253c0 con 0x55f92db4e4a0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db56710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc0318f0 con 0x55f92db4e4a0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.831+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005de0 tx=0x7fa9d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1948b9fb869030 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.832+0000 7fa9ca7fc640 10 client.29756 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.832+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/62270217 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4028440 con 0x55f92db4e4a0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.832+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/62270217 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fa9d40285e0 con 0x55f92db4e4a0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.832+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/62270217 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9d4028820 con 0x55f92db4e4a0 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.832+0000 7fa9eba41740 10 client.29756 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:50.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55f92db67a70 con 0x55f92db4e4a0 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/62270217 <== mon.2 v2:172.21.15.117:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa9d4026070 con 0x55f92db4e4a0 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 10 client.29756 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 10 client.29756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 20 client.29756 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92db69310 0x55f92db6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 4 client.29756 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:50.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.833+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f92dac1850 con 0x55f92db69310 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.834+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92db69310 0x55f92db6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.837+0000 7fa9e27fc640 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92db69310 0x55f92db6b7b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f40ac39c0febc3 server_cookie=59f67a6f21e7d59c in_seq=0 out_seq=0 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.837+0000 7fa9ca7fc640 10 client.29756 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.838+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/62270217 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f92dac1850 con 0x55f92db69310 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.838+0000 7fa9ca7fc640 10 client.29756 handle_command_reply: tid=0 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 client.29756 shutdown 2024-09-15T22:37:50.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 msgr2=0x55f92db58f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005de0 tx=0x7fa9d4004730 comp rx=0 tx=0).stop 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 shutdown_connections 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92db69310 0x55f92db6b7b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92db56710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 --2- 172.21.15.77:0/62270217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4e4a0 0x55f92db58f20 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005de0 tx=0x7fa9d4004730 comp rx=0 tx=0).stop 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 >> 172.21.15.77:0/62270217 conn(0x55f92db4aa30 msgr2=0x55f92db50710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.839+0000 7fa9ca7fc640 0 client.29756 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:50.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.841+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 shutdown_connections 2024-09-15T22:37:50.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.842+0000 7fa9eba41740 1 -- 172.21.15.77:0/62270217 wait complete. 2024-09-15T22:37:50.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.842+0000 7fa9eba41740 20 client.29756 trim_cache size 0 max 16384 2024-09-15T22:37:50.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 Processor -- start 2024-09-15T22:37:50.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 -- start start 2024-09-15T22:37:50.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db4aa30 0x55f92db654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db54aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b1270 con 0x55f92db4aa30 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b2db0 con 0x55f92db59680 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.873+0000 7fa9eba41740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b1940 con 0x55f92db4edc0 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48942/0 (socket says 172.21.15.77:48942) 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 -- 172.21.15.77:0/4011991223 learned_addr learned my addr 172.21.15.77:0/4011991223 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db4aa30 0x55f92db654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 -- 172.21.15.77:0/4011991223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 msgr2=0x55f92db54aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db54aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 -- 172.21.15.77:0/4011991223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db4aa30 msgr2=0x55f92db654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db4aa30 0x55f92db654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.874+0000 7fa9e37fe640 1 -- 172.21.15.77:0/4011991223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92db55180 con 0x55f92db59680 2024-09-15T22:37:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.875+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0049d0 tx=0x7fa9dc09c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5111965b6f08eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.875+0000 7fa9d8ff9640 1 -- 172.21.15.77:0/4011991223 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0b4020 con 0x55f92db59680 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.875+0000 7fa9d8ff9640 1 -- 172.21.15.77:0/4011991223 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc0d0040 con 0x55f92db59680 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.875+0000 7fa9d8ff9640 1 -- 172.21.15.77:0/4011991223 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0cf070 con 0x55f92db59680 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- 172.21.15.77:0/4011991223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 msgr2=0x55f92db50600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0049d0 tx=0x7fa9dc09c440 comp rx=0 tx=0).stop 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- 172.21.15.77:0/4011991223 shutdown_connections 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db59680 0x55f92db50600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4edc0 0x55f92db54aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 --2- 172.21.15.77:0/4011991223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db4aa30 0x55f92db654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- 172.21.15.77:0/4011991223 >> 172.21.15.77:0/4011991223 conn(0x55f92db56870 msgr2=0x55f92db4fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- 172.21.15.77:0/4011991223 shutdown_connections 2024-09-15T22:37:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- 172.21.15.77:0/4011991223 wait complete. 2024-09-15T22:37:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.877+0000 7fa9eba41740 1 -- start start 2024-09-15T22:37:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.878+0000 7fa9eba41740 1 Processor -- start 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.878+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 0x55f92db5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.878+0000 7fa9eba41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 0x55f92db5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 0x55f92db5d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:41900/0 (socket says 172.21.15.77:41900) 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9e2ffd640 1 -- 172.21.15.77:0/1307549650 learned_addr learned my addr 172.21.15.77:0/1307549650 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92dbdb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b26e0 con 0x55f92db59680 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41b2010 con 0x55f92db4edc0 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41af270 con 0x55f92db4aa30 2024-09-15T22:37:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.879+0000 7fa9eba41740 10 client.0 _ll_register_callbacks cb 0x7fa9d400bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.880+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92dbdb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.881+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1307549650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 msgr2=0x55f92db5d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.881+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 0x55f92db5d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.881+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 msgr2=0x55f92db5dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.881+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.881+0000 7fa9e37fe640 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92dbdbde0 con 0x55f92db59680 2024-09-15T22:37:50.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.882+0000 7fa9e37fe640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92dbdb8a0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ce020 tx=0x7fa9dc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cb6b4318b3456 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.882+0000 7fa9ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:50.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.882+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0cf030 con 0x55f92db59680 2024-09-15T22:37:50.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.883+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fa9dc0d03f0 con 0x55f92db59680 2024-09-15T22:37:50.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.883+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9dc0d0040 con 0x55f92db59680 2024-09-15T22:37:50.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.884+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9dc0b4070 con 0x55f92db59680 2024-09-15T22:37:50.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.884+0000 7fa9eba41740 10 client.21156 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:50.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.884+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f92dbdcfd0 con 0x55f92db59680 2024-09-15T22:37:50.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc0a0960 con 0x55f92db59680 2024-09-15T22:37:50.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa9dc0b0ad0 con 0x55f92db59680 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 10 client.21156 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 10 client.21156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 20 client.21156 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92dbdd6a0 0x55f92dbdfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 4 client.21156 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.885+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa9dc0d0040 con 0x55f92dbdd6a0 2024-09-15T22:37:50.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.886+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92dbdd6a0 0x55f92dbdfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.887+0000 7fa9e2ffd640 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92dbdd6a0 0x55f92dbdfa90 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc5906580010229 server_cookie=374e7fe82825aebd in_seq=0 out_seq=0 2024-09-15T22:37:50.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.887+0000 7fa9ca7fc640 10 client.21156 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.893+0000 7fa9ca7fc640 1 -- 172.21.15.77:0/1307549650 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa9dc0d0040 con 0x55f92dbdd6a0 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.893+0000 7fa9ca7fc640 10 client.21156 handle_command_reply: tid=0 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.893+0000 7fa9eba41740 1 client.21156 shutdown 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 msgr2=0x55f92dbdb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92dbdb8a0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ce020 tx=0x7fa9dc009200 comp rx=0 tx=0).stop 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 shutdown_connections 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f92dbdd6a0 0x55f92dbdfa90 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f92db59680 0x55f92dbdb8a0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f92db4edc0 0x55f92db5dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 --2- 172.21.15.77:0/1307549650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f92db4aa30 0x55f92db5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 >> 172.21.15.77:0/1307549650 conn(0x55f92db56870 msgr2=0x55f92db49350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9ca7fc640 0 client.21156 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 shutdown_connections 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.894+0000 7fa9eba41740 1 -- 172.21.15.77:0/1307549650 wait complete. 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.895+0000 7fa9eba41740 20 client.21156 trim_cache size 0 max 16384 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.895+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 msgr2=0x7fa9d00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 0x7fa9d00a58f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005d00 tx=0x7fa9d400b410 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 msgr2=0x7fa9e419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00eb10 tx=0x7fa9dc00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 shutdown_connections 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa9d00a3430 0x7fa9d00a58f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9e4108290 0x7fa9e419ce80 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4103740 0x7fa9e4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 --2- 172.21.15.77:0/1281896285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9e4102d90 0x7fa9e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 >> 172.21.15.77:0/1281896285 conn(0x7fa9e40fc7d0 msgr2=0x7fa9e4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 shutdown_connections 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:50.896+0000 7fa9e9668640 1 -- 172.21.15.77:0/1281896285 wait complete. 2024-09-15T22:37:51.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 193492 13.63 MB/sec execute 592 sec latency 202.985 ms 2024-09-15T22:37:51.727 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:37:51.727 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:52.110 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:52.532 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:52 smithi077 ceph-mon[30980]: pgmap v804: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 383 op/s 2024-09-15T22:37:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:52 smithi196 ceph-mon[35109]: pgmap v804: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 383 op/s 2024-09-15T22:37:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:52 smithi117 ceph-mon[35021]: pgmap v804: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 383 op/s 2024-09-15T22:37:52.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 193909 13.65 MB/sec execute 593 sec latency 143.645 ms 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 -- 172.21.15.77:0/1038765631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 msgr2=0x7fbf04103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/1038765631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04103120 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402a480 comp rx=0 tx=0).stop 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 -- 172.21.15.77:0/1038765631 shutdown_connections 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/1038765631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 0x7fbf0410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/1038765631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 0x7fbf0410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/1038765631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04103120 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 -- 172.21.15.77:0/1038765631 >> 172.21.15.77:0/1038765631 conn(0x7fbf040fc7a0 msgr2=0x7fbf040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 -- 172.21.15.77:0/1038765631 shutdown_connections 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.626+0000 7fbf08ce0640 1 -- 172.21.15.77:0/1038765631 wait complete. 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 Processor -- start 2024-09-15T22:37:52.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 -- start start 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 0x7fbf04198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111f70 con 0x7fbf04102d40 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111df0 con 0x7fbf0410c170 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.627+0000 7fbf08ce0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041120f0 con 0x7fbf04103660 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60516/0 (socket says 172.21.15.77:60516) 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 -- 172.21.15.77:0/3213567757 learned_addr learned my addr 172.21.15.77:0/3213567757 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf01d9b640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 0x7fbf04198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf02d9d640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 -- 172.21.15.77:0/3213567757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 msgr2=0x7fbf04198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 0x7fbf04198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.642 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 -- 172.21.15.77:0/3213567757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 msgr2=0x7fbf0419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 -- 172.21.15.77:0/3213567757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419d400 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf0259c640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef403f030 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef4004440 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419d690 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4035070 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.628+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a4fd0 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.629+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0406b070 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.631+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbef4042030 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.631+0000 7fbef37fe640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 0x7fbeec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.631+0000 7fbf01d9b640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 0x7fbeec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.631+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fbef40f03f0 con 0x7fbf04102d40 2024-09-15T22:37:52.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.632+0000 7fbf01d9b640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 0x7fbeec0a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbf040feda0 tx=0x7fbef8005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.637+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbef40bc3f0 con 0x7fbf04102d40 2024-09-15T22:37:52.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.921+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf041071e0 con 0x7fbf04102d40 2024-09-15T22:37:52.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.921+0000 7fbef37fe640 1 -- 172.21.15.77:0/3213567757 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11989 (secure 0 0 0) 0x7fbef40c06f0 con 0x7fbf04102d40 2024-09-15T22:37:52.923 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:37:52.923 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"btime":"2024-09-15T22:28:31:293853+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:28:30.441480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:37:52.923 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-09-15T22:37:52.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 msgr2=0x7fbeec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 0x7fbeec0a5870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbf040feda0 tx=0x7fbef8005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 msgr2=0x7fbf04198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef4004040 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 shutdown_connections 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fbeec0a33b0 0x7fbeec0a5870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf04103660 0x7fbf04198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 --2- 172.21.15.77:0/3213567757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf04102d40 0x7fbf04198450 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 >> 172.21.15.77:0/3213567757 conn(0x7fbf040fc7a0 msgr2=0x7fbf040fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 shutdown_connections 2024-09-15T22:37:52.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:52.924+0000 7fbf08ce0640 1 -- 172.21.15.77:0/3213567757 wait complete. 2024-09-15T22:37:53.487 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:53 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3213567757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:53 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3213567757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:53 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3213567757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 194269 13.66 MB/sec execute 594 sec latency 148.644 ms 2024-09-15T22:37:53.847 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:37:54.191 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:54.476 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:54 smithi077 ceph-mon[30980]: pgmap v805: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 427 op/s 2024-09-15T22:37:54.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:54 smithi196 ceph-mon[35109]: pgmap v805: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 427 op/s 2024-09-15T22:37:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:54 smithi117 ceph-mon[35021]: pgmap v805: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 427 op/s 2024-09-15T22:37:54.629 INFO:tasks.workunit.client.0.smithi077.stdout: 10 194647 13.67 MB/sec execute 595 sec latency 158.895 ms 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 -- 172.21.15.77:0/3653048854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 msgr2=0x7f9300107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 --2- 172.21.15.77:0/3653048854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300107b80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009930 tx=0x7f92f402a420 comp rx=0 tx=0).stop 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 -- 172.21.15.77:0/3653048854 shutdown_connections 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 --2- 172.21.15.77:0/3653048854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 0x7f930010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 --2- 172.21.15.77:0/3653048854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300107b80 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 --2- 172.21.15.77:0/3653048854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f9300103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.746+0000 7f93048ff640 1 -- 172.21.15.77:0/3653048854 >> 172.21.15.77:0/3653048854 conn(0x7f93000fc7d0 msgr2=0x7f93000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.752 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.747+0000 7f93048ff640 1 -- 172.21.15.77:0/3653048854 shutdown_connections 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.747+0000 7f93048ff640 1 -- 172.21.15.77:0/3653048854 wait complete. 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.747+0000 7f93048ff640 1 Processor -- start 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.747+0000 7f93048ff640 1 -- start start 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 0x7f9300101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9300112090 con 0x7f9300103740 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9300111f10 con 0x7f9300108290 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f93048ff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9300112210 con 0x7f9300102d90 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:41940/0 (socket says 172.21.15.77:41940) 2024-09-15T22:37:54.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60544/0 (socket says 172.21.15.77:60544) 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 -- 172.21.15.77:0/640907069 learned_addr learned my addr 172.21.15.77:0/640907069 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fed9d640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 0x7f9300101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 -- 172.21.15.77:0/640907069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 msgr2=0x7f9300101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 0x7f9300101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 -- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 msgr2=0x7f9300100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fe59c640 1 -- 172.21.15.77:0/640907069 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93001a6ad0 con 0x7f9300102d90 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.748+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f92fe59c640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f92f000eb90 tx=0x7f92f000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f92ef7fe640 1 -- 172.21.15.77:0/640907069 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0004d20 con 0x7f9300102d90 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93001a6dc0 con 0x7f9300102d90 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f92ef7fe640 1 -- 172.21.15.77:0/640907069 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f000ee90 con 0x7f9300102d90 2024-09-15T22:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f92ef7fe640 1 -- 172.21.15.77:0/640907069 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0010830 con 0x7f9300102d90 2024-09-15T22:37:54.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.749+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93001a7220 con 0x7f9300102d90 2024-09-15T22:37:54.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.753+0000 7f92ef7fe640 1 -- 172.21.15.77:0/640907069 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f92f0005020 con 0x7f9300102d90 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.753+0000 7f92ef7fe640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 0x7f92e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 Processor -- start 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f92ef7fe640 1 -- 172.21.15.77:0/640907069 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f92f00d06e0 con 0x7f9300102d90 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 0x7f92e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 -- start start 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b376692a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b37669d370 0x55b37669f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b3765f82f0 con 0x55b37669d370 2024-09-15T22:37:54.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b37659fcf0 con 0x55b376693b40 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f9306cd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3765f42b0 con 0x55b3766925d0 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 0x7f92e80a5870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f92f4004770 tx=0x7f92f4031400 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.754+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b376692a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b376692a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:41954/0 (socket says 172.21.15.77:41954) 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fdd9b640 1 -- 172.21.15.77:0/2302023275 learned_addr learned my addr 172.21.15.77:0/2302023275 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:48998/0 (socket says 172.21.15.77:48998) 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fed9d640 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b37669d370 0x55b37669f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 -- 172.21.15.77:0/2302023275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 msgr2=0x55b376692a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b376692a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 -- 172.21.15.77:0/2302023275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b37669d370 msgr2=0x55b37669f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b37669d370 0x55b37669f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 -- 172.21.15.77:0/2302023275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3765843a0 con 0x55b376693b40 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f92f00cdaa0 tx=0x7f92f009c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495d00694c192b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92ecff9640 1 -- 172.21.15.77:0/2302023275 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f00b4020 con 0x55b376693b40 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92ecff9640 1 -- 172.21.15.77:0/2302023275 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f009c630 con 0x55b376693b40 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.755+0000 7f92ecff9640 1 -- 172.21.15.77:0/2302023275 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0098ca0 con 0x55b376693b40 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 -- 172.21.15.77:0/2302023275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 msgr2=0x55b376692090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f92f00cdaa0 tx=0x7f92f009c4c0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 -- 172.21.15.77:0/2302023275 shutdown_connections 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b37669d370 0x55b37669f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b376692a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2302023275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b376693b40 0x55b376692090 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 -- 172.21.15.77:0/2302023275 >> 172.21.15.77:0/2302023275 conn(0x55b37665b310 msgr2=0x55b37668ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 -- 172.21.15.77:0/2302023275 shutdown_connections 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.756+0000 7f9306cd8740 1 -- 172.21.15.77:0/2302023275 wait complete. 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 -- start start 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 Processor -- start 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b37669cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60550/0 (socket says 172.21.15.77:60550) 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f92fdd9b640 1 -- 172.21.15.77:0/2400034874 learned_addr learned my addr 172.21.15.77:0/2400034874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b37669a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b37669cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b3766a00c0 con 0x55b376693b40 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3765b3060 con 0x55b37669d370 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b3766a0240 con 0x55b3766925d0 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.757+0000 7f9306cd8740 10 client.0 _ll_register_callbacks cb 0x55b3763068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fed9d640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b37669a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 -- 172.21.15.77:0/2400034874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 msgr2=0x55b37669cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b37669cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 -- 172.21.15.77:0/2400034874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 msgr2=0x55b37669a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b37669a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3763693c0 con 0x55b376693b40 2024-09-15T22:37:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b37669cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f92f4031470 tx=0x7f92f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd8b32970536b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.758+0000 7f92e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4035040 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f009c630 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f92f4003b40 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f92f4003d40 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f9306cd8740 10 client.21174 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b3766abbb0 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f009c630 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f92f404a5d0 con 0x55b376693b40 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f9306cd8740 10 client.21174 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:54.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.759+0000 7f9306cd8740 10 client.21174 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f9306cd8740 20 client.21174 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b3766af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f9306cd8740 4 client.21174 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b376605860 con 0x55b3766acfc0 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b3766af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f92e5ffb640 10 client.21174 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.760+0000 7f92fe59c640 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b3766af440 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64179a5382cd058d server_cookie=da64960e99e07dd8 in_seq=0 out_seq=0 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f92e5ffb640 1 -- 172.21.15.77:0/2400034874 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b376605860 con 0x55b3766acfc0 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f92e5ffb640 10 client.21174 handle_command_reply: tid=0 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f9306cd8740 1 client.21174 shutdown 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 msgr2=0x55b37669a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f92f4031470 tx=0x7f92f4009260 comp rx=0 tx=0).stop 2024-09-15T22:37:54.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 shutdown_connections 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.761+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b3766af440 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b37669a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f92e5ffb640 0 client.21174 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b37669a010 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 --2- 172.21.15.77:0/2400034874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b37669cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 >> 172.21.15.77:0/2400034874 conn(0x55b37665b310 msgr2=0x55b3766949c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 shutdown_connections 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 1 -- 172.21.15.77:0/2400034874 wait complete. 2024-09-15T22:37:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.762+0000 7f9306cd8740 20 client.21174 trim_cache size 0 max 16384 2024-09-15T22:37:54.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.780+0000 7f9306cd8740 1 Processor -- start 2024-09-15T22:37:54.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 -- start start 2024-09-15T22:37:54.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b3766a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b3766b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b1260 con 0x55b376693b40 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b3766a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f92fdd9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60562/0 (socket says 172.21.15.77:60562) 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f92fdd9b640 1 -- 172.21.15.77:0/3585463319 learned_addr learned my addr 172.21.15.77:0/3585463319 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b2da0 con 0x55b37669d370 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f92fed9d640 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b3766b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.781+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b1930 con 0x55b3766925d0 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 -- 172.21.15.77:0/3585463319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 msgr2=0x55b3766a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b3766a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 -- 172.21.15.77:0/3585463319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 msgr2=0x55b3766b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b3766b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 -- 172.21.15.77:0/3585463319 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37668f960 con 0x55b376693b40 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f92f404abb0 tx=0x7f92f402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530057502ac888dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92e7fff640 1 -- 172.21.15.77:0/3585463319 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4035070 con 0x55b376693b40 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92e7fff640 1 -- 172.21.15.77:0/3585463319 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f4003940 con 0x55b376693b40 2024-09-15T22:37:54.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.782+0000 7f92e7fff640 1 -- 172.21.15.77:0/3585463319 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f403ccd0 con 0x55b376693b40 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.783+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 msgr2=0x55b3766b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.783+0000 7f9306cd8740 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f92f404abb0 tx=0x7f92f402a420 comp rx=0 tx=0).stop 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 shutdown_connections 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b37669d370 0x55b3766b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766b4d10 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 --2- 172.21.15.77:0/3585463319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3766925d0 0x55b3766a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 >> 172.21.15.77:0/3585463319 conn(0x55b37665b310 msgr2=0x55b3766988e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 shutdown_connections 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 -- 172.21.15.77:0/3585463319 wait complete. 2024-09-15T22:37:54.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 -- start start 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.784+0000 7f9306cd8740 1 Processor -- start 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b37669d370 0x55b376716ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b26d0 con 0x55b376693b40 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b2000 con 0x55b3766925d0 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93001af260 con 0x55b37669d370 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f9306cd8740 10 client.0 _ll_register_callbacks cb 0x7f92f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49030/0 (socket says 172.21.15.77:49030) 2024-09-15T22:37:54.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 -- 172.21.15.77:0/1520408796 learned_addr learned my addr 172.21.15.77:0/1520408796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fed9d640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b37669d370 0x55b376716ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 -- 172.21.15.77:0/1520408796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b37669d370 msgr2=0x55b376716ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b37669d370 0x55b376716ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 -- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 msgr2=0x55b3766a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.785+0000 7f92fe59c640 1 -- 172.21.15.77:0/1520408796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b376695120 con 0x55b3766925d0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f92fe59c640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f92f0005800 tx=0x7f92f0005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2333f323c51e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f92e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f00b4030 con 0x55b3766925d0 2024-09-15T22:37:54.791 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f92f009a930 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f4003940 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.786+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f92f009bf90 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 10 client.29783 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b376718180 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f4003940 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f92f0098ca0 con 0x55b3766925d0 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 10 client.29783 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 10 client.29783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 20 client.29783 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b37671a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 4 client.29783 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:54.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.787+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3765ffd70 con 0x55b3766acfc0 2024-09-15T22:37:54.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.788+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b37671a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.788+0000 7f92fdd9b640 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b37671a190 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78b89108d7934bad server_cookie=4c8ac57c232cdef4 in_seq=0 out_seq=0 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.788+0000 7f92e5ffb640 10 client.29783 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f92e5ffb640 1 -- 172.21.15.77:0/1520408796 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3765ffd70 con 0x55b3766acfc0 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f92e5ffb640 10 client.29783 handle_command_reply: tid=0 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 client.29783 shutdown 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 msgr2=0x55b3766a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f92f0005800 tx=0x7f92f0005960 comp rx=0 tx=0).stop 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 shutdown_connections 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x55b3766acfc0 0x55b37671a190 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b37669d370 0x55b376716ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.795 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:54.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f92e5ffb640 0 client.29783 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:37:54.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b376693b40 0x55b3766a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 --2- 172.21.15.77:0/1520408796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3766925d0 0x55b3766a1630 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 >> 172.21.15.77:0/1520408796 conn(0x55b37665b310 msgr2=0x55b376694930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 shutdown_connections 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 1 -- 172.21.15.77:0/1520408796 wait complete. 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.790+0000 7f9306cd8740 20 client.29783 trim_cache size 0 max 16384 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 msgr2=0x7f92e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 0x7f92e80a5870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f92f4004770 tx=0x7f92f4031400 comp rx=0 tx=0).stop 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 msgr2=0x7f93001009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f92f000eb90 tx=0x7f92f000c6a0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 shutdown_connections 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f92e80a33b0 0x7f92e80a5870 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9300108290 0x7f9300101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9300103740 0x7f9300100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 --2- 172.21.15.77:0/640907069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9300102d90 0x7f93001009f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 >> 172.21.15.77:0/640907069 conn(0x7f93000fc7d0 msgr2=0x7f9300105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 shutdown_connections 2024-09-15T22:37:54.797 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:54.791+0000 7f93048ff640 1 -- 172.21.15.77:0/640907069 wait complete. 2024-09-15T22:37:55.630 INFO:tasks.workunit.client.0.smithi077.stdout: 10 195021 13.67 MB/sec execute 596 sec latency 138.449 ms 2024-09-15T22:37:55.744 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:37:56.072 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:56.484 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:56 smithi077 ceph-mon[30980]: pgmap v806: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 313 op/s 2024-09-15T22:37:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:56 smithi196 ceph-mon[35109]: pgmap v806: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 313 op/s 2024-09-15T22:37:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:56 smithi117 ceph-mon[35021]: pgmap v806: 129 pgs: 129 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.2 MiB/s wr, 313 op/s 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 -- 172.21.15.77:0/302176618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b0110190 msgr2=0x7f63b006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 --2- 172.21.15.77:0/302176618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b0110190 0x7f63b006e4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009be0 tx=0x7f63a802a6f0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 -- 172.21.15.77:0/302176618 shutdown_connections 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 --2- 172.21.15.77:0/302176618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b011b140 0x7f63b011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 --2- 172.21.15.77:0/302176618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 --2- 172.21.15.77:0/302176618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b0110190 0x7f63b006e4f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.622+0000 7f63af59e640 1 -- 172.21.15.77:0/302176618 >> 172.21.15.77:0/302176618 conn(0x7f63b0073180 msgr2=0x7f63b0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.623+0000 7f63af59e640 1 -- 172.21.15.77:0/302176618 shutdown_connections 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.623+0000 7f63af59e640 1 -- 172.21.15.77:0/302176618 wait complete. 2024-09-15T22:37:56.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.624+0000 7f63af59e640 1 Processor -- start 2024-09-15T22:37:56.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.624+0000 7f63af59e640 1 -- start start 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b0110190 0x7f63b01149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 0x7f63b0114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:41992/0 (socket says 172.21.15.77:41992) 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63add9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b0110190 0x7f63b01149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63ae59c640 1 -- 172.21.15.77:0/2831620790 learned_addr learned my addr 172.21.15.77:0/2831620790 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111370 con 0x7f63b0110190 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01111f0 con 0x7f63b011b140 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.625+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01114f0 con 0x7f63b006ea30 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 0x7f63b0114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 -- 172.21.15.77:0/2831620790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 msgr2=0x7f63b0114f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 0x7f63b0114f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 -- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b0110190 msgr2=0x7f63b01149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b0110190 0x7f63b01149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01157c0 con 0x7f63b006ea30 2024-09-15T22:37:56.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.626+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63a8034040 tx=0x7f63a8030da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.628+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 0x7f63b0114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.628+0000 7f639f7fe640 1 -- 172.21.15.77:0/2831620790 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8041020 con 0x7f63b006ea30 2024-09-15T22:37:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.628+0000 7f639f7fe640 1 -- 172.21.15.77:0/2831620790 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8003e50 con 0x7f63b006ea30 2024-09-15T22:37:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.628+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01bbf50 con 0x7f63b006ea30 2024-09-15T22:37:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.628+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01bc2f0 con 0x7f63b006ea30 2024-09-15T22:37:56.631 INFO:tasks.workunit.client.0.smithi077.stdout: 10 195443 13.69 MB/sec execute 597 sec latency 131.643 ms 2024-09-15T22:37:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.630+0000 7f639f7fe640 1 -- 172.21.15.77:0/2831620790 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8036040 con 0x7f63b006ea30 2024-09-15T22:37:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.633+0000 7f639f7fe640 1 -- 172.21.15.77:0/2831620790 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f63a8044020 con 0x7f63b006ea30 2024-09-15T22:37:56.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.633+0000 7f63b7155740 1 Processor -- start 2024-09-15T22:37:56.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 -- start start 2024-09-15T22:37:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df16270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 0x56331df23940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331de7c2f0 con 0x56331df17d20 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56331de23cf0 con 0x56331df21550 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63b7155740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56331de782b0 con 0x56331df167b0 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42000/0 (socket says 172.21.15.77:42000) 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.634+0000 7f63ae59c640 1 -- 172.21.15.77:0/2750500683 learned_addr learned my addr 172.21.15.77:0/2750500683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df16270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63add9b640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 0x56331df23940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 -- 172.21.15.77:0/2750500683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 msgr2=0x56331df23940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 0x56331df23940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 -- 172.21.15.77:0/2750500683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 msgr2=0x56331df16270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df16270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 -- 172.21.15.77:0/2750500683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331de083a0 con 0x56331df167b0 2024-09-15T22:37:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63add9b640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 0x56331df23940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.635+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63a80edf80 tx=0x7f63a80f4f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df46dac2cc707ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.636+0000 7f639cff9640 1 -- 172.21.15.77:0/2750500683 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8103470 con 0x56331df167b0 2024-09-15T22:37:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.636+0000 7f639f7fe640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 0x7f63980a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.636+0000 7f639cff9640 1 -- 172.21.15.77:0/2750500683 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a80fe070 con 0x56331df167b0 2024-09-15T22:37:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.636+0000 7f639cff9640 1 -- 172.21.15.77:0/2750500683 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80bd130 con 0x56331df167b0 2024-09-15T22:37:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.637+0000 7f639f7fe640 1 -- 172.21.15.77:0/2831620790 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f63a80f0ce0 con 0x7f63b006ea30 2024-09-15T22:37:56.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.637+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 0x7f63980a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 0x7f63980a59c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63b006b4a0 tx=0x7f63a4008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 -- 172.21.15.77:0/2750500683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 msgr2=0x56331df16bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63a80edf80 tx=0x7f63a80f4f10 comp rx=0 tx=0).stop 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 -- 172.21.15.77:0/2750500683 shutdown_connections 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df21550 0x56331df23940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df167b0 0x56331df16bf0 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63a80edf80 tx=0x7f63a80f4f10 comp rx=0 tx=0).stop 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 --2- 172.21.15.77:0/2750500683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df16270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.638+0000 7f63b7155740 1 -- 172.21.15.77:0/2750500683 >> 172.21.15.77:0/2750500683 conn(0x56331dedf310 msgr2=0x56331df12e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.639+0000 7f63b7155740 1 -- 172.21.15.77:0/2750500683 shutdown_connections 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.639+0000 7f63b7155740 1 -- 172.21.15.77:0/2750500683 wait complete. 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.639+0000 7f63b7155740 1 -- start start 2024-09-15T22:37:56.641 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.639+0000 7f63b7155740 1 Processor -- start 2024-09-15T22:37:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 0x56331df1e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df2d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56331df24010 con 0x56331df21550 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56331de37060 con 0x56331df1ee40 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56331df24190 con 0x56331df17d20 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42016/0 (socket says 172.21.15.77:42016) 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63ae59c640 1 -- 172.21.15.77:0/3096229305 learned_addr learned my addr 172.21.15.77:0/3096229305 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63b7155740 10 client.0 _ll_register_callbacks cb 0x56331db8a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63add9b640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 0x56331df1e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63aed9d640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df2d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.640+0000 7f63ae59c640 1 -- 172.21.15.77:0/3096229305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 msgr2=0x56331df2d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63ae59c640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df2d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63ae59c640 1 -- 172.21.15.77:0/3096229305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 msgr2=0x56331df1e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63ae59c640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 0x56331df1e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63ae59c640 1 -- 172.21.15.77:0/3096229305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331dbed3c0 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63add9b640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 0x56331df1e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63ae59c640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63a8004af0 tx=0x7f63a80305a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2793b9ac08ea221 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f6395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80bcf20 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a80fe070 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f63a8003b00 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.641+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63a80bd130 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 10 client.29707 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56331df24010 con 0x56331df17d20 2024-09-15T22:37:56.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a80bd7d0 con 0x56331df17d20 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f63a8004680 con 0x56331df17d20 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 10 client.29707 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 10 client.29707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 20 client.29707 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df312b0 0x56331df33730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 4 client.29707 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f63a80fe070 con 0x56331df312b0 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.642+0000 7f63add9b640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df312b0 0x56331df33730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.643+0000 7f6395ffb640 10 client.29707 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:56.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.643+0000 7f63add9b640 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df312b0 0x56331df33730 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfac65bc1fe7afca server_cookie=7451097297441227 in_seq=0 out_seq=0 2024-09-15T22:37:56.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f6395ffb640 1 -- 172.21.15.77:0/3096229305 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f63a80fe070 con 0x56331df312b0 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f6395ffb640 10 client.29707 handle_command_reply: tid=0 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 client.29707 shutdown 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 msgr2=0x56331df1e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63a8004af0 tx=0x7f63a80305a0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 shutdown_connections 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df312b0 0x56331df33730 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df2d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df21550 0x56331df1e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 --2- 172.21.15.77:0/3096229305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df1e3c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 >> 172.21.15.77:0/3096229305 conn(0x56331dedf310 msgr2=0x56331df18b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.644+0000 7f6395ffb640 0 client.29707 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:56.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.645+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 shutdown_connections 2024-09-15T22:37:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.645+0000 7f63b7155740 1 -- 172.21.15.77:0/3096229305 wait complete. 2024-09-15T22:37:56.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.645+0000 7f63b7155740 20 client.29707 trim_cache size 0 max 16384 2024-09-15T22:37:56.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.659+0000 7f63b7155740 1 Processor -- start 2024-09-15T22:37:56.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.659+0000 7f63b7155740 1 -- start start 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df2d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df1ee40 0x56331df2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f639000fe90 con 0x56331df17d20 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63900119d0 con 0x56331dedf310 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63b7155740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6390010560 con 0x56331df1ee40 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:49074/0 (socket says 172.21.15.77:49074) 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 -- 172.21.15.77:0/558665662 learned_addr learned my addr 172.21.15.77:0/558665662 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63aed9d640 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df1ee40 0x56331df2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 -- 172.21.15.77:0/558665662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df1ee40 msgr2=0x56331df2d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df1ee40 0x56331df2d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 -- 172.21.15.77:0/558665662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 msgr2=0x56331df2d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df2d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 -- 172.21.15.77:0/558665662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331df211b0 con 0x56331dedf310 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.660+0000 7f63ae59c640 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63a80c1040 tx=0x7f63a80306a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd954430af25f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f6397fff640 1 -- 172.21.15.77:0/558665662 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80b9f60 con 0x56331dedf310 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f6397fff640 1 -- 172.21.15.77:0/558665662 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a80fe070 con 0x56331dedf310 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f6397fff640 1 -- 172.21.15.77:0/558665662 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80c1480 con 0x56331dedf310 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 -- 172.21.15.77:0/558665662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 msgr2=0x56331df1cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63a80c1040 tx=0x7f63a80306a0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 -- 172.21.15.77:0/558665662 shutdown_connections 2024-09-15T22:37:56.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df1ee40 0x56331df2d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331df17d20 0x56331df2d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 --2- 172.21.15.77:0/558665662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331dedf310 0x56331df1cc90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 -- 172.21.15.77:0/558665662 >> 172.21.15.77:0/558665662 conn(0x56331df1ea60 msgr2=0x56331df20440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.661+0000 7f63b7155740 1 -- 172.21.15.77:0/558665662 shutdown_connections 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 -- 172.21.15.77:0/558665662 wait complete. 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 -- start start 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 Processor -- start 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331dedf310 0x56331df24980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df9a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6390011300 con 0x56331dedf310 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6390010c30 con 0x56331df1ee40 2024-09-15T22:37:56.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63b7155740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f639000de90 con 0x56331df17d20 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63ae59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331dedf310 0x56331df24980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63add9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63add9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42026/0 (socket says 172.21.15.77:42026) 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.662+0000 7f63add9b640 1 -- 172.21.15.77:0/2198166172 learned_addr learned my addr 172.21.15.77:0/2198166172 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63b7155740 10 client.0 _ll_register_callbacks cb 0x7f63b00688b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:56.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63aed9d640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df9a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 -- 172.21.15.77:0/2198166172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 msgr2=0x56331df9a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df9a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 -- 172.21.15.77:0/2198166172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331dedf310 msgr2=0x56331df24980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331dedf310 0x56331df24980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 -- 172.21.15.77:0/2198166172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331df9a540 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63add9b640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56331df17250 tx=0x7f63a00039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d0655526197c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f6395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0005220 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f63a00053c0 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a80fe070 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63a002d050 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63b7155740 10 client.29713 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.663+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56331df9b880 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a80fe070 con 0x56331df17d20 2024-09-15T22:37:56.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f63a80f9d20 con 0x56331df17d20 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 10 client.29713 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 10 client.29713 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 20 client.29713 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df21550 0x56331df9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 4 client.29713 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56331de89860 con 0x56331df21550 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.664+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df21550 0x56331df9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f6395ffb640 10 client.29713 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63ae59c640 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df21550 0x56331df9e020 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d40cd735ddf469c server_cookie=8592502e7b554d7f in_seq=0 out_seq=0 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f6395ffb640 1 -- 172.21.15.77:0/2198166172 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56331de89860 con 0x56331df21550 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f6395ffb640 10 client.29713 handle_command_reply: tid=0 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 client.29713 shutdown 2024-09-15T22:37:56.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 msgr2=0x56331df24ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56331df17250 tx=0x7f63a00039c0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 shutdown_connections 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x56331df21550 0x56331df9e020 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56331df1ee40 0x56331df9a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56331df17d20 0x56331df24ec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 --2- 172.21.15.77:0/2198166172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56331dedf310 0x56331df24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f6395ffb640 0 client.29713 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 >> 172.21.15.77:0/2198166172 conn(0x56331df1ea60 msgr2=0x56331df11780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 shutdown_connections 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 1 -- 172.21.15.77:0/2198166172 wait complete. 2024-09-15T22:37:56.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.665+0000 7f63b7155740 20 client.29713 trim_cache size 0 max 16384 2024-09-15T22:37:56.673 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:56.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 msgr2=0x7f63980a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 0x7f63980a59c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63b006b4a0 tx=0x7f63a4008040 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 msgr2=0x7f63b011a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63a8034040 tx=0x7f63a8030da0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 shutdown_connections 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.666+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f63980a3500 0x7f63980a59c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63b011b140 0x7f63b0114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b0110190 0x7f63b01149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 --2- 172.21.15.77:0/2831620790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b006ea30 0x7f63b011a960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 >> 172.21.15.77:0/2831620790 conn(0x7f63b0073180 msgr2=0x7f63b011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 shutdown_connections 2024-09-15T22:37:56.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:56.667+0000 7f63af59e640 1 -- 172.21.15.77:0/2831620790 wait complete. 2024-09-15T22:37:57.492 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:37:57.630 INFO:tasks.workunit.client.0.smithi077.stdout: 10 195913 13.71 MB/sec execute 598 sec latency 160.118 ms 2024-09-15T22:37:57.786 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:58.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 -- 172.21.15.77:0/920799621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe808101270 msgr2=0x7fe80810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 --2- 172.21.15.77:0/920799621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe808101270 0x7fe80810d7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe800009ca0 tx=0x7fe80002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 -- 172.21.15.77:0/920799621 shutdown_connections 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 --2- 172.21.15.77:0/920799621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe808101270 0x7fe80810d7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 --2- 172.21.15.77:0/920799621 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8081008d0 0x7fe808100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 --2- 172.21.15.77:0/920799621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081068d0 0x7fe808106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 -- 172.21.15.77:0/920799621 >> 172.21.15.77:0/920799621 conn(0x7fe8080fc610 msgr2=0x7fe8080fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 -- 172.21.15.77:0/920799621 shutdown_connections 2024-09-15T22:37:58.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 -- 172.21.15.77:0/920799621 wait complete. 2024-09-15T22:37:58.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.383+0000 7fe807fff640 1 Processor -- start 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 -- start start 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 0x7fe808197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 0x7fe80819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe808110d40 con 0x7fe8081008d0 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe808110bc0 con 0x7fe8081068d0 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe807fff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe808110ec0 con 0x7fe808101270 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe806ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe806ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60622/0 (socket says 172.21.15.77:60622) 2024-09-15T22:37:58.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe806ffd640 1 -- 172.21.15.77:0/2224693251 learned_addr learned my addr 172.21.15.77:0/2224693251 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe8077fe640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 0x7fe80819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.384+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 0x7fe808197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 -- 172.21.15.77:0/2224693251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 msgr2=0x7fe808197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 0x7fe808197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 -- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 msgr2=0x7fe8081971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 -- 172.21.15.77:0/2224693251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80819c190 con 0x7fe8081068d0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 0x7fe808197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe8077fe640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 0x7fe80819ba90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe800005e30 tx=0x7fe800003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe7fffff640 1 -- 172.21.15.77:0/2224693251 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800031650 con 0x7fe8081068d0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe7fffff640 1 -- 172.21.15.77:0/2224693251 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8000317f0 con 0x7fe8081068d0 2024-09-15T22:37:58.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe7fffff640 1 -- 172.21.15.77:0/2224693251 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80002f970 con 0x7fe8081068d0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.385+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80819c420 con 0x7fe8081068d0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.386+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe808069f70 con 0x7fe8081068d0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.388+0000 7fe7fffff640 1 -- 172.21.15.77:0/2224693251 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe80002fb30 con 0x7fe8081068d0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.389+0000 7fe7fffff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 0x7fe7f40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.389+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 0x7fe7f40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.389+0000 7fe7fffff640 1 -- 172.21.15.77:0/2224693251 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fe8000f1300 con 0x7fe8081068d0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.389+0000 7fe80f7d8740 1 Processor -- start 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 -- start start 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 0x7fe7f40a5870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80095f0 tx=0x7fe7f8005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b1c550 0x55a941b1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a941a772f0 con 0x55a941b12d20 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a941a1ecf0 con 0x55a941b1c550 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a941a732b0 con 0x55a941b117b0 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe8067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe8067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42064/0 (socket says 172.21.15.77:42064) 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe8067fc640 1 -- 172.21.15.77:0/2455858538 learned_addr learned my addr 172.21.15.77:0/2455858538 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe806ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.390+0000 7fe8077fe640 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b1c550 0x55a941b1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.396 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 -- 172.21.15.77:0/2455858538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b1c550 msgr2=0x55a941b1e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b1c550 0x55a941b1e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 -- 172.21.15.77:0/2455858538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 msgr2=0x55a941b11270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b11270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 -- 172.21.15.77:0/2455858538 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a941a033a0 con 0x55a941b117b0 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009e10 tx=0x7fe7f000e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b972f3f56e391ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe7fd7fa640 1 -- 172.21.15.77:0/2455858538 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0017070 con 0x55a941b117b0 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe7fd7fa640 1 -- 172.21.15.77:0/2455858538 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f0013950 con 0x55a941b117b0 2024-09-15T22:37:58.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.391+0000 7fe7fd7fa640 1 -- 172.21.15.77:0/2455858538 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00129a0 con 0x55a941b117b0 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2455858538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 msgr2=0x55a941b11bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009e10 tx=0x7fe7f000e5a0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2455858538 shutdown_connections 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b1c550 0x55a941b1e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b117b0 0x55a941b11bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.401 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2455858538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2455858538 >> 172.21.15.77:0/2455858538 conn(0x55a941ada310 msgr2=0x55a941b0de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2455858538 shutdown_connections 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.392+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2455858538 wait complete. 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 -- start start 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 Processor -- start 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b1c550 0x55a941b19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a941b30260 con 0x55a941b1c550 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a941a32060 con 0x55a941b117b0 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a941b303e0 con 0x55a941b12d20 2024-09-15T22:37:58.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe80f7d8740 10 client.0 _ll_register_callbacks cb 0x55a9417858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe806ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42072/0 (socket says 172.21.15.77:42072) 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 -- 172.21.15.77:0/628007421 learned_addr learned my addr 172.21.15.77:0/628007421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8077fe640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b1c550 0x55a941b19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 -- 172.21.15.77:0/628007421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 msgr2=0x55a941b1bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b1bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 -- 172.21.15.77:0/628007421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b1c550 msgr2=0x55a941b19710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b1c550 0x55a941b19710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.393+0000 7fe8067fc640 1 -- 172.21.15.77:0/628007421 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9417e83c0 con 0x55a941b12d20 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe8067fc640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0004810 tx=0x7fe7f00123d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dff61b4dad23dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-09-15T22:37:58.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0017040 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fe7f0012720 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f0013950 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7f0012920 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe80f7d8740 10 client.29722 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a941b3bc90 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f001c450 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.394+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe7f0023070 con 0x55a941b12d20 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 10 client.29722 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 10 client.29722 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 20 client.29722 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941b3cfe0 0x55a941b3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 4 client.29722 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7f0013950 con 0x55a941b3cfe0 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe806ffd640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941b3cfe0 0x55a941b3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe806ffd640 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941b3cfe0 0x55a941b3f3d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b0db4d435811b75 server_cookie=d43680a7bb9a92bb in_seq=0 out_seq=0 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.395+0000 7fe7ee7fc640 10 client.29722 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/628007421 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe7f0013950 con 0x55a941b3cfe0 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe7ee7fc640 10 client.29722 handle_command_reply: tid=0 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 client.29722 shutdown 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 msgr2=0x55a941b19190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0004810 tx=0x7fe7f00123d0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 shutdown_connections 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941b3cfe0 0x55a941b3f3d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b1c550 0x55a941b19710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b12d20 0x55a941b19190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/628007421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.396+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 >> 172.21.15.77:0/628007421 conn(0x55a941ada310 msgr2=0x55a941b13b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.397+0000 7fe7ee7fc640 0 client.29722 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.397+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 shutdown_connections 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.397+0000 7fe80f7d8740 1 -- 172.21.15.77:0/628007421 wait complete. 2024-09-15T22:37:58.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.397+0000 7fe80f7d8740 20 client.29722 trim_cache size 0 max 16384 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.414+0000 7fe80f7d8740 1 Processor -- start 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.414+0000 7fe80f7d8740 1 -- start start 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b39500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8080728c0 con 0x55a941b12d20 2024-09-15T22:37:58.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081aa360 con 0x55a941b117b0 2024-09-15T22:37:58.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe808072f90 con 0x55a941b1c550 2024-09-15T22:37:58.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe806ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b39500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b1d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42084/0 (socket says 172.21.15.77:42084) 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8077fe640 1 -- 172.21.15.77:0/2859452506 learned_addr learned my addr 172.21.15.77:0/2859452506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 -- 172.21.15.77:0/2859452506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 msgr2=0x55a941b1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 -- 172.21.15.77:0/2859452506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 msgr2=0x55a941b39500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b39500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.415+0000 7fe8067fc640 1 -- 172.21.15.77:0/2859452506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a941b0e0b0 con 0x55a941b12d20 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.416+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b18170 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0012610 tx=0x7fe7f0004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138b5d69771652b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.416+0000 7fe7fcff9640 1 -- 172.21.15.77:0/2859452506 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0025440 con 0x55a941b12d20 2024-09-15T22:37:58.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.416+0000 7fe7fcff9640 1 -- 172.21.15.77:0/2859452506 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f00255e0 con 0x55a941b12d20 2024-09-15T22:37:58.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.416+0000 7fe7fcff9640 1 -- 172.21.15.77:0/2859452506 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f001c450 con 0x55a941b12d20 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2859452506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 msgr2=0x55a941b18170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b18170 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0012610 tx=0x7fe7f0004640 comp rx=0 tx=0).stop 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2859452506 shutdown_connections 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b12d20 0x55a941b18170 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2859452506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b117b0 0x55a941b39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2859452506 >> 172.21.15.77:0/2859452506 conn(0x55a941ada310 msgr2=0x55a941b17a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2859452506 shutdown_connections 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.417+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2859452506 wait complete. 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.418+0000 7fe80f7d8740 1 -- start start 2024-09-15T22:37:58.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 Processor -- start 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 0x55a941b43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b12d20 0x55a941b44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe808073d30 con 0x55a941b117b0 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe808073660 con 0x55a941b12d20 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8080708c0 con 0x55a941b1c550 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe80f7d8740 10 client.0 _ll_register_callbacks cb 0x7fe7f8009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:58.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe8077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe8077fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b34670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:42096/0 (socket says 172.21.15.77:42096) 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe8077fe640 1 -- 172.21.15.77:0/2451171491 learned_addr learned my addr 172.21.15.77:0/2451171491 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.419+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b12d20 0x55a941b44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 0x55a941b43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 -- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 msgr2=0x55a941b34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b34670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 -- 172.21.15.77:0/2451171491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b12d20 msgr2=0x55a941b44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b12d20 0x55a941b44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.420+0000 7fe806ffd640 1 -- 172.21.15.77:0/2451171491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a941b34c30 con 0x55a941b117b0 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.421+0000 7fe806ffd640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 0x55a941b43f20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x55a941b3f910 tx=0x7fe7f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2088313d1972c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:58.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.421+0000 7fe7ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.421+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8029620 con 0x55a941b117b0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.421+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7fe7f80297c0 con 0x55a941b117b0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.421+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f00255e0 con 0x55a941b117b0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.422+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe7f803a050 con 0x55a941b117b0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.422+0000 7fe80f7d8740 10 client.21210 fetch_fsmap learned FSMap version 39 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.422+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a941b35f50 con 0x55a941b117b0 2024-09-15T22:37:58.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.422+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f00255e0 con 0x55a941b117b0 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe7f8031070 con 0x55a941b117b0 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe80f7d8740 10 client.21210 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe80f7d8740 10 client.21210 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe80f7d8740 20 client.21210 populate_metadata read hostname 'smithi077' 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941ba7af0 0x55a941ba9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:58.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.424+0000 7fe80f7d8740 4 client.21210 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:37:58.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.425+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a941a84860 con 0x55a941ba7af0 2024-09-15T22:37:58.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.425+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941ba7af0 0x55a941ba9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:58.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.426+0000 7fe8067fc640 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941ba7af0 0x55a941ba9f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf30c729ba06e172 server_cookie=bcabb39f668f5953 in_seq=0 out_seq=0 2024-09-15T22:37:58.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.426+0000 7fe7ee7fc640 10 client.21210 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:58.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.426+0000 7fe7ee7fc640 1 -- 172.21.15.77:0/2451171491 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a941a84860 con 0x55a941ba7af0 2024-09-15T22:37:58.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.426+0000 7fe7ee7fc640 10 client.21210 handle_command_reply: tid=0 2024-09-15T22:37:58.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.427+0000 7fe80f7d8740 1 client.21210 shutdown 2024-09-15T22:37:58.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 msgr2=0x55a941b43f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 0x55a941b43f20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x55a941b3f910 tx=0x7fe7f8004730 comp rx=0 tx=0).stop 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 shutdown_connections 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x55a941ba7af0 0x55a941ba9f10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe7ee7fc640 0 client.21210 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a941b1c550 0x55a941b34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a941b12d20 0x55a941b44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 --2- 172.21.15.77:0/2451171491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a941b117b0 0x55a941b43f20 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 >> 172.21.15.77:0/2451171491 conn(0x55a941ada310 msgr2=0x55a941b18280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 shutdown_connections 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.428+0000 7fe80f7d8740 1 -- 172.21.15.77:0/2451171491 wait complete. 2024-09-15T22:37:58.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.429+0000 7fe80f7d8740 20 client.21210 trim_cache size 0 max 16384 2024-09-15T22:37:58.431 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:37:58.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 msgr2=0x7fe7f40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 0x7fe7f40a5870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80095f0 tx=0x7fe7f8005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:58.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 msgr2=0x7fe80819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:58.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 0x7fe80819ba90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe800005e30 tx=0x7fe800003f10 comp rx=0 tx=0).stop 2024-09-15T22:37:58.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 shutdown_connections 2024-09-15T22:37:58.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7f40a33b0 0x7fe7f40a5870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8081068d0 0x7fe80819ba90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe808101270 0x7fe808197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 --2- 172.21.15.77:0/2224693251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8081008d0 0x7fe8081971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:58.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.431+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 >> 172.21.15.77:0/2224693251 conn(0x7fe8080fc610 msgr2=0x7fe8080fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:58.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.432+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 shutdown_connections 2024-09-15T22:37:58.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:37:58.432+0000 7fe807fff640 1 -- 172.21.15.77:0/2224693251 wait complete. 2024-09-15T22:37:58.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:37:58 smithi077 ceph-mon[30980]: pgmap v807: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 460 op/s 2024-09-15T22:37:58.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:37:58 smithi196 ceph-mon[35109]: pgmap v807: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 460 op/s 2024-09-15T22:37:58.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:37:58 smithi117 ceph-mon[35021]: pgmap v807: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 460 op/s 2024-09-15T22:37:58.630 INFO:tasks.workunit.client.0.smithi077.stdout: 10 196388 13.74 MB/sec execute 599 sec latency 188.932 ms 2024-09-15T22:37:59.221 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:37:59.528 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:59.630 INFO:tasks.workunit.client.0.smithi077.stdout: 10 cleanup 600 sec 2024-09-15T22:38:00.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 -- 172.21.15.77:0/273079343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89981055b0 msgr2=0x7f899810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 --2- 172.21.15.77:0/273079343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89981055b0 0x7f899810d130 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8990009ca0 tx=0x7f899002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 -- 172.21.15.77:0/273079343 shutdown_connections 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 --2- 172.21.15.77:0/273079343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89981055b0 0x7f899810d130 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 --2- 172.21.15.77:0/273079343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8998100e30 0x7f8998104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.027+0000 7f899f3ef640 1 --2- 172.21.15.77:0/273079343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f8998100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- 172.21.15.77:0/273079343 >> 172.21.15.77:0/273079343 conn(0x7f8998077210 msgr2=0x7f89980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- 172.21.15.77:0/273079343 shutdown_connections 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- 172.21.15.77:0/273079343 wait complete. 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 Processor -- start 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- start start 2024-09-15T22:38:00.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f89981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 0x7f899819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8998110e90 con 0x7f8998100e30 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8998110d10 con 0x7f89981055b0 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.028+0000 7f899f3ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8998111010 con 0x7f8998100480 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899d18b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f89981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899d98c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 0x7f899819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58004/0 (socket says 172.21.15.77:58004) 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899d98c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 0x7f899819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:42318/0 (socket says 172.21.15.77:42318) 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899d18b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f89981983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:35592/0 (socket says 172.21.15.77:35592) 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 -- 172.21.15.77:0/850982852 learned_addr learned my addr 172.21.15.77:0/850982852 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:00.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 -- 172.21.15.77:0/850982852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 msgr2=0x7f89981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f89981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 -- 172.21.15.77:0/850982852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 msgr2=0x7f899819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 0x7f899819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 -- 172.21.15.77:0/850982852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819d360 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899c98a640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9f0 tx=0x7f898800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f898e7fc640 1 -- 172.21.15.77:0/850982852 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800cd80 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819d650 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f898e7fc640 1 -- 172.21.15.77:0/850982852 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8988010910 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f898e7fc640 1 -- 172.21.15.77:0/850982852 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988018990 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.029+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a4f30 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f898e7fc640 1 -- 172.21.15.77:0/850982852 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8988010ac0 con 0x7f8998100e30 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f89a17c8740 1 Processor -- start 2024-09-15T22:38:00.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f89a17c8740 1 -- start start 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f898e7fc640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 0x7f89840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fbb40 0x55769b3fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.032+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b407760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89a17c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55769b3602f0 con 0x55769b3fa5d0 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89a17c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55769b307cf0 con 0x55769b405370 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89a17c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55769b35c2b0 con 0x55769b3fbb40 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f898e7fc640 1 -- 172.21.15.77:0/850982852 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f89880d0340 con 0x7f8998100e30 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899d98c640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 0x7f89840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58018/0 (socket says 172.21.15.77:58018) 2024-09-15T22:38:00.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 -- 172.21.15.77:0/4061283982 learned_addr learned my addr 172.21.15.77:0/4061283982 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899d98c640 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b407760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899d18b640 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fbb40 0x55769b3fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 -- 172.21.15.77:0/4061283982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fbb40 msgr2=0x55769b3fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fbb40 0x55769b3fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 -- 172.21.15.77:0/4061283982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 msgr2=0x55769b407760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b407760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 -- 172.21.15.77:0/4061283982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55769b2ec3a0 con 0x55769b3fa5d0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899d98c640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 0x7f89840a5870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8990004af0 tx=0x7f8990031c90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f899c98a640 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f8988098d10 tx=0x7f8988056d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836079a6d825afee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89837fe640 1 -- 172.21.15.77:0/4061283982 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880d0170 con 0x55769b3fa5d0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89837fe640 1 -- 172.21.15.77:0/4061283982 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89880cde50 con 0x55769b3fa5d0 2024-09-15T22:38:00.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.033+0000 7f89837fe640 1 -- 172.21.15.77:0/4061283982 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880bc310 con 0x55769b3fa5d0 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- 172.21.15.77:0/4061283982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 msgr2=0x55769b3faa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f8988098d10 tx=0x7f8988056d70 comp rx=0 tx=0).stop 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- 172.21.15.77:0/4061283982 shutdown_connections 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b407760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fa5d0 0x55769b3faa10 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 --2- 172.21.15.77:0/4061283982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fbb40 0x55769b3fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- 172.21.15.77:0/4061283982 >> 172.21.15.77:0/4061283982 conn(0x55769b3c3310 msgr2=0x55769b3f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- 172.21.15.77:0/4061283982 shutdown_connections 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- 172.21.15.77:0/4061283982 wait complete. 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.034+0000 7f89a17c8740 1 -- start start 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 Processor -- start 2024-09-15T22:38:00.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b404da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b4027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55769b408070 con 0x55769b3fbb40 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55769b31b060 con 0x55769b405370 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55769b4081f0 con 0x55769b3fa5d0 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f89a17c8740 10 client.0 _ll_register_callbacks cb 0x55769b06e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58020/0 (socket says 172.21.15.77:58020) 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 -- 172.21.15.77:0/3698431211 learned_addr learned my addr 172.21.15.77:0/3698431211 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899d98c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b4027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899d18b640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b404da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 -- 172.21.15.77:0/3698431211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 msgr2=0x55769b404da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b404da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 -- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 msgr2=0x55769b4027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b4027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899c98a640 1 -- 172.21.15.77:0/3698431211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55769b0d13c0 con 0x55769b3fbb40 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.035+0000 7f899d98c640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b4027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f899c98a640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f898809c2d0 tx=0x7f8988018d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b9f47c17de8912 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880d0170 con 0x55769b3fbb40 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f89880bc2f0 con 0x55769b3fbb40 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89880cde50 con 0x55769b3fbb40 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89880ba050 con 0x55769b3fbb40 2024-09-15T22:38:00.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 10 client.21222 fetch_fsmap learned FSMap version 39 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55769b413b50 con 0x55769b3fbb40 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89880cde50 con 0x55769b3fbb40 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f89880591b0 con 0x55769b3fbb40 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 10 client.21222 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 10 client.21222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 20 client.21222 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b414ea0 0x55769b417290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 4 client.21222 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.036+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55769b36d860 con 0x55769b414ea0 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.037+0000 7f899d18b640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b414ea0 0x55769b417290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.037+0000 7f8980ff9640 10 client.21222 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.037+0000 7f899d18b640 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b414ea0 0x55769b417290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45559f7c5b5ddb8f server_cookie=a8d72fc76ba9a08c in_seq=0 out_seq=0 2024-09-15T22:38:00.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f8980ff9640 1 -- 172.21.15.77:0/3698431211 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55769b36d860 con 0x55769b414ea0 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f8980ff9640 10 client.21222 handle_command_reply: tid=0 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 client.21222 shutdown 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 msgr2=0x55769b402050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f898809c2d0 tx=0x7f8988018d60 comp rx=0 tx=0).stop 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 shutdown_connections 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b414ea0 0x55769b417290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b405370 0x55769b4027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b402050 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 --2- 172.21.15.77:0/3698431211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b404da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f8980ff9640 0 client.21222 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 >> 172.21.15.77:0/3698431211 conn(0x55769b3c3310 msgr2=0x55769b3fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 shutdown_connections 2024-09-15T22:38:00.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 1 -- 172.21.15.77:0/3698431211 wait complete. 2024-09-15T22:38:00.053 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.038+0000 7f89a17c8740 20 client.21222 trim_cache size 0 max 16384 2024-09-15T22:38:00.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.052+0000 7f89a17c8740 1 Processor -- start 2024-09-15T22:38:00.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 -- start start 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b3f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fbb40 0x55769b3f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89981aee60 con 0x55769b405370 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b09a0 con 0x55769b3fbb40 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f89a17c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89981af530 con 0x55769b3fa5d0 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58022/0 (socket says 172.21.15.77:58022) 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 -- 172.21.15.77:0/2727712221 learned_addr learned my addr 172.21.15.77:0/2727712221 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d18b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b3f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899c98a640 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fbb40 0x55769b3f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 -- 172.21.15.77:0/2727712221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 msgr2=0x55769b3f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b3f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 -- 172.21.15.77:0/2727712221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fbb40 msgr2=0x55769b3f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fbb40 0x55769b3f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 -- 172.21.15.77:0/2727712221 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55769b3fd120 con 0x55769b405370 2024-09-15T22:38:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.053+0000 7f899d98c640 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f8990035000 tx=0x7f8990049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a955e07be4b512bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f8982ffd640 1 -- 172.21.15.77:0/2727712221 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900319d0 con 0x55769b405370 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f8982ffd640 1 -- 172.21.15.77:0/2727712221 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990036040 con 0x55769b405370 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f8982ffd640 1 -- 172.21.15.77:0/2727712221 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899002fc60 con 0x55769b405370 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 -- 172.21.15.77:0/2727712221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 msgr2=0x55769b41a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f8990035000 tx=0x7f8990049000 comp rx=0 tx=0).stop 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 -- 172.21.15.77:0/2727712221 shutdown_connections 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b405370 0x55769b41a2b0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fbb40 0x55769b3f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 --2- 172.21.15.77:0/2727712221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b3fa5d0 0x55769b3f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 -- 172.21.15.77:0/2727712221 >> 172.21.15.77:0/2727712221 conn(0x55769b3c3310 msgr2=0x55769b3f54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.054+0000 7f89a17c8740 1 -- 172.21.15.77:0/2727712221 shutdown_connections 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 -- 172.21.15.77:0/2727712221 wait complete. 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 -- start start 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 Processor -- start 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b417fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b405370 0x55769b41a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b02d0 con 0x55769b3fbb40 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89981afc00 con 0x55769b3fa5d0 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.055+0000 7f89a17c8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89981ace60 con 0x55769b405370 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:42366/0 (socket says 172.21.15.77:42366) 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899c98a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b417fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 -- 172.21.15.77:0/724171677 learned_addr learned my addr 172.21.15.77:0/724171677 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f89a17c8740 10 client.0 _ll_register_callbacks cb 0x7f8994005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d98c640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b405370 0x55769b41a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 -- 172.21.15.77:0/724171677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b405370 msgr2=0x55769b41a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b405370 0x55769b41a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 -- 172.21.15.77:0/724171677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 msgr2=0x55769b417fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b417fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 -- 172.21.15.77:0/724171677 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55769b41a7c0 con 0x55769b3fa5d0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f899d18b640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55769b408220 tx=0x7f8994004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef48736b3e3c07f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f8980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994003c60 con 0x55769b3fa5d0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8990036040 con 0x55769b3fa5d0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f8994003e00 con 0x55769b3fa5d0 2024-09-15T22:38:00.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f899402f050 con 0x55769b3fa5d0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f89a17c8740 10 client.29831 fetch_fsmap learned FSMap version 39 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.056+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55769b4837f0 con 0x55769b3fa5d0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990036040 con 0x55769b3fa5d0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8994027880 con 0x55769b3fa5d0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 10 client.29831 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 10 client.29831 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 20 client.29831 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b483f40 0x55769b486360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 4 client.29831 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55769b367d70 con 0x55769b483f40 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.057+0000 7f899c98a640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b483f40 0x55769b486360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f8980ff9640 10 client.29831 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f899c98a640 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b483f40 0x55769b486360 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94fe3282255583f3 server_cookie=cdfe1fbe95b70c5a in_seq=0 out_seq=0 2024-09-15T22:38:00.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f8980ff9640 1 -- 172.21.15.77:0/724171677 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55769b367d70 con 0x55769b483f40 2024-09-15T22:38:00.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f8980ff9640 10 client.29831 handle_command_reply: tid=0 2024-09-15T22:38:00.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 client.29831 shutdown 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 msgr2=0x55769b417a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55769b408220 tx=0x7f8994004100 comp rx=0 tx=0).stop 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 shutdown_connections 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x55769b483f40 0x55769b486360 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55769b405370 0x55769b41a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55769b3fbb40 0x55769b417fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f8980ff9640 0 client.29831 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 --2- 172.21.15.77:0/724171677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55769b3fa5d0 0x55769b417a70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 >> 172.21.15.77:0/724171677 conn(0x55769b3c3310 msgr2=0x55769b3fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 shutdown_connections 2024-09-15T22:38:00.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 1 -- 172.21.15.77:0/724171677 wait complete. 2024-09-15T22:38:00.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.058+0000 7f89a17c8740 20 client.29831 trim_cache size 0 max 16384 2024-09-15T22:38:00.065 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:00.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 msgr2=0x7f89840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 0x7f89840a5870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8990004af0 tx=0x7f8990031c90 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 msgr2=0x7f89981988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9f0 tx=0x7f898800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 shutdown_connections 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f89840a33b0 0x7f89840a5870 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89981055b0 0x7f899819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8998100e30 0x7f89981988f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 --2- 172.21.15.77:0/850982852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8998100480 0x7f89981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 >> 172.21.15.77:0/850982852 conn(0x7f8998077210 msgr2=0x7f89980feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 shutdown_connections 2024-09-15T22:38:00.066 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:00.059+0000 7f899f3ef640 1 -- 172.21.15.77:0/850982852 wait complete. 2024-09-15T22:38:00.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:00 smithi077 ceph-mon[30980]: pgmap v808: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.6 MiB/s wr, 316 op/s 2024-09-15T22:38:00.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:00 smithi196 ceph-mon[35109]: pgmap v808: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.6 MiB/s wr, 316 op/s 2024-09-15T22:38:00.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:00 smithi117 ceph-mon[35021]: pgmap v808: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.6 MiB/s wr, 316 op/s 2024-09-15T22:38:00.630 INFO:tasks.workunit.client.0.smithi077.stdout: 7 cleanup 601 sec 2024-09-15T22:38:00.779 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:38:01.055 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: 0 cleanup 602 sec 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: Operation Count AvgLat MaxLat 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: ---------------------------------------- 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: NTCreateX 264114 10.043 34281.208 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: Close 193966 0.003 0.297 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: Rename 11197 45.131 34938.566 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: Unlink 53385 14.567 34860.127 2024-09-15T22:38:01.369 INFO:tasks.workunit.client.0.smithi077.stdout: Qpathinfo 239650 5.774 11889.788 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: Qfileinfo 41798 0.001 0.047 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: Qfsinfo 43981 0.291 38.627 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: Sfileinfo 21514 6.162 26539.778 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: Find 92697 0.886 4467.934 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: WriteX 130570 2.373 34655.342 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: ReadX 415094 0.088 901.725 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: LockX 862 5.014 54.560 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: UnlockX 862 4.243 102.406 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: Flush 18518 13.374 1365.426 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-09-15T22:38:01.370 INFO:tasks.workunit.client.0.smithi077.stdout:Throughput 13.7381 MB/sec 10 clients 10 procs max_latency=34938.573 ms 2024-09-15T22:38:01.371 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:38:01.371 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 -- 172.21.15.77:0/635929381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000102790 msgr2=0x7f50001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 --2- 172.21.15.77:0/635929381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000102790 0x7f50001067e0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 -- 172.21.15.77:0/635929381 shutdown_connections 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 --2- 172.21.15.77:0/635929381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 0x7f500010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 --2- 172.21.15.77:0/635929381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000102790 0x7f50001067e0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 --2- 172.21.15.77:0/635929381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000101de0 0x7f50001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 -- 172.21.15.77:0/635929381 >> 172.21.15.77:0/635929381 conn(0x7f50000fd8c0 msgr2=0x7f50000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 -- 172.21.15.77:0/635929381 shutdown_connections 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.591+0000 7f500673c640 1 -- 172.21.15.77:0/635929381 wait complete. 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 Processor -- start 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 -- start start 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000102790 0x7f5000074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 0x7f5000074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5000111f70 con 0x7f5000101de0 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5000111df0 con 0x7f5000106f10 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f500673c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50001120f0 con 0x7f5000102790 2024-09-15T22:38:01.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58040/0 (socket says 172.21.15.77:58040) 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f4ffffff640 1 -- 172.21.15.77:0/526817540 learned_addr learned my addr 172.21.15.77:0/526817540 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000102790 0x7f5000074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.592+0000 7f5004cd9640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 0x7f5000074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 -- 172.21.15.77:0/526817540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000102790 msgr2=0x7f5000074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000102790 0x7f5000074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 -- 172.21.15.77:0/526817540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 msgr2=0x7f5000074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 0x7f5000074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 -- 172.21.15.77:0/526817540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50001a6900 con 0x7f5000101de0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffffff640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000e9f0 tx=0x7f4ff000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffd7fa640 1 -- 172.21.15.77:0/526817540 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000cd80 con 0x7f5000101de0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50001a6bf0 con 0x7f5000101de0 2024-09-15T22:38:01.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffd7fa640 1 -- 172.21.15.77:0/526817540 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0010910 con 0x7f5000101de0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f4ffd7fa640 1 -- 172.21.15.77:0/526817540 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00188f0 con 0x7f5000101de0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.593+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50001a7100 con 0x7f5000101de0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.595+0000 7f4ffd7fa640 1 -- 172.21.15.77:0/526817540 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ff0010ac0 con 0x7f5000101de0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 Processor -- start 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f4ffd7fa640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 -- start start 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f4ffd7fa640 1 -- 172.21.15.77:0/526817540 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4ff00d0460 con 0x7f5000101de0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5004cd9640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf18b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf198760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558caf0f12f0 con 0x558caf18b5d0 2024-09-15T22:38:01.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558caf098cf0 con 0x558caf196370 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5008b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558caf0ed2b0 con 0x558caf18cb40 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f5004cd9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.596+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58050/0 (socket says 172.21.15.77:58050) 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1802703525 learned_addr learned my addr 172.21.15.77:0/1802703525 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4ffffff640 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf18b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1802703525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 msgr2=0x558caf18b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf18b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1802703525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 msgr2=0x558caf198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f5004cd9640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8005fd0 tx=0x7f4ff8011b40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1802703525 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caf07d3a0 con 0x558caf18b5d0 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed03b7a66a110ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4ff67fc640 1 -- 172.21.15.77:0/1802703525 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8040070 con 0x558caf18b5d0 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4ff67fc640 1 -- 172.21.15.77:0/1802703525 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8031870 con 0x558caf18b5d0 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.597+0000 7f4ff67fc640 1 -- 172.21.15.77:0/1802703525 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8035040 con 0x558caf18b5d0 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- 172.21.15.77:0/1802703525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 msgr2=0x558caf18ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe8005be0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- 172.21.15.77:0/1802703525 shutdown_connections 2024-09-15T22:38:01.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf18ba10 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 --2- 172.21.15.77:0/1802703525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf18b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- 172.21.15.77:0/1802703525 >> 172.21.15.77:0/1802703525 conn(0x558caf154310 msgr2=0x558caf187c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- 172.21.15.77:0/1802703525 shutdown_connections 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- 172.21.15.77:0/1802703525 wait complete. 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.598+0000 7f5008b15740 1 -- start start 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 Processor -- start 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf192fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 0x558caf193570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558caf199010 con 0x558caf196370 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558caf0ac060 con 0x558caf18b5d0 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558caf199190 con 0x558caf18cb40 2024-09-15T22:38:01.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5008b15740 10 client.0 _ll_register_callbacks cb 0x558caedff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf192fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5004cd9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 0x558caf193570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:42396/0 (socket says 172.21.15.77:42396) 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 -- 172.21.15.77:0/2735236854 learned_addr learned my addr 172.21.15.77:0/2735236854 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 -- 172.21.15.77:0/2735236854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 msgr2=0x558caf192fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf192fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 -- 172.21.15.77:0/2735236854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 msgr2=0x558caf193570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 0x558caf193570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 -- 172.21.15.77:0/2735236854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caee623c0 con 0x558caf18b5d0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f5004cd9640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 0x558caf193570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4ffffff640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c6a0 tx=0x7f4ff009c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab262f8b03609bed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.599+0000 7f4fdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff009b8b0 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe8031870 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4ff009ba50 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ff00b5200 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 10 client.29843 fetch_fsmap learned FSMap version 39 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558caf1a4ab0 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8031870 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4ff00981b0 con 0x558caf18b5d0 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 10 client.29843 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 10 client.29843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 20 client.29843 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf1a5ec0 0x558caf1a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 4 client.29843 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558caf0fe860 con 0x558caf1a5ec0 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.600+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf1a5ec0 0x558caf1a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.601+0000 7f4fdf7fe640 10 client.29843 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.601+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf1a5ec0 0x558caf1a82b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73d77d24c17d170 server_cookie=89300bc2bdea45fc in_seq=0 out_seq=0 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/2735236854 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558caf0fe860 con 0x558caf1a5ec0 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f4fdf7fe640 10 client.29843 handle_command_reply: tid=0 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 client.29843 shutdown 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 msgr2=0x558caf195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c6a0 tx=0x7f4ff009c590 comp rx=0 tx=0).stop 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 shutdown_connections 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf1a5ec0 0x558caf1a82b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf196370 0x558caf193570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf192fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 --2- 172.21.15.77:0/2735236854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf18b5d0 0x558caf195d20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 >> 172.21.15.77:0/2735236854 conn(0x558caf154310 msgr2=0x558caf18d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f4fdf7fe640 0 client.29843 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:01.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 shutdown_connections 2024-09-15T22:38:01.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 1 -- 172.21.15.77:0/2735236854 wait complete. 2024-09-15T22:38:01.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.603+0000 7f5008b15740 20 client.29843 trim_cache size 0 max 16384 2024-09-15T22:38:01.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 Processor -- start 2024-09-15T22:38:01.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 -- start start 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18b5d0 0x558caf1a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf18e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b11c0 con 0x558caf18cb40 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b2d00 con 0x558caf196370 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5008b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b1890 con 0x558caf18b5d0 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58078/0 (socket says 172.21.15.77:58078) 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1793553468 learned_addr learned my addr 172.21.15.77:0/1793553468 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f4ffffff640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18b5d0 0x558caf1a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.617+0000 7f5004cd9640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf18e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1793553468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18b5d0 msgr2=0x558caf1a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18b5d0 0x558caf1a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1793553468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 msgr2=0x558caf18e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf18e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 -- 172.21.15.77:0/1793553468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caf1a8910 con 0x558caf18cb40 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f5004cd9640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf18e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80053f0 tx=0x7f4fe8047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6029f9dc46ff787a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4ff5ffb640 1 -- 172.21.15.77:0/1793553468 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8040040 con 0x558caf18cb40 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4ff5ffb640 1 -- 172.21.15.77:0/1793553468 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe80035f0 con 0x558caf18cb40 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.618+0000 7f4ff5ffb640 1 -- 172.21.15.77:0/1793553468 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8035040 con 0x558caf18cb40 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- 172.21.15.77:0/1793553468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 msgr2=0x558caf18e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80053f0 tx=0x7f4fe8047040 comp rx=0 tx=0).stop 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- 172.21.15.77:0/1793553468 shutdown_connections 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf18e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18cb40 0x558caf18e120 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 --2- 172.21.15.77:0/1793553468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18b5d0 0x558caf1a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- 172.21.15.77:0/1793553468 >> 172.21.15.77:0/1793553468 conn(0x558caf154310 msgr2=0x558caf1983d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- 172.21.15.77:0/1793553468 shutdown_connections 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- 172.21.15.77:0/1793553468 wait complete. 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.619+0000 7f5008b15740 1 -- start start 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 Processor -- start 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf190050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf190590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf190ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b2630 con 0x558caf18b5d0 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b1f60 con 0x558caf196370 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50001af1c0 con 0x558caf18cb40 2024-09-15T22:38:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5008b15740 10 client.0 _ll_register_callbacks cb 0x7f4ff009bfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf190050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf190ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf190050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58090/0 (socket says 172.21.15.77:58090) 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f4ffffff640 1 -- 172.21.15.77:0/3282487349 learned_addr learned my addr 172.21.15.77:0/3282487349 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f4fff7fe640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf190590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 -- 172.21.15.77:0/3282487349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 msgr2=0x558caf190590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf190590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 -- 172.21.15.77:0/3282487349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 msgr2=0x558caf190050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf190050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.620+0000 7f5004cd9640 1 -- 172.21.15.77:0/3282487349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caf19ca30 con 0x558caf196370 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f5004cd9640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf190ad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558caf18c070 tx=0x7f4ff8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f690be53573f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:01.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f4fdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801a040 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f4ff800fe00 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe80035f0 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ff8035050 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f5008b15740 10 client.29849 fetch_fsmap learned FSMap version 39 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558caf210e50 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.621+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe80035f0 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f4ff801d030 con 0x558caf196370 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 10 client.29849 fetch_fsmap finished waiting for FSMap version 39 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 10 client.29849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 20 client.29849 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf211620 0x558caf213a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:01.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 4 client.29849 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558caf0f8d70 con 0x558caf211620 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f4ffffff640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf211620 0x558caf213a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f4fdf7fe640 10 client.29849 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.622+0000 7f4ffffff640 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf211620 0x558caf213a60 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b9850a84b89d98 server_cookie=63d5e79a515a6f2a in_seq=0 out_seq=0 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f4fdf7fe640 1 -- 172.21.15.77:0/3282487349 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558caf0f8d70 con 0x558caf211620 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f4fdf7fe640 10 client.29849 handle_command_reply: tid=0 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 client.29849 shutdown 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 msgr2=0x558caf190ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf190ad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558caf18c070 tx=0x7f4ff8004ab0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 shutdown_connections 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x558caf211620 0x558caf213a60 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558caf196370 0x558caf190ad0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f4fdf7fe640 0 client.29849 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558caf18cb40 0x558caf190590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 --2- 172.21.15.77:0/3282487349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558caf18b5d0 0x558caf190050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 >> 172.21.15.77:0/3282487349 conn(0x558caf154310 msgr2=0x558caf186d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 shutdown_connections 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.623+0000 7f5008b15740 1 -- 172.21.15.77:0/3282487349 wait complete. 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f5008b15740 20 client.29849 trim_cache size 0 max 16384 2024-09-15T22:38:01.633 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 msgr2=0x7f4fec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8005fd0 tx=0x7f4ff8011b40 comp rx=0 tx=0).stop 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 msgr2=0x7f50000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000e9f0 tx=0x7f4ff000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 shutdown_connections 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5000106f10 0x7f5000074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5000102790 0x7f5000074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 --2- 172.21.15.77:0/526817540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5000101de0 0x7f50000744f0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 >> 172.21.15.77:0/526817540 conn(0x7f50000fd8c0 msgr2=0x7f50000fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 shutdown_connections 2024-09-15T22:38:01.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:01.624+0000 7f500673c640 1 -- 172.21.15.77:0/526817540 wait complete. 2024-09-15T22:38:02.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:02 smithi077 ceph-mon[30980]: pgmap v809: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 339 op/s 2024-09-15T22:38:02.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:02 smithi196 ceph-mon[35109]: pgmap v809: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 339 op/s 2024-09-15T22:38:02.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:02 smithi117 ceph-mon[35021]: pgmap v809: 129 pgs: 129 active+clean; 1.9 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.5 MiB/s wr, 339 op/s 2024-09-15T22:38:03.332 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-15T22:38:03.332 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T22:38:03.340 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:38:03.340 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:38:03.626 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:03.918 DEBUG:teuthology.parallel:result is None 2024-09-15T22:38:03.918 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T22:38:03.952 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T22:38:03.952 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-15T22:38:03.964 INFO:tasks.quiescer:joining Quiescers 2024-09-15T22:38:03.964 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-15T22:38:03.965 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:04.300 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:04 smithi077 ceph-mon[30980]: pgmap v810: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 367 op/s 2024-09-15T22:38:04.300 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:04 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:04.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 -- 172.21.15.77:0/991603289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4102ef0 msgr2=0x7f95b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 --2- 172.21.15.77:0/991603289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4102ef0 0x7f95b410da40 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009ca0 tx=0x7f95ac02a950 comp rx=0 tx=0).stop 2024-09-15T22:38:04.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 -- 172.21.15.77:0/991603289 shutdown_connections 2024-09-15T22:38:04.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 --2- 172.21.15.77:0/991603289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4102ef0 0x7f95b410da40 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 --2- 172.21.15.77:0/991603289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102570 0x7f95b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 --2- 172.21.15.77:0/991603289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4100e20 0x7f95b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 -- 172.21.15.77:0/991603289 >> 172.21.15.77:0/991603289 conn(0x7f95b40fc7d0 msgr2=0x7f95b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 -- 172.21.15.77:0/991603289 shutdown_connections 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.298+0000 7f95b926c640 1 -- 172.21.15.77:0/991603289 wait complete. 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.299+0000 7f95b926c640 1 Processor -- start 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.299+0000 7f95b926c640 1 -- start start 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.299+0000 7f95b926c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.299+0000 7f95b926c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4102570 0x7f95b4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 0x7f95b41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4110f80 con 0x7f95b4100e20 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4110e00 con 0x7f95b4102570 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4111100 con 0x7f95b4102ef0 2024-09-15T22:38:04.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58096/0 (socket says 172.21.15.77:58096) 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 -- 172.21.15.77:0/4075585286 learned_addr learned my addr 172.21.15.77:0/4075585286 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b27fc640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4102570 0x7f95b4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b37fe640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 0x7f95b41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 -- 172.21.15.77:0/4075585286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 msgr2=0x7f95b41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 0x7f95b41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 -- 172.21.15.77:0/4075585286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4102570 msgr2=0x7f95b4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4102570 0x7f95b4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 -- 172.21.15.77:0/4075585286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41a48b0 con 0x7f95b4100e20 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b37fe640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 0x7f95b41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d950 tx=0x7f95a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f959bfff640 1 -- 172.21.15.77:0/4075585286 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4017070 con 0x7f95b4100e20 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41a4a80 con 0x7f95b4100e20 2024-09-15T22:38:04.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f959bfff640 1 -- 172.21.15.77:0/4075585286 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a4009c30 con 0x7f95b4100e20 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f959bfff640 1 -- 172.21.15.77:0/4075585286 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a400f6b0 con 0x7f95b4100e20 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.300+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a4fc0 con 0x7f95b4100e20 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f959bfff640 1 -- 172.21.15.77:0/4075585286 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f95a40109d0 con 0x7f95b4100e20 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 Processor -- start 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 -- start start 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e429016350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429020c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e428f77280 con 0x55e429021150 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e428f1eb10 con 0x55e429016890 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95bb645740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e428f73240 con 0x55e429015f70 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429020c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58102/0 (socket says 172.21.15.77:58102) 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95b27fc640 1 -- 172.21.15.77:0/2620581743 learned_addr learned my addr 172.21.15.77:0/2620581743 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f95b37fe640 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e429016350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.303+0000 7f959bfff640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 0x7f959c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 -- 172.21.15.77:0/2620581743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 msgr2=0x55e429016350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e429016350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b37fe640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 0x7f959c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 -- 172.21.15.77:0/2620581743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 msgr2=0x55e429020c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429020c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 -- 172.21.15.77:0/2620581743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e428f2b1f0 con 0x55e429021150 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f959bfff640 1 -- 172.21.15.77:0/4075585286 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f95a40cfcc0 con 0x7f95b4100e20 2024-09-15T22:38:04.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b27fc640 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b690 tx=0x7f95a800bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2a3930897ac646 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95997fa640 1 -- 172.21.15.77:0/2620581743 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8003bd0 con 0x55e429021150 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95b37fe640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 0x7f959c0a5870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005fc0 tx=0x7f95ac02a950 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95997fa640 1 -- 172.21.15.77:0/2620581743 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8015070 con 0x55e429021150 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.304+0000 7f95997fa640 1 -- 172.21.15.77:0/2620581743 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004e00 con 0x55e429021150 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- 172.21.15.77:0/2620581743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 msgr2=0x55e429023540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b690 tx=0x7f95a800bb50 comp rx=0 tx=0).stop 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- 172.21.15.77:0/2620581743 shutdown_connections 2024-09-15T22:38:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429023540 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429020c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 --2- 172.21.15.77:0/2620581743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e429016350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- 172.21.15.77:0/2620581743 >> 172.21.15.77:0/2620581743 conn(0x55e429012500 msgr2=0x55e4290128e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- 172.21.15.77:0/2620581743 shutdown_connections 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- 172.21.15.77:0/2620581743 wait complete. 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 -- start start 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.305+0000 7f95bb645740 1 Processor -- start 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e42901e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e42901e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e429023d20 con 0x55e429021150 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e428f31f10 con 0x55e429015f70 2024-09-15T22:38:04.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e429023ea0 con 0x55e429016890 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 10 client.0 _ll_register_callbacks cb 0x55e428efaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58116/0 (socket says 172.21.15.77:58116) 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 -- 172.21.15.77:0/2080266356 learned_addr learned my addr 172.21.15.77:0/2080266356 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b27fc640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e42901e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e42901e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 -- 172.21.15.77:0/2080266356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 msgr2=0x55e42901e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e42901e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 -- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 msgr2=0x55e42901e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e42901e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 -- 172.21.15.77:0/2080266356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e428ce86a0 con 0x55e429021150 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95b37fe640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f95ac003bf0 tx=0x7f95ac0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9558feabb63542b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f9596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac034040 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a8015070 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.306+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f95ac004450 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95ac004650 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 10 client.21258 fetch_fsmap learned FSMap version 41 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e42902f960 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8015070 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f95ac049040 con 0x55e429021150 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 10 client.21258 fetch_fsmap finished waiting for FSMap version 41 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 10 client.21258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 20 client.21258 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e429030e20 0x55e429033210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 4 client.21258 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e428f847f0 con 0x55e429030e20 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e429030e20 0x55e429033210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f9596ff5640 10 client.21258 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.307+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e429030e20 0x55e429033210 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7382b4bbc6463113 server_cookie=a43da9fb07d14937 in_seq=0 out_seq=0 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f9596ff5640 1 -- 172.21.15.77:0/2080266356 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e428f847f0 con 0x55e429030e20 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f9596ff5640 10 client.21258 handle_command_reply: tid=0 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 client.21258 shutdown 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 msgr2=0x55e42901eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f95ac003bf0 tx=0x7f95ac0095c0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 shutdown_connections 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e429030e20 0x55e429033210 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e42901eac0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e42901e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 --2- 172.21.15.77:0/2080266356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e42901e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f9596ff5640 0 client.21258 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:04.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 >> 172.21.15.77:0/2080266356 conn(0x55e429012500 msgr2=0x55e42900d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 shutdown_connections 2024-09-15T22:38:04.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 1 -- 172.21.15.77:0/2080266356 wait complete. 2024-09-15T22:38:04.333 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.308+0000 7f95bb645740 20 client.21258 trim_cache size 0 max 16384 2024-09-15T22:38:04.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.325+0000 7f95bb645740 1 Processor -- start 2024-09-15T22:38:04.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.325+0000 7f95bb645740 1 -- start start 2024-09-15T22:38:04.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e42900d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429038a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41acdf0 con 0x55e429021150 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ae960 con 0x55e429016890 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95bb645740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ad4c0 con 0x55e429015f70 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58118/0 (socket says 172.21.15.77:58118) 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 -- 172.21.15.77:0/1333188927 learned_addr learned my addr 172.21.15.77:0/1333188927 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e42900d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b27fc640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 -- 172.21.15.77:0/1333188927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 msgr2=0x55e42900d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e42900d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 -- 172.21.15.77:0/1333188927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 msgr2=0x55e429038a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429038a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 -- 172.21.15.77:0/1333188927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42901cff0 con 0x55e429021150 2024-09-15T22:38:04.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b27fc640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:04.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f95b37fe640 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f95ac02ae70 tx=0x7f95ac02fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c214543b00fde19f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f9598ff9640 1 -- 172.21.15.77:0/1333188927 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac062070 con 0x55e429021150 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f9598ff9640 1 -- 172.21.15.77:0/1333188927 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac03ab50 con 0x55e429021150 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.326+0000 7f9598ff9640 1 -- 172.21.15.77:0/1333188927 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac030c50 con 0x55e429021150 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- 172.21.15.77:0/1333188927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 msgr2=0x55e429038fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f95ac02ae70 tx=0x7f95ac02fce0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- 172.21.15.77:0/1333188927 shutdown_connections 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e429038fb0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429016890 0x55e429038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 --2- 172.21.15.77:0/1333188927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429015f70 0x55e42900d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- 172.21.15.77:0/1333188927 >> 172.21.15.77:0/1333188927 conn(0x55e429012500 msgr2=0x55e42900ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- 172.21.15.77:0/1333188927 shutdown_connections 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- 172.21.15.77:0/1333188927 wait complete. 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.327+0000 7f95bb645740 1 -- start start 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 Processor -- start 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e429029120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e429029660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ae290 con 0x55e429021150 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41adb90 con 0x55e429015f70 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41aadf0 con 0x55e429016890 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95bb645740 10 client.0 _ll_register_callbacks cb 0x7f95ac02ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e429029120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58124/0 (socket says 172.21.15.77:58124) 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e429029660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b37fe640 1 -- 172.21.15.77:0/1000447492 learned_addr learned my addr 172.21.15.77:0/1000447492 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 -- 172.21.15.77:0/1000447492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 msgr2=0x55e429029660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e429029660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 -- 172.21.15.77:0/1000447492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 msgr2=0x55e4290260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 -- 172.21.15.77:0/1000447492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e429026630 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b37fe640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f95b2ffd640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e429029120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95a40cd080 tx=0x7f95a40d21d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8af0d8eb68d8717 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.328+0000 7f9596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40d1040 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f95a4098280 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ac03ab50 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f95a4020040 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 10 client.29864 fetch_fsmap learned FSMap version 41 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e42909c220 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac03ab50 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f95a40860b0 con 0x55e429015f70 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 10 client.29864 fetch_fsmap finished waiting for FSMap version 41 2024-09-15T22:38:04.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 10 client.29864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 20 client.29864 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e42909c990 0x55e42909ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 4 client.29864 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e428f7ed00 con 0x55e42909c990 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.329+0000 7f95b27fc640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e42909c990 0x55e42909ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.330+0000 7f9596ff5640 10 client.29864 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.330+0000 7f95b27fc640 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e42909c990 0x55e42909ed80 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc6bf610f63fff2 server_cookie=592529dd492bd491 in_seq=0 out_seq=0 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.330+0000 7f9596ff5640 1 -- 172.21.15.77:0/1000447492 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e428f7ed00 con 0x55e42909c990 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.330+0000 7f9596ff5640 10 client.29864 handle_command_reply: tid=0 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.330+0000 7f95bb645740 1 client.29864 shutdown 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 msgr2=0x55e429029120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e429029120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95a40cd080 tx=0x7f95a40d21d0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 shutdown_connections 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55e42909c990 0x55e42909ed80 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e429021150 0x55e4290260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e429016890 0x55e429029660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 --2- 172.21.15.77:0/1000447492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e429015f70 0x55e429029120 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 >> 172.21.15.77:0/1000447492 conn(0x55e429012500 msgr2=0x55e42900d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f9596ff5640 0 client.29864 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 shutdown_connections 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 1 -- 172.21.15.77:0/1000447492 wait complete. 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95bb645740 20 client.29864 trim_cache size 0 max 16384 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:04.346 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "0ffa1a84-c5aa-4b46-acf7-04fc0b6cee3e", 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 msgr2=0x7f959c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 0x7f959c0a5870 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005fc0 tx=0x7f95ac02a950 comp rx=0 tx=0).stop 2024-09-15T22:38:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 msgr2=0x7f95b41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.331+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d950 tx=0x7f95a400de10 comp rx=0 tx=0).stop 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 shutdown_connections 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f959c0a33b0 0x7f959c0a5870 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95b4102ef0 0x7f95b41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95b4102570 0x7f95b4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 --2- 172.21.15.77:0/4075585286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95b4100e20 0x7f95b41992f0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 >> 172.21.15.77:0/4075585286 conn(0x7f95b40fc7d0 msgr2=0x7f95b4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 shutdown_connections 2024-09-15T22:38:04.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:04.332+0000 7f95b926c640 1 -- 172.21.15.77:0/4075585286 wait complete. 2024-09-15T22:38:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:04 smithi196 ceph-mon[35109]: pgmap v810: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 367 op/s 2024-09-15T22:38:04.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:04 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:04 smithi117 ceph-mon[35021]: pgmap v810: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 367 op/s 2024-09-15T22:38:04.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:04 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:05.094 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:38:05.194 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:05.349 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:05.349 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: scrub queued for path: / 2024-09-15T22:38:05.349 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: scrub summary: active paths [/] 2024-09-15T22:38:05.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:05 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:05.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: scrub queued for path: / 2024-09-15T22:38:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: scrub summary: active paths [/] 2024-09-15T22:38:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:05 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: scrub queued for path: / 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: scrub summary: active paths [/] 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:05 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:05.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 -- 172.21.15.77:0/4272123146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 msgr2=0x7f0d30101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4272123146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d30101200 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009900 tx=0x7f0d2002a480 comp rx=0 tx=0).stop 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 -- 172.21.15.77:0/4272123146 shutdown_connections 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4272123146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102ef0 0x7f0d3010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4272123146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102570 0x7f0d301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4272123146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d30101200 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 -- 172.21.15.77:0/4272123146 >> 172.21.15.77:0/4272123146 conn(0x7f0d300fc7d0 msgr2=0x7f0d300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 -- 172.21.15.77:0/4272123146 shutdown_connections 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 -- 172.21.15.77:0/4272123146 wait complete. 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.710+0000 7f0d35e98640 1 Processor -- start 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 -- start start 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102570 0x7f0d300fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:05.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102ef0 0x7f0d300fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0d30110f50 con 0x7f0d30100e20 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d30110dd0 con 0x7f0d30102ef0 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d35e98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d301110d0 con 0x7f0d30102570 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58130/0 (socket says 172.21.15.77:58130) 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 -- 172.21.15.77:0/4082262737 learned_addr learned my addr 172.21.15.77:0/4082262737 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d27fff640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102570 0x7f0d300fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2ffff640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102ef0 0x7f0d300fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 -- 172.21.15.77:0/4082262737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102570 msgr2=0x7f0d300fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102570 0x7f0d300fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 -- 172.21.15.77:0/4082262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102ef0 msgr2=0x7f0d300fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102ef0 0x7f0d300fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 -- 172.21.15.77:0/4082262737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d301a6920 con 0x7f0d30100e20 2024-09-15T22:38:05.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.711+0000 7f0d2f7fe640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006010 tx=0x7f0d20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.712+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2003f070 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.712+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d301a6bb0 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.712+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d20031c30 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.712+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20035030 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.712+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d301a70e0 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.713+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d30101200 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.714+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d20042070 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.715+0000 7f0d2d7fa640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 0x7f0d1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.715+0000 7f0d27fff640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 0x7f0d1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.715+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9438+0+0 (secure 0 0 0) 0x7f0d200f09c0 con 0x7f0d30100e20 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.715+0000 7f0d27fff640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 0x7f0d1c0a5870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d18005e10 tx=0x7f0d18005da0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:05.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.719+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d200f2e20 con 0x7f0d30100e20 2024-09-15T22:38:05.880 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:05.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.933+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d301a72b0 con 0x7f0d30100e20 2024-09-15T22:38:05.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.934+0000 7f0d2d7fa640 1 -- 172.21.15.77:0/4082262737 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11991 (secure 0 0 0) 0x7f0d20031de0 con 0x7f0d30100e20 2024-09-15T22:38:05.937 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:05.937 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"btime":"2024-09-15T22:38:04:214041+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:38:03.867398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[14604,24458,24424,24439,24440]},"id":1}]} 2024-09-15T22:38:05.937 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.936+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 msgr2=0x7f0d1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.936+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 0x7f0d1c0a5870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d18005e10 tx=0x7f0d18005da0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.936+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 msgr2=0x7f0d300fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.936+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006010 tx=0x7f0d20003c10 comp rx=0 tx=0).stop 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 shutdown_connections 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f0d1c0a33b0 0x7f0d1c0a5870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d30102ef0 0x7f0d300fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d30102570 0x7f0d300fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 --2- 172.21.15.77:0/4082262737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d30100e20 0x7f0d300fcc80 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:05.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 >> 172.21.15.77:0/4082262737 conn(0x7f0d300fc7d0 msgr2=0x7f0d30069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:05.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 shutdown_connections 2024-09-15T22:38:05.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:05.937+0000 7f0d35e98640 1 -- 172.21.15.77:0/4082262737 wait complete. 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: pgmap v811: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: scrub summary: idle 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:38:06.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:06 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/4082262737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: pgmap v811: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-15T22:38:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: scrub summary: idle 2024-09-15T22:38:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:06.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:38:06.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:06 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/4082262737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: pgmap v811: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: scrub summary: idle 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' 2024-09-15T22:38:06.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:06 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/4082262737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:07.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:07 smithi077 ceph-mon[30980]: osdmap e130: 12 total, 12 up, 12 in 2024-09-15T22:38:07.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:07 smithi196 ceph-mon[35109]: osdmap e130: 12 total, 12 up, 12 in 2024-09-15T22:38:07.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:07 smithi117 ceph-mon[35021]: osdmap e130: 12 total, 12 up, 12 in 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.042+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/1150959793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08106a80 msgr2=0x7fdf08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/1150959793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08106a80 0x7fdf08106e60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009930 tx=0x7fdef802a420 comp rx=0 tx=0).stop 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/1150959793 shutdown_connections 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/1150959793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08101420 0x7fdf0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/1150959793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08100a80 0x7fdf08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/1150959793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08106a80 0x7fdf08106e60 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/1150959793 >> 172.21.15.77:0/1150959793 conn(0x7fdf080fc7a0 msgr2=0x7fdf080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/1150959793 shutdown_connections 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/1150959793 wait complete. 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 Processor -- start 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.043+0000 7fdf0dcaa640 1 -- start start 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08100a80 0x7fdf081992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08106a80 0x7fdf081a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08110ea0 con 0x7fdf08101420 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08110d20 con 0x7fdf08106a80 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08111020 con 0x7fdf08100a80 2024-09-15T22:38:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58138/0 (socket says 172.21.15.77:58138) 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 -- 172.21.15.77:0/3628521124 learned_addr learned my addr 172.21.15.77:0/3628521124 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf077fe640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08100a80 0x7fdf081992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 -- 172.21.15.77:0/3628521124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08100a80 msgr2=0x7fdf081992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08100a80 0x7fdf081992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 -- 172.21.15.77:0/3628521124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08106a80 msgr2=0x7fdf081a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08106a80 0x7fdf081a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 -- 172.21.15.77:0/3628521124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf081a4890 con 0x7fdf08101420 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdefffff640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fdef000d950 tx=0x7fdef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf057fa640 1 -- 172.21.15.77:0/3628521124 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0017070 con 0x7fdf08101420 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf057fa640 1 -- 172.21.15.77:0/3628521124 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef0009c30 con 0x7fdf08101420 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a4a60 con 0x7fdf08101420 2024-09-15T22:38:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf057fa640 1 -- 172.21.15.77:0/3628521124 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000f6b0 con 0x7fdf08101420 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.044+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a4f70 con 0x7fdf08101420 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf057fa640 1 -- 172.21.15.77:0/3628521124 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdef00109d0 con 0x7fdf08101420 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 Processor -- start 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 -- start start 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f63df582e0 con 0x55f63dffd680 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f63deffce0 con 0x55f63dff24a0 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.047+0000 7fdf10083740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f63df542a0 con 0x55f63dff2dc0 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58154/0 (socket says 172.21.15.77:58154) 2024-09-15T22:38:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 -- 172.21.15.77:0/2203007659 learned_addr learned my addr 172.21.15.77:0/2203007659 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf057fa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 0x7fdef40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf07fff640 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf077fe640 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf07fff640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 0x7fdef40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf057fa640 1 -- 172.21.15.77:0/3628521124 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7fdef00cfeb0 con 0x7fdf08101420 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 -- 172.21.15.77:0/2203007659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 msgr2=0x55f63dffd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 -- 172.21.15.77:0/2203007659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 msgr2=0x55f63dff2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 -- 172.21.15.77:0/2203007659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63dee4440 con 0x55f63dffd680 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdefffff640 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fdef009ba40 tx=0x7fdef0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fcbbf7773fb101 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.048+0000 7fdf07fff640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 0x7fdef40a5870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf00008fd0 tx=0x7fdf00006ce0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdefdffb640 1 -- 172.21.15.77:0/2203007659 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0020070 con 0x55f63dffd680 2024-09-15T22:38:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdefdffb640 1 -- 172.21.15.77:0/2203007659 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef009bd40 con 0x55f63dffd680 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdefdffb640 1 -- 172.21.15.77:0/2203007659 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00cd430 con 0x55f63dffd680 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 -- 172.21.15.77:0/2203007659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 msgr2=0x55f63dfffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fdef009ba40 tx=0x7fdef0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 -- 172.21.15.77:0/2203007659 shutdown_connections 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dfffa70 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 --2- 172.21.15.77:0/2203007659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 -- 172.21.15.77:0/2203007659 >> 172.21.15.77:0/2203007659 conn(0x55f63dfeea30 msgr2=0x55f63dfeee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 -- 172.21.15.77:0/2203007659 shutdown_connections 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.049+0000 7fdf10083740 1 -- 172.21.15.77:0/2203007659 wait complete. 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 -- start start 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 Processor -- start 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dffcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f63e000250 con 0x55f63dffd680 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f63df13050 con 0x55f63dff24a0 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f63e0003d0 con 0x55f63dff2dc0 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf10083740 10 client.0 _ll_register_callbacks cb 0x55f63df53160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58158/0 (socket says 172.21.15.77:58158) 2024-09-15T22:38:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.050+0000 7fdf07fff640 1 -- 172.21.15.77:0/903591694 learned_addr learned my addr 172.21.15.77:0/903591694 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf077fe640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dffcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdefffff640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 -- 172.21.15.77:0/903591694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 msgr2=0x55f63dffa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 -- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 msgr2=0x55f63dffcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dffcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 -- 172.21.15.77:0/903591694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63dcc93c0 con 0x55f63dffd680 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf07fff640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014f40 tx=0x7fdf00004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fd127cd19e08c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00028060 con 0x55f63dffd680 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef009bd40 con 0x55f63dffd680 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7fdf00019630 con 0x55f63dffd680 2024-09-15T22:38:08.078 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf0000de80 con 0x55f63dffd680 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf10083740 10 client.21282 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f63e00bbf0 con 0x55f63dffd680 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.051+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef009bd40 con 0x55f63dffd680 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fdf00022030 con 0x55f63dffd680 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 10 client.21282 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 10 client.21282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 20 client.21282 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63e00d3e0 0x55f63e00f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 4 client.21282 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63df65850 con 0x55f63e00d3e0 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf077fe640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63e00d3e0 0x55f63e00f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdee6ffd640 10 client.21282 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.052+0000 7fdf077fe640 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63e00d3e0 0x55f63e00f860 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb886997037a9ecb server_cookie=8ac2f359dee3bfe5 in_seq=0 out_seq=0 2024-09-15T22:38:08.079 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdee6ffd640 1 -- 172.21.15.77:0/903591694 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f63df65850 con 0x55f63e00d3e0 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdee6ffd640 10 client.21282 handle_command_reply: tid=0 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 client.21282 shutdown 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 msgr2=0x55f63dffa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014f40 tx=0x7fdf00004b50 comp rx=0 tx=0).stop 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 shutdown_connections 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63e00d3e0 0x55f63e00f860 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dffd680 0x55f63dffa710 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dff2dc0 0x55f63dffa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 --2- 172.21.15.77:0/903591694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dffcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 >> 172.21.15.77:0/903591694 conn(0x55f63dfeea30 msgr2=0x55f63dff4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdee6ffd640 0 client.21282 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 shutdown_connections 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 1 -- 172.21.15.77:0/903591694 wait complete. 2024-09-15T22:38:08.080 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.053+0000 7fdf10083740 20 client.21282 trim_cache size 0 max 16384 2024-09-15T22:38:08.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.071+0000 7fdf10083740 1 Processor -- start 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.071+0000 7fdf10083740 1 -- start start 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081ad030 con 0x55f63dff2dc0 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081aebd0 con 0x55f63dff24a0 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf10083740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081ad700 con 0x55f63dfeea30 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58162/0 (socket says 172.21.15.77:58162) 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 -- 172.21.15.77:0/2790868168 learned_addr learned my addr 172.21.15.77:0/2790868168 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdefffff640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf077fe640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.086 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 -- 172.21.15.77:0/2790868168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 msgr2=0x55f63e0094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 -- 172.21.15.77:0/2790868168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 msgr2=0x55f63dff8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 -- 172.21.15.77:0/2790868168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63dff9180 con 0x55f63dff2dc0 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf077fe640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdf07fff640 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000b990 tx=0x7fdf00006c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea03f0e3296b8ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.072+0000 7fdefd7fa640 1 -- 172.21.15.77:0/2790868168 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00028040 con 0x55f63dff2dc0 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdefd7fa640 1 -- 172.21.15.77:0/2790868168 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf00003e50 con 0x55f63dff2dc0 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdefd7fa640 1 -- 172.21.15.77:0/2790868168 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004040 con 0x55f63dff2dc0 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 -- 172.21.15.77:0/2790868168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 msgr2=0x55f63dffc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000b990 tx=0x7fdf00006c00 comp rx=0 tx=0).stop 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 -- 172.21.15.77:0/2790868168 shutdown_connections 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63dffc140 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63dff8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 --2- 172.21.15.77:0/2790868168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 -- 172.21.15.77:0/2790868168 >> 172.21.15.77:0/2790868168 conn(0x55f63dffa870 msgr2=0x55f63dffba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 -- 172.21.15.77:0/2790868168 shutdown_connections 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.073+0000 7fdf10083740 1 -- 172.21.15.77:0/2790868168 wait complete. 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 -- start start 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 Processor -- start 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63e001900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081ae500 con 0x55f63dff2dc0 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081ade00 con 0x55f63dff24a0 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081ab030 con 0x55f63dfeea30 2024-09-15T22:38:08.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf10083740 10 client.0 _ll_register_callbacks cb 0x7fdf00014bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58178/0 (socket says 172.21.15.77:58178) 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf07fff640 1 -- 172.21.15.77:0/252439099 learned_addr learned my addr 172.21.15.77:0/252439099 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdefffff640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63e001900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.074+0000 7fdf077fe640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 -- 172.21.15.77:0/252439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 msgr2=0x55f63e0013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 -- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 msgr2=0x55f63e001900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63e001900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 -- 172.21.15.77:0/252439099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63dff2bf0 con 0x55f63dff2dc0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf07fff640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fdf0001a870 tx=0x7fdf0000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d39165b88f29ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:08.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00040060 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7fdf00003970 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf00003e50 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf00047050 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf10083740 10 client.21288 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f63e0781e0 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.075+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf00003e50 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fdf00022050 con 0x55f63dff2dc0 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 10 client.21288 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 10 client.21288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 20 client.21288 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63dffd680 0x55f63e07a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 4 client.21288 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f63df5fd60 con 0x55f63dffd680 2024-09-15T22:38:08.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf077fe640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63dffd680 0x55f63e07a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdf077fe640 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63dffd680 0x55f63e07a890 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4016ad6c231d52 server_cookie=cb72f0db207d117b in_seq=0 out_seq=0 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.076+0000 7fdee6ffd640 10 client.21288 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdee6ffd640 1 -- 172.21.15.77:0/252439099 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f63df5fd60 con 0x55f63dffd680 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdee6ffd640 10 client.21288 handle_command_reply: tid=0 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 client.21288 shutdown 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 msgr2=0x55f63e001e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fdf0001a870 tx=0x7fdf0000b560 comp rx=0 tx=0).stop 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 shutdown_connections 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55f63dffd680 0x55f63e07a890 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f63dff2dc0 0x55f63e001e40 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63dff24a0 0x55f63e001900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 --2- 172.21.15.77:0/252439099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f63dfeea30 0x55f63e0013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 >> 172.21.15.77:0/252439099 conn(0x55f63dffa870 msgr2=0x55f63dffba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdee6ffd640 0 client.21288 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 shutdown_connections 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 1 -- 172.21.15.77:0/252439099 wait complete. 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf10083740 20 client.21288 trim_cache size 0 max 16384 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2024-09-15T22:38:08.092 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:08.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 msgr2=0x7fdef40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.077+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 0x7fdef40a5870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf00008fd0 tx=0x7fdf00006ce0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 msgr2=0x7fdf08199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fdef000d950 tx=0x7fdef000de10 comp rx=0 tx=0).stop 2024-09-15T22:38:08.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 shutdown_connections 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdef40a33b0 0x7fdef40a5870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf08106a80 0x7fdf081a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf08101420 0x7fdf08199810 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 --2- 172.21.15.77:0/3628521124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf08100a80 0x7fdf081992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 >> 172.21.15.77:0/3628521124 conn(0x7fdf080fc7a0 msgr2=0x7fdf080fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 shutdown_connections 2024-09-15T22:38:08.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:08.078+0000 7fdf0dcaa640 1 -- 172.21.15.77:0/3628521124 wait complete. 2024-09-15T22:38:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:08 smithi077 ceph-mon[30980]: pgmap v813: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-09-15T22:38:08.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:08 smithi077 ceph-mon[30980]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:08 smithi196 ceph-mon[35109]: pgmap v813: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-09-15T22:38:08.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:08 smithi196 ceph-mon[35109]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:08 smithi117 ceph-mon[35021]: pgmap v813: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 240 op/s 2024-09-15T22:38:08.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:08 smithi117 ceph-mon[35021]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-15T22:38:08.790 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24458 quiesce db --cancel --all' 2024-09-15T22:38:08.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24458 quiesce db --cancel --all 2024-09-15T22:38:09.810 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:38:09.810 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:09.965 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:10.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:10 smithi077 ceph-mon[30980]: pgmap v814: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-09-15T22:38:10.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:10 smithi196 ceph-mon[35109]: pgmap v814: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-09-15T22:38:10.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:10 smithi117 ceph-mon[35021]: pgmap v814: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 2.0 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.4 MiB/s wr, 243 op/s 2024-09-15T22:38:11.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 -- 172.21.15.77:0/3247946324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 msgr2=0x7f4724106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 --2- 172.21.15.77:0/3247946324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f4724106e60 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f4720009930 tx=0x7f472002a420 comp rx=0 tx=0).stop 2024-09-15T22:38:11.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 -- 172.21.15.77:0/3247946324 shutdown_connections 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 --2- 172.21.15.77:0/3247946324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724101420 0x7f472410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 --2- 172.21.15.77:0/3247946324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724100a80 0x7f4724100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 --2- 172.21.15.77:0/3247946324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f4724106e60 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 -- 172.21.15.77:0/3247946324 >> 172.21.15.77:0/3247946324 conn(0x7f47240fc7a0 msgr2=0x7f47240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 -- 172.21.15.77:0/3247946324 shutdown_connections 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 -- 172.21.15.77:0/3247946324 wait complete. 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.606+0000 7f472c0bf640 1 Processor -- start 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 -- start start 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 0x7f4724102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4724110e80 con 0x7f4724106a80 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4724110d00 con 0x7f4724101420 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472c0bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4724111000 con 0x7f4724100a80 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f4729e5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472965a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 0x7f4724102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f4729e5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43440/0 (socket says 172.21.15.77:43440) 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f4729e5b640 1 -- 172.21.15.77:0/133445844 learned_addr learned my addr 172.21.15.77:0/133445844 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37012/0 (socket says 172.21.15.77:37012) 2024-09-15T22:38:11.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472965a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 0x7f4724102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.77:56420/0 (socket says 172.21.15.77:56420) 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 -- 172.21.15.77:0/133445844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 msgr2=0x7f4724102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 -- 172.21.15.77:0/133445844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 msgr2=0x7f4724102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 0x7f4724102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 -- 172.21.15.77:0/133445844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47241a68f0 con 0x7f4724106a80 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f4729e5b640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.607+0000 7f472a65c640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e940 tx=0x7f471c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.608+0000 7f471affd640 1 -- 172.21.15.77:0/133445844 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c004d20 con 0x7f4724106a80 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.608+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241a6be0 con 0x7f4724106a80 2024-09-15T22:38:11.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.608+0000 7f471affd640 1 -- 172.21.15.77:0/133445844 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471c005020 con 0x7f4724106a80 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.608+0000 7f471affd640 1 -- 172.21.15.77:0/133445844 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0116b0 con 0x7f4724106a80 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.608+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47241a7120 con 0x7f4724106a80 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.610+0000 7f471affd640 1 -- 172.21.15.77:0/133445844 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f471c0054d0 con 0x7f4724106a80 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.610+0000 7f472e498740 1 Processor -- start 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.610+0000 7f472e498740 1 -- start start 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef056d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef0562e200 con 0x55ef056c9a00 2024-09-15T22:38:11.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef055d4dc0 con 0x55ef056d4be0 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472e498740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef0562a1c0 con 0x55ef056ca320 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f471affd640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 0x7f47100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.77:43450/0 (socket says 172.21.15.77:43450) 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 -- 172.21.15.77:0/3697775279 learned_addr learned my addr 172.21.15.77:0/3697775279 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472a65c640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 0x7f47100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f472965a640 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef056d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f471affd640 1 -- 172.21.15.77:0/133445844 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7f471c0d0100 con 0x7f4724106a80 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 -- 172.21.15.77:0/3697775279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 msgr2=0x55ef056d6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef056d6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 -- 172.21.15.77:0/3697775279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 msgr2=0x55ef056c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.611+0000 7f4729e5b640 1 -- 172.21.15.77:0/3697775279 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0539f620 con 0x55ef056ca320 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.612+0000 7f4729e5b640 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4720009a50 tx=0x7f4720031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a65c24533d62282 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.612+0000 7f472a65c640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 0x7f47100a5870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f471c0cd510 tx=0x7f471c091530 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.612+0000 7f470ffff640 1 -- 172.21.15.77:0/3697775279 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720035040 con 0x55ef056ca320 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.612+0000 7f470ffff640 1 -- 172.21.15.77:0/3697775279 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47200319d0 con 0x55ef056ca320 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.612+0000 7f470ffff640 1 -- 172.21.15.77:0/3697775279 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720004a00 con 0x55ef056ca320 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- 172.21.15.77:0/3697775279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 msgr2=0x55ef056d46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4720009a50 tx=0x7f4720031660 comp rx=0 tx=0).stop 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- 172.21.15.77:0/3697775279 shutdown_connections 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef056d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d46a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- 172.21.15.77:0/3697775279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- 172.21.15.77:0/3697775279 >> 172.21.15.77:0/3697775279 conn(0x55ef056c5f90 msgr2=0x55ef056c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- 172.21.15.77:0/3697775279 shutdown_connections 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- 172.21.15.77:0/3697775279 wait complete. 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- start start 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 Processor -- start 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.613+0000 7f472e498740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef056d77b0 con 0x55ef056d4be0 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472e498740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef055c8c50 con 0x55ef056ca320 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472e498740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ef056d7930 con 0x55ef056c9a00 2024-09-15T22:38:11.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472e498740 10 client.0 _ll_register_callbacks cb 0x55ef055daba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37026/0 (socket says 172.21.15.77:37026) 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 -- 172.21.15.77:0/2252240715 learned_addr learned my addr 172.21.15.77:0/2252240715 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f4729e5b640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 -- 172.21.15.77:0/2252240715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 msgr2=0x55ef056d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 -- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 msgr2=0x55ef056d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 -- 172.21.15.77:0/2252240715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef056d2470 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f4729e5b640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472965a640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4714012e00 tx=0x7f471400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255613f7d71a15a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f470d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714004040 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7f47140041e0 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47200319d0 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4714024080 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.614+0000 7f472e498740 10 client.21300 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ef056e3380 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47200319d0 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f4714021040 con 0x55ef056d4be0 2024-09-15T22:38:11.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 10 client.21300 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 10 client.21300 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 20 client.21300 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef056e4840 0x55ef056e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 4 client.21300 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef0563b770 con 0x55ef056e4840 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.615+0000 7f4729e5b640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef056e4840 0x55ef056e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f4729e5b640 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef056e4840 0x55ef056e6c30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee8fa1167500163 server_cookie=381ed3977f306d28 in_seq=0 out_seq=0 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f470d7fa640 10 client.21300 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f470d7fa640 1 -- 172.21.15.77:0/2252240715 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef0563b770 con 0x55ef056e4840 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f470d7fa640 10 client.21300 handle_command_reply: tid=0 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 client.21300 shutdown 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 msgr2=0x55ef056d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4714012e00 tx=0x7f471400b390 comp rx=0 tx=0).stop 2024-09-15T22:38:11.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 shutdown_connections 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef056e4840 0x55ef056e6c30 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056d1c70 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f470d7fa640 0 client.21300 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 --2- 172.21.15.77:0/2252240715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 >> 172.21.15.77:0/2252240715 conn(0x55ef056c5f90 msgr2=0x55ef056c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 shutdown_connections 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.616+0000 7f472e498740 1 -- 172.21.15.77:0/2252240715 wait complete. 2024-09-15T22:38:11.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.617+0000 7f472e498740 20 client.21300 trim_cache size 0 max 16384 2024-09-15T22:38:11.638 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 Processor -- start 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 -- start start 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b1310 con 0x55ef056d4be0 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b2eb0 con 0x55ef056ca320 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.635+0000 7f472e498740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b19e0 con 0x55ef056c9a00 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37028/0 (socket says 172.21.15.77:37028) 2024-09-15T22:38:11.639 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 -- 172.21.15.77:0/3355000367 learned_addr learned my addr 172.21.15.77:0/3355000367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f4729e5b640 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 -- 172.21.15.77:0/3355000367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 msgr2=0x55ef056e0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056e0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 -- 172.21.15.77:0/3355000367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 msgr2=0x55ef056ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 -- 172.21.15.77:0/3355000367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef056d06e0 con 0x55ef056d4be0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f472965a640 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f471400ebb0 tx=0x7f4714012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6410b55579d83e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f470f7fe640 1 -- 172.21.15.77:0/3355000367 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140104b0 con 0x55ef056d4be0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f470f7fe640 1 -- 172.21.15.77:0/3355000367 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4714010650 con 0x55ef056d4be0 2024-09-15T22:38:11.640 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.636+0000 7f470f7fe640 1 -- 172.21.15.77:0/3355000367 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714005030 con 0x55ef056d4be0 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- 172.21.15.77:0/3355000367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 msgr2=0x55ef056ec4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f471400ebb0 tx=0x7f4714012590 comp rx=0 tx=0).stop 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- 172.21.15.77:0/3355000367 shutdown_connections 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056d4be0 0x55ef056ec4c0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056ca320 0x55ef056ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 --2- 172.21.15.77:0/3355000367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056c9a00 0x55ef056e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- 172.21.15.77:0/3355000367 >> 172.21.15.77:0/3355000367 conn(0x55ef056c5f90 msgr2=0x55ef056cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- 172.21.15.77:0/3355000367 shutdown_connections 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- 172.21.15.77:0/3355000367 wait complete. 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.637+0000 7f472e498740 1 -- start start 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 Processor -- start 2024-09-15T22:38:11.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef0574d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b27e0 con 0x55ef056c9a00 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b20e0 con 0x55ef056d4be0 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47241af310 con 0x55ef056ca320 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472e498740 10 client.0 _ll_register_callbacks cb 0x7f4714012a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37032/0 (socket says 172.21.15.77:37032) 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 -- 172.21.15.77:0/390422342 learned_addr learned my addr 172.21.15.77:0/390422342 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef0574d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f4729e5b640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 -- 172.21.15.77:0/390422342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 msgr2=0x55ef056d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 -- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 msgr2=0x55ef0574d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef0574d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.638+0000 7f472965a640 1 -- 172.21.15.77:0/390422342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0574d9c0 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f472965a640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f471400eef0 tx=0x7f4714010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3057ec66f78aba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140104b0 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7f4714005500 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4714010650 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f47140035f0 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f472e498740 10 client.21306 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ef0574ec20 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4714010650 con 0x55ef056c9a00 2024-09-15T22:38:11.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.639+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f471402f6b0 con 0x55ef056c9a00 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 10 client.21306 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 10 client.21306 resolve_mds: validated gid 24458 aka daemon mds.b 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 20 client.21306 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef0574f240 0x55ef05751630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 4 client.21306 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55ef05635c80 con 0x55ef0574f240 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f4729e5b640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef0574f240 0x55ef05751630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f4729e5b640 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef0574f240 0x55ef05751630 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a620eb1addd53cc1 server_cookie=24b6f8d13e37885f in_seq=0 out_seq=0 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f470d7fa640 10 client.21306 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f470d7fa640 1 -- 172.21.15.77:0/390422342 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55ef05635c80 con 0x55ef0574f240 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f470d7fa640 10 client.21306 handle_command_reply: tid=0 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.640+0000 7f472e498740 1 client.21306 shutdown 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 msgr2=0x55ef056d8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f471400eef0 tx=0x7f4714010b50 comp rx=0 tx=0).stop 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 shutdown_connections 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55ef0574f240 0x55ef05751630 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef056d4be0 0x55ef0574d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ef056ca320 0x55ef056d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f470d7fa640 0 client.21306 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 --2- 172.21.15.77:0/390422342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef056c9a00 0x55ef056d8110 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 >> 172.21.15.77:0/390422342 conn(0x55ef056c5f90 msgr2=0x55ef056d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 shutdown_connections 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 1 -- 172.21.15.77:0/390422342 wait complete. 2024-09-15T22:38:11.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472e498740 20 client.21306 trim_cache size 0 max 16384 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 42, 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24458, 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 88, 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": {} 2024-09-15T22:38:11.651 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.641+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 msgr2=0x7f47100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 0x7f47100a5870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f471c0cd510 tx=0x7f471c091530 comp rx=0 tx=0).stop 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 msgr2=0x7f47241030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e940 tx=0x7f471c00ee00 comp rx=0 tx=0).stop 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 shutdown_connections 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47100a33b0 0x7f47100a5870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4724106a80 0x7f47241030d0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4724101420 0x7f4724102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 --2- 172.21.15.77:0/133445844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4724100a80 0x7f4724102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:11.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 >> 172.21.15.77:0/133445844 conn(0x7f47240fc7a0 msgr2=0x7f472406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:11.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 shutdown_connections 2024-09-15T22:38:11.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:11.642+0000 7f472c0bf640 1 -- 172.21.15.77:0/133445844 wait complete. 2024-09-15T22:38:12.311 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:12 smithi077 ceph-mon[30980]: pgmap v815: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-09-15T22:38:12.343 INFO:tasks.quiescer:done joining Quiescers 2024-09-15T22:38:12.343 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-15T22:38:12.353 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-15T22:38:12.353 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:38:12.353 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:38:12.353 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-15T22:38:12.383 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:12.408 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-15T22:38:12.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:38:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:12 smithi196 ceph-mon[35109]: pgmap v815: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-09-15T22:38:12.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:12 smithi117 ceph-mon[35021]: pgmap v815: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 235 op/s 2024-09-15T22:38:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- 172.21.15.77:0/437158315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 msgr2=0x7f90f0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 --2- 172.21.15.77:0/437158315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0100e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009930 tx=0x7f90e402a420 comp rx=0 tx=0).stop 2024-09-15T22:38:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- 172.21.15.77:0/437158315 shutdown_connections 2024-09-15T22:38:12.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 --2- 172.21.15.77:0/437158315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 0x7f90f010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 --2- 172.21.15.77:0/437158315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0100e10 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 --2- 172.21.15.77:0/437158315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 0x7f90f0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- 172.21.15.77:0/437158315 >> 172.21.15.77:0/437158315 conn(0x7f90f0077990 msgr2=0x7f90f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- 172.21.15.77:0/437158315 shutdown_connections 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- 172.21.15.77:0/437158315 wait complete. 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 Processor -- start 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.829+0000 7f90f610a640 1 -- start start 2024-09-15T22:38:12.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 0x7f90f01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 0x7f90f019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110e00 con 0x7f90f01009b0 2024-09-15T22:38:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110c80 con 0x7f90f0101350 2024-09-15T22:38:12.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0110f80 con 0x7f90f01069c0 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37058/0 (socket says 172.21.15.77:37058) 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 -- 172.21.15.77:0/1884069694 learned_addr learned my addr 172.21.15.77:0/1884069694 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90eeffd640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 0x7f90f01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90effff640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 0x7f90f019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 -- 172.21.15.77:0/1884069694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 msgr2=0x7f90f019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 0x7f90f019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 -- 172.21.15.77:0/1884069694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 msgr2=0x7f90f01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 0x7f90f01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 -- 172.21.15.77:0/1884069694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019c270 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ef7fe640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c970 tx=0x7f90e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0007bb0 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e0007d50 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000f450 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019c560 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.830+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a3df0 con 0x7f90f01009b0 2024-09-15T22:38:12.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.832+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f0100e10 con 0x7f90f01009b0 2024-09-15T22:38:12.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.833+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f90e0005020 con 0x7f90f01009b0 2024-09-15T22:38:12.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.833+0000 7f90ecff9640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 0x7f90d80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.834+0000 7f90eeffd640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 0x7f90d80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.834+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7f90e00cfd80 con 0x7f90f01009b0 2024-09-15T22:38:12.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.834+0000 7f90eeffd640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 0x7f90d80a59a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e40058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:12.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.838+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90e0014030 con 0x7f90f01009b0 2024-09-15T22:38:13.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 -- 172.21.15.77:0/3644460226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf01082a0 msgr2=0x7f2bf010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/3644460226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf01082a0 0x7f2bf010ec40 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009cd0 tx=0x7f2be802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 -- 172.21.15.77:0/3644460226 shutdown_connections 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/3644460226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf01082a0 0x7f2bf010ec40 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/3644460226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 0x7f2bf0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/3644460226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf0102d30 0x7f2bf0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 -- 172.21.15.77:0/3644460226 >> 172.21.15.77:0/3644460226 conn(0x7f2bf00fc7d0 msgr2=0x7f2bf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 -- 172.21.15.77:0/3644460226 shutdown_connections 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.970+0000 7f2bf65f1640 1 -- 172.21.15.77:0/3644460226 wait complete. 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 Processor -- start 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 -- start start 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 0x7f2bf01a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf01082a0 0x7f2bf019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01120a0 con 0x7f2bf0102d30 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0111f20 con 0x7f2bf01082a0 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf65f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0112220 con 0x7f2bf01036e0 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37080/0 (socket says 172.21.15.77:37080) 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 -- 172.21.15.77:0/2060797848 learned_addr learned my addr 172.21.15.77:0/2060797848 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2bf4b8e640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf01082a0 0x7f2bf019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2be7fff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 0x7f2bf01a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 -- 172.21.15.77:0/2060797848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 msgr2=0x7f2bf01a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 0x7f2bf01a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 -- 172.21.15.77:0/2060797848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf01082a0 msgr2=0x7f2bf019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf01082a0 0x7f2bf019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.971+0000 7f2beffff640 1 -- 172.21.15.77:0/2060797848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf019ad00 con 0x7f2bf0102d30 2024-09-15T22:38:13.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2beffff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000cd80 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf019aff0 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be0010910 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be00188f0 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.972+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a72e0 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.973+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0103110 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.974+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2be0010ac0 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.975+0000 7f2bedffb640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 0x7f2bdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.975+0000 7f2be7fff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 0x7f2bdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.975+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7f2be00d0510 con 0x7f2bf0102d30 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.975+0000 7f2be7fff640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 0x7f2bdc0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00fdf00 tx=0x7f2bd8005f90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:12.979+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2be009cb40 con 0x7f2bf0102d30 2024-09-15T22:38:13.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.032+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f90f0062630 con 0x7f90d80a34e0 2024-09-15T22:38:13.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.045+0000 7f90ecff9640 1 -- 172.21.15.77:0/1884069694 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f90f0062630 con 0x7f90d80a34e0 2024-09-15T22:38:13.048 INFO:teuthology.orchestra.run.smithi077.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:12:34", "first": "2024-09-15T22:13:00", "last": "2024-09-15T22:38:00", "last_pruned": "2024-09-15T22:38:00", "created_count": 26, "pruned_count": 20, "active": true} 2024-09-15T22:38:13.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.047+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 msgr2=0x7f90d80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.047+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 0x7f90d80a59a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e40058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 msgr2=0x7f90f0197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c970 tx=0x7f90e000ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 shutdown_connections 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f90d80a34e0 0x7f90d80a59a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90f01069c0 0x7f90f019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0101350 0x7f90f01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 --2- 172.21.15.77:0/1884069694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90f01009b0 0x7f90f0197280 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 >> 172.21.15.77:0/1884069694 conn(0x7f90f0077990 msgr2=0x7f90f0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 shutdown_connections 2024-09-15T22:38:13.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.048+0000 7f90f610a640 1 -- 172.21.15.77:0/1884069694 wait complete. 2024-09-15T22:38:13.062 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 -- 172.21.15.77:0/128553056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7641009b0 msgr2=0x7fe764100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 --2- 172.21.15.77:0/128553056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7641009b0 0x7fe764100e10 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009930 tx=0x7fe74c02a420 comp rx=0 tx=0).stop 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 -- 172.21.15.77:0/128553056 shutdown_connections 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 --2- 172.21.15.77:0/128553056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764101350 0x7fe76410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 --2- 172.21.15.77:0/128553056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7641009b0 0x7fe764100e10 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 --2- 172.21.15.77:0/128553056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641069c0 0x7fe764106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 -- 172.21.15.77:0/128553056 >> 172.21.15.77:0/128553056 conn(0x7fe764077990 msgr2=0x7fe7640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 -- 172.21.15.77:0/128553056 shutdown_connections 2024-09-15T22:38:13.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.163+0000 7fe769077640 1 -- 172.21.15.77:0/128553056 wait complete. 2024-09-15T22:38:13.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 Processor -- start 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 -- start start 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641009b0 0x7fe7640fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 0x7fe7640fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe764110e00 con 0x7fe764101350 2024-09-15T22:38:13.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe764110c80 con 0x7fe7641069c0 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe769077640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe764110f80 con 0x7fe7641009b0 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe76259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe76259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37098/0 (socket says 172.21.15.77:37098) 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.164+0000 7fe76259c640 1 -- 172.21.15.77:0/3692730506 learned_addr learned my addr 172.21.15.77:0/3692730506 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe762d9d640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641009b0 0x7fe7640fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76359e640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 0x7fe7640fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 -- 172.21.15.77:0/3692730506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641009b0 msgr2=0x7fe7640fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641009b0 0x7fe7640fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 -- 172.21.15.77:0/3692730506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 msgr2=0x7fe7640fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 0x7fe7640fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 -- 172.21.15.77:0/3692730506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641a6850 con 0x7fe764101350 2024-09-15T22:38:13.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76359e640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 0x7fe7640fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe76259c640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009230 tx=0x7fe74c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c03f030 con 0x7fe764101350 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe74c003ad0 con 0x7fe764101350 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641a6ae0 con 0x7fe764101350 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c035070 con 0x7fe764101350 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.165+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a6fd0 con 0x7fe764101350 2024-09-15T22:38:13.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.166+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764100e10 con 0x7fe764101350 2024-09-15T22:38:13.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.168+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe74c042030 con 0x7fe764101350 2024-09-15T22:38:13.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.169+0000 7fe75bfff640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 0x7fe7500a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.169+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9526+0+0 (secure 0 0 0) 0x7fe74c0f0a00 con 0x7fe764101350 2024-09-15T22:38:13.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.169+0000 7fe762d9d640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 0x7fe7500a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.169+0000 7fe762d9d640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 0x7fe7500a5920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe75400bb60 tx=0x7fe75400b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.172+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe74c0bcff0 con 0x7fe764101350 2024-09-15T22:38:13.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.190+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bf010c2c0 con 0x7f2bf0102d30 2024-09-15T22:38:13.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.191+0000 7f2bedffb640 1 -- 172.21.15.77:0/2060797848 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11993 (secure 0 0 0) 0x7f2be009c440 con 0x7f2bf0102d30 2024-09-15T22:38:13.194 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:13.194 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":42,"btime":"2024-09-15T22:38:07:243258+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:38:06.428603+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:38:13.194 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 42 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 msgr2=0x7f2bdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 0x7f2bdc0a5870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00fdf00 tx=0x7f2bd8005f90 comp rx=0 tx=0).stop 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 msgr2=0x7f2bf01a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 shutdown_connections 2024-09-15T22:38:13.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f2bdc0a33b0 0x7f2bdc0a5870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bf01082a0 0x7f2bf019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf01036e0 0x7f2bf01a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 --2- 172.21.15.77:0/2060797848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bf0102d30 0x7f2bf01a0410 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 >> 172.21.15.77:0/2060797848 conn(0x7f2bf00fc7d0 msgr2=0x7f2bf00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 shutdown_connections 2024-09-15T22:38:13.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.194+0000 7f2bf65f1640 1 -- 172.21.15.77:0/2060797848 wait complete. 2024-09-15T22:38:13.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.369+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 --> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe7641036b0 con 0x7fe7500a3460 2024-09-15T22:38:13.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.381+0000 7fe75bfff640 1 -- 172.21.15.77:0/3692730506 <== mgr.15309 v2:172.21.15.77:6800/3746048513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe7641036b0 con 0x7fe7500a3460 2024-09-15T22:38:13.383 INFO:teuthology.orchestra.run.smithi077.stdout:/ 1m 6m3h 2024-09-15T22:38:13.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 msgr2=0x7fe7500a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 0x7fe7500a5920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe75400bb60 tx=0x7fe75400b390 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 msgr2=0x7fe7640fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009230 tx=0x7fe74c004270 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 shutdown_connections 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fe7500a3460 0x7fe7500a5920 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7641069c0 0x7fe7640fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe764101350 0x7fe7640fd320 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 --2- 172.21.15.77:0/3692730506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7641009b0 0x7fe7640fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 >> 172.21.15.77:0/3692730506 conn(0x7fe764077990 msgr2=0x7fe764069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 shutdown_connections 2024-09-15T22:38:13.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:13.383+0000 7fe769077640 1 -- 172.21.15.77:0/3692730506 wait complete. 2024-09-15T22:38:13.396 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-15T22:38:13.406 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-15T22:38:13.563 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:13 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2060797848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:13.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:13 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2060797848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:13.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:13 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2060797848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:13.933 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-15T22:38:14.178 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:14 smithi077 ceph-mon[30980]: pgmap v816: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 259 op/s 2024-09-15T22:38:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:14 smithi077 ceph-mon[30980]: from='client.21312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:14 smithi077 ceph-mon[30980]: from='client.21324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:14 smithi077 ceph-mon[30980]: osdmap e131: 12 total, 12 up, 12 in 2024-09-15T22:38:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:14 smithi196 ceph-mon[35109]: pgmap v816: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 259 op/s 2024-09-15T22:38:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:14 smithi196 ceph-mon[35109]: from='client.21312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:14 smithi196 ceph-mon[35109]: from='client.21324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:14 smithi196 ceph-mon[35109]: osdmap e131: 12 total, 12 up, 12 in 2024-09-15T22:38:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:14 smithi117 ceph-mon[35021]: pgmap v816: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 259 op/s 2024-09-15T22:38:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:14 smithi117 ceph-mon[35021]: from='client.21312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:14 smithi117 ceph-mon[35021]: from='client.21324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:14.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:14 smithi117 ceph-mon[35021]: osdmap e131: 12 total, 12 up, 12 in 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- 172.21.15.77:0/258767248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac102790 msgr2=0x7f44ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 --2- 172.21.15.77:0/258767248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac102790 0x7f44ac1067e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f449c009900 tx=0x7f449c02a410 comp rx=0 tx=0).stop 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- 172.21.15.77:0/258767248 shutdown_connections 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 --2- 172.21.15.77:0/258767248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac106f10 0x7f44ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 --2- 172.21.15.77:0/258767248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac102790 0x7f44ac1067e0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 --2- 172.21.15.77:0/258767248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac101de0 0x7f44ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- 172.21.15.77:0/258767248 >> 172.21.15.77:0/258767248 conn(0x7f44ac0fd8c0 msgr2=0x7f44ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- 172.21.15.77:0/258767248 shutdown_connections 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- 172.21.15.77:0/258767248 wait complete. 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 Processor -- start 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.698+0000 7f44b39c3640 1 -- start start 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac102790 0x7f44ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 0x7f44ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac111f90 con 0x7f44ac101de0 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac111e10 con 0x7f44ac106f10 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac112110 con 0x7f44ac102790 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37116/0 (socket says 172.21.15.77:37116) 2024-09-15T22:38:14.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 -- 172.21.15.77:0/3268550220 learned_addr learned my addr 172.21.15.77:0/3268550220 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 0x7f44ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac102790 0x7f44ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 -- 172.21.15.77:0/3268550220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac102790 msgr2=0x7f44ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac102790 0x7f44ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 -- 172.21.15.77:0/3268550220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 msgr2=0x7f44ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 0x7f44ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 -- 172.21.15.77:0/3268550220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac1a69e0 con 0x7f44ac101de0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 0x7f44ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b175f640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f44a800d950 tx=0x7f44a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44a27fc640 1 -- 172.21.15.77:0/3268550220 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8017030 con 0x7f44ac101de0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac1a6cd0 con 0x7f44ac101de0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44a27fc640 1 -- 172.21.15.77:0/3268550220 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a8004020 con 0x7f44ac101de0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44a27fc640 1 -- 172.21.15.77:0/3268550220 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80047a0 con 0x7f44ac101de0 2024-09-15T22:38:14.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.699+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1a7200 con 0x7f44ac101de0 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.701+0000 7f44a27fc640 1 -- 172.21.15.77:0/3268550220 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f44a80041d0 con 0x7f44ac101de0 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 Processor -- start 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 -- start start 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44a27fc640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 0x7f44980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da598c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da5999760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556da58f22f0 con 0x556da598c5d0 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556da5899cf0 con 0x556da5997370 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556da58ee2b0 con 0x556da598db40 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b0f5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 0x7f44980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b0f5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37130/0 (socket says 172.21.15.77:37130) 2024-09-15T22:38:14.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44a27fc640 1 -- 172.21.15.77:0/3268550220 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f44a80cfb40 con 0x7f44ac101de0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.702+0000 7f44b0f5e640 1 -- 172.21.15.77:0/2168552485 learned_addr learned my addr 172.21.15.77:0/2168552485 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b175f640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da598c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 -- 172.21.15.77:0/2168552485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 msgr2=0x556da598c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da598c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 -- 172.21.15.77:0/2168552485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 msgr2=0x556da5999760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b1f60640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da5999760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da5999760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 -- 172.21.15.77:0/2168552485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da587e3a0 con 0x556da598c5d0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b1f60640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da5999760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 0x7f44980a5870 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c470 tx=0x7f44a40099d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f449c009900 tx=0x7f449c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775043630fb0e63f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44977fe640 1 -- 172.21.15.77:0/2168552485 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c03f070 con 0x556da598c5d0 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44977fe640 1 -- 172.21.15.77:0/2168552485 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c0317b0 con 0x556da598c5d0 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.703+0000 7f44977fe640 1 -- 172.21.15.77:0/2168552485 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c035070 con 0x556da598c5d0 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- 172.21.15.77:0/2168552485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 msgr2=0x556da598ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f449c009900 tx=0x7f449c032ce0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- 172.21.15.77:0/2168552485 shutdown_connections 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da5999760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da598ca10 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/2168552485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da598c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- 172.21.15.77:0/2168552485 >> 172.21.15.77:0/2168552485 conn(0x556da5955310 msgr2=0x556da5988c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- 172.21.15.77:0/2168552485 shutdown_connections 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- 172.21.15.77:0/2168552485 wait complete. 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.704+0000 7f44b5d9c740 1 -- start start 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 Processor -- start 2024-09-15T22:38:14.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598c5d0 0x556da5996da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556da599a010 con 0x556da598db40 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556da58ad060 con 0x556da598c5d0 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556da599a190 con 0x556da5997370 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b5d9c740 10 client.0 _ll_register_callbacks cb 0x556da56008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37142/0 (socket says 172.21.15.77:37142) 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 -- 172.21.15.77:0/902246164 learned_addr learned my addr 172.21.15.77:0/902246164 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b175f640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598c5d0 0x556da5996da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b1f60640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 -- 172.21.15.77:0/902246164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 msgr2=0x556da59947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 -- 172.21.15.77:0/902246164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598c5d0 msgr2=0x556da5996da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598c5d0 0x556da5996da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.705+0000 7f44b0f5e640 1 -- 172.21.15.77:0/902246164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da56633c0 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f449c031550 tx=0x7f449c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589a08b9eeff153f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c03f070 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f449c004440 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c0317b0 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f449c054050 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 10 client.21336 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556da59a5a90 con 0x556da598db40 2024-09-15T22:38:14.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c0317b0 con 0x556da598db40 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f449c004a90 con 0x556da598db40 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 10 client.21336 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 10 client.21336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 20 client.21336 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da59a6de0 0x556da59a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 4 client.21336 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.706+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556da58ff860 con 0x556da59a6de0 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b175f640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da59a6de0 0x556da59a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b175f640 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da59a6de0 0x556da59a91d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=460d62f51983cba server_cookie=539f9af2d870099 in_seq=0 out_seq=0 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f4494ff9640 10 client.21336 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f4494ff9640 1 -- 172.21.15.77:0/902246164 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556da58ff860 con 0x556da59a6de0 2024-09-15T22:38:14.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f4494ff9640 10 client.21336 handle_command_reply: tid=0 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b5d9c740 1 client.21336 shutdown 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 msgr2=0x556da5994050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f449c031550 tx=0x7f449c0315b0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.707+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 shutdown_connections 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da59a6de0 0x556da59a91d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598db40 0x556da5994050 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/902246164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598c5d0 0x556da5996da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f4494ff9640 0 client.21336 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 >> 172.21.15.77:0/902246164 conn(0x556da5955310 msgr2=0x556da598e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 shutdown_connections 2024-09-15T22:38:14.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 1 -- 172.21.15.77:0/902246164 wait complete. 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.708+0000 7f44b5d9c740 20 client.21336 trim_cache size 0 max 16384 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.720+0000 7f44b5d9c740 1 Processor -- start 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.720+0000 7f44b5d9c740 1 -- start start 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 0x556da5993910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1b11e0 con 0x556da598c5d0 2024-09-15T22:38:14.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1b2d80 con 0x556da598db40 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1b18b0 con 0x556da5997370 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37144/0 (socket says 172.21.15.77:37144) 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 -- 172.21.15.77:0/3587212908 learned_addr learned my addr 172.21.15.77:0/3587212908 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 0x556da5993910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 -- 172.21.15.77:0/3587212908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 msgr2=0x556da59ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 -- 172.21.15.77:0/3587212908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 msgr2=0x556da5993910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 0x556da5993910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 -- 172.21.15.77:0/3587212908 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da5994b50 con 0x556da598c5d0 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b1f60640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:14.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 0x556da5993910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f44b175f640 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009610 tx=0x7f44a8097e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e32f980d1cba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f4496ffd640 1 -- 172.21.15.77:0/3587212908 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a809fe40 con 0x556da598c5d0 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f4496ffd640 1 -- 172.21.15.77:0/3587212908 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a80a4700 con 0x556da598c5d0 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.721+0000 7f4496ffd640 1 -- 172.21.15.77:0/3587212908 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8086070 con 0x556da598c5d0 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- 172.21.15.77:0/3587212908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 msgr2=0x556da5988e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009610 tx=0x7f44a8097e60 comp rx=0 tx=0).stop 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- 172.21.15.77:0/3587212908 shutdown_connections 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da5997370 0x556da59ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da598db40 0x556da5993910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/3587212908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da5988e10 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- 172.21.15.77:0/3587212908 >> 172.21.15.77:0/3587212908 conn(0x556da5955310 msgr2=0x556da5988700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- 172.21.15.77:0/3587212908 shutdown_connections 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- 172.21.15.77:0/3587212908 wait complete. 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 -- start start 2024-09-15T22:38:14.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.722+0000 7f44b5d9c740 1 Processor -- start 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da59a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da59aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1b26b0 con 0x556da598c5d0 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1b1fb0 con 0x556da5997370 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1af1e0 con 0x556da598db40 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b5d9c740 10 client.0 _ll_register_callbacks cb 0x7f44a8004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37150/0 (socket says 172.21.15.77:37150) 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 -- 172.21.15.77:0/1808792906 learned_addr learned my addr 172.21.15.77:0/1808792906 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da59a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b1f60640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da59aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 -- 172.21.15.77:0/1808792906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 msgr2=0x556da59a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da59a9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 -- 172.21.15.77:0/1808792906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 msgr2=0x556da59aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da59aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 -- 172.21.15.77:0/1808792906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da59aac70 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b1f60640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da59aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f44b175f640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f44a80a41d0 tx=0x7f44a809fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a26e10c948fa62 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.723+0000 7f4494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80cd180 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a80a4700 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f44a80860c0 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f44a8098100 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 10 client.21342 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556da5a117c0 con 0x556da598c5d0 2024-09-15T22:38:14.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a80a4700 con 0x556da598c5d0 2024-09-15T22:38:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f44a809bcf0 con 0x556da598c5d0 2024-09-15T22:38:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 10 client.21342 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 10 client.21342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-15T22:38:14.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 20 client.21342 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da5a11dd0 0x556da5a141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 4 client.21342 mds_command: new command op to 14604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 --> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556da58f9d70 con 0x556da5a11dd0 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.724+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da5a11dd0 0x556da5a141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f4494ff9640 10 client.21342 ms_handle_connect on v2:172.21.15.77:6840/3032989520 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b0f5e640 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da5a11dd0 0x556da5a141c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb955b6ca6f8021d server_cookie=844d93ef7122a4fc in_seq=0 out_seq=0 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f4494ff9640 1 -- 172.21.15.77:0/1808792906 <== mds.4 v2:172.21.15.77:6840/3032989520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556da58f9d70 con 0x556da5a11dd0 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f4494ff9640 10 client.21342 handle_command_reply: tid=0 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 client.21342 shutdown 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 msgr2=0x556da59aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f44a80a41d0 tx=0x7f44a809fe40 comp rx=0 tx=0).stop 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 shutdown_connections 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:6840/3032989520,v1:172.21.15.77:6841/3032989520] conn(0x556da5a11dd0 0x556da5a141c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556da5997370 0x556da59aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556da598db40 0x556da59a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 --2- 172.21.15.77:0/1808792906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556da598c5d0 0x556da59aca90 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 >> 172.21.15.77:0/1808792906 conn(0x556da5955310 msgr2=0x556da59958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f4494ff9640 0 client.21342 ms_handle_reset on v2:172.21.15.77:6840/3032989520 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 shutdown_connections 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 1 -- 172.21.15.77:0/1808792906 wait complete. 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.725+0000 7f44b5d9c740 20 client.21342 trim_cache size 0 max 16384 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 msgr2=0x7f44980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 0x7f44980a5870 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c470 tx=0x7f44a40099d0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 msgr2=0x7f44ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f44a800d950 tx=0x7f44a800de10 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 shutdown_connections 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f44980a33b0 0x7f44980a5870 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ac106f10 0x7f44ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44ac102790 0x7f44ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 --2- 172.21.15.77:0/3268550220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44ac101de0 0x7f44ac0744f0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 >> 172.21.15.77:0/3268550220 conn(0x7f44ac0fd8c0 msgr2=0x7f44ac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 shutdown_connections 2024-09-15T22:38:14.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:14.726+0000 7f44b39c3640 1 -- 172.21.15.77:0/3268550220 wait complete. 2024-09-15T22:38:16.358 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:16 smithi077 ceph-mon[30980]: pgmap v818: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 5.6 MiB/s wr, 225 op/s 2024-09-15T22:38:16.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:16 smithi196 ceph-mon[35109]: pgmap v818: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 5.6 MiB/s wr, 225 op/s 2024-09-15T22:38:16.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:16 smithi117 ceph-mon[35021]: pgmap v818: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 5.6 MiB/s wr, 225 op/s 2024-09-15T22:38:17.391 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-15T22:38:17.649 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 -- 172.21.15.77:0/2247341980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4106a80 msgr2=0x7fadf4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 --2- 172.21.15.77:0/2247341980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4106a80 0x7fadf4106e60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7faddc009930 tx=0x7faddc02a420 comp rx=0 tx=0).stop 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 -- 172.21.15.77:0/2247341980 shutdown_connections 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 --2- 172.21.15.77:0/2247341980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4101420 0x7fadf410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 --2- 172.21.15.77:0/2247341980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4100a80 0x7fadf4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 --2- 172.21.15.77:0/2247341980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4106a80 0x7fadf4106e60 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 -- 172.21.15.77:0/2247341980 >> 172.21.15.77:0/2247341980 conn(0x7fadf40fc7a0 msgr2=0x7fadf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 -- 172.21.15.77:0/2247341980 shutdown_connections 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.138+0000 7fadf92d6640 1 -- 172.21.15.77:0/2247341980 wait complete. 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 Processor -- start 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 -- start start 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4100a80 0x7fadf41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4106a80 0x7fadf419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4110ec0 con 0x7fadf4101420 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4110d40 con 0x7fadf4100a80 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf92d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4111040 con 0x7fadf4106a80 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37162/0 (socket says 172.21.15.77:37162) 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf27fc640 1 -- 172.21.15.77:0/696366025 learned_addr learned my addr 172.21.15.77:0/696366025 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:18.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.139+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4100a80 0x7fadf41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf37fe640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4106a80 0x7fadf419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 -- 172.21.15.77:0/696366025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4106a80 msgr2=0x7fadf419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4106a80 0x7fadf419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 -- 172.21.15.77:0/696366025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4100a80 msgr2=0x7fadf41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4100a80 0x7fadf41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 -- 172.21.15.77:0/696366025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf419d490 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf27fc640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fade400d770 tx=0x7fade400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadebfff640 1 -- 172.21.15.77:0/696366025 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4017070 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadebfff640 1 -- 172.21.15.77:0/696366025 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4005500 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf419d780 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadebfff640 1 -- 172.21.15.77:0/696366025 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4004e00 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.140+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41a5060 con 0x7fadf4101420 2024-09-15T22:38:18.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.142+0000 7fadebfff640 1 -- 172.21.15.77:0/696366025 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fade4004fa0 con 0x7fadf4101420 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 Processor -- start 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 -- start start 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 0x55e1360a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a55d0 0x55e1360a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e13600b2f0 con 0x55e1360b0370 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e135fb2cf0 con 0x55e1360a6b40 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadfb6af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e1360072b0 con 0x55e1360a55d0 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37176/0 (socket says 172.21.15.77:37176) 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadf27fc640 1 -- 172.21.15.77:0/2588390793 learned_addr learned my addr 172.21.15.77:0/2588390793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a55d0 0x55e1360a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.143+0000 7fadf37fe640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 0x55e1360a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadebfff640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 0x7fade00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf37fe640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 0x7fade00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadebfff640 1 -- 172.21.15.77:0/696366025 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fade40cfb90 con 0x7fadf4101420 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 -- 172.21.15.77:0/2588390793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a55d0 msgr2=0x55e1360a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a55d0 0x55e1360a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 -- 172.21.15.77:0/2588390793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 msgr2=0x55e1360a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 0x55e1360a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 -- 172.21.15.77:0/2588390793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e135f973a0 con 0x55e1360b0370 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf37fe640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 0x55e1360a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf27fc640 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fade40cce30 tx=0x7fade4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201c397539901f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fade97fa640 1 -- 172.21.15.77:0/2588390793 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4020070 con 0x55e1360b0370 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fadf37fe640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 0x7fade00a5870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fadec009fd0 tx=0x7fadec004a40 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fade97fa640 1 -- 172.21.15.77:0/2588390793 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade40046c0 con 0x55e1360b0370 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.144+0000 7fade97fa640 1 -- 172.21.15.77:0/2588390793 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade40d4040 con 0x55e1360b0370 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- 172.21.15.77:0/2588390793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 msgr2=0x55e1360b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fade40cce30 tx=0x7fade4009c30 comp rx=0 tx=0).stop 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- 172.21.15.77:0/2588390793 shutdown_connections 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360b2760 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a55d0 0x55e1360a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 --2- 172.21.15.77:0/2588390793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a6b40 0x55e1360a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- 172.21.15.77:0/2588390793 >> 172.21.15.77:0/2588390793 conn(0x55e13606e310 msgr2=0x55e1360a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- 172.21.15.77:0/2588390793 shutdown_connections 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- 172.21.15.77:0/2588390793 wait complete. 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.145+0000 7fadfb6af740 1 -- start start 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 Processor -- start 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e1360b2fb0 con 0x55e1360b0370 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e135fc6060 con 0x55e1360a55d0 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e1360b3130 con 0x55e1360a6b40 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadfb6af740 10 client.0 _ll_register_callbacks cb 0x55e135d198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37178/0 (socket says 172.21.15.77:37178) 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 -- 172.21.15.77:0/3864963791 learned_addr learned my addr 172.21.15.77:0/3864963791 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf27fc640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 -- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 msgr2=0x55e1360ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 -- 172.21.15.77:0/3864963791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 msgr2=0x55e1360afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 -- 172.21.15.77:0/3864963791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135d7c3c0 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadf37fe640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fadec00f040 tx=0x7fadec004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990470ff811dd0db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.146+0000 7fadda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec020070 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fadec025660 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e136018860 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fadec02e050 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 10 client.21354 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e1360beb30 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e136018860 con 0x55e1360b0370 2024-09-15T22:38:18.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fadec01a390 con 0x55e1360b0370 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 10 client.21354 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 10 client.21354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 20 client.21354 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e1360bfff0 0x55e1360c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 4 client.21354 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e136012d70 con 0x55e1360bfff0 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.147+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e1360bfff0 0x55e1360c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.148+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e1360bfff0 0x55e1360c2420 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d57e3a4a4e546484 server_cookie=bf92fa1016758bf8 in_seq=0 out_seq=0 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.148+0000 7fadda7fc640 10 client.21354 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.148+0000 7fadda7fc640 1 -- 172.21.15.77:0/3864963791 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e136012d70 con 0x55e1360bfff0 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.148+0000 7fadda7fc640 10 client.21354 handle_command_reply: tid=0 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.148+0000 7fadfb6af740 1 client.21354 shutdown 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 msgr2=0x55e1360ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fadec00f040 tx=0x7fadec004b50 comp rx=0 tx=0).stop 2024-09-15T22:38:18.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 shutdown_connections 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e1360bfff0 0x55e1360c2420 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360b0370 0x55e1360ad650 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 --2- 172.21.15.77:0/3864963791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadda7fc640 0 client.21354 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 >> 172.21.15.77:0/3864963791 conn(0x55e13606e310 msgr2=0x55e1360a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 shutdown_connections 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 1 -- 172.21.15.77:0/3864963791 wait complete. 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.149+0000 7fadfb6af740 20 client.21354 trim_cache size 0 max 16384 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.161+0000 7fadfb6af740 1 Processor -- start 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.161+0000 7fadfb6af740 1 -- start start 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360b0370 0x55e1360b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41aef90 con 0x55e1360a6b40 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41b0b30 con 0x55e1360a55d0 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadfb6af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41af660 con 0x55e1360b0370 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37182/0 (socket says 172.21.15.77:37182) 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 -- 172.21.15.77:0/1129253173 learned_addr learned my addr 172.21.15.77:0/1129253173 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf37fe640 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360b0370 0x55e1360b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 -- 172.21.15.77:0/1129253173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360b0370 msgr2=0x55e1360b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360b0370 0x55e1360b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 -- 172.21.15.77:0/1129253173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 msgr2=0x55e1360a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 -- 172.21.15.77:0/1129253173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1360abff0 con 0x55e1360a6b40 2024-09-15T22:38:18.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fadf27fc640 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fade400d740 tx=0x7fade40ccf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a554a72b9c064bda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fade8ff9640 1 -- 172.21.15.77:0/1129253173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4020040 con 0x55e1360a6b40 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.162+0000 7fade8ff9640 1 -- 172.21.15.77:0/1129253173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade409ba80 con 0x55e1360a6b40 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fade8ff9640 1 -- 172.21.15.77:0/1129253173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade40980f0 con 0x55e1360a6b40 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- 172.21.15.77:0/1129253173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 msgr2=0x55e1360b87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fade400d740 tx=0x7fade40ccf10 comp rx=0 tx=0).stop 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- 172.21.15.77:0/1129253173 shutdown_connections 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360b0370 0x55e1360b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a6b40 0x55e1360b87a0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 --2- 172.21.15.77:0/1129253173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360a55d0 0x55e1360a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- 172.21.15.77:0/1129253173 >> 172.21.15.77:0/1129253173 conn(0x55e13606e310 msgr2=0x55e1360a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- 172.21.15.77:0/1129253173 shutdown_connections 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- 172.21.15.77:0/1129253173 wait complete. 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.163+0000 7fadfb6af740 1 -- start start 2024-09-15T22:38:18.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 Processor -- start 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360b0370 0x55e1360a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41b0460 con 0x55e1360a55d0 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41afd60 con 0x55e1360b0370 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41acf90 con 0x55e1360a6b40 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37188/0 (socket says 172.21.15.77:37188) 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadfb6af740 10 client.0 _ll_register_callbacks cb 0x7fadec00cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 -- 172.21.15.77:0/542458516 learned_addr learned my addr 172.21.15.77:0/542458516 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf27fc640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf37fe640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360b0370 0x55e1360a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 -- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 msgr2=0x55e1360a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 -- 172.21.15.77:0/542458516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360b0370 msgr2=0x55e1360a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360b0370 0x55e1360a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.164+0000 7fadf2ffd640 1 -- 172.21.15.77:0/542458516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1360aa740 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadf2ffd640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7faddc009900 tx=0x7faddc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3ac6ec5ccc4a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc035040 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7faddc005020 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e135f8f170 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faddc005220 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 10 client.21360 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e13612f500 con 0x55e1360a55d0 2024-09-15T22:38:18.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e135f8f170 con 0x55e1360a55d0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7faddc0035f0 con 0x55e1360a55d0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 10 client.21360 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 10 client.21360 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 20 client.21360 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e13612fbd0 0x55e136131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 4 client.21360 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.165+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 --> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e135fdc9b0 con 0x55e13612fbd0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadf27fc640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e13612fbd0 0x55e136131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadf27fc640 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e13612fbd0 0x55e136131fc0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1d16f36b276608d server_cookie=690463c19a528e66 in_seq=0 out_seq=0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadda7fc640 10 client.21360 ms_handle_connect on v2:172.21.15.196:6836/3513024753 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadda7fc640 1 -- 172.21.15.77:0/542458516 <== mds.3 v2:172.21.15.196:6836/3513024753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e135fdc9b0 con 0x55e13612fbd0 2024-09-15T22:38:18.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadda7fc640 10 client.21360 handle_command_reply: tid=0 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadfb6af740 1 client.21360 shutdown 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 msgr2=0x55e1360a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.166+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7faddc009900 tx=0x7faddc031400 comp rx=0 tx=0).stop 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 shutdown_connections 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:6836/3513024753,v1:172.21.15.196:6837/3513024753] conn(0x55e13612fbd0 0x55e136131fc0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1360b0370 0x55e1360a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1360a6b40 0x55e1360a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 --2- 172.21.15.77:0/542458516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1360a55d0 0x55e1360a9490 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadda7fc640 0 client.21360 ms_handle_reset on v2:172.21.15.196:6836/3513024753 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 >> 172.21.15.77:0/542458516 conn(0x55e13606e310 msgr2=0x55e1360a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 shutdown_connections 2024-09-15T22:38:18.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 1 -- 172.21.15.77:0/542458516 wait complete. 2024-09-15T22:38:18.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadfb6af740 20 client.21360 trim_cache size 0 max 16384 2024-09-15T22:38:18.186 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 msgr2=0x7fade00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 0x7fade00a5870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fadec009fd0 tx=0x7fadec004a40 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 msgr2=0x7fadf4198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fade400d770 tx=0x7fade400dc30 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 shutdown_connections 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fade00a33b0 0x7fade00a5870 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadf4106a80 0x7fadf419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf4101420 0x7fadf4198a20 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 --2- 172.21.15.77:0/696366025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4100a80 0x7fadf41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.167+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 >> 172.21.15.77:0/696366025 conn(0x7fadf40fc7a0 msgr2=0x7fadf406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.187 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.168+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 shutdown_connections 2024-09-15T22:38:18.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:18.168+0000 7fadf92d6640 1 -- 172.21.15.77:0/696366025 wait complete. 2024-09-15T22:38:18.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:18 smithi077 ceph-mon[30980]: pgmap v819: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.8 MiB/s wr, 193 op/s 2024-09-15T22:38:18.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:18 smithi196 ceph-mon[35109]: pgmap v819: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.8 MiB/s wr, 193 op/s 2024-09-15T22:38:18.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:18 smithi117 ceph-mon[35021]: pgmap v819: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 4.8 MiB/s wr, 193 op/s 2024-09-15T22:38:18.983 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:38:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:19.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 -- 172.21.15.77:0/2640585722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0101de0 msgr2=0x7fa0c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 --2- 172.21.15.77:0/2640585722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0101de0 0x7fa0c01021c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009900 tx=0x7fa0b402a410 comp rx=0 tx=0).stop 2024-09-15T22:38:19.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 -- 172.21.15.77:0/2640585722 shutdown_connections 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 --2- 172.21.15.77:0/2640585722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0106f10 0x7fa0c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 --2- 172.21.15.77:0/2640585722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 0x7fa0c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 --2- 172.21.15.77:0/2640585722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0101de0 0x7fa0c01021c0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 -- 172.21.15.77:0/2640585722 >> 172.21.15.77:0/2640585722 conn(0x7fa0c00fd8c0 msgr2=0x7fa0c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 -- 172.21.15.77:0/2640585722 shutdown_connections 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.717+0000 7fa0c7239640 1 -- 172.21.15.77:0/2640585722 wait complete. 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 Processor -- start 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 -- start start 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0101de0 0x7fa0c0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 0x7fa0c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0111f90 con 0x7fa0c0106f10 2024-09-15T22:38:19.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0111e10 con 0x7fa0c0101de0 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c7239640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0112110 con 0x7fa0c0102790 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45998/0 (socket says 172.21.15.77:45998) 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 -- 172.21.15.77:0/3649350436 learned_addr learned my addr 172.21.15.77:0/3649350436 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0bffff640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 0x7fa0c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0101de0 0x7fa0c0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 -- 172.21.15.77:0/3649350436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 msgr2=0x7fa0c01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 0x7fa0c01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 -- 172.21.15.77:0/3649350436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0101de0 msgr2=0x7fa0c0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0101de0 0x7fa0c0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.755 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 -- 172.21.15.77:0/3649350436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019d430 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b970 tx=0x7fa0b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.718+0000 7fa0bdffb640 1 -- 172.21.15.77:0/3649350436 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8019070 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.719+0000 7fa0bdffb640 1 -- 172.21.15.77:0/3649350436 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b800cb40 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.719+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019d720 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.719+0000 7fa0bdffb640 1 -- 172.21.15.77:0/3649350436 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8007650 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.719+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a5000 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.721+0000 7fa0bdffb640 1 -- 172.21.15.77:0/3649350436 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa0b801c070 con 0x7fa0c0106f10 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.721+0000 7fa0c9612740 1 Processor -- start 2024-09-15T22:38:19.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.721+0000 7fa0c9612740 1 -- start start 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.721+0000 7fa0bdffb640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 0x7fa0b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5635d4b182f0 con 0x5635d4bb25d0 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5635d4abfcf0 con 0x5635d4bb3b40 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c9612740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635d4b142b0 con 0x5635d4bc1780 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 0x7fa0b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bdffb640 1 -- 172.21.15.77:0/3649350436 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fa0b80cf5f0 con 0x7fa0c0106f10 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46012/0 (socket says 172.21.15.77:46012) 2024-09-15T22:38:19.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 -- 172.21.15.77:0/585115438 learned_addr learned my addr 172.21.15.77:0/585115438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 -- 172.21.15.77:0/585115438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 msgr2=0x5635d4bc3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 -- 172.21.15.77:0/585115438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 msgr2=0x5635d4bb2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bb2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 -- 172.21.15.77:0/585115438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d4aa43a0 con 0x5635d4bb25d0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 0x7fa0b00a5870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac012960 tx=0x7fa0ac00bb70 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0bffff640 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac010740 tx=0x7fa0ac01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47fd901190303a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.722+0000 7fa0aaffd640 1 -- 172.21.15.77:0/585115438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac024070 con 0x5635d4bb25d0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0aaffd640 1 -- 172.21.15.77:0/585115438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac010e00 con 0x5635d4bb25d0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0aaffd640 1 -- 172.21.15.77:0/585115438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac020970 con 0x5635d4bb25d0 2024-09-15T22:38:19.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 -- 172.21.15.77:0/585115438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 msgr2=0x5635d4bb2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac010740 tx=0x7fa0ac01c000 comp rx=0 tx=0).stop 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 -- 172.21.15.77:0/585115438 shutdown_connections 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bb2a10 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 --2- 172.21.15.77:0/585115438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 -- 172.21.15.77:0/585115438 >> 172.21.15.77:0/585115438 conn(0x5635d4b7b310 msgr2=0x5635d4baec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 -- 172.21.15.77:0/585115438 shutdown_connections 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.723+0000 7fa0c9612740 1 -- 172.21.15.77:0/585115438 wait complete. 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 -- start start 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 Processor -- start 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 0x5635d4bc11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bbe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5635d4bc4420 con 0x5635d4bc1780 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5635d4ad3060 con 0x5635d4bb3b40 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5635d4bc45a0 con 0x5635d4bb25d0 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c9612740 10 client.0 _ll_register_callbacks cb 0x5635d48268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46014/0 (socket says 172.21.15.77:46014) 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 -- 172.21.15.77:0/1636470027 learned_addr learned my addr 172.21.15.77:0/1636470027 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 0x5635d4bc11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 -- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 msgr2=0x5635d4bc11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 0x5635d4bc11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0bffff640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bbe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 -- 172.21.15.77:0/1636470027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 msgr2=0x5635d4bbe470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bbe470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c57d6640 1 -- 172.21.15.77:0/1636470027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d48893c0 con 0x5635d4bc1780 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.724+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 0x5635d4bc11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80ccac0 tx=0x7fa0b8012800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd94cfd9f395b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b4060 con 0x5635d4bc1780 2024-09-15T22:38:19.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fa0b80cf180 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ac010e00 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0b80ccdd0 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 10 client.21372 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5635d4bcffc0 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac010e00 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fa0b80b6ce0 con 0x5635d4bc1780 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 10 client.21372 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 10 client.21372 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 20 client.21372 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4bd1420 0x5635d4bd3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 4 client.21372 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.725+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635d4b25860 con 0x5635d4bd1420 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.726+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4bd1420 0x5635d4bd3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.726+0000 7fa09ffff640 10 client.21372 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.726+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4bd1420 0x5635d4bd3880 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db41aff5fe0b3080 server_cookie=816b192a36d8146f in_seq=0 out_seq=0 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa09ffff640 1 -- 172.21.15.77:0/1636470027 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635d4b25860 con 0x5635d4bd1420 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa09ffff640 10 client.21372 handle_command_reply: tid=0 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 client.21372 shutdown 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 msgr2=0x5635d4bbec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80ccac0 tx=0x7fa0b8012800 comp rx=0 tx=0).stop 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 shutdown_connections 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4bd1420 0x5635d4bd3880 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bc1780 0x5635d4bbec00 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bbe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 --2- 172.21.15.77:0/1636470027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb25d0 0x5635d4bc11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa09ffff640 0 client.21372 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 >> 172.21.15.77:0/1636470027 conn(0x5635d4b7b310 msgr2=0x5635d4bb4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 shutdown_connections 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 1 -- 172.21.15.77:0/1636470027 wait complete. 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.727+0000 7fa0c9612740 20 client.21372 trim_cache size 0 max 16384 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 Processor -- start 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 -- start start 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb3b40 0x5635d4bbdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 0x5635d4bd9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01acd60 con 0x5635d4bb25d0 2024-09-15T22:38:19.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01ae900 con 0x5635d4bc1780 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c9612740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01ad430 con 0x5635d4bb3b40 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46026/0 (socket says 172.21.15.77:46026) 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/3607278385 learned_addr learned my addr 172.21.15.77:0/3607278385 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0bffff640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb3b40 0x5635d4bbdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 0x5635d4bd9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/3607278385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb3b40 msgr2=0x5635d4bbdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb3b40 0x5635d4bbdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/3607278385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 msgr2=0x5635d4bd9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 0x5635d4bd9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/3607278385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d4bafdd0 con 0x5635d4bb25d0 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.740+0000 7fa0c57d6640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 0x5635d4bd9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:19.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009900 tx=0x7fa0b40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d784f0095345acce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0aa7fc640 1 -- 172.21.15.77:0/3607278385 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b404a070 con 0x5635d4bb25d0 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0aa7fc640 1 -- 172.21.15.77:0/3607278385 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b4005500 con 0x5635d4bb25d0 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0aa7fc640 1 -- 172.21.15.77:0/3607278385 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4035040 con 0x5635d4bb25d0 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 -- 172.21.15.77:0/3607278385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 msgr2=0x5635d4baee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009900 tx=0x7fa0b40058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 -- 172.21.15.77:0/3607278385 shutdown_connections 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bc1780 0x5635d4bd9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bb3b40 0x5635d4bbdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 --2- 172.21.15.77:0/3607278385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4baee10 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 -- 172.21.15.77:0/3607278385 >> 172.21.15.77:0/3607278385 conn(0x5635d4b7b310 msgr2=0x5635d4bae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 -- 172.21.15.77:0/3607278385 shutdown_connections 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.741+0000 7fa0c9612740 1 -- 172.21.15.77:0/3607278385 wait complete. 2024-09-15T22:38:19.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 -- start start 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 Processor -- start 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01ae230 con 0x5635d4bb25d0 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01adb30 con 0x5635d4bb3b40 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01aad60 con 0x5635d4bc1780 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c9612740 10 client.0 _ll_register_callbacks cb 0x7fa0b8097300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46032/0 (socket says 172.21.15.77:46032) 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/2744141040 learned_addr learned my addr 172.21.15.77:0/2744141040 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:19.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 msgr2=0x5635d4bc6050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc6050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/2744141040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 msgr2=0x5635d4bc5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.742+0000 7fa0c4fd5640 1 -- 172.21.15.77:0/2744141040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d4c3b120 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c4fd5640 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009230 tx=0x7fa0b4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09e26cefb96edc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4035050 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fa0b4004e70 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b4005500 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0b404a040 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 10 client.21378 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5635d4c3c2d0 con 0x5635d4bb25d0 2024-09-15T22:38:19.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b4005500 con 0x5635d4bb25d0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fa0b403dbd0 con 0x5635d4bb25d0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 10 client.21378 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 10 client.21378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 20 client.21378 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4c3c9c0 0x5635d4c3ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 4 client.21378 mds_command: new command op to 24439 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.743+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 --> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635d4b1fd70 con 0x5635d4c3c9c0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0bffff640 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4c3c9c0 0x5635d4c3ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0bffff640 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4c3c9c0 0x5635d4c3ee40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f7c6092bf6192a1 server_cookie=5a037252c880e7ec in_seq=0 out_seq=0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa09ffff640 10 client.21378 ms_handle_connect on v2:172.21.15.196:6838/1624290607 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa09ffff640 1 -- 172.21.15.77:0/2744141040 <== mds.2 v2:172.21.15.196:6838/1624290607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635d4b1fd70 con 0x5635d4c3c9c0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa09ffff640 10 client.21378 handle_command_reply: tid=0 2024-09-15T22:38:19.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 client.21378 shutdown 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 msgr2=0x5635d4bc55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009230 tx=0x7fa0b4005a40 comp rx=0 tx=0).stop 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 shutdown_connections 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:6838/1624290607,v1:172.21.15.196:6839/1624290607] conn(0x5635d4c3c9c0 0x5635d4c3ee40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5635d4bc1780 0x5635d4bc6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5635d4bb3b40 0x5635d4bc5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa09ffff640 0 client.21378 ms_handle_reset on v2:172.21.15.196:6838/1624290607 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 --2- 172.21.15.77:0/2744141040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635d4bb25d0 0x5635d4bc55d0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.744+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 >> 172.21.15.77:0/2744141040 conn(0x5635d4b7b310 msgr2=0x5635d4bc04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 shutdown_connections 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c9612740 1 -- 172.21.15.77:0/2744141040 wait complete. 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c9612740 20 client.21378 trim_cache size 0 max 16384 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 msgr2=0x7fa0b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 0x7fa0b00a5870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac012960 tx=0x7fa0ac00bb70 comp rx=0 tx=0).stop 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 msgr2=0x7fa0c019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b970 tx=0x7fa0b800be30 comp rx=0 tx=0).stop 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 shutdown_connections 2024-09-15T22:38:19.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fa0b00a33b0 0x7fa0b00a5870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0c0106f10 0x7fa0c019cd50 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c0102790 0x7fa0c01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 --2- 172.21.15.77:0/3649350436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0101de0 0x7fa0c0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 >> 172.21.15.77:0/3649350436 conn(0x7fa0c00fd8c0 msgr2=0x7fa0c00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 shutdown_connections 2024-09-15T22:38:19.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:19.745+0000 7fa0c7239640 1 -- 172.21.15.77:0/3649350436 wait complete. 2024-09-15T22:38:20.517 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:20 smithi077 ceph-mon[30980]: pgmap v820: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.9 MiB/s wr, 186 op/s 2024-09-15T22:38:20.544 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:38:20.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:20 smithi196 ceph-mon[35109]: pgmap v820: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.9 MiB/s wr, 186 op/s 2024-09-15T22:38:20.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:20 smithi117 ceph-mon[35021]: pgmap v820: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 4.9 MiB/s wr, 186 op/s 2024-09-15T22:38:20.788 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 -- 172.21.15.77:0/905620284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c102d30 msgr2=0x7f912c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 --2- 172.21.15.77:0/905620284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c102d30 0x7f912c103110 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f911c009930 tx=0x7f911c02a420 comp rx=0 tx=0).stop 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 -- 172.21.15.77:0/905620284 shutdown_connections 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 --2- 172.21.15.77:0/905620284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1082a0 0x7f912c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 --2- 172.21.15.77:0/905620284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c1036e0 0x7f912c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 --2- 172.21.15.77:0/905620284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c102d30 0x7f912c103110 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 -- 172.21.15.77:0/905620284 >> 172.21.15.77:0/905620284 conn(0x7f912c0fc7d0 msgr2=0x7f912c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 -- 172.21.15.77:0/905620284 shutdown_connections 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.313+0000 7f9131a4d640 1 -- 172.21.15.77:0/905620284 wait complete. 2024-09-15T22:38:21.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 Processor -- start 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 -- start start 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c102d30 0x7f912c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1036e0 0x7f912c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1120a0 con 0x7f912c1082a0 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f912c111f20 con 0x7f912c1036e0 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9131a4d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f912c112220 con 0x7f912c102d30 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46052/0 (socket says 172.21.15.77:46052) 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 -- 172.21.15.77:0/2028396162 learned_addr learned my addr 172.21.15.77:0/2028396162 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f9123fff640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1036e0 0x7f912c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912affd640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c102d30 0x7f912c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 -- 172.21.15.77:0/2028396162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c102d30 msgr2=0x7f912c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c102d30 0x7f912c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 -- 172.21.15.77:0/2028396162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1036e0 msgr2=0x7f912c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1036e0 0x7f912c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 -- 172.21.15.77:0/2028396162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c19d550 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.314+0000 7f912b7fe640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f912400eaa0 tx=0x7f912400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.315+0000 7f9130a4b640 1 -- 172.21.15.77:0/2028396162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124004d20 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.315+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c19d840 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.315+0000 7f9130a4b640 1 -- 172.21.15.77:0/2028396162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9124005020 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.315+0000 7f9130a4b640 1 -- 172.21.15.77:0/2028396162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240116d0 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.315+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c1a5120 con 0x7f912c1082a0 2024-09-15T22:38:21.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9130a4b640 1 -- 172.21.15.77:0/2028396162 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f91240051d0 con 0x7f912c1082a0 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 Processor -- start 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 -- start start 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad16c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.317+0000 7f9133e26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bad0c52f0 con 0x556bad15f5d0 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9133e26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bad06ccf0 con 0x556bad160b40 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9133e26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bad0c12b0 con 0x556bad16a370 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9130a4b640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 0x7f91180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46060/0 (socket says 172.21.15.77:46060) 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 -- 172.21.15.77:0/2591235463 learned_addr learned my addr 172.21.15.77:0/2591235463 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9123fff640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad16c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9123fff640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912b7fe640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 0x7f91180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 -- 172.21.15.77:0/2591235463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 msgr2=0x556bad16c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad16c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 -- 172.21.15.77:0/2591235463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 msgr2=0x556bad15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad15f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 -- 172.21.15.77:0/2591235463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad0513a0 con 0x556bad15f5d0 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9123fff640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad15f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:21.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9130a4b640 1 -- 172.21.15.77:0/2028396162 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f91240d01c0 con 0x7f912c1082a0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912affd640 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f911c009930 tx=0x7f911c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf396f4faacd665 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f912b7fe640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 0x7f91180a5870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9124098570 tx=0x7f912406de90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9121ffb640 1 -- 172.21.15.77:0/2591235463 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c040070 con 0x556bad15f5d0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9121ffb640 1 -- 172.21.15.77:0/2591235463 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f911c005020 con 0x556bad15f5d0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.318+0000 7f9121ffb640 1 -- 172.21.15.77:0/2591235463 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c035040 con 0x556bad15f5d0 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 -- 172.21.15.77:0/2591235463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 msgr2=0x556bad15fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f911c009930 tx=0x7f911c031630 comp rx=0 tx=0).stop 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 -- 172.21.15.77:0/2591235463 shutdown_connections 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad16c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad15fa10 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 --2- 172.21.15.77:0/2591235463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad15f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 -- 172.21.15.77:0/2591235463 >> 172.21.15.77:0/2591235463 conn(0x556bad128310 msgr2=0x556bad15bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 -- 172.21.15.77:0/2591235463 shutdown_connections 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.319+0000 7f9133e26740 1 -- 172.21.15.77:0/2591235463 wait complete. 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 -- start start 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 Processor -- start 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad15f5d0 0x556bad169c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad167420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bad16d010 con 0x556bad160b40 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556bad080060 con 0x556bad15f5d0 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556bad16d190 con 0x556bad16a370 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9133e26740 10 client.0 _ll_register_callbacks cb 0x556bacdd38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46064/0 (socket says 172.21.15.77:46064) 2024-09-15T22:38:21.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 -- 172.21.15.77:0/2989512980 learned_addr learned my addr 172.21.15.77:0/2989512980 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9123fff640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad15f5d0 0x556bad169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9123fff640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 -- 172.21.15.77:0/2989512980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 msgr2=0x556bad167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad167420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 -- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad15f5d0 msgr2=0x556bad169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad15f5d0 0x556bad169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f912affd640 1 -- 172.21.15.77:0/2989512980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bace363c0 con 0x556bad160b40 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.320+0000 7f9123fff640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f912affd640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be90bc972663516 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c040030 con 0x556bad160b40 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f911c0053e0 con 0x556bad160b40 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911c005020 con 0x556bad160b40 2024-09-15T22:38:21.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f911c035070 con 0x556bad160b40 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 10 client.21390 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bad1789a0 con 0x556bad160b40 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f911c005020 con 0x556bad160b40 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f911c03c9e0 con 0x556bad160b40 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 10 client.21390 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 10 client.21390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 20 client.21390 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad179cf0 0x556bad17c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 4 client.21390 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.321+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bad0d2860 con 0x556bad179cf0 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.322+0000 7f9123fff640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad179cf0 0x556bad17c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.322+0000 7f9123fff640 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad179cf0 0x556bad17c0e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dab660da8c4b3f7e server_cookie=ba26f601bf28eeb7 in_seq=0 out_seq=0 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.322+0000 7f910affd640 10 client.21390 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f910affd640 1 -- 172.21.15.77:0/2989512980 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bad0d2860 con 0x556bad179cf0 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f910affd640 10 client.21390 handle_command_reply: tid=0 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 client.21390 shutdown 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 msgr2=0x556bad166ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c031b10 comp rx=0 tx=0).stop 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 shutdown_connections 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad179cf0 0x556bad17c0e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad16a370 0x556bad167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad160b40 0x556bad166ee0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 --2- 172.21.15.77:0/2989512980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad15f5d0 0x556bad169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 >> 172.21.15.77:0/2989512980 conn(0x556bad128310 msgr2=0x556bad161930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f910affd640 0 client.21390 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 shutdown_connections 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 1 -- 172.21.15.77:0/2989512980 wait complete. 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.323+0000 7f9133e26740 20 client.21390 trim_cache size 0 max 16384 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 Processor -- start 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 -- start start 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 0x556bad1673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad16a370 0x556bad16b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1aef50 con 0x556bad15f5d0 2024-09-15T22:38:21.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1b0af0 con 0x556bad16a370 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.336+0000 7f9133e26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1af620 con 0x556bad160b40 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46066/0 (socket says 172.21.15.77:46066) 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 -- 172.21.15.77:0/2416035265 learned_addr learned my addr 172.21.15.77:0/2416035265 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f912affd640 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 0x556bad1673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 -- 172.21.15.77:0/2416035265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 msgr2=0x556bad1673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 0x556bad1673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 -- 172.21.15.77:0/2416035265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad16a370 msgr2=0x556bad16b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad16a370 0x556bad16b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 -- 172.21.15.77:0/2416035265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad15a310 con 0x556bad15f5d0 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f912affd640 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 0x556bad1673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f9123fff640 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f912c0697d0 tx=0x7f91140115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541a9d2b6e200df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f91217fa640 1 -- 172.21.15.77:0/2416035265 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114022070 con 0x556bad15f5d0 2024-09-15T22:38:21.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f91217fa640 1 -- 172.21.15.77:0/2416035265 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9114005500 con 0x556bad15f5d0 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.337+0000 7f91217fa640 1 -- 172.21.15.77:0/2416035265 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911401e600 con 0x556bad15f5d0 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- 172.21.15.77:0/2416035265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 msgr2=0x556bad166ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f912c0697d0 tx=0x7f91140115b0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- 172.21.15.77:0/2416035265 shutdown_connections 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad16a370 0x556bad16b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad160b40 0x556bad1673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 --2- 172.21.15.77:0/2416035265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad15f5d0 0x556bad166ea0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- 172.21.15.77:0/2416035265 >> 172.21.15.77:0/2416035265 conn(0x556bad128310 msgr2=0x556bad16c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- 172.21.15.77:0/2416035265 shutdown_connections 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- 172.21.15.77:0/2416035265 wait complete. 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.338+0000 7f9133e26740 1 -- start start 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 Processor -- start 2024-09-15T22:38:21.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad15f5d0 0x556bad17f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad17f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1b0420 con 0x556bad16a370 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1afd20 con 0x556bad160b40 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f912c1acf50 con 0x556bad15f5d0 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9133e26740 10 client.0 _ll_register_callbacks cb 0x7f911c004790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46072/0 (socket says 172.21.15.77:46072) 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 -- 172.21.15.77:0/716565572 learned_addr learned my addr 172.21.15.77:0/716565572 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad15f5d0 0x556bad17f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f912affd640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad17f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 -- 172.21.15.77:0/716565572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad15f5d0 msgr2=0x556bad17f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad15f5d0 0x556bad17f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 -- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 msgr2=0x556bad17f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad17f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 -- 172.21.15.77:0/716565572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bad15b510 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f9123fff640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f911400b010 tx=0x7f9114004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca600191a2e919 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.339+0000 7f910affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911402a020 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f91140048c0 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9114005500 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f911403f050 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 10 client.21396 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bad162ae0 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9114005500 con 0x556bad16a370 2024-09-15T22:38:21.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7f91140266d0 con 0x556bad16a370 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 10 client.21396 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 10 client.21396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 20 client.21396 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad1e4430 0x556bad1e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 4 client.21396 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 --> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bad0ccd70 con 0x556bad1e4430 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.340+0000 7f912affd640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad1e4430 0x556bad1e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f910affd640 10 client.21396 ms_handle_connect on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f912affd640 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad1e4430 0x556bad1e6850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3877fbc0b8834603 server_cookie=33fa398991c6b6dc in_seq=0 out_seq=0 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f910affd640 1 -- 172.21.15.77:0/716565572 <== mds.1 v2:172.21.15.117:6836/3263285576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bad0ccd70 con 0x556bad1e4430 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f910affd640 10 client.21396 handle_command_reply: tid=0 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 client.21396 shutdown 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 msgr2=0x556bad17a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f911400b010 tx=0x7f9114004da0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 shutdown_connections 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:6836/3263285576,v1:172.21.15.117:6837/3263285576] conn(0x556bad1e4430 0x556bad1e6850 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bad16a370 0x556bad17a480 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556bad160b40 0x556bad17f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 --2- 172.21.15.77:0/716565572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556bad15f5d0 0x556bad17f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 >> 172.21.15.77:0/716565572 conn(0x556bad128310 msgr2=0x556bad165770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f910affd640 0 client.21396 ms_handle_reset on v2:172.21.15.117:6836/3263285576 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 shutdown_connections 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.341+0000 7f9133e26740 1 -- 172.21.15.77:0/716565572 wait complete. 2024-09-15T22:38:21.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9133e26740 20 client.21396 trim_cache size 0 max 16384 2024-09-15T22:38:21.360 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:21.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 msgr2=0x7f91180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.360 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 0x7f91180a5870 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9124098570 tx=0x7f912406de90 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 msgr2=0x7f912c19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f912400eaa0 tx=0x7f912400eea0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 shutdown_connections 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f91180a33b0 0x7f91180a5870 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f912c1082a0 0x7f912c19ce70 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f912c1036e0 0x7f912c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 --2- 172.21.15.77:0/2028396162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f912c102d30 0x7f912c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 >> 172.21.15.77:0/2028396162 conn(0x7f912c0fc7d0 msgr2=0x7f912c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 shutdown_connections 2024-09-15T22:38:21.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:21.342+0000 7f9131a4d640 1 -- 172.21.15.77:0/2028396162 wait complete. 2024-09-15T22:38:22.113 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:38:22.360 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:22.480 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:22 smithi077 ceph-mon[30980]: pgmap v821: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.9 MiB/s wr, 146 op/s 2024-09-15T22:38:22.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:22 smithi196 ceph-mon[35109]: pgmap v821: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.9 MiB/s wr, 146 op/s 2024-09-15T22:38:22.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:22 smithi117 ceph-mon[35021]: pgmap v821: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.9 MiB/s wr, 146 op/s 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 -- 172.21.15.77:0/2206218072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 msgr2=0x7fedd4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 --2- 172.21.15.77:0/2206218072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4100ee0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009930 tx=0x7fedc802a420 comp rx=0 tx=0).stop 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 -- 172.21.15.77:0/2206218072 shutdown_connections 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 --2- 172.21.15.77:0/2206218072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 0x7fedd410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 --2- 172.21.15.77:0/2206218072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4100ee0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 --2- 172.21.15.77:0/2206218072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 -- 172.21.15.77:0/2206218072 >> 172.21.15.77:0/2206218072 conn(0x7fedd40fc7a0 msgr2=0x7fedd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 -- 172.21.15.77:0/2206218072 shutdown_connections 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.878+0000 7feddc3e7640 1 -- 172.21.15.77:0/2206218072 wait complete. 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 Processor -- start 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 -- start start 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 0x7fedd4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd41030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110e80 con 0x7fedd4100a80 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4110d00 con 0x7fedd4101420 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7feddc3e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4111000 con 0x7fedd4106a80 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46090/0 (socket says 172.21.15.77:46090) 2024-09-15T22:38:22.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 -- 172.21.15.77:0/1700775241 learned_addr learned my addr 172.21.15.77:0/1700775241 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda984640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd41030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedd9982640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 0x7fedd4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 -- 172.21.15.77:0/1700775241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 msgr2=0x7fedd41030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd41030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 -- 172.21.15.77:0/1700775241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 msgr2=0x7fedd4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 0x7fedd4102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 -- 172.21.15.77:0/1700775241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a6960 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda984640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedda183640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c970 tx=0x7fedcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.879+0000 7fedd37fe640 1 -- 172.21.15.77:0/1700775241 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc007bb0 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.880+0000 7fedd37fe640 1 -- 172.21.15.77:0/1700775241 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc007d50 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.880+0000 7fedd37fe640 1 -- 172.21.15.77:0/1700775241 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00f450 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.880+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a6c50 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.880+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a7160 con 0x7fedd4100a80 2024-09-15T22:38:22.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedd37fe640 1 -- 172.21.15.77:0/1700775241 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fedcc005020 con 0x7fedd4100a80 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 Processor -- start 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 -- start start 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedd37fe640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 0x7fedbc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 0x5559715d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55597153e2f0 con 0x5559715e3370 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559714e5cf0 con 0x5559715d9b40 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.882+0000 7fedde7c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55597153a2b0 con 0x5559715d85d0 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46098/0 (socket says 172.21.15.77:46098) 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 -- 172.21.15.77:0/2363964665 learned_addr learned my addr 172.21.15.77:0/2363964665 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd37fe640 1 -- 172.21.15.77:0/1700775241 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fedcc0cfe80 con 0x7fedd4100a80 2024-09-15T22:38:22.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda183640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 0x7fedbc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd9982640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 0x5559715d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 -- 172.21.15.77:0/2363964665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 msgr2=0x5559715d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 0x5559715d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 -- 172.21.15.77:0/2363964665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 msgr2=0x5559715d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 -- 172.21.15.77:0/2363964665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559714ca3a0 con 0x5559715e3370 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd9982640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 0x5559715d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda183640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 0x7fedbc0a58f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0a4510 tx=0x7fedcc00cf60 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedda984640 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013ba0 tx=0x7fedc400a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e31b3bc298a911 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd0dea640 1 -- 172.21.15.77:0/2363964665 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4013310 con 0x5559715e3370 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd0dea640 1 -- 172.21.15.77:0/2363964665 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4003e50 con 0x5559715e3370 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.883+0000 7fedd0dea640 1 -- 172.21.15.77:0/2363964665 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc40043a0 con 0x5559715e3370 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- 172.21.15.77:0/2363964665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 msgr2=0x5559715e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013ba0 tx=0x7fedc400a6d0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- 172.21.15.77:0/2363964665 shutdown_connections 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715e3370 0x5559715e5760 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d85d0 0x5559715d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 --2- 172.21.15.77:0/2363964665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- 172.21.15.77:0/2363964665 >> 172.21.15.77:0/2363964665 conn(0x5559715a1310 msgr2=0x5559715d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- 172.21.15.77:0/2363964665 shutdown_connections 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- 172.21.15.77:0/2363964665 wait complete. 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.884+0000 7fedde7c0740 1 -- start start 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 Processor -- start 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d9b40 0x5559715dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715e3370 0x5559715e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5559715e6070 con 0x5559715d85d0 2024-09-15T22:38:22.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559714f9060 con 0x5559715e3370 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5559715e61f0 con 0x5559715d9b40 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedde7c0740 10 client.0 _ll_register_callbacks cb 0x55597124c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46106/0 (socket says 172.21.15.77:46106) 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 -- 172.21.15.77:0/1812021241 learned_addr learned my addr 172.21.15.77:0/1812021241 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedd9982640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d9b40 0x5559715dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715e3370 0x5559715e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 -- 172.21.15.77:0/1812021241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d9b40 msgr2=0x5559715dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d9b40 0x5559715dfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 -- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715e3370 msgr2=0x5559715e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715e3370 0x5559715e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 -- 172.21.15.77:0/1812021241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559712af3c0 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedda984640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fedc400afd0 tx=0x7fedc4004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a994761dfd78e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.885+0000 7fedb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4027040 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fedc4004120 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc4003e50 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fedc4033050 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 10 client.21408 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5559715f1ae0 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4003e50 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fedc4005020 con 0x5559715d85d0 2024-09-15T22:38:22.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 10 client.21408 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 10 client.21408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 20 client.21408 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559715f2ee0 0x5559715f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 4 client.21408 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.886+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55597154b860 con 0x5559715f2ee0 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.887+0000 7fedd9982640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559715f2ee0 0x5559715f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.887+0000 7fedb9ffb640 10 client.21408 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.887+0000 7fedd9982640 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559715f2ee0 0x5559715f52d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cd030689cfefea server_cookie=c821ebc5357753b8 in_seq=0 out_seq=0 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedb9ffb640 1 -- 172.21.15.77:0/1812021241 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55597154b860 con 0x5559715f2ee0 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedb9ffb640 10 client.21408 handle_command_reply: tid=0 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 client.21408 shutdown 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 msgr2=0x5559715e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fedc400afd0 tx=0x7fedc4004020 comp rx=0 tx=0).stop 2024-09-15T22:38:22.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 shutdown_connections 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x5559715f2ee0 0x5559715f52d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715e3370 0x5559715e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715d9b40 0x5559715dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedb9ffb640 0 client.21408 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 --2- 172.21.15.77:0/1812021241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715e2d40 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 >> 172.21.15.77:0/1812021241 conn(0x5559715a1310 msgr2=0x5559715da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 shutdown_connections 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 1 -- 172.21.15.77:0/1812021241 wait complete. 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.888+0000 7fedde7c0740 20 client.21408 trim_cache size 0 max 16384 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 Processor -- start 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 -- start start 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 0x5559715ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715db660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b1180 con 0x5559715d9b40 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b2d20 con 0x5559715d85d0 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedde7c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b1850 con 0x5559715e3370 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46110/0 (socket says 172.21.15.77:46110) 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 -- 172.21.15.77:0/3509430722 learned_addr learned my addr 172.21.15.77:0/3509430722 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedda984640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 0x5559715ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedda984640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 -- 172.21.15.77:0/3509430722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 msgr2=0x5559715db660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715db660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 -- 172.21.15.77:0/3509430722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 msgr2=0x5559715ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.928 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 0x5559715ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 -- 172.21.15.77:0/3509430722 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559715e5f50 con 0x5559715d9b40 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedda984640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 0x5559715ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedd9982640 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fedc8031550 tx=0x7fedc802aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d23967f224e257e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedbbfff640 1 -- 172.21.15.77:0/3509430722 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc804b070 con 0x5559715d9b40 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedbbfff640 1 -- 172.21.15.77:0/3509430722 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8005500 con 0x5559715d9b40 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.901+0000 7fedbbfff640 1 -- 172.21.15.77:0/3509430722 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8035040 con 0x5559715d9b40 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- 172.21.15.77:0/3509430722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 msgr2=0x5559715db120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fedc8031550 tx=0x7fedc802aa60 comp rx=0 tx=0).stop 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- 172.21.15.77:0/3509430722 shutdown_connections 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715db660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d9b40 0x5559715db120 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 --2- 172.21.15.77:0/3509430722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d85d0 0x5559715ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- 172.21.15.77:0/3509430722 >> 172.21.15.77:0/3509430722 conn(0x5559715a1310 msgr2=0x5559715e2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- 172.21.15.77:0/3509430722 shutdown_connections 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- 172.21.15.77:0/3509430722 wait complete. 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 -- start start 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.902+0000 7fedde7c0740 1 Processor -- start 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715dd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b2650 con 0x5559715d85d0 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b1f50 con 0x5559715d9b40 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41af180 con 0x5559715e3370 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 10 client.0 _ll_register_callbacks cb 0x7fedc4013560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46124/0 (socket says 172.21.15.77:46124) 2024-09-15T22:38:22.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 -- 172.21.15.77:0/831498244 learned_addr learned my addr 172.21.15.77:0/831498244 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedd9982640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 -- 172.21.15.77:0/831498244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 msgr2=0x5559715dd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715dd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 -- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 msgr2=0x5559715dcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715dcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 -- 172.21.15.77:0/831498244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559715dd8a0 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedd9982640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedda984640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005590 tx=0x7fedc4013b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3be73d7aa582c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc403c060 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fedc4005000 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.903+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8005500 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fedc400ab50 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 10 client.21414 fetch_fsmap learned FSMap version 42 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55597165d7d0 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8005500 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 42) ==== 8386+0+0 (secure 0 0 0) 0x7fedc4036040 con 0x5559715d85d0 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 10 client.21414 fetch_fsmap finished waiting for FSMap version 42 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 10 client.21414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 20 client.21414 populate_metadata read hostname 'smithi077' 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55597165def0 0x555971660310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 4 client.21414 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 --> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555971545d70 con 0x55597165def0 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.904+0000 7fedd9982640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55597165def0 0x555971660310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedd9982640 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55597165def0 0x555971660310 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93c6d4797668c13 server_cookie=d3cb35cbb5485ab5 in_seq=0 out_seq=0 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedb9ffb640 10 client.21414 ms_handle_connect on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedb9ffb640 1 -- 172.21.15.77:0/831498244 <== mds.0 v2:172.21.15.117:6838/1252344498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555971545d70 con 0x55597165def0 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedb9ffb640 10 client.21414 handle_command_reply: tid=0 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 client.21414 shutdown 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 msgr2=0x5559715ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005590 tx=0x7fedc4013b30 comp rx=0 tx=0).stop 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 shutdown_connections 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:6838/1252344498,v1:172.21.15.117:6839/1252344498] conn(0x55597165def0 0x555971660310 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5559715e3370 0x5559715dd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5559715d9b40 0x5559715dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedb9ffb640 0 client.21414 ms_handle_reset on v2:172.21.15.117:6838/1252344498 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 --2- 172.21.15.77:0/831498244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559715d85d0 0x5559715ebac0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 >> 172.21.15.77:0/831498244 conn(0x5559715a1310 msgr2=0x5559715d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 shutdown_connections 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 1 -- 172.21.15.77:0/831498244 wait complete. 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7fedde7c0740 20 client.21414 trim_cache size 0 max 16384 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 msgr2=0x7fedbc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 0x7fedbc0a58f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0a4510 tx=0x7fedcc00cf60 comp rx=0 tx=0).stop 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 msgr2=0x7fedd4102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:22.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.905+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c970 tx=0x7fedcc00ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 shutdown_connections 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fedbc0a3430 0x7fedbc0a58f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedd4106a80 0x7fedd41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedd4101420 0x7fedd4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 --2- 172.21.15.77:0/1700775241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedd4100a80 0x7fedd4102650 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 >> 172.21.15.77:0/1700775241 conn(0x7fedd40fc7a0 msgr2=0x7fedd41048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 shutdown_connections 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:22.906+0000 7feddc3e7640 1 -- 172.21.15.77:0/1700775241 wait complete. 2024-09-15T22:38:23.589 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-15T22:38:23.590 INFO:tasks.fwd_scrub:done joining 2024-09-15T22:38:23.590 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-15T22:38:23.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/238016786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 msgr2=0x7f4f74101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/238016786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74101f00 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009930 tx=0x7f4f6802a420 comp rx=0 tx=0).stop 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/238016786 shutdown_connections 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/238016786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f74102440 0x7f4f7410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/238016786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74101f00 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/238016786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74107ab0 0x7f4f74107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/238016786 >> 172.21.15.77:0/238016786 conn(0x7f4f740fd7c0 msgr2=0x7f4f740ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/238016786 shutdown_connections 2024-09-15T22:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/238016786 wait complete. 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 Processor -- start 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.701+0000 7f4f7aa5e640 1 -- start start 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74102440 0x7f4f74198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f74107ab0 0x7f4f7419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74111eb0 con 0x7f4f74101aa0 2024-09-15T22:38:23.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74111d30 con 0x7f4f74102440 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f7aa5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74112030 con 0x7f4f74107ab0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46136/0 (socket says 172.21.15.77:46136) 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 -- 172.21.15.77:0/445111310 learned_addr learned my addr 172.21.15.77:0/445111310 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f737fe640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74102440 0x7f4f74198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 -- 172.21.15.77:0/445111310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f74107ab0 msgr2=0x7f4f7419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f74107ab0 0x7f4f7419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 -- 172.21.15.77:0/445111310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74102440 msgr2=0x7f4f74198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74102440 0x7f4f74198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 -- 172.21.15.77:0/445111310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7419d410 con 0x7f4f74101aa0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f73fff640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c890 tx=0x7f4f6400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.702+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64004220 con 0x7f4f74101aa0 2024-09-15T22:38:23.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.703+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f640043c0 con 0x7f4f74101aa0 2024-09-15T22:38:23.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.703+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7419d700 con 0x7f4f74101aa0 2024-09-15T22:38:23.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.703+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f640049e0 con 0x7f4f74101aa0 2024-09-15T22:38:23.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.703+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741a4f90 con 0x7f4f74101aa0 2024-09-15T22:38:23.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.703+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f74101f00 con 0x7f4f74101aa0 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.705+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4f640079b0 con 0x7f4f74101aa0 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.705+0000 7f4f717fa640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 0x7f4f5c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.705+0000 7f4f737fe640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 0x7f4f5c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.705+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f4f640d00d0 con 0x7f4f74101aa0 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.706+0000 7f4f737fe640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 0x7f4f5c0a5880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009900 tx=0x7f4f680058d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:23.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.709+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f64014030 con 0x7f4f74101aa0 2024-09-15T22:38:23.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.876+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f74199770 con 0x7f4f74101aa0 2024-09-15T22:38:23.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.878+0000 7f4f717fa640 1 -- 172.21.15.77:0/445111310 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+25340 (secure 0 0 0) 0x7f4f6409c720 con 0x7f4f74101aa0 2024-09-15T22:38:23.879 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:23.880 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":131,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:38:14.116978+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/999829703":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/3279424814":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3785009540":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1809193257":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/1437248041":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1809193257":"2024-09-16T22:12:24.385790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]},{"pool":4,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:23.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 msgr2=0x7f4f5c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 0x7f4f5c0a5880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009900 tx=0x7f4f680058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 msgr2=0x7f4f74198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c890 tx=0x7f4f6400cd50 comp rx=0 tx=0).stop 2024-09-15T22:38:23.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 shutdown_connections 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f4f5c0a33c0 0x7f4f5c0a5880 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f74107ab0 0x7f4f7419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f74102440 0x7f4f74198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 --2- 172.21.15.77:0/445111310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f74101aa0 0x7f4f74198420 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 >> 172.21.15.77:0/445111310 conn(0x7f4f740fd7c0 msgr2=0x7f4f740fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 shutdown_connections 2024-09-15T22:38:23.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.880+0000 7f4f7aa5e640 1 -- 172.21.15.77:0/445111310 wait complete. 2024-09-15T22:38:23.892 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:05:16.403426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:09:51.900322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:09:54.068598+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 130, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:10:00.447354+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 130, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:38:23.893 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:38:23.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 -- 172.21.15.77:0/674427495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1035a0 msgr2=0x7f8a6c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 --2- 172.21.15.77:0/674427495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1035a0 0x7f8a6c10ba80 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009900 tx=0x7f8a6002a480 comp rx=0 tx=0).stop 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 -- 172.21.15.77:0/674427495 shutdown_connections 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 --2- 172.21.15.77:0/674427495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a6c10c0d0 0x7f8a6c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 --2- 172.21.15.77:0/674427495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c1035a0 0x7f8a6c10ba80 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 --2- 172.21.15.77:0/674427495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c102c80 0x7f8a6c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 -- 172.21.15.77:0/674427495 >> 172.21.15.77:0/674427495 conn(0x7f8a6c0fc690 msgr2=0x7f8a6c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 -- 172.21.15.77:0/674427495 shutdown_connections 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.994+0000 7f8a7227b640 1 -- 172.21.15.77:0/674427495 wait complete. 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 Processor -- start 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 -- start start 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c1035a0 0x7f8a6c100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a6c10c0d0 0x7f8a6c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c111eb0 con 0x7f8a6c102c80 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c111d30 con 0x7f8a6c1035a0 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a7227b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c112030 con 0x7f8a6c10c0d0 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46156/0 (socket says 172.21.15.77:46156) 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 -- 172.21.15.77:0/3938044731 learned_addr learned my addr 172.21.15.77:0/3938044731 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 -- 172.21.15.77:0/3938044731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a6c10c0d0 msgr2=0x7f8a6c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a6c10c0d0 0x7f8a6c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 -- 172.21.15.77:0/3938044731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c1035a0 msgr2=0x7f8a6c100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c1035a0 0x7f8a6c100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.995+0000 7f8a6bfff640 1 -- 172.21.15.77:0/3938044731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c1a6930 con 0x7f8a6c102c80 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a6bfff640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d950 tx=0x7f8a5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c017070 con 0x7f8a6c102c80 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c1a6c20 con 0x7f8a6c102c80 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c009c30 con 0x7f8a6c102c80 2024-09-15T22:38:23.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00f6b0 con 0x7f8a6c102c80 2024-09-15T22:38:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.996+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1a71c0 con 0x7f8a6c102c80 2024-09-15T22:38:24.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.997+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c103060 con 0x7f8a6c102c80 2024-09-15T22:38:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.998+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8a5c0109d0 con 0x7f8a6c102c80 2024-09-15T22:38:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.998+0000 7f8a697fa640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 0x7f8a540a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.999+0000 7f8a6b7fe640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 0x7f8a540a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.999+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f8a5c0cfd70 con 0x7f8a6c102c80 2024-09-15T22:38:24.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:23.999+0000 7f8a6b7fe640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 0x7f8a540a5880 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8a6002a9a0 tx=0x7f8a60000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.003+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a5c009de0 con 0x7f8a6c102c80 2024-09-15T22:38:24.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.170+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a6c02d7c0 con 0x7f8a6c102c80 2024-09-15T22:38:24.172 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.170+0000 7f8a697fa640 1 -- 172.21.15.77:0/3938044731 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f8a5c09c3c0 con 0x7f8a6c102c80 2024-09-15T22:38:24.172 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 msgr2=0x7f8a540a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 0x7f8a540a5880 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8a6002a9a0 tx=0x7f8a60000bc0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 msgr2=0x7f8a6c1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d950 tx=0x7f8a5c00de10 comp rx=0 tx=0).stop 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 shutdown_connections 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8a540a33c0 0x7f8a540a5880 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.172+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a6c10c0d0 0x7f8a6c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.173+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a6c1035a0 0x7f8a6c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.173+0000 7f8a7227b640 1 --2- 172.21.15.77:0/3938044731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c102c80 0x7f8a6c1008d0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.173+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 >> 172.21.15.77:0/3938044731 conn(0x7f8a6c0fc690 msgr2=0x7f8a6c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.173+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 shutdown_connections 2024-09-15T22:38:24.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.173+0000 7f8a7227b640 1 -- 172.21.15.77:0/3938044731 wait complete. 2024-09-15T22:38:24.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:38:24.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 -- 172.21.15.77:0/2992117893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70106a30 msgr2=0x7f7c70106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 --2- 172.21.15.77:0/2992117893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70106a30 0x7f7c70106e10 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009930 tx=0x7f7c6002a440 comp rx=0 tx=0).stop 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 -- 172.21.15.77:0/2992117893 shutdown_connections 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 --2- 172.21.15.77:0/2992117893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 0x7f7c7010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 --2- 172.21.15.77:0/2992117893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c70100a20 0x7f7c70100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 --2- 172.21.15.77:0/2992117893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70106a30 0x7f7c70106e10 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 -- 172.21.15.77:0/2992117893 >> 172.21.15.77:0/2992117893 conn(0x7f7c700fc740 msgr2=0x7f7c700feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 -- 172.21.15.77:0/2992117893 shutdown_connections 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.288+0000 7f7c762c9640 1 -- 172.21.15.77:0/2992117893 wait complete. 2024-09-15T22:38:24.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 Processor -- start 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 -- start start 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 0x7f7c70198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c70106a30 0x7f7c7019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70110e70 con 0x7f7c70100a20 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70110cf0 con 0x7f7c701013c0 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c762c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70110ff0 con 0x7f7c70106a30 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46166/0 (socket says 172.21.15.77:46166) 2024-09-15T22:38:24.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6ffff640 1 -- 172.21.15.77:0/540541230 learned_addr learned my addr 172.21.15.77:0/540541230 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6f7fe640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 0x7f7c70198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6ffff640 1 -- 172.21.15.77:0/540541230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c70106a30 msgr2=0x7f7c7019cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.289+0000 7f7c6ffff640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c70106a30 0x7f7c7019cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6ffff640 1 -- 172.21.15.77:0/540541230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 msgr2=0x7f7c70198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6ffff640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 0x7f7c70198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6ffff640 1 -- 172.21.15.77:0/540541230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7019d500 con 0x7f7c70100a20 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6ffff640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009230 tx=0x7f7c60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6003f030 con 0x7f7c70100a20 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c60003ad0 con 0x7f7c70100a20 2024-09-15T22:38:24.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60035070 con 0x7f7c70100a20 2024-09-15T22:38:24.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7019d790 con 0x7f7c70100a20 2024-09-15T22:38:24.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.290+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c701a5080 con 0x7f7c70100a20 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.291+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c70100e80 con 0x7f7c70100a20 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.292+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c60042030 con 0x7f7c70100a20 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.293+0000 7f7c6d7fa640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 0x7f7c580a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.293+0000 7f7c6f7fe640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 0x7f7c580a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.293+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f7c600f05f0 con 0x7f7c70100a20 2024-09-15T22:38:24.296 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.293+0000 7f7c6f7fe640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 0x7f7c580a5880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64008040 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.297+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c600bcc40 con 0x7f7c70100a20 2024-09-15T22:38:24.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.464+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c701a5250 con 0x7f7c70100a20 2024-09-15T22:38:24.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.465+0000 7f7c6d7fa640 1 -- 172.21.15.77:0/540541230 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) ==== 104+0+11 (secure 0 0 0) 0x7f7c600bc540 con 0x7f7c70100a20 2024-09-15T22:38:24.466 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-15T22:38:24.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 msgr2=0x7f7c580a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 0x7f7c580a5880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64008040 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 msgr2=0x7f7c70198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009230 tx=0x7f7c60004270 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 shutdown_connections 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f7c580a33c0 0x7f7c580a5880 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c70106a30 0x7f7c7019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c701013c0 0x7f7c70198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 --2- 172.21.15.77:0/540541230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70100a20 0x7f7c70198510 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 >> 172.21.15.77:0/540541230 conn(0x7f7c700fc740 msgr2=0x7f7c70104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 shutdown_connections 2024-09-15T22:38:24.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.467+0000 7f7c762c9640 1 -- 172.21.15.77:0/540541230 wait complete. 2024-09-15T22:38:24.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:24 smithi077 ceph-mon[30980]: pgmap v822: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 144 KiB/s wr, 38 op/s 2024-09-15T22:38:24.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/445111310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:24.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:24 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3938044731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:24.479 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:38:24.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 -- 172.21.15.77:0/3607433333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0101320 msgr2=0x7f93d010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 --2- 172.21.15.77:0/3607433333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0101320 0x7f93d010d850 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009cd0 tx=0x7f93c802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 -- 172.21.15.77:0/3607433333 shutdown_connections 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 --2- 172.21.15.77:0/3607433333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0101320 0x7f93d010d850 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 --2- 172.21.15.77:0/3607433333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93d0100980 0x7f93d0100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 --2- 172.21.15.77:0/3607433333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0106990 0x7f93d0106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 -- 172.21.15.77:0/3607433333 >> 172.21.15.77:0/3607433333 conn(0x7f93d00779d0 msgr2=0x7f93d00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 -- 172.21.15.77:0/3607433333 shutdown_connections 2024-09-15T22:38:24.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 -- 172.21.15.77:0/3607433333 wait complete. 2024-09-15T22:38:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.581+0000 7f93d823c640 1 Processor -- start 2024-09-15T22:38:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 -- start start 2024-09-15T22:38:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0101320 0x7f93d0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93d0106990 0x7f93d019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0110dd0 con 0x7f93d0100980 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0110c50 con 0x7f93d0106990 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0110f50 con 0x7f93d0101320 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46174/0 (socket says 172.21.15.77:46174) 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 -- 172.21.15.77:0/3841595785 learned_addr learned my addr 172.21.15.77:0/3841595785 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d57d7640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0101320 0x7f93d0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 -- 172.21.15.77:0/3841595785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0101320 msgr2=0x7f93d0197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0101320 0x7f93d0197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 -- 172.21.15.77:0/3841595785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93d0106990 msgr2=0x7f93d019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93d0106990 0x7f93d019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 -- 172.21.15.77:0/3841595785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d019c2f0 con 0x7f93d0100980 2024-09-15T22:38:24.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d5fd8640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00d770 tx=0x7f93cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc017070 con 0x7f93d0100980 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc005500 con 0x7f93d0100980 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d019c5e0 con 0x7f93d0100980 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.582+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc004e00 con 0x7f93d0100980 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.583+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a3e70 con 0x7f93d0100980 2024-09-15T22:38:24.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.584+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0100de0 con 0x7f93d0100980 2024-09-15T22:38:24.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.585+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f93cc004fa0 con 0x7f93d0100980 2024-09-15T22:38:24.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.585+0000 7f93c6ffd640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 0x7f93bc0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.585+0000 7f93d57d7640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 0x7f93bc0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.585+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f93cc0cfa80 con 0x7f93d0100980 2024-09-15T22:38:24.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.586+0000 7f93d57d7640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 0x7f93bc0a5880 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f93d00fffb0 tx=0x7f93c000b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.590+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93cc09c0d0 con 0x7f93d0100980 2024-09-15T22:38:24.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:24 smithi196 ceph-mon[35109]: pgmap v822: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 144 KiB/s wr, 38 op/s 2024-09-15T22:38:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/445111310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:24.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:24 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3938044731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:24 smithi117 ceph-mon[35021]: pgmap v822: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 144 KiB/s wr, 38 op/s 2024-09-15T22:38:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/445111310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:24.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:24 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3938044731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:24.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.760+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f93d002d7c0 con 0x7f93d0100980 2024-09-15T22:38:24.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.761+0000 7f93c6ffd640 1 -- 172.21.15.77:0/3841595785 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) ==== 100+0+11 (secure 0 0 0) 0x7f93cc09b9d0 con 0x7f93d0100980 2024-09-15T22:38:24.762 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2024-09-15T22:38:24.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 msgr2=0x7f93bc0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 0x7f93bc0a5880 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f93d00fffb0 tx=0x7f93c000b390 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 msgr2=0x7f93d0197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00d770 tx=0x7f93cc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 shutdown_connections 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f93bc0a33c0 0x7f93bc0a5880 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93d0106990 0x7f93d019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93d0101320 0x7f93d0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 --2- 172.21.15.77:0/3841595785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93d0100980 0x7f93d0197300 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 >> 172.21.15.77:0/3841595785 conn(0x7f93d00779d0 msgr2=0x7f93d0103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 shutdown_connections 2024-09-15T22:38:24.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.763+0000 7f93d823c640 1 -- 172.21.15.77:0/3841595785 wait complete. 2024-09-15T22:38:24.775 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 -- 172.21.15.77:0/3454280598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb1810e430 msgr2=0x7fdb18110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 --2- 172.21.15.77:0/3454280598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb1810e430 0x7fdb18110820 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009ca0 tx=0x7fdb1002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 -- 172.21.15.77:0/3454280598 shutdown_connections 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 --2- 172.21.15.77:0/3454280598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb1810e430 0x7fdb18110820 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 --2- 172.21.15.77:0/3454280598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb181011a0 0x7fdb1810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 --2- 172.21.15.77:0/3454280598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb18100880 0x7fdb18100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 -- 172.21.15.77:0/3454280598 >> 172.21.15.77:0/3454280598 conn(0x7fdb180fc690 msgr2=0x7fdb180feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.886+0000 7fdb20277640 1 -- 172.21.15.77:0/3454280598 shutdown_connections 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 -- 172.21.15.77:0/3454280598 wait complete. 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 Processor -- start 2024-09-15T22:38:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 -- start start 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb18100880 0x7fdb1819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb1810e430 0x7fdb1819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18114180 con 0x7fdb181011a0 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18114000 con 0x7fdb1810e430 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb20277640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18114300 con 0x7fdb18100880 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb1d812640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb1d812640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46194/0 (socket says 172.21.15.77:46194) 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.887+0000 7fdb1d812640 1 -- 172.21.15.77:0/2012248190 learned_addr learned my addr 172.21.15.77:0/2012248190 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1e814640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb1810e430 0x7fdb1819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 -- 172.21.15.77:0/2012248190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb18100880 msgr2=0x7fdb1819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb18100880 0x7fdb1819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 -- 172.21.15.77:0/2012248190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb1810e430 msgr2=0x7fdb1819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb1810e430 0x7fdb1819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 -- 172.21.15.77:0/2012248190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1819f6e0 con 0x7fdb181011a0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb1d812640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c970 tx=0x7fdb0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08007bb0 con 0x7fdb181011a0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb08007d50 con 0x7fdb181011a0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1819f9d0 con 0x7fdb181011a0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800f450 con 0x7fdb181011a0 2024-09-15T22:38:24.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.888+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb181a7260 con 0x7fdb181011a0 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.889+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb18100c60 con 0x7fdb181011a0 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.890+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdb08005020 con 0x7fdb181011a0 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.891+0000 7fdb0f7fe640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 0x7fdb040a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.891+0000 7fdb1e013640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 0x7fdb040a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.891+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fdb080cff50 con 0x7fdb181011a0 2024-09-15T22:38:24.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.891+0000 7fdb1e013640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 0x7fdb040a5880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009710 tx=0x7fdb14006c90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:24.895+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb08014030 con 0x7fdb181011a0 2024-09-15T22:38:25.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.060+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdb18109620 con 0x7fdb181011a0 2024-09-15T22:38:25.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.060+0000 7fdb0f7fe640 1 -- 172.21.15.77:0/2012248190 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v131) ==== 103+0+11 (secure 0 0 0) 0x7fdb0809c5a0 con 0x7fdb181011a0 2024-09-15T22:38:25.062 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 msgr2=0x7fdb040a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 0x7fdb040a5880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009710 tx=0x7fdb14006c90 comp rx=0 tx=0).stop 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 msgr2=0x7fdb1819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c970 tx=0x7fdb0800ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 shutdown_connections 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdb040a33c0 0x7fdb040a5880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.064 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb1810e430 0x7fdb1819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb181011a0 0x7fdb1819ac30 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 --2- 172.21.15.77:0/2012248190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb18100880 0x7fdb1819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 >> 172.21.15.77:0/2012248190 conn(0x7fdb180fc690 msgr2=0x7fdb180fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 shutdown_connections 2024-09-15T22:38:25.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.062+0000 7fdb20277640 1 -- 172.21.15.77:0/2012248190 wait complete. 2024-09-15T22:38:25.075 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-15T22:38:25.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 -- 172.21.15.77:0/813981637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000103590 msgr2=0x7fe00010ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 --2- 172.21.15.77:0/813981637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000103590 0x7fe00010ba70 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009900 tx=0x7fdff402a480 comp rx=0 tx=0).stop 2024-09-15T22:38:25.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 -- 172.21.15.77:0/813981637 shutdown_connections 2024-09-15T22:38:25.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 --2- 172.21.15.77:0/813981637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 0x7fe00010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.175 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 --2- 172.21.15.77:0/813981637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000103590 0x7fe00010ba70 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 --2- 172.21.15.77:0/813981637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000102c70 0x7fe000103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 -- 172.21.15.77:0/813981637 >> 172.21.15.77:0/813981637 conn(0x7fe0000fc6c0 msgr2=0x7fe0000feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 -- 172.21.15.77:0/813981637 shutdown_connections 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.173+0000 7fe006731640 1 -- 172.21.15.77:0/813981637 wait complete. 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 Processor -- start 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 -- start start 2024-09-15T22:38:25.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 0x7fe000198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 0x7fe00019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe000111ef0 con 0x7fe000102c70 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe000111d70 con 0x7fe000103590 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe006731640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112070 con 0x7fe00010c090 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fdffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fdffffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46212/0 (socket says 172.21.15.77:46212) 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fdffffff640 1 -- 172.21.15.77:0/1186685821 learned_addr learned my addr 172.21.15.77:0/1186685821 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.174+0000 7fe004cce640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 0x7fe00019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdfff7fe640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 0x7fe000198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 -- 172.21.15.77:0/1186685821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 msgr2=0x7fe00019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 0x7fe00019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 -- 172.21.15.77:0/1186685821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 msgr2=0x7fe000198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 0x7fe000198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 -- 172.21.15.77:0/1186685821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019d3d0 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdfff7fe640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 0x7fe000198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffffff640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c920 tx=0x7fdff000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0007c70 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff0007e10 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019d6c0 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0018890 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.175+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a4f50 con 0x7fe000102c70 2024-09-15T22:38:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.176+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000103050 con 0x7fe000102c70 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.177+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdff0018a70 con 0x7fe000102c70 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.178+0000 7fdffd7fa640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 0x7fdfe80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.178+0000 7fdfff7fe640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 0x7fdfe80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.178+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7fdff00d02b0 con 0x7fe000102c70 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.178+0000 7fdfff7fe640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 0x7fdfe80a5880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdff4000c00 tx=0x7fdff4032d10 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:25.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.182+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdff0014030 con 0x7fe000102c70 2024-09-15T22:38:25.331 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/540541230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3841595785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:25 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2012248190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.395+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe000106b00 con 0x7fe000102c70 2024-09-15T22:38:25.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.403+0000 7fdffd7fa640 1 -- 172.21.15.77:0/1186685821 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742870 (secure 0 0 0) 0x7fdff000de20 con 0x7fe000102c70 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 26, 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout: "active_gid": 15309, 2024-09-15T22:38:25.415 INFO:teuthology.orchestra.run.smithi077.stdout: "active_name": "x", 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addrs": { 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6800", 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3746048513 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-09-15T22:38:25.416 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6801", 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3746048513 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addr": "172.21.15.77:6801/3746048513", 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "active_change": "2024-09-15T22:12:24.386011+0000", 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "active_mgr_features": 4540701547738038271, 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "available": true, 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "standbys": [ 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.417 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24817, 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "z", 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.418 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:38:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.420 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.421 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.424 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-15T22:38:25.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.430 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-15T22:38:25.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.432 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:38:25.433 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.434 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-15T22:38:25.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.436 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-15T22:38:25.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.438 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:25.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:25.440 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.443 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:38:25.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.445 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.448 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.450 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.451 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.452 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.453 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-15T22:38:25.455 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.458 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-15T22:38:25.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:38:25.461 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-15T22:38:25.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.463 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.464 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-15T22:38:25.465 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.466 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.467 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.468 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:38:25.469 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.470 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-15T22:38:25.471 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.472 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.474 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-15T22:38:25.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.476 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.477 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-15T22:38:25.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.480 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.481 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.482 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.483 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.485 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-15T22:38:25.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.488 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.489 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:38:25.490 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-15T22:38:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-15T22:38:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-15T22:38:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-15T22:38:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:38:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:38:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-15T22:38:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:38:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-15T22:38:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-15T22:38:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:38:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:38:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:38:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:38:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:38:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:38:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.581 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:38:25.582 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.583 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.584 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.585 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:38:25.586 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.587 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.588 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.589 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:38:25.590 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.592 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.593 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.594 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:38:25.595 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.596 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.597 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.598 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.599 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.600 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:38:25.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.602 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.603 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.604 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.605 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.606 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.607 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.608 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.609 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.610 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-15T22:38:25.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.615 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.616 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.617 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.618 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.620 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.621 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.622 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.624 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.625 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.626 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.627 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.628 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-15T22:38:25.630 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:25.631 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.632 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.633 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.634 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.637 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.639 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.640 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-15T22:38:25.645 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.646 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-15T22:38:25.647 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:38:25.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.650 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.651 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.655 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.656 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.657 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.659 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-15T22:38:25.661 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.662 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.663 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.664 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.666 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.667 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.669 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.670 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.672 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-15T22:38:25.674 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:25.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-15T22:38:25.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.678 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:25.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.681 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.682 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.684 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.686 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.690 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.691 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.692 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.693 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-15T22:38:25.694 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.696 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.697 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.698 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.699 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.700 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.701 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.704 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.705 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.706 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.708 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.709 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.710 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.711 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-15T22:38:25.712 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-15T22:38:25.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:38:25.716 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.717 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.718 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.720 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.721 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.722 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.724 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.725 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.726 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.729 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.731 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.732 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.733 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.734 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-15T22:38:25.737 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.739 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.740 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.742 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.745 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.746 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.748 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.751 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.753 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.754 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.755 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.756 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.758 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.760 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.762 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.764 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.765 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.768 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.771 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.772 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.775 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.776 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.777 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.778 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.779 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.780 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.781 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.782 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-15T22:38:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.785 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.786 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.788 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.789 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.791 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.792 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.793 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.794 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.795 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.796 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.797 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.799 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-15T22:38:25.800 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.801 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.805 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.806 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.809 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.810 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.812 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:38:25.813 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-15T22:38:25.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.817 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.818 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.819 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.820 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.822 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.823 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.824 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:25.825 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.828 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.829 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.830 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.831 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.832 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.834 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.838 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.839 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.840 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.841 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.842 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-15T22:38:25.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.844 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.845 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.846 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.848 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.849 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.850 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.852 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.853 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-15T22:38:25.858 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.862 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.866 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.867 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.869 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.870 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.871 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.872 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.874 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.875 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.876 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.877 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.878 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.882 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.884 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.885 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.886 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-15T22:38:25.889 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.892 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.894 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.895 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.896 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.898 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.899 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.900 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.901 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.904 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.906 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.907 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.908 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.910 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-15T22:38:25.911 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.912 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.915 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.916 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.918 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.921 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.924 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.925 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.926 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.927 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.928 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.929 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.930 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.932 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.934 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.935 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.937 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.941 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.942 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-15T22:38:25.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.944 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.945 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.946 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.947 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.948 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.950 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.951 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.952 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.953 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.954 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.955 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.956 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.957 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24848, 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "y", 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-15T22:38:25.958 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.959 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.960 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.963 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.964 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.965 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.966 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-15T22:38:25.967 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.969 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.970 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.971 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-15T22:38:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:38:25.974 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-15T22:38:25.976 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.977 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-15T22:38:25.978 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.979 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.980 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.981 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:38:25.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-15T22:38:25.983 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.984 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-15T22:38:25.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.986 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.987 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:25.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.989 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.990 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.991 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.992 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.993 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.994 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-15T22:38:25.995 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.996 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:25.997 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:25.998 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:25.999 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.000 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.001 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.002 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.003 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.004 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.005 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:38:26.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.008 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.009 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-15T22:38:26.010 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.011 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:38:26.012 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.013 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.014 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.015 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:38:26.016 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.017 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:38:26.018 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.019 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.020 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.021 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.022 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:38:26.023 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.024 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.025 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.026 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.028 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-15T22:38:26.029 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.030 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.031 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.032 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:38:26.033 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.034 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.035 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:38:26.036 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.037 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.038 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-15T22:38:26.039 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.040 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.041 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.042 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.043 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-15T22:38:26.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.045 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:38:26.046 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.047 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.048 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.049 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.051 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.053 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.054 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.055 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.056 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.057 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.058 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-15T22:38:26.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.060 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-15T22:38:26.061 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.062 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.063 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-15T22:38:26.064 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.065 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.066 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.067 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.068 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-15T22:38:26.069 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-15T22:38:26.070 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.071 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.072 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.073 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.074 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.075 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:38:26.076 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.077 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.078 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.079 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.080 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.081 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.082 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.083 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.085 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.086 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.087 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.088 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:38:26.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.090 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.091 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:38:26.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.095 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.097 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.098 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.099 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:38:26.100 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.101 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.102 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.104 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.105 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.106 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.107 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.108 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.109 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.110 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.112 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:38:26.113 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.114 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.115 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.116 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.117 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.118 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.119 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:38:26.120 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.123 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.124 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.125 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.126 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.128 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.129 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.130 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.131 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.134 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:38:26.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.138 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.140 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-15T22:38:26.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.142 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.143 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.144 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:26.146 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.147 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.148 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.150 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.151 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.154 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.155 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.158 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.159 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-15T22:38:26.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:26.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.164 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.165 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.166 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-15T22:38:26.167 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.168 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.169 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.170 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.172 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.173 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-15T22:38:26.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.176 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.180 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-15T22:38:26.181 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.182 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.183 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.183 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-15T22:38:26.183 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.184 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.185 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.186 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.187 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.188 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.190 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-15T22:38:26.192 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.193 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-15T22:38:26.195 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-15T22:38:26.196 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:38:26.198 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.199 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.202 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.203 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.204 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-15T22:38:26.206 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-15T22:38:26.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.210 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.211 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.212 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.214 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.215 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.217 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.218 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.220 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.221 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.222 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.223 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.224 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.225 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.226 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.227 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.228 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-15T22:38:26.229 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.230 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.231 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.232 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.233 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.234 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.235 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-15T22:38:26.236 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.237 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.238 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.239 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.240 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.241 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.242 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.243 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-15T22:38:26.244 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-15T22:38:26.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-15T22:38:26.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:38:26.249 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-15T22:38:26.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.251 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.252 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.253 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.255 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.260 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.262 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.263 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.265 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.266 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.267 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.268 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.269 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.270 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.272 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.273 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.274 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.275 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.277 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.278 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.279 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.286 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.287 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.290 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.291 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.292 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.294 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.295 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.297 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.298 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.299 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.300 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.302 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.303 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.305 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.307 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.308 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.311 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.313 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.315 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.316 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.318 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.320 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.321 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.322 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.323 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.325 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-15T22:38:26.327 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.328 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.329 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.331 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.332 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.333 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:26.334 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.335 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.337 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.339 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.340 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.341 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.344 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.346 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.349 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.350 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.352 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.353 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.354 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.355 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.356 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.357 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.358 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.359 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.360 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.361 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.362 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.363 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.365 INFO:teuthology.orchestra.run.smithi077.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.366 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.368 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.369 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.371 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.374 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.376 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.377 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.379 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.380 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.381 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.382 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-15T22:38:26.384 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-15T22:38:26.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.385 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.388 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.389 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.390 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.391 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-15T22:38:26.392 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.393 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.394 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.395 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-15T22:38:26.396 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.397 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.398 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.399 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-15T22:38:26.400 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.401 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.403 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.404 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.405 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.407 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.409 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.410 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.414 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.415 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.416 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.418 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.421 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.422 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.423 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-15T22:38:26.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.425 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.427 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.428 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.429 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.430 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.432 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-15T22:38:26.434 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.435 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.436 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:38:26.437 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.439 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-15T22:38:26.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.443 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.444 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.445 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.446 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.447 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.448 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.449 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.450 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.452 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.453 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-15T22:38:26.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.456 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.457 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.459 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.461 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.462 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.463 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.464 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.465 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.467 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.468 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.470 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.473 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.475 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.476 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.477 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.478 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-15T22:38:26.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.481 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.482 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.483 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.484 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.485 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.488 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.490 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.491 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.492 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: "modules": [ 2024-09-15T22:38:26.493 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "iostat", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "nfs", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "restful", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "snap_schedule" 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.494 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.495 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.498 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.499 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.500 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.501 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.502 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.503 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.504 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-09-15T22:38:26.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-09-15T22:38:26.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.507 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.509 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.510 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.511 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:38:26.512 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-09-15T22:38:26.513 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.514 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.515 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:38:26.516 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:38:26.517 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.518 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.519 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:38:26.520 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.521 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.522 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.523 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.524 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.525 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.526 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.527 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.528 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.529 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-09-15T22:38:26.530 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.531 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.532 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.533 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:26.535 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.536 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.538 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.539 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.541 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.542 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:38:26.543 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.544 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-09-15T22:38:26.545 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.546 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.547 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.548 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.549 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.550 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:38:26.551 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.552 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.553 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.554 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.555 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-09-15T22:38:26.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:38:26.561 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.562 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-09-15T22:38:26.563 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.564 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-09-15T22:38:26.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.571 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-09-15T22:38:26.573 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.574 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.575 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:38:26.578 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.579 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.580 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.582 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.583 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.584 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.585 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.586 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.587 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.588 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.589 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.591 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:38:26.592 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:38:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-09-15T22:38:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.598 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.599 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:38:26.600 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.601 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-09-15T22:38:26.602 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.603 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.604 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.605 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-09-15T22:38:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.612 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:38:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-09-15T22:38:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.626 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:38:26.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.632 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.633 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.634 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:38:26.635 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.636 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.638 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.639 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.640 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:38:26.644 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.645 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.646 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.647 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:38:26.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.651 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.652 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.654 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.655 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.657 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.659 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:38:26.664 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:38:26.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.670 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.674 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.675 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:26.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:38:26.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.680 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-09-15T22:38:26.682 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:26.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.687 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.688 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.689 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.690 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.692 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.693 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-09-15T22:38:26.694 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.695 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:26.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.698 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:26.699 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.700 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.701 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.704 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.705 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-09-15T22:38:26.706 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.707 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.708 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.710 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.715 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-09-15T22:38:26.716 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:38:26.717 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.720 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.721 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:38:26.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.723 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.734 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.736 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.737 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.739 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.740 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.741 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.742 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.743 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.744 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.745 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.747 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-09-15T22:38:26.748 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.752 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.753 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.754 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.756 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.757 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.758 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.759 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.760 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.762 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.764 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.766 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.767 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-09-15T22:38:26.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.769 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.770 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.771 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.772 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.774 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.776 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.777 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.779 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.780 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.781 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.782 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.783 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-09-15T22:38:26.787 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.788 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.789 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-09-15T22:38:26.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.791 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.792 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.794 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.795 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.796 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.798 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.799 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.800 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.801 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.803 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.804 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.806 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.816 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.817 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.818 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.819 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-09-15T22:38:26.820 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-09-15T22:38:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.822 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.823 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.827 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.828 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.829 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.830 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.833 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.835 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.837 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.838 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.839 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.841 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:26.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.844 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.845 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:26.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.848 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:38:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.882 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.883 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.890 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.891 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.892 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.893 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.894 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.895 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.896 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.897 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-09-15T22:38:26.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.900 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.901 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.904 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.905 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.907 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.908 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.909 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.912 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.915 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.916 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.917 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.918 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.920 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.921 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.922 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.923 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.924 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.926 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-09-15T22:38:26.927 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-09-15T22:38:26.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.929 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.930 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-09-15T22:38:26.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.933 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-09-15T22:38:26.934 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.935 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-09-15T22:38:26.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.942 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.943 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.944 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.945 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.947 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.948 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.949 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.950 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.951 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.952 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.953 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.956 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.957 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.958 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.959 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.960 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.961 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.962 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.963 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.964 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.965 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.966 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.967 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.968 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.969 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.970 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-09-15T22:38:26.971 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-09-15T22:38:26.975 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.977 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.978 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.979 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.980 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-09-15T22:38:26.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.983 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.984 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.985 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.986 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.987 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.988 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.989 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.990 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.991 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.993 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.994 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-09-15T22:38:26.995 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.996 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.997 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:26.998 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:26.999 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:27.000 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:27.001 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:27.002 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.003 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.004 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.004 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.004 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:27.005 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:27.006 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.007 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:27.008 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:27.009 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.010 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.011 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.013 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:38:27.014 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-09-15T22:38:27.016 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:27.017 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.018 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.019 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.021 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-09-15T22:38:27.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.024 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.025 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.027 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "services": { 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard": "https://172.21.15.77:8443/" 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "always_on_modules": { 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "octopus": [ 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-15T22:38:27.028 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "pacific": [ 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-15T22:38:27.029 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "quincy": [ 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.030 INFO:teuthology.orchestra.run.smithi077.stdout: "reef": [ 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "squid": [ 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-09-15T22:38:27.031 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "last_failure_osd_epoch": 91, 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "active_clients": [ 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-09-15T22:38:27.032 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 528490834 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "libcephsqlite", 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.033 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2292750152 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 905744937 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3133963273 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.035 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 415005644 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3292829639 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.036 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.037 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.037 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-09-15T22:38:27.037 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:27.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/540541230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3841595785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:25 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2012248190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.038 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:26 smithi117 ceph-mon[35021]: pgmap v823: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:26 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1186685821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/540541230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3841595785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:25 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2012248190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:26 smithi196 ceph-mon[35109]: pgmap v823: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-09-15T22:38:27.039 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:26 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1186685821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:38:27.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 msgr2=0x7fdfe80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 0x7fdfe80a5880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdff4000c00 tx=0x7fdff4032d10 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 msgr2=0x7fe0001983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c920 tx=0x7fdff000cde0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 shutdown_connections 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7fdfe80a33c0 0x7fdfe80a5880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe00010c090 0x7fe00019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe000103590 0x7fe000198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 --2- 172.21.15.77:0/1186685821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102c70 0x7fe0001983e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 >> 172.21.15.77:0/1186685821 conn(0x7fe0000fc6c0 msgr2=0x7fe0000fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 shutdown_connections 2024-09-15T22:38:27.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:25.408+0000 7fe006731640 1 -- 172.21.15.77:0/1186685821 wait complete. 2024-09-15T22:38:27.046 DEBUG:tasks.check_counter:Getting stats from a 2024-09-15T22:38:27.046 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-15T22:38:27.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:26 smithi077 ceph-mon[30980]: pgmap v823: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 134 KiB/s wr, 35 op/s 2024-09-15T22:38:27.046 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:26 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1186685821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:38:27.304 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 457911, 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 402419, 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 189541952, 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 301023143, 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 45, 2024-09-15T22:38:27.845 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 35.262265633, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 20.467318367, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 18.912995392, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 1.192242698, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 402417, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 23.252491044, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000057782 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 812750, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.121620302, 2024-09-15T22:38:27.846 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000149 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 47663952, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 166993312 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 516471, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 367278, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 243748562, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 231645007, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 53, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 23.535327636, 2024-09-15T22:38:27.847 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 9.021606081, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 17.173941949, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 1.459059902, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 367275, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 14.651017233, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000039891 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 839166, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.108752083, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000129 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.848 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 5456, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 133680 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 54866, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 88284, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 12472578, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 179255205, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 9, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 5.545405884, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 1.945900604, 2024-09-15T22:38:27.849 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 2.002588480, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 1.550810957, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 88284, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 5.277215896, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000059775 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 99078, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.018568127, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000187 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1830992, 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 962272 2024-09-15T22:38:27.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 44317, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 42.479586432, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000958539 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:27.851 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 25947, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.509171510, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000096703 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 96089, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 96016, 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-09-15T22:38:27.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 96016, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 565.868489391, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005893481 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 72, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 20, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 1, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 756, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 58, 2024-09-15T22:38:27.853 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 614, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 84, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 673, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 266, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 1, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 1, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 5, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 152977, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 87554, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 70, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 103, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:27.854 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 1061, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 2, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 57, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 2, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 67, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 1008, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 5, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 260330, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 1008, 2024-09-15T22:38:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 117, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 117, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 891, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 4, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 2946, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 39869, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 27458, 2024-09-15T22:38:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 11232, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 4009, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 67943 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 45, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 688, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 395, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 588, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 62, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 9, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 13327, 2024-09-15T22:38:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 13318, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 22, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 2085, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 2, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 3, 2024-09-15T22:38:27.858 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 62474, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 62474, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 107, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-09-15T22:38:27.859 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 107, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 7, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 356303656, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 22005, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 116.030285234, 2024-09-15T22:38:27.860 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.005272905 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 627, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 14095, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 13468, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 40, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 534, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 494, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 770, 2024-09-15T22:38:27.861 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 28535, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 27765, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 1, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 14823, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 14822, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 81792, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223520 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 144246, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 13, 2024-09-15T22:38:27.862 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 96089, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 1844, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 45, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 13263, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 145.460229692, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.010967370 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1999, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 14.252446079, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.007129787 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.863 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 30888, 2024-09-15T22:38:27.864 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 31.271385014, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001012412 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:27.866 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.867 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.867 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.867 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-09-15T22:38:27.867 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.868 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-09-15T22:38:27.869 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 33306, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 151.546310482, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004550120 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2781, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 70.975963837, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.025521741 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 9, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.131446779, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.014605197 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.871 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 24, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 26.800401637, 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 1.116683401 2024-09-15T22:38:27.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 428, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.332849958, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000777686 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.873 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-09-15T22:38:27.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 13318, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 125.097455913, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.009393111 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-09-15T22:38:27.875 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 4, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 2, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 10577, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 5288, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1137, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:27.876 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:27.877 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:27.878 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 383853, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 338, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:27.879 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 190, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 2265330, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 16222, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.880 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 51438, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 355152551, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 32, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 51438, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 51438, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 450.645907718, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.008760953 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-09-15T22:38:27.881 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 51438, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 52246 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 51438, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1447, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 49991, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 439, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 22444, 2024-09-15T22:38:27.882 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 265, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 1, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 26731, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1008, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:27.883 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-09-15T22:38:27.884 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 131, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 43, 2024-09-15T22:38:27.885 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 16, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 4, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 1030, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 298 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-09-15T22:38:27.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 2096 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 5, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 26636, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 13318, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.887 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1029244, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 347178001, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1029244, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1029244, 2024-09-15T22:38:27.888 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 347178001, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:27.889 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 51438, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 356607557, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 51438, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 356607557, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:27.890 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 51438, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 51438, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 51438, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 51438, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.891 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 13318, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1455006, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 13318, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:27.892 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 439, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1455006, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x564d11206960": { 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:27.893 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 62474, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 352109352, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 62474, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 22005, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 352109352, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.894 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:27.895 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:28.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:28 smithi077 ceph-mon[30980]: pgmap v824: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 120 KiB/s wr, 36 op/s 2024-09-15T22:38:28.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:28 smithi196 ceph-mon[35109]: pgmap v824: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 120 KiB/s wr, 36 op/s 2024-09-15T22:38:28.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:28 smithi117 ceph-mon[35021]: pgmap v824: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 120 KiB/s wr, 36 op/s 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds.exported=2 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds.imported=2 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=45 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=688 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:28.810 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:28.810 DEBUG:tasks.check_counter:Getting stats from d 2024-09-15T22:38:28.810 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-15T22:38:29.056 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:29.606 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:29.606 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001132891, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000207632, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:29.607 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:29.608 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 7, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 357, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3552, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 299232, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.025271814, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.017837663, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.003980950, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:29.609 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 355, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.028615194, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000080606 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:38:29.610 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 299232 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 473, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 462, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1658592, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 286352, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.062943437, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.026791592, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.048969284, 2024-09-15T22:38:29.611 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 461, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.022403188, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000048596 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1658592, 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 286352 2024-09-15T22:38:29.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:29.613 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:29.614 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:29.615 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148771, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 32, 2024-09-15T22:38:29.616 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:29.617 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:29.618 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:29.619 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 476, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1591745, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 476, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 476, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1591745, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:29.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:29.621 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:30.362 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-15T22:38:30.362 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-15T22:38:30.362 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-15T22:38:30.362 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-15T22:38:30.362 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-15T22:38:30.363 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:38:30.363 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:38:30.363 DEBUG:tasks.check_counter:Getting stats from g 2024-09-15T22:38:30.363 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-15T22:38:30.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:30 smithi077 ceph-mon[30980]: pgmap v825: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 13 op/s 2024-09-15T22:38:30.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:30 smithi196 ceph-mon[35109]: pgmap v825: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 13 op/s 2024-09-15T22:38:30.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:30 smithi117 ceph-mon[35021]: pgmap v825: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 13 op/s 2024-09-15T22:38:30.614 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2861, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2836, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 107398892, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1033454, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.326846846, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.103868178, 2024-09-15T22:38:31.119 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.161099290, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.059856205, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2835, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.102757583, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000036246 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.120 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1806768, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 292832 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2397, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2747, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 113857729, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1539283, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 9, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.229141409, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.055965617, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.108663694, 2024-09-15T22:38:31.121 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.050911653, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2744, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.087112728, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000031746 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 777616 2024-09-15T22:38:31.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 966, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 966, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 134270425, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 306376, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.153676372, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.018831938, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.107388633, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.021883836, 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:31.123 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 965, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.035671179, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000036964 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.124 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 9406, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.205508069, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000021848 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:31.125 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.126 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 14075, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 14075, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-09-15T22:38:31.127 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 14075, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-09-15T22:38:31.128 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:31.129 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:38:31.130 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-09-15T22:38:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-09-15T22:38:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 62474, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-09-15T22:38:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 107, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 7, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 356303656, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 356294946, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 62474 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 629, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 13994, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 13365, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 27, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 43, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 16, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 14075, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 14145, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 70, 2024-09-15T22:38:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 67892, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215328 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-09-15T22:38:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-09-15T22:38:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-09-15T22:38:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 4, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 2, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 4650, 2024-09-15T22:38:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 267882, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 829, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 190, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 11647172, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 74619, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-09-15T22:38:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 5686, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 5686, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5686, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 13.016736166, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002289260 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5686, 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 5687 2024-09-15T22:38:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 5686, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 5686, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 3435, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 2249, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-09-15T22:38:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-09-15T22:38:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-09-15T22:38:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 131, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 43, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 11, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 13, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 2, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-09-15T22:38:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-09-15T22:38:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 6220, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 354936951, 2024-09-15T22:38:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 6220, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 6220, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 354936951, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5686, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 352131604, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5686, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 352131604, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 5686, 2024-09-15T22:38:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 5686, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 5686, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5686, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55eef52dcb40": { 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:31.168 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:31.963 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:31.963 DEBUG:tasks.check_counter:Getting stats from j 2024-09-15T22:38:31.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-15T22:38:32.209 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:32.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:32 smithi077 ceph-mon[30980]: pgmap v826: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-15T22:38:32.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:32 smithi196 ceph-mon[35109]: pgmap v826: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-15T22:38:32.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:32 smithi117 ceph-mon[35021]: pgmap v826: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001229331, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000379437, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 7, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 358, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3552, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 299408, 2024-09-15T22:38:32.731 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.024765425, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.017094389, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.004174661, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 356, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.027383074, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000076918 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:32.732 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 299408 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 475, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 464, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1659136, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 287632, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-09-15T22:38:32.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.067529206, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.028171526, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.051790645, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 463, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.019789683, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000042742 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:32.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1659136, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 287632 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:32.735 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:32.736 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:32.737 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:32.738 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148731, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 31, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-09-15T22:38:32.739 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-09-15T22:38:32.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 478, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1592005, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 478, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 478, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1592005, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-09-15T22:38:32.741 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-09-15T22:38:32.742 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:32.742 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-09-15T22:38:32.742 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:38:33.488 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:38:33.489 DEBUG:tasks.check_counter:Getting stats from b 2024-09-15T22:38:33.489 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-15T22:38:33.871 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:34.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:34 smithi077 ceph-mon[30980]: pgmap v827: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-15T22:38:34.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:34 smithi196 ceph-mon[35109]: pgmap v827: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-15T22:38:34.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:34 smithi117 ceph-mon[35021]: pgmap v827: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 254868, 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 359717, 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 66648081, 2024-09-15T22:38:34.748 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 553375337, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 359, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 27.687615731, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 15.253095379, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 11.432227508, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 2.661421280, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 359717, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 32.944406443, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000091584 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 512913, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.110058137, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000214 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 96688, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 689952 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1012183, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1205187, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 404832418, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1141813500, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 13, 2024-09-15T22:38:34.750 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 62.151228420, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 26.369881589, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 42.119466275, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 3.950234001, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1205189, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 119.548043178, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000099194 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.751 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1948871, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.334270703, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000171 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 165613088, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 546720432 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 391949, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 633385, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 89917068, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 568143724, 2024-09-15T22:38:34.752 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 21, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 8, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 26.488531095, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 11.477169940, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 14.504096396, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 2.381034420, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 633378, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 76.332138376, 2024-09-15T22:38:34.753 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000120515 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 916696, 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.160791682, 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000175 2024-09-15T22:38:34.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2740560, 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 628432 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-15T22:38:34.755 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 74478, 2024-09-15T22:38:34.756 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 153.230811578, 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002057396 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:34.757 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 69592, 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 15.193399958, 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000218321 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.758 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 314772, 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 280555, 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 280555, 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4357.563690420, 2024-09-15T22:38:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.015531940 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 34217, 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:34.760 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 439, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 28, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 16, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 8335, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 3962, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 4362, 2024-09-15T22:38:34.761 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 11, 2024-09-15T22:38:34.762 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 2645, 2024-09-15T22:38:34.762 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 388, 2024-09-15T22:38:34.762 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2024-09-15T22:38:34.762 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 3, 2024-09-15T22:38:34.762 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 17, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 433952, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 253759, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 34214, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 31, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:34.763 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 2258, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 16, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 314, 2024-09-15T22:38:34.764 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 7, 2024-09-15T22:38:34.765 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 319, 2024-09-15T22:38:34.765 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:34.765 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 7752, 2024-09-15T22:38:34.765 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:34.765 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 6, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 7752, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:34.766 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 1390, 2024-09-15T22:38:34.767 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 33, 2024-09-15T22:38:34.767 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 1357, 2024-09-15T22:38:34.767 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 6362, 2024-09-15T22:38:34.767 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 18, 2024-09-15T22:38:34.767 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 63901, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 192864, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 120693, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 31948, 2024-09-15T22:38:34.768 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 8839, 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 194573 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 407, 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 85, 2024-09-15T22:38:34.769 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 52, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 64, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 1440, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 84, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 36869, 2024-09-15T22:38:34.770 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 36785, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 22, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 6308, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 33, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 16, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-15T22:38:34.771 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 50, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 189879, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 127938, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 122901, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 66978, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 5037, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 324, 2024-09-15T22:38:34.772 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 196, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 187, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 137, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 6, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 9, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 547482121, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 1095435861, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 43592, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 625.751223948, 2024-09-15T22:38:34.773 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.014354726 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 5323, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 42402, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 37079, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 211, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 512, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 301, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 8335, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 83373, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 75038, 2024-09-15T22:38:34.774 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 3, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 43238, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 43235, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 111072, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 231712 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 442007, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 3002, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 314772, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 5023, 2024-09-15T22:38:34.775 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 6086, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 36989, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1193.082350520, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.032255058 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5756, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 121.890993838, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.021176336 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.776 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 85649, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 650.268900931, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007592253 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.777 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000245965, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000245965 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.779 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 26, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.016068182, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000618007 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.780 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 119, 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.853120551, 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007169080 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.781 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 26, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 5.685835786, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.218685991 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.028968475, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002896847 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.782 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 105967, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1384.623821050, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.013066556 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7503, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 255.260719590, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.034021154 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 95, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.030202728, 2024-09-15T22:38:34.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.010844239 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 20, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 8.760064264, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.438003213 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-15T22:38:34.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 380, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 84.412308351, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.222137653 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1220, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 8.728603484, 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007154593 2024-09-15T22:38:34.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 9, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.021659116, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002406568 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 36785, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 642.899827589, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.017477227 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:34.788 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 7, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 5, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 32234, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 16117, 2024-09-15T22:38:34.789 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1548, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:34.790 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:34.791 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:34.792 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 1037734, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 2110, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 264, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 3, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 190, 2024-09-15T22:38:34.793 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 18998375, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 154621, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-15T22:38:34.794 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 130289, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 1104973981, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 37, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 130289, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 130289, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1516.101347522, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.011636449 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 130289, 2024-09-15T22:38:34.795 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 133398 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 130289, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 8740, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 121549, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 104, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 891, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 955, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 44514, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 525, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-15T22:38:34.796 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 28, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 74585, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 7785, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 1393, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-15T22:38:34.797 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-15T22:38:34.798 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 131, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 49, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-15T22:38:34.799 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 1937, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 681 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 6367 2024-09-15T22:38:34.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 5, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 73570, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 36785, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:34.801 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1658996, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 391467380, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1658996, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1658996, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 391467380, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 130289, 2024-09-15T22:38:34.803 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1109009946, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 130289, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1109009946, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-15T22:38:34.804 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 130289, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 130289, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 130289, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 130289, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 36785, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 4035965, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 36785, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:34.806 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 922, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 4035965, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x563d2e0be960": { 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:34.807 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 189879, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1091241557, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 189879, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 43592, 2024-09-15T22:38:34.808 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1091241557, 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:34.809 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds.exported=16 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=407 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=85 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=26 2024-09-15T22:38:35.680 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=20 2024-09-15T22:38:35.680 DEBUG:tasks.check_counter:Getting stats from e 2024-09-15T22:38:35.681 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-15T22:38:35.928 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3135, 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3135, 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 391012149, 2024-09-15T22:38:36.424 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 996116, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.447236112, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.088988210, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.258026491, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.077311149, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3134, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.125683840, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000040103 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.425 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2328, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2300, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 351132566, 2024-09-15T22:38:36.426 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 864202, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.534777772, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.061159855, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.299075287, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.181210112, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2299, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.088262006, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000038391 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.427 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1825872, 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 295552 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.428 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2220, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2573, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 352918769, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1573167, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 9, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.382443650, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.052693196, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.260180420, 2024-09-15T22:38:36.429 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.052245385, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2570, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.088999733, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000034630 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 868160 2024-09-15T22:38:36.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10972, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.321445713, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000029296 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.431 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-15T22:38:36.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 43265, 2024-09-15T22:38:36.433 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 18424, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 24841, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 16945, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 2, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:36.434 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 6, 2024-09-15T22:38:36.435 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:36.436 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-15T22:38:36.437 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:38:36.438 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-15T22:38:36.439 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 189879, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 137, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 17, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 547482121, 2024-09-15T22:38:36.440 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 1095435861, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 1095390364, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 189879 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-15T22:38:36.441 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 5312, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 42326, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 37014, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 200, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 371, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 171, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 43265, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 43560, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 295, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-15T22:38:36.442 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 108408, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215328 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.443 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.444 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.445 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.446 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.447 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.448 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:36.451 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.453 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.454 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.456 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 7, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 5, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 5, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 4710, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.457 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:36.458 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:36.459 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 462063, 2024-09-15T22:38:36.460 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 6095, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 190, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:36.461 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 44004004, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 320411, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 7136, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 26, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 4, 2024-09-15T22:38:36.462 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 7136, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7136, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 14.599743328, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002045928 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7136, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7137 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 7136, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 7136, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-15T22:38:36.463 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 4512, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 2621, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-15T22:38:36.464 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 2, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:36.465 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-15T22:38:36.466 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 131, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 43, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 11, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 1, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-15T22:38:36.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:38:36.468 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 7679, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1094341647, 2024-09-15T22:38:36.469 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 7679, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7679, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1094341647, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.470 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:36.471 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:36.472 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:36.472 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:36.472 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7136, 2024-09-15T22:38:36.472 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1091274514, 2024-09-15T22:38:36.472 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7136, 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1091274514, 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.473 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:36.474 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7136, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 7136, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7136, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7136, 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:36.475 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:36.476 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:36.477 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.478 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55ccbabe4b40": { 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:36.479 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:36.481 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T22:38:36.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:36 smithi077 ceph-mon[30980]: pgmap v828: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:36.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:36 smithi196 ceph-mon[35109]: pgmap v828: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:36.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:36 smithi117 ceph-mon[35021]: pgmap v828: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:37.279 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:37.279 DEBUG:tasks.check_counter:Getting stats from h 2024-09-15T22:38:37.280 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-15T22:38:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:37.990 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T22:38:37.990 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:37.990 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 333812, 2024-09-15T22:38:37.991 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 229122, 2024-09-15T22:38:37.991 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 142230068, 2024-09-15T22:38:37.991 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 156456554, 2024-09-15T22:38:37.991 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 41, 2024-09-15T22:38:37.991 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-09-15T22:38:37.992 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 17.275896675, 2024-09-15T22:38:37.992 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 7.661105411, 2024-09-15T22:38:37.992 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 11.424639118, 2024-09-15T22:38:37.992 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.715599624, 2024-09-15T22:38:37.992 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 229120, 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 8.173089287, 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000035671 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:37.993 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 515262, 2024-09-15T22:38:37.994 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.078543666, 2024-09-15T22:38:37.994 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000152 2024-09-15T22:38:37.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:37.994 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:37.994 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 286663, 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 325328, 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 100767658, 2024-09-15T22:38:37.995 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 317831869, 2024-09-15T22:38:37.996 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 13, 2024-09-15T22:38:37.996 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-15T22:38:37.996 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 17.849057322, 2024-09-15T22:38:37.997 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 7.604193899, 2024-09-15T22:38:37.997 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 12.017140453, 2024-09-15T22:38:37.997 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.767660267, 2024-09-15T22:38:37.997 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:37.997 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 325327, 2024-09-15T22:38:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 20.769345211, 2024-09-15T22:38:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000063841 2024-09-15T22:38:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:37.998 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 561617, 2024-09-15T22:38:37.999 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.089973499, 2024-09-15T22:38:37.999 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000160 2024-09-15T22:38:37.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:37.999 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 63297872, 2024-09-15T22:38:38.000 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 164717280 2024-09-15T22:38:38.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.000 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:38.000 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 444414, 2024-09-15T22:38:38.000 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 328600, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 212876369, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 172838071, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 50, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 8, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 18.971301595, 2024-09-15T22:38:38.001 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 7.132359756, 2024-09-15T22:38:38.002 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 14.013674369, 2024-09-15T22:38:38.002 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.985250807, 2024-09-15T22:38:38.002 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:38.002 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 328598, 2024-09-15T22:38:38.002 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 14.214023779, 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000043256 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 741939, 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.098910755, 2024-09-15T22:38:38.003 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000133 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 782384 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-15T22:38:38.004 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-15T22:38:38.005 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-15T22:38:38.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.005 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-15T22:38:38.005 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:38.005 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:38.006 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37629, 2024-09-15T22:38:38.006 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 36.818631925, 2024-09-15T22:38:38.006 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000978464 2024-09-15T22:38:38.007 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.007 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:38.007 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 21500, 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.938859305, 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000090179 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.009 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-15T22:38:38.009 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 136776, 2024-09-15T22:38:38.009 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 90610, 2024-09-15T22:38:38.009 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-15T22:38:38.009 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 90610, 2024-09-15T22:38:38.010 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 603.823262034, 2024-09-15T22:38:38.010 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006663980 2024-09-15T22:38:38.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.010 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-15T22:38:38.010 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 46166, 2024-09-15T22:38:38.011 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:38.011 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:38.011 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 29, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 2, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 890, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 117, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 701, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 72, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 702, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 322, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:38.012 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 5, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 192981, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 91831, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 46165, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 115, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 1787, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 2, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 93, 2024-09-15T22:38:38.013 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 2, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 72, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 1135, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 5, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 260330, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 6, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 1135, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:38.014 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 130, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 130, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 1005, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 8, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 3309, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 35102, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 28752, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 9318, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 3435, 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 69057 2024-09-15T22:38:38.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 202, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 570, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 336, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 494, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 219, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 10, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 11023, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 11013, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:38.016 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 22, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 2184, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 2, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 2, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:38.017 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 54794, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 54794, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 89, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 89, 2024-09-15T22:38:38.018 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 6, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 287164414, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 18723, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 118.943523517, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006352802 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-15T22:38:38.019 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 724, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 11931, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 11207, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 47, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 486, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 439, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 891, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 22073, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 21182, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 12882, 2024-09-15T22:38:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 12882, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 79884, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 223520 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 186717, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 13, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 136776, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 1476, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 1087, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10953, 2024-09-15T22:38:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 306.072887733, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.027944205 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4589, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 34.045082935, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007418845 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-15T22:38:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 25309, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 44.347070937, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001752225 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35885, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 137.119219497, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003821073 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2477, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 48.581999549, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.019613241 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.188663037, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.018866303 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-15T22:38:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 34, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.596575554, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.017546339 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 340, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.302469677, 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000889616 2024-09-15T22:38:38.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.033 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11013, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 32.569293115, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002957349 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.034 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 10141, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 10141, 2024-09-15T22:38:38.035 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1571, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:38.036 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:38.037 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:38.038 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 400398, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 353, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:38.039 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 190, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 2565303, 2024-09-15T22:38:38.040 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 18237, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 43905, 2024-09-15T22:38:38.041 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 285663282, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 30, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 43905, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 43905, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 392.757094291, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008945611 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 43905, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 44784 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.042 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 43905, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 1671, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 42234, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 536, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 19259, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 318, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-15T22:38:38.043 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 2, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 22105, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1135, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:38.044 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-15T22:38:38.045 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 131, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 43, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-09-15T22:38:38.046 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 14, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 2, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 1032, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 367 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.047 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 2197 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 5, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 22026, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 11013, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:38.048 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1064885, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 352310795, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1064885, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:38.049 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1064885, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 352310795, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:38.050 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 43905, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 286867547, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 43905, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 286867547, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.051 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:38.052 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 43905, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 43905, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 43905, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 43905, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.053 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 11013, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1204265, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 11013, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 536, 2024-09-15T22:38:38.054 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1204265, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55fe3204e960": { 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 54794, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 282970110, 2024-09-15T22:38:38.055 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 54794, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 18723, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 282970110, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.056 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:38.057 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T22:38:38.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:38 smithi077 ceph-mon[30980]: pgmap v829: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:38.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:38 smithi196 ceph-mon[35109]: pgmap v829: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:38.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:38 smithi117 ceph-mon[35021]: pgmap v829: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:38.862 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=202 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=570 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:38.863 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:38.863 DEBUG:tasks.check_counter:Getting stats from k 2024-09-15T22:38:38.863 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-15T22:38:39.105 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:39.615 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T22:38:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:39.615 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2076, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2078, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 478717796, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 660232, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-15T22:38:39.616 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.395504699, 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.045409914, 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.286018590, 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.050328512, 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2076, 2024-09-15T22:38:39.617 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.057433535, 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000027665 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.618 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2635, 2024-09-15T22:38:39.619 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2605, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 476536464, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 962067, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.479946285, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.065729063, 2024-09-15T22:38:39.620 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.368806690, 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.047443771, 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2605, 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.086397275, 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000033165 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.621 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1829600, 2024-09-15T22:38:39.622 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 297760 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3577, 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3932, 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 466642756, 2024-09-15T22:38:39.623 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1916962, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.542007870, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.093710204, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.326737706, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.093634155, 2024-09-15T22:38:39.624 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3929, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.139654948, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000035544 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 780112 2024-09-15T22:38:39.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11958, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.339173032, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000028363 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.626 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-15T22:38:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-15T22:38:39.628 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 45241, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 24054, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 21187, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 10878, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:39.629 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-09-15T22:38:39.630 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:39.631 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-15T22:38:39.632 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:39.633 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 205984, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-15T22:38:39.634 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 136, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 18, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 868754916, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 1422031190, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 1422030591, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 205984 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 1491, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 45122, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 43631, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 72, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 288, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 216, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 45241, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 45294, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 53, 2024-09-15T22:38:39.636 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 98572, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215328 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-15T22:38:39.637 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.639 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.640 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.642 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-15T22:38:39.642 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.642 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.642 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.643 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.644 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.646 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.647 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.648 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.649 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 2827, 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:39.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:39.651 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:39.652 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:39.653 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 305417, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 1816, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:39.654 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 190, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 35891656, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 222456, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-15T22:38:39.655 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 7740, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 7740, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7740, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 14.147749005, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001827874 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-15T22:38:39.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7740, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7741 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 7740, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 7740, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 4704, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-15T22:38:39.657 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 3034, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-09-15T22:38:39.658 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-15T22:38:39.659 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-15T22:38:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 131, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 43, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 11, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 11, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-15T22:38:39.661 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.662 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 8284, 2024-09-15T22:38:39.663 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1421120662, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 8284, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 8284, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1421120662, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7740, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1417947802, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7740, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1417947802, 2024-09-15T22:38:39.665 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-15T22:38:39.666 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7740, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 7740, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7740, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7740, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:39.668 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55fafe45ab40": { 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-15T22:38:39.669 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.670 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T22:38:39.671 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T22:38:40.439 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-15T22:38:40.439 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-15T22:38:40.440 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-15T22:38:40.440 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-15T22:38:40.440 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-15T22:38:40.440 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:40.440 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:40.440 DEBUG:tasks.check_counter:Getting stats from c 2024-09-15T22:38:40.440 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-15T22:38:40.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:40 smithi077 ceph-mon[30980]: pgmap v830: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:40.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:40 smithi196 ceph-mon[35109]: pgmap v830: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:40.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:40 smithi117 ceph-mon[35021]: pgmap v830: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:40.744 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 345458, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 212236, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 152832493, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 46183870, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 41, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 14.030658660, 2024-09-15T22:38:41.466 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 5.310863711, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 11.816855446, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.117791447, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 212236, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5.048373345, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000023786 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 545608, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.070799772, 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000129 2024-09-15T22:38:41.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 19642560, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 15737840 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 432183, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 216730, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 226875042, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 35536541, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 49, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 17.634462117, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 7.436135129, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 13.878994358, 2024-09-15T22:38:41.468 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.017570956, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 216727, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3.164783950, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000014602 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 647430, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.080679865, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000124 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 7632, 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 148400 2024-09-15T22:38:41.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 69521, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 36688, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 33196598, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 13027050, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 20, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 2.848931963, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 1.047007801, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 2.301053065, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.069682081, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 36686, 2024-09-15T22:38:41.470 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.849497913, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000023155 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 99944, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.013874371, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000138 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1841728, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 971616 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-15T22:38:41.471 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5334, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.112782175, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000396097 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2342, 2024-09-15T22:38:41.472 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.126143035, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000053861 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 56061, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 4264, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4264, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 40.399822605, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.009474630 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-15T22:38:41.473 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 51797, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 18, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 2, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 2, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 14, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 14, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 353, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-09-15T22:38:41.474 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 62767, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 5084, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 51796, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 108, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 179, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 2, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 127, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 2, 2024-09-15T22:38:41.475 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 11, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 119, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 260330, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 119, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:41.476 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 119, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 10, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 334, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 3440, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 3078, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 1816, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 324, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 4200 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-15T22:38:41.477 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 366, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 459, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 281, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 366, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 389, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 836, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 836, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:41.478 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 22, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 1502, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 2, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-15T22:38:41.479 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 6323, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6323, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 8, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 8, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-15T22:38:41.480 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 14030146, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2475, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 17.247375927, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.006968636 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 26, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1081, 2024-09-15T22:38:41.481 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1055, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 18, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 424, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 406, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 19, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 1326, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 1307, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 907, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 906, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 64880, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.482 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 60669, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 13, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 56061, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 319, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 1905, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 825, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 27.870553528, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.033782489 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 84, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.158079545, 2024-09-15T22:38:41.483 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001881899 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-15T22:38:41.484 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1173, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.560376055, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003887788 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000150050, 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000150050 2024-09-15T22:38:41.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.487 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-15T22:38:41.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.487 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1145, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.731594118, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000638946 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-15T22:38:41.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 180, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.382283639, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.013234909 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.490 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.491 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 20, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.114385481, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005719274 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:41.492 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.493 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 836, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.582400189, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005481339 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:41.494 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 458, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 229, 2024-09-15T22:38:41.495 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1546, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:41.496 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:41.497 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:41.498 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 296480, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 119, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 190, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:41.499 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 195733, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 708, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-15T22:38:41.500 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 5252, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 10066963, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 31, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 5252, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5252, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 61.059632224, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.011625977 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5252, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5705 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.501 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 5252, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 349, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 4903, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 230, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 2705, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 184, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-15T22:38:41.502 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 1686, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 119, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-15T22:38:41.503 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-15T22:38:41.504 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 131, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 43, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 17, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 5, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 551, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 200 2024-09-15T22:38:41.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 1504 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 5, 2024-09-15T22:38:41.506 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 1672, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 836, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 847158, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 334080165, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 847158, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:41.507 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 847158, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 334080165, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:41.508 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5252, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 10158687, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5252, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 10158687, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:41.509 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5252, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 5252, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5252, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5252, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.510 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 836, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 91724, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 836, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:41.511 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 230, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 91724, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x56517ccc8960": { 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 6323, 2024-09-15T22:38:41.512 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 9835842, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 6323, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2475, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 9835842, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:41.513 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=366 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=459 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:42.400 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:42.400 DEBUG:tasks.check_counter:Getting stats from f 2024-09-15T22:38:42.401 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-15T22:38:42.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:42 smithi077 ceph-mon[30980]: pgmap v831: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-15T22:38:42.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:42 smithi196 ceph-mon[35109]: pgmap v831: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-15T22:38:42.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:42 smithi117 ceph-mon[35021]: pgmap v831: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-15T22:38:42.648 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 4296, 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4298, 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 95460130, 2024-09-15T22:38:43.147 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1366135, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.361855868, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.096092324, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.130372428, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.111072117, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4296, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.139185884, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000032398 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:43.148 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 600, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 952, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 84040956, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 972570, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 9, 2024-09-15T22:38:43.149 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.112571094, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.023510226, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.066992163, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.015201696, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 950, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.045980538, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000048400 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.150 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 783616 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1275, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1247, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 106873737, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 529762, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.182506111, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.041466037, 2024-09-15T22:38:43.151 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.136269328, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.017168879, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1246, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.053948549, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000043297 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1825984, 2024-09-15T22:38:43.152 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 296032 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 9392, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.210023466, 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000022361 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.153 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-15T22:38:43.154 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 11895, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-15T22:38:43.155 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 11895, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 11895, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:43.156 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-15T22:38:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-15T22:38:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:38:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 54794, 2024-09-15T22:38:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 89, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 6, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 287164414, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 287163815, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.162 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 54794 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 736, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 11830, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 11094, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 35, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 51, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 16, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 11895, 2024-09-15T22:38:43.163 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 11998, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 103, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 65324, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-15T22:38:43.164 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.165 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:43.166 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.167 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.169 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.170 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.171 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.172 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:43.173 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-15T22:38:43.174 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-15T22:38:43.176 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 2824, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:43.177 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:43.178 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:43.179 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 271656, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 931, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:43.180 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 190, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 10360276, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 68433, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.181 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 5625, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 5625, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5625, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 10.979753002, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001951956 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-15T22:38:43.182 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5625, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5626 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 5625, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 5625, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 3381, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 2242, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-15T22:38:43.183 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-15T22:38:43.184 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-15T22:38:43.185 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 131, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 43, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 11, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 1, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-15T22:38:43.186 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-15T22:38:43.187 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 6167, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 285789103, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 6167, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:43.188 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 6167, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 285789103, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:43.189 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5625, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 282975577, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5625, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 282975577, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-15T22:38:43.190 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5625, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 5625, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5625, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5625, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.191 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:43.192 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55d41640ab40": { 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:43.193 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:43.194 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-15T22:38:43.871 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-15T22:38:43.871 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-15T22:38:43.872 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-15T22:38:43.872 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-15T22:38:43.872 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-15T22:38:43.872 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:43.872 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:43.872 DEBUG:tasks.check_counter:Getting stats from i 2024-09-15T22:38:43.872 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-15T22:38:44.117 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:44.539 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:44 smithi196 ceph-mon[35109]: pgmap v832: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 13 op/s 2024-09-15T22:38:44.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:44 smithi077 ceph-mon[30980]: pgmap v832: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 13 op/s 2024-09-15T22:38:44.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:44 smithi117 ceph-mon[35021]: pgmap v832: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 13 op/s 2024-09-15T22:38:44.647 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-15T22:38:44.647 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:44.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 833654, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1155266, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 264236129, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1246235261, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 20, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 47.923318829, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 21.034076810, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 30.465603786, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 2.230444741, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1155264, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 31.733268864, 2024-09-15T22:38:44.648 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000027468 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1814584, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.258533815, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000142 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 191883104, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 629663280 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:44.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 324148, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 473401, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 80680948, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 937665554, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 12, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 22.098122982, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 9.724866072, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 10.596649479, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 2.282787596, 2024-09-15T22:38:44.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 473401, 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 13.571618150, 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000028668 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 640659, 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.126487778, 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000197 2024-09-15T22:38:44.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1851792, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 325264 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 682858, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 907588, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 204153319, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 745491781, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 70, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 37.865069122, 2024-09-15T22:38:44.652 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 17.381860842, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 23.138283097, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 1.129736920, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 907585, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 21.262825932, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000023427 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1496276, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.230607122, 2024-09-15T22:38:44.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000154 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 782640 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:44.654 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 142447, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 113.052379628, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000793645 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 82107, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 8.427385502, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000102639 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.655 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 421928, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 392446, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 392446, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1125.398977102, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002867653 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 29482, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:44.656 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 310, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 28, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 32, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 4016, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 2405, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1556, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 55, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 578, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 254, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 6, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 606839, 2024-09-15T22:38:44.657 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 370699, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 29482, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 113, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 5195, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 1, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 42, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 10, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 164, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 2348, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:44.658 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 5, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 260330, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 2348, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 290, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 290, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 2058, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 4, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 9333, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 138502, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:44.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 85518, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 35082, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 15706, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 293448 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 22, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 711, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 407, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 628, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 30, 2024-09-15T22:38:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 10, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 43596, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 43586, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 22, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 2864, 2024-09-15T22:38:44.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 1, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 60, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 205984, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 163417, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 159771, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 46213, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 3646, 2024-09-15T22:38:44.662 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 401, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 273, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 265, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 136, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 4, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 8, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 868754916, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 1422031190, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 70598, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 185.951713634, 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002633951 2024-09-15T22:38:44.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 1502, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 45226, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 43724, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 84, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 740, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 656, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 4016, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 92168, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 88152, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-15T22:38:44.664 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 48563, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 48563, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 93740, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223520 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 639598, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 12, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 421928, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 5996, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 74, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 43512, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 307.924375501, 2024-09-15T22:38:44.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.007076769 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 7666, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 22.651971232, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002954861 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 105683, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 79.044269739, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000747937 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-15T22:38:44.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 181435, 2024-09-15T22:38:44.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 437.010148970, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002408632 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 9090, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 22.452379225, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002470008 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 10, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.156396849, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.015639684 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 68, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 102.702095076, 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 1.510324927 2024-09-15T22:38:44.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1396, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.548006301, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000392554 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-15T22:38:44.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.676 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 43586, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 152.909334209, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003508221 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.677 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 11032, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 11032, 2024-09-15T22:38:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1489, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:44.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:44.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:44.681 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 412791, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 693, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 88, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 1, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:44.682 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 190, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 6819876, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 61653, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-15T22:38:44.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 165882, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 1429990677, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 165882, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 165882, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 641.362434531, 2024-09-15T22:38:44.684 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003866377 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 165882, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 168074 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 165882, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3776, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 162106, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 32, 2024-09-15T22:38:44.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 31, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1428, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 72026, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 734, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 26, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 87909, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2348, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 62, 2024-09-15T22:38:44.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-15T22:38:44.687 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 131, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 43, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 12, 2024-09-15T22:38:44.688 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 2624, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 854 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 2806 2024-09-15T22:38:44.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 5, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 87172, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 43586, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:44.690 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1840656, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 359730330, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1840656, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1840656, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 359730330, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 165882, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1434751935, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 165882, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1434751935, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.692 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 165882, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 165882, 2024-09-15T22:38:44.693 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 165882, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 165882, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 43586, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 4761258, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:44.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 43586, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1428, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 4761258, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x563697134960": { 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:44.695 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 205984, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1417836886, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 205984, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 70598, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1417836886, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:44.696 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=22 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=711 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-15T22:38:45.501 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:45.502 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:45.502 DEBUG:tasks.check_counter:Getting stats from l 2024-09-15T22:38:45.502 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-15T22:38:45.751 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:46.300 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-15T22:38:46.300 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:38:46.300 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 3514, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3516, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 6679303, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1117453, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.284393618, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.091136666, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.075059510, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.093665371, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3514, 2024-09-15T22:38:46.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.123346113, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000035101 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:38:46.302 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 707, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 679, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1867938, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 350658, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.087918939, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.033932541, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.067728876, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.004868354, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 678, 2024-09-15T22:38:46.303 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.035057801, 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000051707 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.304 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1827680, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 299392 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1557, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1913, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4584160, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1279154, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.146045886, 2024-09-15T22:38:46.305 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.055098042, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.034173387, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.043359146, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1911, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.079405189, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000041551 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.306 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 785936 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:46.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8664, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.210117511, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000024251 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.308 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-09-15T22:38:46.309 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 843, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 843, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 843, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-09-15T22:38:46.310 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:38:46.311 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:38:46.312 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-09-15T22:38:46.313 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:38:46.314 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-09-15T22:38:46.315 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6323, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-09-15T22:38:46.316 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 8, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-09-15T22:38:46.317 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 14030146, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 14021436, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 6323 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-09-15T22:38:46.318 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 10, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 967, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 957, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 8, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 24, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 16, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 843, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 981, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 138, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 51968, 2024-09-15T22:38:46.319 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215328 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.320 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-09-15T22:38:46.321 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.321 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.321 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.321 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-09-15T22:38:46.322 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.323 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.323 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.323 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.323 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-09-15T22:38:46.323 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.324 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.324 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.324 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.324 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-09-15T22:38:46.324 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.325 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.325 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.325 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.325 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-09-15T22:38:46.325 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.326 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.327 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.327 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-09-15T22:38:46.327 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.327 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.328 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.328 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-09-15T22:38:46.328 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.328 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.329 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-09-15T22:38:46.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.329 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.330 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.331 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-09-15T22:38:46.331 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.331 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.331 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.332 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-09-15T22:38:46.332 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.332 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.332 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.333 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-09-15T22:38:46.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.333 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.334 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.334 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-09-15T22:38:46.334 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.334 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.334 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.335 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.336 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-09-15T22:38:46.336 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.336 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.336 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.336 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.337 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-09-15T22:38:46.337 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.337 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.337 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.337 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.338 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-09-15T22:38:46.338 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.338 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.338 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.338 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.339 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-09-15T22:38:46.339 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.339 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.339 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.339 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.340 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.341 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.342 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.342 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.342 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-09-15T22:38:46.342 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.342 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-09-15T22:38:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.344 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.344 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.344 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-09-15T22:38:46.344 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.344 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.346 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.346 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-09-15T22:38:46.346 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.346 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.346 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:38:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:38:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-09-15T22:38:46.347 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-09-15T22:38:46.348 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-09-15T22:38:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-09-15T22:38:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-09-15T22:38:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 4, 2024-09-15T22:38:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-09-15T22:38:46.349 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-09-15T22:38:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-09-15T22:38:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 4080, 2024-09-15T22:38:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-09-15T22:38:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.350 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:38:46.351 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:38:46.352 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:38:46.352 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:38:46.352 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:38:46.352 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:38:46.352 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:38:46.353 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:38:46.353 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:38:46.353 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:38:46.353 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:38:46.353 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-09-15T22:38:46.354 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:38:46.354 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-09-15T22:38:46.354 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:38:46.354 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:38:46.354 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-09-15T22:38:46.355 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:38:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:38:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:38:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-09-15T22:38:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-09-15T22:38:46.356 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-09-15T22:38:46.357 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:38:46.357 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:38:46.357 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:38:46.357 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:38:46.357 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 207897, 2024-09-15T22:38:46.358 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 64, 2024-09-15T22:38:46.358 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:38:46.358 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-09-15T22:38:46.358 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-09-15T22:38:46.358 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9416, 2024-09-15T22:38:46.359 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 190, 2024-09-15T22:38:46.360 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:38:46.360 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:38:46.360 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-09-15T22:38:46.360 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-09-15T22:38:46.360 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 655592, 2024-09-15T22:38:46.361 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 3513, 2024-09-15T22:38:46.361 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-09-15T22:38:46.361 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-09-15T22:38:46.361 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-09-15T22:38:46.361 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 5226, 2024-09-15T22:38:46.362 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 5226, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5226, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 8.400810982, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001607503 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5226, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 5227 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.363 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 5226, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 5226, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 3272, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1952, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-09-15T22:38:46.364 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-09-15T22:38:46.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-09-15T22:38:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 131, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 43, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 7, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 10, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 3, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-09-15T22:38:46.367 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.368 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 5774, 2024-09-15T22:38:46.369 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 12580739, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 5774, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5774, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12580739, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.370 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5226, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 9835847, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5226, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 9835847, 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:46.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:46.372 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 5226, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 5226, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 5226, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 5226, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:46.373 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-15T22:38:46.374 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55ff6e254b40": { 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-09-15T22:38:46.375 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-09-15T22:38:46.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-09-15T22:38:46.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.376 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-15T22:38:46.376 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-15T22:38:46.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:46 smithi077 ceph-mon[30980]: pgmap v833: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:46.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:46 smithi196 ceph-mon[35109]: pgmap v833: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:46.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:46 smithi117 ceph-mon[35021]: pgmap v833: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:38:47.151 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:38:47.152 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T22:38:47.161 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T22:38:47.162 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:38:47.162 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:38:47.188 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-09-15T22:38:47.188 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:38:47.245 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:38:47.538 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:48.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 -- 172.21.15.77:0/2917812071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4270101df0 msgr2=0x7f42701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 --2- 172.21.15.77:0/2917812071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4270101df0 0x7f42701021d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f426c009900 tx=0x7f426c02a410 comp rx=0 tx=0).stop 2024-09-15T22:38:48.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 -- 172.21.15.77:0/2917812071 shutdown_connections 2024-09-15T22:38:48.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 --2- 172.21.15.77:0/2917812071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270106f20 0x7f427010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 --2- 172.21.15.77:0/2917812071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f42701027a0 0x7f42701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 --2- 172.21.15.77:0/2917812071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4270101df0 0x7f42701021d0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 -- 172.21.15.77:0/2917812071 >> 172.21.15.77:0/2917812071 conn(0x7f42700fd8b0 msgr2=0x7f42700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 -- 172.21.15.77:0/2917812071 shutdown_connections 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.004+0000 7f42777b5640 1 -- 172.21.15.77:0/2917812071 wait complete. 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 Processor -- start 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 -- start start 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270101df0 0x7f42700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270106f20 0x7f4270074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4270111f80 con 0x7f42701027a0 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4270111e00 con 0x7f4270101df0 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f42777b5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112100 con 0x7f4270106f20 2024-09-15T22:38:48.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f4274d50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f4274d50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43480/0 (socket says 172.21.15.77:43480) 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.005+0000 7f4274d50640 1 -- 172.21.15.77:0/1601370401 learned_addr learned my addr 172.21.15.77:0/1601370401 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 -- 172.21.15.77:0/1601370401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270106f20 msgr2=0x7f4270074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4275551640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270101df0 0x7f42700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270106f20 0x7f4270074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 -- 172.21.15.77:0/1601370401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270101df0 msgr2=0x7f42700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270101df0 0x7f42700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 -- 172.21.15.77:0/1601370401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42701a68c0 con 0x7f42701027a0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f4274d50640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f426000c970 tx=0x7f426000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260007bb0 con 0x7f42701027a0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701a6bb0 con 0x7f42701027a0 2024-09-15T22:38:48.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4260007d50 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000f450 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.006+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a70a0 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.007+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42701021d0 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.008+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4260005020 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.009+0000 7f42667fc640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 0x7f425c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.009+0000 7f4275551640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 0x7f425c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.009+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f42600cff50 con 0x7f42701027a0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.009+0000 7f4275551640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 0x7f425c0a5870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f426c0096c0 tx=0x7f426c000bc0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.061 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.013+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4260014030 con 0x7f42701027a0 2024-09-15T22:38:48.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.192+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4270109b70 con 0x7f42701027a0 2024-09-15T22:38:48.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.192+0000 7f42667fc640 1 -- 172.21.15.77:0/1601370401 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v131) ==== 81+0+944 (secure 0 0 0) 0x7f426009c5a0 con 0x7f42701027a0 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/999829703 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3279424814 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2559994639 2024-09-16T22:02:40.710885+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/573117065 2024-09-16T22:02:40.710885+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3493953936 2024-09-16T22:02:40.710885+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3785009540 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:38:48.194 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3468864668 2024-09-16T22:03:04.101979+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1397043946 2024-09-16T22:03:04.101979+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1809193257 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1437248041 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1757809354 2024-09-16T22:03:04.101979+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/3382222599 2024-09-16T22:02:40.710885+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1261342424 2024-09-16T22:03:04.101979+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1809193257 2024-09-16T22:12:24.385790+0000 2024-09-15T22:38:48.195 INFO:teuthology.orchestra.run.smithi077.stderr:listed 16 entries 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.194+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 msgr2=0x7f425c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.194+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 0x7f425c0a5870 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f426c0096c0 tx=0x7f426c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.194+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 msgr2=0x7f4270074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.194+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f426000c970 tx=0x7f426000ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 shutdown_connections 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f425c0a33b0 0x7f425c0a5870 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4270106f20 0x7f4270074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.196 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42701027a0 0x7f4270074a30 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 --2- 172.21.15.77:0/1601370401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4270101df0 0x7f42700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 >> 172.21.15.77:0/1601370401 conn(0x7f42700fd8b0 msgr2=0x7f42700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 shutdown_connections 2024-09-15T22:38:48.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:48.195+0000 7f42777b5640 1 -- 172.21.15.77:0/1601370401 wait complete. 2024-09-15T22:38:48.581 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:48 smithi077 ceph-mon[30980]: pgmap v834: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:48.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:48 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1601370401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:38:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:48 smithi196 ceph-mon[35109]: pgmap v834: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:48.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:48 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1601370401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:38:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:48 smithi117 ceph-mon[35021]: pgmap v834: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 14 op/s 2024-09-15T22:38:48.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:48 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1601370401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:38:48.967 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T22:38:48.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:48.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:49.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T22:38:49.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:49.137 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T22:38:49.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:49.180 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-09-15T22:38:49.234 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T22:38:49.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:49.235 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T22:38:49.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:49.291 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-09-15T22:38:49.394 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-15T22:38:49.405 INFO:tasks.cephadm:Teardown begin 2024-09-15T22:38:49.405 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T22:38:49.405 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:49.679 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:50.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 -- 172.21.15.77:0/4020739338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0100a80 msgr2=0x7f8ef0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/4020739338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0100a80 0x7f8ef0100ee0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009930 tx=0x7f8ee402a420 comp rx=0 tx=0).stop 2024-09-15T22:38:50.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 -- 172.21.15.77:0/4020739338 shutdown_connections 2024-09-15T22:38:50.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/4020739338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 0x7f8ef010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/4020739338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0100a80 0x7f8ef0100ee0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/4020739338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0106a80 0x7f8ef0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 -- 172.21.15.77:0/4020739338 >> 172.21.15.77:0/4020739338 conn(0x7f8ef00fc7a0 msgr2=0x7f8ef00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 -- 172.21.15.77:0/4020739338 shutdown_connections 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.199+0000 7f8ef53f2640 1 -- 172.21.15.77:0/4020739338 wait complete. 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 Processor -- start 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 -- start start 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0100a80 0x7f8ef00fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 0x7f8ef00fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0110e80 con 0x7f8ef0106a80 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0110d00 con 0x7f8ef0101420 2024-09-15T22:38:50.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.200+0000 7f8ef53f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0111000 con 0x7f8ef0100a80 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39070/0 (socket says 172.21.15.77:39070) 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 -- 172.21.15.77:0/3570185887 learned_addr learned my addr 172.21.15.77:0/3570185887 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eeeffd640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0100a80 0x7f8ef00fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eee7fc640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 0x7f8ef00fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 -- 172.21.15.77:0/3570185887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0100a80 msgr2=0x7f8ef00fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0100a80 0x7f8ef00fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 -- 172.21.15.77:0/3570185887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 msgr2=0x7f8ef00fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 0x7f8ef00fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 -- 172.21.15.77:0/3570185887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a6980 con 0x7f8ef0106a80 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8eef7fe640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800cad0 tx=0x7f8ee800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800ae20 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee8005020 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a6c70 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800f6f0 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.201+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a7130 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.202+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef0100ee0 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.203+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8ee80051d0 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.204+0000 7f8ed7fff640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 0x7f8ed80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.204+0000 7f8eeeffd640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 0x7f8ed80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.205+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f8ee80d0170 con 0x7f8ef0106a80 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.205+0000 7f8eeeffd640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 0x7f8ed80a5870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00058c0 tx=0x7f8ee000a2d0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.209+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ee8014030 con 0x7f8ef0106a80 2024-09-15T22:38:50.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.413+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ef010a000 con 0x7f8ef0106a80 2024-09-15T22:38:50.415 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:50 smithi077 ceph-mon[30980]: pgmap v835: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:50.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.414+0000 7f8ed7fff640 1 -- 172.21.15.77:0/3570185887 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11993 (secure 0 0 0) 0x7f8ee809c7c0 con 0x7f8ef0106a80 2024-09-15T22:38:50.416 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:50.416 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":42,"btime":"2024-09-15T22:38:07:243258+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:38:06.428603+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:38:50.416 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 42 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 msgr2=0x7f8ed80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 0x7f8ed80a5870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00058c0 tx=0x7f8ee000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 msgr2=0x7f8ef00fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800cad0 tx=0x7f8ee800cf90 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 shutdown_connections 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8ed80a33b0 0x7f8ed80a5870 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ef0106a80 0x7f8ef00fd8f0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0101420 0x7f8ef00fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 --2- 172.21.15.77:0/3570185887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef0100a80 0x7f8ef00fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.416+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 >> 172.21.15.77:0/3570185887 conn(0x7f8ef00fc7a0 msgr2=0x7f8ef006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.417+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 shutdown_connections 2024-09-15T22:38:50.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:50.417+0000 7f8ef53f2640 1 -- 172.21.15.77:0/3570185887 wait complete. 2024-09-15T22:38:50.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:50 smithi196 ceph-mon[35109]: pgmap v835: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:50.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:50 smithi117 ceph-mon[35021]: pgmap v835: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-15T22:38:51.086 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:51.333 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:51.597 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:51 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3570185887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:51.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:51 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3570185887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:51.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:51 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3570185887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 -- 172.21.15.77:0/2899881762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 msgr2=0x7f47cc114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 --2- 172.21.15.77:0/2899881762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc114f80 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009900 tx=0x7f47c402a480 comp rx=0 tx=0).stop 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 -- 172.21.15.77:0/2899881762 shutdown_connections 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 --2- 172.21.15.77:0/2899881762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc114f80 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 --2- 172.21.15.77:0/2899881762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 0x7f47cc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 --2- 172.21.15.77:0/2899881762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 0x7f47cc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 -- 172.21.15.77:0/2899881762 >> 172.21.15.77:0/2899881762 conn(0x7f47cc0fd8c0 msgr2=0x7f47cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 -- 172.21.15.77:0/2899881762 shutdown_connections 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.835+0000 7f47d2e92640 1 -- 172.21.15.77:0/2899881762 wait complete. 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 Processor -- start 2024-09-15T22:38:51.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 -- start start 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 0x7f47cc1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 0x7f47cc1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc118570 con 0x7f47cc112b90 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1183f0 con 0x7f47cc074590 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d2e92640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1186f0 con 0x7f47cc074eb0 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39098/0 (socket says 172.21.15.77:39098) 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 -- 172.21.15.77:0/208604688 learned_addr learned my addr 172.21.15.77:0/208604688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d0c2e640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 0x7f47cc1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47c3fff640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 0x7f47cc1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:51.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 -- 172.21.15.77:0/208604688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 msgr2=0x7f47cc1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 0x7f47cc1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 -- 172.21.15.77:0/208604688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 msgr2=0x7f47cc1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 0x7f47cc1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.836+0000 7f47d142f640 1 -- 172.21.15.77:0/208604688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19efc0 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47d142f640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005ab0 tx=0x7f47c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c403f030 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4003ce0 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19f250 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4035070 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.837+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1ab5a0 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.838+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc074970 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.839+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f47c4042030 con 0x7f47cc112b90 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.840+0000 7f47c1ffb640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 0x7f47b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:51.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.840+0000 7f47d0c2e640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 0x7f47b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.840+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f47c40f07f0 con 0x7f47cc112b90 2024-09-15T22:38:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.840+0000 7f47d0c2e640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 0x7f47b40a5870 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47cc19f9d0 tx=0x7f47c8009680 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:51.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:51.843+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47c40bce40 con 0x7f47cc112b90 2024-09-15T22:38:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.046+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47cc1ab800 con 0x7f47cc112b90 2024-09-15T22:38:52.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.047+0000 7f47c1ffb640 1 -- 172.21.15.77:0/208604688 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11993 (secure 0 0 0) 0x7f47c40bc740 con 0x7f47cc112b90 2024-09-15T22:38:52.049 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:52.049 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":42,"btime":"2024-09-15T22:38:07:243258+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1898309607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1898309607},{"type":"v1","addr":"172.21.15.77:6835","nonce":1898309607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/1847869314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":1847869314},{"type":"v1","addr":"172.21.15.77:6837","nonce":1847869314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:09:55.819962+0000","modified":"2024-09-15T22:38:06.428603+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24458,"mds_1":24440,"mds_2":24439,"mds_3":24424,"mds_4":14604},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14592":{"gid":14592,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/2629133747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":2629133747},{"type":"v1","addr":"172.21.15.77:6839","nonce":2629133747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14604":{"gid":14604,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.77:6841/3032989520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3032989520},{"type":"v1","addr":"172.21.15.77:6841","nonce":3032989520}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6833/3132990127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3132990127},{"type":"v1","addr":"172.21.15.196:6833","nonce":3132990127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6833/2823149773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2823149773},{"type":"v1","addr":"172.21.15.117:6833","nonce":2823149773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/889715770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":889715770},{"type":"v1","addr":"172.21.15.196:6835","nonce":889715770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/3112549785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3112549785},{"type":"v1","addr":"172.21.15.117:6835","nonce":3112549785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.196:6837/3513024753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3513024753},{"type":"v1","addr":"172.21.15.196:6837","nonce":3513024753}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6839/1624290607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1624290607},{"type":"v1","addr":"172.21.15.196:6839","nonce":1624290607}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.117:6837/3263285576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3263285576},{"type":"v1","addr":"172.21.15.117:6837","nonce":3263285576}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6839/1252344498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1252344498},{"type":"v1","addr":"172.21.15.117:6839","nonce":1252344498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24458,"qdb_cluster":[24440,24439,24424,24458,14604]},"id":1}]} 2024-09-15T22:38:52.049 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 42 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 msgr2=0x7f47b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 0x7f47b40a5870 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47cc19f9d0 tx=0x7f47c8009680 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 msgr2=0x7f47cc19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005ab0 tx=0x7f47c40042f0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 shutdown_connections 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f47b40a33b0 0x7f47b40a5870 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47cc112b90 0x7f47cc19e890 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47cc074eb0 0x7f47cc1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 --2- 172.21.15.77:0/208604688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47cc074590 0x7f47cc1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 >> 172.21.15.77:0/208604688 conn(0x7f47cc0fd8c0 msgr2=0x7f47cc0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 shutdown_connections 2024-09-15T22:38:52.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:52.049+0000 7f47d2e92640 1 -- 172.21.15.77:0/208604688 wait complete. 2024-09-15T22:38:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:52 smithi196 ceph-mon[35109]: pgmap v836: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T22:38:52.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:52 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/208604688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:52 smithi117 ceph-mon[35021]: pgmap v836: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T22:38:52.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:52 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/208604688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:52.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:52 smithi077 ceph-mon[30980]: pgmap v836: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-15T22:38:52.738 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:52 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/208604688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:52.765 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:38:53.011 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 -- 172.21.15.77:0/621893962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c106f20 msgr2=0x7f5a4c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 --2- 172.21.15.77:0/621893962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c106f20 0x7f5a4c10eaa0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ca0 tx=0x7f5a4402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 -- 172.21.15.77:0/621893962 shutdown_connections 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 --2- 172.21.15.77:0/621893962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c106f20 0x7f5a4c10eaa0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 --2- 172.21.15.77:0/621893962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 --2- 172.21.15.77:0/621893962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 -- 172.21.15.77:0/621893962 >> 172.21.15.77:0/621893962 conn(0x7f5a4c0fd8b0 msgr2=0x7f5a4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:53.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 -- 172.21.15.77:0/621893962 shutdown_connections 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.636+0000 7f5a51f26640 1 -- 172.21.15.77:0/621893962 wait complete. 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 Processor -- start 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 -- start start 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c106f20 0x7f5a4c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c111fa0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c111e20 con 0x7f5a4c106f20 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a51f26640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c112120 con 0x7f5a4c101df0 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a4affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a4b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a4affd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39116/0 (socket says 172.21.15.77:39116) 2024-09-15T22:38:53.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.637+0000 7f5a4affd640 1 -- 172.21.15.77:0/1659039421 learned_addr learned my addr 172.21.15.77:0/1659039421 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4bfff640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c106f20 0x7f5a4c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 -- 172.21.15.77:0/1659039421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 msgr2=0x7f5a4c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 -- 172.21.15.77:0/1659039421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c106f20 msgr2=0x7f5a4c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c106f20 0x7f5a4c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 -- 172.21.15.77:0/1659039421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c19d3b0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4b7fe640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a4affd640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b4a0 tx=0x7f5a4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40014070 con 0x7f5a4c1027a0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c19d6a0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a40010af0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40017d10 con 0x7f5a4c1027a0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.638+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a4f80 con 0x7f5a4c1027a0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.639+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c1021d0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.640+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5a40017070 con 0x7f5a4c1027a0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.641+0000 7f5a48ff9640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 0x7f5a340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.641+0000 7f5a4b7fe640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 0x7f5a340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.641+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f5a400cf2c0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.641+0000 7f5a4b7fe640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 0x7f5a340a5870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00bfd0 tx=0x7f5a3c00b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:53.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.645+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a4009b3d0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.814+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a4c1a52c0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.816+0000 7f5a48ff9640 1 -- 172.21.15.77:0/1659039421 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+25340 (secure 0 0 0) 0x7f5a4009f6d0 con 0x7f5a4c1027a0 2024-09-15T22:38:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-09-15T22:38:53.818 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":131,"fsid":"0ddb746c-73ae-11ef-bceb-c7b262605968","created":"2024-09-15T22:02:08.800745+0000","modified":"2024-09-15T22:38:14.116978+0000","last_up_change":"2024-09-15T22:09:03.651565+0000","last_in_change":"2024-09-15T22:08:40.379432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:05:16.403426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:09:51.900322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:09:54.068598+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:10:00.447354+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"286650d4-5c20-4659-a7ef-db7c243e0aba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6803","nonce":4041675784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6805","nonce":4041675784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6809","nonce":4041675784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":4041675784},{"type":"v1","addr":"172.21.15.77:6807","nonce":4041675784}]},"public_addr":"172.21.15.77:6803/4041675784","cluster_addr":"172.21.15.77:6805/4041675784","heartbeat_back_addr":"172.21.15.77:6809/4041675784","heartbeat_front_addr":"172.21.15.77:6807/4041675784","state":["exists","up"]},{"osd":1,"uuid":"5cddb21e-cf12-4806-9540-0b7c570af2aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6801","nonce":3227694760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6803","nonce":3227694760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6807","nonce":3227694760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3227694760},{"type":"v1","addr":"172.21.15.117:6805","nonce":3227694760}]},"public_addr":"172.21.15.117:6801/3227694760","cluster_addr":"172.21.15.117:6803/3227694760","heartbeat_back_addr":"172.21.15.117:6807/3227694760","heartbeat_front_addr":"172.21.15.117:6805/3227694760","state":["exists","up"]},{"osd":2,"uuid":"3e1d614a-34ca-45c9-b2c2-09d60c5c0821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6801","nonce":3745852928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6803","nonce":3745852928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6807","nonce":3745852928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3745852928},{"type":"v1","addr":"172.21.15.196:6805","nonce":3745852928}]},"public_addr":"172.21.15.196:6801/3745852928","cluster_addr":"172.21.15.196:6803/3745852928","heartbeat_back_addr":"172.21.15.196:6807/3745852928","heartbeat_front_addr":"172.21.15.196:6805/3745852928","state":["exists","up"]},{"osd":3,"uuid":"42be2980-1ab3-42e7-9256-eb759230d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6811","nonce":2407517217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6813","nonce":2407517217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6817","nonce":2407517217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2407517217},{"type":"v1","addr":"172.21.15.77:6815","nonce":2407517217}]},"public_addr":"172.21.15.77:6811/2407517217","cluster_addr":"172.21.15.77:6813/2407517217","heartbeat_back_addr":"172.21.15.77:6817/2407517217","heartbeat_front_addr":"172.21.15.77:6815/2407517217","state":["exists","up"]},{"osd":4,"uuid":"7db7bf42-1fed-4197-9bad-58e8a56f816a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6809","nonce":1062550497}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6811","nonce":1062550497}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6815","nonce":1062550497}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1062550497},{"type":"v1","addr":"172.21.15.117:6813","nonce":1062550497}]},"public_addr":"172.21.15.117:6809/1062550497","cluster_addr":"172.21.15.117:6811/1062550497","heartbeat_back_addr":"172.21.15.117:6815/1062550497","heartbeat_front_addr":"172.21.15.117:6813/1062550497","state":["exists","up"]},{"osd":5,"uuid":"d897eb1d-0f68-4ff4-832b-42deabfc6556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6809","nonce":2355584153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6811","nonce":2355584153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6815","nonce":2355584153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2355584153},{"type":"v1","addr":"172.21.15.196:6813","nonce":2355584153}]},"public_addr":"172.21.15.196:6809/2355584153","cluster_addr":"172.21.15.196:6811/2355584153","heartbeat_back_addr":"172.21.15.196:6815/2355584153","heartbeat_front_addr":"172.21.15.196:6813/2355584153","state":["exists","up"]},{"osd":6,"uuid":"916910cf-a42b-453d-be6b-c237a8305b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6819","nonce":3314328155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6821","nonce":3314328155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6825","nonce":3314328155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3314328155},{"type":"v1","addr":"172.21.15.77:6823","nonce":3314328155}]},"public_addr":"172.21.15.77:6819/3314328155","cluster_addr":"172.21.15.77:6821/3314328155","heartbeat_back_addr":"172.21.15.77:6825/3314328155","heartbeat_front_addr":"172.21.15.77:6823/3314328155","state":["exists","up"]},{"osd":7,"uuid":"5b4aa2ec-81c6-4365-82ec-6b470840358a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6817","nonce":159988175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6819","nonce":159988175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6823","nonce":159988175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":159988175},{"type":"v1","addr":"172.21.15.117:6821","nonce":159988175}]},"public_addr":"172.21.15.117:6817/159988175","cluster_addr":"172.21.15.117:6819/159988175","heartbeat_back_addr":"172.21.15.117:6823/159988175","heartbeat_front_addr":"172.21.15.117:6821/159988175","state":["exists","up"]},{"osd":8,"uuid":"3dc4187f-569d-4da4-ba2e-3c708dd5b99c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6817","nonce":3896356979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6819","nonce":3896356979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6823","nonce":3896356979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3896356979},{"type":"v1","addr":"172.21.15.196:6821","nonce":3896356979}]},"public_addr":"172.21.15.196:6817/3896356979","cluster_addr":"172.21.15.196:6819/3896356979","heartbeat_back_addr":"172.21.15.196:6823/3896356979","heartbeat_front_addr":"172.21.15.196:6821/3896356979","state":["exists","up"]},{"osd":9,"uuid":"5097d6bd-7d2a-45fc-958c-da3902b80859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6827","nonce":3515252544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6829","nonce":3515252544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6833","nonce":3515252544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":3515252544},{"type":"v1","addr":"172.21.15.77:6831","nonce":3515252544}]},"public_addr":"172.21.15.77:6827/3515252544","cluster_addr":"172.21.15.77:6829/3515252544","heartbeat_back_addr":"172.21.15.77:6833/3515252544","heartbeat_front_addr":"172.21.15.77:6831/3515252544","state":["exists","up"]},{"osd":10,"uuid":"67e4b9ba-333a-40a1-a03a-9a31da950d8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6825","nonce":2316398411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6827","nonce":2316398411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6831","nonce":2316398411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2316398411},{"type":"v1","addr":"172.21.15.117:6829","nonce":2316398411}]},"public_addr":"172.21.15.117:6825/2316398411","cluster_addr":"172.21.15.117:6827/2316398411","heartbeat_back_addr":"172.21.15.117:6831/2316398411","heartbeat_front_addr":"172.21.15.117:6829/2316398411","state":["exists","up"]},{"osd":11,"uuid":"35d9686a-188b-4bee-a88a-9c31fe9dc7b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6825","nonce":2953911305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6827","nonce":2953911305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6831","nonce":2953911305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2953911305},{"type":"v1","addr":"172.21.15.196:6829","nonce":2953911305}]},"public_addr":"172.21.15.196:6825/2953911305","cluster_addr":"172.21.15.196:6827/2953911305","heartbeat_back_addr":"172.21.15.196:6831/2953911305","heartbeat_front_addr":"172.21.15.196:6829/2953911305","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:24.706565+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:49.982330+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:14.255602+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:38.991270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:04.766654+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:29.786527+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:06:55.448356+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:19.952668+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:07:45.120279+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:10.753885+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:08:36.813198+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:09:01.966913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/999829703":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/3279424814":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/2559994639":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/573117065":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3493953936":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/3785009540":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6801/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:6801/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/3468864668":"2024-09-16T22:03:04.101979+0000","172.21.15.77:0/1397043946":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/1809193257":"2024-09-16T22:12:24.385790+0000","172.21.15.77:0/1437248041":"2024-09-16T22:12:24.385790+0000","172.21.15.77:6800/1757809354":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6800/3382222599":"2024-09-16T22:02:40.710885+0000","172.21.15.77:0/1261342424":"2024-09-16T22:03:04.101979+0000","172.21.15.77:6801/1809193257":"2024-09-16T22:12:24.385790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]},{"pool":4,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:53.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 msgr2=0x7f5a340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 0x7f5a340a5870 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00bfd0 tx=0x7f5a3c00b390 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 msgr2=0x7f5a4c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b4a0 tx=0x7f5a4000b960 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 shutdown_connections 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f5a340a33b0 0x7f5a340a5870 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a4c106f20 0x7f5a4c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a4c1027a0 0x7f5a4c198940 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 --2- 172.21.15.77:0/1659039421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a4c101df0 0x7f5a4c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 >> 172.21.15.77:0/1659039421 conn(0x7f5a4c0fd8b0 msgr2=0x7f5a4c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 shutdown_connections 2024-09-15T22:38:53.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:53.818+0000 7f5a51f26640 1 -- 172.21.15.77:0/1659039421 wait complete. 2024-09-15T22:38:54.535 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-15T22:38:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:54 smithi196 ceph-mon[35109]: pgmap v837: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-15T22:38:54.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:54 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/1659039421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:54 smithi117 ceph-mon[35021]: pgmap v837: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-15T22:38:54.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:54 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/1659039421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:54.779 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:54.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:54 smithi077 ceph-mon[30980]: pgmap v837: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-15T22:38:54.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:54 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/1659039421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- 172.21.15.77:0/3377116246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 msgr2=0x7f496810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3377116246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f496810eaa0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f4960009ca0 tx=0x7f496002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- 172.21.15.77:0/3377116246 shutdown_connections 2024-09-15T22:38:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3377116246 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f496810eaa0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.311 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3377116246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 0x7f49681067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3377116246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 0x7f49681008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- 172.21.15.77:0/3377116246 >> 172.21.15.77:0/3377116246 conn(0x7f49680fc5c0 msgr2=0x7f49680fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- 172.21.15.77:0/3377116246 shutdown_connections 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- 172.21.15.77:0/3377116246 wait complete. 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 Processor -- start 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.290+0000 7f496f4d9640 1 -- start start 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 0x7f4968101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 0x7f49681019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4968112020 con 0x7f4968106ed0 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111ea0 con 0x7f4968100500 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49681121a0 con 0x7f4968100eb0 2024-09-15T22:38:55.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39134/0 (socket says 172.21.15.77:39134) 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 -- 172.21.15.77:0/3897320459 learned_addr learned my addr 172.21.15.77:0/3897320459 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496d275640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 0x7f4968101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496ca74640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 0x7f49681019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 -- 172.21.15.77:0/3897320459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 msgr2=0x7f49681019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 0x7f49681019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 -- 172.21.15.77:0/3897320459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 msgr2=0x7f4968101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 0x7f4968101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 -- 172.21.15.77:0/3897320459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a6a20 con 0x7f4968106ed0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496da76640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4960031a60 tx=0x7f4960031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960003b70 con 0x7f4968106ed0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960003d10 con 0x7f4968106ed0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.291+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a6cb0 con 0x7f4968106ed0 2024-09-15T22:38:55.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.292+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960004770 con 0x7f4968106ed0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.292+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a71f0 con 0x7f4968106ed0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.292+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968100960 con 0x7f4968106ed0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.294+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f496002f790 con 0x7f4968106ed0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.294+0000 7f495a7fc640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 0x7f49540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.294+0000 7f496d275640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 0x7f49540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.294+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9438+0+0 (secure 0 0 0) 0x7f49600f1210 con 0x7f4968106ed0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.295+0000 7f496d275640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 0x7f49540a5870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4964009990 tx=0x7f4964009110 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:55.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.298+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49600bd970 con 0x7f4968106ed0 2024-09-15T22:38:55.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.503+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f496802d7c0 con 0x7f4968106ed0 2024-09-15T22:38:55.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.507+0000 7f495a7fc640 1 -- 172.21.15.77:0/3897320459 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f49600bd270 con 0x7f4968106ed0 2024-09-15T22:38:55.509 INFO:teuthology.orchestra.run.smithi077.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.509+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 msgr2=0x7f49540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.509+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 0x7f49540a5870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4964009990 tx=0x7f4964009110 comp rx=0 tx=0).stop 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.509+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 msgr2=0x7f4968101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.509+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f4960031a60 tx=0x7f4960031a90 comp rx=0 tx=0).stop 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 shutdown_connections 2024-09-15T22:38:55.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f49540a33b0 0x7f49540a5870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4968106ed0 0x7f4968101f00 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4968100eb0 0x7f49681019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 --2- 172.21.15.77:0/3897320459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4968100500 0x7f4968101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 >> 172.21.15.77:0/3897320459 conn(0x7f49680fc5c0 msgr2=0x7f49680fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 shutdown_connections 2024-09-15T22:38:55.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:55.510+0000 7f496f4d9640 1 -- 172.21.15.77:0/3897320459 wait complete. 2024-09-15T22:38:55.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a[30976]: 2024-09-15T22:38:55.504+0000 7f51212e2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:38:55.832 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.832 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.832 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-mds[71487]: -- 172.21.15.77:0/2940373726 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55700d57d860 con 0x55700d55b400 2024-09-15T22:38:55.832 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: starting mds.a at 2024-09-15T22:38:55.833 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.833 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.833 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-mds[71097]: -- 172.21.15.77:0/2454233792 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55940df53860 con 0x55940df31000 2024-09-15T22:38:55.833 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:38:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: starting mds.g at 2024-09-15T22:38:55.854 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.854 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.854 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-mds[66079]: -- 172.21.15.196:0/1017077334 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56353d491860 con 0x56353d46f400 2024-09-15T22:38:55.854 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: starting mds.i at 2024-09-15T22:38:55.854 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-mds[65173]: -- 172.21.15.196:0/1194879906 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd3c8f3860 con 0x55cd3c8d0800 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: starting mds.l at 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-mds[65778]: -- 172.21.15.196:0/1498637901 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587ddcb9860 con 0x5587ddc96800 2024-09-15T22:38:55.855 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: starting mds.c at 2024-09-15T22:38:55.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-mds[65475]: -- 172.21.15.196:0/1345651157 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55603e8e5860 con 0x55603e8c3000 2024-09-15T22:38:55.856 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:38:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: starting mds.f at 2024-09-15T22:38:55.862 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.862 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-mds[63783]: -- 172.21.15.117:0/3499244093 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558822165860 con 0x558822142800 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: starting mds.e at 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-mds[63468]: -- 172.21.15.117:0/1114273360 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647e4ff7860 con 0x5647e4fd5000 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: starting mds.k at 2024-09-15T22:38:55.863 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-mds[64402]: -- 172.21.15.117:0/2737376956 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56411015f860 con 0x56411013d400 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: starting mds.b at 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-mds[64086]: -- 172.21.15.117:0/3734884954 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdebf9d860 con 0x55cdebf7b000 2024-09-15T22:38:55.864 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:38:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: starting mds.h at 2024-09-15T22:38:56.287 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T22:38:56.540 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: pgmap v838: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: osdmap e132: 12 total, 12 up, 12 in 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-15T22:38:56.582 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:56 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: pgmap v838: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-15T22:38:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:38:56.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:38:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:38:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:38:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: osdmap e132: 12 total, 12 up, 12 in 2024-09-15T22:38:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-15T22:38:56.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:56 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: pgmap v838: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3897320459' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: osdmap e132: 12 total, 12 up, 12 in 2024-09-15T22:38:56.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-15T22:38:56.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:56 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:57.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 -- 172.21.15.77:0/1149159054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0101420 msgr2=0x7f82c010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 --2- 172.21.15.77:0/1149159054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0101420 0x7f82c010d950 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009cd0 tx=0x7f82b802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 -- 172.21.15.77:0/1149159054 shutdown_connections 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 --2- 172.21.15.77:0/1149159054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0101420 0x7f82c010d950 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 --2- 172.21.15.77:0/1149159054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0100a80 0x7f82c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 --2- 172.21.15.77:0/1149159054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0106a80 0x7f82c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 -- 172.21.15.77:0/1149159054 >> 172.21.15.77:0/1149159054 conn(0x7f82c00fc7a0 msgr2=0x7f82c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 -- 172.21.15.77:0/1149159054 shutdown_connections 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.051+0000 7f82c73a8640 1 -- 172.21.15.77:0/1149159054 wait complete. 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 Processor -- start 2024-09-15T22:38:57.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 -- start start 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 0x7f82c0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0106a80 0x7f82c019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0110e80 con 0x7f82c0100a80 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0110d00 con 0x7f82c0106a80 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c73a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0111000 con 0x7f82c0101420 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39186/0 (socket says 172.21.15.77:39186) 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 -- 172.21.15.77:0/3591741733 learned_addr learned my addr 172.21.15.77:0/3591741733 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5945640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0106a80 0x7f82c019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:57.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c4943640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 0x7f82c0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 -- 172.21.15.77:0/3591741733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 msgr2=0x7f82c0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 0x7f82c0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 -- 172.21.15.77:0/3591741733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0106a80 msgr2=0x7f82c019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0106a80 0x7f82c019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c5144640 1 -- 172.21.15.77:0/3591741733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c019d2e0 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.052+0000 7f82c4943640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 0x7f82c0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82c5144640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00d770 tx=0x7f82bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc017070 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc005500 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c019d5d0 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004e00 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.053+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c01a4eb0 con 0x7f82c0100a80 2024-09-15T22:38:57.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.054+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c0100ee0 con 0x7f82c0100a80 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.055+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f82bc004fa0 con 0x7f82c0100a80 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.056+0000 7f82b67fc640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 0x7f82ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.056+0000 7f82c4943640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 0x7f82ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.056+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9780+0+0 (secure 0 0 0) 0x7f82bc0cfce0 con 0x7f82c0100a80 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.056+0000 7f82c4943640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 0x7f82ac0a5870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f82c0105070 tx=0x7f82b0005d90 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:57.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.060+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82bc09c1d0 con 0x7f82c0100a80 2024-09-15T22:38:57.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.265+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f82c010a000 con 0x7f82c0100a80 2024-09-15T22:38:57.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.514+0000 7f82b67fc640 1 -- 172.21.15.77:0/3591741733 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f82bc09bad0 con 0x7f82c0100a80 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 msgr2=0x7f82ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 0x7f82ac0a5870 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f82c0105070 tx=0x7f82b0005d90 comp rx=0 tx=0).stop 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 msgr2=0x7f82c0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00d770 tx=0x7f82bc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 shutdown_connections 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f82ac0a33b0 0x7f82ac0a5870 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c0106a80 0x7f82c019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82c0101420 0x7f82c0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 --2- 172.21.15.77:0/3591741733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82c0100a80 0x7f82c0198330 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:57.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 >> 172.21.15.77:0/3591741733 conn(0x7f82c00fc7a0 msgr2=0x7f82c006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:57.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 shutdown_connections 2024-09-15T22:38:57.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:57.517+0000 7f82c73a8640 1 -- 172.21.15.77:0/3591741733 wait complete. 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6838/2355423856,v1:172.21.15.77:6840/2355423856] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.77:6839/1267111867,v1:172.21.15.77:6841/1267111867] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6832/1084825806,v1:172.21.15.196:6834/1084825806] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6833/1579357616,v1:172.21.15.196:6835/1579357616] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6836/3343746107,v1:172.21.15.196:6838/3343746107] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.196:6837/162910069,v1:172.21.15.196:6839/162910069] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6833/851595398,v1:172.21.15.117:6834/851595398] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6832/1113259909,v1:172.21.15.117:6835/1113259909] up:boot 2024-09-15T22:38:57.521 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6836/3906532968,v1:172.21.15.117:6838/3906532968] up:boot 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: mds.? [v2:172.21.15.117:6837/1470879432,v1:172.21.15.117:6839/1470879432] up:boot 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:57.522 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:57 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:38:57.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6838/2355423856,v1:172.21.15.77:6840/2355423856] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.77:6839/1267111867,v1:172.21.15.77:6841/1267111867] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6832/1084825806,v1:172.21.15.196:6834/1084825806] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6833/1579357616,v1:172.21.15.196:6835/1579357616] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6836/3343746107,v1:172.21.15.196:6838/3343746107] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.196:6837/162910069,v1:172.21.15.196:6839/162910069] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6833/851595398,v1:172.21.15.117:6834/851595398] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6832/1113259909,v1:172.21.15.117:6835/1113259909] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6836/3906532968,v1:172.21.15.117:6838/3906532968] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: mds.? [v2:172.21.15.117:6837/1470879432,v1:172.21.15.117:6839/1470879432] up:boot 2024-09-15T22:38:57.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:57.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:57 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:38:57.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6838/2355423856,v1:172.21.15.77:6840/2355423856] up:boot 2024-09-15T22:38:57.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.77:6839/1267111867,v1:172.21.15.77:6841/1267111867] up:boot 2024-09-15T22:38:57.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6832/1084825806,v1:172.21.15.196:6834/1084825806] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6833/1579357616,v1:172.21.15.196:6835/1579357616] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6836/3343746107,v1:172.21.15.196:6838/3343746107] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.196:6837/162910069,v1:172.21.15.196:6839/162910069] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6833/851595398,v1:172.21.15.117:6834/851595398] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6832/1113259909,v1:172.21.15.117:6835/1113259909] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6836/3906532968,v1:172.21.15.117:6838/3906532968] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: mds.? [v2:172.21.15.117:6837/1470879432,v1:172.21.15.117:6839/1470879432] up:boot 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:57.863 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='mgr.15309 172.21.15.77:0/884276337' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:57.864 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:57 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:38:58.240 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T22:38:58.490 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:58.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: pgmap v840: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:38:58.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:38:58.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:38:58.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: Cluster is now healthy 2024-09-15T22:38:58.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:38:58.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:58 smithi077 ceph-mon[30980]: fsmap 12 up:standby 2024-09-15T22:38:58.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: pgmap v840: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:38:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:38:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:38:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: Cluster is now healthy 2024-09-15T22:38:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:38:58.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:58 smithi196 ceph-mon[35109]: fsmap 12 up:standby 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: pgmap v840: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: Cluster is now healthy 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/3591741733' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:38:58.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:58 smithi117 ceph-mon[35021]: fsmap 12 up:standby 2024-09-15T22:38:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- 172.21.15.77:0/4062991594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 msgr2=0x7f8c201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 --2- 172.21.15.77:0/4062991594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c201029b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009900 tx=0x7f8c1402a480 comp rx=0 tx=0).stop 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- 172.21.15.77:0/4062991594 shutdown_connections 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 --2- 172.21.15.77:0/4062991594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 --2- 172.21.15.77:0/4062991594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c201029b0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 --2- 172.21.15.77:0/4062991594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c20100e20 0x7f8c20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- 172.21.15.77:0/4062991594 >> 172.21.15.77:0/4062991594 conn(0x7f8c200fc7d0 msgr2=0x7f8c200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- 172.21.15.77:0/4062991594 shutdown_connections 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- 172.21.15.77:0/4062991594 wait complete. 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 Processor -- start 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.971+0000 7f8c27503640 1 -- start start 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c20100e20 0x7f8c201984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20110f50 con 0x7f8c20102570 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20110dd0 con 0x7f8c20102ef0 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c201110d0 con 0x7f8c20100e20 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39204/0 (socket says 172.21.15.77:39204) 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 -- 172.21.15.77:0/2991053374 learned_addr learned my addr 172.21.15.77:0/2991053374 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c25aa0640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 -- 172.21.15.77:0/2991053374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c20100e20 msgr2=0x7f8c201984c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c20100e20 0x7f8c201984c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 -- 172.21.15.77:0/2991053374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 msgr2=0x7f8c2019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:58.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 -- 172.21.15.77:0/2991053374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2019d470 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c25aa0640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c24a9e640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006010 tx=0x7f8c14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1403f070 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14031c30 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2019d700 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14035030 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.972+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c201a5040 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.973+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c201021b0 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.974+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8c14042070 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.975+0000 7f8c1e7fc640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 0x7f8c080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.975+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9780+0+0 (secure 0 0 0) 0x7f8c140f0f40 con 0x7f8c20102570 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.975+0000 7f8c2529f640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 0x7f8c080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.976+0000 7f8c2529f640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 0x7f8c080a5870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8c10007c10 tx=0x7f8c100075f0 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:58.978+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c14031de0 con 0x7f8c20102570 2024-09-15T22:38:59.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.146+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c2010a0a0 con 0x7f8c20102570 2024-09-15T22:38:59.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.520+0000 7f8c1e7fc640 1 -- 172.21.15.77:0/2991053374 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7f8c140bd430 con 0x7f8c20102570 2024-09-15T22:38:59.523 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' removed 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 msgr2=0x7f8c080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 0x7f8c080a5870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8c10007c10 tx=0x7f8c100075f0 comp rx=0 tx=0).stop 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 msgr2=0x7f8c20198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006010 tx=0x7f8c14003c10 comp rx=0 tx=0).stop 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 shutdown_connections 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f8c080a33b0 0x7f8c080a5870 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c20102ef0 0x7f8c2019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c20102570 0x7f8c20198a00 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:59.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.523+0000 7f8c27503640 1 --2- 172.21.15.77:0/2991053374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c20100e20 0x7f8c201984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:59.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.524+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 >> 172.21.15.77:0/2991053374 conn(0x7f8c200fc7d0 msgr2=0x7f8c2006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:59.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.524+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 shutdown_connections 2024-09-15T22:38:59.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:38:59.524+0000 7f8c27503640 1 -- 172.21.15.77:0/2991053374 wait complete. 2024-09-15T22:38:59.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:38:59 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:38:59.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:38:59 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:38:59.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:38:59 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:00.237 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:39:00.411+0000 7fd9845e9640 -1 client.15333: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:39:00.413+0000 7fd9845e9640 -1 client.15333: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Exception in thread Thread-31: 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-15T22:39:00.524 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.run() 2024-09-15T22:39:00.525 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-15T22:39:00.525 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.function(*self.args, **self.kwargs) 2024-09-15T22:39:00.525 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-15T22:39:00.525 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: self.prune_snapshots(sched) 2024-09-15T22:39:00.525 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-15T22:39:00.550 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:00.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-mon[30980]: pgmap v841: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:39:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:00.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:00 smithi077 ceph-mon[30980]: osdmap e133: 12 total, 12 up, 12 in 2024-09-15T22:39:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:00 smithi196 ceph-mon[35109]: pgmap v841: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:39:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:00 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:00.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:00 smithi196 ceph-mon[35109]: osdmap e133: 12 total, 12 up, 12 in 2024-09-15T22:39:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:00 smithi117 ceph-mon[35021]: pgmap v841: 129 pgs: 129 active+clean; 1.9 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-15T22:39:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:00 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2991053374' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:00.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:00 smithi117 ceph-mon[35021]: osdmap e133: 12 total, 12 up, 12 in 2024-09-15T22:39:01.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.075+0000 7f6ad0f78640 1 -- 172.21.15.77:0/963442662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 msgr2=0x7f6acc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:01.093 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.075+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/963442662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc1029b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009900 tx=0x7f6ac002a480 comp rx=0 tx=0).stop 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 -- 172.21.15.77:0/963442662 shutdown_connections 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/963442662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/963442662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc1029b0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/963442662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 0x7f6acc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 -- 172.21.15.77:0/963442662 >> 172.21.15.77:0/963442662 conn(0x7f6acc0fc7d0 msgr2=0x7f6acc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 -- 172.21.15.77:0/963442662 shutdown_connections 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 -- 172.21.15.77:0/963442662 wait complete. 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.076+0000 7f6ad0f78640 1 Processor -- start 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 -- start start 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 0x7f6acc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:01.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc1120a0 con 0x7f6acc102570 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc111f20 con 0x7f6acc102ef0 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ad0f78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc112220 con 0x7f6acc100e20 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53628/0 (socket says 172.21.15.77:53628) 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 -- 172.21.15.77:0/2386056050 learned_addr learned my addr 172.21.15.77:0/2386056050 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6aca59c640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 0x7f6acc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6acad9d640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 -- 172.21.15.77:0/2386056050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 msgr2=0x7f6acc198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 0x7f6acc198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 -- 172.21.15.77:0/2386056050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 msgr2=0x7f6acc19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 -- 172.21.15.77:0/2386056050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6acc19d510 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6acad9d640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6ac9d9b640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0006010 tx=0x7f6ac0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.077+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac003f070 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.078+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ac0031c30 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.078+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6acc19d7a0 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.078+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0035030 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.078+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6acc1a50e0 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.078+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6acc1021b0 con 0x7f6acc102570 2024-09-15T22:39:01.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.080+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6ac0042070 con 0x7f6acc102570 2024-09-15T22:39:01.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.080+0000 7f6abb7fe640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 0x7f6ab40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:01.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.080+0000 7f6aca59c640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 0x7f6ab40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:01.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.080+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9284+0+0 (secure 0 0 0) 0x7f6ac00f0880 con 0x7f6acc102570 2024-09-15T22:39:01.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.081+0000 7f6aca59c640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 0x7f6ab40a5870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00bbe0 tx=0x7f6abc00b390 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:01.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.085+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ac0031de0 con 0x7f6acc102570 2024-09-15T22:39:01.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.254+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6acc109090 con 0x7f6acc102570 2024-09-15T22:39:01.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.527+0000 7f6abb7fe640 1 -- 172.21.15.77:0/2386056050 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7f6ac00bcf60 con 0x7f6acc102570 2024-09-15T22:39:01.529 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' removed 2024-09-15T22:39:01.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 msgr2=0x7f6ab40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:01.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 0x7f6ab40a5870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00bbe0 tx=0x7f6abc00b390 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 msgr2=0x7f6acc198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0006010 tx=0x7f6ac0003c10 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 shutdown_connections 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f6ab40a33b0 0x7f6ab40a5870 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6acc102ef0 0x7f6acc19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6acc102570 0x7f6acc198aa0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 --2- 172.21.15.77:0/2386056050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6acc100e20 0x7f6acc198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 >> 172.21.15.77:0/2386056050 conn(0x7f6acc0fc7d0 msgr2=0x7f6acc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 shutdown_connections 2024-09-15T22:39:01.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:01.531+0000 7f6ad0f78640 1 -- 172.21.15.77:0/2386056050 wait complete. 2024-09-15T22:39:01.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:01 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:01.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:01 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:01.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:01 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:02.576 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-15T22:39:02.826 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:02.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:02 smithi077 ceph-mon[30980]: pgmap v843: 97 pgs: 97 active+clean; 307 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:02.831 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:02 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:02.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:02 smithi077 ceph-mon[30980]: osdmap e134: 12 total, 12 up, 12 in 2024-09-15T22:39:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:02 smithi196 ceph-mon[35109]: pgmap v843: 97 pgs: 97 active+clean; 307 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:02 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:02.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:02 smithi196 ceph-mon[35109]: osdmap e134: 12 total, 12 up, 12 in 2024-09-15T22:39:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:02 smithi117 ceph-mon[35021]: pgmap v843: 97 pgs: 97 active+clean; 307 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:02 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2386056050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:02.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:02 smithi117 ceph-mon[35021]: osdmap e134: 12 total, 12 up, 12 in 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 -- 172.21.15.77:0/154185097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534100a80 msgr2=0x7f8534100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 --2- 172.21.15.77:0/154185097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534100a80 0x7f8534100ee0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8520009930 tx=0x7f852002a420 comp rx=0 tx=0).stop 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 -- 172.21.15.77:0/154185097 shutdown_connections 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 --2- 172.21.15.77:0/154185097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534101420 0x7f853410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 --2- 172.21.15.77:0/154185097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534100a80 0x7f8534100ee0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 --2- 172.21.15.77:0/154185097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 0x7f8534106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 -- 172.21.15.77:0/154185097 >> 172.21.15.77:0/154185097 conn(0x7f85340fc7a0 msgr2=0x7f85340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 -- 172.21.15.77:0/154185097 shutdown_connections 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.351+0000 7f853ac1e640 1 -- 172.21.15.77:0/154185097 wait complete. 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 Processor -- start 2024-09-15T22:39:03.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 -- start start 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 0x7f8534198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 0x7f853419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8534110e80 con 0x7f8534101420 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8534110d00 con 0x7f8534100a80 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f853ac1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8534111000 con 0x7f8534106a80 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f8533fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f8533fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53656/0 (socket says 172.21.15.77:53656) 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f8533fff640 1 -- 172.21.15.77:0/2878281429 learned_addr learned my addr 172.21.15.77:0/2878281429 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.352+0000 7f85389ba640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 0x7f8534198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f85391bb640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 0x7f853419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 -- 172.21.15.77:0/2878281429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 msgr2=0x7f853419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 0x7f853419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 -- 172.21.15.77:0/2878281429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 msgr2=0x7f8534198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 0x7f8534198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 -- 172.21.15.77:0/2878281429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853419d420 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f85389ba640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 0x7f8534198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8533fff640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f8520009230 tx=0x7f8520004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852003f030 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8520003ad0 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520035070 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853419d6b0 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.353+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a4ff0 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.354+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85341021c0 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.355+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8520042030 con 0x7f8534101420 2024-09-15T22:39:03.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.356+0000 7f8531ffb640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 0x7f85240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:03.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.356+0000 7f85389ba640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 0x7f85240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:03.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.356+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8754+0+0 (secure 0 0 0) 0x7f85200f03b0 con 0x7f8534101420 2024-09-15T22:39:03.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.357+0000 7f85389ba640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 0x7f85240a5870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8528005d70 tx=0x7f8528005d00 comp rx=0 tx=0).ready entity=mgr.15309 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:03.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.361+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85200bcca0 con 0x7f8534101420 2024-09-15T22:39:03.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:03.536+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f853410a000 con 0x7f8534101420 2024-09-15T22:39:04.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.535+0000 7f8531ffb640 1 -- 172.21.15.77:0/2878281429 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v135) ==== 178+0+0 (secure 0 0 0) 0x7f85200bc5a0 con 0x7f8534101420 2024-09-15T22:39:04.538 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data_ec' removed 2024-09-15T22:39:04.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 msgr2=0x7f85240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:04.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 0x7f85240a5870 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8528005d70 tx=0x7f8528005d00 comp rx=0 tx=0).stop 2024-09-15T22:39:04.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 msgr2=0x7f85341989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f8520009230 tx=0x7f8520004270 comp rx=0 tx=0).stop 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 shutdown_connections 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:6800/3746048513,v1:172.21.15.77:6801/3746048513] conn(0x7f85240a33b0 0x7f85240a5870 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8534106a80 0x7f853419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8534101420 0x7f85341989b0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 --2- 172.21.15.77:0/2878281429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8534100a80 0x7f8534198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 >> 172.21.15.77:0/2878281429 conn(0x7f85340fc7a0 msgr2=0x7f853406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 shutdown_connections 2024-09-15T22:39:04.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-09-15T22:39:04.540+0000 7f853ac1e640 1 -- 172.21.15.77:0/2878281429 wait complete. 2024-09-15T22:39:04.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:04 smithi077 ceph-mon[30980]: pgmap v845: 33 pgs: 33 active+clean; 307 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:04.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:04 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:04.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:04 smithi196 ceph-mon[35109]: pgmap v845: 33 pgs: 33 active+clean; 307 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:04.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:04 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:04 smithi117 ceph-mon[35021]: pgmap v845: 33 pgs: 33 active+clean; 307 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:39:04.862 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:04 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:39:05.303 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:39:05.330 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:39:05.364 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:39:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:05 smithi117 ceph-mon[35021]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:05.612 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:05 smithi117 ceph-mon[35021]: osdmap e135: 12 total, 12 up, 12 in 2024-09-15T22:39:05.719 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-15T22:39:05.719 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:39:05.719 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:05 smithi196 ceph-mon[35109]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:05.719 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:05 smithi196 ceph-mon[35109]: osdmap e135: 12 total, 12 up, 12 in 2024-09-15T22:39:05.744 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:39:05.760 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:39:05.783 INFO:tasks.cephadm:Stopping all daemons... 2024-09-15T22:39:05.784 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-15T22:39:05.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:39:05.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:05 smithi077 ceph-mon[30980]: from='client.? 172.21.15.77:0/2878281429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:39:05.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:05 smithi077 ceph-mon[30980]: osdmap e135: 12 total, 12 up, 12 in 2024-09-15T22:39:06.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:05 smithi077 systemd[1]: Stopping Ceph mon.a for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:06.387 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a[30976]: 2024-09-15T22:39:06.129+0000 7f5126aed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:06.387 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a[30976]: 2024-09-15T22:39:06.129+0000 7f5126aed640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-15T22:39:06.700 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:06 smithi077 podman[147418]: 2024-09-15 22:39:06.386293964 +0000 UTC m=+0.371906035 container died ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-15T22:39:07.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:06 smithi077 podman[147418]: 2024-09-15 22:39:06.798366677 +0000 UTC m=+0.783978779 container cleanup ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-15T22:39:07.068 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:06 smithi077 bash[147418]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:39:07.332 INFO:journalctl@ceph.mon.a.smithi077.stdout:Sep 15 22:39:07 smithi077 podman[147527]: 2024-09-15 22:39:07.066984932 +0000 UTC m=+0.682451718 container remove ac1952b1d15765c0ede9d01327dd4e41e37a9772811625ce2d055720a22db1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-15T22:39:10.177 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.a.service' 2024-09-15T22:39:10.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:10.287 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-15T22:39:10.287 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-15T22:39:10.287 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.b 2024-09-15T22:39:10.613 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:10 smithi117 systemd[1]: Stopping Ceph mon.b for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:11.102 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-b[35017]: 2024-09-15T22:39:10.676+0000 7f1ab5a33640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:11.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-b[35017]: 2024-09-15T22:39:10.676+0000 7f1ab5a33640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:39:11.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:10 smithi117 podman[77268]: 2024-09-15 22:39:10.789426659 +0000 UTC m=+0.242922386 container died b5703de2a2f14ffb3c412f20e5966c0bcc611cc274c135d81b345bbdb5871177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-b, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-15T22:39:11.354 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:11 smithi117 podman[77268]: 2024-09-15 22:39:11.351928189 +0000 UTC m=+0.805423917 container cleanup b5703de2a2f14ffb3c412f20e5966c0bcc611cc274c135d81b345bbdb5871177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-15T22:39:11.354 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 15 22:39:11 smithi117 bash[77268]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-b 2024-09-15T22:39:11.813 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.b.service' 2024-09-15T22:39:11.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:11.858 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-15T22:39:11.858 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-15T22:39:11.858 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.c 2024-09-15T22:39:12.245 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:11 smithi196 systemd[1]: Stopping Ceph mon.c for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:12.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c[35088]: 2024-09-15T22:39:12.243+0000 7fe388df7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c[35088]: 2024-09-15T22:39:12.243+0000 7fe388df7640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:39:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:12 smithi196 podman[77745]: 2024-09-15 22:39:12.346005165 +0000 UTC m=+0.242975754 container died 2b2a1bd3652be1b233e7cc9547b09a6d3609560e6829506046c79248f30d38ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:39:12.996 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:12 smithi196 podman[77745]: 2024-09-15 22:39:12.719256246 +0000 UTC m=+0.616226834 container cleanup 2b2a1bd3652be1b233e7cc9547b09a6d3609560e6829506046c79248f30d38ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-15T22:39:12.997 INFO:journalctl@ceph.mon.c.smithi196.stdout:Sep 15 22:39:12 smithi196 bash[77745]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mon-c 2024-09-15T22:39:13.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mon.c.service' 2024-09-15T22:39:13.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:13.181 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-15T22:39:13.181 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-15T22:39:13.181 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:39:13.501 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:13 smithi077 systemd[1]: Stopping Ceph mgr.x for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:13.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:13 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-15T22:39:13.499+0000 7fda21c8a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:13.832 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:13 smithi077 podman[150150]: 2024-09-15 22:39:13.530484461 +0000 UTC m=+0.147445152 container died fcb0d844aa0a6172571860670fb12670c10f4094997d2c6b04ad089cd96830a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD) 2024-09-15T22:39:14.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:13 smithi077 podman[150150]: 2024-09-15 22:39:13.933450037 +0000 UTC m=+0.550410730 container cleanup fcb0d844aa0a6172571860670fb12670c10f4094997d2c6b04ad089cd96830a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T22:39:14.332 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Sep 15 22:39:13 smithi077 bash[150150]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-x 2024-09-15T22:39:14.525 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-15T22:39:14.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:14.563 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-15T22:39:14.563 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-15T22:39:14.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.y 2024-09-15T22:39:14.862 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:39:14 smithi117 systemd[1]: Stopping Ceph mgr.y for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:15.208 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:39:14 smithi117 podman[77389]: 2024-09-15 22:39:14.903578499 +0000 UTC m=+0.151134116 container died 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:39:15.591 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:39:15 smithi117 podman[77389]: 2024-09-15 22:39:15.322806595 +0000 UTC m=+0.570362213 container cleanup 3d9a106936dc5d58b3bc33fec5f745f8baa60815668cf79a8718a2404b919b3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-15T22:39:15.592 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 15 22:39:15 smithi117 bash[77389]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-y 2024-09-15T22:39:15.740 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-15T22:39:15.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:15.775 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-15T22:39:15.775 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-15T22:39:15.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z 2024-09-15T22:39:16.103 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:15 smithi196 systemd[1]: Stopping Ceph mgr.z for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:16.452 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 podman[77870]: 2024-09-15 22:39:16.12065832 +0000 UTC m=+0.151468321 container stop 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:39:16.452 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 podman[77870]: 2024-09-15 22:39:16.146474775 +0000 UTC m=+0.177284776 container died 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:39:16.736 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 podman[77870]: 2024-09-15 22:39:16.54174805 +0000 UTC m=+0.572558050 container cleanup 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-15T22:39:16.736 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 bash[77870]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z 2024-09-15T22:39:16.964 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-15T22:39:16.987 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 podman[77883]: 2024-09-15 22:39:16.735062388 +0000 UTC m=+0.608250809 container remove 55933dc495ab41f4a40fbbb1f5b38dc1390bbe8154f34aae8809295ef2059c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:39:16.987 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-15T22:39:16.987 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-15T22:39:16.988 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 systemd[1]: Stopped Ceph mgr.z for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:39:16.988 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Sep 15 22:39:16 smithi196 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mgr.z.service: Consumed 12.125s CPU time. 2024-09-15T22:39:16.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:16.998 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-15T22:39:16.998 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-15T22:39:16.998 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.0 2024-09-15T22:39:17.316 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:17 smithi077 systemd[1]: Stopping Ceph osd.0 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:17.577 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:39:17.314+0000 7fdaa3e35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:17.578 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:39:17.314+0000 7fdaa3e35640 -1 osd.0 135 *** Got signal Terminated *** 2024-09-15T22:39:17.578 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0[44568]: 2024-09-15T22:39:17.314+0000 7fdaa3e35640 -1 osd.0 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:23.295 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:22 smithi077 podman[150272]: 2024-09-15 22:39:22.961585105 +0000 UTC m=+5.764750105 container died 31c7c9eab21e44314c60026ae4a9e26b66072b5f5cdd2e86000e1678eab0f5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-15T22:39:23.582 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:23 smithi077 podman[150272]: 2024-09-15 22:39:23.499155105 +0000 UTC m=+6.302320108 container cleanup 31c7c9eab21e44314c60026ae4a9e26b66072b5f5cdd2e86000e1678eab0f5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:39:23.582 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:23 smithi077 bash[150272]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0 2024-09-15T22:39:24.081 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:23 smithi077 podman[150299]: 2024-09-15 22:39:23.770635963 +0000 UTC m=+0.791655885 container remove 31c7c9eab21e44314c60026ae4a9e26b66072b5f5cdd2e86000e1678eab0f5ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:39:24.469 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.082352162 +0000 UTC m=+0.016364502 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:39:24.469 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.199277689 +0000 UTC m=+0.133290021 container create 4f0263a91b52854414b5c7d8f87a46e381e0b708b42a9592db9045a8fcfac50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-15T22:39:24.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.468901965 +0000 UTC m=+0.402914306 container init 4f0263a91b52854414b5c7d8f87a46e381e0b708b42a9592db9045a8fcfac50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:39:24.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.472275594 +0000 UTC m=+0.406287928 container start 4f0263a91b52854414b5c7d8f87a46e381e0b708b42a9592db9045a8fcfac50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731) 2024-09-15T22:39:24.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.5523525 +0000 UTC m=+0.486364832 container attach 4f0263a91b52854414b5c7d8f87a46e381e0b708b42a9592db9045a8fcfac50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T22:39:24.832 INFO:journalctl@ceph.osd.0.smithi077.stdout:Sep 15 22:39:24 smithi077 podman[150389]: 2024-09-15 22:39:24.655731449 +0000 UTC m=+0.589743781 container died 4f0263a91b52854414b5c7d8f87a46e381e0b708b42a9592db9045a8fcfac50c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-15T22:39:25.520 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.0.service' 2024-09-15T22:39:25.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:25.560 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-15T22:39:25.560 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-15T22:39:25.560 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.1 2024-09-15T22:39:25.862 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:25 smithi117 systemd[1]: Stopping Ceph osd.1 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:26.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:39:25.884+0000 7fc278308640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:26.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:39:25.884+0000 7fc278308640 -1 osd.1 135 *** Got signal Terminated *** 2024-09-15T22:39:26.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1[39486]: 2024-09-15T22:39:25.884+0000 7fc278308640 -1 osd.1 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:31.612 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:31 smithi117 podman[77520]: 2024-09-15 22:39:31.372697877 +0000 UTC m=+5.607798859 container died faa5da50531228b03298498413e31f74a7ae074e01ce4178992742b2d2de0160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-15T22:39:32.028 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:31 smithi117 podman[77520]: 2024-09-15 22:39:31.792484937 +0000 UTC m=+6.027585906 container cleanup faa5da50531228b03298498413e31f74a7ae074e01ce4178992742b2d2de0160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-15T22:39:32.028 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:31 smithi117 bash[77520]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1 2024-09-15T22:39:32.362 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77538]: 2024-09-15 22:39:32.027198154 +0000 UTC m=+0.657033932 container remove faa5da50531228b03298498413e31f74a7ae074e01ce4178992742b2d2de0160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-15T22:39:32.764 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.340471484 +0000 UTC m=+0.015548979 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:39:32.765 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.469554424 +0000 UTC m=+0.144631909 container create d53e21a2dc7248b665b7e3778cef99575f3a181fc11ec81f90c64309530e0485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T22:39:33.112 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.763969754 +0000 UTC m=+0.439047241 container init d53e21a2dc7248b665b7e3778cef99575f3a181fc11ec81f90c64309530e0485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-15T22:39:33.112 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.767402379 +0000 UTC m=+0.442479864 container start d53e21a2dc7248b665b7e3778cef99575f3a181fc11ec81f90c64309530e0485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:39:33.112 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.872393255 +0000 UTC m=+0.547470778 container attach d53e21a2dc7248b665b7e3778cef99575f3a181fc11ec81f90c64309530e0485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T22:39:33.113 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 15 22:39:32 smithi117 podman[77625]: 2024-09-15 22:39:32.9590237 +0000 UTC m=+0.634101192 container died d53e21a2dc7248b665b7e3778cef99575f3a181fc11ec81f90c64309530e0485 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:39:33.638 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.1.service' 2024-09-15T22:39:33.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:33.679 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-15T22:39:33.679 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-15T22:39:33.679 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.2 2024-09-15T22:39:33.996 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:33 smithi196 systemd[1]: Stopping Ceph osd.2 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:34.353 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:39:33.995+0000 7f1e9381c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:34.353 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:39:33.995+0000 7f1e9381c640 -1 osd.2 135 *** Got signal Terminated *** 2024-09-15T22:39:34.354 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2[39968]: 2024-09-15T22:39:33.995+0000 7f1e9381c640 -1 osd.2 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:39.791 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:39 smithi196 podman[78006]: 2024-09-15 22:39:39.487074169 +0000 UTC m=+5.608521140 container died ff7fb82ba6e8562a8ec34fdba7c32795150d3e08234f9aa2a34398634d6b07e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-15T22:39:40.103 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:39 smithi196 podman[78006]: 2024-09-15 22:39:39.887255197 +0000 UTC m=+6.008702157 container cleanup ff7fb82ba6e8562a8ec34fdba7c32795150d3e08234f9aa2a34398634d6b07e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:39:40.103 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:39 smithi196 bash[78006]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2 2024-09-15T22:39:40.449 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78025]: 2024-09-15 22:39:40.12627702 +0000 UTC m=+0.637830460 container remove ff7fb82ba6e8562a8ec34fdba7c32795150d3e08234f9aa2a34398634d6b07e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:39:40.726 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.350763997 +0000 UTC m=+0.015659797 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:39:40.727 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.481359131 +0000 UTC m=+0.146254929 container create 36df08f142b23671d71e59df2708545a700da8b928bd32cea5ce755bd52a6072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-15T22:39:40.727 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.725770072 +0000 UTC m=+0.390665881 container init 36df08f142b23671d71e59df2708545a700da8b928bd32cea5ce755bd52a6072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-15T22:39:41.104 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.729534317 +0000 UTC m=+0.394430117 container start 36df08f142b23671d71e59df2708545a700da8b928bd32cea5ce755bd52a6072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-15T22:39:41.104 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.834148273 +0000 UTC m=+0.499044082 container attach 36df08f142b23671d71e59df2708545a700da8b928bd32cea5ce755bd52a6072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-15T22:39:41.104 INFO:journalctl@ceph.osd.2.smithi196.stdout:Sep 15 22:39:40 smithi196 podman[78113]: 2024-09-15 22:39:40.9276435 +0000 UTC m=+0.592539310 container died 36df08f142b23671d71e59df2708545a700da8b928bd32cea5ce755bd52a6072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-15T22:39:41.607 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.2.service' 2024-09-15T22:39:41.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:41.660 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-15T22:39:41.660 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-15T22:39:41.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.3 2024-09-15T22:39:41.951 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:41 smithi077 systemd[1]: Stopping Ceph osd.3 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:42.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:41 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:41.949+0000 7f215690f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:42.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:41 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:41.949+0000 7f215690f640 -1 osd.3 135 *** Got signal Terminated *** 2024-09-15T22:39:42.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:41 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:41.949+0000 7f215690f640 -1 osd.3 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:44.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:43.713+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:44.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:44.043+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:44.582 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:44 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:44.140+0000 7f214ef1d640 -1 osd.3 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:20.565876+0000 front 2024-09-15T22:39:20.565912+0000 (oldest deadline 2024-09-15T22:39:44.066002+0000) 2024-09-15T22:39:45.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:44 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:44.756+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:45.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:45.137+0000 7f214ef1d640 -1 osd.3 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:20.565876+0000 front 2024-09-15T22:39:20.565912+0000 (oldest deadline 2024-09-15T22:39:44.066002+0000) 2024-09-15T22:39:45.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:45 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:45.042+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:45.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:45.029+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:45.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:44.990+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:46.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:45 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:45.798+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:46.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:46.013+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:46.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:45.987+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:46.582 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:46 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3[51168]: 2024-09-15T22:39:46.179+0000 7f214ef1d640 -1 osd.3 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:20.565876+0000 front 2024-09-15T22:39:20.565912+0000 (oldest deadline 2024-09-15T22:39:44.066002+0000) 2024-09-15T22:39:46.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:46 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:46.086+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:47.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:46 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:46.759+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:47.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:46.991+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:47.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:46.978+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:47.482 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:47 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:47.130+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:47.756 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:47 smithi077 podman[150493]: 2024-09-15 22:39:47.511210777 +0000 UTC m=+5.660648402 container died e3abf7208a3b5ffb7c7b624fd1c7514731c6c04242f23d7c16061be400819bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-15T22:39:48.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:47 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:47.807+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:48.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:48 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:48.105+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:48.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150493]: 2024-09-15 22:39:48.024738278 +0000 UTC m=+6.174175902 container cleanup e3abf7208a3b5ffb7c7b624fd1c7514731c6c04242f23d7c16061be400819bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:39:48.333 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 bash[150493]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3 2024-09-15T22:39:48.333 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150509]: 2024-09-15 22:39:48.268163032 +0000 UTC m=+0.747726445 container remove e3abf7208a3b5ffb7c7b624fd1c7514731c6c04242f23d7c16061be400819bf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-15T22:39:48.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:47.949+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:48.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:47.977+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:48.832 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150605]: 2024-09-15 22:39:48.483336537 +0000 UTC m=+0.015669075 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:39:48.832 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150605]: 2024-09-15 22:39:48.581579442 +0000 UTC m=+0.113911980 container create c339c8055b0b9c6d6909b324f459c850cd2bbedaa0fa2cd4ad4b190495f1dd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:39:48.832 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150605]: 2024-09-15 22:39:48.817900678 +0000 UTC m=+0.350233217 container init c339c8055b0b9c6d6909b324f459c850cd2bbedaa0fa2cd4ad4b190495f1dd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:39:48.832 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150605]: 2024-09-15 22:39:48.821507351 +0000 UTC m=+0.353839892 container start c339c8055b0b9c6d6909b324f459c850cd2bbedaa0fa2cd4ad4b190495f1dd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-15T22:39:49.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:48 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:48.845+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:49.331 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:49 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:49.150+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:49.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:48 smithi077 podman[150605]: 2024-09-15 22:39:48.926413243 +0000 UTC m=+0.458745797 container attach c339c8055b0b9c6d6909b324f459c850cd2bbedaa0fa2cd4ad4b190495f1dd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-15T22:39:49.332 INFO:journalctl@ceph.osd.3.smithi077.stdout:Sep 15 22:39:49 smithi077 podman[150605]: 2024-09-15 22:39:49.006612561 +0000 UTC m=+0.538945112 container died c339c8055b0b9c6d6909b324f459c850cd2bbedaa0fa2cd4ad4b190495f1dd8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T22:39:49.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:49.004+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:49.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:48.934+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:49.688 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.3.service' 2024-09-15T22:39:49.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:49.730 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-15T22:39:49.730 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-15T22:39:49.730 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.4 2024-09-15T22:39:50.014 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:49 smithi117 systemd[1]: Stopping Ceph osd.4 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:50.014 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:49 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:49.884+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:50.268 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:39:50.073+0000 7f32a314c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:50.268 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:39:50.073+0000 7f32a314c640 -1 osd.4 135 *** Got signal Terminated *** 2024-09-15T22:39:50.269 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4[45967]: 2024-09-15T22:39:50.073+0000 7f32a314c640 -1 osd.4 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:50.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:49.889+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:50.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:50.055+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:50.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:50 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:50.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:51.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:50 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:50.843+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:51.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:50.873+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:51.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:51.019+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:51.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:51 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:51.215+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:52.346 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:51.890+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:52.346 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:52 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:52.028+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:52.362 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:51 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:51.875+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:52.362 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:51 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:51.875+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:52.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:52 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:52.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:52.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:52 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:52.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:53.081 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:52 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:52.689+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:53.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:52 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:52.916+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:53.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:53 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:53.035+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:53.362 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:52 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:52.899+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:53.362 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:52 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:52.899+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:53.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:53 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:53.158+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:53.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:53 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:53.158+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:54.084 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:53 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:53.654+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:54.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:53 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:53.863+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:54.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:53 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:53.863+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:54.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:53 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:53.849+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:54.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:53 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:53.933+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:54.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:54 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:54.017+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:54.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:54.139+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:54.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:54.139+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:55.081 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:54.694+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:55.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:54 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:54.874+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:55.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:54 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:54.854+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:55.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:54 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:54.854+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:55.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:55.038+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:55.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:54 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:54.915+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:55.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:54 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:54.915+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:39:55.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:55.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:55.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:55.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:55.824 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:55 smithi117 podman[77728]: 2024-09-15 22:39:55.53336108 +0000 UTC m=+5.605707040 container died 6dad0b316c4ccbb3c3bd406cf9ff9ec2a603b06938544a2cb8e0bccf9f14118c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:39:55.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:39:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:39:55.579+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:39:56.081 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:55 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:55.709+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:56.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:55.905+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:56.112 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:55 smithi117 podman[77728]: 2024-09-15 22:39:55.984836805 +0000 UTC m=+6.057182773 container cleanup 6dad0b316c4ccbb3c3bd406cf9ff9ec2a603b06938544a2cb8e0bccf9f14118c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T22:39:56.112 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:55 smithi117 bash[77728]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4 2024-09-15T22:39:56.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:55.822+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:56.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:55 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:55.822+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:56.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:55.891+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:56.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:55 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:55.891+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:39:56.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:56 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:56.027+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:56.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:56.114+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:56.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:56.114+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:56.612 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77757]: 2024-09-15 22:39:56.227769098 +0000 UTC m=+0.655538026 container remove 6dad0b316c4ccbb3c3bd406cf9ff9ec2a603b06938544a2cb8e0bccf9f14118c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:39:56.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:39:56 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:39:56.546+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:39:56.915 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77843]: 2024-09-15 22:39:56.552380037 +0000 UTC m=+0.015903091 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:39:56.915 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77843]: 2024-09-15 22:39:56.670250119 +0000 UTC m=+0.133773163 container create 4c9dd8e842e82fca92a811a81985997b998bb5e83889e5c2c5877725eafdb56d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-15T22:39:56.915 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:56 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:56.841+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:56.916 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:56 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:56.841+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:57.074 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:56.688+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:57.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:57 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:57.073+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:57.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:57 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:57.073+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:57.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:56 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:56.914+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:57.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:56 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:56.914+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:39:57.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:57 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:57.050+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:57.362 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:56 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:56.917+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:57.362 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77843]: 2024-09-15 22:39:56.914404029 +0000 UTC m=+0.377927072 container init 4c9dd8e842e82fca92a811a81985997b998bb5e83889e5c2c5877725eafdb56d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-15T22:39:57.362 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77843]: 2024-09-15 22:39:56.917831705 +0000 UTC m=+0.381354768 container start 4c9dd8e842e82fca92a811a81985997b998bb5e83889e5c2c5877725eafdb56d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T22:39:57.362 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:56 smithi117 podman[77843]: 2024-09-15 22:39:56.997805896 +0000 UTC m=+0.461328954 container attach 4c9dd8e842e82fca92a811a81985997b998bb5e83889e5c2c5877725eafdb56d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:39:57.362 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 15 22:39:57 smithi117 podman[77843]: 2024-09-15 22:39:57.104330501 +0000 UTC m=+0.567853545 container died 4c9dd8e842e82fca92a811a81985997b998bb5e83889e5c2c5877725eafdb56d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T22:39:57.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.4.service' 2024-09-15T22:39:57.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:39:57 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:39:57.549+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:39:57.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:39:57.881 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-15T22:39:57.882 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-15T22:39:57.882 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.5 2024-09-15T22:39:58.058 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:57 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:57.689+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:58.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:57 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:57.888+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:58.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:57 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:57.804+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:58.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:57 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:57.804+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:58.200 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:58.044+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:58.200 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:57 smithi196 systemd[1]: Stopping Ceph osd.5 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:39:58.201 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:57 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:57.961+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:58.201 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:57 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:57.961+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:39:58.201 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:58.198+0000 7f7f3e904640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:39:58.201 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:58.198+0000 7f7f3e904640 -1 osd.5 135 *** Got signal Terminated *** 2024-09-15T22:39:58.201 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:58.198+0000 7f7f3e904640 -1 osd.5 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:39:58.331 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:58 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:58.056+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:58.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:58 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:58.056+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:58.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:39:58 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:39:58.595+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:39:59.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:58 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:58.735+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:39:59.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:58 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:58.841+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:39:59.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:58 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:58.787+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:39:59.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:58 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:58.787+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:39:59.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:39:59.076+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:39:59.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:59.002+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:39:59.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:59.002+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:39:59.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:59.096+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:39:59.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:39:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:39:59.096+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:39:59.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:39:59.572+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:00.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:39:59 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:39:59.781+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:00.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:59 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:59.840+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:00.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:39:59 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:39:59.840+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:00.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:59 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:59.774+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:00.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:39:59 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:39:59.774+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:00.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:00.097+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:00.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:00.097+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:00.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:59.989+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:40:00.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:39:59 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:39:59.989+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:40:00.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:00.083+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:00.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:00.083+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:00.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:00 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:00.578+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:01.067 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:00.777+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:01.068 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:00.777+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:01.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:00 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:00.744+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:01.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:00 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:00.744+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:01.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:00 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:00.844+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:01.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:00 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:00.844+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:01.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:01.066+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:01.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:01.066+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:01.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:01.146+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:01.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:01.146+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:01.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:01.013+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:40:01.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:01.013+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:40:01.986 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:01.610+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:02.053 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:01.739+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:02.053 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:01 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:01.739+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:02.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:01.767+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:02.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:01.767+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:02.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:01.830+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:02.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:01 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:01.830+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:02.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:02.051+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:02.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:02.051+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:02.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:02.051+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:02.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:01.984+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:40:02.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:01 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:01.984+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:40:02.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:02.176+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:02.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:02.176+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:02.950 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:02.637+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:02.950 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:02.637+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:03.070 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:02.702+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:03.070 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:02 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:02.702+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:03.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:02.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:03.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:02.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:03.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:02.861+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:03.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:02 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:02.861+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:03.331 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:03.069+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:03.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:03.069+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:03.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:03.069+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:03.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:02.948+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:22.046276+0000 front 2024-09-15T22:39:22.046113+0000 (oldest deadline 2024-09-15T22:39:43.146135+0000) 2024-09-15T22:40:03.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:02 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5[46716]: 2024-09-15T22:40:02.948+0000 7f7f3af1a640 -1 osd.5 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.147996+0000 front 2024-09-15T22:39:30.148090+0000 (oldest deadline 2024-09-15T22:39:54.848065+0000) 2024-09-15T22:40:03.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:03 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:03.191+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:03.354 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:03 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:03.191+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:04.069 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:03.737+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:04.070 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:03 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:03.737+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:04.076 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:03 smithi196 podman[78240]: 2024-09-15 22:40:03.761939395 +0000 UTC m=+5.675532739 container died 85a224fcbb61c2a1e3e415b9f68e6b8a28d45ee015973c514827225544c4c5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-15T22:40:04.076 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:03 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:03.613+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:04.076 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:03 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:03.613+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:04.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:03.753+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:04.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:03.753+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:04.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:03.833+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:04.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:03 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:03.833+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:04.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:04.068+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:04.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:04.068+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:04.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:04.068+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:04.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:04 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:04.186+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:04.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:04 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:04.186+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:04.353 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:04 smithi196 podman[78240]: 2024-09-15 22:40:04.191158195 +0000 UTC m=+6.104751539 container cleanup 85a224fcbb61c2a1e3e415b9f68e6b8a28d45ee015973c514827225544c4c5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:40:04.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:04 smithi196 bash[78240]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5 2024-09-15T22:40:04.853 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:04 smithi196 podman[78257]: 2024-09-15 22:40:04.476208607 +0000 UTC m=+0.709522783 container remove 85a224fcbb61c2a1e3e415b9f68e6b8a28d45ee015973c514827225544c4c5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:40:04.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:04 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:04.640+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:04.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:04 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:04.641+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:05.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:04.723+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:05.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:04.723+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:05.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:04.802+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:05.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:04.802+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:05.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:04.710+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:05.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:04.710+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:05.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:04 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:04.710+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:05.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:04 smithi196 podman[78343]: 2024-09-15 22:40:04.778456166 +0000 UTC m=+0.016348607 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:05.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:04 smithi196 podman[78343]: 2024-09-15 22:40:04.887738387 +0000 UTC m=+0.125630832 container create d98e3a9bee0f6b26df36023be6af3275a847e3f022d059490d9309b929ae07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-15T22:40:05.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:05 smithi196 podman[78343]: 2024-09-15 22:40:05.121356136 +0000 UTC m=+0.359248577 container init d98e3a9bee0f6b26df36023be6af3275a847e3f022d059490d9309b929ae07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-15T22:40:05.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:05 smithi196 podman[78343]: 2024-09-15 22:40:05.124629564 +0000 UTC m=+0.362522008 container start d98e3a9bee0f6b26df36023be6af3275a847e3f022d059490d9309b929ae07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-15T22:40:05.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:05.081+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:05.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:05.081+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:05.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:05.081+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:05.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:05 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:05.174+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:05.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:05 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:05.174+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:05.604 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:05 smithi196 podman[78343]: 2024-09-15 22:40:05.213312193 +0000 UTC m=+0.451204646 container attach d98e3a9bee0f6b26df36023be6af3275a847e3f022d059490d9309b929ae07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-15T22:40:05.604 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 15 22:40:05 smithi196 podman[78343]: 2024-09-15 22:40:05.321591345 +0000 UTC m=+0.559483793 container died d98e3a9bee0f6b26df36023be6af3275a847e3f022d059490d9309b929ae07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:40:05.909 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:05 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:05.618+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:05.909 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:05 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:05.618+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:05.932 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.5.service' 2024-09-15T22:40:05.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:05.971 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-15T22:40:05.972 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-15T22:40:05.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.6 2024-09-15T22:40:05.999 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:05.679+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:05.999 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:05 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:05.679+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:06.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:05.791+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:06.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:05.791+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:06.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:05.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:06.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:05.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:06.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:05 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:05.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:06.080+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:06.080+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:06.080+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 systemd[1]: Stopping Ceph osd.6 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:06.270+0000 7f7127d90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:06.272 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:06.270+0000 7f7127d90640 -1 osd.6 135 *** Got signal Terminated *** 2024-09-15T22:40:06.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:06 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:06.133+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:06.353 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:06 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:06.133+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:06.582 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:06.270+0000 7f7127d90640 -1 osd.6 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:07.081 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:06.661+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:07.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:06 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:06.661+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:07.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:06 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:06.642+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:07.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:06 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:06.642+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:07.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:06 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:06.788+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:07.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:06 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:06.788+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:07.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:06 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:06.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:07.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:06 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:06.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:07.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:06 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:06.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:07.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:07 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:07.120+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:07.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:07 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:07.120+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:07.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:07 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:07.120+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:07.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:07 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:07.151+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:07.605 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:07 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:07.151+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:08.081 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:07 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:07.691+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:08.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:07 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:07.691+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:08.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:07 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:07.649+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:08.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:07 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:07.649+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:08.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:07 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:07.649+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:08.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:07 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:07.763+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:08.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:07 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:07.763+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:08.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:07 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:07.665+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:08.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:07 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:07.665+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:08.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:07 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:07.665+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:08.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:08 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:08.150+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:08.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:08 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:08.150+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:08.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:08 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:08.150+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:08.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:08 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:08.194+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:08.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:08 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:08.194+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:09.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:08 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:08.703+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:09.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:08 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:08.703+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:09.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:08 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:08.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:09.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:08 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:08.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:09.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:08 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:08.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:09.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:08 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:08.716+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:09.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:08 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:08.716+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:09.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:08 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:08.645+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:09.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:08 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:08.645+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:09.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:08 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:08.645+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:09.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:09.133+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:09.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:09.133+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:09.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:09.133+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:09.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:09 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:09.220+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:09.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:09 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:09.220+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:10.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:09.697+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:10.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:09 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:09.697+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:10.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:09 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:09.696+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:10.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:09 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:09.696+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:10.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:09 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:09.696+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:10.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:09.718+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:10.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:09.718+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:10.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:09.689+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:10.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:09.689+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:10.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:09.689+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:10.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:10.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:10.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:10.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:10.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:10.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:10.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:10.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:10.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:10 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:10.194+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:10.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:10 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:10.194+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:11.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:10.702+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:26.822707+0000 front 2024-09-15T22:39:26.822681+0000 (oldest deadline 2024-09-15T22:39:52.122830+0000) 2024-09-15T22:40:11.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:10 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6[58061]: 2024-09-15T22:40:10.702+0000 7f71243a6640 -1 osd.6 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:38.525082+0000 front 2024-09-15T22:39:38.524897+0000 (oldest deadline 2024-09-15T22:40:00.225016+0000) 2024-09-15T22:40:11.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:10 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:10.654+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:11.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:10 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:10.654+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:11.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:10 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:10.654+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:11.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:10.670+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:11.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:10.670+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:11.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:10.684+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:11.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:10.684+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:11.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:10 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:10.684+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:11.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:11.121+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:11.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:11.121+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:11.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:11.121+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:11.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:11 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:11.121+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:11.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:11.190+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:11.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:11.190+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:11.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:11.190+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:12.082 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:11 smithi077 podman[150711]: 2024-09-15 22:40:11.715117191 +0000 UTC m=+5.551060387 container died bc0e860bcefd5a30cd338747b01f2920906a515ac6f612b7cba3a73519edd808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:12.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:11.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:12.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:11.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:12.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:11 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:11.647+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:12.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:11.696+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:12.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:11.696+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:12.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:11.696+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:12.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:11.703+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:12.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:11.703+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:12.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:11 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:11.703+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:12 smithi077 podman[150711]: 2024-09-15 22:40:12.12061427 +0000 UTC m=+5.956557506 container cleanup bc0e860bcefd5a30cd338747b01f2920906a515ac6f612b7cba3a73519edd808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:12 smithi077 bash[150711]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:12.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:12.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:12.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:12.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:12 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:12.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:12.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:12.181+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:12.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:12.181+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:12.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:12.181+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:12.788 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:12 smithi077 podman[150727]: 2024-09-15 22:40:12.380916283 +0000 UTC m=+0.664681595 container remove bc0e860bcefd5a30cd338747b01f2920906a515ac6f612b7cba3a73519edd808 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T22:40:13.044 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:12 smithi077 podman[150816]: 2024-09-15 22:40:12.690214853 +0000 UTC m=+0.016005452 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:13.044 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:12 smithi077 podman[150816]: 2024-09-15 22:40:12.803309035 +0000 UTC m=+0.129099677 container create 761029be8cc249573a13bfd5b2059b4cc2258f0c1f080b654b4f587787a2b960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-15T22:40:13.044 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:13 smithi077 podman[150816]: 2024-09-15 22:40:13.0431557 +0000 UTC m=+0.368946299 container init 761029be8cc249573a13bfd5b2059b4cc2258f0c1f080b654b4f587787a2b960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-15T22:40:13.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:12.640+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:13.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:12.640+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:13.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:12 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:12.640+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:12.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:12.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:12.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:12.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:12.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:12.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:13.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:12.718+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:13 smithi077 podman[150816]: 2024-09-15 22:40:13.046609497 +0000 UTC m=+0.372400097 container start 761029be8cc249573a13bfd5b2059b4cc2258f0c1f080b654b4f587787a2b960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:13 smithi077 podman[150816]: 2024-09-15 22:40:13.118059335 +0000 UTC m=+0.443849957 container attach 761029be8cc249573a13bfd5b2059b4cc2258f0c1f080b654b4f587787a2b960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.6.smithi077.stdout:Sep 15 22:40:13 smithi077 podman[150816]: 2024-09-15 22:40:13.235390016 +0000 UTC m=+0.561180623 container died 761029be8cc249573a13bfd5b2059b4cc2258f0c1f080b654b4f587787a2b960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:13 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:13.145+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:13 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:13.145+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:13 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:13.145+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:13.332 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:13 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:13.145+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:13.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:13.157+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:13.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:13.157+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:13.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:13.157+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:14.006 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.6.service' 2024-09-15T22:40:14.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:14.085 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-15T22:40:14.085 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-15T22:40:14.086 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.7 2024-09-15T22:40:14.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:13.616+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:14.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:13.616+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:14.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:13 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:13.616+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:14.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:13.747+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:14.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:13.747+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:14.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:13.747+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:14.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:13.683+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:14.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:13.683+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:14.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:13.683+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:14.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:13 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:13.683+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:14.488 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:14 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:14.184+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:14.488 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:14 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:14.184+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:14.488 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:14 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:14.184+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:14.488 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:14 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:14.184+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:14.503 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 systemd[1]: Stopping Ceph osd.7 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:14.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:14.196+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:14.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:14.196+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:14.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:14.196+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:14.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:14.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:14.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:14.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:14.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:14.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:14.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:14.680+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:14.862 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.502+0000 7f0eb4659640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:14.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.502+0000 7f0eb4659640 -1 osd.7 135 *** Got signal Terminated *** 2024-09-15T22:40:14.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.502+0000 7f0eb4659640 -1 osd.7 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:14.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.718+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:14.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.718+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:14.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:14 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:14.718+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:15.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:14.625+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:15.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:14.625+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:15.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:14 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:14.625+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:15.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:15 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:15.136+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:15.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:15 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:15.136+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:15.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:15 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:15.137+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:15.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:15 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:15.137+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:15.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:15.211+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:15.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:15.211+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:15.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:15.211+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:16.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:15.663+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:16.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:15.663+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:16.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:15 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:15.663+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:16.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:15.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:16.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:15.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:16.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:15.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:16.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:15.716+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:16.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:15.716+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:16.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:15.716+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:16.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:15 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:15.716+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:16.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:16 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:16.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:16.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:16 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:16.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:16.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:16 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:16.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:16.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:16 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:16.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:16.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:16.204+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:16.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:16.204+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:16.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:16.204+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:16.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:16.204+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:17.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:16.691+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:17.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:16.691+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:17.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:16 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:16.691+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:17.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:16.691+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:17.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:16.691+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:17.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:16.691+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:17.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:16.691+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.705399+0000 front 2024-09-15T22:39:54.705429+0000 (oldest deadline 2024-09-15T22:40:15.805629+0000) 2024-09-15T22:40:17.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:16.688+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:17.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:16.688+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:17.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:16.688+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:17.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:16.688+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:17.483 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:17.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:17.483 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:17.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:17.483 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:17.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:17.483 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:17 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:17.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:17.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:17.231+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:17.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:17.231+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:17.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:17.231+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:17.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:17.231+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:18.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:17.715+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:18.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:17.715+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:18.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:17 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:17.715+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:18.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:17.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:18.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:17.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:18.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:17.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:18.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:17.711+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.705399+0000 front 2024-09-15T22:39:54.705429+0000 (oldest deadline 2024-09-15T22:40:15.805629+0000) 2024-09-15T22:40:18.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:17.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:18.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:17.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:18.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:17.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:18.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:17.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:18.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:17 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:17.701+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:18.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:18 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:18.170+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:18.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:18 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:18.170+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:18.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:18 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:18.170+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:18.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:18 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:18.170+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:18.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:18 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:18.170+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:18.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:18.218+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:18.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:18.218+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:18.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:18.218+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:18.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:18.218+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:19.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:18.672+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:19.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:18.672+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:19.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:18.672+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:19.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:18 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:18.672+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:18.683+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:30.702309+0000 front 2024-09-15T22:39:30.702267+0000 (oldest deadline 2024-09-15T22:39:53.602421+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:18.683+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.602908+0000 front 2024-09-15T22:39:33.602935+0000 (oldest deadline 2024-09-15T22:39:59.502997+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:18.683+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:45.404142+0000 front 2024-09-15T22:39:45.404013+0000 (oldest deadline 2024-09-15T22:40:11.304080+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7[52609]: 2024-09-15T22:40:18.683+0000 7f0eacc67640 -1 osd.7 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.705399+0000 front 2024-09-15T22:39:54.705429+0000 (oldest deadline 2024-09-15T22:40:15.805629+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:18.722+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:18.722+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:19.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:18.722+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:19.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:18.722+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:19.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:18 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:18.722+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:19.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:19.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:19.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:19.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:19.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:19.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:19.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:19.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:19.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:19 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:19.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:19.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:19.256+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:19.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:19.256+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:19.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:19.256+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:19.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:19.257+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:20.012 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:19.677+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:20.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:19.677+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:20.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:19.677+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:20.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:19.677+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:20.013 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:19 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:19.677+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:20.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:19.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:20.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:19.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:20.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:19.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:20.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:19 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:19.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:20.288 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:20 smithi117 podman[77947]: 2024-09-15 22:40:20.066067913 +0000 UTC m=+5.789234210 container died f0925eb253da5171a362b3195211349cd4d4ab910f212a8c9e24738f81ab60e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:40:20.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:20.149+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:20.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:20.149+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:20.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:20.149+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:20.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:20.149+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:20.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:20 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:20.149+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:20.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:20.264+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:20.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:20.264+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:20.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:20.264+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:20.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:20.264+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:20.612 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:20 smithi117 podman[77947]: 2024-09-15 22:40:20.443459097 +0000 UTC m=+6.166625397 container cleanup f0925eb253da5171a362b3195211349cd4d4ab910f212a8c9e24738f81ab60e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:20.612 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:20 smithi117 bash[77947]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7 2024-09-15T22:40:21.100 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:20 smithi117 podman[77963]: 2024-09-15 22:40:20.719679783 +0000 UTC m=+0.655151933 container remove f0925eb253da5171a362b3195211349cd4d4ab910f212a8c9e24738f81ab60e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:40:21.100 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:20.653+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:21.100 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:20.653+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:21.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:20.653+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:21.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:20.653+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:21.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:20.653+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:21.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:20.706+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:21.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:20.706+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:21.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:20.706+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:21.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:20 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:20.706+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:21.362 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.002299953 +0000 UTC m=+0.016147395 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:21.363 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.114278949 +0000 UTC m=+0.128126378 container create 052518ac4ce853569b3cd7b4a034dd5f34d80247a88032fe453b41ed0260774a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:40:21.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:21.195+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:21.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:21.195+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:21.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:21.195+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:21.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:21.195+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:21.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:21 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:21.195+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:21.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:21.308+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:21.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:21.308+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:21.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:21.308+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:21.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:21.308+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:21.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:21.613+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:21.613+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:21.613+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:21.613+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:21 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:21.613+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.386477624 +0000 UTC m=+0.400325053 container init 052518ac4ce853569b3cd7b4a034dd5f34d80247a88032fe453b41ed0260774a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.389892779 +0000 UTC m=+0.403740217 container start 052518ac4ce853569b3cd7b4a034dd5f34d80247a88032fe453b41ed0260774a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.492051814 +0000 UTC m=+0.505899255 container attach 052518ac4ce853569b3cd7b4a034dd5f34d80247a88032fe453b41ed0260774a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:40:21.863 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 15 22:40:21 smithi117 podman[78061]: 2024-09-15 22:40:21.574085948 +0000 UTC m=+0.587933389 container died 052518ac4ce853569b3cd7b4a034dd5f34d80247a88032fe453b41ed0260774a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:40:22.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:21.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:22.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:21.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:22.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:21.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:22.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:21 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:21.667+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:22.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.7.service' 2024-09-15T22:40:22.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:22.271 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-15T22:40:22.271 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-15T22:40:22.271 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.8 2024-09-15T22:40:22.570 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 systemd[1]: Stopping Ceph osd.8 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:22.570 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:22.345+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:22.570 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:22.345+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:22.570 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:22.345+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:22.570 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:22.345+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:22.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:22.183+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:22.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:22.183+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:22.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:22.183+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:22.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:22.183+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:22.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:22 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:22.183+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:22.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.568+0000 7f09364ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:22.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.568+0000 7f09364ae640 -1 osd.8 135 *** Got signal Terminated *** 2024-09-15T22:40:22.853 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.568+0000 7f09364ae640 -1 osd.8 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:22.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.662+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:22.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.662+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:22.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.662+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:22.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:22 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:22.662+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:23.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:22.637+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:23.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:22.637+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:23.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:22.637+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:23.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:22.637+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:23.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:22 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:22.637+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:23.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:23.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:23.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:23.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:23.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:23.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:23.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:23.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:23.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:23 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:23.162+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:23.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:23.317+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:23.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:23.317+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:23.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:23.317+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:23.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:23.317+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:23.880 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:23.651+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:23.880 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:23.651+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:23.881 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:23.651+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:23.881 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:23 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:23.651+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:24.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:23.616+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:24.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:23.616+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:24.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:23.616+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:24.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:23.616+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:24.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:23 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:23.616+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:24.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:24.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:24.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:24.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:24.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:24.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:24.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:24.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:24.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:24 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:24.143+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:24.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:24.347+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:24.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:24.347+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:24.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:24.347+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:24.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:24.347+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:24.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:24.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:24.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:24.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:24.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:24.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:24 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:24.579+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:25.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:24.666+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:25.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:24.666+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:25.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:24.666+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:25.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:24.666+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:25.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:24.666+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:02.678147+0000 front 2024-09-15T22:40:02.678189+0000 (oldest deadline 2024-09-15T22:40:23.778250+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:25.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:25 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:25.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:25.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:25.323+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:25.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:25.323+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:25.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:25.323+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:25.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:25.323+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:26.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:25.685+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:26.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:25.685+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:26.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:25.685+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:26.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:25.685+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:26.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:25 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:25.685+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:02.678147+0000 front 2024-09-15T22:40:02.678189+0000 (oldest deadline 2024-09-15T22:40:23.778250+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:26.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:25 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:25.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:26.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:26.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:26.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:26.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:26.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:26.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:26 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:26.202+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:26.670 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:26.371+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:26.670 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:26.371+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:26.670 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:26.371+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:26.670 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:26.371+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:26.670 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:26.371+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:26.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:26 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:26.593+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:27.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:26.668+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:28.972267+0000 front 2024-09-15T22:39:28.972207+0000 (oldest deadline 2024-09-15T22:39:54.872392+0000) 2024-09-15T22:40:27.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:26.668+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.173384+0000 front 2024-09-15T22:39:37.173362+0000 (oldest deadline 2024-09-15T22:40:02.374033+0000) 2024-09-15T22:40:27.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:26.668+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:43.475360+0000 front 2024-09-15T22:39:43.475239+0000 (oldest deadline 2024-09-15T22:40:07.575163+0000) 2024-09-15T22:40:27.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:26.668+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:53.877327+0000 front 2024-09-15T22:39:53.877240+0000 (oldest deadline 2024-09-15T22:40:18.577278+0000) 2024-09-15T22:40:27.104 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:26 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8[53644]: 2024-09-15T22:40:26.668+0000 7f092eabc640 -1 osd.8 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:02.678147+0000 front 2024-09-15T22:40:02.678189+0000 (oldest deadline 2024-09-15T22:40:23.778250+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:27.536 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:27 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:27.156+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:27.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:27 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:27.360+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:27.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:27 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:27.360+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:27.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:27 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:27.360+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:27.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:27 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:27.360+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:27.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:27 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:27.360+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:27.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:27.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:27.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:27.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:27.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:27.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:27 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:27.598+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:28 smithi196 podman[78445]: 2024-09-15 22:40:28.212084787 +0000 UTC m=+5.744184662 container died 4882d279b52bf2eb5d02fffd2eff1e1fde45027920a2c3ae3fb0a1e980c6ef6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:28.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:28.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:28.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:28.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:28.538 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:28.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:28.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:28 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:28.190+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:28.845 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:28 smithi196 podman[78445]: 2024-09-15 22:40:28.652067826 +0000 UTC m=+6.184167699 container cleanup 4882d279b52bf2eb5d02fffd2eff1e1fde45027920a2c3ae3fb0a1e980c6ef6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:40:28.846 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:28 smithi196 bash[78445]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:28.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:28 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:28.561+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:29.103 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:28 smithi196 podman[78473]: 2024-09-15 22:40:28.845189336 +0000 UTC m=+0.631095053 container remove 4882d279b52bf2eb5d02fffd2eff1e1fde45027920a2c3ae3fb0a1e980c6ef6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.140433009 +0000 UTC m=+0.016016713 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.262735206 +0000 UTC m=+0.138318932 container create 008fe0b2cc6344d2429f168ca758ffef1fe36a24a93a004c3b8e8201a61a540a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:29 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:29.392+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:29 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:29.392+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:29 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:29.392+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:29 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:29.392+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:29.491 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:29 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:29.392+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:29.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:29 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:29.175+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:29.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.49039723 +0000 UTC m=+0.365980914 container init 008fe0b2cc6344d2429f168ca758ffef1fe36a24a93a004c3b8e8201a61a540a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:29.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.49513289 +0000 UTC m=+0.370716577 container start 008fe0b2cc6344d2429f168ca758ffef1fe36a24a93a004c3b8e8201a61a540a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T22:40:29.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.573512173 +0000 UTC m=+0.449095867 container attach 008fe0b2cc6344d2429f168ca758ffef1fe36a24a93a004c3b8e8201a61a540a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T22:40:29.854 INFO:journalctl@ceph.osd.8.smithi196.stdout:Sep 15 22:40:29 smithi196 podman[78561]: 2024-09-15 22:40:29.684330373 +0000 UTC m=+0.559914061 container died 008fe0b2cc6344d2429f168ca758ffef1fe36a24a93a004c3b8e8201a61a540a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:29.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:29 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:29.592+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:30.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.8.service' 2024-09-15T22:40:30.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:30.402 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-15T22:40:30.402 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-15T22:40:30.402 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.9 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:30.429 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.168+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:30.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:30.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:30.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:30.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:30.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:30.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:30.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:30.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:30.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:30 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:30.388+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:30.694 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 systemd[1]: Stopping Ceph osd.9 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:30.864 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:30 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:30.590+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:31.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.692+0000 7fbac60a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:31.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.692+0000 7fbac60a0640 -1 osd.9 135 *** Got signal Terminated *** 2024-09-15T22:40:31.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:30 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:30.692+0000 7fbac60a0640 -1 osd.9 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:31.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:31 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:31.203+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:31.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:31.426+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:31.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:31.426+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:31.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:31.426+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:31.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:31.426+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:31.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:31 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:31.426+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:31.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:31 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:31.542+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:32.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:32.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:32.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:32.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:32.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:32.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:32 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:32.176+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:32.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:32.433+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:32.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:32.433+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:32.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:32.433+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:32.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:32.433+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:32.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:32.433+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:32.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:32 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:32.568+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:33.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:33 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:33.199+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:33.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:33.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:33.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:33.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:33.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:33.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:33 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:33.479+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:33.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:33 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:33.578+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:34.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:34 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:34.196+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:34.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:34.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:34.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:34.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:34.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:34.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:34 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:34.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:35.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:34 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:34.611+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:21.942982+0000 front 2024-09-15T22:39:21.942959+0000 (oldest deadline 2024-09-15T22:39:44.843094+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:27.144455+0000 front 2024-09-15T22:39:27.144383+0000 (oldest deadline 2024-09-15T22:39:51.744585+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:35.245727+0000 front 2024-09-15T22:39:35.245825+0000 (oldest deadline 2024-09-15T22:40:01.145849+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.447105+0000 front 2024-09-15T22:39:46.447084+0000 (oldest deadline 2024-09-15T22:40:09.947325+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:54.648644+0000 front 2024-09-15T22:39:54.648446+0000 (oldest deadline 2024-09-15T22:40:18.148510+0000) 2024-09-15T22:40:35.582 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:35 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9[64754]: 2024-09-15T22:40:35.194+0000 7fbabe6ae640 -1 osd.9 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:01.951601+0000 front 2024-09-15T22:40:01.951701+0000 (oldest deadline 2024-09-15T22:40:24.851636+0000) 2024-09-15T22:40:35.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:35.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:35.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:35.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:35.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:35.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:35 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:35.521+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:36.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:36.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:36.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:35 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:35.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:36.581 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:36 smithi077 podman[150931]: 2024-09-15 22:40:36.216742484 +0000 UTC m=+5.617805504 container died 344ae49ca4e55552ad71e6ea7d21132072ad498a28772956a3f5b3212e2cee13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-15T22:40:36.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:36.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:36.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:36.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:36.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:36.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:36.507+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:37.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:36 smithi077 podman[150931]: 2024-09-15 22:40:36.626180494 +0000 UTC m=+6.027243554 container cleanup 344ae49ca4e55552ad71e6ea7d21132072ad498a28772956a3f5b3212e2cee13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True) 2024-09-15T22:40:37.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:36 smithi077 bash[150931]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9 2024-09-15T22:40:37.082 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:36 smithi077 podman[150951]: 2024-09-15 22:40:36.844366905 +0000 UTC m=+0.625557032 container remove 344ae49ca4e55552ad71e6ea7d21132072ad498a28772956a3f5b3212e2cee13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:40:37.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:37.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:37.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:37.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:37.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:37.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:37.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:36 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:36.620+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:37.499 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.137774443 +0000 UTC m=+0.015615277 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:37.499 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.253665379 +0000 UTC m=+0.131506202 container create dab74cd613a27812bf92aeb7fc1797abedbf7ec37d7a3f1062d6c089854623c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T22:40:37.499 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.498367987 +0000 UTC m=+0.376208822 container init dab74cd613a27812bf92aeb7fc1797abedbf7ec37d7a3f1062d6c089854623c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-15T22:40:37.832 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.502097247 +0000 UTC m=+0.379938069 container start dab74cd613a27812bf92aeb7fc1797abedbf7ec37d7a3f1062d6c089854623c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:40:37.832 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.564584071 +0000 UTC m=+0.442424908 container attach dab74cd613a27812bf92aeb7fc1797abedbf7ec37d7a3f1062d6c089854623c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:37.832 INFO:journalctl@ceph.osd.9.smithi077.stdout:Sep 15 22:40:37 smithi077 podman[151038]: 2024-09-15 22:40:37.68801471 +0000 UTC m=+0.565855550 container died dab74cd613a27812bf92aeb7fc1797abedbf7ec37d7a3f1062d6c089854623c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:40:37.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:37.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:37.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:37.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:37.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:37.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:37 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:37.528+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:38.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:38.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:38.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:38.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:38.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:38.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:38.113 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:37 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:37.627+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:38.298 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.9.service' 2024-09-15T22:40:38.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:38.379 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-15T22:40:38.379 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-15T22:40:38.379 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.10 2024-09-15T22:40:38.731 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 systemd[1]: Stopping Ceph osd.10 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:38.731 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:38.731 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.635+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.729+0000 7fcaa62e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.729+0000 7fcaa62e0640 -1 osd.10 135 *** Got signal Terminated *** 2024-09-15T22:40:38.732 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:38 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:38.729+0000 7fcaa62e0640 -1 osd.10 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:38.859 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:38.860 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:38.860 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:38.860 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:38.860 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:38.860 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:38 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:38.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:39.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:39.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:39.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:39.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:39.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:39.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:39 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:39.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:39.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:39.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:39.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:39.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:39.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:39.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:39.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:39 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:39.591+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:40.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:40.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:40.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:40.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:40.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:40.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:40 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:40.576+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:40.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:40 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:40.574+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:41.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:41.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:41.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:41.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:41.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:41.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:41.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:41 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:41.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:41.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:41.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:41.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:41.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:41.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:41.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:41.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:41 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:41.535+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:42.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:42 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:42.574+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:42.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:42.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:42.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:42.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:42.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:42.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:42.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:42 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:42.529+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:43.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:43 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:43.567+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.696528+0000 front 2024-09-15T22:39:17.696595+0000 (oldest deadline 2024-09-15T22:39:43.596684+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-15T22:39:29.398395+0000 front 2024-09-15T22:39:29.398413+0000 (oldest deadline 2024-09-15T22:39:51.698590+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:37.599594+0000 front 2024-09-15T22:39:37.599716+0000 (oldest deadline 2024-09-15T22:40:04.600177+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.901315+0000 front 2024-09-15T22:39:46.901441+0000 (oldest deadline 2024-09-15T22:40:12.201388+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:52.201867+0000 front 2024-09-15T22:39:52.201904+0000 (oldest deadline 2024-09-15T22:40:16.902046+0000) 2024-09-15T22:40:43.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:39:58.003122+0000 front 2024-09-15T22:39:58.003111+0000 (oldest deadline 2024-09-15T22:40:23.903021+0000) 2024-09-15T22:40:43.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:43 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10[59184]: 2024-09-15T22:40:43.523+0000 7fca9e8ee640 -1 osd.10 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:10.205559+0000 front 2024-09-15T22:40:10.205509+0000 (oldest deadline 2024-09-15T22:40:34.905369+0000) 2024-09-15T22:40:44.589 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:44 smithi117 podman[78167]: 2024-09-15 22:40:44.327471767 +0000 UTC m=+5.756291140 container died 7e5e31cd18159160058b6f49d17bf3b0a6477f24af6dbf95946dae4546e0040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T22:40:44.839 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:44 smithi117 podman[78167]: 2024-09-15 22:40:44.670657306 +0000 UTC m=+6.099476676 container cleanup 7e5e31cd18159160058b6f49d17bf3b0a6477f24af6dbf95946dae4546e0040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-15T22:40:44.839 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:44 smithi117 bash[78167]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10 2024-09-15T22:40:44.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:44.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:44 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:44.584+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:45.112 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:44 smithi117 podman[78183]: 2024-09-15 22:40:44.838532688 +0000 UTC m=+0.507491478 container remove 7e5e31cd18159160058b6f49d17bf3b0a6477f24af6dbf95946dae4546e0040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:40:45.542 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.136954705 +0000 UTC m=+0.015374774 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:45.542 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.239150183 +0000 UTC m=+0.117570288 container create d4c260815146b9dc7bf2555a9daa5b678ae6a3ec56efd1b0bead00e7f4f2a1c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-15T22:40:45.543 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.458073403 +0000 UTC m=+0.336493464 container init d4c260815146b9dc7bf2555a9daa5b678ae6a3ec56efd1b0bead00e7f4f2a1c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:40:45.543 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.461245669 +0000 UTC m=+0.339665741 container start d4c260815146b9dc7bf2555a9daa5b678ae6a3ec56efd1b0bead00e7f4f2a1c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-15T22:40:45.543 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.541379446 +0000 UTC m=+0.419799516 container attach d4c260815146b9dc7bf2555a9daa5b678ae6a3ec56efd1b0bead00e7f4f2a1c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:40:45.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:45.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:45.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:45.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:45.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:45.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:45.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:45 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:45.564+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:45.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 15 22:40:45 smithi117 podman[78269]: 2024-09-15 22:40:45.654951081 +0000 UTC m=+0.533371149 container died d4c260815146b9dc7bf2555a9daa5b678ae6a3ec56efd1b0bead00e7f4f2a1c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:40:46.231 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.10.service' 2024-09-15T22:40:46.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:46.270 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-15T22:40:46.270 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-15T22:40:46.270 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.11 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 systemd[1]: Stopping Ceph osd.11 for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:46.589 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:46.590 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:46.590 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.519+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:46.590 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.586+0000 7feaa577d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:46.590 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.586+0000 7feaa577d640 -1 osd.11 135 *** Got signal Terminated *** 2024-09-15T22:40:46.590 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:46 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:46.586+0000 7feaa577d640 -1 osd.11 135 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:40:47.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:47.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:47.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:47.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:47.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:47.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:47.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:47 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:47.486+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:48.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:48.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:48.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:48.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:48.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:48.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:48.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:48 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:48.513+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:49.856 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:49.857 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:49 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:49.544+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-15T22:40:25.252772+0000 front 2024-09-15T22:40:25.252756+0000 (oldest deadline 2024-09-15T22:40:49.352913+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:50.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:50 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:50.554+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-15T22:40:25.252772+0000 front 2024-09-15T22:40:25.252756+0000 (oldest deadline 2024-09-15T22:40:49.352913+0000) 2024-09-15T22:40:51.853 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-09-15T22:39:17.640948+0000 front 2024-09-15T22:39:17.640914+0000 (oldest deadline 2024-09-15T22:39:44.641759+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-09-15T22:39:33.943959+0000 front 2024-09-15T22:39:33.944048+0000 (oldest deadline 2024-09-15T22:39:59.844147+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-09-15T22:39:46.146900+0000 front 2024-09-15T22:39:46.146781+0000 (oldest deadline 2024-09-15T22:40:10.846836+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-15T22:39:50.847380+0000 front 2024-09-15T22:39:50.847441+0000 (oldest deadline 2024-09-15T22:40:16.147573+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-09-15T22:40:00.848709+0000 front 2024-09-15T22:40:00.848627+0000 (oldest deadline 2024-09-15T22:40:25.548742+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-09-15T22:40:07.249590+0000 front 2024-09-15T22:40:07.249652+0000 (oldest deadline 2024-09-15T22:40:33.149545+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-15T22:40:14.850623+0000 front 2024-09-15T22:40:14.850487+0000 (oldest deadline 2024-09-15T22:40:40.650986+0000) 2024-09-15T22:40:51.854 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:51 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11[60494]: 2024-09-15T22:40:51.579+0000 7fea9dd8b640 -1 osd.11 135 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-09-15T22:40:25.252772+0000 front 2024-09-15T22:40:25.252756+0000 (oldest deadline 2024-09-15T22:40:49.352913+0000) 2024-09-15T22:40:52.321 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:52 smithi196 podman[78664]: 2024-09-15 22:40:52.041738917 +0000 UTC m=+5.578546733 container died 621f829e8b042831bfe4fde310dbd01457a6f64fffdc8d20c464e898f39b09fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-15T22:40:52.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:52 smithi196 podman[78664]: 2024-09-15 22:40:52.411637923 +0000 UTC m=+5.948445752 container cleanup 621f829e8b042831bfe4fde310dbd01457a6f64fffdc8d20c464e898f39b09fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T22:40:52.603 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:52 smithi196 bash[78664]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11 2024-09-15T22:40:53.022 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:52 smithi196 podman[78680]: 2024-09-15 22:40:52.704663478 +0000 UTC m=+0.653310159 container remove 621f829e8b042831bfe4fde310dbd01457a6f64fffdc8d20c464e898f39b09fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-15T22:40:53.275 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:52.925391426 +0000 UTC m=+0.016855257 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:40:53.275 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:53.035887269 +0000 UTC m=+0.127351086 container create 1ab6b797df624b6f860dd93eca6a04647343ef8254db86faac058eda0a252d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-15T22:40:53.275 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:53.274897818 +0000 UTC m=+0.366361635 container init 1ab6b797df624b6f860dd93eca6a04647343ef8254db86faac058eda0a252d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-15T22:40:53.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:53.278140065 +0000 UTC m=+0.369603895 container start 1ab6b797df624b6f860dd93eca6a04647343ef8254db86faac058eda0a252d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-15T22:40:53.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:53.358074941 +0000 UTC m=+0.449538770 container attach 1ab6b797df624b6f860dd93eca6a04647343ef8254db86faac058eda0a252d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-15T22:40:53.604 INFO:journalctl@ceph.osd.11.smithi196.stdout:Sep 15 22:40:53 smithi196 podman[78767]: 2024-09-15 22:40:53.466323305 +0000 UTC m=+0.557787133 container died 1ab6b797df624b6f860dd93eca6a04647343ef8254db86faac058eda0a252d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-15T22:40:54.061 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@osd.11.service' 2024-09-15T22:40:54.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:54.102 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-15T22:40:54.102 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-15T22:40:54.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.a 2024-09-15T22:40:54.397 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:54 smithi077 systemd[1]: Stopping Ceph mds.a for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:54.654 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: 2024-09-15T22:40:54.395+0000 7f4698bab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:54.654 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:54 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a[71465]: 2024-09-15T22:40:54.395+0000 7f4698bab640 -1 mds.a *** got signal Terminated *** 2024-09-15T22:40:55.831 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:55 smithi077 podman[151141]: 2024-09-15 22:40:55.56569195 +0000 UTC m=+1.269964238 container died 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:40:56.090 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:55 smithi077 podman[151141]: 2024-09-15 22:40:55.913055945 +0000 UTC m=+1.617328233 container cleanup 887f7da9ecf31d841f7f5d2f5f00ed705db9175200bf0768ebfb5553e5295bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-15T22:40:56.090 INFO:journalctl@ceph.mds.a.smithi077.stdout:Sep 15 22:40:55 smithi077 bash[151141]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-a 2024-09-15T22:40:56.308 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.a.service' 2024-09-15T22:40:56.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:40:56.346 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-15T22:40:56.346 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-15T22:40:56.346 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.d 2024-09-15T22:40:56.676 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:40:56 smithi077 systemd[1]: Stopping Ceph mds.d for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:40:57.081 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:40:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d[70282]: 2024-09-15T22:40:56.675+0000 7f6484d5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:40:57.082 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:40:56 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d[70282]: 2024-09-15T22:40:56.675+0000 7f6484d5e640 -1 mds.d *** got signal Terminated *** 2024-09-15T22:40:59.832 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:40:59 smithi077 podman[151271]: 2024-09-15 22:40:59.554418312 +0000 UTC m=+2.984630134 container died 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-15T22:41:00.303 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:41:00 smithi077 podman[151271]: 2024-09-15 22:41:00.01746327 +0000 UTC m=+3.447675091 container cleanup 1afd74b55798154c59dc312a1b39c96e0b5ad28fb0e9639796de5cfad5603602 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-15T22:41:00.303 INFO:journalctl@ceph.mds.d.smithi077.stdout:Sep 15 22:41:00 smithi077 bash[151271]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-d 2024-09-15T22:41:00.440 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.d.service' 2024-09-15T22:41:00.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:00.473 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-15T22:41:00.474 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-15T22:41:00.474 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.g 2024-09-15T22:41:00.831 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:00 smithi077 systemd[1]: Stopping Ceph mds.g for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:00.832 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: 2024-09-15T22:41:00.804+0000 7f75ca55c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:00.832 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:00 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g[71081]: 2024-09-15T22:41:00.804+0000 7f75ca55c640 -1 mds.g *** got signal Terminated *** 2024-09-15T22:41:03.823 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:03 smithi077 podman[151393]: 2024-09-15 22:41:03.571136589 +0000 UTC m=+2.873762185 container died 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-15T22:41:04.082 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 podman[151393]: 2024-09-15 22:41:04.005062745 +0000 UTC m=+3.307688344 container cleanup 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:41:04.082 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 bash[151393]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g 2024-09-15T22:41:04.492 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.g.service' 2024-09-15T22:41:04.516 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 podman[151406]: 2024-09-15 22:41:04.256894634 +0000 UTC m=+0.683593391 container remove 3f91ebb6c0df056a91f59d12d302809035626f91e9031b57085ec9098ffe20de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-g, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:41:04.516 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.g.service: Deactivated successfully. 2024-09-15T22:41:04.516 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 systemd[1]: Stopped Ceph mds.g for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:41:04.516 INFO:journalctl@ceph.mds.g.smithi077.stdout:Sep 15 22:41:04 smithi077 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.g.service: Consumed 15.162s CPU time. 2024-09-15T22:41:04.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:04.527 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-15T22:41:04.527 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-15T22:41:04.527 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.j 2024-09-15T22:41:04.831 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:04 smithi077 systemd[1]: Stopping Ceph mds.j for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:05.133 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j[69899]: 2024-09-15T22:41:04.873+0000 7f7469884640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:05.134 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:04 smithi077 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j[69899]: 2024-09-15T22:41:04.873+0000 7f7469884640 -1 mds.j *** got signal Terminated *** 2024-09-15T22:41:08.082 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:07 smithi077 podman[151514]: 2024-09-15 22:41:07.811473876 +0000 UTC m=+3.061323081 container died 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-15T22:41:08.454 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:08 smithi077 podman[151514]: 2024-09-15 22:41:08.260244712 +0000 UTC m=+3.510093959 container cleanup 966d312e222aa4df8fca3f54a5dab910e0188c6b4b7f3f143bcd05e20f55ac93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:41:08.454 INFO:journalctl@ceph.mds.j.smithi077.stdout:Sep 15 22:41:08 smithi077 bash[151514]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-j 2024-09-15T22:41:08.674 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.j.service' 2024-09-15T22:41:08.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:08.706 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-15T22:41:08.706 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-15T22:41:08.706 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.b 2024-09-15T22:41:09.037 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:08 smithi117 systemd[1]: Stopping Ceph mds.b for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:09.362 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:41:09.035+0000 7fb92f565640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:09.362 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:09 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b[64398]: 2024-09-15T22:41:09.035+0000 7fb92f565640 -1 mds.b *** got signal Terminated *** 2024-09-15T22:41:11.862 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:11 smithi117 podman[78388]: 2024-09-15 22:41:11.607588222 +0000 UTC m=+2.692121549 container stop 8b356d65b8696b82f4b7d2f42c2b1a39ac5da867a62910646b90830ba2628380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-15T22:41:11.862 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:11 smithi117 podman[78388]: 2024-09-15 22:41:11.635014522 +0000 UTC m=+2.719547926 container died 8b356d65b8696b82f4b7d2f42c2b1a39ac5da867a62910646b90830ba2628380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-15T22:41:12.196 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:11 smithi117 podman[78388]: 2024-09-15 22:41:11.993530038 +0000 UTC m=+3.078063374 container cleanup 8b356d65b8696b82f4b7d2f42c2b1a39ac5da867a62910646b90830ba2628380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:41:12.196 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:11 smithi117 bash[78388]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b 2024-09-15T22:41:12.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.b.service' 2024-09-15T22:41:12.450 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:12 smithi117 podman[78400]: 2024-09-15 22:41:12.195082307 +0000 UTC m=+0.579698438 container remove 8b356d65b8696b82f4b7d2f42c2b1a39ac5da867a62910646b90830ba2628380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-15T22:41:12.450 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:12 smithi117 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-15T22:41:12.451 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:12 smithi117 systemd[1]: Stopped Ceph mds.b for 0ddb746c-73ae-11ef-bceb-c7b262605968. 2024-09-15T22:41:12.451 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 15 22:41:12 smithi117 systemd[1]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.b.service: Consumed 14min 29.027s CPU time. 2024-09-15T22:41:12.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:12.463 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-15T22:41:12.464 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-15T22:41:12.464 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.e 2024-09-15T22:41:12.790 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:12 smithi117 systemd[1]: Stopping Ceph mds.e for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:13.112 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: 2024-09-15T22:41:12.789+0000 7f360a701640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:13.112 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:12 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e[63779]: 2024-09-15T22:41:12.789+0000 7f360a701640 -1 mds.e *** got signal Terminated *** 2024-09-15T22:41:15.899 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:15 smithi117 podman[78510]: 2024-09-15 22:41:15.616101361 +0000 UTC m=+2.932489789 container died 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:41:16.283 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:15 smithi117 podman[78510]: 2024-09-15 22:41:15.972560865 +0000 UTC m=+3.288949249 container cleanup 9fb8f1e2bfe34d98cb9e1f22fa801da717e0ca908938bfda1390167ddb62b65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T22:41:16.283 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 15 22:41:15 smithi117 bash[78510]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-e 2024-09-15T22:41:16.449 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.e.service' 2024-09-15T22:41:16.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:16.482 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-15T22:41:16.482 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-15T22:41:16.482 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.h 2024-09-15T22:41:16.862 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:16 smithi117 systemd[1]: Stopping Ceph mds.h for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:16.862 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: 2024-09-15T22:41:16.784+0000 7f04810c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:16.862 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:16 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h[64082]: 2024-09-15T22:41:16.784+0000 7f04810c5640 -1 mds.h *** got signal Terminated *** 2024-09-15T22:41:19.862 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:19 smithi117 podman[78631]: 2024-09-15 22:41:19.610738358 +0000 UTC m=+2.918225756 container died 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-15T22:41:20.261 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:19 smithi117 podman[78631]: 2024-09-15 22:41:19.984034439 +0000 UTC m=+3.291521837 container cleanup 8b45cdfbec19b0cd1f002f2745c5c44328fccf9fdd50828ebd89b50aca608a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:41:20.261 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 15 22:41:19 smithi117 bash[78631]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-h 2024-09-15T22:41:20.415 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.h.service' 2024-09-15T22:41:20.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:20.448 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-15T22:41:20.448 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-15T22:41:20.448 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.k 2024-09-15T22:41:20.855 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:20 smithi117 systemd[1]: Stopping Ceph mds.k for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:21.112 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: 2024-09-15T22:41:20.854+0000 7ffbbc7f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:21.112 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:20 smithi117 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k[63464]: 2024-09-15T22:41:20.854+0000 7ffbbc7f2640 -1 mds.k *** got signal Terminated *** 2024-09-15T22:41:23.898 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:23 smithi117 podman[78753]: 2024-09-15 22:41:23.621170914 +0000 UTC m=+2.951618566 container died a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-15T22:41:24.264 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:23 smithi117 podman[78753]: 2024-09-15 22:41:23.979477273 +0000 UTC m=+3.309924931 container cleanup a54f2434676ddf32e213a077250470a097fda2fd234cf451294da44f5f5cb398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-15T22:41:24.264 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 15 22:41:23 smithi117 bash[78753]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-k 2024-09-15T22:41:24.396 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.k.service' 2024-09-15T22:41:24.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:24.428 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-15T22:41:24.428 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-15T22:41:24.428 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.c 2024-09-15T22:41:24.745 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:24 smithi196 systemd[1]: Stopping Ceph mds.c for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:25.007 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: 2024-09-15T22:41:24.743+0000 7fd2902f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:25.007 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:24 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c[65774]: 2024-09-15T22:41:24.743+0000 7fd2902f2640 -1 mds.c *** got signal Terminated *** 2024-09-15T22:41:27.887 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:27 smithi196 podman[78881]: 2024-09-15 22:41:27.603782902 +0000 UTC m=+2.966106255 container died c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-15T22:41:28.203 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:27 smithi196 podman[78881]: 2024-09-15 22:41:27.977111875 +0000 UTC m=+3.339435231 container cleanup c3b84b97d5afa7a3169938373dc8e6ca931ab8f783a8c1ae5076c21233bcc43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T22:41:28.204 INFO:journalctl@ceph.mds.c.smithi196.stdout:Sep 15 22:41:27 smithi196 bash[78881]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-c 2024-09-15T22:41:28.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.c.service' 2024-09-15T22:41:28.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:28.372 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-15T22:41:28.372 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-15T22:41:28.372 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.f 2024-09-15T22:41:28.603 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:28 smithi196 systemd[1]: Stopping Ceph mds.f for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:29.103 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: 2024-09-15T22:41:28.671+0000 7f3c2bb20640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:29.103 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:28 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f[65471]: 2024-09-15T22:41:28.671+0000 7f3c2bb20640 -1 mds.f *** got signal Terminated *** 2024-09-15T22:41:31.866 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:31 smithi196 podman[79013]: 2024-09-15 22:41:31.604110808 +0000 UTC m=+3.015453408 container died a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:41:32.298 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:31 smithi196 podman[79013]: 2024-09-15 22:41:31.939029232 +0000 UTC m=+3.350371822 container cleanup a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:41:32.298 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:31 smithi196 bash[79013]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f 2024-09-15T22:41:32.299 INFO:journalctl@ceph.mds.f.smithi196.stdout:Sep 15 22:41:32 smithi196 podman[79025]: 2024-09-15 22:41:32.090074329 +0000 UTC m=+0.485798327 container remove a14020065d39bf1af4373ee115e5ec25344fd1690be578034cbd6dbb396f2221 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:41:32.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.f.service' 2024-09-15T22:41:32.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:32.347 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-15T22:41:32.347 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-15T22:41:32.347 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.i 2024-09-15T22:41:32.603 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:32 smithi196 systemd[1]: Stopping Ceph mds.i for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:33.103 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: 2024-09-15T22:41:32.691+0000 7fc25855d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:33.103 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:32 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i[66075]: 2024-09-15T22:41:32.691+0000 7fc25855d640 -1 mds.i *** got signal Terminated *** 2024-09-15T22:41:35.960 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:35 smithi196 podman[79133]: 2024-09-15 22:41:35.612775971 +0000 UTC m=+3.044471529 container died 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:41:36.219 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:36 smithi196 podman[79133]: 2024-09-15 22:41:36.051107912 +0000 UTC m=+3.482803470 container cleanup 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-15T22:41:36.220 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:36 smithi196 bash[79133]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i 2024-09-15T22:41:36.477 INFO:journalctl@ceph.mds.i.smithi196.stdout:Sep 15 22:41:36 smithi196 podman[79147]: 2024-09-15 22:41:36.219035476 +0000 UTC m=+0.604190235 container remove 956abfeb375a13b39c491bff698019d26f23f784080f8a9939d9ffd4847eaebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:41:36.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.i.service' 2024-09-15T22:41:36.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:36.513 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-15T22:41:36.514 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-15T22:41:36.514 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.l 2024-09-15T22:41:36.830 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:36 smithi196 systemd[1]: Stopping Ceph mds.l for 0ddb746c-73ae-11ef-bceb-c7b262605968... 2024-09-15T22:41:37.103 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: 2024-09-15T22:41:36.829+0000 7f2328a0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:41:37.103 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:36 smithi196 ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l[65169]: 2024-09-15T22:41:36.829+0000 7f2328a0d640 -1 mds.l *** got signal Terminated *** 2024-09-15T22:41:39.964 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:39 smithi196 podman[79257]: 2024-09-15 22:41:39.609081161 +0000 UTC m=+2.871982636 container died db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-15T22:41:40.215 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:40 smithi196 podman[79257]: 2024-09-15 22:41:40.05448934 +0000 UTC m=+3.317390812 container cleanup db40991e161a3844f1d9cd9af2e9a956ed5a50d65416a218924417a055458335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-15T22:41:40.215 INFO:journalctl@ceph.mds.l.smithi196.stdout:Sep 15 22:41:40 smithi196 bash[79257]: ceph-0ddb746c-73ae-11ef-bceb-c7b262605968-mds-l 2024-09-15T22:41:40.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ddb746c-73ae-11ef-bceb-c7b262605968@mds.l.service' 2024-09-15T22:41:40.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:41:40.465 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-15T22:41:40.465 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:41:40.669 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:41:43.215 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:41:43.415 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:41:45.871 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:41:46.073 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T22:41:48.571 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:41:48.605 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:41:48.637 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:41:48.663 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-15T22:41:48.665 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi077/crash 2024-09-15T22:41:48.666 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:41:48.707 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:41:48.707 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:41:48.709 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi117/crash 2024-09-15T22:41:48.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:41:48.748 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:41:48.748 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:41:48.750 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi196/crash 2024-09-15T22:41:48.751 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:41:48.803 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:41:48.803 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:41:48.804 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-15T22:41:48.804 DEBUG:teuthology.orchestra.run.smithi077:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-15T22:41:48.899 INFO:tasks.cephadm:Compressing logs... 2024-09-15T22:41:48.899 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:41:48.901 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:41:48.904 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:41:48.939 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:41:48.940 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:41:48.942 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:41:48.942 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-15T22:41:48.942 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:41:48.943 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-15T22:41:48.944 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-15T22:41:48.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log: /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-15T22:41:48.944 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-15T22:41:48.945 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.f.log/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.z.log: 2024-09-15T22:41:48.945 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:41:48.947 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:41:48.948 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-15T22:41:48.948 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-15T22:41:48.948 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-15T22:41:48.950 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:41:48.950 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:41:48.950 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-15T22:41:48.950 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-15T22:41:48.953 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-15T22:41:48.954 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:41:48.955 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:41:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:41:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-15T22:41:48.961 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-15T22:41:48.963 INFO:teuthology.orchestra.run.smithi117.stderr:gzip/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.10.log: -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-15T22:41:48.963 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:41:48.964 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-15T22:41:48.966 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:41:48.966 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-15T22:41:48.966 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-15T22:41:48.966 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-15T22:41:48.966 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:41:49.045 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.8% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:41:49.120 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:41:49.141 INFO:teuthology.orchestra.run.smithi196.stderr: 90.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:41:49.143 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:41:49.147 INFO:teuthology.orchestra.run.smithi077.stderr: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:41:49.150 INFO:teuthology.orchestra.run.smithi117.stderr: 82.9% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:41:49.152 INFO:teuthology.orchestra.run.smithi117.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:41:49.195 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-15T22:41:49.206 INFO:teuthology.orchestra.run.smithi117.stderr: 92.7% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-15T22:41:49.210 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:41:49.236 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:41:49.261 INFO:teuthology.orchestra.run.smithi077.stderr: 84.9% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:41:49.281 INFO:teuthology.orchestra.run.smithi077.stderr: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:41:49.283 INFO:teuthology.orchestra.run.smithi117.stderr: 87.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:41:49.373 INFO:teuthology.orchestra.run.smithi117.stderr: 90.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:41:49.502 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:41:49.564 INFO:teuthology.orchestra.run.smithi077.stderr: 94.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-15T22:41:49.598 INFO:teuthology.orchestra.run.smithi077.stderr: 94.7% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-15T22:41:49.808 INFO:teuthology.orchestra.run.smithi196.stderr: 94.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-15T22:41:50.326 INFO:teuthology.orchestra.run.smithi077.stderr: 90.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-15T22:41:55.499 INFO:teuthology.orchestra.run.smithi196.stderr: 95.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-15T22:41:57.971 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-15T22:42:12.497 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-15T22:43:07.852 INFO:teuthology.orchestra.run.smithi077.stderr: 95.6% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-15T22:43:13.669 INFO:teuthology.orchestra.run.smithi117.stderr: 93.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-15T22:43:41.991 INFO:teuthology.orchestra.run.smithi077.stderr: 91.4% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-15T22:43:42.551 INFO:teuthology.orchestra.run.smithi077.stderr: 91.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-15T22:43:50.643 INFO:teuthology.orchestra.run.smithi077.stderr: 91.2% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-15T22:44:54.415 INFO:teuthology.orchestra.run.smithi077.stderr: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-15T22:44:59.840 INFO:teuthology.orchestra.run.smithi196.stderr: 95.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-15T22:45:45.902 INFO:teuthology.orchestra.run.smithi077.stderr: 93.5% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-15T22:45:45.905 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-09-15T22:45:45.905 INFO:teuthology.orchestra.run.smithi077.stderr:real 3m56.989s 2024-09-15T22:45:45.905 INFO:teuthology.orchestra.run.smithi077.stderr:user 4m57.132s 2024-09-15T22:45:45.905 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m11.237s 2024-09-15T22:45:59.416 INFO:teuthology.orchestra.run.smithi117.stderr: 95.5% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-15T22:47:57.697 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-15T22:48:32.426 INFO:teuthology.orchestra.run.smithi117.stderr: 91.5% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-15T22:49:25.017 INFO:teuthology.orchestra.run.smithi117.stderr: 94.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-15T22:49:59.736 INFO:teuthology.orchestra.run.smithi196.stderr: 91.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-15T22:51:20.219 INFO:teuthology.orchestra.run.smithi117.stderr: 91.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-15T22:52:21.788 INFO:teuthology.orchestra.run.smithi196.stderr: 91.3% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-15T22:53:19.339 INFO:teuthology.orchestra.run.smithi117.stderr: 91.1% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-15T22:53:37.177 INFO:teuthology.orchestra.run.smithi117.stderr: 91.2% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-15T22:54:45.359 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-15T22:59:08.256 INFO:teuthology.orchestra.run.smithi117.stderr: 93.5% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-15T22:59:23.221 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-15T22:59:23.223 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-15T22:59:23.223 INFO:teuthology.orchestra.run.smithi196.stderr:real 17m34.305s 2024-09-15T22:59:23.224 INFO:teuthology.orchestra.run.smithi196.stderr:user 8m58.770s 2024-09-15T22:59:23.224 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m24.709s 2024-09-15T23:02:08.990 INFO:teuthology.orchestra.run.smithi117.stderr: 92.9% -- replaced with /var/log/ceph/0ddb746c-73ae-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-15T23:02:09.055 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T23:02:09.055 INFO:teuthology.orchestra.run.smithi117.stderr:real 20m20.137s 2024-09-15T23:02:09.055 INFO:teuthology.orchestra.run.smithi117.stderr:user 9m4.644s 2024-09-15T23:02:09.056 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m26.254s 2024-09-15T23:02:09.056 INFO:tasks.cephadm:Archiving logs... 2024-09-15T23:02:09.057 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi077/log 2024-09-15T23:02:09.058 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:02:58.130 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi117/log 2024-09-15T23:02:58.131 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:04:26.882 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi196/log 2024-09-15T23:04:26.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:05:54.671 INFO:tasks.cephadm:Removing cluster... 2024-09-15T23:05:54.672 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force 2024-09-15T23:05:54.875 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T23:05:55.519 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force 2024-09-15T23:05:56.221 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T23:05:57.079 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ddb746c-73ae-11ef-bceb-c7b262605968 --force 2024-09-15T23:05:57.612 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0ddb746c-73ae-11ef-bceb-c7b262605968 2024-09-15T23:05:58.351 INFO:tasks.cephadm:Removing cephadm ... 2024-09-15T23:05:58.351 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:05:58.371 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:05:58.391 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:05:58.408 INFO:tasks.cephadm:Teardown complete 2024-09-15T23:05:58.408 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T23:05:58.420 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T23:05:58.420 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:05:58.423 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:05:58.435 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:05:58.776 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T23:05:58.787 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T23:05:58.787 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:05:58.790 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:05:58.793 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:05:58.805 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:05:58.807 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:05:58.807 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:05:58.838 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:05:58.838 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-15T23:05:58.839 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +1038us[ +941us] +/- 42ms 2024-09-15T23:05:58.839 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 -1200us[-1200us] +/- 50ms 2024-09-15T23:05:58.839 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 21 +70us[ +70us] +/- 60ms 2024-09-15T23:05:58.839 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:05:58.880 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:05:58.880 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-15T23:05:58.880 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +1013us[ +917us] +/- 42ms 2024-09-15T23:05:58.881 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1227us[-1227us] +/- 50ms 2024-09-15T23:05:58.881 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +27us[ -68us] +/- 60ms 2024-09-15T23:05:58.881 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:05:58.911 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:05:58.911 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-09-15T23:05:58.912 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +999us[ +881us] +/- 42ms 2024-09-15T23:05:58.912 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1228us[-1228us] +/- 50ms 2024-09-15T23:05:58.912 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +22us[ +22us] +/- 60ms 2024-09-15T23:05:58.912 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:05:58.913 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T23:05:58.931 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T23:05:58.932 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T23:05:58.946 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:05:59.129 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:05:59.236 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:05:59.359 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:05:59.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:05:59.393 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:05:59.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:05:59.425 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:05:59.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:05:59.454 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T23:05:59.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T23:05:59.476 INFO:teuthology.task.internal:Duration was 4217.066286 seconds 2024-09-15T23:05:59.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T23:05:59.487 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T23:05:59.487 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:05:59.490 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:05:59.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:05:59.549 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:05:59.551 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:05:59.553 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:05:59.939 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T23:05:59.940 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-09-15T23:05:59.940 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:05:59.964 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T23:05:59.964 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:05:59.995 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-15T23:05:59.996 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:06:00.034 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T23:06:00.034 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:06:00.037 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:06:00.039 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:06:00.116 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T23:06:00.116 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:06:00.118 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:06:00.121 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:06:02.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T23:06:02.480 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T23:06:02.481 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:06:02.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:06:02.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:06:02.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T23:06:02.866 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:02.870 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:02.873 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:02.899 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-09-15T23:06:02.909 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-15T23:06:02.916 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-15T23:06:02.946 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:02.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:06:02.985 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:03.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:06:03.002 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:06:03.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:06:03.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T23:06:03.031 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T23:06:03.033 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi077 2024-09-15T23:06:03.033 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:06:03.123 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi117 2024-09-15T23:06:03.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:06:03.164 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906227/remote/smithi196 2024-09-15T23:06:03.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:06:03.207 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T23:06:03.208 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:06:03.211 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:06:03.213 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:06:03.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T23:06:03.273 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T23:06:03.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T23:06:03.286 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T23:06:03.286 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:06:03.289 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:06:03.291 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:06:03.301 INFO:teuthology.orchestra.run.smithi077.stdout: 265726 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:06 /home/ubuntu/cephtest 2024-09-15T23:06:03.305 INFO:teuthology.orchestra.run.smithi117.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:06 /home/ubuntu/cephtest 2024-09-15T23:06:03.316 INFO:teuthology.orchestra.run.smithi196.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:06 /home/ubuntu/cephtest 2024-09-15T23:06:03.317 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T23:06:03.331 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T23:06:03.357 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4217.066285848618 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T23:06:03.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:06:03.447 INFO:teuthology.run:pass